Skip to content

1.1.0: audit follow-ups (gallery repeats, tutorial coverage, native usage) #284

Description

@jeremymanning

Tracking issue for the three audits run on 2026-09-05 before the 1.1.0 re-cut (repeats in the gallery, tutorial coverage of the public API, native-hypertools usage). Everything here ships in 1.1.0. Constraint: the five launch tutorials the Bluesky thread links (market_sectors, weather_decades, painting_embeddings, conversation_shape, morph_shapes_zoo) keep their pages and clips.

A. Gallery repeats and misdescriptions (examples/)

  • Delete plot_shape_morph.py (strict subset of animate_morph_zoo.py); fold the "all seven shapes" note into animate_morph_zoo; retarget animate_surface_morph.py's reference
  • Merge plot_TSNE.py + plot_UMAP.py + plot_digits.py into one example
  • Merge plot_PPCA.py into plot_missing_data.py
  • Merge animate_MDS.py into animate.py
  • Merge chemtrails.py + precog.py into one trails page (keep animate_trails_mix.py)
  • Merge plot_clusters.py/2/3 into one three-cell example
  • Fix misdescribed docstrings: plot_story_trajectories.py (execute the pipeline for real), plot_hue.py (categorical hue), animate_surface_morph.py (changelog prose), save_movie.py (hidden align=), plot_datasets_tour.py ("five vs four")
  • 'hyper' -> 'HyperAlign' in analyze.py, plot_align.py
  • Gallery ordering (within_subsection_order) and the "Story trajectories" entry in docs/tutorials.rst
  • Retitle conversation_trajectories and hugging_face_embeddings so they read as distinct from their siblings

B. Native usage rewrites

  • load_digits -> hyp.load('digits') (three examples); load_dataset -> hyp.load(...) (streaming_data, hugging_face_embeddings)
  • plot_procrustes.py: private procrustes() -> hyp.align(model='Procrustes')
  • plot_gensim_text.py: text2mat -> hyp.plot(texts, vectorizer=..., semantic=...)
  • plot_autoencoders.py: ax.scatter -> hyp.plot(..., ax=)
  • animate_surface_morph.py: drop the hand downsample; set_alpha -> alpha= (c4ed79c + morph alpha fix)
  • Remove dead sentence-transformers / pykalman import guards (six files) (examples done; tutorials in progress) now that extras install on demand
  • projectile_kalman.ipynb and stock_forecasting.ipynb hand ax.plot panels -> hyp.plot(..., reduce=None, ndims=2, ax=)
  • Library gap: isotropic (whole-array) option for Normalize (feefcbe: Normalize(mode='isotropic'), 21 tests), so the three morph files stop hand-rolling normalize_shape(); tests + docs
  • Tighten tests/test_examples_are_native.py DEFECT_MARKERS with the signatures the audit listed (with a per-file allowlist for deliberate ax= demos)

C. Coverage: new and rebuilt tutorials (each executed, in docs/tutorials.rst, install cell, gate entry)

  • hierarchy.ipynb: column-MultiIndex frame, per-sector traces + mean, names=, per-level alpha=, hierarchical predict= and return_model trace_data

  • animate_forecast.py + tutorial: animate=True, predict='Kalman', forecast_trail=True, forecast_hue/palette/fmt

  • io.ipynb: hyp.load of local files / URL / HF streaming / DataFrame, hyp.save round trips, save_path formats, HypertoolsIOError

  • pipelines.ipynb: return_model on every dispatcher, .transform, pipeline= replay, apply_model, supported_models(), bundle keys, Pipeline directly

  • manip.ipynb: hyp.manip Smooth/Resample/ZScore/Normalize and chains, animate='window' + focused=

  • Rebuild plot.ipynb: saving executed, animate=, names=, colorbar=, alpha= list, size/elev/azim, labels, font=, label_alpha=, backend='plotly', impute=, resample=, exception classes

  • Rebuild align.ipynb: Procrustes, SRM, align= inside plot/analyze, fitted aligner reuse, local image

  • Rebuild analyze.ipynb: call every stage it advertises, hypertools figures instead of seaborn heatmaps

  • Rebuild reduce.ipynb: describe curve drawn, UMAP, autoencoders, return_model

  • Extend stock_forecasting.ipynb (Chronos section) and projectile_kalman.ipynb (sklearn imputers vs Kalman)

  • Library gap found by the morph rewrite: alpha= does not reach the travelling morph cloud (matplotlib and plotly); fix + tests, then drop the set_alpha call in animate_surface_morph.py

D. Verification and release

  • Every tutorial executed with scripts/execute_tutorial.py; measured output cells recorded in the gate
  • Full suite (4118 passed), ruff, sphinx -W gallery build, Colab feature tour (both backends, fresh runtimes)
  • Gallery republished from the release commit (3a9b868); artifacts rebuilt; master pushed; - [ ] CI green; v1.1.0 moved; tag CI green; draft release updated

Found while fixing (all fixed for 1.1.0)

  • Glyph 8722 warning on caller-supplied ax= panels: tick labels under the sans-serif alias have no per-glyph fallback -> the ax= path sets the tick-label font list (217a2a6)
  • hyp.io.lsl_stream() left its inlet open -> LSLStream destroys it on close/with/GC/exit (61f3e7e)
  • PPCA divide by zero encountered in log on wide data -> slogdet (54b4b2e)
  • hyp.load(DataFrame) raised TypeError -> passthrough for loaded data (a8d073a)
  • hyp.plot(x, pipeline=<hand-built Pipeline with raw sklearn stages>) raised -> fit-or-transform replay of raw steps, Pipeline.is_fitted (3340595)
  • DEFECT_MARKERS tightened with the audit signatures (6b6bcbc); not added: weather's trailing rolling(12).mean() (hypertools Smooth is centred; visual rewrites deferred to 1.2 per maintainer)

Also fixed on the way (not in the original audit): HF vectorizers crashing with the default semantic= (#285, 1b34b1d), pipeline replay of hand-built Pipelines (3340595), LSL inlet destroy (bb07321), docs-gate literal for the market notebook (d724181), gensim example title overline, stale gallery output pages.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions