Skip to content

Install Pyraview 0.4.2 from its pre-built release wheels, never from source - #330

Merged
stevevanhooser merged 3 commits into
mainfrom
pyraview-prebuilt-wheels
Oct 15, 2026
Merged

stevevanhooser merged 3 commits into
mainfrom
pyraview-prebuilt-wheels

Conversation

@stevevanhooser

Copy link
Copy Markdown
Contributor

Summary

Pyraview isn't on PyPI, and building its sdist needs CMake and a C++ compiler. The v0.4.2 GitHub release ships a wheel for each platform and CPython version, with the compiled library bundled inside. After this PR, NDI-python and the uv-based installer get pyraview only from those wheels. If no wheel fits, the install fails; it never compiles.

  • pyproject.toml: new extra pyraview = ["pyraview==0.4.2"]. It's declared by name with an exact pin, not a git URL or sdist URL. It's not in gui, so pip install ".[gui]" still works without a find-links flag. The installer will request ndi[gui,napari,pyraview].
  • install/pyraview-wheels.html: a committed flat index (--find-links) with all 54 v0.4.2 wheels. Each link is the release download URL plus a #sha256= fragment. The sdist is not listed.
  • install/pyraview-constraints.txt: pyraview==0.4.2 with one --hash per wheel. These are the same hashes as the index; this file is what makes uv check them (see below).
  • tools/make_pyraview_index.py <tag>: regenerates both files from the GitHub API, using each release asset's digest. --check rebuilds them in memory and fails if the committed files differ. --release-json reads a saved API response.
  • ndi_install.py: removed the Pyraview entry from DEPENDENCIES and the clone + pip install <checkout> build (pip_install_dependency). The existing editable install now requests the pyraview extra with --find-links install/pyraview-wheels.html --only-binary pyraview. Its other behaviour is unchanged.
  • README (developer install section): the one-line command for pip, and the equivalent for uv.
  • tools/check_pyraview_wheel.py: the CI proof. It imports pyraview and asserts that _lib_path is the library bundled inside the package. It also asserts that .dist-info/WHEEL has Root-Is-Purelib: false and that its tag set is exactly the tag set of one wheel in the committed index. A wheel built locally from the sdist would carry linux_x86_64 rather than manylinux_*, a different macOS target, and so on. It then prints that release wheel's filename and sha256.

Which find-links mechanism, and why: (b), the committed index

I picked (b) over GitHub's undocumented releases/expanded_assets/<tag> page because it pins hashes and doesn't depend on a page GitHub might change. I tested how uv and pip each read it, locally with uv 0.11.32 against a real downloaded 0.4.2 wheel:

local file path install/pyraview-wheels.html raw URL (served as text/plain)
uv ❌ refused ("Failed to read --find-links directory … Not a directory"). uv takes only a directory or a URL. ✅
pip ✅ ❌ skipped ("Content-Type: text/plain … only supported … text/html")

So each tool uses the form it can read:

  • uv (CI, the future double-click installer): -f https://raw.githubusercontent.com/Waltham-Data-Science/NDI-python/<ref>/install/pyraview-wheels.html. CI uses this commit's SHA; the installer will use main or a tag.
  • pip (ndi_install.py, README): the local path in the checkout.

What is hash-verified, and what is not

I checked each of these locally by corrupting hashes, and CI checks the uv case on every leg:

install path hashes checked? how
pip install … --find-links install/pyraview-wheels.html (ndi_install.py) yes pip checks the #sha256 fragment on every link: "Expected sha256 000… Got 2a55…"
uv pip install -f <raw URL> alone no uv 0.11.32 ignores find-links #sha256 fragments; a corrupted fragment still installs, with or without --verify-hashes
uv pip install -f <raw URL> -c install/pyraview-constraints.txt yes uv enforces hashes from a constraints file ("Hash mismatch … Computed: sha256:2a55…"), without requiring hashes for every other package. Works as a local path or URL. CI checks this on every leg with a corrupted copy.
uv tool install … -c constraints / --with-requirements / UV_REQUIRE_HASHES=1 no uv tool install checks no pyraview hash in any of those forms. A corrupted constraints file still installed.

For the double-click installer (next PR): as of uv 0.11.32, uv tool install can't enforce these hashes. The installer can create its environment with uv venv + uv pip install -c install/pyraview-constraints.txt (hash-enforced). Or it can accept uv tool install, which still never compiles (--no-build-package pyraview) but trusts TLS to GitHub for the wheel's contents. --check in CI proves the committed hashes are GitHub's own asset digests.

The constraints file also protects against a squatter publishing pyraview on PyPI: uv would reject any file whose hash isn't listed. The pip path checks link fragments only for links that come from our index.

No source builds: belt and braces

  • The index has no sdist.
  • Every install command we control forbids building pyraview: --only-binary pyraview for pip and uv pip (uv pip has no --no-build-package), and --no-build-package pyraview for uv tool install.
  • CI also hides cmake, cc, c++, gcc, g++, clang, clang++, cl and git. On Linux and macOS they are shadowed by scripts that always fail. On Windows, PATH dirs containing them are dropped, since native lookup ignores shell shadows. The step checks they're really unusable before going on.

CI proof

The install-without-git job from #328 is now a matrix, rather than a second job. Each leg runs with git and the compilers hidden and uses Python 3.12:
ubuntu-latest (x86_64), ubuntu-24.04-arm (aarch64), windows-latest, macos-latest (arm64), macos-15-intel, and Rosetta: macos-latest with cpython-3.12-macos-x86_64 installed by uv, which asserts platform.machine() == "x86_64" and tags containing macosx_ and x86_64.

Each leg runs:

  1. Negative check: .[pyraview] without -f must fail and must not build anything.
  2. Negative check: a corrupted constraints file must give "hash mismatch".
  3. Install .[tutorials,pyraview] with the raw-URL index, the constraints file and --only-binary pyraview.
  4. python -c "import pyraview, os; print(pyraview.__file__, pyraview._lib_path)", then check_pyraview_wheel.py.
  5. The pip path, using the local .html file.
  6. The existing import / ndi check / NDIcalc steps.

The Linux x86_64 leg also runs uv tool install … --no-build-package pyraview (the installer's command form) and make_pyraview_index.py --check against the GitHub API.

The Linux x86_64 leg keeps the check name "Install without git", so a required-check rule naming it still matches. timeout-minutes: 45 bounds a hung run. Note that it doesn't bound time spent queued for a runner label: if macos-15-intel is ever retired, drop that leg.

The existing test, windows-smoke and ced-integration jobs run ndi_install.py, so they now take the pip + local-index path on Linux 3.10–3.12, Windows 3.10/3.12 and Linux 3.14. Before this PR they compiled Pyraview from a clone.

Installed wheel per leg: see the comment below; I'll post it from the CI logs once the run finishes.

Local verification (sandbox)

  • black, ruff, and the new and touched tests pass. That's tests/test_pyraview_wheel_index.py, test_ndi_install_exit_codes.py and test_ndi_install_timeouts.py; the full suite needs the archive-URL dependencies, which this sandbox can't fetch, so CI runs it.
  • With one real downloaded wheel (cp312 manylinux x86_64, sha256 2a550481… = GitHub's digest): uv selected the right wheel from the index; pip installed it and check_pyraview_wheel.py passed. I also ran the hash experiments in the table above.
  • I could not install from GitHub release URLs through uv here: the sandbox proxy gives 401 on the redirect. CI is the proof for every platform.

Tests

tests/test_pyraview_wheel_index.py checks offline:

  • the index lists only wheels of the pinned version, every link carries a hash, and there's no sdist;
  • every supported CPython has a wheel on every platform;
  • the constraints file has the same hashes as the index;
  • the generator reproduces both committed files byte for byte (and drops the sdist, mltbx, etc.);
  • pyraview isn't in gui and isn't a URL;
  • ndi_install.py no longer clones Pyraview and passes --find-links + --only-binary pyraview.

The two ndi_install tests that covered the removed pip_install_dependency and its 1800 s literal timeout were updated to match.

No bridge YAML changes: no MATLAB interface changed.

🤖 Generated with Claude Code

https://claude.ai/code/session_0121enpth5oMvjLdL1tP6uAG


Generated by Claude Code

Pyraview is not on PyPI, and its sdist needs CMake and a C++ compiler.
Its GitHub release carries a wheel per platform and CPython version with
the compiled library inside, so:

- pyproject.toml: new `pyraview` extra, `pyraview==0.4.2` by name (not
  folded into `gui`, so `pip install .[gui]` needs no extra flag).
- install/pyraview-wheels.html: committed find-links flat index, one link
  per release wheel with its #sha256 fragment; the sdist is left out.
  install/pyraview-constraints.txt: the same hashes as a constraints file,
  which is how uv is made to verify them (uv ignores find-links hash
  fragments; pip checks them). tools/make_pyraview_index.py regenerates
  both from a release tag via the GitHub API, or --check compares.
- ndi_install.py: drop the Pyraview clone and its from-source pip build;
  the editable install now takes the `pyraview` extra with --find-links
  install/pyraview-wheels.html --only-binary pyraview.
- CI: install-without-git becomes a matrix (Linux x86_64/aarch64,
  Windows, macOS arm64, macos-15-intel, x86_64 CPython under Rosetta)
  that hides git, CMake and the compilers, installs .[tutorials,pyraview]
  from the raw-URL index with --only-binary pyraview and the hashed
  constraints, and proves with tools/check_pyraview_wheel.py that the
  installed WHEEL tags are exactly a release wheel's. Negative checks: no
  find-links fails without compiling, and a corrupted hash is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0121enpth5oMvjLdL1tP6uAG
cryptography 49 dropped its universal2 macOS wheels, so on an Intel Mac
50.0.2 resolved to the sdist and uv tried a Rust + C build. The new
macos-intel and macos-rosetta-x86_64 CI legs, which hide the compilers,
caught it. Cap it below 49 on x86_64 macOS only; 48.0.1 is the last
release with a wheel there. Every other platform keeps the latest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0121enpth5oMvjLdL1tP6uAG

Copy link
Copy Markdown
Contributor Author

MATLAB/Python bridge completeness is red, and not because of this PR. test_matlab_bridge_hashes.py::test_no_entry_has_drifted flags src/ndi/calc/stimulus/ndi_matlab_python_bridge.yaml: tuningcurve -- MATLAB has moved since 63043f393. NDI-matlab merged VH-Lab/NDI-matlab@49a875c94 ("Fix tuningcurve dealing over a single stimulus property") at 23:34 UTC yesterday, after main's last green CI run (21:36). This PR touches no bridge file, so main will show the same failure on its next run.

I checked the diff. On the Python side nothing needs porting. That entry is porting_deferred, and ndi_calc_stimulus_tuningcurve.calculate is still a stub with no deal loop, so the bug the commit fixes has no Python counterpart yet. The fix would be just the hash bump:

-    matlab_last_sync_hash: "63043f393"
+    matlab_last_sync_hash: "49a875c94"

(or tools/bridge/bump_sync_hash.py). The decision_log could also note that the deal loop, once ported, must index with the loop variable, per 49a875c94.

I haven't put this in this PR because it's unrelated to installing Pyraview. Your call: it can go in a separate one-line PR, or I can add it here if you'd rather.

The Intel-macOS legs failed for a reason this PR did surface: cryptography ≥49 has no x86_64 macOS wheel, so it was being compiled with Rust. Fixed in 47b6c91, which caps it below 49 on Intel Macs only.


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

Installed wheel per leg (CI run on 47b6c91, from each job's INSTALLED RELEASE WHEEL line; uv and pip installed the same file on every leg):

leg platform.machine() release wheel installed sha256
Linux x86_64 (ubuntu-latest) x86_64 pyraview-0.4.2-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl 2a550481…
Linux aarch64 (ubuntu-24.04-arm) aarch64 pyraview-0.4.2-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl 3ad3bcf8…
Windows (windows-latest) AMD64 pyraview-0.4.2-cp312-cp312-win_amd64.whl df12ad18…
macOS arm64 (macos-latest) arm64 pyraview-0.4.2-cp312-cp312-macosx_11_0_arm64.whl 55c06a89…
macOS Intel (macos-15-intel) x86_64 pyraview-0.4.2-cp312-cp312-macosx_10_13_x86_64.whl 68d8f401…
Rosetta (macos-latest, cpython-3.12-macos-x86_64) x86_64 pyraview-0.4.2-cp312-cp312-macosx_10_13_x86_64.whl 68d8f401…

On every leg, with git, CMake and the C/C++ compilers hidden:

  • .[pyraview] without --find-links failed at resolution, without building anything.
  • A corrupted constraints file failed with "Hash mismatch". The hash uv computed is the one listed above for that leg.
  • .[tutorials,pyraview] then installed, and python -m ndi check passed 20/20.

On the Linux x86_64 leg:

  • uv tool install … --no-build-package pyraview took the same manylinux wheel.
  • make_pyraview_index.py --check reported OK: 54 wheels, hashes match GitHub's digests for v0.4.2.

Every check on 47b6c91 is green except MATLAB/Python bridge completeness. That failure is the NDI-matlab tuningcurve drift described above, which is unrelated to this PR. Symmetry tests passed on rerun; the earlier failure was an api.github.com rate limit during MATLAB setup, before any test ran.


Generated by Claude Code

@stevevanhooser
stevevanhooser merged commit 335c466 into main Oct 15, 2026
15 checks passed
@stevevanhooser
stevevanhooser deleted the pyraview-prebuilt-wheels branch October 15, 2026 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants