Skip to content

Commit 91b8233

Browse files
claudeclaude
committed
Merge NDI-matlab main into V2
main at 695e8d8 (PR #1020, ProgressBarWindow) and the commits before it that V2 did not have (147). Merged without conflict; requirements.txt and tests/requirements.txt take main's openMINDS_MATLAB v0.12.0 and keep V2's other lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FxKLA8Tbf9cpvDe7fuVeKf
2 parents b93b139 + 695e8d8 commit 91b8233

105 files changed

Lines changed: 13085 additions & 292 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎find_napari_debug_log.sh‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
#!/usr/bin/env zsh
2+
# find_napari_debug_log.sh - locate and print the napariViewLightsheet debug
3+
# log written by ndi.test.cloud.lightsheet_blob_cloud_roundtrip.
4+
#
5+
# The demo defaults DebugLogFile to fullfile(tempdir, 'NapariLightSheetDebugging.txt')
6+
# which on macOS resolves under $TMPDIR (typically /var/folders/xx/yyy/T/),
7+
# not /tmp. This script looks in the usual spots, picks the newest match,
8+
# and cats it.
9+
10+
set -u
11+
setopt NULL_GLOB EXTENDED_GLOB
12+
13+
name="NapariLightSheetDebugging.txt"
14+
15+
# Places to check, ordered by likelihood on macOS.
16+
locations=(
17+
"${TMPDIR:-}/$name" # matches MATLAB's tempdir
18+
"/tmp/$name" # a user who set DebugLogFile explicitly
19+
"$HOME/$name" # or dropped it in their home
20+
"/private/tmp/$name" # macOS's real /tmp target
21+
)
22+
23+
# De-duplicate, drop empties, and keep only paths that actually exist.
24+
typeset -a found
25+
for p in "${locations[@]}"; do
26+
[[ -n "$p" && -f "$p" ]] && found+=("$p")
27+
done
28+
29+
# Wider net if the shortlist misses: recurse under /var/folders and $TMPDIR
30+
# (macOS parks per-user tempdirs there). Prune common noise; keep the
31+
# recent ones.
32+
if (( ${#found} == 0 )); then
33+
for root in "${TMPDIR:-}" /var/folders /private/var/folders /tmp; do
34+
[[ -n "$root" && -d "$root" ]] || continue
35+
while IFS= read -r hit; do
36+
[[ -n "$hit" ]] && found+=("$hit")
37+
done < <(find "$root" -maxdepth 6 -type f -name "$name" 2>/dev/null)
38+
done
39+
fi
40+
41+
if (( ${#found} == 0 )); then
42+
print -u2 "No $name found. If your demo ran with a custom DebugLogFile,"
43+
print -u2 "point me at it: $0 /full/path/to/logfile"
44+
print -u2 "Or check the value of tempdir in MATLAB (>> tempdir)."
45+
exit 1
46+
fi
47+
48+
# Pick the newest match if multiple.
49+
newest="$found[1]"
50+
newest_mtime=$(stat -f %m "$newest" 2>/dev/null || stat -c %Y "$newest" 2>/dev/null)
51+
for f in "$found[@]"; do
52+
m=$(stat -f %m "$f" 2>/dev/null || stat -c %Y "$f" 2>/dev/null)
53+
if [[ -n "$m" && "$m" -gt "$newest_mtime" ]]; then
54+
newest="$f"
55+
newest_mtime="$m"
56+
fi
57+
done
58+
59+
print -u2 "=== Reading: $newest ==="
60+
if (( ${#found} > 1 )); then
61+
print -u2 "(also found: ${(j:, :)found[2,-1]})"
62+
fi
63+
print -u2 ""
64+
65+
cat -- "$newest"

‎lightsheet_test_blob_ingest_and_view.m‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
%LIGHTSHEET_TEST_BLOB_INGEST_AND_VIEW - end-to-end smoke test on the synthetic blob
2+
%
3+
% Builds a small OME-Zarr fixture on disk (a couple of MB), ingests it
4+
% into a fresh NDI session with the blosc-zstd fast path, and launches
5+
% the napari viewer on the resulting pyramid. Use this to sanity-check
6+
% the ingest -> document -> viewer pipeline without touching the real
7+
% SmartSPIM data. Runs in under a minute on a laptop.
8+
%
9+
% Edit CONFIG below if you want to keep the fixture / session between
10+
% runs, change the blob size, or point at a different napari launcher.
11+
%
12+
% See also: ndi.test.lightsheet.makeBlobFixture,
13+
% ndi.test.lightsheet.ingestBlobFixture,
14+
% ndi.fun.doc.lightsheet.fromOMEZarr,
15+
% ndi.fun.doc.lightsheet.view
16+
17+
% -------------------- CONFIG -----------------------------------------
18+
% Leave scratch dirs empty to land under tempname (fresh every run).
19+
sessionDir = ''; % e.g. '~/ndi-sessions/blob-demo'
20+
zarrDir = ''; % e.g. '~/ndi-sessions/blob-demo/zarr'
21+
subjectName = 'blob@vhlab';
22+
23+
% Blob size / shape. 128^3 uint16 across 2 channels ~= 16 MB raw; small
24+
% enough for a fast smoke test, big enough that the pyramid has real
25+
% levels to look at.
26+
shape = [128 128 128];
27+
numChannels = 2;
28+
chunkShape = [32 32 32];
29+
numLevels = 3;
30+
31+
% Ingest options. blosc-zstd exercises the encodeChunk fast path; keep
32+
% clevel modest for a smoke test.
33+
codec = 'blosc-zstd';
34+
clevel = 5;
35+
36+
% Napari launcher. Same wrapper the real-data script uses.
37+
launcher = '/usr/local/bin/napariViewLightsheet';
38+
% ---------------------------------------------------------------------
39+
40+
fprintf('Building blob fixture and ingesting ...\n');
41+
[S, P, info] = ndi.test.lightsheet.ingestBlobFixture( ...
42+
'sessionDir', sessionDir, ...
43+
'zarrDir', zarrDir, ...
44+
'subject', subjectName, ...
45+
'Shape', shape, ...
46+
'NumChannels', numChannels, ...
47+
'ChunkShape', chunkShape, ...
48+
'NumLevels', numLevels, ...
49+
'codec', codec, ...
50+
'clevel', clevel);
51+
52+
fprintf('Session: %s\n', char(S.path()));
53+
fprintf('Zarr: %s\n', info.zarrPath);
54+
fprintf('Pyramid: %s\n', P.id());
55+
56+
fprintf('Launching napari viewer ...\n');
57+
ndi.fun.doc.lightsheet.view(S, P.id(), 'launcher', launcher);

‎requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,4 +11,4 @@ https://github.com/Waltham-Data-Science/NDI-compress-matlabp
1111
https://github.com/Waltham-Data-Science/ndi-ontology-matlab
1212
https://github.com/VH-Lab/pyraview
1313
https://github.com/bastibe/Violinplot-Matlab
14-
fex://134212-openminds-metadata-models-for-matlab/0.11.0
14+
https://github.com/openMetadataInitiative/openMINDS_MATLAB@v0.12.0

‎src/ndi/+ndi/+cloud/+api/+feedback/deleteFeedback.m‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
function [b, answer, apiResponse, apiURL] = deleteFeedback(feedbackID)
2+
%DELETEFEEDBACK User-facing wrapper to withdraw one of your own problem reports.
3+
%
4+
% [B, ANSWER, APIRESPONSE, APIURL] = ndi.cloud.api.feedback.deleteFeedback(FEEDBACKID)
5+
%
6+
% Deletes a report previously sent with ndi.cloud.api.feedback.submitFeedback.
7+
% Only the user who submitted a report can delete it; any other report
8+
% (or one that does not exist) is answered with 404.
9+
%
10+
% Inputs:
11+
% feedbackID - The id returned by submitFeedback.
12+
%
13+
% Outputs:
14+
% b - True if the call succeeded, false otherwise.
15+
% answer - A confirmation message on success.
16+
% apiResponse - The full matlab.net.http.ResponseMessage object.
17+
% apiURL - The URL that was called.
18+
%
19+
% See also: ndi.cloud.api.feedback.submitFeedback,
20+
% ndi.cloud.api.implementation.feedback.DeleteFeedback
21+
22+
arguments
23+
feedbackID (1,1) string
24+
end
25+
26+
% 1. Create an instance of the implementation class.
27+
api_call = ndi.cloud.api.implementation.feedback.DeleteFeedback('feedbackID', feedbackID);
28+
29+
% 2. Call the execute method and return its outputs directly.
30+
[b, answer, apiResponse, apiURL] = api_call.execute();
31+
32+
end

‎src/ndi/+ndi/+cloud/+api/+feedback/submitFeedback.m‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
function [b, feedbackID, apiResponse, apiURL] = submitFeedback(app, category, description, options)
2+
%SUBMITFEEDBACK User-facing wrapper to submit an in-app problem report to NDI Cloud.
3+
%
4+
% [B, FEEDBACKID, APIRESPONSE, APIURL] = ndi.cloud.api.feedback.submitFeedback(APP, CATEGORY, DESCRIPTION, ...)
5+
%
6+
% Sends a problem report to POST /feedback. The reporter is the
7+
% logged-in user. The server redacts anything that looks like a bearer
8+
% token or a signed S3 URL credential before storing the report, but
9+
% callers should still scrub logs before sending.
10+
%
11+
% Inputs:
12+
% app - Name of the client sending the report (e.g., "lightsheetZarr-viewer").
13+
% category - One of "bug", "performance" or "other".
14+
% description - The user's free-text description of the problem.
15+
%
16+
% Optional name-value inputs:
17+
% appVersion - Version of the client (e.g., "NDI-matlab 2.0").
18+
% datasetId - Cloud dataset id the report concerns.
19+
% documentId - Document id the report concerns.
20+
% environment - Struct describing the client environment (os, versions, ...).
21+
% state - Struct with client state useful for diagnosis.
22+
% log - Recent log lines.
23+
% traceback - Error stack, if any.
24+
%
25+
% Server limits: 1 MB per request; description 20,000 characters, log
26+
% 500,000, traceback 50,000; environment 20 kB and state 200 kB as JSON;
27+
% 10 reports per user per hour.
28+
%
29+
% Outputs:
30+
% b - True if the call succeeded, false otherwise.
31+
% feedbackID - The id of the stored report (use it with deleteFeedback).
32+
% apiResponse - The full matlab.net.http.ResponseMessage object.
33+
% apiURL - The URL that was called.
34+
%
35+
% See also: ndi.cloud.api.feedback.deleteFeedback,
36+
% ndi.cloud.api.implementation.feedback.SubmitFeedback
37+
38+
arguments
39+
app (1,1) string {mustBeNonzeroLengthText}
40+
category (1,1) string {mustBeMember(category, ["bug", "performance", "other"])}
41+
description (1,1) string {mustBeNonzeroLengthText}
42+
options.appVersion (1,1) string = missing
43+
options.datasetId (1,1) string = missing
44+
options.documentId (1,1) string = missing
45+
options.environment struct {mustBeScalarOrEmpty} = struct.empty
46+
options.state struct {mustBeScalarOrEmpty} = struct.empty
47+
options.log (1,1) string = missing
48+
options.traceback (1,1) string = missing
49+
end
50+
51+
feedbackStruct = struct( ...
52+
'schemaVersion', 1, ...
53+
'app', app, ...
54+
'category', category, ...
55+
'description', description);
56+
57+
stringFields = ["appVersion", "datasetId", "documentId", "log", "traceback"];
58+
for fieldName = stringFields
59+
if ~ismissing(options.(fieldName))
60+
feedbackStruct.(fieldName) = options.(fieldName);
61+
end
62+
end
63+
for fieldName = ["environment", "state"]
64+
if ~isempty(options.(fieldName))
65+
feedbackStruct.(fieldName) = options.(fieldName);
66+
end
67+
end
68+
69+
% 1. Create an instance of the implementation class.
70+
api_call = ndi.cloud.api.implementation.feedback.SubmitFeedback('feedbackStruct', feedbackStruct);
71+
72+
% 2. Call the execute method and return its outputs directly.
73+
[b, feedbackID, apiResponse, apiURL] = api_call.execute();
74+
75+
end

‎src/ndi/+ndi/+cloud/+api/+files/getFileUploadURL.m‎

Lines changed: 24 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
1-
function [b, answer, apiResponse, apiURL] = getFileUploadURL(cloudDatasetID, cloudFileUID)
1+
function [b, answer, apiResponse, apiURL] = getFileUploadURL(cloudDatasetID, cloudFileUID, options)
22
%GETFILEUPLOADURL Get a pre-signed URL for uploading a single file.
33
%
44
% [B, ANSWER, APIRESPONSE, APIURL] = ndi.cloud.api.files.getFileUploadURL(CLOUDDATASETID, CLOUDFILEUID)
5+
% [...] = ndi.cloud.api.files.getFileUploadURL(..., 'checksumCRC32', CRC)
56
%
67
% Retrieves a URL that can be used to PUT the contents of a single file
78
% for a dataset.
@@ -10,6 +11,17 @@
1011
% cloudDatasetID - The ID of the dataset.
1112
% cloudFileUID - The unique identifier that will be used for the file.
1213
%
14+
% Name-Value Inputs:
15+
% checksumCRC32 - (Optional) The file's CRC-32 as base64 of its
16+
% 4-byte big-endian value (ndi.fun.file.crc32).
17+
% When given, the URL signs the
18+
% x-amz-checksum-crc32 header, so the upload must
19+
% send it: pass the same value to putFiles as
20+
% 'checksumCRC32'. S3 then rejects content that
21+
% does not match, and the cloud records the value
22+
% as the file's crc32.
23+
% Default "" (no checksum).
24+
%
1325
% Outputs:
1426
% b - True if the call succeeded, false otherwise.
1527
% answer - The pre-signed upload URL (string) on success, or an error struct on failure.
@@ -19,17 +31,26 @@
1931
% Example:
2032
% [success, upload_url] = ndi.cloud.api.files.getFileUploadURL('d-12345', 'f-newfile');
2133
%
22-
% See also: ndi.cloud.api.implementation.files.GetFileUploadURL
34+
% % With a checksum, so a corrupted upload is refused:
35+
% crc = ndi.fun.file.crc32('myfile.dat');
36+
% [success, upload_url] = ndi.cloud.api.files.getFileUploadURL('d-12345', 'f-newfile', ...
37+
% 'checksumCRC32', crc);
38+
% ndi.cloud.api.files.putFiles(upload_url, 'myfile.dat', 'checksumCRC32', crc);
39+
%
40+
% See also: ndi.cloud.api.implementation.files.GetFileUploadURL,
41+
% ndi.fun.file.crc32
2342

2443
arguments
2544
cloudDatasetID (1,1) string
2645
cloudFileUID (1,1) string
46+
options.checksumCRC32 (1,1) string = ""
2747
end
2848

2949
% 1. Create an instance of the implementation class.
3050
api_call = ndi.cloud.api.implementation.files.GetFileUploadURL(...
3151
'cloudDatasetID', cloudDatasetID, ...
32-
'cloudFileUID', cloudFileUID);
52+
'cloudFileUID', cloudFileUID, ...
53+
'checksumCRC32', options.checksumCRC32);
3354

3455
% 2. Call the execute method and return its outputs directly.
3556
[b, answer, apiResponse, apiURL] = api_call.execute();

‎src/ndi/+ndi/+cloud/+api/+files/getSignedURLSet.m‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,9 @@
2727
% (or the wrapped struct form MATLAB unpacks the
2828
% server's map into) and whose values are pre-signed
2929
% GET URLs.
30+
% `fileInfo` is an Nx1 struct array (uid, uploaded,
31+
% size, crc32) for this page's files; crc32 is '' when
32+
% none was recorded. Empty if the server predates it.
3033
% On failure, the server error payload.
3134
% apiResponse - The full matlab.net.http.ResponseMessage object.
3235
% apiURL - The URL that was called.

‎src/ndi/+ndi/+cloud/+api/+files/getSignedURLSetAll.m‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@
2929
% pages, expiresAt, files.
3030
% `files` is a struct whose field names are file UIDs
3131
% and whose values are pre-signed GET URLs.
32+
% `fileInfo` is an Nx1 struct array (uid, uploaded,
33+
% size, crc32) across all pages; crc32 is '' when none
34+
% was recorded.
3235
% apiResponse - The matlab.net.http.ResponseMessage from the last page.
3336
% apiURL - The URL of the last page.
3437
%

‎src/ndi/+ndi/+cloud/+api/+files/listFiles.m‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@
2929
% uploaded - A 0/1 flag indicating if the file has been uploaded.
3030
% sourceDatasetId - The cloudDatasetId of the file's parent dataset.
3131
% size - The file size in bytes.
32+
% crc32 - The base64 big-endian CRC-32 of the content,
33+
% recorded at upload ('' when none was recorded).
34+
% Compare with ndi.fun.file.crc32 of a local copy.
3235
%
3336
% Example:
3437
% % Get the first page of files

‎src/ndi/+ndi/+cloud/+api/+files/listFilesAll.m‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,8 @@
3030
% uploaded - 0/1 flag: whether the file is uploaded.
3131
% sourceDatasetId - The parent dataset's cloud id.
3232
% size - The file size in bytes.
33+
% crc32 - Base64 big-endian CRC-32 recorded at
34+
% upload ('' when none was recorded).
3335
% apiResponse - The matlab.net.http.ResponseMessage object from the LAST
3436
% page call.
3537
% apiURL - The URL that was called for the LAST page.

0 commit comments

Comments
 (0)