Skip to content

Commit 5c79009

Browse files
committed
Lightsheet: find migrated pyramids by method; label from notes
Two decisions of Jess, 2026-10-15 (DID-matlab f363d98 builds the migrator side): 1. A migrated lightsheet pyramid's observation carries method FBbi:00000369 'selective plane illumination fluorescence microscopy'. ndi.vintage.lightsheetPyramidDocs now finds migrated pyramids as observations with interaction.method.node == that CURIE, one query (ndi.v2.fieldQuery, so subject_interaction is asked too); v1 pyramids are found as before (isa lightsheetZarrPyramid). The term lives in the new ndi.vintage.lightsheetMethod, which replaces ndi.vintage.lightsheetBuilderName (deleted: nothing else used it; the fixture names the software entity literally). The builder software entity is still written; it no longer marks the pyramid. 2. ndi.vintage.lightsheetPyramid reads the label from the observation's statement.notes (the part before the migrator's separator, when the batch pass did not run), pyramid_name from the store body description as before, and pyramid_type is always '' (dropped). Tests: TestLightsheetMigrated pins the method against the real migrator's output (replacing the builder-name check), the label after the pass, and the reader's separator against a no-pass migration; TestLightsheetMigratedSession adds a decoy observation of the builder software with no method, which must not be listed; TestLightsheetVintage reads the label from notes, with and without the separator. Fixtures carry the method and the label on notes, and a description without label or type. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FxKLA8Tbf9cpvDe7fuVeKf
1 parent 9b46cef commit 5c79009

8 files changed

Lines changed: 174 additions & 82 deletions

File tree

‎src/ndi/+ndi/+vintage/lightsheetBuilderName.m‎

Lines changed: 0 additions & 19 deletions
This file was deleted.
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
function m = lightsheetMethod()
2+
%LIGHTSHEETMETHOD The method term that marks a migrated lightsheet pyramid.
3+
%
4+
% M = ndi.vintage.lightsheetMethod()
5+
%
6+
% M is struct('node', 'FBbi:00000369', 'name', 'selective plane
7+
% illumination fluorescence microscopy') -- light-sheet microscopy (checked
8+
% on OLS). DID-matlab +migrators_j/lightsheet_zarr_pyramid.m sets it as
9+
% `interaction.method` on the intensity observation it makes of EVERY v1
10+
% lightsheetZarrPyramid, OME-Zarr and SmartSPIM alike (Jess 2026-10-15). It
11+
% is what says "lightsheet pyramid" rather than "some fluorescence
12+
% measurement", so ndi.vintage.lightsheetPyramidDocs finds migrated pyramids
13+
% by its node. It must stay the migrator's, character for character;
14+
% ndi.unittest.vintage.TestLightsheetMigrated pins it against the migrator's
15+
% output when DID-matlab's migrators are on the path.
16+
%
17+
% (Until 2026-10-15 the marker was the name of the builder `software`
18+
% entity the observation's software_id names. That entity is still
19+
% written, as the record of what built the pyramid; it no longer marks it.)
20+
%
21+
% See also: ndi.vintage.lightsheetPyramidDocs.
22+
23+
m = struct('node', 'FBbi:00000369', ...
24+
'name', 'selective plane illumination fluorescence microscopy');
25+
end

‎src/ndi/+ndi/+vintage/lightsheetPyramid.m‎

Lines changed: 30 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -26,11 +26,21 @@
2626
% (LEVELRECS, ndi.vintage.lightsheetLevels), which the
2727
% same pass completed from the pyramid's units and
2828
% channel names.
29-
% label, pyramid_name, pyramid_type
30-
% the migration keeps these ONLY as words in the store
31-
% body's `description` ('label "..."', 'NGFF multiscales
32-
% name "..."', '... type "..."'), so they are parsed
33-
% back out of it; '' when there is no store body.
29+
% label the observation's `statement.notes` (Jess
30+
% 2026-10-15: the label stays there, permanently). When
31+
% the batch pass did not run, the notes also carry, after
32+
% a newline and '-- store body: ', the text the pass
33+
% would have moved to the store body; only the part
34+
% before the LAST such separator is the label. The
35+
% separator is DID-matlab's lightsheet_zarr_pyramid.m /
36+
% foldGenericFiles.m NOTES_SEPARATOR, character for
37+
% character.
38+
% pyramid_name parsed from the store body's `description`
39+
% ('NGFF multiscales name "..."'); '' when there is no
40+
% store body (or no SESSION).
41+
% pyramid_type '' always: the migration DROPS it (Jess 2026-10-15; it
42+
% repeats the levels' summary statistic, and the v1
43+
% docs say not to trust it).
3444
% pipeline_version ''. It became a `version` text_assertion about the
3545
% software entity, not read here.
3646
% byte_order level 0's, if given.
@@ -53,6 +63,8 @@
5363
'shape_level0', [], 'voxel_size_level0', [], 'voxel_size_units', '', ...
5464
'translation_level0', [], 'dtype', '', 'pipeline_version', '', 'byte_order', '');
5565

66+
p.label = labelOf(charOr(ndi.v2.blockOf(props, 'statement', 'notes', '')));
67+
5668
p.dtype = charOr(ndi.v2.blockOf(props, 'value', 'source_data_type'));
5769
if isempty(p.dtype)
5870
p.dtype = charOr(ndi.v2.blockOf(props, 'value', 'data_type'));
@@ -66,10 +78,7 @@
6678
sp = stores{k}.document_properties;
6779
d = blockValue(sp, 'description');
6880
if ~isempty(d)
69-
d = charOr(d);
70-
p.label = quoted(d, 'label');
71-
p.pyramid_name = quoted(d, 'NGFF multiscales name');
72-
p.pyramid_type = quoted(d, 'NGFF multiscales type');
81+
p.pyramid_name = quoted(charOr(d), 'NGFF multiscales name');
7382
end
7483
keys = blockValue(sp, 'keys');
7584
if ~isempty(keys) && isempty(frame)
@@ -112,6 +121,18 @@
112121
end
113122
end
114123

124+
function label = labelOf(notes)
125+
% The label: NOTES up to the LAST separator DID-matlab's migrator writes
126+
% before the text its store-record pass moves out (present only when that pass
127+
% did not run), else all of NOTES.
128+
SEPARATOR = sprintf('\n-- store body: ');
129+
label = notes;
130+
at = strfind(notes, SEPARATOR);
131+
if ~isempty(at)
132+
label = notes(1:at(end) - 1);
133+
end
134+
end
135+
115136
function s = quoted(text, label)
116137
s = '';
117138
tok = regexp(text, [regexptranslate('escape', label) ' "([^"]*)"'], 'tokens', 'once');

‎src/ndi/+ndi/+vintage/lightsheetPyramidDocs.m‎

Lines changed: 24 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -8,45 +8,41 @@
88
%
99
% HOW A MIGRATED PYRAMID IS RECOGNISED. DID-matlab
1010
% +migrators_j/lightsheet_zarr_pyramid.m makes each v1 pyramid an intensity
11-
% `observation` (its id kept) whose software_id names an `entity` of type
12-
% software called ndi.vintage.lightsheetBuilderName() -- 'NDI lightsheet
13-
% pyramid builder (ndi.fun.doc.lightsheet.makePyramid)' (Jess
14-
% 2026-10-14, lightsheet point 5). Nothing else on the observation says
15-
% "lightsheet": its class is `observation`, its value kind `intensity` and
16-
% its variable NCIT:C184706 'Fluorescence Intensity', all of which any
17-
% fluorescence measurement shares. So a migrated pyramid is an observation
18-
% whose software is THAT entity, found in two steps (the entity by name,
19-
% then the observations pointing at it) because a query sees one document
20-
% at a time.
11+
% `observation` (its id kept) whose `interaction.method` is
12+
% ndi.vintage.lightsheetMethod() -- FBbi:00000369 'selective plane
13+
% illumination fluorescence microscopy' (Jess 2026-10-15). Its class
14+
% (`observation`), value kind (`intensity`) and variable (NCIT:C184706
15+
% 'Fluorescence Intensity') are shared by any fluorescence measurement; the
16+
% method is what says light-sheet. So a migrated pyramid is an observation
17+
% whose method node is that term, found in ONE query on the field path
18+
% `interaction.method.node` (`interaction` declares `method`, an
19+
% ontology_term {node, name}; ndi.v2.fieldQuery also asks the block's name
20+
% before 2026-10-08, `subject_interaction`).
21+
%
22+
% (Until 2026-10-15 it was found through the builder `software` entity's
23+
% name. The entity is still written; it no longer marks the pyramid.)
2124
%
2225
% This is why lightsheet has no ndi.vintage.map row. ndi.vintage.entryFor
2326
% keys on the class name alone, and `observation` / `sampled_body` are every
2427
% V_eta statement and array: a row would reroute the edge and field reads
2528
% ndi.vintage.edge / field do for EVERY observation in a session (e.g. the
2629
% time-reference reads in ndi.file.navigator.bodies), not just pyramids.
2730
%
28-
% The software entity is matched in both shapes did-schema has used: an
29-
% `entity` with name (current) or a `software` document with name (before
30-
% 2026-10-08).
31-
%
32-
% See also: ndi.vintage.lightsheetLevelDocs, ndi.vintage.lightsheetPyramid.
31+
% See also: ndi.vintage.lightsheetMethod, ndi.vintage.lightsheetLevelDocs,
32+
% ndi.vintage.lightsheetPyramid.
3333

34-
BUILDER = ndi.vintage.lightsheetBuilderName();
34+
M = ndi.vintage.lightsheetMethod();
3535

3636
docs = session.database_search(ndi.query('', 'isa', 'lightsheetZarrPyramid'));
3737
docs = reshape(docs, 1, []);
3838

39-
sw = session.database_search(ndi.query('entity.name', 'exact_string', BUILDER, '') | ...
40-
ndi.query('software.name', 'exact_string', BUILDER, ''));
41-
seen = {};
42-
for i = 1:numel(sw)
43-
obs = session.database_search(ndi.v2.isaQuery('observation') & ...
44-
ndi.query('', 'depends_on', 'software_id', sw{i}.id()));
45-
for j = 1:numel(obs)
46-
id = obs{j}.id();
47-
if any(strcmp(seen, id)), continue; end
48-
seen{end+1} = id; %#ok<AGROW>
49-
docs{end+1} = obs{j}; %#ok<AGROW>
50-
end
39+
obs = session.database_search(ndi.v2.isaQuery('observation') & ...
40+
ndi.v2.fieldQuery('interaction.method.node', 'exact_string', M.node));
41+
seen = cellfun(@(d) d.id(), docs, 'UniformOutput', false);
42+
for j = 1:numel(obs)
43+
id = obs{j}.id();
44+
if any(strcmp(seen, id)), continue; end
45+
seen{end+1} = id; %#ok<AGROW>
46+
docs{end+1} = obs{j}; %#ok<AGROW>
5147
end
5248
end

‎tests/+ndi/+unittest/+vintage/LightsheetFixtures.m‎

Lines changed: 14 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,19 +7,22 @@
77
% did2.build.key: a regular axis's origin and spacing are {value,
88
% source_value} pairs) and the store-record
99
% half of +did2/+convert/foldGenericFiles.m, DID-matlab
10-
% claude/ecstatic-pascal-md91fw at 000aeb8), AFTER the batch pass:
10+
% claude/ecstatic-pascal-md91fw at f363d98), AFTER the batch pass:
1111
%
1212
% pyramid an `observation` (value kind `intensity`) with the v1 pyramid's
1313
% id; entity_id = the subject, instrument_id = the element,
1414
% software_id = the builder `entity`; value.data_type 'uint16',
1515
% value.source_data_type '<u2' (the v1 dtype), data_body true;
16-
% variable NCIT:C184706 'Fluorescence Intensity'.
17-
% software an `entity` of type software named
18-
% ndi.vintage.lightsheetBuilderName().
16+
% variable NCIT:C184706 'Fluorescence Intensity'; method
17+
% ndi.vintage.lightsheetMethod() (FBbi:00000369, Jess
18+
% 2026-10-15), the mark ndi.vintage.lightsheetPyramidDocs finds;
19+
% notes = the v1 label (Jess 2026-10-15).
20+
% software an `entity` of type software named for the builder (the
21+
% migrator's BUILDER; no longer the finder's mark).
1922
% store the v1 fileReference as an UNHELD `opaque_body` owned by the
2023
% observation, format EMPTY:0000259 OME-Zarr, carrying the
2124
% full-resolution keys and a description holding the pyramid's
22-
% label / NGFF name / type.
25+
% NGFF multiscales name (no label, no type: Jess 2026-10-15).
2326
% level a `sampled_body` with the v1 level's id, owner_id = the
2427
% observation; keys t,c,z,y,x (t in second, c named by channel,
2528
% z/y/x converted to meter with the micrometre source kept);
@@ -45,7 +48,9 @@
4548
s.depends_on = struct('name', {'entity_id', 'instrument_id', 'software_id'}, ...
4649
'document_id', {'subj_ls_1', 'elem_scope_1', softwareId});
4750
s.statement = struct('variable', ...
48-
struct('node', 'NCIT:C184706', 'name', 'Fluorescence Intensity'));
51+
struct('node', 'NCIT:C184706', 'name', 'Fluorescence Intensity'), ...
52+
'notes', 'brain 7 cleared');
53+
s.interaction = struct('method', ndi.vintage.lightsheetMethod());
4954
s.value = struct('data_type', 'uint16', 'source_data_type', '<u2', ...
5055
'data_body', true);
5156
end
@@ -54,7 +59,7 @@
5459
s = ndi.unittest.vintage.LightsheetFixtures.etaDoc('entity', {'base'}, id);
5560
s.depends_on = struct('name', {}, 'document_id', {});
5661
s.entity = struct('type', struct('node', '', 'name', 'software'), ...
57-
'name', ndi.vintage.lightsheetBuilderName());
62+
'name', 'NDI lightsheet pyramid builder (ndi.fun.doc.lightsheet.makePyramid)');
5863
end
5964

6065
function s = store(id, ownerId)
@@ -66,8 +71,8 @@
6671
'format', struct('node', 'EMPTY:0000259', 'name', 'OME-Zarr'), ...
6772
'filename', 'brain7.ome.zarr', ...
6873
'description', ['OME-Zarr store, not held in the database; ' ...
69-
'label "brain 7 cleared"; NGFF multiscales name "mean"; ' ...
70-
'NGFF multiscales type "mean"; .zattrs MD5 0123456789abcdef'], ...
74+
'NGFF multiscales name "mean"; recorded by location, not ' ...
75+
'ingested; .zattrs MD5 0123456789abcdef'], ...
7176
'redundant', false);
7277
end
7378

‎tests/+ndi/+unittest/+vintage/TestLightsheetMigrated.m‎

Lines changed: 53 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
% ndi.session.dir, and asks the same readers.
1313
%
1414
% NEEDS DID-matlab's lightsheet migrators (claude/ecstatic-pascal-md91fw,
15-
% 000aeb8 and later) and the composition did-schema they target, so
15+
% f363d98 and later: the method and the label on notes, Jess 2026-10-15) and the composition did-schema they target, so
1616
% test-import-v2.yml runs it on the `next` leg only. It is not filtered on
1717
% `current`; it is not selected there.
1818
%
@@ -60,27 +60,66 @@ function migrateAndWrite(testCase)
6060

6161
methods (Test)
6262

63-
function testTheBuilderNameIsTheMigrators(testCase)
63+
function testTheMethodIsTheMigrators(testCase)
6464
% ndi.vintage.lightsheetPyramidDocs finds migrated pyramids by
65-
% this one string; it must be the migrator's, character for
66-
% character.
67-
names = {};
65+
% this method node (Jess 2026-10-15), on the path
66+
% interaction.method.node; it must be the migrator's, character
67+
% for character, and at that path.
68+
obs = [];
6869
for k = 1:numel(testCase.Out)
69-
d = testCase.Out{k};
70-
if strcmp(d.document_class.class_name, 'entity') || ...
71-
strcmp(d.document_class.class_name, 'software')
72-
n = ndi.v2.blockOf(d, 'software', 'name', '');
73-
if ~isempty(n), names{end+1} = char(n); end %#ok<AGROW>
74-
end
70+
if strcmp(testCase.Out{k}.base.id, 'pyr_ls_1'), obs = testCase.Out{k}; end
7571
end
76-
testCase.verifyTrue(any(strcmp(names, ndi.vintage.lightsheetBuilderName())), ...
77-
sprintf('software entities in the migration: %s', strjoin(names, ' | ')));
72+
testCase.assertNotEmpty(obs, 'the pyramid''s observation');
73+
m = ndi.v2.blockOf(obs, 'interaction', 'method', []);
74+
if iscell(m), m = m{1}; end
75+
testCase.assertTrue(isstruct(m) && isfield(m, 'node'), ...
76+
'the migrated pyramid carries interaction.method');
77+
M = ndi.vintage.lightsheetMethod();
78+
testCase.verifyEqual(char(m.node), M.node);
79+
testCase.verifyEqual(char(m.name), M.name);
80+
end
81+
82+
function testTheLabelStaysOnTheObservation(testCase)
83+
% Jess 2026-10-15: the label is the observation's notes after the
84+
% batch pass, and the store body's description holds the NGFF
85+
% multiscales name only.
86+
d = testCase.Session.database_search(ndi.query('base.id', 'exact_string', 'pyr_ls_1'));
87+
testCase.assertNumElements(d, 1);
88+
testCase.verifyEqual(char(ndi.v2.blockOf(d{1}.document_properties, ...
89+
'statement', 'notes', '')), 'brain 7 cleared');
90+
[p, vintage] = ndi.vintage.lightsheetPyramid(d{1}, testCase.Session);
91+
testCase.verifyEqual(vintage, 'V_eta');
92+
testCase.verifyEqual(p.label, 'brain 7 cleared');
93+
testCase.verifyEqual(p.pyramid_name, 'mean');
94+
testCase.verifyEqual(p.pyramid_type, '', 'pyramid_type is dropped');
95+
end
96+
97+
function testWithoutThePassTheReaderStillGetsTheLabel(testCase)
98+
% A bare v1_to_v2 call leaves the multiscales name after the
99+
% migrator's separator in notes; the reader's separator must be
100+
% the same literal, so the label it reads is the label alone.
101+
out = did2.convert.v1_to_v2({ndi.unittest.vintage.TestLightsheetMigrated.v1Pyramid()}, ...
102+
'Validate', true, 'TargetVersion', 'V_eta');
103+
testCase.assertEmpty(out.quarantine, ...
104+
ndi.unittest.vintage.TestLightsheetMigrated.reasons(out));
105+
obs = [];
106+
for k = 1:numel(out.migrated)
107+
d = out.migrated{k}.toStruct();
108+
if strcmp(d.base.id, 'pyr_ls_1'), obs = d; end
109+
end
110+
testCase.assertNotEmpty(obs);
111+
notes = char(ndi.v2.blockOf(obs, 'statement', 'notes', ''));
112+
testCase.verifySubstring(notes, 'NGFF multiscales name "mean"', ...
113+
'without the pass the name is still on the notes');
114+
p = ndi.vintage.lightsheetPyramid(obs);
115+
testCase.verifyEqual(p.label, 'brain 7 cleared');
78116
end
79117

80118
function testTheMigratedPyramidIsFoundAndListed(testCase)
81119
docs = ndi.vintage.lightsheetPyramidDocs(testCase.Session);
82120
ids = cellfun(@(d) d.id(), docs, 'UniformOutput', false);
83-
testCase.verifyEqual(ids, {'pyr_ls_1'}, 'the pyramid keeps its id');
121+
testCase.verifyEqual(ids, {'pyr_ls_1'}, ...
122+
'the pyramid keeps its id, and is found by its method');
84123
rows = ndi.gui.app.LightsheetZarrManager.pyramidRows(testCase.Session);
85124
testCase.verifyNumElements(rows, 1);
86125
testCase.verifyEqual(rows(1).subjectID, 'subj_ls_1');

‎tests/+ndi/+unittest/+vintage/TestLightsheetMigratedSession.m‎

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,14 @@ function needV2(testCase)
4141
mkdir(bytes);
4242
testCase.Member0 = F.tempFile(bytes, 'uid_level0_chunk0', uint8(1:8));
4343
m1 = F.tempFile(bytes, 'uid_level0_chunk1', uint8(9:16));
44+
% A DECOY: an intensity observation with the same builder
45+
% software but no light-sheet method. The finder keys on the
46+
% method (Jess 2026-10-15), so it must NOT be listed.
47+
decoy = F.pyramid('obs_not_lightsheet', 'sw_ls_1');
48+
decoy.interaction = struct('method', struct('node', '', 'name', ''));
49+
decoy.statement.notes = '';
4450
docs = {F.software('sw_ls_1'), ...
45-
F.pyramid('pyr_ls_1', 'sw_ls_1'), ...
51+
F.pyramid('pyr_ls_1', 'sw_ls_1'), decoy, ...
4652
F.store('fref_store_1', 'pyr_ls_1'), ...
4753
F.level('lvl_0', 'pyr_ls_1', 1, '', {testCase.Member0, m1}), ...
4854
F.level('lvl_max1', 'pyr_ls_1', 2, 'max', {}), ...
@@ -61,7 +67,8 @@ function testTheMigratedPyramidIsFound(testCase)
6167
docs = ndi.vintage.lightsheetPyramidDocs(testCase.Session);
6268
ids = cellfun(@(d) d.id(), docs, 'UniformOutput', false);
6369
testCase.verifyEqual(ids, {'pyr_ls_1'}, ...
64-
'exactly the observation whose software is the pyramid builder');
70+
['exactly the observation whose method is light-sheet ' ...
71+
'microscopy, not every observation of the builder software']);
6572
end
6673

6774
function testLevelTableReadsTheMigratedLevels(testCase)
@@ -113,7 +120,8 @@ function testTheManagerListsTheMigratedPyramid(testCase)
113120
rows = ndi.gui.app.LightsheetZarrManager.pyramidRows(testCase.Session);
114121
testCase.verifyNumElements(rows, 1);
115122
testCase.verifyEqual(rows(1).id, 'pyr_ls_1');
116-
testCase.verifyEqual(rows(1).label, 'brain 7 cleared');
123+
testCase.verifyEqual(rows(1).label, 'brain 7 cleared', ...
124+
'the label is the observation''s notes (Jess 2026-10-15)');
117125
testCase.verifyEqual(rows(1).subjectID, 'subj_ls_1');
118126
testCase.verifyEqual(rows(1).nLevels, 3);
119127
testCase.verifyEqual(rows(1).reduction, 'max/mean');

0 commit comments

Comments
 (0)