Repository navigation
Wait up to 60 s for each bulk document download chunk - #331
Merged
Merged
Conversation
downloadDocumentCollection polls each chunk's presigned ZIP URL while the server builds the archive, and gave up after 20 s. A chunk of 2000 large documents can take longer, which surfaced as a TimeoutError partway through downloading a dataset. Raise the default to 60 s in both downloadDocumentCollection and _download_chunk_zip, matching NDI-matlab#1030, and update the bridge entry's default to match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T41FUDkgY9DAstG7LzZHaV
tuningcurve: NDI-matlab 49a875c94 (VH-Lab/NDI-matlab#1029) fixed the single-property deal index in calculate; the Python calculate is still a stub with no deal loop, so nothing to port. Recorded in decision_log. downloadDocumentCollection: NDI-matlab 2669a8525 (the paired claude/download-timeout-60s branch) raises the default Timeout to 60 s, which this PR ports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T41FUDkgY9DAstG7LzZHaV
Matches NDI-matlab 5b0a3a87a on the paired branch: 60 s is still too short for a chunk of large documents while the server builds its ZIP. Bridge default and sync hash follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T41FUDkgY9DAstG7LzZHaV
stevevanhooser
changed the title
Matches NDI-matlab e82a607f0 on the paired branch. Bridge default and sync hash follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T41FUDkgY9DAstG7LzZHaV
stevevanhooser
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Raises the default
timeoutfor bulk document downloads from 20 to 60 seconds, to match VH-Lab/NDI-matlab#1030.Why
For each chunk of up to 2000 documents,
downloadDocumentCollectionpolls the chunk's presigned ZIP URL while the server's background job builds the archive. That job may run for up to 15 minutes, but the client gave up after 20 seconds. A chunk of large documents can take longer than that to build, and the download then failed partway through: aTimeoutErrorin Python, a 404 from curl in MATLAB. This happened when downloading the calculators paper's example dataset (4549 documents).Changes
src/ndi/cloud/download.py: thetimeoutdefault is 60.0 in bothdownloadDocumentCollectionand_download_chunk_zip, and the docstring now says "Default 60 matches MATLAB".src/ndi/cloud/ndi_matlab_python_bridge.yaml: thetimeoutdefault fordownloadDocumentCollectionis "60.0", andmatlab_last_sync_hashis NDI-matlabe82a607f0(the head of the paired NDI-matlab branch, which CI checks out).src/ndi/calc/stimulus/ndi_matlab_python_bridge.yaml:tuningcurve'smatlab_last_sync_hashmoves to49a875c94(Fix tuningcurve calculator dealing over a single stimulus property VH-Lab/NDI-matlab#1029, the single-property deal fix). Nothing to port, because the Pythoncalculateis still a stub with no deal loop. Thedecision_logrecords this.CHANGELOG.md: an entry under Unreleased / Fixed.Merge VH-Lab/NDI-matlab#1030 first, with a merge commit, so
e82a607f0stays in NDI-matlabmainhistory.Testing
black --checkandruff checkpass ondownload.py.tests/test_matlab_bridge_hashes.pypasses locally against the paired NDI-matlab branch.https://claude.ai/code/session_01T41FUDkgY9DAstG7LzZHaV
🤖 Generated with Claude Code
Generated by Claude Code