Merge pull request #331 from Waltham-Data-Science/claude/download-timeout-60s

Wait up to 60 s for each bulk document download chunk

Author: stevevanhooser

CHANGELOG.md

@@ -23,6 +23,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ### Fixed
 
+- **Bulk document downloads wait up to 60 s for each chunk** (mirroring
+  NDI-matlab#1030). `downloadDocumentCollection` polls each chunk's
+  presigned ZIP URL while the server builds the archive, and gave up after
+  20 s; a chunk of 2000 large documents can take longer, which surfaced as a
+  `TimeoutError` (in MATLAB, a 404 from curl) partway through downloading a
+  dataset. The default `timeout` is now 60 s in both
+  `downloadDocumentCollection` and `_download_chunk_zip`.
+
 - **`ndi.probe.timeseries.readtimeseries()` names its first argument
   `timeref_or_epoch`, matching MATLAB and `ndi.element.timeseries`**
   (NDI-python#166). It was called `epoch`, so the two classes could not be

src/ndi/calc/stimulus/ndi_matlab_python_bridge.yaml

@@ -18,7 +18,7 @@ classes:
     type: class
     status: porting_deferred
     matlab_path: "src/ndi/+ndi/+calc/+stimulus/tuningcurve.m"
-    matlab_last_sync_hash: "63043f393"
+    matlab_last_sync_hash: "49a875c94"
     python_path: "ndi/calc/stimulus/tuningcurve.py"
     python_class: "ndi_calc_stimulus_tuningcurve"
     inherits: "ndi.calculator"
@@ -50,6 +50,11 @@ classes:
       the MATLAB without a populated session to check against is how a
       plausible wrong answer gets committed, and this entry has already
       carried one claim too many.
+      RE-EXAMINED AT 49a875c94 (VH-Lab/NDI-matlab#1029): calculate now sets
+      the deal index to the loop index when one property is dealt (it was
+      fixed at 1, so every dealt curve used the first value) and accumulates
+      deal_log_str. Nothing to follow here: the port's calculate is the stub
+      described above and has no deal loop. Hash bumped only.
 
     methods:
       - name: tuningcurve

src/ndi/cloud/download.py

@@ -114,7 +114,7 @@ def _check_zip_ratio(zf: Any) -> None:
 
 def _download_chunk_zip(
     url: str,
-    timeout: float = 20.0,
+    timeout: float = 60.0,
     retry_interval: float = 1.0,
 ) -> list[dict[str, Any]]:
     """Download and extract a bulk-download ZIP from a presigned S3 URL.
@@ -214,7 +214,7 @@ def downloadDocumentCollection(
     dataset_id: str,
     doc_ids: list[str] | None = None,
     chunk_size: int = 2000,
-    timeout: float = 20.0,
+    timeout: float = 60.0,
     retry_interval: float = 1.0,
     progress: Callable[[str], None] | None = None,
     *,
@@ -234,7 +234,7 @@ def downloadDocumentCollection(
         chunk_size: Number of documents per bulk-download request.
             Default 2000 matches MATLAB.
         timeout: Seconds to wait for each chunk's ZIP to become
-            available. Default 20 matches MATLAB.
+            available. Default 60 matches MATLAB.
         retry_interval: Seconds between polling attempts for each
             chunk. Default 1 matches MATLAB.
         progress: Optional callback for status messages.

src/ndi/cloud/ndi_matlab_python_bridge.yaml

@@ -617,7 +617,7 @@ functions:
   # --- download.py functions ---
   - name: downloadDocumentCollection
     matlab_path: "+ndi/+cloud/+download/downloadDocumentCollection.m"
-    matlab_last_sync_hash: "e1a641f0b"
+    matlab_last_sync_hash: "e82a607f0"
     python_path: "ndi/cloud/download.py"
     input_arguments:
       - name: dataset_id
@@ -633,7 +633,7 @@ functions:
       - name: timeout
         type_matlab: "double"
         type_python: "float"
-        default: "20.0"
+        default: "60.0"
       - name: retry_interval
         type_python: "float"
         default: "1.0"