Status: Proposed research mission; no field effectiveness established. Research orbit: HUBBLE.
Original proposal · Domain model · Protocol
Thesis. An assistant is useful only if analysts can distinguish sourced observation from model inference. System. Give the model a versioned, access-controlled evidence store; require each claim to cite a record ID and time interval, with unsupported claims labeled as hypotheses. A deterministic checker confirms citation existence and access scope before a report is shown. Inputs. Synthetic incident cases and separately authorized, redacted records. Falsifiable metric. Supported-claim precision, missing-citation rate and analyst time to verify conclusions compared with a text-only baseline. Limitation. An existing citation can still be misinterpreted; human review and evidence replay remain necessary. See NIST AI RMF and NIST incident response.
Treat the statement “An assistant is useful only if analysts can distinguish sourced observation from model inference.” as a hypothesis with a comparator and a population, not as a demonstrated benefit. Implement the precise mechanism described above; preserve the original evidence boundary and endpoint. Use the domain model to state variables and units, but justify which assumptions apply to this particular mechanism rather than copying the model mechanically.
Observation contract: collect only the fields needed to estimate this mission's endpoint. Candidate domain fields are Case identifier, time split, evidence availability, score, label provenance, recommendation, human outcome. Bind every record to source/version, valid-time interval, missingness reason and a fixture or collection permission. The domain resource is Telemanom SMAP/MSL anomaly data and original study; its applicability is a candidate starting point, not an assertion that every field required by this mission exists there. When fields or labels are absent, design a synthetic or owned-system fixture and report the gap.
Baseline and falsification: use the original comparison stated above and at least one applicable simple baseline from Rule-based queue; uncalibrated model; always-abstain policy; same model without evidence check. Match information, tuning, observation and resource budgets. Define an absolute acceptance floor and minimum meaningful improvement before collecting the final test results. Reject the thesis if the benefit disappears on an untouched partition, if the resource-normalized baseline dominates it or if a declared invariant fails. Choose numerical floors with the mission owner; this dossier does not invent measured thresholds.
Adverse-case matrix: Missing evidence; contradictory sources; benign drift; unseen channel; reduced prevalence. Choose the cases relevant to this mechanism and explain any exclusion. Add a negative control in which the intervention should have no effect. Preserve ambiguous outcomes instead of assigning unsupported truth labels.
Graduate investigation: Temporal leakage; unsupported explanations; selective labels; workload-dependent intervention. Write a rival explanation for any apparent improvement; identify the smallest additional observation that distinguishes it from the proposed mechanism. Perform a sensitivity analysis over uncertain assumptions and a leave-one-environment/scenario-out evaluation. A performance difference within a synthetic fixture establishes only behavior of that fixture.
Review artifacts: model/assumption ledger; versioned input contract; baseline implementation; preregistered endpoint; adverse-case results with uncertainty; resource-use record; negative findings; independent review notes. The practical starting budget is Offline curated cases and public telemetry; model output is analysis advice with bounded authority. Independent reviewer sign-off remains pending.
Domain references and data caveats · Applicable resource ledger