[arXiv Research] Agentic Workflow Improvements — 2026-10-15

[github-actions[bot]]

Summary

25 papers screened, 15 relevant, 3 opportunities identified.

---

Actionable Opportunities

Context and memory management — Fresh-context restart primitive

Paper: Scaling Discovery Requires Forgetting
Authors: Shangyin Tan, Ben Zhou, Lakshya A Agrawal
Published: 2026-10-14
Effort: medium
Rationale: The paper reports that accumulated history anchors agents to earlier attempts, causing long-running and parallel discovery to plateau, while discarding search state restores exploration. gh-aw already provides sub-agents and persistent memory, so it can implement controlled forgetting while preserving explicitly selected durable artifacts rather than carrying the full transcript.

---

Engine conformance and harness evaluation — Add a frozen harness conformance benchmark

Paper: Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents
Authors: Yuyao Ge, Shenghua Liu, Peizhong Gao
Published: 2026-10-14
Effort: medium
Rationale: Doc2Harness separates harness construction from downstream execution and shows that even the strongest reported system passes only 53.05%, with permission cases particularly discriminative and performance varying by subsystem. gh-aw has multiple pluggable engines plus security-critical harness behavior, so a frozen, contract-driven test corpus would measure implementation fidelity independently of model/task quality and expose unsafe permission mismatches that aggregate end-to-end tests can hide.

---

Safe-output validation — Add non-vacuity checks to safe outputs

Paper: Beyond LLM Judge: Mechanical Verification for Agentic Autoformalization
Authors: Anna Mei, Michael R. Douglas
Published: 2026-10-14
Effort: medium
Rationale: The paper reduces incorrectly accepted formalizations by pairing semantic LLM review with deterministic numerical and vacuity checks. Applying the same separation to gh-aw would catch safe-output validators that look plausible but are tautological, ignore a field, or fail at numerical boundaries, while keeping enforcement reproducible and independent of model judgment.

---

Papers Analyzed

Paper	Published	Relevant	Area
Co-Evolving Acting and Forecasting for Language Agents	2026-10-14	Yes	Agent planning and evaluation
Price of Anarchy of First-Price Auctions in Agentic Markets	2026-10-14	No	—
Player-1	2026-10-14	No	—
One Task, Many Interfaces	2026-10-14	Yes	Cross-engine tool-interface robustness
Beyond LLM Judge	2026-10-14	Yes	Safe-output validation
Scaling Discovery Requires Forgetting	2026-10-14	Yes	Context and memory management
PhySGA	2026-10-14	No	—
PerSo	2026-10-14	No	—
ActiveEvidence	2026-10-14	Yes	Tool-call optimization
The Wealth of Agents	2026-10-14	Yes	Sub-agent orchestration
Bayesian Learning to Select Fair Classifiers	2026-10-14	No	—
Hindsight-Privileged RLVR	2026-10-14	No	—
Average-Reward Policy Identification	2026-10-14	No	—
From Training to Escalation	2026-10-14	Yes	Security and supply-chain isolation
SwarmRSI	2026-10-14	Yes	Evaluation and iterative sub-agent workflows
What Should a Software Team Spend on Agents?	2026-10-14	Yes	Cost and token optimization
From Relevance to Validity	2026-10-14	Yes	MCP tool selection and validation
Doc2Harness	2026-10-14	Yes	Harness testing
TreeFlex	2026-10-14	Yes	Context and checkpoint management
Reflect After Failure	2026-10-14	Yes	Failure recovery and memory
MMLA-RSI	2026-10-14	No	—
CASI	2026-10-14	Yes	Evaluation and credit assignment
TANGO	2026-10-14	No	—
Harness Policy Distillation	2026-10-14	Yes	Runtime contract verification
Long-Horizon RL Needs Muon	2026-10-14	No	—

---

Next Steps

• Investigate: prototype periodic fresh-session restarts and measure diversity, verifier score, token use, and success rate (effort: medium)
• Investigate: prototype ten contract-derived harness scenarios across two engines with permission and safe-output cases (effort: medium)
• Investigate: prototype mutation-based non-vacuity tests for two safe-output operations (effort: medium)

Quick-Win Agentic Prompts

@copilot Implement: Add a configurable workflow primitive that periodically checkpoints only selected evidence and best candidates to cache-memory or repo-memory, starts a fresh agent or sub-agent session without prior conversation history, and injects the compact checkpoint into the new session. in the gh-aw Context and memory management component. Rationale: The paper reports that accumulated history anchors agents to earlier attempts, causing long-running and parallel discovery to plateau, while discarding search state restores exploration. gh-aw already provides sub-agents and persistent memory, so it can implement controlled forgetting while preserving explicitly selected durable artifacts rather than carrying the full transcript. Source: Scaling Discovery Requires Forgetting.

@copilot Implement: Create a Doc2Harness-style gh-aw conformance suite that gives an engine/harness implementation only the documented interface contract, compiles a representative workflow corpus, freezes the resulting harness, and evaluates it in isolated environments across separate subsystems—tool invocation, workspace mutation, safe outputs, network/firewall behavior, and especially permission enforcement. Report per-subsystem scores as well as overall task success so engine-specific regressions and ranking reversals remain visible. in the gh-aw Engine conformance and harness evaluation component. Rationale: Doc2Harness separates harness construction from downstream execution and shows that even the strongest reported system passes only 53.05%, with permission cases particularly discriminative and performance varying by subsystem. gh-aw has multiple pluggable engines plus security-critical harness behavior, so a frozen, contract-driven test corpus would measure implementation fidelity independently of model/task quality and expose unsafe permission mismatches that aggregate end-to-end tests can hide. Source: Doc2Harness.

@copilot Implement: Extend safe-output schema compilation and gh aw audit with mechanical postcondition tests: generate valid, boundary, and deliberately invalid fixtures; mutate each constrained field; and require the validator to accept intended outputs while rejecting violations and demonstrating that every declared constraint can affect the verdict. in the gh-aw Safe-output validation component. Rationale: The paper reduces incorrectly accepted formalizations by pairing semantic LLM review with deterministic numerical and vacuity checks. Applying the same separation to gh-aw would catch safe-output validators that look plausible but are tautological, ignore a field, or fail at numerical boundaries, while keeping enforcement reproducible and independent of model judgment. Source: Beyond LLM Judge.

References: §38707214484

Generated by 🔬 arXiv Paper Researcher: GitHub Agentic Workflows · codex · gpt56 · 213 AIC · ⌖ 13.1 AIC · ⊞ 18.1K · ◷

• expires on Oct 29, 2026, 12:47 AM UTC-08:00