Commit History
-
Wave 21: adversarial-review fixes — all 9 verified findings closed
Baladithya Balamurugan +1: Claude Fable 53 reviewers (correctness/security/fidelity) + independent verifier over the Stage-0 pipeline commit; every P0/P1 empirically confirmed before fixing. P0: reverse_unified_diff silently corrupted file-add/delete patches (git apply rc 128: naive +/- swap inverts the ---/+++ order and leaves a stale `new file mode` header). Fix: route them to the UNREVERSED_MARKER fallback; regression test with a /dev/null patch. P1: DECONTAMINATION_LIST omitted SWE-Gym's 11 eval repos (the architecture named both families) — added; V3 now fully closed; regression test. P1: COMPOSER_RECIPE_MAPPING.md:20 still asserted KL(teacher||student) as blog-stated — corrected to direction-unverified per F-12. P1: build_corpus budget was a soft ceiling (check-then-spend overshot by one rollout) — now a pre-charge hard ceiling; test asserts cost <= budget. P2: dedup stats double-counted rows that were both within-run and cross-gen dups (disjoint partition now); run_id path-traversal guard o…3bbcf21 -
Wave 21: Stage-0 dataset pipeline — swesmith engine, rollout harness, gates, contract
Baladithya Balamurugan +1: Claude Fable 5Implements research/deepread/13-synthesis-architecture.md (ADR-016): the "point at a repo -> dataset" pipeline the critical review architected, closing every verified P0 from research/deepread/12-verified-findings.md. New modules (+96 tests, full suite 511 passed / 66 skipped): - datagen/repo_gate.py: SPDX-ish license detection -> 3 tiers (fail-closed) + benchmark DECONTAMINATION vs the SWE-bench eval-repo list (closes V3 — zero decontamination existed anywhere). - datagen/swesmith_adapter.py + [swesmith] extra: SWE-smith as the synthesis engine (closes V4 buy-vs-build). Handles the patch-semantics INVERSION (SWE-smith's patch INTRODUCES the bug; golden_diff = reverse_unified_diff) + strategy provenance sidecar + difficulty priors. - datagen/trajectory.py: canonical trajectory IR (closes D-11). ToolCall canonical_form = v1 divergence-gate action algebra (replaces the whitespace stub, D-3); to_policy_row = THE policy-visible serializer, sentinel-tested to never leak gol…9a2ce20 -
Wave 21: deep-read critical review — 8 source clusters re-read, findings verified
Baladithya Balamurugan +1: Claude Fable 58 Sonnet readers re-fetched every primary source (Composer 2.5 blog verbatim, Composer 2 techreport 2603.24477 full HTML, SWE-smith/SWE-Gym/R2E-Gym/SWE-bench, SDPO/OPSD, GRPO family + Comedy-of-Estimators, DiLoCo line, world-model cluster + anti-evidence, LATS/ToT/rStar/Tree-GRPO/SWE-Search/Symphony/Socratic-SWE, TRL/verl/SkyRL live docs, SWE-MiniSandbox) and cross-checked the repo's research notes + vision docs against them. 2 adversarial critics (fidelity, design); every P0/P1 finding independently verified against code + source quotes — 0 refuted. (Feasibility critic stalled on capacity; flagged unreviewed.) Top verified findings: - SDPO "mathematically the same" claim is WRONG (Cursor cites it as background; ours is a third, blog-inspired design) — to be corrected. - Envisioned tree pipeline had 4 structural breaks: seed-trace/oracle disjointness, NO rollout harness (SFT corpus had no producer), divergence gate uncomputable (whitespace-stub normalizer), no Sandbox.fork(). - …2a16b30
-
Wave 20: fix SageMaker smoke — torch-2.7 DLC + drop vllm pin (the real conflict)
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)Second live run peeled the fundamental conflict: trl 1.5.1 requires transformers>=4.56.2, which references torch.float8_e8m0fnu (torch>=2.7). So pinning transformers DOWN (last commit) was impossible, and the torch-2.6 DLC was the wrong base. Fixes: - launcher + Dockerfile: use the torch-2.7 DLC 2.7.1-gpu-py312-cu128-ubuntu22.04-sagemaker-v1.26 (pullable, verified). - requirements.txt: drop vllm — vllm==0.8.5 hard-pins torch==2.6.0 and fights the 2.7 base. vLLM-colocate is a rollout speed optimization, not what the smoke proves; the smoke uses model.generate rollout. Resolver dry-run now clean (trl pulls transformers>=4.56.2, satisfiable on torch 2.7). - launcher: --no-vllm → --vllm (opt-in, OFF by default); vLLM needs a baked image with a torch-2.7-matched vllm>=0.9. - quickstart: document the full trl→transformers→torch chain + the vLLM note. Two failed runs cost ~$0.30 total (365 + 316 billable sec). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply…a578ad9 -
Wave 20: fix SageMaker smoke — pin transformers<4.53 (torch 2.6 float8 skew)
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)First live run (composer-grpo-smoke-...18-35-31) reached Training then failed at AutoModel.from_pretrained: AttributeError module 'torch' has no attribute 'float8_e8m0fnu'. transformers 4.53+ references that dtype at import (MXFP4 quant) but it only exists in torch 2.7; the DLC ships torch 2.6 (vllm 0.8.5 pins 2.6). The bare ">=4.46" silently pulled 4.53+. Pin >=4.51.1,<4.53 (4.51.1 = vllm 0.8.5's floor, verified against PyPI metadata). Cost of the failed run: 365 billable sec ≈ $0.15. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>498dbd7 -
Wave 20: SageMaker GRPO smoke artifacts (F3) — runnable g5.2xlarge GSM8K
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)The F3 runnable-now RL smoke, with two live-verified corrections to the design doc's recipe: - DLC tag is cu126 + requires the -v1.25 build suffix (NOT the cu124 bare tag in the AWS docs page; resolved against the live ECR registry). - sagemaker SDK pinned <3: v3 is an API rewrite that dropped sagemaker.estimator.Estimator / sagemaker.pytorch. Artifacts: - examples/gsm8k_grpo/run_sagemaker.py: GPU+vLLM-colocate variant of run.py (same GSM8K RLVR reward, plain GRPO alpha=beta=0), CLI/JSON hyperparams, writes to SM_MODEL_DIR. - examples/gsm8k_grpo/run_sagemaker_launch.py: sagemaker-v2 Estimator driver. Default --image dlc uses the stock PyTorch DLC + source_dir (no local 15GB cross-arch build); --image <ecr> uses a prebuilt baked image. Stages a minimal source_dir (framework + script + requirements). - examples/gsm8k_grpo/requirements.txt: trl + vllm==0.8.5 (torch-2.6 line) + RL stack, installed at job start atop the DLC's torch 2.6/cu126. - d…7453f13 -
Wave 20: Tier-0 fidelity fixes — k1-in-reward KL + Composer-2 behavior rewards
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)Closes the two cheapest "next-level" gaps from the F5 fidelity audit (Rubric A items c2, h, i). Both reproduce Composer-2 §4.1/§4.2 exactly, without the hint mystery. 33 new tests; full suite 415 passed / 66 skipped. (1) k1-in-reward KL (F5 #1, the highest-fidelity-leverage single change). Composer-2 §4.1 chooses the k1 estimator IN THE REWARD (-log r); TRL uses k3 IN THE LOSS. The 2025/26 evidence (arXiv:2512.21852, verl k1-only, TRL #4967) says k1-in-reward improves OOD generalization where k3-in-reward can collapse — exactly the "take any model to the next level" axis. - trainer/kl_in_reward.py: pure, tested math. k1/k3 per-sequence estimators + apply_kl_in_reward, which adjusts GRPO advantages by -coef·(KL - group_mean(KL)). test_kl_in_reward.py PROVES this exactly equals folding -coef·KL into the reward then re-running GRPO's group-mean baseline (9 group/gen combos) — the justification for not forking TRL's scoring code. Exact only without std-norm (Dr.GRPO …41289bf -
Wave 20: close F4 s3:// gap — live-S3 DiLoCo allreduce smoke (AWS_SMOKE-gated)
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)ObjectStoreAllReduce's S3 branches (_init_fsspec/_put/_exists/_get over s3fs) previously had only mock coverage; F4 flagged "s3:// never exercised against real S3" as the highest-value unproven gap. Adds test_s3_rendezvous_allreduce_across_replicas to test_serverless_local.py (torchft-free, so it runs on Apple Silicon CI too): 2 OS processes call ObjectStoreAllReduce.allreduce() over an s3:// rendezvous, every rank must end with the cross-rank mean. Reuses the existing spawn-importable _replica_compute_and_sync. Gated on AWS_SMOKE=1; skips cleanly otherwise. Verified live 2026-06-09 against s3://amazon-sagemaker-386931836011-us-west-2-7597bf4d9a3d/diloco-rdv/: both ranks converged to identical weights (max|diff|=0.00e+00), both round_000000/rank_000{0,1}.pt objects present in S3 — proving the PUT->poll->GET->mean path and S3 strong read-after-write consistency hold cross-process. Full file: 10 passed, 1 skipped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>bd37412 -
Wave 20: 5-facet AWS-native architecture design (F1-F5)
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)Answers the dataset-gen+SFT-vs-RL-vs-BOTH question with 5 grounded facet designs from the aws-native-architecture-design workflow: - F1 systems-framing: BOTH = two loops at two timescales, not two phases; component->loop mapping table; SFT-first ordering; S3 dataset contract (6 typed prefixes); repo delta (tree_controller, s3_layout, sft_floor). - F2 aws-datagen: per-stage native verdict (Glue 5.0 Spark ingest -> Bedrock batch replay -> AWS Batch array on Spot test-exec -> EMR Serverless normalize), Step Functions orchestration. - F3 rl-sagemaker: runnable-now g5.2xlarge GSM8K GRPO smoke (<$1), colocated vLLM, BYO container from PyTorch DLC; live-verified quota. - F4 decoupled-diloco-s3: N single-instance jobs, ObjectStoreAllReduce PUT-poll-mean over S3, strong-consistency correctness, IAM gotcha, cheapest validating-run ladder. - F5 fidelity-audit: rubric vs Composer 2.5/Composer-2 + load-bearing papers; 5 FULLY-REPLICATED, k3-vs-k1 KL documented infidelity, missing…50e3f0e -
R14: harden live-Docker tests against transient daemon contention
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)The Phase-8 verifiers flagged 2 live-Docker DockerSandbox tests as flaky under concurrent container resource pressure (full suite + review agents spawning containers) — passing in isolation. Added _retry_docker(): bounded retry (3x, backoff) on transient daemon errors (timeout/resource/conflict/500/connection), re-raising genuine repeatable failures. Wired into the 4-gate inversion test and the network-disabled test. 27 passed / 1 skipped, lint-clean. Closes the last addressable Phase-8 verification finding; backlog now fully resolved on this host (remainder is user-gated GPU/token only). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>c647fe9 -
docs: commit the Wave-2/3 review + Phase-8 verification audit trail (JSON findings)
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>b1adfc9 -
Phase 8: B4-final (last 2 living-doc stale counts) + final verification disposition
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)- B4-final: USER_GUIDE.md:678 + INTEGRATION_RECIPES.md:926 '115-test suite' → 266/62 (the last current-framed stale counts the independent verifier found outside the earlier grep scope; dated-historical + _archive mentions correctly left). - Recorded the Phase-8 disposition: full suite 381 passed/65 skipped/0 failed; independent verifier confirms B1-E3 resolved; submodule-export design note; final backlog disposition (zero actionable-on-host items open; remainder is user-gated GPU/token or tracked pre-existing lint debt). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>c59d939 -
docs: file R13 (pre-existing serverless ruff B904 debt) as tracked backlog item
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>ace4fac -
Wave 3 cleanup: close deferred-LOW review items R5/R6/R11
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)- R5: EKSExecutor.cancel + SageMakerExecutor.cancel now re-raise genuinely unexpected errors instead of swallowing ALL exceptions; only already-terminated signals are idempotent no-ops (EKS: 404/409; SM: ResourceNotFound + already-terminal ValidationException via new _is_already_terminal). + a test that an AccessDenied-class error propagates. - R6: EKSExecutor.collect() result dicts include a 'result' key (the S3 rendezvous URI, or None) for cross-backend shape uniformity with Local/Modal/SageMaker. - R11: seed torch.manual_seed(0) in spike-006 test_alternating_batches_loss_ decreases so the loss-trend assertion is reproducible and no longer flaky under CPU contention (it failed only in the contended full suite, passed in isolation). Root cause was unseeded nondeterminism, not a real regression. All Wave-3 review findings (R1-R11) now closed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>7d9dbbc -
Wave 3: close the HIGH review findings (kill-switch wiring, HeldoutSplit, EKS entrypoint bug)
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)Phase-7 reconciliation of the concurrent review team's findings (research/review-*.json). HIGH: - R1: wire HeldOutGuard into ComposerReplicationTrainer — OPTIONAL + OFF by default (no heldout_guard => byte-identical legacy behavior). When configured, _maybe_update_killswitch folds in-loop reward + injected heldout_eval_fn() + token-mean KL into the guard at the logging cadence; fires a hard (CollapseStopError) or soft (control.should_training_stop) halt. The #2 collapse safeguard now actually fires instead of being dead code. + test_killswitch_integration.py. - R2: build composer_replication/safety/holdout.py — HeldoutSplit disjointness enforcer (id-set + optional content-hash; .split()/.validate()/.assert_disjoint; raises HeldoutOverlapError listing the leak). The un-built second half of C1 that keeps the guard's proxy-real gap signal meaningful. +10 tests. Re-exported from safety/__init__ (resolves the dangling refs). - R3: EKS entrypoint contract bug — replica_e…bd0c358 -
Wave 2: 4 new modules (kill-switch, EKS/SageMaker executors, DockerSandbox) + B4/B7 completion
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)Built by the parallel execution team (worktree-isolated), integrated + tested here. New modules (all CPU-testable via mock/lazy-import; optional deps gated): - composer_replication/safety/ — HeldOutGuard run-level collapse kill-switch (held-out-declines-while-reward-rises streak + KL-to-init hard-stop 0.08 + proxy-real hacking-gap; EMA-denoised, latched-fire, CollapseStopError). The documented #2 safeguard for the self-evolving flywheel. 23 tests. - composer_replication/diloco/serverless/eks.py — EKSExecutor satisfying the ServerlessExecutor Protocol via a SINGLE Kubernetes Indexed Job → N rank-ordered ReplicaHandles, gang-cancel (Background propagation), REPLICA_RANK via the downward API, S3 rendezvous (IRSA). 28 tests (mock BatchV1/CoreV1). - composer_replication/diloco/serverless/sagemaker.py — SageMakerExecutor (boto3 create_training_job, same S3 rendezvous, status mapping). +13-test module written during integration (the build agent shipped it test-less). - compos…7a55e1e -
Wave 1: fix 8 failing tests + unblock Docker E2E + dep/doc debt
Baladithya Balamurugan +1: Claude Opus 4.8 (1M context)Backlog-resolution Wave 1 (branch backlog/goal-resolution-2026-06). Bugs fixed: - B1 (P0): generate spikes/007 synthetic_session_with_error.jsonl fixture (never committed — .gitignore *.jsonl whitelisted the sibling but not this one) → the 8 failing test_trace_examples_adapter tests now pass. Added the missing .gitignore whitelist line. - B2 (P2): [dev] extra was un-installable on Apple Silicon (pulled Linux-only torchft-nightly). Dropped diloco from base [dev]; added [dev-full] for Linux. - B3 (P2): [serverless] extra now includes s3fs/boto3/kubernetes (needed for real S3 rendezvous + the EKSExecutor/SageMakerExecutor adapters). Newly unblocked: - D1 (…-245d): Docker substrate E2E — Docker now available on host; both gates (4-gate substrate inversion + cache-scrub-in-container) PASS. Long-standing "hardware-blocked" item closed. Doc/API debt: - B4: reconciled divergent test counts (115/176/210/232) to one canonical figure measured this session: 266 passed / 62 skipp…c11cf49 -
Merge docs/refine-2026-06: documentation refinement pass
Brings the ccode-ultracode docs engagement (4 waves + 1 reconciliation) into main: channel-3 provenance correction, gap-honesty, dead-link fixes, archived point-in-time wave reviews (with redirect stubs), new OVERVIEW.md, ADR-014 indexing. Reconciled the main-lags-master foot-gun guidance to its RESOLVED state (main is canonical and synced). Docs-only; no source touched.4e6e82e -
docs(reconcile): main-lags-master foot-gun is RESOLVED, not live
The refine brief predated the 2026-06-09 branch fix. main==master now and main is canonical, so "git checkout master after cloning" is stale/backwards. Reframe README + USER_GUIDE + OVERVIEW to "main is canonical & synced; fresh clone works as-is; ImportError = stale clone, git checkout main".d7e4b4e -
docs: add refine-2026-06 engagement summary
Top-level summary of the docs/refine-2026-06 engagement: every file touched across waves 1-3, what changed and why, what was deliberately left alone, two maintainer notes for issues outside docs-only scope (the off-limits ADR-002-channel2-sdpo dead link in examples/, and the API_REFERENCE config-factory gap), and the verification proofs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>5e64616 -
docs: main is canonical branch; retire main-lags-master foot-gun
Record the 2026-06-09 decision that `main` is the canonical branch on both HF repos. main==master converged via fast-forward; push to main going forward. Reframe gotcha #1 from "main lags master" to "RESOLVED, keep it synced" — the Modal SHA pins stay correct but their stale-main rationale no longer applies.ace6dd4 -
docs(wave3): add OVERVIEW.md, index ADR-014, fold in adversarial-review fixes
New-docs + freshness wave, plus corrections from an adversarial review pass on the wave1 commit. Docs-only; no source files touched. New: - docs/OVERVIEW.md — a 5-minute, honestly-scoped newcomer tour: what the framework is, the three channels WITH provenance (1 Dr.GRPO + 2 SDPO = genuine Composer replication; 3 trace-replay-DPO = the framework's own additive channel, NOT Cursor's recipe), what's proven (CPU SDPO-fires, the A1 8B Modal run, GSM8K GRPO, $0.98/trace), and what's gapped (Docker e2e, A2-A4 ladder). Linked from README + both _archive READMEs. ADR index: - Add the missing ADR-014 row + a provenance note recording the channel-3 correction it carries. Adversarial-review corrections (to the wave1 edits): - Drop the parent-commit SHA mislabelled as "HEAD" in the VISION_VALIDATION status banner and the ALTERED_MINDS runnability note; keep the date + ADR ref. - Re-attribute the A2-A4 gap claim: cite ADR-014 only for what it states ("the A1 run used dr_grpo"; obj…e130879 -
docs(wave2): archive point-in-time wave reviews + dated review bundles
Move historical, point-in-time artifacts into _archive/ (preserving git history via rename) and leave one-line redirect stubs at every original path so nothing 404s — including the prose references baked into immutable accepted ADRs (ADR-007/008/012) and off-limits spike verdicts, which cannot be edited to point at the new location. Docs-only; no source files touched. Moved -> docs/research/_archive/ : WAVE_7_10_FINAL_REVIEW, WAVE_13/14/15_FINAL_REVIEW (cross-model reviews). Moved -> docs/_archive/ : DEEP_WORK_LOOP_LOG, WAVE_COMPOSER_DATAGEN_RL_2026-05-29, and the two dated review bundles (reviews/cross-family-adr-008-009-010-2026-05-29/ and reviews/final-verify-deep-work-2026-05-29/). Each origin path keeps a redirect stub pointing to the archived copy and to the current source-of-truth (METHODOLOGY / BACKLOG / V1_V8_COVERAGE / ADRs). Added docs/_archive/README.md and docs/research/_archive/README.md indexing what was archived and why ("point-in-time, superseded by current d…f00833d -
docs(wave1): correctness pass — channel-3 provenance, gap honesty, dead links
Propagate ADR-014 ground truth into the living docs and fix every dead relative link. Docs-only; no source files touched. Provenance (Fact A) — stop blurring the additive trace-replay-DPO channel into Cursor's recipe: - README v0.1 roadmap cell, HF_REPO_LAYOUT v0/v1 variant rows: reframe "Composer recipe" = channels 1 (Dr.GRPO) + 2 (SDPO) only; trace-replay is the framework's own addition (link ADR-014). Gap honesty (Fact D) — do not imply the full A1-A4 LMA ladder is runnable: - VISION_VALIDATION status banner + ALTERED_MINDS_TIE_IN Phase-3: only A1 (GRPO-only) has a real Modal runner; A2/A3/A4 are scaffold + plan-builder only, blocked on dataset construction. Real 8B run is additionally budget-gated. PO menu / strip_thinking / main-lag (Facts B, E, F): - VISION_VALIDATION: base RL objective is a selectable menu (default Dr.GRPO) per ADR-014, not hardcoded. - ALTERED_MINDS_TIE_IN: note SDPO requires strip_thinking=False on real traces (~67% of error-recovery turns are…20e3bd9 -
docs+seeds: file remaining-work issues (Seeds) + project-state doc
Initialize git-native issue tracking (.seeds/) and file 6 remaining-work issues with dependency wiring: - A2 SDPO-only arm (build runner + error-trace dataset) [ready] - A3 replay-DPO-only arm (needs preference corpus) [blocked on A2] - A4 combined arm + final A0-A4 comparison [blocked on A2+A3] - Higher-lr PO-objective sweep (make DAPO/GSPO clip-higher fire) [ready] - Docker substrate e2e (hardware-gated) [ready] - SECURITY: rotate exposed HF token (user-only, P1) [ready] Add docs/PROJECT_STATE_AND_REMAINING_WORK.md: single-page snapshot of what is proven, what remains, the Channel-3-not-from-Composer provenance correction (per ADR-014), and the load-bearing gotchas (main-lags-master, strip_thinking=False for SDPO, OUTPUT_DIR clobber, objective-diagnostic logging). Channel-3 framing corrected to "framework's own additive research channel, NOT Composer's recipe" throughout.fb13ea3 -
seeds: sync 2026-06-09
8d2e6fc
-
feat(trainer): policy-optimization objective MENU (ADR-014)
Adds make_po_config(objective=..., **overrides) — RL's base objective is no longer hardcoded to Dr.GRPO. Six selectable named presets over trl 1.5.0's verified GRPOConfig knob-space (introspected the installed package, not a GitHub snapshot): grpo | vanilla GRPO (std-norm advantage) dr_grpo | DEFAULT; no length-std bias (Composer 2.5's base objective) bnpo | batch-normalized dapo | decoupled clip-higher (epsilon_high=0.28) + overlong mask + KL off gspo | sequence-level importance ratio (Qwen3; long-CoT / MoE stable) cispo | detached-IS REINFORCE (every token keeps a gradient; MiniMax-M1) Every preset is PURE CONFIG (trl already implements each loss_type branch + importance_sampling_level/epsilon_high) — no custom _compute_loss. Drift guards assert loss_type / IS-level / epsilon_high actually applied so a preset can't silently degrade (e.g. GSPO with IS overridden back to token raises). 10 unit tests green against real trl 1.5.0. Research-grounded: Composer …aae66fa
-
docs(adr-013): close acceptance-gate boxes 1-5; only user-gated spend remains
The LMA-side runner scaffold (the one outstanding non-budget-gated box) now exists in the sister repo: llm-mental-alterations/composer_replication_runs/ with 15 green mock-driven tests (collect+pass with no torch/trl/modal/framework installed), documented in LMA ADR-0017. Marks boxes 1-5 done; the only open item is the explicit user go/no-go on real LMA-checkpoint / Modal budget spend.7b34ebf -
fix(phase-8): close all 5 cross-family final-verify findings + regression tests
The deep-work-loop exit gate (2 independent confirmations). A final 3-family scatter of the session diff found and we fixed: - [P0 latent] SDPO sentinel-mask used only student_response_valid; now requires BOTH valid masks AND non-sentinel indices (a future divergent teacher tail could have distilled against clamped position 0). - [P1 convergent, GPT-5.5+Gemini] curriculum shared one n_effort denominator for turns+think_tokens; split into independent n_turns/n_think counters. - [P1] docker E2E test ran pip-install under --network none (would fail when the gate activates); replaced with stdlib python -c expression runner, no network. - [P1] MMLU reward: Answer: A or B hedge scored full credit; _HEDGE_RE now penalizes it as multiple-answers. - [P1] HackMonitor read-markers matched import in important / cat in concatenate and scanned the agent patch as read-evidence; whole-word regex + exclude patch/diff payloads from BOTH layers (a legit patch can no longer self-incriminate…678d10b -
feat(b4-gpu+b6): GPU train-proof on A10G + docker-gated substrate E2E test
B4-GPU: modal_b4_gpu_smoke.py ran real Qwen2.5-0.5B-Instruct on a Modal A10G in bf16, 30 steps of the real 3-channel composition. PASS: all-finite, SDPO channel fired nonzero (max JSD 0.1855), loss converged 4.7262 -> 0.0050 monotone. This is the genuine "it trains on GPU" proof the CPU smoke could not give. Result + curve in gpu_smoke_result.json; README documents both CPU and GPU proofs. B6: test_docker_substrate_e2e.py — the docker-gated 4-gate substrate-inversion E2E now EXISTS (was only referenced). Runs the gates against a real python:3.11-slim container + real subprocess pytest on a minimal synthetic feature-deletion task, plus a cache-scrub-in-container test. Skips cleanly with no Docker (2 skipped here), runs the moment a Docker host executes the suite. ADR-010 [~] gate updated: mechanics gate is now ready-to-close, not just wired. Cost: ~1-3 USD A10G. No LMA budget touched.c009712 -
feat(wave-b): ADR-013 LMA integration + B4 end-to-end SDPO-fires proof + doc refresh
ADR-013: composer_replication/integrations/altered_minds/ (generic, framework-side) - MMLUFormatReward: structured-answer reward, scores only the final letter (not rationale style), unparseable/multiple-answer penalties, option-order randomization, always-C exploit detectable via logged option distribution. - dual_kl_logger: KL(policy||altered-init) AND KL(policy||unaltered-base) as the washout-vs-amplification instrument (neither optimized). - channel_ladder_configs: A0-A4 isolated-channel ladder (alpha=0.02/beta=0.05), replacing the uninterpretable combined alpha=0.2/beta=0.4 recipe per the cross-family research critique (SDPO can AMPLIFY an altered model). B4: examples/altered_minds_channel_ladder/run.py PROVES the SDPO channel FIRES nonzero (JSD=0.0565, grad flows) through the REAL shipped collator alignment indices — the thing the old smoke could not show (it only proved init). Honest stub-with-differing-tokens proof; real-model path gated behind ALTERED_MINDS_REAL_MODE…21647a4 -
feat(wave-a): close ADR-011 (SDPO alignment indices) + ADR-012 (review findings)
B1/ADR-011: collator emits student/teacher_response_idx + valid masks via _mask_to_padded_indices; loss sentinel-masks padding. Strict SDPO no longer raises against the real collator (the regression my review-fix introduced). B2/ADR-012: - k1-KL: found TRL 1.5.0 uses k3 not k1; corrected docstring honestly + documenting test. - hint routing: style/communication/effort sites now reach the judge (error-kind aware). - HackMonitor: added patch-provenance layer defeating string-concat obfuscation; ADR-010 language corrected AST->signature+patch-provenance. - curriculum: optional turns/think_tokens effort signals, backward-compatible. 210 passed / 16 skipped (was 192). Two Opus-4.8 workers in parallel.d02d724 -
architect: ADR-011/012/013 + research (alignment-index fix, review-findings closure, LMA channel-ladder)
b4a584a -
review+fix: cross-family adversarial ADR review (owed item) + remediation
Ran the owed 5-family adversarial review of ADR-008/009/010 (unblocked once OpenRouter credits were restored). 4/5 clean reviews (GPT-5.5, Gemini 3.1 Pro, DeepSeek V4 Pro, Grok 4.3; Kimi starved its token budget). Verdicts: ADR-008 3-REJECT/1-fix, ADR-009 unanimous-fix, ADR-010 3-REJECT/1-fix — all REJECTs were "accepted-status outran the evidence," not "design wrong." The review caught a real policy-corrupting P0 the solo pre-mortem missed: the SDPO alignment guard was a shape-check, so hint-shifted teacher tokens could silently misalign and distill garbage into the policy. Verified every convergent finding against code, then fixed (192 pass/16 skip, was 187): - SDPO alignment (composer_trainer.py): require collator-emitted student_response_idx/teacher_response_idx; gather aligned post-hint logits before JSD; strict-mode raises on missing/mismatched indices. Tests rewritten to the corrected contract (different-length sequences now exercised). - Sandbox scrub (sandbox.py): boo…185cce2 -
docs: wave summary — Composer 2.5 data-gen + targeted-RL textual-feedback
8498e8f -
feat(datagen): ADR-010 FeatureDeletionEnv synthetic-data subsystem; accepted
Track A of the deep-work-loop — Composer 2.5's named synthetic-data generator (Feature Deletion) as a reusable subsystem. New composer_replication.datagen: - schema.py: FeatureDeletionTask (broken_repo + FAIL_TO_PASS reward target + PASS_TO_PASS functional guard; golden_diff HELD OUT of repr/observation). - sandbox.py: Sandbox Protocol + FakeSandbox (unit tests) + LocalSubprocessSandbox (real); SANDBOX_DENYLIST blocks find/strings/unzip/decompilers/git (the tools Composer's reported reward-hacks used to recover deleted signatures). - monitor.py: HackMonitor — flags cache/bytecode-provenance hacks so the grader masks reward (defense-in-depth behind the sandbox lockdown). - curriculum.py: DifficultyCurriculum — online pass-rate gate; up-weights the ~0.5 frontier (w ∝ p(1-p)), retires aced tasks, quarantines all-fail tasks (raw rate, not smoothed). Implements the blog's "select for harder tasks dynamically". - validator.py: 4-gate solvability validator (baseline-green / del…9336af3 -
feat(hints): ADR-009 layered HintGenerator; accepted
Track B of the deep-work-loop. Extends hint_generator.py from a flat dispatch registry into a layered, cost-ordered HintGenerator behind a typed Protocol — the SDPO textual-feedback hint source (Composer 2.5's hint mechanism is unstated in every Cursor artifact; this is our reconstruction). Layers (tried cheapest-first, first non-None wins): 1. TemplateHintGenerator — the existing 5 templates (free, deterministic; byte-identical to dispatch(), preserved). 2. RawErrorHintGenerator — raw env/tool error text as the hint (free; covers any message-bearing site templates miss). SDPO "environment feedback as conditioning signal". 3. LLMJudgeHintGenerator — <=2-sentence corrective hint via an injected complete() callable (optional, OFF unless provided; in-memory + disk cache keyed on error-context hash; covers style/communication/effort). CompositeHintGenerator.as_collator_hook() returns a callable matching the existing CollatorConfig.hint_generator signature (…84740d4 -
feat(trainer): ADR-008 gate-3 live GRPO+SDPO smoke PASS; ADR-008 accepted
The composer_grpo_sdpo_smoke instantiates a real trl.GRPOTrainer via ComposerReplicationTrainer on Qwen2.5-0.5B with the Dr. GRPO config (loss_type=dr_grpo, scale_rewards=none, num_iterations=1) and alpha_sdpo=1.0, runs 1 step, and confirms loss/sdpo_kl is logged — the SDPO channel is wired into the live Dr. GRPO loop. PASS. Surfaced + fixed a real API drift: TRL 1.5.0 dropped GRPOConfig.max_prompt_length. All 4 ADR-008 acceptance gates green -> status proposed->accepted; gate checkboxes ticked with evidence; index updated.2a34df4 -
feat(trainer): ADR-008 Dr.GRPO config + SDPO strict-alignment guard
Phase-6 execution of ADR-008 (gates 1, 2, 5 green; gate 3 = the live GRPOTrainer smoke runs detached separately). - make_dr_grpo_config(): builds trl.GRPOConfig to the Dr. GRPO recipe (loss_type=dr_grpo => no length-standardization bias; scale_rewards=none => no std-dev advantage normalization; num_iterations=1 => single-epoch), with a drift-guard assertion. TRL 1.5.0 natively supports these knobs and its own help text cites the Dr. GRPO paper for both. - ComposerReplicationTrainer.strict_sdpo_alignment (default True): the student/teacher logit shape-mismatch path now RAISES instead of silently zeroing the SDPO channel (closes the ADR-008 trust-gap, composer_trainer.py lines 158-160). Opt-out to warn-and-skip for production resilience. - prime_rl/composer_loss.py: NotImplementedError(alpha_sdpo>0) message now points at the TRL host as the SDPO home (documents the ADR-006 amendment). - Tests: 7 new (config knobs asserted; strict raises / non-strict skips / aligned fir…bde5c5e -
docs(adr): add ADR-008/009/010 (Dr.GRPO+SDPO, layered hints, FeatureDeletionEnv)
Phase-4 of the deep-work-loop. Three proposed ADRs translating the Composer 2.5 data-gen + targeted-textual-feedback research into durable decisions, each with mechanized acceptance gates (status stays `proposed` until gates green): - ADR-008: target Dr. GRPO (length-std off, no std-norm, k1 KL, Adam, single-epoch — per arXiv:2603.24477) and host the live SDPO channel in the TRL ComposerReplicationTrainer. amends ADR-006 (SDPO needs full logits -> TRL-only; PRIME-RL hosts channels 1+3). preserves ADR-007. - ADR-009: layered HintGenerator (template -> raw-error -> LLM-judge -> SDPO sibling-bootstrap) behind a typed Protocol; preserves the existing templates + CollatorConfig.hint_generator hook. - ADR-010: FeatureDeletionEnv synthetic-data subsystem inverting OSS SWE substrates (SWE-bench-Lite/SWE-Gym/R2E-Gym/SWE-rebench) + online difficulty gate + reward-hacking safeguards. ADR-006 patched: Amended-by header + stale matrix row qualified. Index added. Prior-ADR audit (ad…36ab61e -
research: Composer 2.5 data-gen + targeted-textual-feedback deep-research wave
Phase-3 of the deep-work-loop bringing Composer 2.5's dataset-generation and targeted-RL-with-textual-feedback methods into the framework. 5 new research docs (130KB), all with primary-source citations: - 06-feature-deletion-datagen.md: Feature Deletion env design (online pass-rate difficulty gate, reward-hacking safeguards, 5 OSS substrates w/ HF ids + licenses, deletion mechanics, FeatureDeletionEnv + TRL reward_fn adapter). - 07-sdpo-hint-generator.md: layered HintGenerator (template -> raw-error -> LLM-judge -> introspection -> learned -> SDPO sibling-bootstrap), actual template strings, judge prompt, slots into existing CollatorConfig hook. - 08-sdpo-grpo-integration.md: ComposerGRPOTrainer(GRPOTrainer) design adding SDPO KL at error turns on top of Dr. GRPO; PRIME-RL recipe blocked (log-probs only), TRL subclass is the host; CPU smoke plan. - 09-composer-blog-delta-2026.md: blog re-extraction delta; found the Composer 2 arXiv tech report; SDPO successful-rollout-as…6049d00 -
examples: add sdpo_real_trace_train_smoke — close the forward+backward+step link
The framework proved the SDPO data path (validate_real_trace_alignment: 832/832 mask alignment on real ~/.claude traces) and the loss math (test_gradient_flow: SDPO routes finite grads on TinyLM) in isolation, but never connected them: an actual compose_loss forward + backward + optimizer.step() on a real HF model fed by the real-trace collator. This is the never-implemented composer_replication.examples. sdpo_with_real_traces_production module that Modal stage_4_sdpo_smoke referenced. It is now real and PASSES on Qwen2.5-0.5B-Instruct over 6 real error-bearing sessions: step 0: total=2.36307 lm_ce=2.33588 sdpo_jsd=0.02718 finite=True step 1: total=2.32758 lm_ce=2.30190 sdpo_jsd=0.02568 finite=True SDPO channel fired (>0): True | param moved: True (max|Δ|=6.22e-05) Gates: (1) no crash, (2) finite loss, (3) SDPO channel fires >0 (shape-gate passes, teacher forward contributes real signal — not the silent no-op the empty-placeholder stage_4 would give), (4) a real param moves af…7090729
-
Wave 21c: verify PRIME-RL adapter parity against upstream source (byte-for-byte)
The recipe's only automated parity check (test_parity_with_prime_rl_default_loss_fn) is skip-marked whenever prime-rl isn't importable in the framework venv — i.e. essentially always, since we deliberately keep prime-rl's heavy deps (vLLM, pydantic config trees, flash-attn) out of our test env. The fallback was an in-file reimplementation (_reference_default_loss), so a shared bug could pass silently. This closes that gap against the ACTUAL upstream source. ## What was done Built an out-of-band parity harness that: - clones PrimeIntellect-ai/prime-rl (shallow) - builds an isolated venv with ONLY torch+beartype+jaxtyping+numpy - loads upstream src/prime_rl/trainer/rl/loss.py directly by path, stubbing the two modules it imports (prime_rl.configs.trainer, prime_rl.utils.utils) so we skip the vLLM/pydantic dependency tree entirely - runs identical inputs through our loss_fn and upstream default_loss_fn ## Result 24/24 cases match (12 seeds × 2 regimes: tiny-perturbation + wide-diver…c98928e -
Wave 21b: skip zero-signal SDPO on empty-recovery error turns + real-trace validation
Found by validating the Wave 21 pipeline against REAL Claude Code session logs (738 local sessions, 66 with is_error:true) rather than only the synthetic fixture. Two outcomes: a collator fix and a permanent validation example. ## Collator fix: empty-recovery error turns On real traces, ~67% of error sites have EMPTY recovery content when strip_thinking=True — because the error-RECOVERY turn is frequently pure [THINKING] (the model reasons about the failure, then silently retries a tool), and stripping thinking empties it. The old code set any_errors=True and injected a hint the moment hint_text existed, BEFORE checking recovery content, so an empty-recovery turn produced an all-ignore_index sdpo_loss_mask: a zero-signal SDPO row that wastes a forward pass and dilutes the channel. Fix: only treat a turn as an SDPO error site when BOTH a hint was produced AND the recovery turn has content (`hint_text and turn.get("content")`). Applied symmetrically to `_build_hint_injected_trace` (te…d61036a -
Wave 21: close both Wave 20 debt items — chat-template alignment + structural is_error
Two architectural-debt items flagged "Wave 20 candidate" in the framework doc, both CPU-only, both now fixed and regression-guarded. ## 1. SDPO mask chat-template drift (was ~67% aligned, now ~100%) `ComposerDataCollator._build_segment_mask` built the sdpo_loss_mask (and the aligned-student response_mask) by tokenizing each content segment in isolation and concatenating. That ignored the scaffolding tokens apply_chat_template inserts around every message (<|im_start|>{role}\n ... <|im_end|>\n, BOS), so mask bits drifted left of the real content tokens — the residual ~33% contamination documented in the Wave 19 production audit. Fix: new `_build_chat_aligned_mask` derives the mask from per-message apply_chat_template prefix deltas. For message k it computes the token span len(template(msgs[:k+1])) - len(template(msgs[:k])), then locates the content run inside that span by subsequence match and marks ONLY those positions as loss. Falls back to whole-span marking if the content run can…6806cf7 -
Wave 20: ModalSpawnExecutor — finish the Modal-backed serverless executor
Wave 17 shipped a NotImplementedError skeleton for ModalExecutor pinned by test_skeleton_executors.py. Wave 18 hardened the skeleton contract to be strict per-code-path. This wave adds the real Modal-backed executor as ModalSpawnExecutor (sibling class), preserving the skeleton contract for backward compat while making DiLoCo across Modal containers actually runnable. Design choices vs the v0 skeleton's docstring: 1. **User-provided modal.Function instead of internal app construction.** The skeleton showed a pattern where the executor builds its own modal.App and registers run_replica internally. That couples the executor to image/GPU/Volume choices the user actually wants to own. ModalSpawnExecutor takes a *pre-decorated* modal.Function from the caller. The user defines: @app.function(gpu="H100:4", image=img, volumes={"/vol": vol}, secrets=[...], timeout=4*3600) def diloco_replica(rank: int, rendezvous_uri: str, world_size: int, **k…a384097
-
Wave 19: production-grade SDPO via ComposerDataCollator + adapter + collator fixes
Adds the full production data path Wave 18 deferred: ClaudeCodeIngester → adapter → ComposerDataCollator → compose_loss with proper hint injection at detected error sites. Two rounds of 3-reviewer cross-family review caught a critical SDPO mask alignment bug in round 1; round 2 verified the fix. NEW infrastructure: - composer_replication/ingestion/trace_examples.py: claude_states_to_trace_examples() adapter walks ClaudeCodeIngester output, detects [TOOL_RESULT (ERROR)] tagged user turns, marks the recovery assistant turn with tool_error="<kind>" so ComposerDataCollator._is_error_turn picks it up. Default classifier handles file_not_found, permission_denied, command_not_found, syntax_error, connection_error; users can pass custom error_kind_fn. Backward-scan finds errors even when intervening user turns separate the error from recovery. - composer_replication/ingestion/tests/test_trace_examples_adapter.py: 14 tests pinning the adapter contract (error detection, classifica…03bf323
-
Wave 18: 14 backlog items closed + 3-reviewer cross-family review
Polish wave grinding down 14 P0/P1/P2 items from Wave 7 audit + Wave 11/12 self-critique + Wave 16/17 deferrals. Two passes of 3-reviewer cross-family review (Gemini 3.1 Pro / Grok 4.3 / Sonnet 4.6) caught 6 real issues in round 1; round 2 verified all fixed plus 2 new flagged items addressed. Doc honesty (P0): - README.md scoping change at line 29: dropped "any HuggingFace base model" for the more honest "any HF causal LM with a chat template (Qwen, Llama, Mistral, DeepSeek, Phi, Gemma families)". Added qualifier that Spike 006 zeroed the SDPO channel due to shape misalignment, so SDPO is verified via the integration test suite + end-to-end examples, not Spike 006. - spikes/005-integrated-trainer-skeleton/verl_path/README.md NEW — explicit STATUS: design-only marker per Wave 7 audit Objection 3. Documents what to do before relying on the VeRL path (install Ray, run parity test against trl_path). Test honesty (P1): - spikes/006-real-hf-model-smoke/verdict.md numerical c…54efac8 -
Wave 17: close all 5 audit FLAGs + SDPO context alignment + serverless re-exports
Three-track follow-up to Wave 16, with cross-model review (gemini-3.1-pro APPROVED, grok-4.3 surfaced one BLOCKER that verified false-positive empirically, deepseek-v4-pro burned all max_tokens on hidden reasoning producing no visible output): Wave 17a — serverless package re-exports (real code bug) composer_replication/diloco/serverless/__init__.py now re-exports ModalExecutor and HFJobsExecutor alongside LocalProcessExecutor. The package docstring already documented `from composer_replication.diloco.serverless import ModalExecutor` as the public API but the import would have failed because the names weren't in __init__.py's import list or __all__. Both modal.py and hf_jobs.py import-safely (they don't import the optional `modal` / `huggingface_hub` packages at module level — those are deferred to class method calls), so adding the re-exports doesn't introduce a new dependency. Pinned by a regression test (test_public_reexports_include_all_executors in test_s…a84c060 -
Wave 16: install ergonomics + gradient evidence + SDPO end-to-end example
Five-track polish wave with mandatory cross-model adversarial review (deepseek-v4-pro + gemini-3.1-pro-preview, route-fidelity-verified via direct-urllib scatter; reviewers caught 2 real bugs that BOTH made it into the staged diff and would have shipped without the review): Wave 16a — Install ergonomics (extras pinning) pyproject.toml dropped three extras whose lower bounds were unsatisfiable on public PyPI and would have blocked any fresh user: - monarch>=0.4.1: PyPI 'monarch' tops out at 0.1.11 and is unrelated to Meta's actor framework; the real Monarch ships as torchmonarch-nightly with platform constraints. - prime-rl>=0.5: 'prime-rl' is not registered on PyPI at all; Prime Intellect publishes from source only. - data-juicer>=1.0: 'data-juicer' is not registered on PyPI; the closest match (py-data-juicer) has broken transitive deps. All three integrations import lazily, so the framework still loads cleanly without them. docs/TROUBLESHOO…c0a5ab7