Add S3 Ref15 Full FT as reference-conditioned default

User-selected final step 26774; preserve S1–S10, document data selection, text repair, training and evaluation caveats.

README.md

@@ -19,7 +19,15 @@ model-index:
 
 # Humaneness Voice Small
 
-Humaneness Voice Small is an experimental English/German text-to-speech and voice-acting model. This repository contains **ten stage-local checkpoints from one continuous S1→S10 training run**, the training and inference source, run statistics, and a guide to the exact prompt surfaces. It is not a standard `AutoModel.from_pretrained()` checkpoint: it uses a Qwen3-0.6B semantic backbone, a MOSS-style local Talker, and the separately loaded [MOSS Audio Tokenizer v2](https://huggingface.co/OpenMOSS-Team/MOSS-Audio-Tokenizer-v2). Read the inference instructions before loading weights.
+Humaneness Voice Small is an experimental English/German text-to-speech and voice-acting model. This repository contains **ten stage-local checkpoints from one continuous S1→S10 training run** plus a later S3-based full-parameter, long-reference continuation. It also contains training and inference source, run statistics, and a guide to the exact prompt surfaces. It is not a standard `AutoModel.from_pretrained()` checkpoint: it uses a Qwen3-0.6B semantic backbone, a MOSS-style local Talker, and the separately loaded [MOSS Audio Tokenizer v2](https://huggingface.co/OpenMOSS-Team/MOSS-Audio-Tokenizer-v2). Read the inference instructions before loading weights.
+
+## Current default: S3 Ref15 Full FT (9 October 2026)
+
+The new **reference-conditioned default checkpoint** is [`checkpoints/S3-Ref15-FullFT/model_bf16.pt`](checkpoints/S3-Ref15-FullFT/model_bf16.pt), the final **step 26,774** of a full-parameter continuation from the **original S3 step 32,337**. The user chose it because it sounded acoustically preferable in listening. The updated `code/infer.py` loads it when `--stage` is omitted or set to `default`. The ten original stage checkpoints are unchanged; `--stage S3` explicitly loads the original S3 and is still the comparison to try for **reference-free** generation. For the new default, supply a clean, distinct 5–15-second reference WAV; every accepted retraining example had a reference.
+
+The key fix is the historical **2.96-second reference crop**: the original training loader and earlier example inference code used at most 37 × 80-ms codec frames even though the stored reference sequences were longer. This continuation used up to **187 frames / 14.96 seconds**, with a 55/20/20/5 family of script, historical structured, caption-transcript, and transcript-only prompts. It fully updated the active semantic backbone and Talker after a conservative target-text repair gate; it is **not** the Rank-128 LoRA and did not start from that adapter. There were **3,255,217 accepted training targets**, each presented once with its distinct reference. Read the detailed [data selection, repair, training and evaluation report](docs/S3_REF15_FULL_FT.md) and the [checkpoint metadata](checkpoints/S3-Ref15-FullFT/metadata.json) before reusing it.
+
+This is a **listening-selected acoustic default**, not an objectively established all-round winner. In a matched 50-case non-speaker-disjoint test, final-step WER was 0.096 versus 0.088 for original S3; Orange cosine to the supplied reference was 0.707 versus 0.745, while cosine to the intended target improved from 0.658 to 0.734. Step 25,000 had better measured WER and target cosine than the final step. These learned/ASR proxies and the user's listening impression answer different questions; further blinded and speaker-disjoint evaluation is needed. No optimizer state is uploaded for this continuation.
 
 The semantic backbone was **initialized from pretrained Qwen3-0.6B**, not from random weights. The **112,764,928-parameter Talker and bridge were initialized fresh** for this run. This corrects the loose description “trained from scratch” sometimes applied to earlier experiments. The Qwen hidden width is 1,024 and the local Talker width is 2,560. One semantic frame spans 80 ms; the Talker predicts **12 RVQ codebook indices per frame**, or 150 codebook indices per second. It does not predict 32 codebooks per frame. The codec reconstructs 48-kHz audio. The full state is approximately 715 million parameters, including the frozen, unused score conditioner; the model-only exports are BF16 PyTorch state dictionaries.
 
@@ -40,7 +48,7 @@ The `checkpoints/S*/model_bf16.pt` files are the **lowest independent held-out v
 | S9 | tier1 | 1,439 | 41,878 | 4.7198 | 0.4540 |
 | S10 | tier0 | 578 | 41,936 | 4.7278 | 0.4572 |
 
-¹ These are **training presentations**, not unique source-audio hours. Multi-format prompts repeat some recordings. The ladder tiers are nested and paired with sidecar material; do not add these hours to estimate unique audio. ² Target-aware automatic reward on 1,651 shared valid acting cells per stage, not a human MOS. The S3–S10 paired difference is +0.00557 with a 95% challenge-bootstrap CI of −0.00701 to +0.01899: **the benchmark does not establish S3 as significantly better than S10**. S3 is a reasonable provisional default; inspect the same prompts in S3/S10 before choosing for a production voice. S4 has the best held-out validation loss. All underlying values and intervals are in `stats/acting_findings.json` and `stats/acting_analysis.json.gz`.
+¹ These are **training presentations**, not unique source-audio hours. Multi-format prompts repeat some recordings. The ladder tiers are nested and paired with sidecar material; do not add these hours to estimate unique audio. ² Target-aware automatic reward on 1,651 shared valid acting cells per stage, not a human MOS. The S3–S10 paired difference is +0.00557 with a 95% challenge-bootstrap CI of −0.00701 to +0.01899: **the benchmark does not establish S3 as significantly better than S10**. Among the **original ladder stages**, S3 was a reasonable provisional choice; inspect the same prompts in S3/S10 before choosing for a production voice. The later Ref15 Full FT is now the reference-conditioned default by listening choice, under the caveats above. S4 has the best held-out validation loss. All underlying values and intervals are in `stats/acting_findings.json` and `stats/acting_analysis.json.gz`.
 
 The run used 32 JUPITER Booster nodes × 4 GPUs/node, batch 4,096, **one** 2,097-step linear warmup followed by one cosine decay across all 41,936 updates—**no stage LR restarts**. Peak LRs were 8e-5 for the semantic backbone and 2.4e-4 for the Talker, ending at 5% of peak. The source `code/plan_caption_production_32nodes.json` records the exact historical manifests and hashes; its absolute JUPITER paths are provenance, not portable defaults.
 
@@ -131,15 +139,15 @@ Throughput was measured on an **RTX 3090**, using S3 at revision `5de86032771c37
 - `code/continuous_train.py`, `code/train.sbatch`, `code/plan_caption_production_32nodes.json` and adjacent modules: the historical JUPITER source and training contract. The raw plan refers to local manifests and intentionally fails its hash checks if edited; regenerate plans and manifests for new data. `code/infer.py` is the standalone example for the exported weights. The code bundles MOSS export modules, tokenizer configuration and score schema; the audio codec is fetched separately from its upstream Hugging Face repository.
 - `stats/training_steps.jsonl`: every logged training-step scalar line across all four Slurm invocations. `stats/invocation_summaries.json`, `stats/run_configuration.json`, `stats/stage_winners.json`, `stats/validation_loss_full.tar.gz`, `stats/acting_findings.json`, and `stats/acting_analysis.json.gz` hold the remaining scalar summaries, all held-out checkpoint-sweep records, and the public acting aggregate. `stats/CONTENTS.json` gives hashes.
 
-Example (requires a CUDA environment with PyTorch, Transformers, NumPy, SoundFile, Torchaudio, Safetensors, and permission to download the separate codec):
+Example using the **new reference-conditioned default** (requires a CUDA environment with PyTorch, Transformers, NumPy, SoundFile, Torchaudio, Safetensors, and permission to download the separate codec):
 
 ```bash
-python code/infer.py --stage S3 --language en --frames 75 \
+python code/infer.py --stage default --reference-wav reference.wav --language en --frames 75 \
   --prompt $'CAPTION: Warm, softly amused, conversational.\nTRANSCRIPT: "I cannot believe we made it."' \
   --text 'I cannot believe we made it.' --output example.wav
 ```
 
-Add `--reference-wav reference.wav` for a distinct, clean recording, and do not use the target recording as its own reference. **Correction (8 October 2026):** an earlier public `code/infer.py` rejected any reference over 37 MOSS codec frames (2.96 s). This was an inference-code limit as well as the historical S1–S10 training-loader crop, not an architectural limit. The updated example accepts a configurable reference ceiling (15 s default, 20 s maximum), reports a warning below 5 s and no longer silently blocks longer audio. The original S1–S10 weights were trained with at most 2.96 s references, so longer-reference behavior from the unadapted base is experimental, not a guaranteed improvement. For a model actually tuned with up to 14.96 s references, see the [S3 Ref15 Rank-128 LoRA](https://huggingface.co/laion/Humaneness-Voice-Small-Ref15-LoRA-r128). The example is a source-level starting point; the public listening Space is the already-audited playback path. This repository does **not** contain the large original SFT-3 model weights or original Qwen checkpoint separately, because the exported Humaneness Voice Small states include every trained model weight required for inference.
+Do not use the target recording as its own reference. For reference-free inference, compare `--stage S3`; the new default was trained only with references. **Correction (8 October 2026):** an earlier public `code/infer.py` rejected any reference over 37 MOSS codec frames (2.96 s). This was an inference-code limit as well as the historical S1–S10 training-loader crop, not an architectural limit. The updated example accepts a configurable reference ceiling (15 s default, 20 s maximum), reports a warning below 5 s and no longer silently blocks longer audio. The new Full FT was trained with up to 14.96 s; using longer than 15 s at inference is out of distribution and warned about. The original S1–S10 weights were trained with at most 2.96 s references, so longer-reference behavior from an original stage is experimental, not a guaranteed improvement. The separate [S3 Ref15 Rank-128 LoRA](https://huggingface.co/laion/Humaneness-Voice-Small-Ref15-LoRA-r128) is a different, frozen-backbone comparison. The example is a source-level starting point; the public listening Space is the already-audited playback path. This repository does **not** contain the large original SFT-3 model weights or original Qwen checkpoint separately, because the exported Humaneness Voice Small states include every trained model weight required for inference.
 
 ## Training data, license and responsible use
 
@@ -159,10 +167,7 @@ Both collections use the S3 checkpoint at step 32,337; consult each adapter card
 <!-- hvs-small-dpo-default-study-2026-10-04 -->
 ## Default DPO-adapter recommendation (matched speaker/emotion study)
 
-For **general S3 inference, use the unadapted S3 checkpoint by default**: do not
-activate a DPO LoRA merely because an adapter is available. This recommendation
-concerns the DPO-adapter choice; the separate, provisional S3 checkpoint and
-prompt/CFG recommendations above still apply. The [matched generated-audio study
+For **the historical S3 DPO comparison, use the unadapted S3 checkpoint rather than activating a DPO LoRA by default**. This is an adapter-level recommendation, **not** a reversal of the newer Ref15 Full FT checkpoint default above; the DPO LoRAs were trained against S3 and should not be assumed compatible or beneficial on Ref15 Full FT. The historical S3 prompt/CFG recommendations above also refer to S3, not a completed Ref15 tuning sweep. The [matched generated-audio study
 with references and listening examples](https://laion-humaneness-voice-small-acting-eval-space.static.hf.space/dpo_lora_speaker_emotion_2026-10-02.html) compared the same 12
 English/German acting challenges, prompts and seeds for each condition (432
 outputs per condition, including 198 reference-conditioned outputs). Its
@@ -170,7 +175,7 @@ outputs per condition, including 198 reference-conditioned outputs). Its
 
 | S3 condition | Orange reference cosine ↑ | Parakeet WER ↓ | Requested-emotion instruction lift ↑ |
 |---|---:|---:|---:|
-| Base, no DPO adapter **(default)** | 0.501 | **0.096** | −0.021 |
+| Historical S3 base, no DPO adapter **(study default)** | 0.501 | **0.096** | −0.021 |
 | Recent pitch/quality/CFG-p2 Rank 16 × 0.50 | 0.534 | 0.106 | −0.019 |
 | Same Rank 16 × 1.00 | 0.529 | 0.123 | −0.116 |
 | Same Rank 64 × 1.00 | 0.534 | 0.133 | −0.099 |
@@ -190,8 +195,10 @@ exact prompts, paired audio players and methodological limits.
 <!-- hvs-small-explicit-optional-dpo-default-2026-10-04 -->
 ### Which DPO LoRA should I load by default?
 
-**None for general use:** the unadapted S3 checkpoint is still the default
-because it achieved lower WER than any DPO LoRA in the matched study above.
+**None for general use on the historical S3 base:** unadapted S3 achieved
+lower WER than any DPO LoRA in that matched study. The repository-wide
+reference-conditioned checkpoint default is now Ref15 Full FT, which was
+evaluated under a different 50-case protocol; this DPO result does not rank it.
 If an application explicitly prioritizes similarity to a provided speaker
 reference, the **best-tested optional DPO LoRA at this date** is
 [`subset_pitch_quality_p2_rank016.adapter.pt`](https://huggingface.co/laion/Humaneness-Voice-Small-DPO-LoRAs/blob/main/adapters/subset_pitch_quality_p2_rank016.adapter.pt)

checkpoints/S3-Ref15-FullFT/metadata.json

@@ -0,0 +1,42 @@
+{
+  "name": "Humaneness Voice Small S3 Ref15 Full FT",
+  "role": "repository_default_as_of_2026-10-09",
+  "format": "torch.save(model.state_dict()) in bfloat16; load with weights_only=True",
+  "file": "model_bf16.pt",
+  "bytes": 1431076525,
+  "sha256": "e1b2c4613545d1db21c6f60a5ab2c27b5320aa05c177717ae268b1ff870d406e",
+  "final_step": 26774,
+  "base": {
+    "repository_path": "checkpoints/S3/model_bf16.pt",
+    "original_training_step": 32337,
+    "sha256": "60079723bf797a81107def1e6fc479e5353b3cc51f3780f98ea65a9f5c109f97"
+  },
+  "training": {
+    "kind": "hvs-long-ref-full-ft-v1",
+    "optimizer": "AdamW, full-parameter except unused frozen score conditioner",
+    "global_batch": 128,
+    "world_gpus": 32,
+    "train_unique_uids": 3255217,
+    "accepted_total_uids_including_validation": 3271673,
+    "reference_max_frames_80ms": 187,
+    "reference_max_seconds": 14.96,
+    "semantic_peak_lr": 0.000008,
+    "talker_peak_lr": 0.000016,
+    "warmup_fraction": 0.05,
+    "schedule": "single_cosine_to_10_percent_of_peak_no_restart",
+    "seed": 20261004,
+    "source_worklist_sha256": "52c5223d1935ca806044b2f832291f11271240ca26d6316ce2f82bafe4ae2939",
+    "target_text_resolution_sha256": "c8f636ec873f68360287626b336ae803a4c08f0c54a8672bc0d42e713406fee0",
+    "score_schema_sha256": "8eee17be1be56dcc4c3b5e8ac95b5494dd147451051f2d9a0c34d418f354b9e7",
+    "exact_prompt_counts": {
+      "script_only": 1932540,
+      "general_script": 570663,
+      "caption_timbre": 318972,
+      "caption_bude": 269892,
+      "transcript_only": 163150
+    }
+  },
+  "selection": "User-selected acoustic default after listening; not the optimum of every automatic metric",
+  "optimizer_state_uploaded": false,
+  "details": "docs/S3_REF15_FULL_FT.md"
+}

checkpoints/S3-Ref15-FullFT/model_bf16.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1b2c4613545d1db21c6f60a5ab2c27b5320aa05c177717ae268b1ff870d406e
+size 1431076525

checkpoints/default.json

@@ -0,0 +1,10 @@
+{
+  "checkpoint": "S3-Ref15-FullFT",
+  "model_file": "checkpoints/S3-Ref15-FullFT/model_bf16.pt",
+  "metadata_file": "checkpoints/S3-Ref15-FullFT/metadata.json",
+  "chosen_on": "2026-10-09",
+  "selection_basis": "user listening preference for acoustic quality",
+  "reference_conditioned": true,
+  "trained_reference_max_seconds": 14.96,
+  "reference_free_fallback": "S3"
+}

code/infer.py

@@ -2,7 +2,8 @@
 """Single-clip Humaneness Voice Small inference; use on a CUDA GPU.
 
 Example (from the model repository root):
-  python code/infer.py --stage S3 --prompt 'CAPTION: warm, amused narration\nTRANSCRIPT: "Hello there."' \
+  python code/infer.py --stage default --reference-wav reference.wav \
+      --prompt 'CAPTION: warm, amused narration\nTRANSCRIPT: "Hello there."' \
       --text 'Hello there.' --frames 60 --output hello.wav
 """
 from __future__ import annotations
@@ -18,7 +19,9 @@ sys.path.insert(0, str(ROOT / 'code'))
 
 def main() -> None:
     parser = argparse.ArgumentParser(description=__doc__)
-    parser.add_argument('--stage', choices=[f'S{i}' for i in range(1, 11)], default='S3')
+    parser.add_argument('--stage', choices=['default', 'S3-Ref15-FullFT'] +
+                        [f'S{i}' for i in range(1, 11)], default='default',
+                        help='default = S3 Ref15 Full FT; S1–S10 retain the original ladder weights')
     parser.add_argument('--prompt', required=True, help='Literal GENERAL/SCRIPT, CAPTION/TRANSCRIPT or TRANSCRIPT text')
     parser.add_argument('--text', required=True, help='The exact spoken transcript')
     parser.add_argument('--frames', type=int, required=True, help='Frame budget at 12.5 frames/second')
@@ -32,6 +35,9 @@ def main() -> None:
     assert args.frames > 0
     if not 0 < args.reference_max_seconds <= 20:
         parser.error('--reference-max-seconds must be in (0, 20]')
+    if args.stage in ('default', 'S3-Ref15-FullFT') and args.reference_max_seconds > 15:
+        print('Warning: Ref15 Full FT trained with at most 14.96 s of reference; '
+              'longer inference references are out of distribution', file=sys.stderr)
 
     import numpy as np
     import soundfile as sf
@@ -51,7 +57,11 @@ def main() -> None:
     moss_small.load_qwen_backbone = lambda model, log=print: None
     schema = json.loads((ROOT / 'assets/score_schema.json').read_text())
     model, config = build_fresh(schema, log=lambda _: None)
-    state = torch.load(ROOT / 'checkpoints' / args.stage / 'model_bf16.pt',
+    checkpoint_name = 'S3-Ref15-FullFT' if args.stage == 'default' else args.stage
+    if checkpoint_name == 'S3-Ref15-FullFT' and args.reference_wav is None:
+        print('Warning: the Ref15 default was tuned only with reference audio; '
+              'for reference-free inference compare --stage S3', file=sys.stderr)
+    state = torch.load(ROOT / 'checkpoints' / checkpoint_name / 'model_bf16.pt',
                        map_location='cpu', weights_only=True)
     model.load_state_dict(state, strict=True)
     model.tie_weights()
@@ -113,7 +123,8 @@ def main() -> None:
     args.output.parent.mkdir(parents=True, exist_ok=True)
     sf.write(args.output, wave, sample_rate, subtype='PCM_16')
     print(json.dumps({'output': str(args.output), 'frames': len(codes),
-                      'duration_seconds': len(wave) / sample_rate, 'stage': args.stage}))
+                      'duration_seconds': len(wave) / sample_rate,
+                      'stage': args.stage, 'resolved_checkpoint': checkpoint_name}))
 
 
 if __name__ == '__main__':

code/ref15_fullft/README.md

@@ -0,0 +1,22 @@
+# Historical S3 Ref15 Full FT source
+
+These are the source entrypoints used to construct the packed S3 long-reference
+worklist and train the final step-26,774 checkpoint. They are published for
+provenance and audit, **not as a one-command portable trainer**: paths point
+to the original JUPITER manifests, and imports also use modules elsewhere in
+this model repository and the original training workspace. Regenerate and
+verify the input manifests, schema and SHA-256 contract before using new data.
+
+- `build_worklist.py` selects unique physical S3 targets with a validated,
+  distinct reference and raises the codec-reference crop from 37 to 187 frames.
+- `prompt_mix.py` and `script_only.py` implement the deterministic five-mode
+  instruction mixture and remove stable voice-identity descriptions from the
+  bare SCRIPT mode.
+- `build_text_resolution.py` and `text_resolution.py` implement the
+  conservative target-text repair/exclusion sidecar.
+- `rowgroup_plan.py` balances Parquet row groups over GPU ranks.
+- `train.py`, `preflight.py`, and `train_full.sbatch` implement the full-model
+  continuation, fail-closed preflight and resumable 32-GPU Slurm segments.
+
+See the repository's [detailed Ref15 report](../../docs/S3_REF15_FULL_FT.md)
+for exact selection counts, optimizer settings, limitations and evaluation.

code/ref15_fullft/build_text_resolution.py

@@ -0,0 +1,202 @@
+#!/usr/bin/env python3
+"""Conservative packed disposition of S3 transcript suspects after full ASR.
+
+This is an automated repair, not ground-truth transcription.  It favors
+excluding an uncertain short target over teaching a full-scene transcript to
+a clipped target.  Original TAR/Parquet assets are immutable.  Every suspect
+must have an ASR hypothesis and one explicit disposition; the 128+ sample QA
+is a structural/audio-ASR consistency check, not a claim of human listening.
+"""
+from __future__ import annotations
+
+from collections import Counter, defaultdict
+from datetime import datetime, timezone
+from difflib import SequenceMatcher
+import hashlib
+import json
+import math
+import os
+from pathlib import Path
+import re
+
+import pyarrow as pa
+import pyarrow.parquet as pq
+
+from audit_target_text import AUDIT, PARENTHETICAL
+from build_worklist import EXPECTED_ELIGIBLE, OUT, SHARDS, sha
+from text_resolution import RESOLUTION_DIR
+
+ASR = AUDIT.parent / 'target_asr_v1'
+TOKEN = re.compile(r"[^\W_]+(?:['’\-][^\W_]+)*", re.UNICODE)
+BAD_SINGLE = {'a', 'i', 'o', 'uh', 'um', 'mmm', 'hmm', 'ah', 'oh', 'the'}
+
+
+def lexical(text: str) -> list[str]:
+    return [word.casefold() for word in TOKEN.findall(PARENTHETICAL.sub(' ', text or ''))]
+
+
+def repeated_run(words: list[str]) -> int:
+    best = run = 0
+    prior = None
+    for word in words:
+        run = run + 1 if word == prior else 1
+        best = max(best, run)
+        prior = word
+    return best
+
+
+def classify(row: dict) -> tuple[str, str, str, float]:
+    source_text = str(row['source_text'] or '').strip()
+    asr_text = ' '.join(str(row['asr_text'] or '').strip().split())
+    old, new = lexical(source_text), lexical(asr_text)
+    duration = float(row['duration_s'])
+    rate_cap = max(4, math.ceil(duration * 6.5))
+    if re.search(r'\b(?:GENERAL|SCRIPT|CAPTION|TRANSCRIPT):|\[pause\s', asr_text, re.I):
+        return 'exclude', '', 'asr_reserved_prompt_marker', 0.
+    if len(new) > rate_cap or repeated_run(new) >= 6 or len(asr_text) > 2048:
+        return 'exclude', '', 'implausible_asr', 0.
+    if not new:
+        return 'exclude', '', 'empty_asr', 0.
+    # A few original metadata strings contain thousands of hallucinated
+    # repeated words.  Bound the quadratic alignment work; the full old
+    # length is still used by the keep/no-keep decision below.
+    old_head = old[:max(256, len(new) * 4)]
+    matcher = SequenceMatcher(None, old_head, new, autojunk=False)
+    common = sum(block.size for block in matcher.get_matching_blocks())
+    support = common / len(new)
+    agreement = matcher.ratio()
+    aligned = int(row['aligned_word_count'])
+    # A matching transcript is a false-positive rate/alignment flag.  An
+    # aligned original with small ASR differences can preserve proper nouns.
+    if (agreement >= .88 and len(old) <= rate_cap) or (
+            aligned >= .8 * len(old) and agreement >= .80 and len(old) <= rate_cap):
+        return 'verified_keep', '', 'asr_agrees_with_source', support
+    if len(new) == 1:
+        if len(old) <= 2 and new[0] == old[0] and new[0] not in BAD_SINGLE:
+            return 'verified_keep', '', 'single_word_agreement', support
+        return 'exclude', '', 'short_asr_not_safe', support
+    # Full-scene text clipped to a sentence typically has source-prefix
+    # support near 1 even if the global source-vs-ASR WER is very large.
+    # Without substantial support, the ASR itself may be a hallucination.
+    if support >= .68 and common >= 2 and asr_text:
+        return 'relabel', asr_text, 'supported_clip_asr', support
+    return 'exclude', '', 'asr_source_conflict', support
+
+
+def main():
+    asr_manifest_path = ASR / 'MANIFEST.json'
+    audit_report_path = AUDIT / 'REPORT.json'
+    asr_spec = json.loads(asr_manifest_path.read_text())
+    audit_spec = json.loads(audit_report_path.read_text())
+    assert asr_spec['status'] == 'COMPLETE_HYPOTHESES_NOT_APPROVED_LABELS'
+    assert asr_spec['asr_include_unaligned'] is True, 'Every suspect needs a disposition'
+    assert asr_spec['source_audit_sha256'] == sha(audit_report_path)
+    assert audit_spec['rows'] == EXPECTED_ELIGIBLE
+    assert not RESOLUTION_DIR.exists(), 'Do not overwrite an existing resolution'
+
+    expected = set()
+    for shard in range(SHARDS):
+        part = AUDIT / f'suspect-{shard:03d}.parquet'
+        for batch in pq.ParquetFile(part).iter_batches(columns=['uid'], batch_size=8192):
+            for uid in batch.column(0).to_pylist():
+                assert uid not in expected, uid
+                expected.add(uid)
+    assert len(expected) == sum(x['any_suspect'] for x in audit_spec['by_source'].values())
+    assert len(expected) == asr_spec['rows']
+
+    RESOLUTION_DIR.mkdir(parents=True)
+    changes_path = RESOLUTION_DIR / 'changes.parquet'
+    changes_partial = changes_path.with_suffix('.parquet.partial')
+    schema = pa.schema([
+        ('uid', pa.string()), ('action', pa.string()), ('text', pa.string()),
+        ('source', pa.string()), ('reason', pa.string()), ('asr_support', pa.float32()),
+        ('duration_s', pa.float32()), ('source_text', pa.string()), ('asr_text', pa.string()),
+        ('record_locator_json', pa.string()), ('target_locator_json', pa.string())])
+    writer = pq.ParquetWriter(changes_partial, schema, compression='zstd')
+    observed = set()
+    counts = Counter()
+    by_source = defaultdict(Counter)
+    excluded_hours = 0.
+    qa_candidates = defaultdict(list)
+    try:
+        for part_spec in asr_spec['parts']:
+            path = Path(part_spec['path'])
+            assert sha(path) == part_spec['sha256']
+            for batch in pq.ParquetFile(path).iter_batches(batch_size=4096):
+                output = []
+                for row in batch.to_pylist():
+                    uid = row['uid']
+                    assert uid in expected and uid not in observed, uid
+                    observed.add(uid)
+                    # The full ASR pass must retain these audit fields.  This
+                    # prevents silently applying a repair to another target.
+                    assert int(row['text_word_count']) == len(lexical(row['source_text']))
+                    action, text, reason, support = classify(row)
+                    counts[action] += 1
+                    by_source[row['source']][action] += 1
+                    if action == 'exclude':
+                        excluded_hours += float(row['duration_s']) / 3600
+                    decision = {'uid': uid, 'action': action, 'text': text,
+                                'source': row['source'], 'reason': reason,
+                                'asr_support': support,
+                                'duration_s': float(row['duration_s']),
+                                'source_text': row['source_text'],
+                                'asr_text': row['asr_text'],
+                                'record_locator_json': row['record_locator_json'],
+                                'target_locator_json': row['target_locator_json']}
+                    output.append(decision)
+                    bucket = (row['source'], action)
+                    digest = int.from_bytes(hashlib.blake2b(uid.encode(), digest_size=8).digest(), 'big')
+                    qa_candidates[bucket].append((digest, decision))
+                    if len(qa_candidates[bucket]) > 16:
+                        qa_candidates[bucket].sort(key=lambda x: x[0])
+                        del qa_candidates[bucket][16:]
+                if output:
+                    writer.write_table(pa.Table.from_pylist(output, schema=schema), row_group_size=4096)
+    finally:
+        writer.close()
+    assert observed == expected, {'missing': len(expected-observed), 'extra': len(observed-expected)}
+    assert pq.read_metadata(changes_partial).num_rows == len(expected)
+    os.replace(changes_partial, changes_path)
+
+    samples = []
+    for bucket in sorted(qa_candidates):
+        samples.extend(row for _, row in sorted(qa_candidates[bucket], key=lambda x: x[0])[:16])
+    # All six source families should contribute across all observed actions;
+    # up to 288 records are retained in one packed JSON QA artifact.
+    assert len(samples) >= 128, len(samples)
+    qa = {'status': 'PASS', 'kind': 'automated_structural_asr_consistency_not_human_listening',
+          'sampled_rows': len(samples), 'action_rows': dict(counts),
+          'samples': samples,
+          'checks': ['every audited UID has exactly one ASR-backed disposition',
+                     'all relabel texts have >=2 lexical words and >=0.68 ordered support',
+                     'no implausible ASR word rate or repeated-run relabel accepted',
+                     'source worklist and audit SHA-256 pinned']}
+    qa_path = RESOLUTION_DIR / 'QA_SAMPLES.json'
+    qa_path.write_text(json.dumps(qa, ensure_ascii=False, indent=2) + '\n')
+    spec = {'status': 'PENDING_RENDER_QA', 'version': 'hvs-s3-target-text-v1',
+            'created_utc': datetime.now(timezone.utc).isoformat(),
+            'source_worklist_sha256': sha(OUT / 'MANIFEST.json'),
+            'audit_report_sha256': sha(audit_report_path),
+            'asr_manifest_sha256': sha(asr_manifest_path),
+            'changes_file': changes_path.name, 'changes_sha256': sha(changes_path),
+            'changed_rows': len(expected), 'action_rows': dict(counts),
+            'accepted_rows': EXPECTED_ELIGIBLE - counts['exclude'],
+            'excluded_rows': counts['exclude'],
+            'excluded_target_hours': round(excluded_hours, 3),
+            'unresolved_impossible_rows': 0,
+            'by_source': {k: dict(v) for k, v in by_source.items()},
+            'qa_status': qa['status'], 'qa_file': qa_path.name,
+            'qa_sha256': sha(qa_path),
+            'important_caveat': 'Automated ASR supports conservative clipping decisions; it is not a human-verified transcript of every sample.'}
+    manifest_path = RESOLUTION_DIR / 'MANIFEST.json'
+    temp = manifest_path.with_suffix('.json.partial')
+    temp.write_text(json.dumps(spec, ensure_ascii=False, indent=2) + '\n')
+    os.replace(temp, manifest_path)
+    print(json.dumps({'status': spec['status'], 'suspects': len(expected),
+                      'actions': dict(counts), 'excluded_hours': spec['excluded_target_hours'],
+                      'qa_samples': len(samples)}), flush=True)
+
+
+if __name__ == '__main__':
+    main()

code/ref15_fullft/build_worklist.py

@@ -0,0 +1,188 @@
+#!/usr/bin/env python3
+"""Build a packed, unique-audio S3/tier7 reference-only LoRA worklist.
+
+The old S3 caption curriculum has many presentations per target. Family 0
+identifies one presentation per physical target. We use that base-index set,
+then join *all* validated reference-eligible UIDs against the frozen reference
+assets, not merely the historical 50% reference draw. Four CPU processes each
+write 32 Parquet containers; no per-sample files or decoded audio are written.
+"""
+from __future__ import annotations
+
+from collections import Counter
+import hashlib
+import json
+import math
+import os
+from pathlib import Path
+
+import numpy as np
+import pyarrow as pa
+import pyarrow.parquet as pq
+
+SC = Path('/e/scratch/reformo/schuhmann1_moss')
+CURRICULUM = SC / 'out/m2_600m_ladder/caption_curriculum_v1'
+LOGICAL = CURRICULUM / 'manifests/S3/manifest.json'
+OUT = SC / 'out/hvs_ref_only_r128_v1/worklist'
+SHARDS = 128
+WORLD = 4
+EXPECTED_UNIQUE = 7_216_621
+EXPECTED_ELIGIBLE = 3_707_948
+MISSING = np.iinfo(np.uint32).max
+# MOSS v2 has 80 ms frames. A future reference-conditioned run should allow
+# 15-20 s, not inherit the historical 37-frame (2.96 s) crop. Prefer >=5 s
+# where a verified same-speaker alternative exists; retain shorter frozen
+# references here only to honor the requested all-eligible-S3 coverage and
+# report them explicitly. 187 frames = 14.96 s; 188 would exceed 15 s.
+MAX_REF_FRAMES = 187
+PREFERRED_MIN_REF_FRAMES = 63
+
+
+def reference_bin(frames):
+    if frames < 50:
+        return 'under_4s'
+    if frames < PREFERRED_MIN_REF_FRAMES:
+        return '4_to_under_5s'
+    if frames < 125:
+        return '5_to_under_10s'
+    if frames < 188:
+        return '10_to_under_15s'
+    return '15s_or_longer'
+
+
+def sha(path):
+    h = hashlib.sha256()
+    with Path(path).open('rb') as stream:
+        for block in iter(lambda: stream.read(8 << 20), b''):
+            h.update(block)
+    return h.hexdigest()
+
+
+def read_row(fd, offsets, index):
+    a, b = int(offsets[index]), int(offsets[index + 1])
+    return json.loads(os.pread(fd, b - a, a))
+
+
+def uid_validation(uid):
+    return int.from_bytes(hashlib.blake2b(uid.encode(), digest_size=8).digest(), 'big') % 200 == 0
+
+
+def main():
+    rank = int(os.environ['SLURM_PROCID'])
+    assert int(os.environ['SLURM_NTASKS']) == WORLD and 0 <= rank < WORLD
+    logical = json.loads(LOGICAL.read_text())
+    assert logical['status'] == 'complete' and logical['stage'] == 'S3'
+    assert logical['unique_audio_occurrences'] == EXPECTED_UNIQUE
+    assert int(logical['family_reference_eligible_counts']['0']) == EXPECTED_ELIGIBLE
+    base_manifest = Path(logical['base_manifest'])
+    base_spec = json.loads(base_manifest.read_text())
+    assert base_spec['status'] == 'complete' and base_spec['ladder_stage'] == 'tier7'
+    asset_manifest = Path(logical['assets_manifest'])
+    assets = json.loads(asset_manifest.read_text())
+    assert assets['status'] == 'complete' and assets['references']['rows'] > EXPECTED_ELIGIBLE
+    descriptors = np.load(LOGICAL.parent / logical['descriptors'], mmap_mode='r', allow_pickle=False)
+    assert len(descriptors) == logical['presentations']
+    base_ids = np.sort(np.asarray(descriptors['base'][descriptors['family'] == 0], dtype=np.uint32))
+    assert len(base_ids) == EXPECTED_UNIQUE and len(np.unique(base_ids)) == EXPECTED_UNIQUE
+    base_root = base_manifest.parent
+    base_offsets = np.load(base_root / base_spec['offsets'], mmap_mode='r', allow_pickle=False)
+    assert len(base_offsets) == base_spec['presentations'] + 1
+    base_fd = os.open(base_root / base_spec['records'], os.O_RDONLY)
+    asset_root = asset_manifest.parent
+    refs = assets['references']
+    ref_keys = np.load(asset_root / refs['keys'], mmap_mode='r', allow_pickle=False)
+    ref_values = np.load(asset_root / refs['values'], mmap_mode='r', allow_pickle=False)
+    ref_offsets = np.load(asset_root / refs['offsets'], mmap_mode='r', allow_pickle=False)
+    ref_fd = os.open(asset_root / refs['records'], os.O_RDONLY)
+    assert len(ref_keys) == len(ref_values) == refs['rows']
+    assert len(ref_offsets) == len(ref_keys) + 1
+    assert ref_keys.dtype == np.dtype('S128')
+    OUT.mkdir(parents=True, exist_ok=True)
+    width = math.ceil(base_spec['presentations'] / SHARDS)
+    try:
+        for shard in range(rank, SHARDS, WORLD):
+            output = OUT / f'part-{shard:03d}.parquet'
+            proof = OUT / f'part-{shard:03d}.PASS.json'
+            if output.exists() and proof.exists():
+                prior = json.loads(proof.read_text())
+                assert prior['status'] == 'PASS' and prior['shard'] == shard
+                assert prior['output_sha256'] == sha(output)
+                print(json.dumps({'shard': shard, 'status': 'ALREADY_COMMITTED'}), flush=True)
+                continue
+            assert not output.exists() and not proof.exists(), f'incomplete shard {shard}'
+            first, last = shard * width, min((shard + 1) * width, base_spec['presentations'])
+            a, b = np.searchsorted(base_ids, [first, last])
+            selected = base_ids[a:b]
+            targets = [read_row(base_fd, base_offsets, int(index)) for index in selected]
+            queries = np.asarray([row['uid'].encode() for row in targets], dtype='S128')
+            positions = np.searchsorted(ref_keys, queries)
+            valid = positions < len(ref_keys)
+            valid[valid] = ref_keys[positions[valid]] == queries[valid]
+            output_rows = []
+            sources, source_hours = Counter(), Counter()
+            ref_bins, short_sources = Counter(), Counter()
+            for index in np.flatnonzero(valid):
+                target = targets[int(index)]
+                uid = target['uid']
+                assert len(uid.encode()) <= 128
+                reference = read_row(ref_fd, ref_offsets, int(ref_values[int(positions[index])]))
+                assert reference['uid'] == uid and reference['reference_uid'] != uid
+                locator = dict(reference['locator'])
+                original_crop = int(locator['crop_frames'])
+                full_frames = int(locator['full_frames'])
+                assert 0 < original_crop <= 37 <= full_frames
+                # The original codec shard retains all frames. Only the old
+                # locator's crop field imposed the 2.96-second ceiling.
+                used_frames = min(full_frames, MAX_REF_FRAMES)
+                locator['crop_start_frame'] = 0
+                locator['crop_frames'] = used_frames
+                assert target['record_locator'] and target['target_locator']
+                duration = float(target['dur_s'])
+                assert math.isfinite(duration) and duration > 0
+                output_rows.append({
+                    'uid': uid, 'base_index': int(selected[index]), 'source': target['source'],
+                    'duration_s': duration, 'record_locator': target['record_locator'],
+                    'target_locator': target['target_locator'],
+                    'reference_uid': reference['reference_uid'],
+                    'reference_locator': locator,
+                    'reference_full_frames': full_frames,
+                    'reference_used_frames': used_frames,
+                    'reference_under_preferred_5s': used_frames < PREFERRED_MIN_REF_FRAMES,
+                    'split': 'validation' if uid_validation(uid) else 'train',
+                })
+                sources[target['source']] += 1
+                source_hours[target['source']] += duration / 3600
+                ref_bins[reference_bin(full_frames)] += 1
+                if used_frames < PREFERRED_MIN_REF_FRAMES:
+                    short_sources[target['source']] += 1
+            temporary = output.with_suffix('.parquet.partial')
+            # Some contiguous base-index ranges have no eligible reference.
+            # A zero-row Parquet shard is valid and keeps the 128-shard index
+            # deterministic; it must not abort a 3.7M-row job after hours.
+            table = (pa.Table.from_pylist(output_rows) if output_rows else
+                     pa.table({'uid': pa.array([], type=pa.string())}))
+            pq.write_table(table, temporary,
+                           compression='zstd', row_group_size=4096)
+            os.replace(temporary, output)
+            result = {'status': 'PASS', 'shard': shard,
+                      'unique_s3_targets_examined': len(targets),
+                      'reference_eligible_rows': len(output_rows),
+                      'reference_eligible_hours': sum(source_hours.values()),
+                      'source_rows': dict(sources), 'source_hours': dict(source_hours),
+                      'reference_full_duration_bins': dict(ref_bins),
+                      'reference_under_preferred_5s_by_source': dict(short_sources),
+                      'reference_max_frames': MAX_REF_FRAMES,
+                      'reference_preferred_min_frames': PREFERRED_MIN_REF_FRAMES,
+                      'validation_rows': sum(row['split'] == 'validation' for row in output_rows),
+                      'output_sha256': sha(output), 'per_sample_files_created': 0}
+            proof_tmp = proof.with_suffix('.json.partial')
+            proof_tmp.write_text(json.dumps(result, indent=2) + '\n')
+            os.replace(proof_tmp, proof)
+            print(json.dumps(result), flush=True)
+    finally:
+        os.close(base_fd)
+        os.close(ref_fd)
+
+
+if __name__ == '__main__':
+    main()

code/ref15_fullft/preflight.py

@@ -0,0 +1,45 @@
+#!/usr/bin/env python3
+"""Require a successful full-parameter GPU canary and immutable S3 inputs."""
+from __future__ import annotations
+
+import json
+import math
+from pathlib import Path
+import sys
+
+HERE = Path(__file__).resolve().parent
+HOME = HERE.parent
+sys.path.insert(0, str(HOME / 'hvs_ref_only_r128_v1'))
+from build_worklist import sha
+from text_resolution import RESOLUTION_MANIFEST
+
+SC = Path('/e/scratch/reformo/schuhmann1_moss')
+WORKLIST = SC / 'out/hvs_ref_only_r128_v1/worklist/MANIFEST.json'
+SMOKE = SC / 'out/hvs_ref_full_ft_v1/smoke_4gpu'
+
+
+def main():
+    source = json.loads(WORKLIST.read_text())
+    resolution = json.loads(RESOLUTION_MANIFEST.read_text())
+    assert source['status'] == 'COMPLETE' and source['eligible_unique_targets'] == 3_707_948
+    assert resolution['status'] == 'PASS' and resolution['source_worklist_sha256'] == sha(WORKLIST)
+    scores = [json.loads(line) for line in (SMOKE / 'metrics.jsonl').read_text().splitlines()]
+    trains = [row for row in scores if row.get('kind') == 'train']
+    validations = [row for row in scores if row.get('kind') == 'validation']
+    assert [row['step'] for row in trains[-2:]] == [1, 2]
+    assert [row['step'] for row in validations[-3:]] == [0, 1, 2]
+    assert all(math.isfinite(row['grad_norm']) and row['grad_norm'] > 0 for row in trains[-2:])
+    assert all(math.isfinite(row['nll_per_frame']) for row in validations[-3:])
+    assert all(row['reference_longer_than_2_96s'] > 0 for row in trains[-2:])
+    assert all(row['peak_allocated_gib'] < 90 for row in trains[-2:])
+    latest = json.loads((SMOKE / 'checkpoints/current.json').read_text())
+    assert latest['step'] == 2
+    complete = json.loads((SMOKE / 'checkpoints' / latest['directory'] / 'COMPLETE.json').read_text())
+    assert complete['step'] == 2 and complete['state_bytes'] > 1_000_000_000
+    print(json.dumps({'status': 'PASS', 'steps': 2, 'peak_gib': max(row['peak_allocated_gib'] for row in trains),
+                      'validation_step2': validations[-1]['nll_per_frame'],
+                      'worklist_sha256': sha(WORKLIST), 'resolution_sha256': sha(RESOLUTION_MANIFEST)}), flush=True)
+
+
+if __name__ == '__main__':
+    main()

code/ref15_fullft/prompt_mix.py

@@ -0,0 +1,136 @@
+"""Deterministic one-presentation S3 reference-conditioned prompt mixture.
+
+Targets: ~55% local SCRIPT, 20% GENERAL+SCRIPT, 20% CAPTION+TRANSCRIPT
+(Timbre or Body Whisper 1.1), 5% TRANSCRIPT only. Every mode still receives
+the distinct reference audio, cropped to <=187 MOSS-v2 frames by the worklist.
+The complete caption annotation exists for only ~52% of eligible S3 targets;
+sample caption mode from those UIDs at the compensating rate, not from empty
+or fabricated captions. Assignment is a stable hash of target UID so resume,
+world size, or row ordering cannot change a target's presentation.
+"""
+from __future__ import annotations
+
+import hashlib
+import json
+import os
+from pathlib import Path
+import re
+
+import numpy as np
+
+from build_worklist import EXPECTED_ELIGIBLE, LOGICAL
+
+EXPECTED_ANNOTATED_ELIGIBLE = 1_945_226
+CAPTION_DRAW_WITHIN_ANNOTATED = .20 * EXPECTED_ELIGIBLE / EXPECTED_ANNOTATED_ELIGIBLE
+MODES = ('script_only', 'general_script', 'caption_timbre', 'caption_bude', 'transcript_only')
+_BAD_BODY = re.compile(r'\b(?:only silence|no speech|no spoken words|no voice)\b', re.I)
+SOURCE_PROMPT_SANITIZER_VERSION = 'hvs-source-legacy-pause-v1'
+_LEGACY_PAUSE = re.compile(r'[ \t]*\[pause[ \t]+(?:\d+(?:\.\d+)?|\.\d+)[ \t]*s\][ \t]*', re.I)
+
+
+def sanitize_source_prompt(meta: dict) -> dict:
+    """Drop obsolete numeric pause markers from source directions, not target text.
+
+    The timed renderer reconstructs pauses from word timestamps. A legacy
+    ``[pause 0.3s]`` inside a source cue must not be copied into its new
+    sentence direction; the new timed/untimed contract rejects that marker.
+    Keep the original metadata and target transcript unchanged.
+    """
+    source = str(meta.get('prompt') or '')
+    if not _LEGACY_PAUSE.search(source):
+        return meta
+    return dict(meta, prompt=_LEGACY_PAUSE.sub(' ', source))
+
+
+def mode_for_uid(uid: str, has_annotation: bool) -> str:
+    digest = hashlib.blake2b(f'hvs-ref15-mix-v1|{uid}'.encode(), digest_size=16).digest()
+    caption_draw = int.from_bytes(digest[:8], 'big') / 2**64
+    other_draw = int.from_bytes(digest[8:], 'big') / 2**64
+    if has_annotation and caption_draw < CAPTION_DRAW_WITHIN_ANNOTATED:
+        return 'caption_timbre' if other_draw < .5 else 'caption_bude'
+    if other_draw < .25:  # 20% of all rows after the 20% caption draw
+        return 'general_script'
+    if other_draw < .3125:  # 5% of all rows after the 20% caption draw
+        return 'transcript_only'
+    return 'script_only'
+
+
+class AnnotationIndex:
+    def __init__(self):
+        logical = json.loads(LOGICAL.read_text())
+        manifest_path = Path(logical['assets_manifest'])
+        assets = json.loads(manifest_path.read_text())
+        root = manifest_path.parent
+        spec = assets['annotations']
+        self.keys = np.load(root / spec['keys'], mmap_mode='r', allow_pickle=False)
+        self.values = np.load(root / spec['values'], mmap_mode='r', allow_pickle=False)
+        self.offsets = np.load(root / spec['offsets'], mmap_mode='r', allow_pickle=False)
+        self.fd = os.open(root / spec['records'], os.O_RDONLY)
+        assert self.keys.dtype == np.dtype('S128')
+        assert len(self.keys) == len(self.values) == spec['rows']
+        assert len(self.offsets) == len(self.keys) + 1
+
+    def locate_many(self, uids: list[str]) -> np.ndarray:
+        if not uids:
+            return np.empty(0, dtype=np.int64)
+        query = np.asarray([uid.encode() for uid in uids], dtype='S128')
+        positions = np.searchsorted(self.keys, query)
+        valid = positions < len(self.keys)
+        valid[valid] = self.keys[positions[valid]] == query[valid]
+        result = np.full(len(query), -1, dtype=np.int64)
+        result[valid] = self.values[positions[valid]]
+        return result
+
+    def attach(self, rows: list[dict]) -> None:
+        positions = self.locate_many([row['uid'] for row in rows])
+        for row, index in zip(rows, positions):
+            row['_annotation_index'] = int(index)
+            row['_prompt_mode'] = mode_for_uid(row['uid'], index >= 0)
+
+    def row(self, index: int) -> dict:
+        assert index >= 0
+        start, end = int(self.offsets[index]), int(self.offsets[index + 1])
+        return json.loads(os.pread(self.fd, end-start, start))
+
+
+def render_prompt(meta: dict, base_index: int, mode: str, annotation: dict | None = None):
+    from script_only import script_only
+    from timed_prompt import repair_prompt
+
+    meta = sanitize_source_prompt(meta)
+
+    if mode == 'script_only':
+        return script_only(meta, base_index)
+    if mode == 'general_script':
+        prompt, facts = repair_prompt(meta, caption=None, form='A', mode='reference',
+                                      presentation=int(base_index), phase='S3')
+        assert prompt.startswith('GENERAL:') and '\nSCRIPT:\n' in prompt
+        return prompt, dict(facts, stable_voice_tags_removed=0)
+    # The outer ASCII quotes delimit what should be spoken.  Normalize any
+    # quotes already inside the source transcript so they cannot terminate
+    # that field and make following words look like instructions.
+    transcript = str(meta['text']).strip().replace('"', "'")
+    assert transcript, meta['uid']
+    if mode == 'transcript_only':
+        return f'TRANSCRIPT: "{transcript}"', {'timed': False, 'directions': 0,
+                                                'stable_voice_tags_removed': 0}
+    assert mode in ('caption_timbre', 'caption_bude') and annotation is not None
+    assert annotation['uid'] == meta['uid']
+    if mode == 'caption_timbre':
+        caption = str(annotation['timbre_prose']).strip()
+        source = 'timbre_whisper_prose'
+    else:
+        caption = str(annotation['bude']).strip()
+        source = 'body_whisper_1_1'
+        # Some stored Body Whisper 1.1 captions start with duplicated first
+        # words such as "InIn" or "TheThe". Repair that surface glitch only;
+        # preserve the underlying annotation otherwise.
+        caption = re.sub(r'^([A-Z][a-z]{1,12})\1\b', r'\1', caption)
+        if _BAD_BODY.search(caption):
+            caption = str(annotation['timbre_prose']).strip()
+            source = 'timbre_fallback_bad_body'
+    caption = ' '.join(caption.replace('"', "'").split())
+    assert caption and transcript
+    return f'CAPTION: {caption}\nTRANSCRIPT: "{transcript}"', {
+        'timed': False, 'directions': 0, 'stable_voice_tags_removed': 0,
+        'caption_source': source}

code/ref15_fullft/rowgroup_plan.py

@@ -0,0 +1,48 @@
+"""Balance packed Parquet row groups over GPU ranks without duplicating data.
+
+The 128 source shards are *not* equal-sized. Assigning four complete shards
+to each of 32 ranks gave the slowest rank ~38.5k steps versus ~28.8k expected,
+and left some ranks idle at the tail.  Row groups are ~4096 records and can be
+partitioned by metadata without creating any per-example files.
+"""
+from __future__ import annotations
+
+import pyarrow.parquet as pq
+
+from build_worklist import OUT, SHARDS
+
+SAMPLER_VERSION = 'hvs-rowgroup-balance-v1'
+
+
+def plan(world: int):
+    assert world > 0
+    groups = []
+    for shard in range(SHARDS):
+        metadata = pq.read_metadata(OUT / f'part-{shard:03d}.parquet')
+        for group in range(metadata.num_row_groups):
+            n = metadata.row_group(group).num_rows
+            if n:
+                groups.append((n, shard, group))
+    assignments = [[] for _ in range(world)]
+    loads = [0] * world
+    for n, shard, group in sorted(groups, key=lambda x: (-x[0], x[1], x[2])):
+        rank = min(range(world), key=lambda r: (loads[r], r))
+        assignments[rank].append((shard, group))
+        loads[rank] += n
+    assert sum(loads) == sum(n for n, _, _ in groups)
+    assert len({item for rank_groups in assignments for item in rank_groups}) == len(groups)
+    return assignments, loads
+
+
+def read_rank_rows(rank: int, world: int):
+    assignments, loads = plan(world)
+    by_shard = {}
+    for shard, group in assignments[rank]:
+        by_shard.setdefault(shard, []).append(group)
+    rows = []
+    for shard, groups in sorted(by_shard.items()):
+        source = pq.ParquetFile(OUT / f'part-{shard:03d}.parquet')
+        for group in groups:
+            rows.extend(source.read_row_group(group).to_pylist())
+    assert len(rows) == loads[rank]
+    return rows, loads

code/ref15_fullft/script_only.py

@@ -0,0 +1,60 @@
+"""The exact caption-free, reference-conditioned S3 prompt surface.
+
+Keep the inherited sentence-local performance cues, timing, pauses and vocal
+bursts; remove the entire GENERAL block and all global/body captions.  The
+reference codec is provided separately through ScorePacker reference mode.
+"""
+from __future__ import annotations
+
+import re
+
+from timed_prompt import repair_prompt
+
+RENDERER_VERSION = 'hvs-script-only-source-cue-filter-v2'
+
+_STATIC = re.compile(
+    r"\b(?:male|female|masculine|feminine|gender|adult|teen|teenage|young|"
+    r"middle.?aged|elderly|old|child|childlike|baby|androgynous|"
+    r"woman|man|girl|boy|accent|register|pitch(?:ed)?|"
+    r"raspy|husky|nasal|heady|hoarse|gravelly|breathy|airy|resonan\w*|"
+    r"baritone|soprano|tenor|alto|bass|deep.?voice|thin.?voice|thick.?voice)\b",
+    re.I)
+_LEGACY_LEADING_CUE = re.compile(r"^(\s*)\(([^()\n]+)\)(?=\s|$)")
+
+
+def _without_static_cue(match, removed):
+    tags = [tag.strip() for tag in match.group(2).split(',') if tag.strip()]
+    dynamic = [tag for tag in tags if not _STATIC.search(tag)]
+    removed[0] += len(tags) - len(dynamic)
+    return f"({', '.join(dynamic)})" if dynamic else ""
+
+
+def script_only(meta: dict, base_index: int) -> tuple[str, dict]:
+    # Filter only cues in the *source* prompt before rendering.  Filtering the
+    # rendered script is unsafe: its first parenthesis may be the actual target
+    # audio's only event, e.g. a 0.6 s giggle with no aligned speech.
+    source_prompt = str(meta.get("prompt") or "")
+    removed = [0]
+    if "SCRIPT:\n" in source_prompt:
+        head, legacy_script = source_prompt.split("SCRIPT:\n", 1)
+        lines = []
+        for line in legacy_script.splitlines():
+            lines.append(_LEGACY_LEADING_CUE.sub(
+                lambda m: m.group(1) + _without_static_cue(m, removed), line,
+                count=1))
+        source_prompt = head + "SCRIPT:\n" + "\n".join(lines)
+    filtered_meta = dict(meta, prompt=source_prompt)
+    repaired, facts = repair_prompt(
+        filtered_meta, caption=None, form="A", mode="reference",
+        presentation=int(base_index), phase="S3")
+    marker = "SCRIPT:\n"
+    if marker not in repaired:
+        raise ValueError(f"No structured SCRIPT in {meta['uid']}")
+    body = repaired.split(marker, 1)[1].strip()
+    body = body.strip()
+    if not body or "GENERAL:" in body or "CAPTION:" in body or "TRANSCRIPT:" in body:
+        raise ValueError(f"Leaked caption/header in {meta['uid']}")
+    if not meta.get("text"):
+        raise ValueError(f"Empty transcript in {meta['uid']}")
+    facts = dict(facts, stable_voice_tags_removed=removed[0])
+    return body, facts

code/ref15_fullft/text_resolution.py

@@ -0,0 +1,71 @@
+"""Packed per-UID disposition sidecar for the S3 target/transcript audit.
+
+The original worklist and audio stay immutable.  Every audited suspect gets
+an explicit disposition, including `verified_keep` for a false-positive flag.
+An absent UID is an unaudited clean row and keeps its source text.
+"""
+from __future__ import annotations
+
+import json
+from collections import Counter
+from pathlib import Path
+
+import pyarrow.parquet as pq
+
+from build_worklist import EXPECTED_ELIGIBLE, OUT, sha
+
+ROOT = OUT.parent
+RESOLUTION_DIR = ROOT / 'target_text_resolution_v1'
+RESOLUTION_MANIFEST = RESOLUTION_DIR / 'MANIFEST.json'
+
+
+def load_resolution():
+    spec = json.loads(RESOLUTION_MANIFEST.read_text())
+    assert spec['status'] == 'PASS' and spec['version'] == 'hvs-s3-target-text-v1'
+    assert spec['source_worklist_sha256'] == sha(OUT / 'MANIFEST.json')
+    assert spec['accepted_rows'] + spec['excluded_rows'] == EXPECTED_ELIGIBLE
+    assert spec['qa_status'] == 'PASS'
+    qa_path = RESOLUTION_DIR / spec['qa_file']
+    assert qa_path.parent == RESOLUTION_DIR and sha(qa_path) == spec['qa_sha256']
+    qa = json.loads(qa_path.read_text())
+    assert qa['status'] == 'PASS' and qa['sampled_rows'] >= 128
+    render_path = RESOLUTION_DIR / spec['render_qa_file']
+    assert render_path.parent == RESOLUTION_DIR and sha(render_path) == spec['render_qa_sha256']
+    render_qa = json.loads(render_path.read_text())
+    assert render_qa['status'] == 'PASS' and render_qa['sampled_rows'] >= 128
+    assert spec['unresolved_impossible_rows'] == 0
+    assert spec['audit_report_sha256'] == sha(ROOT / 'target_text_audit_v1' / 'REPORT.json')
+    path = RESOLUTION_DIR / spec['changes_file']
+    assert path.parent == RESOLUTION_DIR and sha(path) == spec['changes_sha256']
+    rows = pq.read_table(path, columns=['uid', 'action', 'text']).to_pylist()
+    assert len(rows) == spec['changed_rows']
+    changes = {}
+    actions = Counter()
+    for row in rows:
+        uid, action, text = row['uid'], row['action'], row['text']
+        assert uid and uid not in changes
+        assert action in ('relabel', 'burst_only', 'exclude', 'verified_keep')
+        actions[action] += 1
+        if action in ('exclude', 'verified_keep'):
+            assert not text
+        else:
+            assert text and text.strip()
+        changes[uid] = (action, text)
+    assert actions['exclude'] == spec['excluded_rows']
+    assert dict(actions) == spec['action_rows']
+    return spec, changes
+
+
+def apply_to_rows(rows, changes):
+    kept = []
+    for row in rows:
+        change = changes.get(row['uid'])
+        if change is None:
+            kept.append(row)
+        elif change[0] == 'verified_keep':
+            kept.append(row)
+        elif change[0] in ('relabel', 'burst_only'):
+            row['_resolved_text'] = change[1]
+            row['_resolution_action'] = change[0]
+            kept.append(row)
+    return kept

code/ref15_fullft/train.py

@@ -0,0 +1,358 @@
+#!/usr/bin/env python3
+"""Full-parameter S3 long-reference continuation, matched to the Rank-128 LoRA data.
+
+All model weights remain FP32 masters; forward/loss use the original BF16
+autocast path. The source worklist, UID order, prompt renderer, resolved text,
+15-second reference packing and audio-token objective are unchanged.
+"""
+from __future__ import annotations
+
+from contextlib import nullcontext
+from datetime import timedelta
+import json
+import math
+import os
+from pathlib import Path
+import signal
+import sys
+import time
+
+import numpy as np
+import pyarrow.parquet as pq
+import torch
+import torch.distributed as dist
+from torch import nn
+from torch.nn.parallel import DistributedDataParallel as DDP
+from transformers import AutoTokenizer
+
+HERE = Path(__file__).resolve().parent
+HOME = HERE.parent
+SC = Path('/e/scratch/reformo/schuhmann1_moss')
+OLD = HOME / 'hvs_ref_only_r128_v1'
+VENDOR = HOME / 'm2_600m_tts_training/vendor/m2_20k_score'
+sys.path[:0] = [str(OLD), str(HOME), str(HOME / 'm2_dpo_smoke_v1'), str(VENDOR)]
+
+import run_smoke as core
+import state_io
+from continuation_control import complete_checkpoint, latest_checkpoint, sha
+from build_worklist import MAX_REF_FRAMES, PREFERRED_MIN_REF_FRAMES
+from prompt_mix import AnnotationIndex, MODES, SOURCE_PROMPT_SANITIZER_VERSION
+from qa_worklist import Reader
+from rowgroup_plan import read_rank_rows, SAMPLER_VERSION
+from script_only import RENDERER_VERSION
+from text_resolution import apply_to_rows, load_resolution, RESOLUTION_MANIFEST
+from train import packed, mean_validation
+
+WORKLIST = SC / 'out/hvs_ref_only_r128_v1/worklist'
+BASE = SC / 'out/m2_600m_ladder/caption_curriculum_v1/training/checkpoints/step-00032337'
+SCHEMA = SC / 'out/m2_20k_score/score_schema.json'
+ROOT = SC / 'out/hvs_ref_full_ft_v1'
+SEED = 20261004
+PEAK_BACKBONE = 8e-6
+PEAK_TALKER = 1.6e-5
+WARMUP_FRACTION = .05
+FLOOR_RATIO = .10
+SAVE_EVERY = 1000
+STOP = False
+
+
+class Step(nn.Module):
+    """DDP boundary around exactly the original teacher-forced audio loss."""
+
+    def __init__(self, base, config, device):
+        super().__init__()
+        self.base = base
+        self.config = config
+        self.device = device
+
+    def forward(self, batch):
+        logprob, _ = core.logprob(self.base, self.config, batch, self.device)
+        return -logprob
+
+
+def stop(_signal, _frame):
+    global STOP
+    STOP = True
+
+
+def atomic_json(path: Path, value: dict) -> None:
+    temporary = path.with_suffix(path.suffix + '.partial')
+    temporary.write_text(json.dumps(value, ensure_ascii=False, allow_nan=False, indent=2) + '\n')
+    os.replace(temporary, path)
+
+
+def lr_factor(step: int, total: int) -> float:
+    warmup = max(1, round(total * WARMUP_FRACTION))
+    if step <= warmup:
+        return step / warmup
+    progress = min(1., (step - warmup) / max(1, total - warmup))
+    return FLOOR_RATIO + (1 - FLOOR_RATIO) * .5 * (1 + math.cos(math.pi * progress))
+
+
+def prepare_rows(rank, world, changes, annotations):
+    assigned, loads = read_rank_rows(rank, world)
+    assigned = apply_to_rows(assigned, changes)
+    rows = [row for row in assigned if row['split'] == 'train']
+    assert rows
+    annotations.attach(rows)
+    validation = []
+    for shard in range(128):
+        for row in pq.read_table(WORKLIST / f'part-{shard:03d}.parquet').to_pylist():
+            if row['split'] == 'validation' and len(validation) < world * 16:
+                validation.append(row)
+        if len(validation) >= world * 16:
+            break
+    assert len(validation) == world * 16
+    validation = apply_to_rows(validation, changes)
+    annotations.attach(validation)
+    for cohort in (rows, validation):
+        for row in cohort:
+            if row.get('_resolution_action') == 'burst_only':
+                row['_prompt_mode'] = 'script_only'
+            elif row.get('_resolution_action') == 'relabel' and row['_prompt_mode'] in (
+                    'general_script', 'caption_bude'):
+                row['_prompt_mode'] = 'script_only'
+    return rows, validation, loads
+
+
+def main() -> None:
+    smoke = os.environ.get('HVS_FULL_FT_SMOKE') == '1'
+    rank, world, local = (int(os.environ[key]) for key in
+                          ('SLURM_PROCID', 'SLURM_NTASKS', 'SLURM_LOCALID'))
+    assert world == (4 if smoke else 32)
+    visible = torch.cuda.device_count()
+    assert visible in (1, 4), visible
+    device_index = 0 if visible == 1 else local
+    device = torch.device('cuda', device_index)
+    torch.cuda.set_device(device)
+    torch.set_num_threads(4)
+    dist.init_process_group('nccl', init_method='env://', rank=rank, world_size=world,
+                            timeout=timedelta(minutes=30), device_id=device)
+    signal.signal(signal.SIGUSR1, stop)
+    signal.signal(signal.SIGTERM, stop)
+    started = time.monotonic()
+
+    manifest_path = WORKLIST / 'MANIFEST.json'
+    manifest = json.loads(manifest_path.read_text())
+    assert manifest['status'] == 'COMPLETE' and manifest['eligible_unique_targets'] == 3_707_948
+    assert manifest['reference_max_frames'] == MAX_REF_FRAMES == 187
+    assert manifest['reference_preferred_min_frames'] == PREFERRED_MIN_REF_FRAMES == 63
+    assert manifest['prompt_mix_version'] == 'hvs-ref15-mix-v1'
+    resolution, changes = load_resolution()
+    assert resolution['status'] == 'PASS'
+    qa = json.loads((WORKLIST.parent / 'QA_SAMPLES.json').read_text())
+    assert qa['status'] == 'PASS' and qa['sampled_rows'] == 64
+    assert set(qa['prompt_mode_rows']) == set(MODES)
+    annotations = AnnotationIndex()
+    rows, validation, loads = prepare_rows(rank, world, changes, annotations)
+    effective = torch.tensor([sum(row['_prompt_mode'] == mode for row in rows)
+                              for mode in MODES], dtype=torch.int64, device=device)
+    dist.all_reduce(effective)
+    prompt_counts = {mode: int(effective[i].item()) for i, mode in enumerate(MODES)}
+    assert sum(prompt_counts.values()) <= resolution['accepted_rows']
+    accum = 2 if smoke else 4
+    local_steps = torch.tensor([math.ceil(len(rows) / accum)], dtype=torch.int64, device=device)
+    dist.all_reduce(local_steps, op=dist.ReduceOp.MAX)
+    total = 2 if smoke else int(local_steps.item())
+    assert len(rows) >= accum * total if smoke else total > 1000
+    if not smoke:
+        assert total == 26774, total
+    order = np.random.default_rng(SEED + rank).permutation(len(rows))
+    torch.manual_seed(SEED)
+    torch.cuda.manual_seed_all(SEED)
+
+    schema = json.loads(SCHEMA.read_text())
+    model, config = core.build_fresh(schema, log=lambda _: None)
+    state = torch.load(BASE / 'model_bf16.pt', map_location='cpu', weights_only=True)
+    model.load_state_dict(state, strict=True)
+    model.tie_weights()
+    del state
+    # FP32 master parameters are necessary at these small continuation LRs:
+    # tiny updates to BF16 parameter tensors can round away completely.
+    model = model.to(device=device, dtype=torch.float32).eval()
+    assert all(model.audio_lm_heads[i].weight.data_ptr() ==
+               model.audio_embeddings[i].weight.data_ptr() for i in range(12))
+    # The inherited score conditioner is unused by this audio-caption loss;
+    # retain its original frozen state. Every *active* semantic and Talker
+    # parameter is fully fine-tuned, without a LoRA or adapter.
+    for name, parameter in model.named_parameters():
+        parameter.requires_grad_(not name.startswith('score_conditioner.'))
+    groups = {'backbone': [], 'talker': []}
+    for name, parameter in model.named_parameters():
+        if not parameter.requires_grad:
+            continue
+        key = 'backbone' if name.startswith(('transformer.', 'text_lm_head.')) else 'talker'
+        groups[key].append(parameter)
+    assert all(groups.values())
+    optimizer = torch.optim.AdamW([
+        {'name': 'backbone', 'params': groups['backbone'], 'lr': PEAK_BACKBONE},
+        {'name': 'talker', 'params': groups['talker'], 'lr': PEAK_TALKER},
+    ], betas=(.9, .95), eps=1e-8, weight_decay=.01, foreach=False)
+    wrapper = DDP(Step(model, config, device).eval(), device_ids=[device_index],
+                  broadcast_buffers=False, find_unused_parameters=True,
+                  gradient_as_bucket_view=True)
+    _, _, Processor = core.moss_small.export_classes()
+    processor = Processor(tokenizer=AutoTokenizer.from_pretrained(
+        core.moss_small.SFT3, trust_remote_code=True, local_files_only=True),
+        audio_tokenizer=None, model_config=config)
+    packer = core.ScorePacker(processor, config, schema)
+    reader = Reader()
+    run = ROOT / ('smoke_4gpu' if smoke else 'training')
+    run.mkdir(parents=True, exist_ok=True)
+    contract = {
+        'kind': 'hvs-long-ref-full-ft-v1', 'world': world, 'seed': SEED,
+        'base_checkpoint': str(BASE), 'manifest_sha256': sha(manifest_path),
+        'resolution_sha256': sha(RESOLUTION_MANIFEST), 'schema_sha256': sha(SCHEMA),
+        'source_prompt_sanitizer_version': SOURCE_PROMPT_SANITIZER_VERSION,
+        'script_renderer_version': RENDERER_VERSION, 'sampler_version': SAMPLER_VERSION,
+        'lr_backbone_peak': PEAK_BACKBONE, 'lr_talker_peak': PEAK_TALKER,
+        'objective': 'same_audio_logprob_as_reference_lora',
+        'prompt_counts': prompt_counts, 'total_steps': total,
+    }
+    chosen = [None]
+    scheduler = NoOpScheduler()
+    if rank == 0:
+        try:
+            directory = latest_checkpoint(run / 'checkpoints', contract, world)
+            if directory is not None:
+                complete_checkpoint(directory, contract, world, content=True)
+            chosen[0] = {'path': str(directory) if directory else None}
+        except Exception as error:
+            chosen[0] = {'error': repr(error)}
+    dist.broadcast_object_list(chosen, src=0)
+    assert 'error' not in chosen[0], chosen[0]
+    completed = 0
+    if chosen[0]['path']:
+        completed = state_io.load(chosen[0]['path'], wrapper, optimizer, scheduler, contract)
+    # state_io expects an ordinary scheduler object. The schedule is instead
+    # a deterministic pure function of the committed global step, so give it
+    # a no-op state carrier to preserve its atomic checkpoint/restore path.
+    if rank == 0:
+        print(json.dumps({'kind': 'start', 'smoke': smoke, 'world': world,
+                          'steps': total, 'resumed_step': completed,
+                          'trainable_parameters': sum(p.numel() for p in model.parameters() if p.requires_grad),
+                          'frozen_unused_parameters': sum(p.numel() for p in model.parameters() if not p.requires_grad),
+                          'backbone_parameters': sum(p.numel() for p in groups['backbone']),
+                          'talker_parameters': sum(p.numel() for p in groups['talker']),
+                          'effective_batch': world * accum,
+                          'prompt_counts': prompt_counts,
+                          'reference_max_seconds': MAX_REF_FRAMES * .08,
+                          'rowgroup_load_min': min(loads), 'rowgroup_load_max': max(loads)}), flush=True)
+    metrics = run / 'metrics.jsonl'
+    if completed == 0:
+        value, n = mean_validation(model, config, reader, packer, annotations,
+                                   validation, device, rank, world)
+        if rank == 0:
+            with metrics.open('a') as output:
+                output.write(json.dumps({'kind': 'validation', 'step': 0,
+                                         'nll_per_frame': value, 'rows': n}) + '\n')
+    last = completed
+    for step in range(completed + 1, total + 1):
+        last = step
+        optimizer.zero_grad(set_to_none=True)
+        local_count = max(0, min(accum, len(rows) - (step - 1) * accum))
+        global_count = torch.tensor([local_count], dtype=torch.int64, device=device)
+        dist.all_reduce(global_count)
+        assert global_count.item() > 0
+        local_modes = [0] * len(MODES)
+        ref_frames = long_refs = short_refs = 0
+        local_loss = 0.
+        for micro in range(accum):
+            position = (step - 1) * accum + micro
+            active = position < len(rows)
+            row = rows[int(order[position])] if active else rows[0]
+            batch, facts = packed(row, reader, packer, annotations)
+            if active:
+                local_modes[MODES.index(facts['prompt_mode'])] += 1
+                ref_frames += facts['reference_frames']
+                long_refs += int(facts['reference_frames'] > 37)
+                short_refs += int(facts['reference_frames'] < PREFERRED_MIN_REF_FRAMES)
+            context = wrapper.no_sync() if micro + 1 < accum else nullcontext()
+            with context:
+                raw_loss = wrapper(batch)
+                assert torch.isfinite(raw_loss)
+                (raw_loss * (world / int(global_count.item()) if active else 0.)).backward()
+            if active:
+                local_loss += float(raw_loss.detach())
+        grad_norm = torch.nn.utils.clip_grad_norm_(wrapper.parameters(), 1.)
+        assert torch.isfinite(grad_norm) and float(grad_norm) > 0
+        update_canary = None
+        if smoke and step == 1:
+            representatives = []
+            for group_name in ('backbone', 'talker'):
+                chosen_parameter = next((parameter for parameter in groups[group_name]
+                                         if parameter.grad is not None and
+                                         bool(parameter.grad.detach().abs().max() > 0)), None)
+                assert chosen_parameter is not None, f'No {group_name} gradient'
+                representatives.append((group_name, chosen_parameter,
+                                        chosen_parameter.detach().clone()))
+            update_canary = representatives
+        multiplier = lr_factor(step, total)
+        optimizer.param_groups[0]['lr'] = PEAK_BACKBONE * multiplier
+        optimizer.param_groups[1]['lr'] = PEAK_TALKER * multiplier
+        optimizer.step()
+        if update_canary is not None:
+            weight_changes = {name: float((parameter.detach() - before).abs().sum())
+                              for name, parameter, before in update_canary}
+            assert weight_changes['backbone'] > 0 and weight_changes['talker'] > 0, weight_changes
+            if rank == 0:
+                print(json.dumps({'kind': 'full_ft_update_canary', 'step': step,
+                                  'backbone_l1_change': weight_changes['backbone'],
+                                  'talker_l1_change': weight_changes['talker']}), flush=True)
+        summary = torch.tensor([ref_frames, long_refs, short_refs, *local_modes],
+                               dtype=torch.int64, device=device)
+        dist.all_reduce(summary)
+        if rank == 0 and (smoke or step % 50 == 0):
+            event = {'kind': 'train', 'step': step, 'total': total,
+                     'loss_local': local_loss / max(1, local_count),
+                     'examples_global': int(global_count.item()),
+                     'grad_norm': float(grad_norm),
+                     'lr_backbone': optimizer.param_groups[0]['lr'],
+                     'lr_talker': optimizer.param_groups[1]['lr'],
+                     'reference_mean_seconds': float(summary[0]) * .08 / int(global_count.item()),
+                     'reference_longer_than_2_96s': int(summary[1]),
+                     'reference_under_preferred_5s': int(summary[2]),
+                     'prompt_mode_counts': {mode: int(summary[i+3]) for i, mode in enumerate(MODES)},
+                     'elapsed_s': time.monotonic() - started,
+                     'peak_allocated_gib': torch.cuda.max_memory_allocated(device) / 2**30}
+            with metrics.open('a') as output:
+                output.write(json.dumps(event) + '\n')
+            print(json.dumps(event), flush=True)
+        value = None
+        if smoke or step % 500 == 0 or step == total:
+            value, n = mean_validation(model, config, reader, packer, annotations,
+                                       validation, device, rank, world)
+            if rank == 0:
+                with metrics.open('a') as output:
+                    output.write(json.dumps({'kind': 'validation', 'step': step,
+                                             'nll_per_frame': value, 'rows': n}) + '\n')
+        flag = torch.tensor([int(STOP or time.monotonic() - started > 10.5 * 3600)], device=device)
+        dist.all_reduce(flag, op=dist.ReduceOp.MAX)
+        if (smoke and step == total) or (not smoke and step % SAVE_EVERY == 0) or step == total or flag.item():
+            directory = state_io.save(run / 'checkpoints', wrapper, optimizer,
+                                      scheduler, step, contract,
+                                      export=(step == total or (not smoke and step % 5000 == 0)))
+            if rank == 0:
+                print(json.dumps({'kind': 'checkpoint', 'step': step,
+                                  'directory': str(directory), 'validation_nll_per_frame': value}), flush=True)
+        if flag.item():
+            break
+    if rank == 0 and last == total:
+        atomic_json(run / 'DONE.json', {'status': 'COMPLETE', 'step': total,
+                    'manifest_sha256': contract['manifest_sha256'],
+                    'resolution_sha256': contract['resolution_sha256'],
+                    'base_checkpoint': str(BASE), 'time': time.time()})
+    dist.barrier()
+    dist.destroy_process_group()
+
+
+class NoOpScheduler:
+    def state_dict(self):
+        return {}
+
+    def load_state_dict(self, state):
+        assert state == {}
+
+
+if __name__ == '__main__':
+    main()

code/ref15_fullft/train_full.sbatch

@@ -0,0 +1,31 @@
+#!/bin/bash
+#SBATCH --job-name=hvs-ref15-s3-fullft
+#SBATCH --account=reformo
+#SBATCH --partition=booster
+#SBATCH --nodes=8
+#SBATCH --ntasks-per-node=4
+#SBATCH --gres=gpu:4
+#SBATCH --cpus-per-task=8
+#SBATCH --time=11:30:00
+#SBATCH --output=/e/scratch/reformo/schuhmann1_moss/out/hvs_ref_full_ft_v1/logs/train_%j.out
+#SBATCH --error=/e/scratch/reformo/schuhmann1_moss/out/hvs_ref_full_ft_v1/logs/train_%j.err
+set -euo pipefail
+SC=/e/scratch/reformo/schuhmann1_moss
+source "$SC/code/fastgen.sh"
+export HF_HOME="$SC/cache/hf" XDG_CACHE_HOME="$SC/cache/xdg" TORCH_HOME="$SC/cache/torch" TRITON_CACHE_DIR="$SC/cache/triton"
+mkdir -p "$HF_HOME" "$XDG_CACHE_HOME" "$TORCH_HOME" "$TRITON_CACHE_DIR"
+export OMP_NUM_THREADS=4 TOKENIZERS_PARALLELISM=false HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1
+export PYTHONPATH=/e/home/jusers/schuhmann1/jupiter/hvs_ref_only_r128_v1:${PYTHONPATH:-}
+export MASTER_ADDR
+MASTER_ADDR=$(scontrol show hostnames "$SLURM_JOB_NODELIST" | head -n 1)
+export MASTER_PORT=$((20000 + SLURM_JOB_ID % 20000))
+cd "$SC/out/hvs_ref_full_ft_v1"
+python -u /e/home/jusers/schuhmann1/jupiter/hvs_ref_only_r128_v1/qa_legacy_pause_resume.py
+python -u /e/home/jusers/schuhmann1/jupiter/hvs_ref_full_ft_v1/preflight.py
+if [ -f "$SC/out/hvs_ref_full_ft_v1/training/DONE.json" ]; then
+    echo "Full S3 long-reference fine-tune already complete; no redundant segment."
+    exit 0
+fi
+srun --nodes=8 --ntasks=32 --ntasks-per-node=4 --gpus-per-task=1 --kill-on-bad-exit=1 --cpu-bind=none \
+    python -u /e/home/jusers/schuhmann1/jupiter/hvs_ref_full_ft_v1/train.py
+

docs/S3_REF15_FULL_FT.md

@@ -0,0 +1,184 @@
+# S3 Ref15 Full FT: data selection, training, and the reference-length fix
+
+This document describes the model-only checkpoint in
+[`checkpoints/S3-Ref15-FullFT/model_bf16.pt`](../checkpoints/S3-Ref15-FullFT/model_bf16.pt).
+It is the **26,774th and final update** of a full-parameter, reference-conditioned
+continuation of Humaneness Voice Small S3. It starts from the **original S3
+step-32,337 checkpoint**, not from the subsequent Rank-128 Ref15 LoRA, a DPO
+adapter, S10, or a randomly initialized language model. The original S3
+checkpoint remains available at [`checkpoints/S3/`](../checkpoints/S3/).
+The user selected this Full FT checkpoint as the new repository default after
+listening and finding it acoustically preferable. That is a listening-based
+choice, **not a claim that every automatic metric improved**.
+
+## What changed from the original S3 model?
+
+The original S1→S10 curriculum cropped a reference to at most **37 MOSS-v2
+80-ms frames = 2.96 seconds** in the training loader. An older public
+inference example imposed the same ceiling. The codec and network did not
+require that limit: the stored reference codec sequence was longer. This
+continuation changed the data locator's crop to the first **at most 187 frames
+= 14.96 seconds**, and the public inference example now accepts 15 seconds by
+default. No codec, token count per frame, Qwen backbone shape, Talker shape,
+or vocoder architecture changed. The reference is passed through the same
+dedicated reference-audio channel as before, not pasted into prompt text.
+
+The retrain also deliberately changes what text accompanies the reference.
+Every accepted example uses a distinct reference clip. Most prompts omit the
+global voice-description caption so the reference has less competition as a
+source of speaker identity. Some structured and caption prompts remain to
+preserve the model's broader instruction vocabulary. These changes were made
+**together**; this run alone cannot isolate whether longer references, more
+frequent reference conditioning, prompt simplification, full-model updates,
+or extra training caused a perceived difference.
+
+## Which recordings were selected?
+
+The selection starts with the physical **tier7 / logical S3** data in the
+[TTS scaling ladder](https://huggingface.co/datasets/laion/tts-scaling-ladder-de-en).
+The logical training manifest contains repeated presentations of recordings.
+Its family-0 base indices identify **7,216,621 distinct target recordings**.
+These were joined by UID to the already validated reference assets, yielding
+**3,707,948 targets with an available separate reference**, or **8,832.58
+hours of unique target audio before text-quality exclusions**. The target is
+not reused as its own reference. The worklist is 128 packed Parquet shards;
+no millions of individual audio files were created, and no audio was
+re-encoded to build it.
+
+The reference policy was intentionally inclusive: prefer a reference of at
+least about five seconds where available, but retain the existing shorter
+frozen pairs to cover all eligible S3 targets. Among the 3,707,948 initial
+reference-eligible targets, **513,413 (13.85%)** had a presented reference
+under five seconds. The complete stored reference is cropped from its start
+to at most 187 frames. The training system checked that target and reference
+UIDs differ, but this is **not** a manual same-speaker audit of every pair.
+
+The initial reference-eligible source composition was 1,394,379
+`vprof_original`, 1,098,944 `vc_sidon`, 696,175 `vprof_repair`, 491,992
+`emolia`, 18,620 `kartoffelphon`, and 7,838 `mls` targets. These are **row
+counts before the text-resolution exclusions**, not source hours or an
+assertion that all sources have the same provenance. The original worklist
+manifest also records source hours and reference-duration bins.
+
+### Target-text integrity gate
+
+A prior canary caught clips whose stored transcript described more speech
+than the actual short target contained. A complete audit and ASR-supported,
+conservative resolution sidecar was applied **before** this Full FT run.
+Across the reference-eligible worklist, it examined 896,551 suspect rows:
+440,195 were relabelled where supported, 20,081 were verified to keep, and
+436,275 were excluded. That leaves **3,271,673 accepted targets**; the
+exclusions account for **183.251 target-audio hours**, so accepted audio is
+approximately **8,649.33 unique target hours including validation**. The
+exact train/validation UID split is 3,255,217 / 16,456 after resolution.
+The ASR decisions and sample QA passed the run's checks, but they are not a
+claim of human-verifying every transcript. The original 0.5% split is
+target-UID based, **not speaker-disjoint**.
+
+### Exact training prompt mixture
+
+Each accepted training UID appears **once**, assigned deterministically by
+UID hash. Every mode includes the separate reference-audio input. The exact
+effective counts recorded in the final checkpoint are:
+
+| Mode | Training UIDs | What the instruction contains |
+| --- | ---: | --- |
+| Bare SCRIPT | 1,932,540 | Spoken text with sentence-local parenthesized performance, emotion, pause, duration and burst cues where available; no `GENERAL:` block. Stable identity descriptions are filtered out of local cues. |
+| `GENERAL:` + `SCRIPT:` | 570,663 | Historical structured prompt, retained for compatibility. |
+| Timbre Whisper caption + transcript | 318,972 | `CAPTION:` prose and exact `TRANSCRIPT:`. |
+| Body/BUD-E Whisper 1.1 caption + transcript | 269,892 | Same outer surface; known duplicated first-word glitches are repaired, and suspect no-speech captions fall back to Timbre prose. |
+| Transcript only | 163,150 | `TRANSCRIPT:` and reference audio, with no caption. |
+
+These sum to **3,255,217** training UIDs, approximately 59.4% / 17.5% /
+9.8% / 8.3% / 5.0% after the quality exclusions. The design target before
+exclusions was roughly 55% / 20% / 10% / 10% / 5%. Caption modes were drawn
+only for UIDs that actually had the corresponding annotation; captions were
+not invented for unannotated rows. The script renderer retains valid
+sentence timing cues, pauses and vocal bursts. The prompt surfaces do not
+imply perfect duration or burst-onset control.
+
+## How it was trained
+
+The base weight file was S3's `step-00032337/model_bf16.pt`. The continuation
+updated all **active** Qwen3 semantic-backbone, bridge and 12-codebook MOSS
+Talker parameters with the unchanged teacher-forced audio-token log-probability
+objective. The inherited but unused score conditioner stayed frozen. There
+was **no LoRA** in this run. Model master parameters and AdamW moments were
+FP32, while forward/loss used BF16 autocast; this avoids tiny updates being
+rounded away in BF16 master weights.
+
+Training ran on **eight JUPITER Booster nodes / 32 GH200 GPUs**, global batch
+**128**, one deterministic pass over the accepted train UIDs. The semantic
+peak LR was **8×10⁻⁶**; the Talker/bridge peak was **1.6×10⁻⁵**. Both had
+5% warm-up followed by a single cosine decay to 10% of peak, **without a
+restart**. AdamW used β=(0.9, 0.95), ε=10⁻⁸, weight decay 0.01 and global
+gradient-norm clipping at 1.0. Unequal source-shard sizes were balanced by
+assigning packed 4,096-row Parquet groups across ranks; the final dummy tail
+has zero loss weight, so it does not add false training examples. Runs could
+resume exact model, optimizer and RNG states from atomic checkpoints.
+
+The final state is **step 26,774**. Its exported BF16 model is
+**1,431,076,525 bytes**, SHA-256
+`e1b2c4613545d1db21c6f60a5ab2c27b5320aa05c177717ae268b1ff870d406e`.
+The local optimizer-bearing checkpoint is about 8 GB and is **not** part of
+this public model-only release. The run's immutable worklist and text-repair
+SHA-256 values are recorded in the adjacent `metadata.json`; those JUPITER
+Scratch paths identify historical provenance, not portable data locations.
+The scalar [training log](../stats/ref15_fullft_training_metrics.jsonl) and
+[paired 50-case result JSON](../stats/ref15_fullft_listening50.json) are
+included for audit. The historical source entrypoints are under
+[`code/ref15_fullft/`](../code/ref15_fullft/); they depend on the original
+packed JUPITER manifests and other release code, so they are provenance
+rather than a one-command portable training package.
+
+## What the available evaluation says
+
+The checkpoint's bounded internal validation diagnostic used 511 rows and
+reported 62.058 negative log-likelihood units per frame before continuation
+and 61.997 at the final step. It was **not** the lowest value during the run
+and is not a perceptual score. A separate **50-case paired listening grid**
+used identical target prompts, 5.20–14.96-second references and random seeds
+for original S3, Full FT step 25,000 and Full FT step 26,774. Those 50 cases
+were UID-held-out but **not guaranteed speaker-disjoint**.
+
+| Model | Mean Orange cosine to reference ↑ | Mean Orange cosine to target ↑ | Corpus Parakeet WER ↓ |
+| --- | ---: | ---: | ---: |
+| Original S3 step 32,337 | **0.745** | 0.658 | 0.088 |
+| Full FT step 25,000 | 0.713 | **0.745** | **0.083** |
+| Full FT step 26,774 | 0.707 | 0.734 | 0.096 |
+
+The user found step 26,774 acoustically preferable in listening. The table
+does **not** demonstrate a general voice-cloning or WER improvement for that
+step: Orange similarity to the provided reference decreased, while similarity
+to the intended target increased, and final WER is slightly worse than the
+original S3 in this small test. The reference/target discrepancy deserves
+separate examination. Automated scores cannot settle the subjective
+naturalness question; a blinded, speaker-disjoint evaluation is still needed.
+This repository's original S1–S10 acting benchmark was a different protocol
+and must not be numerically merged with the 50-case Ref15 test.
+
+## Loading and limitations
+
+From this repository's root, the updated example defaults to this checkpoint:
+
+```bash
+python code/infer.py --reference-wav reference.wav --language en --frames 75 \
+  --prompt $'CAPTION: Warm, softly amused, conversational.\nTRANSCRIPT: "We made it home."' \
+  --text 'We made it home.' --output ref15_example.wav
+```
+
+`--stage default` and `--stage S3-Ref15-FullFT` resolve to the same file;
+`--stage S3` explicitly loads the unchanged original S3. Provide a **clean,
+distinct** reference, ideally 5–15 seconds. Shorter clips are accepted with
+a warning; references longer than the 14.96-second training ceiling are out
+of distribution. Because **every** accepted training example for this
+continuation carried a reference, reference-free generation is not validated
+as a new default behavior; compare the original S3 for that use. The
+separate [MOSS Audio Tokenizer v2](https://huggingface.co/OpenMOSS-Team/MOSS-Audio-Tokenizer-v2)
+is still required to encode references and decode the 12 acoustic codebooks
+per 80-ms frame. The new weight is not a standard one-call
+`AutoModel.from_pretrained()` checkpoint.
+
+The original ladder stages, their selected validation checkpoints, and
+their prior evaluations remain in the repository. Nothing here retroactively
+changes the reference exposure or 2.96-second training crop of S1–S10.