Implemented Large1.3 reinforcement learning

How the new reinforcement-learning adapter was trained

The final Large1.3Alpha full-SFT model (step9,799) remains the frozen base. A new rank128 attention adapter in both the semantic backbone and local Talker was optimized using VESPO sequence importance weights with separately group-normalized reward components, a group-relative policy-gradient method. This is not the flow-matching GROW method used for the separate Voice Converter model.

For each source example, eight candidates were sampled in a reference-conditioned group and eight in a separate no-reference group. Both groups use the same source utterance and are accepted or rejected together. Training uses equal real-speech/Gemini proportions and balanced English, German, French, Dutch and Spanish source examples. Candidate transcripts are checked with multilingual Parakeet ASR; WER above15%, invalid audio or truncation disqualifies a candidate.

ComponentWeightMeaning
Content Enjoyment33%Humaneness EARS Medium audiobox:CE, scaled by10.
Two-emotion matching33%1 minus mean squared error to two FIXED strongest supported emotions in the source target audio, after scaling by4. Heads are not reselected from each candidate or from the speaker reference.
Genuineness34%Humaneness EARS Medium genuineness_0_6, scaled by6.

The reporting reward is0.33×CE/10 +0.33×(1−MSE_top2)+0.34×genuineness/6. Policy advantages normalize each component within its group; sampled KL and a gold SFT anchor (each0.02) regularize the update. Speaker similarity and VoiceCLAP are NOT reward terms in this run. Learned scores are useful proxies, not human listener ratings or proof of a quality improvement.

Final recorded progress: 1500 attempted groups; 1452 successful optimizer updates. Rejected groups remain logged. Checkpoint files retain adapter weights, AdamW, all rank RNG states, schedule and sampler position.

Checkpoint adapter SHA256: c26031b9e1932ae0e76b29ec894b49bb6f719a9ff997cf733eae8bbf443f9fb2. The selected checkpoint is the completed final run, not a reward-selected “best” or a smoke checkpoint.