Good idea, so we just ran it: the answers don't depend on option position. Same clips, options shuffled, letters mapped back to labels (DuplexJev-32B-Para v1.3, one order per request, no averaging):

• 29 clips; 4-way emotion and 4-way turn state with 15 distinct option orders each, gender with both orders.
• 98.5% of answers are identical across all orders (emotion 686/696, turn 685/696, gender 58/58).
• When the model is confident (≥ 0.8), zero flips, and the winning label's probability barely moves (median change 0.01 for emotion, 0.04 for turn).
• The few flips all come from 2 genuinely ambiguous clips per question, where the model was only 0.55–0.67 sure to begin with.

For benchmarks we still average over a few orders, which settles even those borderline clips (29/29 match) at almost no cost, since the orders share the audio prefix. We'll run the same check on the full benchmark set and add it to the repo. Thanks for the suggestion!

This is a fantastic review, thank you! You reproduced the whole pipeline from our scripts and then went further than we did. That's exactly why we published every model's raw answers, and it's great to see someone put them to work.

You're right that the headline should stand on clips we never trained on, so we're adopting your split as the main number. On those 1,035 items, DuplexJev still comes out on top:

• DuplexJev-32B-Para v1.3: 50.2
• Gemini 3.8 Flash: 47.8
• Qwen3-Omni-30B: 45.0

Your split also shows how broad the lead is. Of the 11 corpora in the 1,035, DuplexJev is first (or tied first) on 8, with big margins where only the voice carries the answer:

• TESS (emotion with neutral words): 38.3 vs 17.0 Flash / 25.5 Qwen
• MSPB (prosody): 77.3 vs 67.2 / 71.9
• IEMOCAP non-verbal (laughs, sighs, breaths): 27.1 vs 21.1 / 21.1

And as your bootstrap shows, the 5.2-point lead over Qwen3-Omni holds firmly (95% CI 2.2 to 8.2).

MELD is a great catch. It's TV dialogue where the words themselves carry most of the emotion, the one setting where models that read the words have the edge. We're adding conversational dialogue emotion data in our next training round and will report MELD separately so everyone can follow the progress.

We'll update the README and scripts to report the 1,035 split as the headline, with the trained-corpus items listed separately. Thanks again, reviews like this make the work better!

You're right again, and thank you! Two reviews in, you've made our benchmark reporting better than when we posted it.

To your question: v1.3's emotion supervision stops at those four labels (angry, happy, neutral, sad). Fear, disgust, surprise and calm were never answer targets, so nothing is lost at readout; the model was simply never taught them. Your split shows it cleanly:

• On the four labels it was trained on: 83/116 (72%), vs 46 for Qwen3-Omni and 39 for Gemini 3.8 Flash.
• Same four labels, unseen corpora only (TESS, RAVDESS, SAVEE): 29/54, vs 22 and 16.
• On labels it never saw: 3/84. That's the gap to close.

Expanding to 10 emotion classes is scheduled for next week. When it ships, we'll report LISTEN neutral-text your way: trained labels vs. the rest.

And good catch on IEMOCAP. PARA alone is 26.0 vs 25.0 for GPT-audio-1.5, and on the 51 EM clips Qwen3-Omni and GPT-audio lead. The README reports per-condition numbers, so no pooled figure there.

One more thing to watch for this week: we're preparing to open-source sophon-realtime, our full-duplex conversational model, with DuplexJev as its ears. Would love your eyes on it too.