Measured comparison against VITS/Piper on CPU, and a question about prosody control

[aradix85]

I have been evaluating OptiSpeech as a possible replacement for Piper in a screen
reader context (NVDA), and ended up with numbers that seem worth sharing, since
#18 and #23 ask roughly the same question and nobody has posted measurements yet.

Setup

• Intel Core Ultra 7 258V, 8 cores, Windows 11, Python 3.12
• onnxruntime 1.27.0, CPUExecutionProvider only (no OpenVINO, no GPU)
• espeak-ng 1.52 for phonemization, called through its shared library so that
  phonemization cost (~0.03 ms) does not pollute the numbers
• Median of 5 runs after a warm-up call, full chain: text to phonemes to audio

Models: mush42__optispeech-lightspeech-en-us-emily and
mush42__optispeech-convnext-en-us-emily (via OpenVoiceOS/phoonnx-optispeech),
against Piper en_US-lessac-high and a Dutch medium VITS voice.

Latency

model	load	short label	full sentence	RTF
Piper lessac-high (VITS)	3102 ms	197.5 ms	520 ms	0.318
Piper nl medium (VITS, streaming)	2695 ms	247.5 ms	513 ms	0.266
OptiSpeech lightspeech	625 ms	22.2 ms	45 ms	0.011
OptiSpeech convnext	717 ms	23.0 ms	52 ms	0.015

Short label = "Settings", the kind of string a screen reader speaks on every
keystroke. About 10x faster, and 4x faster to load, which matters when the
synthesizer is initialized at screen reader startup.

Worth noting the medium VITS voice is not faster than the high one, so this is
an architecture difference rather than a model size difference.

Control factors

These behave exactly as documented, which is the part Piper has no equivalent for:

p_factor 0.8 -> 1.2 : mean F0 209.7 -> 228.3 Hz
e_factor 0.8 -> 1.3 : RMS 0.118 -> 0.142, duration unchanged
d_factor 0.8 -> 1.3 : duration 2.30 -> 3.50 s

Three independent, predictable knobs.

Question about prosody

The thing I am actually chasing is intonation rather than speed. Measuring F0
contours (autocorrelation, 10 ms hop) on question versus statement:

"Is the file saved?"     start -> end of utterance
  Piper,      no "?" in phonemes:  177.8 -> 143.9 Hz  (-33.8)
  Piper,      with "?":            195.0 -> 143.6 Hz  (-51.4)
  OptiSpeech, no "?" in phonemes:  271.8 -> 175.6 Hz  (-96.2)
  OptiSpeech, with "?":            273.1 -> 215.0 Hz  (-58.1)

Two things stand out. OptiSpeech does respond to the question mark: the final
pitch lands about 39 Hz higher and the fall is roughly halved. Piper does not
move at all (0.3 Hz). So the punctuation token is clearly reaching your duration
and pitch predictors and doing something useful.

But the contour still falls overall, so a question does not read as a question,
just as a less emphatic statement. For a screen reader this matters more than it
might elsewhere: a blind user relies on intonation to tell a question from a
statement without seeing the punctuation.

Hence the question: is there room in the current design to get more expressive
prosody out of the model, or is per-utterance p_factor as far as it goes? I saw
#19 suggesting style diffusion and your reply offering to add it as an optional
component, so I do not want to duplicate that discussion. I am asking something
narrower: given that the pitch predictor already reacts to punctuation, would
something like per-token or per-phrase pitch scaling, or richer punctuation
conditioning, be a reasonable thing to expose? Or is that fighting the
lightweight design goal?

Also, since the README still says final model design decisions are being made:
is the ONNX model format stable enough that voices trained now will keep
working? That determines whether it makes sense for people to start training
voices for languages that have none yet, which for me is Dutch.

Thanks for the model, and for sonata-nvda. The speed here is not marginal, it
is the difference between neural TTS being usable for keystroke echo and not.