Skip to content

Subtalker paramater discrepancy between server and cli #26

Description

@esdevver

When the parameters are checked against defaults in tts-server.cpp, it sets subtalker_top_k to top_k, and subtalker_top_p to top_p. Keeping these at default parameters would have parity with the behaviour of the cli tool.

// Sampling overrides ride straight into the ABI; the subtalker
// mirrors the talker knobs so the HTTP surface stays a single
// coherent set. A temperature of zero selects greedy decoding
// on both.
p.seed = req.seed;
if (req.max_new_tokens != -1) {
p.max_new_tokens = req.max_new_tokens;
}
if (req.top_k != -1) {
p.top_k = req.top_k;
p.subtalker_top_k = req.top_k;
}
if (!std::isnan(req.temperature)) {
if (req.temperature == 0.0f) {
p.do_sample = false;
p.subtalker_do_sample = false;
} else {
p.temperature = req.temperature;
p.subtalker_temperature = req.temperature;
}
}
if (!std::isnan(req.top_p)) {
p.top_p = req.top_p;
p.subtalker_top_p = req.top_p;
}
if (!std::isnan(req.repetition_penalty)) {
p.repetition_penalty = req.repetition_penalty;
}

I recommend either removing lines 279 and 287, or properly exposing subtalker parameters (temp, top_k and top_p) to the REST API.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions