Skip to content

fix(vllm): filter None kwargs before passing to SamplingParams#21375

Open
andreafresa wants to merge 1 commit intorun-llama:mainfrom
andreafresa:fix/vllm-filter-none-sampling-params
Open

fix(vllm): filter None kwargs before passing to SamplingParams#21375
andreafresa wants to merge 1 commit intorun-llama:mainfrom
andreafresa:fix/vllm-filter-none-sampling-params

Conversation

@andreafresa
Copy link
Copy Markdown

@andreafresa andreafresa commented Apr 13, 2026

Summary

Fixes #21371

llama-index-llms-vllm hardcodes best_of in _model_kwargs, but vLLM >= 0.19.0 removed best_of from SamplingParams. Since _model_kwargs unconditionally included it, every .complete() call raised:

TypeError: Unexpected keyword argument 'best_of'

Root causebase.py line 257:

return {**base_kwargs}  # passes all keys including best_of=None

Fix — filter out None-valued keys before returning:

return {k: v for k, v in base_kwargs.items() if v is not None}

This way any parameter whose default is None (including best_of) is simply omitted, making the integration robust to future vLLM API removals without breaking explicitly set values.

Test plan

  • Vllm.complete() no longer raises TypeError with vLLM >= 0.19.0
  • Parameters explicitly set by the user (e.g. temperature=0.7) are still forwarded correctly
  • best_of is still forwarded when explicitly set to a non-None value (for older vLLM versions)

vLLM >= 0.19.0 removed `best_of` from SamplingParams. Since `_model_kwargs`
unconditionally included it, every `.complete()` call raised:

    TypeError: Unexpected keyword argument 'best_of'

Filter out None-valued keys so removed or unset parameters are never
forwarded to SamplingParams, making the integration robust to future
vLLM API changes.

Fixes run-llama#21371
@dosubot dosubot bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Apr 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: llama-index-llms-vllm: best_of removed from vLLM SamplingParams causes TypeError on .complete()

1 participant