Improve geothermal qualitative extraction + fix LiteLLM runtime bug - #521
Conversation
Restore geothermal schema/plugin to the latest tuned recall state (v2.2.2 + collection prompt tuning), and add adaptive OpenAI service retry behavior that removes provider-rejected call kwargs (e.g., temperature) and retries safely. Also adds runtime-focused tests for unsupported-kwarg recovery and non-recoverable error passthrough. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #521 +/- ##
==========================================
+ Coverage 63.78% 63.85% +0.06%
==========================================
Files 78 78
Lines 7445 7445
Branches 755 755
==========================================
+ Hits 4749 4754 +5
+ Misses 2536 2531 -5
Partials 160 160
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR improves COMPASS’ geothermal electricity one-shot extraction configuration (schema + text-collection prompting) and hardens the OpenAI/LiteLLM call path to dynamically retry after provider-specific rejections of unsupported top-level kwargs.
Changes:
- Added adaptive handling in
OpenAIServiceto detect provider “unsupported parameter” styleBadRequestErrors, drop the offending kwarg, and retry. - Updated geothermal extraction configuration: replaced boolean
collection_promptswith an explicit prompt chain and tuned the geothermal qualitative schema (including removing out-of-scope district features). - Added unit tests covering the unsupported-kwarg retry behavior and kwarg detection logic.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
compass/services/openai.py |
Adds unsupported-kwarg detection and retry logic for resilient OpenAI/LiteLLM runtime behavior. |
tests/python/unit/services/test_services_openai.py |
Adds targeted tests for retrying without unsupported kwargs and parsing unsupported params from error text. |
compass/extraction/geothermal_electricity/plugin_config.yaml |
Switches to explicit collection prompt(s) for geothermal text retention decisions. |
compass/extraction/geothermal_electricity/geothermal_schema.json |
Updates geothermal qualitative schema definitions and removes extra district features to match intended scope. |
Address PR review feedback: correct the module docstring typo and bound the number of unsupported top-level kwarg drops per request so provider rejections cannot trigger unbounded paid retries.
ppinchuk
left a comment
There was a problem hiding this comment.
Hey Bryon, thank you for this PR.
I am reverting the LiteLLM changes for now because I am not convinced this is the best approach to solve the problem you were experiencing. Let's touch base when you are back to discuss the specific errors you were seeing and what would be the best approach to resolve them.
Summary
This PR finalizes the geothermal qualitative extraction improvements and hardens LiteLLM/OpenAI runtime behavior for provider-specific unsupported kwargs.
Geothermal extraction updates
v2.2.2plugin_config.yamlLiteLLM runtime hardening
compass/services/openai.pyfor recoverable provider rejections of top-level kwargstemperature) and retry the same callTests
Validation
pixi run -e pdev pytest -q tests/python/unit/services/test_services_openai.py -rapP