Skip to content

fix: make Forecaster runtime device-safe - #246

Open
gyx09212214-prog wants to merge 1 commit into
AI4Finance-Foundation:masterfrom
gyx09212214-prog:codex/forecaster-runtime-guards
Open

fix: make Forecaster runtime device-safe#246
gyx09212214-prog wants to merge 1 commit into
AI4Finance-Foundation:masterfrom
gyx09212214-prog:codex/forecaster-runtime-guards

Conversation

@gyx09212214-prog

@gyx09212214-prog gyx09212214-prog commented Jun 12, 2026

Copy link
Copy Markdown

Summary

  • choose float16 only when CUDA is available and default to float32 on CPU-only runtimes
  • make GPU memory reporting tolerant of missing CUDA/NVML instead of failing the app import path
  • add FINGPT_TORCH_DTYPE override support with explicit validation, plus unit coverage for dtype resolution, NVML handling, and CUDA cache cleanup

Why this matters

The Forecaster demo currently assumes CUDA fp16/NVML availability. That can break local CPU-only or Windows smoke tests before users reach the actual forecasting flow. Keeping runtime hardware checks best-effort, while failing fast on invalid explicit dtype overrides, makes the demo easier to validate without changing the model path for CUDA users.

Scope

This addresses the NVML and CPU fp16 runtime failure patterns reported in #119 without changing model loading, prompts, market data access, or generated forecasting behavior.

Validation

  • python -m pytest fingpt\FinGPT_Forecaster\tests\test_runtime_utils.py -q - 8 passed
  • git diff --check - clean
  • Full Gradio/model inference was not run because it requires Hugging Face credentials, the Llama-2 base model, LoRA weights, and market data API keys.

@gyx09212214-prog
gyx09212214-prog force-pushed the codex/forecaster-runtime-guards branch from 7705882 to dc0627a Compare June 15, 2026 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant