Skip to content

fail hermes runs on api errors - #984

Merged
aaronjmars merged 2 commits into
aeonfun:mainfrom
Svector-anu:fix/hermes-api-errors-upstream
Aug 29, 2026
Merged

fail hermes runs on api errors#984
aaronjmars merged 2 commits into
aeonfun:mainfrom
Svector-anu:fix/hermes-api-errors-upstream

Conversation

@Svector-anu

Copy link
Copy Markdown
Contributor

what

make the hermes adapter fail when the cli prints an http 4xx/5xx response while exiting zero. add and register a focused adapter regression test.

why

fork run 33150824247 looked successful at the workflow level, but the main harness output was only HTTP 400: modelCode: does not exist. hermes exited zero, so the adapter wrapped that error text in a success envelope; the scorer then repeated the same error and no health score was recorded.

the exact model source was traced to stale captured config: model.default: tencent/hy3:free. the official live nous catalog updated 2026-08-28 contains tencent/hy3 and no tencent/hy3:free. RH_MODEL_ARG=default was not passed to hermes because the adapter already suppresses that sentinel.

fix

  • detect hermes stdout containing an http 4xx/5xx response
  • emit the real api diagnostic on stderr
  • exit nonzero instead of creating a false success envelope
  • cover both normal output and the zero-exit api-error case

verification

mutation with only the detection removed:

rc=1
hermes HTTP error returned rc=0, want 1

restored:

hermes adapter error tests passed

real fork dispatch 33189422348 now fails at the run step with:

run-harness hermes failed: read-only: workspace write-locked via bwrap hermes API error: HTTP 400: modelCode: does not exist

this pr fixes false-green classification. selecting a current nous model and refreshing HERMES_AUTH remains an operator credential/configuration action; the adapter does not silently rewrite it.

@aaronjmars
aaronjmars merged commit 1924c4f into aeonfun:main Aug 29, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants