Repository navigation
Conversation
Redrob Cowork asks for images through the Redrob gateway, but the key that pays for them belongs to this engine: it lives in the auth store (or REDROB_API_KEY), the desktop app never reads it, and no plugin API hands it out. A Cowork plugin could only reach it by registering a second `redrob` auth hook, which would replace "Connect Redrob" in onboarding, or by fishing it out of an undocumented hook argument. So the tool is a built-in, and the key stays where it is. image_generate posts the prompt to the gateway's /chat/completions with modalities ["image", "text"] (relayed since console #230), writes the returned picture under artifacts/, and returns it as an attachment, which is what puts it inline in the conversation and in front of the model. The file name is chosen before anything is spent and an existing file is never overwritten. Writing is asked as `edit`, like `write`, so plan mode denies it. A gateway refusal (an empty balance, say) and a model that answers in words alone are reported with the gateway's or the model's own message. The registry offers the tool only while a Redrob credential is available, read per request: without one it would be a tool that fails on every call.
flxkm522
added a commit
that referenced
this pull request
Oct 12, 2026
…tored Squash of the redrob-code ENGINE work that was spread across seven open PRs into a single change on develop. The Electron/desktop app and the build-off/Quality-Lab stack move to the alpha/desktop branch separately and are not part of this. Rolled up: - #67 insights route + /v1/chat/completions (serve the OpenAI face for real, forward x-redrob-session to the Console). - #68 regenerate the JS SDK that had fallen behind the routes. - #69-#74 the media tools and endpoints: image_generate (edit / reference images), speech_generate (mp3), POST /redrob/speech and POST /redrob/transcribe, all on the engine own Redrob credential. - #79 the shared on-device labellers (route-labeller, work-labeller) as source workspace packages, consumed downstream by Cowork/Office/Design. Integration fix found by combining the PRs: - packages/server/src/routes.ts: the chat-completion handler reads LLMClient inside the request, so it was a per-request requirement of the embedded API that Layer.provide(serviceLayer) did not discharge, leaking as a residual requirement and flipping the web.handler signature so the untouched sdk-next caller stopped type-checking. LLMClient is a global service, so it is now provided once via HttpRouter.provideRequest inside makeRoutes. Review + lean/documentation pass (behaviour-preserving): - image_generate / speech_generate: size-limit messages derive from their constants instead of hand-kept 10 MB / 50 MB literals (drift-proof). - speech_generate: characters() uses [...text].length (idiomatic code-point count). - core/insights.ts: tightened the ErrorEnvelope describe() union guard (identical runtime behaviour). - plugin/host.ts: removed a misleading comment on skill.list; it matches the uniform mutable() pattern now. - docs/LOCAL-ENGINE-API.md: documented the shipped /redrob/speech and /redrob/transcribe endpoints (sections 1c, 1d). Verified: bun turbo typecheck 20/20; bun install --frozen-lockfile clean; engine tool/server tests 42/42; core 6/6; work-labeller 77/77.
Author
|
Superseded by #80, which consolidates all redrob-code engine PRs into a single squashed PR on |
flxkm522
added a commit
that referenced
this pull request
Oct 12, 2026
…tored Squash of the redrob-code ENGINE work that was spread across seven open PRs into a single change on develop. The Electron/desktop app and the build-off/Quality-Lab stack move to the alpha/desktop branch separately and are not part of this. Rolled up: - #67 insights route + /v1/chat/completions (serve the OpenAI face for real, forward x-redrob-session to the Console). - #68 regenerate the JS SDK that had fallen behind the routes. - #69-#74 the media tools and endpoints: image_generate (edit / reference images), speech_generate (mp3), POST /redrob/speech and POST /redrob/transcribe, all on the engine own Redrob credential. - #79 the shared on-device labellers (route-labeller, work-labeller) as source workspace packages, consumed downstream by Cowork/Office/Design. Integration fix found by combining the PRs: - packages/server/src/routes.ts: the chat-completion handler reads LLMClient inside the request, so it was a per-request requirement of the embedded API that Layer.provide(serviceLayer) did not discharge, leaking as a residual requirement and flipping the web.handler signature so the untouched sdk-next caller stopped type-checking. LLMClient is a global service, so it is now provided once via HttpRouter.provideRequest inside makeRoutes. Review + lean/documentation pass (behaviour-preserving): - image_generate / speech_generate: size-limit messages derive from their constants instead of hand-kept 10 MB / 50 MB literals (drift-proof). - speech_generate: characters() uses [...text].length (idiomatic code-point count). - core/insights.ts: tightened the ErrorEnvelope describe() union guard (identical runtime behaviour). - plugin/host.ts: removed a misleading comment on skill.list; it matches the uniform mutable() pattern now. - docs/LOCAL-ENGINE-API.md: documented the shipped /redrob/speech and /redrob/transcribe endpoints (sections 1c, 1d). Verified: bun turbo typecheck 20/20; bun install --frozen-lockfile clean; engine tool/server tests 42/42; core 6/6; work-labeller 77/77.
flxkm522
added a commit
that referenced
this pull request
Oct 12, 2026
…cs and compat tests One change on develop that both consolidates the redrob-code ENGINE work and documents + pins the on-device labelling it introduces. The Electron/desktop app and the build-off / Quality-Lab stack are the separate alpha/desktop drafts and are NOT part of this. == Engine consolidation (was seven PRs) == - #67 insights route + /v1/chat/completions (serve the OpenAI face for real; forward x-redrob-session to the Console). - #68 regenerate the JS SDK that had fallen behind the routes. - #69-#74 the media tools and endpoints: image_generate (edit / reference images), speech_generate (mp3), POST /redrob/speech and POST /redrob/transcribe, all on the engine own Redrob credential. - #79 the shared on-device labellers (route-labeller, work-labeller) as source workspace packages, consumed downstream by Cowork/Office/Design. Integration fix: the chat-completion handler reads LLMClient inside the request, so it was a per-request requirement of the embedded API that Layer.provide(serviceLayer) did not discharge, leaking as a residual requirement and breaking the untouched sdk-next caller. LLMClient is a global service, so it is now provided once via HttpRouter.provideRequest inside makeRoutes (packages/server/src/routes.ts). Lean/doc pass (behaviour-preserving): size-limit messages derive from their constants; speech_generate characters() uses [...text].length; the insights describe() union guard is tightened; a misleading plugin/host comment removed; /redrob/speech + /redrob/transcribe documented in LOCAL-ENGINE-API.md; and the httpapi route-coverage harness gained scenarios for the two /redrob/* routes. == On-device labelling: docs + compatibility (answering the CTO review) == - docs/ON-DEVICE-LABELLING.md (new): the single reference for WHY there is encoder code (we wrap pinned off-the-shelf models -- distiluse, multilingual-e5-base -- not our own encoders, for on-device privacy + one implementation across products), WHERE the tensors live (only the small trained layers are ours and in git: work-head.json 159KB, prototypes.json 232KB; base weights are hash-pinned downloads), the model-free-CI vs opt-in-eval story, and the backwards-compat model for third-party clients. Includes the pending checklist to move the e5 release asset from redrob-cowork to redrob-code (SHA pins unchanged; the url is not flipped now because a not-yet-uploaded release would 404). - httpapi-chat-completion.test.ts: two new v1-contract cases -- the newer optional fields (max_completion_tokens, reasoning_effort, user) stay accepted, and unknown request fields (a Console-style redrob block, a bogus key) are IGNORED not rejected. The second pins the exact forward/third-party-compat guarantee and fails CI if the schema rejects excess props. - test.yml: the labellers job comment now spells out that the unit suites are model-free and CI-safe, and the real ~140MB/~278MB encoders are needed ONLY by the opt-in eval/model dev commands, which stay off the PR path. Verified: bun turbo typecheck 20/20; bun install --frozen-lockfile clean; engine tool/server tests + the 7 chat-completion contract cases pass; core 6/6; work-labeller 77/77.
its-janghoon
added a commit
that referenced
this pull request
Oct 12, 2026
…tes (#85) image_generate and speech_generate on the engine's Redrob credential, registered only when one is connected. POST /redrob/speech and /redrob/transcribe relay to the Console audio API without the app holding the key; 503 before anything is sent when there is no key, 400 for input the gateway would refuse. From the closed #69, #71, #73, #74; image editing (#70) left out.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Option 1 from the media-output plan: image generation runs inside the engine, using the Redrob credential the engine already holds.
Why a built-in tool, not a Cowork plugin
The Redrob key belongs to Redrob Code. It lives in
auth.jsonorREDROB_API_KEY, Cowork's server deliberately never reads it, and no plugin API hands it out. A Cowork plugin could only reach the key in two bad ways:redrobauth hook.ProviderAuthkeeps one hook per provider, so this would replace "Connect Redrob" in onboarding.provider.keyfrom an undocumentedchat.paramsargument. That's fragile, and it's handling a secret.A built-in avoids both. The key stays where it is, and the image is billed to the same account as the chat.
What it does
image_generate(src/tool/image_generate.ts):POST {CONSOLE_URL}/chat/completionswithmodalities: ["image","text"]. Default model isgoogle/gemini-2.5-flash-image;modelcan be overridden. The gateway relays images since mckinley-and-rice/redrob-console#230.artifacts/<slug>.png(or.jpg/.webp/.gif) and returns it as an attachment (data:URL). Cowork already renders tool attachments inline in chat (session-sync.ts→message-list.tsx), and the model sees what it made.logo.png→logo-2.png).edit, likewrite, so plan mode denies it.filepath,model, and the billedcostUsdfrom the gateway'sredrobblock.REDROB_CONSOLE_URLis read per call, the same developer seamCONSOLE_URLalready honours. That lets the test point at a local stand-in.Proof
bun test test/tool/image_generate.test.tsartifacts/; attachment shape;costUsdin metadata;editpermission on the relative path; no overwrite; named model and file name; no-image error; 402 relayed; no key means no request sent; slug and data-URL guardsbun test test/tool/registry.test.tsbun test(all ofpackages/redrob)tui.test.ts,project.test.tsandwrite.test.ts"OS denies write access"; they also fail ondevelopin this sandbox (it runs as root)bun typecheck(turbo, 18 packages)prettier --check,oxlinton the changed filesNot covered: a live call against
console.redrob.ai. Reaching users needs a release and aredrobCodeVersionbump in Cowork.