feat(sdk): add image input and turn usage - #2487
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
上一阶段已经打通 Query、Session、流式事件、取消和权限响应。外部应用要完成一个最基础的多模态 Agent 调用,还缺少图片上下文和可计量的单轮结果。
本次参考 Codex TypeScript SDK 已验证的最小输入模型:调用方传字符串,或传有序的
text/local_image数组;用量只作为完成结果的事实返回。功能继续走现有 Agent Runtime,没有增加第二套 Runtime、进程协议或图片处理实现。变更
Input/UserInput:支持字符串,以及text、local_image的有序组合;多段文字按顺序合并,图片路径单独传给 Host。AgentInputAttachment;相对路径按 Session workspace 解析。TurnTokenUsage,聚合当前 turn 的多轮模型用量;CLI 复用同一实现,删除原有重复逻辑。Result.usage返回 input/output/total/cached token;可选字段只在所有相关轮次均可确认时返回。0.0.0,本次不发布 npm 或 PyPI。能力边界
兼容性
验证
cargo test -p bitfun-agent-runtime --features agent-runtime dialog_turn --libcargo test -p bitfun-sdk-host -p bitfun-sdk-host-appcargo test -p bitfun-cli json_usage -- --nocapturepnpm --dir sdk/typescript test(65 passed,1 个 Unix-only 用例在 Windows 跳过)pnpm --dir sdk/typescript smoke:consumerpnpm --dir sdk/typescript smoke:nodegit diff --checkcargo check --workspace在 Desktop build script 处因本地缺少预生成的src/mobile-web/dist资源而无法完成;受影响的 Runtime、CLI、SDK Host 和 TypeScript SDK 均已通过直接验证。