Skip to content

fix(wacli): bridge downloaded images into multimodal input#51

Merged
g1e2x87 merged 1 commit into
mainfrom
fix/wacli-inbound-image
Jul 22, 2026
Merged

fix(wacli): bridge downloaded images into multimodal input#51
g1e2x87 merged 1 commit into
mainfrom
fix/wacli-inbound-image

Conversation

@g1e2x87

@g1e2x87 g1e2x87 commented Jul 22, 2026

Copy link
Copy Markdown
Member

Summary

  • Problem: official wacli emits its message webhook before the asynchronous media downloader records messages.local_path, so OpenPRX forwarded only an [image: ...] placeholder.
  • Why it matters: image-only WhatsApp turns reached the agent without image bytes, causing slow path guessing and incorrect replies even though wacli had downloaded the file.
  • What changed: resolve the matching media path from the configured read-only wacli store, confine it to that store, import it into the workspace-owned media artifact store, and emit the existing multimodal image marker.
  • What did not change (scope boundary): no wacli protocol changes, no outbound-media support, and no audio/video/document ingestion changes.

Label Snapshot (required)

  • Risk label (risk: low|medium|high): risk: medium
  • Size label (size: XS|S|M|L|XL, auto-managed/read-only): expected size: M
  • Scope labels (core|agent|channel|config|cron|daemon|doctor|gateway|health|heartbeat|integration|memory|observability|onboard|provider|runtime|security|service|skillforge|skills|tool|tunnel|docs|dependencies|ci|tests|scripts|dev, comma-separated): channel, docs, tests
  • Module labels (<module>: <component>, for example channel: telegram, provider: kimi, tool: shell): channel: wacli
  • Contributor tier label (trusted contributor|experienced contributor|principal contributor|distinguished contributor, auto-managed/read-only; author merged PRs >=5/10/20/50): auto-managed
  • If any auto-label is incorrect, note requested correction: None

Change Metadata

  • Change type (bug|feature|refactor|docs|security|chore): bug
  • Primary scope (runtime|provider|channel|memory|security|ci|docs|multi): channel

Linked Issue

  • Closes # N/A
  • Related # N/A
  • Depends on # N/A
  • Supersedes # N/A

Supersede Attribution (required when Supersedes # is used)

  • Superseded PRs + authors (#<pr> by @<author>, one per line): N/A
  • Integrated scope by source PR (what was materially carried forward): N/A
  • Co-authored-by trailers added for materially incorporated contributors? (Yes/No): No
  • If No, explain why (for example: inspiration-only, no direct code/design carry-over): no superseded work
  • Trailer format check (separate lines, no escaped \n): Pass

Validation Evidence (required)

Commands and result summary:

cargo fmt --all -- --check
CARGO_TARGET_DIR=/opt/worker/tmp/prx-target TMPDIR=/opt/worker/tmp cargo clippy -p openprx --all-targets -- -D warnings
CARGO_TARGET_DIR=/opt/worker/tmp/prx-target TMPDIR=/opt/worker/tmp cargo test -p openprx --quiet
CARGO_TARGET_DIR=/opt/worker/tmp/prx-target TMPDIR=/opt/worker/tmp cargo test -p openprx channels::wacli::tests --lib --quiet
  • Evidence provided (test/log/trace/screenshot/perf): full package tests pass; full lib result is 5707 passed, 0 failed, 6 ignored; wacli-focused result is 40 passed, 0 failed. Regression tests cover delayed DB publication, store confinement, bounded workspace import, oversized rejection, source-path non-exposure, and provider-side data-URI normalization.
  • If any command is intentionally skipped, explain why: Docker CI was not run; the complete native package suite and all-target Clippy were run.

Security Impact (required)

  • New permissions/capabilities? (Yes/No): No
  • New external network calls? (Yes/No): No
  • Secrets/tokens handling changed? (Yes/No): No
  • File system access scope changed? (Yes/No): Yes
  • If any Yes, describe risk and mitigation: wacli image rows are read from the already configured read-only store. Resolved sources must canonicalize beneath store_dir; files are copied through MediaArtifactOwner::import_channel_file, which applies no-follow reads and the effective multimodal size limit. The source path is not placed in model content or failure logs.

Privacy and Data Hygiene (required)

  • Data-hygiene status (pass|needs-follow-up): pass
  • Redaction/anonymization notes: tests use synthetic numeric JIDs and PRX-native actor labels; no incident payloads or host paths are committed.
  • Neutral wording confirmation (use OpenPRX/project-native labels if identity-like wording is needed): Pass

Compatibility / Migration

  • Backward compatible? (Yes/No): Yes
  • Config/env changes? (Yes/No): No
  • Migration needed? (Yes/No): No
  • If yes, exact upgrade steps: N/A. Image understanding requires the existing store_dir setting and wacli sync to run with --download-media.

Human Verification (required)

What was personally validated beyond CI:

  • Verified scenarios: an official-style webhook resolves a downloaded image, imports it under the workspace media directory, and normalizes it into provider-visible image data.
  • Edge cases checked: webhook-before-download timing, aliases table lookup, outside-store source, oversized source, replay ordering, missing media marker, and group-title regression coverage.
  • What was not verified: a new live WhatsApp image turn; that is the post-merge deployed acceptance step.

Side Effects / Blast Radius (required)

  • Affected subsystems/workflows: inbound wacli image messages only; channel factory wiring and channel documentation.
  • Potential unintended effects: an image webhook can wait up to two seconds for wacli to publish local_path; media that remains unavailable falls back to explicit unavailable metadata.
  • Guardrails/monitoring for early detection: the resolver timeout plus existing two-second forward timeout stay below wacli's five-second webhook deadline; logs report admitted byte size or a redacted rejection category.

Agent Collaboration Notes (recommended)

  • Agent tools used (if any): Codex repository inspection, patching, tests, and local service diagnostics; no sub-agents.
  • Workflow/plan summary (if any): reproduce from current service logs, implement in an isolated worktree, add security/timing tests, run full native gates, then open this PR.
  • Verification focus: deterministic media resolution and maintaining workspace filesystem boundaries.
  • Confirmation: naming + architecture boundaries followed (AGENTS.md + CONTRIBUTING.md): Yes

Rollback Plan (required)

  • Fast rollback command/path: revert this PR and redeploy the previous OpenPRX binary.
  • Feature flags or config toggles (if any): none added; removing store_dir disables local media resolution and preserves placeholder fallback behavior.
  • Observable failure symptoms: media_status=unavailable, media-resolution timeout warnings, or inbound image admission rejection warnings.

Risks and Mitigations

  • Risk: slow or failed wacli downloads may not finish inside the two-second enrichment window.
    • Mitigation: bounded polling prevents webhook starvation; unavailable media is represented explicitly and can be retried in a future protocol-level media_ready enhancement.
  • Risk: a tampered local database could reference arbitrary host files.
    • Mitigation: canonical path confinement to store_dir, no-follow bounded import, and no direct source path exposure.

@g1e2x87
g1e2x87 merged commit 3edc681 into main Jul 22, 2026
17 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant