Skip to content

chore: migrate rust/face-recognition to icp-cli#1421

Open
marc0olo wants to merge 31 commits into
masterfrom
chore/migrate-rust-face-recognition-to-icp-cli
Open

chore: migrate rust/face-recognition to icp-cli#1421
marc0olo wants to merge 31 commits into
masterfrom
chore/migrate-rust-face-recognition-to-icp-cli

Conversation

@marc0olo

@marc0olo marc0olo commented Jun 16, 2026

Copy link
Copy Markdown
Member

Summary

  • Replace dfx.json with icp.yaml; custom build pipeline: cargo build --target wasm32-wasip1wasi2ic (rewrites WASI syscalls to IC system API) → wasm-opt (size + SIMD optimisation) → ic-wasm (embed Candid metadata); auto-installs wasi2ic and wasm-opt if missing
  • Add sync step: upload-models-to-canister.sh runs after each deploy — downloads face detection model, auto-generates face recognition model (if Python 3.9–3.12 available), uploads both to the canister, skips if already loaded
  • Restructure source: src/backend/backend/; update workspace Cargo.toml
  • Upgrade ic-cdk 0.140.20, ic-stable-structures 0.60.7, ic-wasi-polyfill 0.4.10.13; remove unused colorgrad; update edition to "2024"; remove channel = "1.79" and channel = "1.79" pin from rust-toolchain.toml; rename wasm32-wasiwasm32-wasip1 throughout
  • Add models_ready() -> bool query so the frontend can detect unloaded models on page load and show an actionable setup message
  • Fix detect() and embedding() in onnx.rs: replace .unwrap() with .ok_or_else() so callers receive a descriptive Err instead of a canister trap when models are not loaded
  • Add post_upgrade model reload: reads model files from stable memory (WASI filesystem) with a metadata guard so upgrades don't require re-uploading 100MB+ of model data
  • Replace Makefile + run.sh with test.sh; Test 4 is context-aware (models_ready() check) so it passes in both CI (no models) and locally (models loaded)
  • Add CI workflow face-recognition.yml (icp-dev-env-rust:1.0.1, installs build-essential for wasi2ic/wasm-opt C++ deps); remove legacy rust-face-recognition-example.yaml
  • Frontend: migrate webpack + dfx declarations to Vite + @icp-sdk/bindgen; align vite.config.js and actor.js with canonical icp-cli pattern (no dfx fallbacks); bump @icp-sdk/core 5.0→5.4, @icp-sdk/bindgen 0.2.2→0.4.0, vite 5.4.11→8.1.3; remove "Onchain" jargon from title/heading
  • download-face-detection-model.sh: fix target path; force IPv4 (--inet4-only/-4)
  • download-face-detection-model.sh: carry over backend/assets/image.png (embedded test image for benchmarking endpoints); update .gitignore to ignore *.onnx at project root
  • Rewrite README.md: remove all "smart contract" jargon; add optional Python 3.9–3.12 prerequisite; document 3 deploy phases; explain model persistence vs persons database behaviour; add benchmarking endpoint usage with icp canister logs
  • Update ic-file-uploader link to current repo (github.com/decide-ai/ic-file-uploader)
  • Delete dfx.json, build.sh, webpack.config.js, run.sh, old src/ tree, old declarations, legacy CI

Test plan

  • icp network start -d && icp deploy && bash test.sh passes — models downloaded/generated/uploaded automatically, all 4 tests green
  • icp canister logs backend is empty after fresh deploy (no panics)
  • Redeployment (icp deploy again) skips model upload (models_ready() returns true)
  • Frontend shows setup warning when models not loaded; shows detection UI once loaded
  • CI workflow face-recognition passes on push

🤖 Generated with Claude Code

marc0olo and others added 2 commits June 19, 2026 08:32
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… 1.0.1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@marc0olo marc0olo force-pushed the chore/migrate-rust-face-recognition-to-icp-cli branch from 55aa0da to 2f7a5b0 Compare June 19, 2026 06:33
marc0olo and others added 24 commits July 6, 2026 10:52
…th canonical pattern

- rust-toolchain.toml: remove channel = "1.79" pin (Cargo 1.79 can't parse edition 2024
  in transitive deps; removing pin uses system stable Rust 1.85+)
- icp.yaml: replace cargo binstall with "which || cargo install" (cargo-binstall is not
  installed by default); remove redundant network definition
- backend/Cargo.toml: ic-cdk 0.14→0.20, ic-stable-structures 0.6→0.7,
  ic-wasi-polyfill 0.4.1→0.13; edition 2021→2024; add cdylib comment
- download-face-detection-model.sh: force IPv4 to avoid hanging IPv6 connections
- frontend: align vite.config.js and actor.js with canonical icp-cli pattern (no dfx
  fallbacks); bump @icp-sdk/core 5.0→5.4, @icp-sdk/bindgen 0.2.2→0.4.0, vite 5.4.11→8.1.3
- CI: add apt-get install build-essential (required by wasi2ic's C++ dependency)
- README: add Node.js v18+, Rust v1.85+, wasi2ic prerequisites; replace make upload-models
  with bash upload-models-to-canister.sh

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
backend/src/benchmarking.rs embeds image.png via include_bytes! at compile time;
the file was not carried over during the src/backend/ → backend/ restructure.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TARGET_DIR set in one icp.yaml command line is not visible in the next since
each command runs in its own shell. Use ./target/wasm32-wasip1/release/ directly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ls not loaded

Replace unwrap() with ok_or_else() in detect() and embedding() so callers get
a descriptive Err when setup_models() hasn't been called yet.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…fore use

The previous structure put "Upload the ONNX models" after "Deploy and test"
which implied the frontend was usable after deploy. Add an explicit note that
the frontend won't work until models are uploaded and setup_models is called.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ing in UI

- Add models_ready() -> bool query to backend/onnx.rs; exposed via lib.rs
- On page load, frontend calls models_ready() and shows an actionable warning
  banner if models haven't been uploaded yet, rather than failing silently
  after the user clicks a button
- Update ic-file-uploader link to current repo: github.com/decide-ai/ic-file-uploader

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…e update instructions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add sync step to icp.yaml that runs upload-models-to-canister.sh after deploy
- upload-models-to-canister.sh now:
  - exits 0 gracefully if model files are missing (CI-safe)
  - skips upload if models_ready() returns true (idempotent on redeployment)
  - re-uploads if models were cleared (e.g. after reinstall)
- README: remove the separate "Upload models" manual step — icp deploy handles it

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…result

- Print models_ready result and upload progress so sync phase output is clear
- Check setup_models() return value for Err variant (icp canister call exits 0
  even when the canister returns Err, so set -e alone doesn't catch failures)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…c step

Remove explicit CI check — the script is naturally CI-safe:
- face detection model is auto-downloaded (~1MB, cached on re-runs)
- face-recognition.onnx absent in CI → exits 0 with clear message
- CI tests pass since they work without loaded models

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
If python3 is available, auto-install facenet-pytorch/torch/onnx and generate
face-recognition.onnx. If python3 is absent (e.g. minimal CI container), skip
gracefully. README updated to reflect that icp deploy handles everything.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Python 3.14 can't build Pillow from source (facenet-pytorch dep);
--prefer-binary downloads pre-built wheels instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…on, UI alignment

- post_upgrade: reload models from stable memory automatically so upgrades don't
  require re-uploading 100MB+ of model data
- upload-models-to-canister.sh: try setup_models() before re-uploading (fast path
  for upgrades); get replica URL from icp network status; pass canister ID to
  ic-file-uploader (no dfx.json needed); detect Python 3.9-3.12 for torch compat;
  use certifi for SSL on macOS python.org installs; auto-download face detection
  model; auto-generate face recognition model if compatible Python available
- .gitignore: replace stale backend/assets/*.onnx entries with *.onnx (models live
  in project root and should never be committed)
- vite.config.js: align exactly with canonical hello_world pattern (const server)
- onnx.rs: add comment that DB is heap memory, cleared on upgrade
- README: remove "smart contract" jargon; add optional Python 3.9-3.12 prerequisite;
  document the 3 deploy phases; explain model persistence vs persons database
- test.sh: make Test 4 context-aware — asserts Ok when models are loaded (local)
  and Err when they are not (CI)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…le and heading

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rove module comment

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…data check

storage::bytes() calls unwrap() and panics on missing files; check both model
files exist before calling setup() to avoid trapping the canister on upgrade
when models haven't been uploaded yet.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…un_recognition

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…struction count

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ection only

run_recognition returns Err when no persons are in the DB which looks broken;
icp canister logs mixes panic traces from old deployments with instruction counts.
Only show run_detection which gives a clean, meaningful output.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…CLI examples

These are development tools, not user-facing features. No need to show CLI commands
that produce confusing output (Err on empty DB, noisy logs from old deployments).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ad script

Calling setup_models() before uploading panics on a fresh deploy because
storage::bytes() calls unwrap() on a non-existent file. The fast path is now
redundant — post_upgrade handles the upgrade case automatically (models_ready()
returns true before the sync step runs), so the models_ready() check is sufficient.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
marc0olo and others added 2 commits July 6, 2026 13:37
… logs for benchmarking

Logs are only captured for update calls; query call logs are not persisted on ICP.
run_recognition (update) logs the instruction count; run_detection (query) does not.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… comments

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates rust/face-recognition from dfx to icp-cli, restructuring the example to the canonical backend/ + frontend/ layout, updating the Rust canister/toolchain, and switching the frontend build from webpack to Vite with @icp-sdk/bindgen.

Changes:

  • Replaced dfx.json/Makefile-era flows with icp.yaml build+sync steps, plus new test.sh and model upload/download automation.
  • Updated Rust backend dependencies/toolchain (incl. ic-cdk to 0.20, WASI target rename) and added models_ready() for the frontend.
  • Migrated frontend to Vite + bindgen and removed legacy webpack + generated declarations.

Reviewed changes

Copilot reviewed 30 out of 42 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
rust/face-recognition/webpack.config.js Removed legacy webpack config (Vite migration).
rust/face-recognition/upload-models-to-canister.sh Reworked model download/generate/upload logic for icp-cli sync step.
rust/face-recognition/test.sh Added icp-cli based smoke tests.
rust/face-recognition/src/frontend/src/.ic-assets.json5 Removed legacy asset headers config (old layout).
rust/face-recognition/src/frontend/assets/.ic-assets.json5 Removed legacy asset headers config (old layout).
rust/face-recognition/src/declarations/backend/index.js Removed dfx-generated JS declarations.
rust/face-recognition/src/declarations/backend/index.d.ts Removed dfx-generated TS declarations.
rust/face-recognition/src/declarations/backend/backend.did.js Removed dfx-generated did JS.
rust/face-recognition/src/declarations/backend/backend.did.d.ts Removed dfx-generated did TS.
rust/face-recognition/src/declarations/backend/backend.did Removed dfx-generated did file (old location).
rust/face-recognition/rust-toolchain.toml Updated target list to wasm32-wasip1 and removed channel pin.
rust/face-recognition/README.md Rewritten docs for icp-cli deploy phases and model persistence behavior.
rust/face-recognition/package.json Simplified to workspace root delegating to frontend/.
rust/face-recognition/icp.yaml Added icp-cli canister definitions with custom build pipeline + sync script.
rust/face-recognition/frontend/vite.config.js Added Vite config + bindgen + dev-server cookie/proxy wiring.
rust/face-recognition/frontend/src/index.js Updated frontend entry to use new actor + models-ready banner.
rust/face-recognition/frontend/src/actor.js Added canonical cookie-based actor wiring (no dfx fallback).
rust/face-recognition/frontend/package.json Added frontend package with Vite + SDK deps.
rust/face-recognition/frontend/main.css Added/updated styling for new UI assets.
rust/face-recognition/frontend/logo_transparent.png Added UI image asset.
rust/face-recognition/frontend/logo_small.png Added UI image asset.
rust/face-recognition/frontend/loader.svg Added loader asset.
rust/face-recognition/frontend/index.html Updated page title/heading and loads src/index.js as module.
rust/face-recognition/frontend/favicon.ico Added favicon asset.
rust/face-recognition/download-face-detection-model.sh Adjusted download flags (force IPv4) and target path behavior.
rust/face-recognition/dfx.json Removed dfx project file.
rust/face-recognition/Cargo.toml Updated workspace member path to backend/.
rust/face-recognition/build.sh Removed legacy build script (replaced by icp.yaml build steps).
rust/face-recognition/backend/src/storage.rs Added WASI filesystem helper functions for model files.
rust/face-recognition/backend/src/onnx.rs Added models_ready() + replaced model unwrap() with descriptive errors.
rust/face-recognition/backend/src/lib.rs Added canister models_ready() endpoint + post-upgrade model reload + candid export.
rust/face-recognition/backend/src/benchmarking.rs Improved benchmarking endpoint docs and formatting helper comment.
rust/face-recognition/backend/Cargo.toml Upgraded deps, removed unused dep, bumped edition, updated versions.
rust/face-recognition/backend/backend.did Added models_ready() to the Candid interface.
rust/face-recognition/.gitignore Updated ignores for ONNX and bindgen output; removed dfx artifacts.
rust/face-recognition/.cargo/config.toml Removed legacy per-target config (moved to icp.yaml build flags).
.github/workflows/rust-face-recognition-example.yaml Removed legacy dfx-based CI workflow.
.github/workflows/face-recognition.yml Added new icp-cli-based CI workflow using icp-dev-env-rust:1.0.1.
Comments suppressed due to low confidence (2)

rust/face-recognition/backend/backend.did:50

  • setup_models returns Result<(), String> in Rust, but backend/backend.did declares setup_models": () -> (). Since you embed this .did into WASM metadata and also use it for frontend bindgen, the interface will be inconsistent and can break callers. Either change the Rust signature to return () or (recommended) update the .did to match the Result (e.g. variant { Ok: null; Err: Error } / variant { Ok: null; Err: text }).
    rust/face-recognition/backend/src/lib.rs:138
  • setup_models currently calls storage::bytes(...) which does std::fs::read(...).unwrap(). If the model files haven't been uploaded yet, this will trap the canister instead of returning a descriptive error. Since setup_models already returns a Result, consider guarding for missing files and returning an Err (and also align the error type with the existing Error { message } candid record for consistency).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread rust/face-recognition/upload-models-to-canister.sh Outdated
…panic in setup_models

- upload-models-to-canister.sh: remove ssl.create_default_context override (caused
  infinite recursion); use SSL_CERT_FILE + REQUESTS_CA_BUNDLE env vars instead
- backend.did: setup_models returns variant { Ok: null; Err: text } not ()
- lib.rs: guard setup_models() with metadata check so missing files return Err
  instead of trapping the canister via storage::bytes().unwrap()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 42 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

rust/face-recognition/backend/src/lib.rs:168

  • post_upgrade currently ignores the result of setup(...). If model reload fails (corrupt/partial files), the canister silently continues with models unloaded, making debugging difficult and potentially causing confusing runtime errors. Logging the error (or trapping explicitly) would make failures visible in canister logs.

Comment thread rust/face-recognition/upload-models-to-canister.sh
…RL; log post_upgrade errors

- REPLICA_URL: try jq → python3 → fallback to localhost:8000 so uploads work
  without Python (e.g. when face-recognition.onnx is provided manually)
- post_upgrade: log setup errors via ic_cdk::println! so failures appear in
  icp canister logs backend instead of being silently swallowed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 42 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

rust/face-recognition/backend/Cargo.toml:5

  • edition = "2024" deviates from the Rust canister examples in this repo (which consistently use edition 2021, e.g. rust/hello_world/backend/Cargo.toml:4). Using 2024 also increases the minimum required Rust toolchain and can break builds in the pinned dev/CI container if it doesn't ship a new-enough rustc. Consider sticking to edition 2021 unless there is a concrete 2024-only requirement.

Comment thread rust/face-recognition/upload-models-to-canister.sh Outdated
… not just <=12)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@marc0olo marc0olo marked this pull request as ready for review July 6, 2026 12:14
@marc0olo marc0olo requested review from a team as code owners July 6, 2026 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants