model_benchmark (C++) only benchmarks text-only decoder models via OgaTokenizer + AppendTokenSequences. There is no C++ equivalent of benchmark/python/benchmark_multimodal.py for vision-language and other multimodal models.
This gap matters for environments where Python is unavailable or undesirable (Android, embedded, CI perf gates, Foundry Local–style deployments) but multimodal models still need end-to-end perf measurement (vision prefill + text decode).
Any plan to support this feature? Thanks!
model_benchmark (C++) only benchmarks text-only decoder models via OgaTokenizer + AppendTokenSequences. There is no C++ equivalent of benchmark/python/benchmark_multimodal.py for vision-language and other multimodal models.
This gap matters for environments where Python is unavailable or undesirable (Android, embedded, CI perf gates, Foundry Local–style deployments) but multimodal models still need end-to-end perf measurement (vision prefill + text decode).
Any plan to support this feature? Thanks!