Extend model compatibility: Any of this options please #380

Activity

1. chmjkb commented on Jun 9, 2025

   

   chmjkb

   on Jun 9, 2025

   Collaborator

   More actions

   Hey @kuriel-dev, thanks for submitting this issue.

   1. What are the benefits of using the mediapipe format? Are you looking to integrate any specific model that is a part of MediaPipe? Unfortunately, that is very unlikely to happen. Supporting different formats is not really on our end, but on ExecuTorch's end, which we depend on. I believe that would be a big effort for ET, and possibly not much reward. Let me know if there are any MediaPipe models that you would like to see within ExecuTorch, so we can dig into this 😅
   2. Do you mean like a website where you could have a UI where you upload safetensors and you would get the exported model back? Nonetheless, one thing I want to mention here is that to convert a model to ExecuTorch you need to have it's PyTorch code and weights first, so that idea would be hard to implement. For a tool that lets you export models quickly, you can check out Optimum-ExecuTorch
   3. Safetensors is a way to store tensors in a safe and efficient manner. It can't be directly used within the ExecuTorch runtime. When you're exporting a model to ExecuTorch, there's a ton of optimizations happening which are serialized in the .pte file. Safetensors files have none of these optimizations.
   4. We'll look into exporting Gemma 3. I believe it's already supported in Optimum, so it should be relatively simple.
2. msluszniak

   added

   enhancement

   on Jun 10, 2025
3. msluszniak

   added the

   Feature

   issue type on Jun 10, 2025
4. kuriel-dev commented on Jun 11, 2025

   

   kuriel-dev

   on Jun 11, 2025

   Last edited by kuriel-dev

   Author

   More actions

   1. Benefits of using MediaPipe:

   • Most importantly, the .task format is highly optimized for mobile and web platforms. In comparison, .pte models are currently not as efficient or optimized as those using MediaPipe’s .task format.

   • Web compatibility: At the moment, ExecuTorch only supports Android and iOS. MediaPipe can make this library more powerful by adding web support, turning it into a more versatile React Native package for AI.

   • Model availability: You can download many models directly from Kaggle repositories in LiteRT format, ready to use — no conversion tools needed.

   • I ran a test using MediaPipe with React Native, and the final build size difference was massive compared to ExecuTorch. ExecuTorch added over 200MB to the final build, while the MediaPipe test added around ~7MB. I don't fully understand why, since I haven’t reviewed the source code of either AI engine yet.

   2. About the converter tool:

   • The idea is to create a ready-to-use converter — possibly as a Docker container rather than a public website. This idea came from frequent issues with Python environments. PyTorch installations often cause conflicts, especially for developers who don’t work with Python regularly or who already have a specific Python version installed globally. This is particularly relevant for React Native developers.

   3. Regarding .pte models:

   • After some research, it turns out that .pte is not an officially recognized model format, which makes finding compatible models very difficult — sometimes even impossible. This brings us back to point No. 2: relying on .pte adds an extra step that can introduce more complexity and potential conflicts.

   4. OK thanks, ❤️

   I'm not trying to force anything — I just want to do my small part to contribute to this tool, which I've grown fond of and truly believe in.

   That's why I suggest a few alternatives. I understand that sometimes changes can mean extra work, but I believe that expanding compatibility could help this project grow in a positive way.

   There's a lot of effort and professionalism in this library, and like everyone else, I just want to see it thrive.

   ❤️React with ❤️1Reacted by Mateusz Sluszniak
5. chmjkb commented on Jun 11, 2025

   

   chmjkb

   on Jun 11, 2025

   Collaborator

   More actions

   First of all, thanks for your input @kuriel-dev, this is very important for us that the community gives us feedback and feature requests and we try to implement those as quickly as possible.

   MediaPipe
   We haven't done comparative benchmarks against MediaPipe, but that's on our roadmap. In terms of LLMs, AFAIK MediaPipe support just the Gemma models, while we don't support them yet, so the most accurate comparison now would be let's say Gemma 3 1B vs Llama 3.2 1B, and I'm pretty sure that the performance will be either on par or better for ET. One way or another, it is unlikely that MediaPipe format will be supported by ExecuTorch anytime soon and there unfortunately isn't much we can do about it for now :(

   Web compatibility
   We want to make react-native-executorch web compatible, and we are planning to contribute to ExecuTorch to make it possible (it's not possible now, because ExecuTorch itself doesn't have a backend that would run on web). This will most be achievable by compiling the ET runtime with XNNPack backend to WASM.

   Bundle size
   To make some backends (XNNPack, CoreML) work with the ExecuTorch runtime, we have to link some native libraries. We wanted the tool to be universal and effortless, so we're just linking all of them (which is a tradeoff, ofc). This is most likely where the bundle size difference comes from. This is a problem that we'll try to put some more emphasis on and hopefully we can do something to reduce it, good point.

   Converter tool
   For effortless model conversion, I highly recommend checking out optimum-executorch. The tool allows for effortless model exporting from the famous Transformers library via CLI. However that this doesn't fully solve the problem that you're referring to. Here's a Dockerfile you can try to use to create an environment for exporting models. It is no longer maintained, because at some point we decided that close to no one uses it (most people just go with our pre-exported models) and installing ExecuTorch is pretty simple and stable nowadays. However I think most of what's in this Dockerfile should still be usable today.

   PTE files
   I'm not sure what you mean by an officially recognized model format. You can check out this doc. It's a format that the ExecuTorch runtime requires to run the models. Given the fact that this is a rather new format, there's not much pre-exported models compared to other frameworks which have been around for a while. I believe this is a matter of time before getting models for this format will be as simple as visiting Kaggle :). For example, there's this repo and as I mentioned, Optimum is in the works and it will be pretty powerful as well.

   Thank you for your support 🫶🏻
6. software-mansion

   locked and limited conversation to collaborators on Aug 20, 2025
7. chmjkb

   converted this issue into a discussion #513 on Aug 20, 2025