Skip to content

feat: added translateImage API function; image in, original and translated text blocks out. - #210

Open
Tallis-Larsen wants to merge 4 commits into
DavidVentura:masterfrom
Tallis-Larsen:expose-ocr
Open

feat: added translateImage API function; image in, original and translated text blocks out.#210
Tallis-Larsen wants to merge 4 commits into
DavidVentura:masterfrom
Tallis-Larsen:expose-ocr

Conversation

@Tallis-Larsen

@Tallis-Larsen Tallis-Larsen commented Jun 11, 2026

Copy link
Copy Markdown

I'm making an (open source) app that requires some basic OCR and translation functionality, but I didn't want to re-implement what has already been done. Therefore, I decided that simply requiring offline-translator as a dependency and offloading all the model management and configurations to it would work to significantly reduce the need to repeat code. This PR simply exposes the translateImageWithOverlay function, allowing any app to get raw OCR and translation data from an image, and handle rendering (if desired) themselves. I think that this would be a significant help to those who wish to implement OCR and translation features into their apps, but do not want the overhead of managing the necessary models (such as myself). Additionally, this change does not modify any existing logic, so it should be quite easy to review and merge. I'm rather new to making PRs to projects other than my own, so please go easy on me :P

@DavidVentura DavidVentura left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you expand on what you want to achieve with your app? I don't really understand the current API design, but maybe I'm missing what you want to do.

Comment thread app/src/main/aidl/dev/davidv/translator/TextBlockResult.aidl Outdated
Comment thread app/src/main/java/dev/davidv/translator/AidlTranslationService.kt
Comment thread README.md Outdated
Removed the service binding instruction for the translation API.
@Tallis-Larsen

Copy link
Copy Markdown
Author

can you expand on what you want to achieve with your app? I don't really understand the current API design, but maybe I'm missing what you want to do.

I'm making a basic alternative to google's circle to search, and among other features, I want to have a translate button in the corner that reads the screen's text via OCR, translates it automatically with the user's preferred language and model, and shows an overlay. I also want the user to be able to search for and copy the text, so the raw data is needed, not just the overlay. Your offline-translator app just so happens to do almost all of this already, it's just not exposed for other apps to use, which is the purpose of this PR. I hope that clears things up a little more. My app's repo isn't public yet, but when it is, it will be here: https://github.com/Tallis-Larsen/square-to-find

@Tallis-Larsen

Copy link
Copy Markdown
Author

New commits remove TextBlock entirely, and now all the caller gets is a list of TextLine and the full translated+untranslated text. Should I also get rid of the full text strings, because they can be derived from the text lines?

@DavidVentura

Copy link
Copy Markdown
Owner

I'm doing some changes with the way words are detected and how they carry style, when that's done I can come back to this, because the API that communicates the structured data is bound to change

@Tallis-Larsen

Copy link
Copy Markdown
Author

Okay no problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants