Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,15 @@ jobs:
name: "x86_64"
runs-on: ubuntu-latest
container:
image: bilelmoussaoui/flatpak-github-actions:gnome-nightly
# Keep in step with runtime-version in the manifest
image: ghcr.io/flathub-infra/flatpak-github-actions:gnome-50
options: --privileged
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- uses: flatpak/flatpak-github-actions/flatpak-builder@master
- uses: flatpak/flatpak-github-actions/flatpak-builder@v6
with:
bundle: livecaptions.flatpak
manifest-path: build-aux/flatpak/net.sapples.LiveCaptions.json
cache-key: flatpak-builder-${{ github.sha }}
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,5 @@ build-dir
.flatpak-builder
result
aprilv0_en-us.april
april-english-dev-01110_en.april
april-english-dev-01110_en.april.part
83 changes: 41 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,74 +52,73 @@ If you forgot, you can initialize submodules like so:
$ git submodule update --init --recursive
```

## Option 1: Building with GNOME Builder (easy)
You can build this easily with GNOME Builder. After cloning, open the project directory in GNOME Builder, download the SDK if it asks you, and click the play button to build and run.
## Option 1: Building from the terminal (Linux)

If you are using Flatpak GNOME Builder and experience issues running this (for example, some cryptic X Window System error), please try using your distro's native packaged version of GNOME Builder instead of Flatpak (e.g. `sudo apt install gnome-builder`).

## Option 2: Building from the terminal (not as easy)
First you must [download ONNXRuntime v1.14.1 (Linux)](https://github.com/microsoft/onnxruntime/releases/download/v1.14.1/onnxruntime-linux-x64-1.14.1.tgz) or [ONNXRuntime v.1.14.1 (Mac OS x86)](https://github.com/microsoft/onnxruntime/releases/download/v1.14.1/onnxruntime-osx-x86_64-1.14.1.tgz), extract it somewhere, and set the environment variables to point to it.

### Make ONNX available
### 1. Install the dependencies
You need GTK 4.12 or newer and libadwaita 1.5 or newer. Ubuntu 22.04 and other older releases are too old; use Nix (Option 2) there.

Linux:
Arch Linux:
```
$ export ONNX_ROOT=/path/to/onnxruntime-linux-x64-1.14.1/
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/onnxruntime-linux-x64-1.14.1/lib
$ sudo pacman -S --needed base-devel git meson ninja cmake pkgconf gtk4 libadwaita libpulse libx11 gettext glib2 desktop-file-utils curl pipewire-pulse
```

Alternatively you should also be able to locally build and install ONNXRuntime, in which case that step shouldn't be necessary.


Mac OS:
Ubuntu 24.04 or newer, Debian 13 or newer:
```
$ export ONNX_ROOT=/path/to/onnxruntime-osx-x86_64-1.14.1/
$ cp /path/to/onnxruntime-osx-x86_64-1.14.1/lib/libonnxruntime.1.14.1.dylib /usr/local/lib/libonnxruntime.1.14.1.dylib
$ cp /path/to/onnxruntime-osx-x86_64-1.14.1/lib/libonnxruntime.dylib /usr/local/lib/libonnxruntime.dylib
$ sudo apt install build-essential git meson ninja-build cmake pkg-config libgtk-4-dev libadwaita-1-dev libpulse-dev libx11-dev gettext libglib2.0-bin desktop-file-utils curl
```

### Other dependencies
You will also need the following prerequisites:
Fedora:
```
pulseaudio
libadwaita
meson
ninja
$ sudo dnf install gcc gcc-c++ git meson ninja-build cmake pkgconf-pkg-config gtk4-devel libadwaita-devel pulseaudio-libs-devel libX11-devel gettext glib2-devel desktop-file-utils curl
```

If you're on macOS, you may have to run the following command for Pulseaudio to be registered as a background service:
Live Captions records audio through the PulseAudio interface. On PipeWire desktops this is provided by `pipewire-pulse`, which most distributions install by default.

### 2. Build
```
brew services restart pulseaudio
$ ./setup.sh
```

and to stop it, just change `restart` for stop.
This builds into `builddir/` and downloads the speech model (about 320 MB). If your distribution packages ONNX Runtime, the system copy is used; otherwise the official ONNX Runtime SDK is downloaded into `subprojects/april-asr/lib`. To use another SDK, set `ONNX_ROOT` to its directory before running `setup.sh`. Run `./setup.sh` again after `git pull`.


### Set up the build

To set up a build, run these commands:
### 3. Run
```
$ meson setup builddir --buildtype release
$ meson devenv -C builddir
$ ./run.sh
```

On first launch, a short benchmark checks that your computer can caption in real time. It needs a result of at least 1.2.

### Build and run
Recognition uses 2 CPU threads by default. To try another number, set `APRIL_ENCODER_THREADS`, for example `APRIL_ENCODER_THREADS=4 ./run.sh`, then re-run the benchmark from the settings window to compare. To use a different model, set `APRIL_MODEL_PATH` to its file.

Now you can build the application. Run the following:
## Option 2: Building with Nix
The flake pins a nixpkgs revision that provides GTK 4.22 and libadwaita 1.9, and downloads the default model. It builds the `april-asr` submodule from the checkout, which requires Nix 2.27 or newer:
```
$ ninja
$ nix build
$ ./result/bin/livecaptions
```

Before being able to run the app, you must also download the model and export `APRIL_MODEL_PATH` to where the model is. For example:
## Option 3: Building a Flatpak
With `flatpak-builder` installed and the Flathub remote added:
```
$ wget https://april.sapples.net/april-english-dev-01110_en.april
$ export APRIL_MODEL_PATH=`pwd`/april-english-dev-01110_en.april
$ flatpak install --user flathub org.gnome.Sdk//50 org.gnome.Platform//50
$ flatpak-builder --user --install --force-clean build-dir build-aux/flatpak/net.sapples.LiveCaptions.json
$ flatpak run net.sapples.LiveCaptions
```

You should now be able to run the app with:
The manifest builds this checkout, and bundles ONNX Runtime and the default model.

## Option 4: Building with GNOME Builder
You can build this with GNOME Builder. After cloning, open the project directory in GNOME Builder, download the SDK if it asks you, and click the play button to build and run.

If you are using Flatpak GNOME Builder and experience issues running this (for example, some cryptic X Window System error), please try using your distro's native packaged version of GNOME Builder instead of Flatpak (e.g. `sudo apt install gnome-builder`).

## macOS
`setup.sh` downloads ONNX Runtime for Linux only. Download [ONNX Runtime for macOS](https://github.com/microsoft/onnxruntime/releases/tag/v1.30.0), extract it, and point `ONNX_ROOT` at it. Install the other dependencies with Homebrew (`gtk4 libadwaita pulseaudio meson ninja cmake pkg-config`) and register PulseAudio as a background service with `brew services restart pulseaudio`. Then:
```
$ src/livecaptions
$ export ONNX_ROOT=/path/to/onnxruntime-osx-arm64-1.30.0
$ cp $ONNX_ROOT/lib/libonnxruntime*.dylib /usr/local/lib/
$ ./setup.sh
$ ./run.sh
```

If you're on MacOS, now, go to security settings and confirm the prompt which asks to allow `libonnxruntime`, then, the application should work as intended.
The first time, confirm the security prompt that asks to allow `libonnxruntime`.
55 changes: 29 additions & 26 deletions build-aux/flatpak/net.sapples.LiveCaptions.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"app-id" : "net.sapples.LiveCaptions",
"id" : "net.sapples.LiveCaptions",
"runtime" : "org.gnome.Platform",
"runtime-version" : "master",
"runtime-version" : "50",
"sdk" : "org.gnome.Sdk",
"command" : "livecaptions",
"finish-args" : [
Expand All @@ -19,15 +19,13 @@
"/share/gtk-doc",
"/share/man",
"/share/pkgconfig",
"/onnx_root",
"/onnxruntime/include",
"/onnxruntime/lib/cmake",
"/onnxruntime/lib/pkgconfig",
"/lib/libaprilasr.so*",
"*.la",
"*.a"
],
"build-options" : {
"env" : {
"ONNX_ROOT" : "/app/onnx_root"
}
},
"modules" : [
{
"name" : "onnxruntime",
Expand All @@ -36,30 +34,23 @@
{
"type" : "archive",
"only-arches" : ["x86_64"],
"dest" : "onnxruntime-1",
"url" : "https://github.com/microsoft/onnxruntime/releases/download/v1.14.1/onnxruntime-linux-x64-1.14.1.tgz",
"sha256" : "010585f534d822bf02d54c7cfc53b57aa3eb0cf34f8fb7503ee519ed18e25d29"
"url" : "https://github.com/microsoft/onnxruntime/releases/download/v1.30.0/onnxruntime-linux-x64-1.30.0.tgz",
"sha256" : "a5ed5a3cac51fbb2e90da632ae43d19212faaa20e76484e62bcb7c23ddb3b3fd"
},

{
"type" : "archive",
"only-arches" : ["aarch64"],
"dest" : "onnxruntime-1",
"url" : "https://github.com/microsoft/onnxruntime/releases/download/v1.14.1/onnxruntime-linux-aarch64-1.14.1.tgz",
"sha256" : "20965e05d2da6a0e6acedda8aca1a8f75c3f4cb1d7932403a9977b753b1b2120"
"url" : "https://github.com/microsoft/onnxruntime/releases/download/v1.30.0/onnxruntime-linux-aarch64-1.30.0.tgz",
"sha256" : "e16a27a8ed330bbc698df7330b0cf56e722f354e3bcc92118682c74ef3c3e3da"
}
],
"build-commands" : [
"mkdir -p $ONNX_ROOT",
"mv onnxruntime-1/* $ONNX_ROOT",

"mkdir -p $FLATPAK_DEST/lib",
"cp $ONNX_ROOT/lib/* $FLATPAK_DEST/lib"
"mkdir -p ${FLATPAK_DEST}/onnxruntime",
"cp -a include lib LICENSE ThirdPartyNotices.txt ${FLATPAK_DEST}/onnxruntime/"
]
},

{
"name": "aprilmodel",
"name" : "aprilmodel",
"buildsystem" : "simple",
"sources" : [
{
Expand All @@ -73,16 +64,28 @@
"install -Dm644 aprilv0_en-us.april -t ${FLATPAK_DEST}/LiveCaptions/models"
]
},

{
"name" : "livecaptions",
"builddir" : true,
"buildsystem" : "meson",
"config-opts" : ["--buildtype=release"],
"build-options" : {
"env" : {
"ONNX_ROOT" : "/app/onnxruntime"
}
},
"sources" : [
{
"type" : "git",
"url" : "https://github.com/abb128/LiveCaptions",
"branch" : "main"
"type" : "dir",
"path" : "../..",
"skip" : [
"builddir",
"_build",
"result",
".flatpak-builder",
"subprojects/april-asr/lib",
"april-english-dev-01110_en.april"
]
}
]
}
Expand Down
8 changes: 4 additions & 4 deletions data/meson.build
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ if desktop_utils.found()
endif

appstream_file = i18n.merge_file(
input: 'net.sapples.LiveCaptions.appdata.xml.in',
output: 'net.sapples.LiveCaptions.appdata.xml',
input: 'net.sapples.LiveCaptions.metainfo.xml.in',
output: 'net.sapples.LiveCaptions.metainfo.xml',
po_dir: '../po',
install: true,
install_dir: join_paths(get_option('datadir'), 'appdata')
install_dir: join_paths(get_option('datadir'), 'metainfo')
)

appstream_util = find_program('appstream-util', required: false)
Expand All @@ -31,7 +31,7 @@ endif

gnome.compile_schemas(build_by_default: true, depend_files: 'net.sapples.LiveCaptions.gschema.xml')
devenv = environment()
devenv.set('GSETTINGS_SCHEMA_DIR', meson.current_build_dir() / 'data')
devenv.set('GSETTINGS_SCHEMA_DIR', meson.current_build_dir())
meson.add_devenv(devenv)

install_data('net.sapples.LiveCaptions.gschema.xml',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@

<name>Live Captions</name>
<summary>Live Captioning for the desktop</summary>
<developer_name translatable="no">abb128</developer_name>
<developer id="net.sapples">
<name translatable="no">abb128</name>
</developer>

<description>
<p>Live Captions is an application that provides realtime automatic subtitles for the Linux desktop.</p>
Expand Down Expand Up @@ -34,17 +36,32 @@

<url type="homepage">https://github.com/abb128/LiveCaptions</url>
<url type="bugtracker">https://github.com/abb128/LiveCaptions/issues</url>
<url type="vcs-browser">https://github.com/abb128/LiveCaptions</url>

<screenshots>
<screenshot type="default">
<image>https://github.com/abb128/LiveCaptions/blob/main/screenshots/0.png?raw=true</image>
<image>https://raw.githubusercontent.com/abb128/LiveCaptions/main/screenshots/0.png</image>
<caption>The caption window, with less confident words faded</caption>
</screenshot>
<screenshot>
<image>https://github.com/abb128/LiveCaptions/blob/main/screenshots/1.png?raw=true</image>
<image>https://raw.githubusercontent.com/abb128/LiveCaptions/main/screenshots/1.png</image>
<caption>Captioning a recorded talk over a video</caption>
</screenshot>
</screenshots>

<releases>
<release date="2026-10-10" version="0.5.0">
<url>https://github.com/abb128/LiveCaptions/releases/tag/v0.5.0</url>
<description>
<p>Ported to current GTK 4 and libadwaita APIs, requiring libadwaita 1.5</p>
<p>Recognition no longer speeds up audio on slow computers, which reduced accuracy; the setup benchmark now requires a result of at least 1.2</p>
<p>Slow-processing warnings now show how far behind recognition is</p>
<p>Sentences are finalized after 1.2 seconds of silence instead of 2.2 seconds</p>
<p>Fixed recognition state carrying over between sentences</p>
<p>Recognition uses two CPU threads by default</p>
<p>Updated ONNX Runtime to 1.30.0</p>
</description>
</release>
<release date="2023-09-22" version="0.4.1">
<url>https://github.com/abb128/LiveCaptions/releases/tag/v0.4.1</url>
<description>
Expand Down Expand Up @@ -97,4 +114,11 @@
<launchable type="desktop-id">net.sapples.LiveCaptions.desktop</launchable>
<project_license>GPL-3.0-or-later</project_license>
<content_rating type="oars-1.1" />
<requires>
<display_length compare="ge">360</display_length>
</requires>
<supports>
<control>pointing</control>
<control>keyboard</control>
</supports>
</component>
6 changes: 3 additions & 3 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 11 additions & 5 deletions flake.nix
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
{
description = "LiveCaptions flake";
inputs = {
# april-asr is a git submodule built by meson's cmake subproject
self.submodules = true;
flake-utils.url = "github:numtide/flake-utils";
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
april-asr = {
Expand All @@ -25,19 +27,23 @@
nativeBuildInputs = with pkgs; [
meson
ninja
wrapGAppsHook
];
buildInputs = with pkgs; [
cmake
pkgconfig
pkg-config
wrapGAppsHook4
appstream-glib # appstream-util
desktop-file-utils # desktop-file-utils
gettext # msgfmt
glib # glib-compile-schemas
libadwaita # libadwaita-1
];
buildInputs = with pkgs; [
gtk4 # gtk4 >= 4.12
libadwaita # libadwaita-1 >= 1.5
libpulseaudio # libpulse
libx11 # x11
onnxruntime
];
# cmake is only used for the april-asr meson subproject
dontUseCmakeConfigure = true;
preFixup = ''
gappsWrapperArgs+=(--prefix APRIL_MODEL_PATH : "${april-asr}")
'';
Expand Down
5 changes: 3 additions & 2 deletions meson.build
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
project('livecaptions', 'c',
version: '0.1.0',
project('livecaptions', 'c', 'cpp',
version: '0.5.0',
meson_version: '>= 0.59.0',
default_options: [ 'warning_level=2',
'werror=false',
'c_std=gnu11',
'cpp_std=c++17',
],
)

Expand Down
2 changes: 1 addition & 1 deletion po/POTFILES
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
data/net.sapples.LiveCaptions.desktop.in
data/net.sapples.LiveCaptions.appdata.xml.in
data/net.sapples.LiveCaptions.metainfo.xml.in
data/net.sapples.LiveCaptions.gschema.xml
src/main.c
src/livecaptions-window.ui
Expand Down
Loading