🌐 VociBlink

Real-time speech translation for live events. The speaker talks. Every audience member reads it instantly, in their own language.

What it does

A speaker starts a session and shares a short code. Anyone can join from their phone or laptop, pick from 33 languages, and watch the translation stream in live, word-by-word — no app install, no account.

sequenceDiagram
    participant S as 🎤 Speaker
    participant B as ⚙️ Backend
    participant LLM as 🧠 LLM
    participant A as 📱 Audience (×N)

    S->>B: speech → text (WebSocket)
    B->>LLM: buffered sentence
    LLM-->>B: streamed translation tokens
    B-->>A: fan-out, per audience language
    A-->>B: reactions, questions
    B-->>S: live audience roster + Q&A

Speech-to-text runs in the speaker's browser (Web Speech API) by default — no audio leaves the device. An optional Gemini Live server-side mode can be enabled for higher accuracy on noisy mics or unsupported browsers.
Translation streams token-by-token through any OpenAI-compatible LLM, with automatic failover to a second provider on rate limits.
Audience members can react with emoji and ask questions back, auto-translated into the speaker's language.

Features


🗣️ Live transcription	Real-time speaker captions, no lag between speaking and text
🌍 33 languages	22 scheduled Indian languages + 11 international
⚡ Streaming translation	Tokens appear as the LLM generates them, not after the full sentence
🔁 Provider failover	Automatic switch to a fallback LLM on 429s
🙋 Live Q&A	Audience questions auto-translate into the speaker's language
💬 Reactions	Lightweight emoji feedback from the crowd
🔊 Optional TTS	Audience can have translations read aloud
🌓 Light & dark themes	Tuned contrast, not a default Tailwind flip
📱 Mobile-first	Tested down to 280px viewports, notch/safe-area aware
🚫 No accounts	Join with a session code, nothing to sign up for

Quick Start (Local)

Backend

cd backend
python -m venv .venv
source .venv/bin/activate       # Windows: .venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env
# Edit .env — set LLM_BASE_URL and LLM_API_KEY at minimum
uvicorn main:app --reload --port 8000

Frontend

cd frontend
npm install
npm run dev
# Opens at http://localhost:5173

Vite proxies /api and /ws to localhost:8000 automatically — no frontend env vars needed for local dev.

Docker (both services)

cp backend/.env.example backend/.env
# Edit backend/.env with your API keys
docker compose up --build
# Frontend: http://localhost:3000  |  Backend: http://localhost:8000

Deploy to Render + Vercel

Step 1 — Backend on Render

Option A — Blueprint (recommended)

Push this repo to GitHub.
Go to render.com → New → Blueprint.
Connect your repo. Render reads render.yaml and creates the service automatically.
Set the required environment variables in the Render dashboard (see table below).

Option B — Manual

New Web Service → connect repo
Root Directory: backend
Runtime: Python 3
Build Command: pip install -r requirements.txt
Start Command: uvicorn main:app --host 0.0.0.0 --port $PORT
Add environment variables (see table below)

Note the service URL — you'll need it for the frontend step. It looks like https://vociblink-backend.onrender.com.

Step 2 — Frontend on Vercel

vercel.com → New Project → import your repo.
Root Directory: frontend (framework auto-detects as Vite).
Add environment variables:

Key	Value
`VITE_API_URL`	`vociblink-backend.onrender.com`
`VITE_WS_URL`	`vociblink-backend.onrender.com`

Deploy.

Step 3 — Allow the frontend origin

Back on the Render dashboard, set:

Key	Value
`CORS_ORIGINS`	`["https://your-project.vercel.app"]`

Redeploy. Done.

Environment Variables

Backend

Variable	Required	Description
`LLM_BASE_URL`	Yes	Base URL of any OpenAI-compatible API
`LLM_API_KEY`	Yes	API key for the primary provider
`LLM_MODEL`	Yes	Model name (default: `mistral-small-latest`)
`LLM_FALLBACK_BASE_URL`	No	Fallback provider URL — auto-used when primary returns 429
`LLM_FALLBACK_API_KEY`	No	Fallback provider API key
`LLM_FALLBACK_MODEL`	No	Fallback model name
`GEMINI_API_KEY`	No	Enables server-side Gemini Live STT; omit to use browser STT only
`GEMINI_MODEL`	No	Gemini Live model (default: `gemini-3.1-flash-live-preview`)
`CORS_ORIGINS`	Prod	JSON array of allowed origins, e.g. `["https://app.vercel.app"]`
`HOST`	No	Bind host (default: `0.0.0.0`)
`PORT`	No	Port (default: `8000`; Render injects this automatically)

Frontend

Variable	When needed	Description
`VITE_API_URL`	Production	Backend HTTP base URL
`VITE_WS_URL`	Production	Backend WebSocket base URL

LLM Providers

Any OpenAI-compatible API works. The backend supports a primary + fallback and switches automatically on rate limits.

Provider	Free Tier	`LLM_BASE_URL`	Suggested `LLM_MODEL`
Mistral	Yes	`https://api.mistral.ai/v1/`	`mistral-small-latest`
NVIDIA NIM	Yes	`https://integrate.api.nvidia.com/v1/`	`meta/llama-3.1-8b-instruct`
Groq	Yes	`https://api.groq.com/openai/v1/`	`llama-3.1-8b-instant`
OpenAI	No	`https://api.openai.com/v1/`	`gpt-4o-mini`

Recommended setup: Mistral as primary (generous free tier) + NVIDIA NIM or Groq as fallback.

Render Free Tier Notes

The free tier spins down after 15 minutes of inactivity. First request after sleep takes ~30–60 seconds.
An active WebSocket connection keeps the service alive.
Set up an uptime monitor (e.g. UptimeRobot pinging /api/health) to prevent cold starts during events.
The backend runs a translation warmup request on startup to reduce first-translation latency.

Tech Stack

Layer	Technology
Frontend	React 18, TypeScript, Tailwind CSS, Vite
Backend	FastAPI, Python 3.12, asyncio
Transport	WebSockets (native)
Speech-to-Text	Browser Web Speech API, optional Gemini Live
Translation	Any OpenAI-compatible LLM
TTS (optional)	Browser SpeechSynthesis API

Project Structure

vociblink/
├── render.yaml                    # Render Blueprint
├── docker-compose.yml
├── backend/
│   ├── main.py                    # FastAPI app, WebSocket handlers, translation pipeline
│   ├── core/config.py             # Env-var settings (pydantic-settings)
│   ├── models/schemas.py          # Pydantic schemas + LANGUAGE_NAMES map
│   ├── services/
│   │   ├── session_manager.py     # In-memory session state (speakers + audience)
│   │   ├── translation_service.py # Streaming LLM translation + provider failover + cache
│   │   ├── broadcast_manager.py   # WebSocket fan-out helpers
│   │   ├── content_filter.py      # Profanity filter (word-boundary aware)
│   │   └── gemini_stt.py          # Optional server-side speech-to-text (Gemini Live)
│   ├── .env.example
│   ├── requirements.txt
│   └── Dockerfile
└── frontend/
    ├── public/
    │   └── pcmWorkletProcessor.js # AudioWorklet: resamples mic input to 16kHz
    ├── src/
    │   ├── pages/                 # Home.tsx, Speaker.tsx, Audience.tsx
    │   ├── components/            # TranscriptBlock, LanguageSelector, MicrophoneButton, …
    │   ├── lib/
    │   │   ├── audio.ts           # TTSPlayer (browser SpeechSynthesis)
    │   │   ├── audioCapture.ts    # Mic capture → AudioWorklet → PCM stream
    │   │   ├── websocket.ts       # Reconnecting WebSocket client + URL helpers
    │   │   ├── speech.ts          # Cross-browser SpeechRecognition wrapper
    │   │   └── theme.tsx          # ThemeProvider + useTheme (light/dark)
    │   ├── styles/globals.css     # CSS custom properties (semantic color tokens)
    │   └── types/index.ts
    ├── vercel.json                # SPA rewrite rule for Vercel
    ├── tailwind.config.js
    ├── vite.config.ts
    └── Dockerfile

Supported Languages

33 languages — 22 scheduled Indian languages + 11 international (click to expand)

Indian: Hindi · Bengali · Telugu · Marathi · Tamil · Urdu · Gujarati · Kannada · Odia · Malayalam · Punjabi · Assamese · Maithili · Sanskrit · Konkani · Nepali · Sindhi · Dogri · Manipuri · Bodo · Santali · Kashmiri

International: English · Spanish · French · German · Portuguese · Italian · Russian · Japanese · Korean · Chinese · Arabic

WebSocket Protocol

Speaker — wss://host/ws/speaker/{session_id}

Direction	Message	Description
→ Backend	`{type:"transcript", text, is_final}`	STT result (interim or final)
→ Backend	`{type:"set_session_title", title}`	Update session title
→ Backend	`{type:"set_question_language", language}`	Language for audience questions
→ Backend	`{type:"ping"}`	Keep-alive
→ Backend	`{type:"end_session"}`	Close session
Backend →	`{type:"session_info", …}`	Session metadata on connect
Backend →	`{type:"audience_list", members}`	Live audience roster
Backend →	`{type:"audience_reaction", emoji, sender_name}`	Emoji reaction
Backend →	`{type:"question", original_text, translated_text, …}`	Audience question

Audience — wss://host/ws/audience/{session_id}?language=hi&name=Alice

Direction	Message	Description
Backend →	`{type:"translation_start"}`	New translation stream beginning
Backend →	`{type:"translation_token", token}`	Streamed translation chunk
Backend →	`{type:"translation_end", is_final}`	Stream complete
Backend →	`{type:"speaker_status", active}`	Speaker online/offline
→ Backend	`{type:"language_change", language}`	Switch translation language
→ Backend	`{type:"reaction", emoji}`	Send emoji reaction
→ Backend	`{type:"question_text", text}`	Send a question

Name		Name	Last commit message	Last commit date
backend		backend
frontend		frontend
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
docker-compose.yml		docker-compose.yml
render.yaml		render.yaml

Navigation Menu

Repository files navigation

🌐 VociBlink

What it does

Features

Quick Start (Local)

Backend

Frontend

Docker (both services)

Deploy to Render + Vercel

Step 1 — Backend on Render

Step 2 — Frontend on Vercel

Step 3 — Allow the frontend origin

Environment Variables

Backend

Frontend

LLM Providers

Render Free Tier Notes

Tech Stack

Project Structure

Supported Languages

WebSocket Protocol

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

🌐 VociBlink

What it does

Features

Quick Start (Local)

Backend

Frontend

Docker (both services)

Deploy to Render + Vercel

Step 1 — Backend on Render

Step 2 — Frontend on Vercel

Step 3 — Allow the frontend origin

Environment Variables

Backend

Frontend

LLM Providers

Render Free Tier Notes

Tech Stack

Project Structure

Supported Languages

WebSocket Protocol

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages