Semantic Beat

Playlist recommender that combines semantic search (FAISS + sentence embeddings) with collaborative filtering (BPR + ALS). You describe your vibe in plain text and get a playlist you can listen to, reorder, and edit.

Repo structure

semantic_beat/
├── backend/
│   ├── main.py            # FastAPI app with 3 endpoints
│   ├── recommender.py     # recommendation pipeline
│   ├── models.py          # model classes for loading the joblib
│   ├── downloader.py      # downloads tracks from SoundCloud via yt-dlp
│   └── downloads/         # cached mp3 files
├── frontend/              # React + Vite app
│   └── src/
│       ├── App.jsx        # main UI component
│       └── App.css        # styling
├── notebook/
│   ├── spotify_recsys.ipynb     # full ML pipeline (training + evaluation)
│   └── download_tracks.ipynb    # track download snippet
├── data/
│   └── mpd_tracks_subset.parquet
├── models/
│   ├── bpr_als_fusion.joblib    # trained BPR + ALS models
│   └── all-MiniLM-L6-v2/       # sentence transformer
└── artist_vector_db/
    ├── artists.faiss            # FAISS index
    └── artists_metadata.parquet

Backend endpoints

Endpoint	Method	What it does
/recommend	POST	Takes a text query, returns a ranked playlist
/replace	POST	Returns a replacement track with low MPD coverage
/download	POST	Downloads a track from SoundCloud and sends the mp3

Setup

1. Create the conda environment

conda create -n rec_sys python=3.11
conda activate rec_sys
pip install fastapi "uvicorn[standard]" polars numpy faiss-cpu sentence-transformers torch joblib implicit scipy yt-dlp

2. Run the backend

cd backend
python -m uvicorn main:app --port 8000

3. Run the frontend

cd frontend
npm install
npm run dev

Then open http://localhost:5173 in your browser.

How it works

Data

We use a subset of the Spotify Million Playlist Dataset (MPD) — 10,000 playlists, ~209k track-playlist pairs. Each row has a playlist id (pid), track/artist/album URIs, names, and the track position in the playlist.

Artist metadata (genres, bios, similar artists, play counts) comes from LastFM — 43,619 artists stored as NDJSON.

Data transformation

1. Ratings from positions: We convert implicit feedback (a track being in a playlist) into a rating using the formula 1 / log2(position + 2). Tracks at the top of a playlist get a higher weight.
2. Sparse matrix: Ratings are stored in a user-item CSR matrix (10k users x ~43k items, 0.05% density) used by the collaborative filtering models.
3. Artist documents: Each LastFM artist is turned into multiple text documents — a style doc (name + genres + similar artists), a bio summary, a stats doc, and chunked bio paragraphs (~800 chars each).
4. Embeddings: All artist documents are encoded into 384-dim vectors using all-MiniLM-L6-v2 (sentence transformer), normalized, and stored in a FAISS IndexFlatIP index for cosine similarity search.

Models

• BPR (Bayesian Personalized Ranking): A learning-to-rank model (128 factors, 200 iterations) that learns which tracks a user prefers over others. Used as the retriever — it returns ~50 candidate tracks per user.
• ALS (Alternating Least Squares): Matrix factorization model (64 factors, 50 iterations). Wrapped in ALSScoreFusionReranker which blends ALS scores (80%) with popularity scores (20%) to re-rank the candidates from BPR.
• FAISS + Sentence Transformer: Semantic search over artist descriptions. The user's text query is embedded and matched against the artist vector DB to find relevant artists.

Recommendation pipeline

1. User types a query like "I like rock and roll with melodic vocals"
2. Semantic search: Query is embedded and matched against the FAISS index → top 10 artists
3. Expansion: Each artist is used as a new query to find 5 more similar artists
4. Track filtering: All tracks from the selected artists are pulled from the MPD subset
5. Collaborative scoring: For users who listen to those artists, BPR retrieves 50 candidates, ALS re-ranks them
6. Score blending: Final score = (1 - w) * ALS_rank_score + w * semantic_score where w is the semantic weight (default 0.4)
7. Top tracks are returned as the playlist