Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ Built for the [Backblaze Generative Media Hackathon](https://backblaze-generativ
- **Firebase mirror:** https://upgradegr-cinemory.web.app, the identical app.
- **Demo video:** [`demo/cinemory-demo.mp4`](demo/cinemory-demo.mp4) (2:50, of
which 95.9s, 56.5%, is live screen capture of this deployed app doing the
real thing, shot as one unbroken five-photo run).
real thing, shot as one unbroken five-photo run). The voice-over and the
music bed under it are both generated, so no third-party track is used.
Watch it on YouTube: **https://youtu.be/mqBw1hc8TxY**

**Check that the live app really is this commit.** The image is stamped at
Expand Down
38 changes: 38 additions & 0 deletions demo/STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,44 @@ _Last updated: 2026-08-03. Deadline: 2026-08-03 5:00pm EDT. $10k. Greece-eligibl
> were true on that date and are very likely superseded. If two entries
> disagree, the higher one wins.

## 2026-08-03: the soundtrack is generated, not licensed (canonical)

> The film had no music because an unlicensed track is a copyright problem we
> will not hand a judge. Generating it removes the question instead of dodging
> it: the bed is as clearly ours as the footage. `minimax-music-2.5` through the
> same Genblaze path every other asset uses, prompted from the occasion the reel
> in the video actually picks (anniversary: warm romantic strings, 96 bpm),
> which is exactly what `Occasion.music_style` has always documented itself as
> being for. Prompt, model and SHA-256 are recorded in
> `demo/cinemory-demo.beats.json` like every other generated artifact.
>
> Two things had to be solved. The model gives **no duration control**, so the
> bed came back 90.26s against a 169.74s film; `demo/mix-music.py` loops it by
> crossfading it into a second copy of itself with a constant-power (`qsin`)
> seam, which on a sustained strings pad is not something you can point at.
> And the **narration stays dominant**: both tracks are measured with EBU R128
> and the bed is attenuated to sit exactly **20.1 LU** under the voice, with
> `amix normalize=0` so the mixer cannot quietly undo the measurement. The
> film's integrated loudness is **-25.9 LUFS before and after**, so the voice is
> not one decibel quieter than the cut that was already approved.
>
> The picture is **stream copied**, never re-encoded, so every footage beat,
> caption window and cut survives byte for byte. 169.74s, still under the 180s
> cap, still ten beats and ten cues.
>
> `scripts/check_video.py` gained a `music:` section. The mix is one AAC track,
> so "exactly one audio stream" still guards against a stray stream but can no
> longer tell narration-only from narration-plus-music. Two new bounds do, and
> they are two-sided on purpose: a **floor** (narration-only leaves 44 windows
> of true dead air between padded lines; with the bed under it that count is 1,
> so a rebuild that skips the music step fails) and a **ceiling** (the mixed
> film must still measure within 1.0 LU of the narration it was built from, so a
> bed mixed anywhere near the voice fails). The bed is also pinned by content
> hash.
>
> Owner action: re-upload `demo/cinemory-demo.mp4` to YouTube and repoint the
> Devpost video link.

## 2026-08-03: video published, project SUBMITTED (canonical)

> The demo video is on YouTube at **https://youtu.be/mqBw1hc8TxY** and Devpost
Expand Down
3 changes: 2 additions & 1 deletion demo/SUBMISSION.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@
use the run.app URL.)*
- **Demo video:** recorded + committed —
[`demo/cinemory-demo.mp4`](cinemory-demo.mp4) (2:50, inside Devpost's 3-min
cap; ElevenLabs voice-over, no music, A/V-sync-gated in CI via the
cap; ElevenLabs voice-over over a generated music bed, A/V-sync-gated in CI
via the
`demo-video` job). Six of its ten beats are **live screen capture of the
deployed app** (95.9s, 56.5% of the runtime), cut from one unbroken take of a
full five-photo run: photos in, occasion picked, all five model calls rolling
Expand Down
5 changes: 5 additions & 0 deletions demo/build-video.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@
CACHE = os.path.join(WORK, "tts-cache")

OUT_MP4 = os.path.join(DEMO, "cinemory-demo.mp4")
MUSIC_BED = os.path.join(ASSETS, "music", "anniversary-bed.mp3")
OUT_SRT = os.path.join(DEMO, "cinemory-demo.en.srt")
OUT_BEATS = os.path.join(DEMO, "cinemory-demo.beats.json")

Expand Down Expand Up @@ -586,6 +587,10 @@ def main() -> int:
"-c:v", "libx264", "-preset", "medium", "-crf", "18", "-pix_fmt", "yuv420p",
"-c:a", "aac", "-b:a", "192k", "-ar", "44100",
"-movflags", "+faststart", tmp_out])
# Lay the generated music bed under the finished cut. This is the last step
# on purpose: it stream copies the picture, so nothing above can drift.
run([sys.executable, os.path.join(DEMO, "mix-music.py"), tmp_out, MUSIC_BED, tmp_out])

for attempt in range(5):
try:
os.replace(tmp_out, OUT_MP4)
Expand Down
14 changes: 14 additions & 0 deletions demo/cinemory-demo.beats.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,20 @@
"voice_id": "21m00Tcm4TlvDq8ikWAM",
"model_id": "eleven_multilingual_v2"
},
"music": {
"asset": "video-assets/music/anniversary-bed.mp3",
"generated": true,
"licence": "generated for this project; no third-party track is used",
"provider": "genblaze",
"platform": "gmicloud",
"model": "minimax-music-2.5",
"prompt": "Instrumental only, no vocals, no voice, no singing, no lyrics. warm romantic strings, 96 beats per minute. Gentle cinematic underscore for a short memory film: soft, warm, unhurried, low dynamic range, nothing percussive or sudden, designed to sit quietly beneath a spoken voice-over.",
"sha256": "095d5c271924360e393cad78a31e639a71267db61a8e7753b3726613ae1eca6a",
"source_seconds": 90.26,
"loop": "crossfaded 2x, 4.0s constant-power (qsin) seam, trimmed to the film",
"narration_lufs": -25.9,
"duck_lu": 20.1
},
"total_seconds": 169.7,
"beats": [
{
Expand Down
Binary file modified demo/cinemory-demo.mp4
Binary file not shown.
148 changes: 148 additions & 0 deletions demo/generate-music.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
#!/usr/bin/env python3
"""Generate the demo video's music bed — Genblaze to MiniMax Music on GMI Cloud.

The soundtrack is **generated, not licensed**. That is the whole point: a track
whose licence we cannot state precisely is a problem to hand a judge, and a
generated bed is as clearly ours as the footage. It goes through the *same*
path every other Cinemory asset takes — the :class:`MediaProvider` port, the
Genblaze ``Pipeline``, GMI Cloud — so nothing here is a side door.

Direction comes from the occasion the reel in the video actually uses, not from
a genre invented for the video. ``Occasion.music_style`` is documented as
"maps to a track/generation prompt on the live path"; this is that path. The
demo take is an **anniversary** reel (see ``demo/video-assets/reels.txt``), so
the bed is built from ``warm romantic strings`` at ``96`` BPM.

Run once; commit the result. ``demo/build-video.py`` muxes the **committed**
mp3 and never calls a provider, so the video gate and a rebuild stay offline.

python demo/generate-music.py # -> video-assets/music/
python demo/generate-music.py --occasion wedding # a different bed

Env: GMI_API_KEY (required)
GENBLAZE_PROVIDER (default gmicloud)
"""
from __future__ import annotations

import argparse
import hashlib
import json
import os
import subprocess
import sys

sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "src"))

from cinemory.adapters.genblaze_provider import ( # noqa: E402
MINIMAX_MUSIC_MODEL,
GenblazeMediaProvider,
)
from cinemory.models import Modality # noqa: E402
from cinemory.occasions import OCCASIONS # noqa: E402

DEMO = os.path.dirname(os.path.abspath(__file__))
MUSIC_DIR = os.path.join(DEMO, "video-assets", "music")
FFPROBE = os.environ.get("FFPROBE", "ffprobe")

#: Structure tags only, no words. ``[Inst]`` is MiniMax's documented
#: instrumental-section tag, and ``lyrics`` is a REQUIRED parameter — so an
#: instrumental bed is requested by giving the model a lyric sheet that asks
#: for playing rather than singing. A vocal under the narration would be worse
#: than the silence this replaces, so the prompt also opens with the negative
#: constraint rather than burying it after the style.
INSTRUMENTAL_LYRICS = "[Intro]\n[Inst]\n[Inst]\n[Outro]"

#: MP3 at the model's top encode. The bed is muxed and re-encoded to AAC on the
#: way into the video, so this is the source master, not the shipped audio.
AUDIO_PARAMS = {"format": "mp3", "sample_rate": 44100, "bitrate": 256000}


def music_prompt(occasion) -> str:
"""The generation prompt for one occasion's bed.

Negative constraint first (see ``INSTRUMENTAL_LYRICS``), then the
occasion's own ``music_style`` and ``tempo`` verbatim, then the fact that
this is underscore — music that has to sit beneath a voice without
competing with it.
"""
return (
"Instrumental only, no vocals, no voice, no singing, no lyrics. "
f"{occasion.music_style}, {occasion.tempo:.0f} beats per minute. "
"Gentle cinematic underscore for a short memory film: soft, warm, "
"unhurried, low dynamic range, nothing percussive or sudden, "
"designed to sit quietly beneath a spoken voice-over."
)


def probe_seconds(path: str) -> float:
out = subprocess.run(
[FFPROBE, "-v", "error", "-show_entries", "format=duration",
"-of", "default=nw=1:nk=1", path],
capture_output=True, text=True, check=True,
)
return float(out.stdout.strip())


def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--occasion", default="anniversary", choices=sorted(OCCASIONS))
args = ap.parse_args()

occ = OCCASIONS[args.occasion]
prompt = music_prompt(occ)
print(f"occasion : {occ.key} ({occ.music_style}, {occ.tempo:.0f} bpm)")
print(f"model : {MINIMAX_MUSIC_MODEL}")
print(f"prompt : {prompt}")

# No storage sink: the bed is a one-off demo asset that gets committed to
# the repo, so persisting it to B2 as well would spend transactions for a
# copy nothing reads. Genblaze still seals its own run manifest either way.
provider = GenblazeMediaProvider(bucket="-")
provider._bucket = None # noqa: SLF001 - explicit "no sink" for this one-off

data = provider.generate(
model=MINIMAX_MUSIC_MODEL,
prompt=prompt,
modality=Modality.AUDIO,
params={"lyrics": INSTRUMENTAL_LYRICS, **AUDIO_PARAMS},
)

os.makedirs(MUSIC_DIR, exist_ok=True)
digest = hashlib.sha256(data).hexdigest()
stem = f"{occ.key}-bed"
mp3 = os.path.join(MUSIC_DIR, f"{stem}.mp3")
with open(mp3, "wb") as fh:
fh.write(data)
seconds = probe_seconds(mp3)

manifest = {
"asset": f"video-assets/music/{stem}.mp3",
"generated": True,
"licence": "generated for this project; no third-party track is used",
"provider": "genblaze",
"platform": "gmicloud",
"model": MINIMAX_MUSIC_MODEL,
"prompt": prompt,
"lyrics": INSTRUMENTAL_LYRICS,
"params": AUDIO_PARAMS,
"occasion": {"key": occ.key, "music_style": occ.music_style, "tempo": occ.tempo},
"sha256": digest,
"size_bytes": len(data),
"seconds": round(seconds, 2),
}
side = os.path.join(MUSIC_DIR, f"{stem}.json")
with open(side, "w", encoding="utf-8") as fh:
json.dump(manifest, fh, indent=2, ensure_ascii=False)
fh.write("\n")

print(f"\nwrote {mp3} ({len(data)} bytes, {seconds:.2f}s)")
print(f"sha256 {digest}")
print(f"wrote {side}")
gb = provider.last_manifest
if gb is not None:
print(f"genblaze manifest sealed: {type(gb).__name__}")
return 0


if __name__ == "__main__":
sys.exit(main())
Loading
Loading