The list can say how long each track is and how good (#567)

defaults.py

@@ -341,6 +341,33 @@
 # and rebuilt from the lists on disk (#628); the copy can be deleted.
 LIST_INDEX_FILE: str = "./data/list_index.db"
 
+# Duration and quality after the size on the list's MP3 and FLAC rows (#567):
+# "::INFO:: 10.3MB 4m31s 320/44.1/JS" - the spelling other servers' lists use.
+# Off by default because it OPENS every audio file, where the scan otherwise
+# asks for nothing but sizes: the first rebuild with it on takes noticeably
+# longer. What it read is kept in LIST_AUDIO_INFO_CACHE, checked against each
+# file's size and modification time, so later rebuilds open only new or changed
+# files. Read with the standard library (audio_info.py); a file it cannot read
+# keeps its size and nothing more.
+LIST_SHOW_AUDIO_INFO: bool = False  # Put duration and bitrate after the size on MP3 and FLAC rows
+# One row per audio file in the lists (about 150 bytes each). Safe to delete:
+# the next rebuild reads every file again.
+LIST_AUDIO_INFO_CACHE: str = "./data/audio_info.db"
+# How many audio files are read at once (#914). On a network mount (NFS, SMB)
+# the time goes into round trips, which overlap. Measured on a real 64,136-file
+# NFS library: one at a time 9.8 files a second, 16 about 73, 64 about 236 (the
+# last partly on a cache warmed by the run before). 64 by default - a plain
+# disk answers 64 requests as readily as it answers 16; on a very old drive
+# or a very small library, lower it. The rebuild's last line says the rate it
+# got, to compare. 1 to 128.
+LIST_AUDIO_INFO_THREADS: int = 64  # Audio files read at once for length and quality
+# The most time one rebuild spends reading audio files it has not read before
+# (#914). A rebuild pauses searches and requests, and the first one with
+# LIST_AUDIO_INFO on has the whole library to read: past this, the list
+# publishes with what was read and the rest wait for the next rebuild. 0 = no
+# limit.
+LIST_AUDIO_INFO_MINUTES: int = 5  # Minutes one rebuild may spend reading new audio files; 0 = no limit
+
 # One row per thing this bot has ever sent, {relative path or archive name ->
 # {name, kind, count}}. Feeds the Stats page's "Most downloaded" table. Not
 # bounded on purpose: a bot can only send what it shares, so the row count is

docs/INSTALL.md

@@ -250,6 +250,16 @@ Two settings decide how the result is split up:
 - **`SEPARATE_VIDEO_LIST`** — publishes film and series as their own list rather than mixing them in with the music. Both travel in the same archive people get by typing your bot's name, so there is no second command to learn. `LIST_VIDEO_EXTENSIONS` says which formats count, and `LIST_VIDEO_COMPANION_EXTENSIONS` (subtitles, `.nfo`, `.sfv`) says which files follow a film into its list when they sit in the same folder - so a release travels whole, while an album's `.nfo` stays with the album. Turn it off if your films and music are already in separate folders and you would rather split by folder.
 - **`RAR_EXTENSIONS`** — which formats make a folder packable with `!rar`. A folder needs one of these to get a row in the album list. Everything else stays listed and directly requestable; this only decides what can be packed. **`MAX_RAR_FOLDER_SIZE`** bounds how large a folder `!rar` will pack — 10 GB by default, which passes a large box set and refuses the folder somebody names hoping it is a library. Set it to 0 for no limit.
 
+**`LIST_SHOW_AUDIO_INFO`** (off by default) adds each MP3 and FLAC file's length and quality after its size -
+`::INFO:: 10.3MB 4m31s 320/44.1/JS`, the way other servers' lists show it (`~245` is a VBR average). Every audio
+file has to be read once. The files are read several at a time (`LIST_AUDIO_INFO_THREADS`, 64), and each rebuild
+spends at most `LIST_AUDIO_INFO_MINUTES` (5) on it, since searches wait while a rebuild runs: on a large library,
+or one on a network drive, the first few rebuilds each publish with part of the library read and the rest showing
+its size alone, until everything has been read once. After that only new files are read, and a rebuild costs what
+it did without the setting. What was read is kept in `data/audio_info.db`; deleting it is safe - the files are
+read again. The rebuild's last line says how fast the files were read; if raising `LIST_AUDIO_INFO_THREADS`
+further does not raise that number, you have found the server's own limit rather than the setting's.
+
 ### If your users queue with AutoQ
 
 AutoQ (the mIRC queue script most of these channels use) pastes list rows into

docs/UPDATES-PUBLIC.md

@@ -2,6 +2,7 @@
 
 ## Unreleased
 
+- **Added: your list can say how long each track is and at what quality.** Turn on *Length and quality in the list* on the Settings page (`LIST_SHOW_AUDIO_INFO`), and every MP3 and FLAC row gets its duration and bitrate/sample rate/channels after the size - `::INFO:: 10.3MB 4m31s 320/44.1/JS` - the way other servers' lists already show it. Every audio file has to be read once, several at a time; on a large library or a network drive that is spread over a few rebuilds, each spending at most 5 minutes on it (*Time limit for reading audio files* under *List rebuild*), so searches are never held up for long. After that only new or changed files are read. A file it cannot read keeps just its size, and search results drop the extra detail before they would cut a filename short.
 - **Added: a checkbox for the update check in the mIRC window too, not just the dashboard.** `/dccore options` has a new box, **Check GitHub for a new DCCore version**, and the window's right-click menu has a matching toggle beside "Check for a new version" - both read and change the same setting the dashboard's checkbox does. Update `dccore.mrc`.
 - **Added: the bot tells you when a new version of DCCore is out.** Once a day it asks GitHub for the latest release - one request, carrying nothing about your bot - and says so in the dashboard's sidebar, in the console's `status` and in the mIRC window. The dashboard has a **Check now** button, and the console (and the mIRC menu) a `checkversion` command. If it cannot reach GitHub it says why, rather than staying quiet. It is on by default and says so at every start; untick *Tell me when a new version is out* on the Settings page (or set `CHECK_FOR_UPDATES = false`) to turn it off. Update `dccore.mrc` for the menu entry.
 - **Added: the list can rebuild itself on a schedule.** Set **Rebuild the list automatically** (Settings, List rebuild) to `daily 04:00`, `weekly sun 04:00`, `monthly 1 03:30` or `every 12h`, and the bot runs the same rebuild `!update` does, on its own clock. If it was off at the scheduled time it catches up once when it starts; a rebuild that fails is tried again at the next scheduled time rather than over and over. The Tools page and the console's `status` show when the next one is. Off by default.

docs/UPDATES.md

@@ -14,6 +14,79 @@ switches. Now `importlib.reload` is wrapped for the test: inside the reload a se
 joined before the reload goes on, so it is refused exactly when the lock is held around the reload. Five runs in a
 row pass; with the `with runtime.config_reload_lock:` removed it fails (as does its neighbour). Test only.
 
+### 🎚️ The list can say how long each track is and how good (#567)
+
+The list gave size and nothing else, while other servers' lists give duration and quality too - which is what
+someone choosing between two copies of an album wants to know: 320 or VBR, is that FLAC really lossless-sized, is
+track 4 the seven-minute version. **`LIST_SHOW_AUDIO_INFO`** (off by default; *Your list* on the Settings page) adds
+them after the size on every MP3 and FLAC row, in the spelling those lists already use, so our own parser and other
+bots' read it unchanged:
+
+```
+!DCCore Artist - Album - 01 - Track.mp3  ::INFO:: 10.3MB 4m31s 320/44.1/JS
+!DCCore Artist - Album - 02 - Track.flac  ::INFO:: 16.7MB 2m5s 1115/44.1/S
+!DCCore Artist - Album - Front.jpg  ::INFO:: 94.4KB
+```
+
+- **Standard library only** (`audio_info.py`, no mutagen). MP3: skip every ID3v2 tag, find the first frame whose
+  successor is where its header says (a lone sync pattern in padding proves nothing), decode it, then read a
+  Xing / Info / VBRI header for the frame count - the only honest duration for VBR; without one it is CBR and the
+  audio bytes over the bitrate. FLAC: `fLaC`, the metadata blocks walked (a big picture block is stepped over, not
+  read), STREAMINFO for rate, channels and samples; the bitrate is the real one, audio bytes over duration.
+- **Spelling decided here** (the issue left it open): duration `4m31s` (minutes go past 59: `72m10s`),
+  then `kbps/kHz/channels` with channels `S` `JS` `DC` `M` or `6ch`. A VBR average is marked with a leading
+  `~` (`~245/44.1/JS`) - no spelling for it was on record.
+- **Anything it cannot read keeps its size and nothing more** - a malformed file, an unknown format, a read error.
+  `read()` never raises; a list build is never taken down by one file.
+- **Built for a network mount, after a live test on one.** The first version read one file at a time during the
+  walk - a stat, a 64 KB read and a seek to the end for an ID3v1 tag each - and on the operator's real library
+  (64,136 files, 1.85 TB, NFS) managed **9.8 files a second**: two hours for one rebuild, with `PAUSE_ON_UPDATE`
+  refusing every search and request meanwhile (Neo's report on #914; the run was stopped, cleanly). On a network
+  mount the time is round trips, not bytes, so it now works the way QuickList - OmenServe's list maker - does:
+  - **One request for an ordinary file.** Opened unbuffered and read through a window that fetches only what it
+    does not hold: one 16 KB read covers the ID3 header, the first frame and its Xing header, or FLAC's STREAMINFO.
+    A big ID3 tag or a picture block in the middle costs one more; the ID3v1 tag is no longer looked for (a request
+    for 128 bytes, 8 ms of a 128 kbps file).
+  - **Many files at once.** Files are only *noted* during the walk; the ones to read are read after it,
+    `LIST_AUDIO_INFO_THREADS` (64) at a time, so the round trips overlap. With 5 ms of simulated latency per
+    request, 16 workers read 35 times as many files a second as one.
+  - **No request at all for an unchanged file.** The cache is loaded into memory once and checked against the size
+    the directory listing already gave - no stat, no read - and written back once.
+  - **A time limit.** `LIST_AUDIO_INFO_MINUTES` (5; 0 = none) bounds the reading one rebuild does: past it no read is
+    started, the list publishes with what was read, the rest keep their size alone and the next rebuild reads them.
+    The dashboard shows *Reading length and quality: n of m files*, which also keeps the stall check fed.
+  - **The rate is said.** The rebuild's last line ends *Read at N files a second, 64 at a time* - on a network
+    mount the ceiling is the server's, so that is what an operator compares when trying another thread count.
+    Measured live on the NFS library (Neo, #914): three ordinary rebuilds of 5-6 minutes each read 62,657 of
+    the 62,699 audio files (42 unreadable) - 17 minutes in total, where the first version needed two hours in one - at about 55-60 files a
+    second with 16. From an empty cache again, 16 read about 73 files a second and 64 about 236 - three to four
+    times as fast, partly on a server cache warmed by the run before. The proposal here had shipped 32 as the
+    cautious middle of that result; asked directly, the operator preferred shipping the number actually measured
+    - 64 by default, range 1 to 128 - since a plain disk answers 64 requests as readily as 16.
+  The two live under *List rebuild* on the Settings page.
+- **The cache**, SQLite at `LIST_AUDIO_INFO_CACHE` (`./data/audio_info.db`, beside the list index), keyed by the
+  row's folder and name. Each list prunes only its own rows, and only when its rebuild **publishes**; a rebuild that
+  fails or is stopped keeps what it read and forgets nothing. A file that could not be read is remembered as such
+  until its size changes, so a broken file is not re-read on every rebuild. The rebuild says how many it read, how
+  many were unchanged and how many are left. A cache that cannot be opened is said, and that list is written
+  size-only. (A cache from the first version is read as it is.)
+- **`@find` keeps the name first.** A result row goes through the line budget as before, but one that would be cut
+  is sent without its audio tail first, so no letter of the name - the part people paste back - is spent on it.
+  Search words still match the whole row, so `@find <artist> 320` narrows to 320 kbps copies.
+- AutoQ is unaffected: the tail is on file rows only, after the size, where its file branch never looks; the
+  `!rar` rows stay exactly as they were.
+
+Stacked on #913, where *Your list* has room since #776 moved the rebuild limits out.
+`tests/test_the_list_says_how_long_and_how_good.py` (42) builds every MP3 and FLAC byte by byte - CBR, both ID3
+tags, a tag bigger than the search window, a false sync, all four channel modes, Xing, Info, VBRI, MPEG-2, FLAC
+mono / 6ch / 96 kHz / a 3 MB picture block, six kinds of broken file - plus the reads each file costs (one, or two
+with cover art), the cache (no request for an unchanged file, a changed size, prune, a stopped rebuild, per-list
+scope, cannot open), reads that provably overlap (a barrier only four concurrent reads pass), the time limit, a
+reader that raises, a real rebuild on, off and out of time, and `@find` at the exact length where the tail decides
+whether the name is cut. Mutation-checked: removing the sync confirmation, the ID3v2 skip, the window's reuse, the
+16 KB first read, the size check, the parallelism, the time limit, the save on a stopped rebuild, the per-list
+prune, the prune on publish or the search fallback each fails a test.
+
 ### 🆕 The bot says when a new version of DCCore is out (#572)
 
 Nothing told an operator that a release existed: the ones who never read the repository - most of them - ran old

list.py

@@ -560,6 +560,18 @@ def strip_info_suffix(rest):
     return filename.strip(), size.strip()
 
 
+# The duration-and-quality tail one of OUR rows carries after its size when
+# LIST_SHOW_AUDIO_INFO is on (#567): "::INFO:: 10.3MB 4m31s 320/44.1/JS", or
+# "~245/44.1/JS" for a VBR average. Anchored to the end, and to the size
+# token right after the marker, so nothing in a filename can match it.
+_AUDIO_TAIL_RE = re.compile(r'(::INFO::\s*\S+)\s+\d+m\d+s\s+~?\d+/[\d.]+/\w+\s*$')
+
+
+def without_audio_info(row):
+    """The row with its audio tail removed, or the row unchanged."""
+    return _AUDIO_TAIL_RE.sub(r'\1', row)
+
+
 def _split_entry_line(line_strip):
     """Pull the filename and size back out of one "!..." master-list line.
 
@@ -1336,7 +1348,13 @@ def _build(shown_match):
                                        f"{shown_match}{R} {BG_CYAN_BLOCK} {BG_RED_BLOCK} ")
                         return f"PRIVMSG {user} :{block_match}\r\n"
 
-                    oserve.queue_message(user, announce.fit_irc_line(_build, match))
+                    line = announce.fit_irc_line(_build, match)
+                    # A row the budget would cut loses its audio tail
+                    # (#567) before a single letter of its name: the name
+                    # is what the reader pastes back to ask for the file.
+                    if line != _build(match) and without_audio_info(match) != match:
+                        line = announce.fit_irc_line(_build, without_audio_info(match))
+                    oserve.queue_message(user, line)
         else:
             print(f"[SEARCH RESULT] 0 Match(es) found for {user} in {channel} on '{search_term}'")
 

settings.conf.sample

@@ -490,6 +490,57 @@
 # and rebuilt from the lists on disk (#628); the copy can be deleted.
 #LIST_INDEX_FILE = ./data/list_index.db
 
+# Add each MP3 and FLAC file's length and quality after its size in your list,
+# e.g. 10.3MB 4m31s 320/44.1/JS. Every audio file has to be read once: on a
+# large library, or one on a network drive, that takes several rebuilds, each
+# limited by the reading time set under List rebuild. After that only new
+# files are read.
+#
+# Duration and quality after the size on the list's MP3 and FLAC rows (#567):
+# "::INFO:: 10.3MB 4m31s 320/44.1/JS" - the spelling other servers' lists use.
+# Off by default because it OPENS every audio file, where the scan otherwise
+# asks for nothing but sizes: the first rebuild with it on takes noticeably
+# longer. What it read is kept in LIST_AUDIO_INFO_CACHE, checked against each
+# file's size and modification time, so later rebuilds open only new or changed
+# files. Read with the standard library (audio_info.py); a file it cannot read
+# keeps its size and nothing more.
+# Put duration and bitrate after the size on MP3 and FLAC rows
+#LIST_SHOW_AUDIO_INFO = false
+
+# Where the length and quality read from your audio files are kept between
+# rebuilds. Safe to delete; the next rebuild reads every file again.
+#
+# One row per audio file in the lists (about 150 bytes each). Safe to delete:
+# the next rebuild reads every file again.
+#LIST_AUDIO_INFO_CACHE = ./data/audio_info.db
+
+# How many audio files are read at once for their length and quality. On a
+# network drive most of the time is waiting, so this is 64 by default; lower
+# it only if a slow or small setup does not benefit. The rebuild says the rate
+# it got, to compare. 1 to 128.
+#
+# How many audio files are read at once (#914). On a network mount (NFS, SMB)
+# the time goes into round trips, which overlap. Measured on a real 64,136-file
+# NFS library: one at a time 9.8 files a second, 16 about 73, 64 about 236 (the
+# last partly on a cache warmed by the run before). 64 by default - a plain
+# disk answers 64 requests as readily as it answers 16; on a very old drive
+# or a very small library, lower it. The rebuild's last line says the rate it
+# got, to compare. 1 to 128.
+# Audio files read at once for length and quality
+#LIST_AUDIO_INFO_THREADS = 64
+
+# The longest one rebuild spends reading audio files it has not read before,
+# since searches wait while it runs. Past it, the list is published and the
+# rest are read by the next rebuild. 0 means no limit.
+#
+# The most time one rebuild spends reading audio files it has not read before
+# (#914). A rebuild pauses searches and requests, and the first one with
+# LIST_AUDIO_INFO on has the whole library to read: past this, the list
+# publishes with what was read and the rest wait for the next rebuild. 0 = no
+# limit.
+# Minutes one rebuild may spend reading new audio files; 0 = no limit
+#LIST_AUDIO_INFO_MINUTES = 5
+
 # Where the count of how often each file was sent is kept, for the Most
 # downloaded table.
 #