From fdab35ae90c87c0ded6679f06e362e1391019fe6 Mon Sep 17 00:00:00 2001 From: chchatzop <35049131+chchatzop@users.noreply.github.com> Date: Fri, 25 Sep 2026 00:59:04 +0300 Subject: [PATCH 1/2] Every feature in one place: the roadmap catches up docs/FUTURE.md's Implemented section is the project's feature list, and it had fallen behind: the fetch replies, the self-running fetch queue, paused bots, the List Browser, automatic list refresh and grabbing, the rebuild that keeps answering, the parallel scan, audio info, quoted searches and the update check are now listed. A fetched archive keeping only one list leaves Planned (done in #399). The README's summary points to the roadmap as the full list. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01AP6LSxkr4n9dMFNSNMogmW --- README.md | 6 +++--- docs/FUTURE.md | 15 ++++++++++++--- docs/UPDATES-PUBLIC.md | 1 + docs/UPDATES.md | 13 +++++++++++++ 4 files changed, 29 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 8b2c3c3..d9f9f20 100644 --- a/README.md +++ b/README.md @@ -11,14 +11,14 @@ It sits in a channel, advertises a catalogue, and serves files over DCC to whoev ## What it does - **Serves files over DCC** — per-user and global queues, configurable slots, `!rar` album packing on demand, and DCC RESUME so an interrupted download continues instead of starting over -- **Keeps a searchable master list** in `.txt`, `.zip` and `.rar`, rebuilt atomically so a failed scan never overwrites a good one -- **Fetches from other bots** — request a file or a whole list, or broadcast one `@find` and collect every reply. In this bot's own search, words in quotes must appear together: `@find "metal church"` +- **Keeps a searchable master list** in `.txt`, `.zip` and `.rar`, rebuilt atomically so a failed scan never overwrites a good one - and keeps answering from the current list while it rebuilds +- **Fetches from other bots** — request a file or a whole list, or broadcast one `@find` and collect every reply. A queued request waits its turn, a bot that is offline is asked again when it returns, and the lists you hold can keep themselves up to date. In this bot's own search, words in quotes must appear together: `@find "metal church"` - **Remembers people** — a user who disconnects keeps their queue for five minutes and resumes on rejoin - **Defends itself** — rolling flood windows, escalating mutes, hostmask bans, and every other bot treated as untrusted - **Is operable** — an authenticated DCC CHAT console, an optional web dashboard, live `!rehash`, and statistics that survive a restart - **Runs where you run it** — long paths, non-ASCII filenames, and Linux, Windows and macOS all in CI on every commit -See [docs/FUTURE.md](docs/FUTURE.md) for the full picture, including what is *not* built yet. +**Every feature, in one place:** the *Implemented* section of [docs/FUTURE.md](docs/FUTURE.md), which also lists what is *not* built yet. ## Quick start diff --git a/docs/FUTURE.md b/docs/FUTURE.md index 7d5a448..3475c55 100644 --- a/docs/FUTURE.md +++ b/docs/FUTURE.md @@ -21,7 +21,10 @@ What DCCore does today, and what it does not do yet. - **One master list**, rebuilt by `!update`, from the dashboard, or by itself on a schedule (`LIST_REBUILD_SCHEDULE`: daily, weekly, monthly or every N hours), published atomically so a failed scan never overwrites a good index. - **Three formats** — `.txt`, `.zip` and `.rar`, all built every time; `LIST_FORMAT` picks which one is offered. -- **Search** — `@find ` against the master list, with results fitted to the IRC line limit. +- **Search** — `@find ` against the master list, with results fitted to the IRC line limit. Words in quotes must appear together, in that order: `@find "metal church" 1986`. +- **Searching and downloading go on while the list rebuilds.** The new list is built beside the one people already have, and the bot pauses only for the few seconds it takes to swap it in (`PAUSE_ON_UPDATE`; `PAUSE_FOR_WHOLE_UPDATE` brings back the old whole-rebuild pause). +- **Several folders scanned at once** (`LIST_SCAN_THREADS`, 16 by default), which is what makes a rebuild over a network drive shorter. +- **Length and quality in the list, if you want them** (`LIST_SHOW_AUDIO_INFO`) — every MP3 and FLAC row gets its duration and bitrate after the size, `::INFO:: 10.3MB 4m31s 320/44.1/JS`, read once per file and remembered. - **Every folder heading says what it holds** — `14 files, 1.20GB` on its own line under the heading, placed so that every program that reads these lists (other DCCore bots, AutoQ, DCCore's own request handling) ignores it. Companion files (`.srt`, `.nfo`, `.sfv`…) travel with the film they belong to when the video list is split out, and stay with an album otherwise. - **A partially unreadable library fails the rebuild** rather than silently publishing a truncated list. @@ -66,6 +69,12 @@ Two pieces were worth doing carefully rather than quickly, and one of them turne - **Broadcast search** — send one `@find` to a channel and collect every bot's reply, grouped under each bot's own parsed header. - **Both DCC directions** — active and passive/reverse SEND, since bots behind NAT use the latter. - **Hostile-input handling** — every other bot is treated as untrusted: admission control, size caps, and zip-slip / zip-bomb guards on any archive received. +- **Their answers are understood** — what OmeNServE, SDFind, SpR, BWI and DCCore itself reply to a request. A queued request shows its place in their queue on the Downloads page and waits for its turn (`FETCH_QUEUED_TIMEOUT`, 12 hours by default); "I don't have that file" or "queue full" ends it at once, in the server's own words. +- **The download queue looks after itself.** A file queued from a bot that is offline waits, and is asked for a minute after the bot comes back. Only `FETCH_MAX_PER_BOT` (3) are asked of one bot at a time, the next going when one arrives, so a big selection does not earn "queue full". A "busy" answer is asked again, three times, ten minutes apart. Unfinished downloads survive a restart. +- **A bot that cannot be reached is paused**, after three failed connections in a row, with a **Resume** button on the Downloads page, instead of failing file after file. When the drive fetched files go to has less than 200 MB free, downloads wait and carry on by themselves once there is room. +- **The List Browser** — every list you have fetched, one row per bot, with a tab for each list its archive holds (music, RAR, video) rather than only the largest. A light says whether the bot's advert shows a newer list than yours; a list you have not opened yet says *New*; each online bot's advertised free slots, queue and speed are shown, as that bot advertises them. One filter searches every held list at once, from an index built as each list arrives, and can be limited to bots that are online now. A bot that never advertises can be added by hand, and a list can be fetched again, or removed, from its own row. +- **Held lists can keep themselves up to date** (`AUTO_REFETCH_LISTS`, off by default): a list is fetched again when its bot's advert shows a different date or file count, or - for a bot whose advert shows no date - once it is 14 days old. Never more often than `AUTO_REFETCH_INTERVAL_HOURS`, and at most `AUTO_REFETCH_MAX_PER_RUN` at a time. +- **Lists can be grabbed by themselves** (`AUTO_GRAB_LISTS`, off by default), on AutoGet's rules: the list of a bot you have none from is asked for one at a time, at most one every 10 minutes, after a random 5-360 second wait, not at all if someone else just asked that bot, and at most three times per bot, 30 minutes apart. Small, slow and "servers only" bots can be skipped, and a list you removed is not grabbed back. ### Operating it @@ -78,7 +87,8 @@ Two pieces were worth doing carefully rather than quickly, and one of them turne - **The launcher is the install.** Extract, double-click (`start-dccore.bat`, `start-dccore.sh`, `start-dccore.command`): the first run opens a setup page in the browser — nickname, server, channels, your nick, the password, the music folder, the dashboard, each with the same **?** explanation the Settings page has, in English, French or Spanish — loopback-only, one-shot, behind a one-time code in the link, and the bot starts the moment you save. No Flask, or no browser: the same questions in the terminal (`configure.py`). On Windows with no Python at all, the launcher offers to download python.org's installer, checks it against a fingerprint pinned in the script, and runs it with both boxes ticked. Every run after that checks the setup and starts the bot. - **Starting with the system, the firewall, the router.** `install-autostart` scripts for Windows (Task Scheduler), Linux (a systemd user unit) and macOS (launchd), each with a remover, each running the launcher so the working directory is right; `allow-firewall.bat` adds the Windows rule for the bot's ports and the setup check names the `ufw`/`firewall-cmd` lines on Linux; port forwarding explained in plain words in both guides. - **Pre-flight check** — `start-dccore.sh check` verifies the setup without opening a socket, and says what stands between the bound ports and the outside on this OS. -- **Every setting explains itself.** The **?** beside each of the 117 settings on the dashboard — and the comment above it in `settings.conf.sample` — is written for the person running the bot, in English, French and Spanish; the dashboard itself is translated the same three ways. +- **It tells you when a new version is out.** Once a day (`CHECK_FOR_UPDATES`) it asks GitHub for the latest release - one request, carrying nothing about your bot - and says so in the dashboard's sidebar, the console's `status` and the mIRC window, each with a way to check now. +- **Every setting explains itself.** The **?** beside every setting on the dashboard — and the comment above it in `settings.conf.sample` — is written for the person running the bot, in English, French and Spanish; the dashboard itself is translated the same three ways. - **Two configuration mechanisms** — `admin_config.py` for Python, `settings.conf` for plain text; the dashboard and console both write to the latter. - **`!rehash`** reloads code and settings live, preserving queues and transfer state. - **Channel adverts** on a timer, with a per-bot theme (five presets, or your own colours). @@ -172,7 +182,6 @@ works, only that the operator says it exists. ### Smaller things worth having - **PER-LIST file exclusions** (`Exclude = .mpu,.db`) — OmenServe has them per list. `LIST_IGNORED_EXTENSIONS` does this globally; scoping it to one folder is the part still missing. -- **A fetched list keeps only the peer's master.** Since the film-and-series split, a DCCore bot's archive carries two `.txt` files, and `list_fetch` picks one - now the master rather than whichever is larger. The films in the other are dropped from the fetched copy. Reading both into one fetched list changes what `_pick_list_file()` returns and the size ceiling that guards it, so it is a change of its own rather than part of the fix. - **Stealth channels** — serve a channel while advertising nothing in it. - **Multi-network** — real in OmenServe, and it would touch every socket path here. diff --git a/docs/UPDATES-PUBLIC.md b/docs/UPDATES-PUBLIC.md index f0951d4..40db7e4 100644 --- a/docs/UPDATES-PUBLIC.md +++ b/docs/UPDATES-PUBLIC.md @@ -2,6 +2,7 @@ ## Unreleased +- **Docs: every feature is listed in one place again.** The roadmap's *Implemented* section ([docs/FUTURE.md](FUTURE.md)) had fallen behind: it now covers what fetching from other bots does today (queued requests, the self-running download queue, paused bots, the List Browser, automatic refresh and grabbing), the rebuild that keeps answering, faster scans, length and quality in the list, quoted searches and the update check. The README's summary points to it. ## v1.13.1 — The Fetch Queue Looks After Itself - **Added: automatic list grabbing (off by default).** Turn on *Grab the lists of bots you have no list from* (`AUTO_GRAB_LISTS`, under *Grabbing lists*) and DCCore asks for the list of each bot that advertises one and whose list you do not have yet - politely: one at a time, at most one every 10 minutes, after a random wait of 5 seconds to 6 minutes, not at all if someone else just asked that bot, and never more than 3 tries per bot, 30 minutes apart. You can skip small or slow bots, bots in "servers only" mode are always skipped, and a list you remove is not fetched back. diff --git a/docs/UPDATES.md b/docs/UPDATES.md index ce6ae3b..84b0d45 100644 --- a/docs/UPDATES.md +++ b/docs/UPDATES.md @@ -4,6 +4,19 @@ All version changes, optimizations, and bug fixes made over time in the DCCore p ## 🟨 Unreleased +### 📚 Every feature in one place: the roadmap catches up + +Where is every feature written down together? In `docs/FUTURE.md`'s *Implemented* section - the README's "What it +does" is a summary that links to it - and it had fallen behind: the #926 PRs, among others, updated both changelogs +but not the roadmap, which that file's own header asks for in the same commit. Now listed: under *The file list*, quoted `@find` phrases, the rebuild that keeps answering (#923), the +parallel scan (#922) and audio length/quality (#914); under *Receiving files from other bots*, understood replies +(#927), the self-running queue (#928), paused bots and the low-disk wait (#929), the List Browser as a whole +(per-archive tabs #399, freshness, *New*, slots/queue/speed, the cross-list filter and Online only, hand-entered bots), +`AUTO_REFETCH_LISTS` with the 14-day rule and `AUTO_GRAB_LISTS`; under *Operating it*, the update check (#572). "The +117 settings" is now "every setting" (131 today, and a count there goes stale with every one added). *A fetched list +keeps only the peer's master* leaves *Planned*: every list in the archive has been kept since #399. The README's +summary gains the rebuild and the fetch queue, and says the roadmap is the full list. + ## 🟩 v1.13.1 (2026-09-24) - "The Fetch Queue Looks After Itself" ### 🧲 Lists are grabbed automatically, on AutoGet's rules (#926) From af12e7ca669ee650bcf9a1a03550356cec611d50 Mon Sep 17 00:00:00 2001 From: chchatzop <35049131+chchatzop@users.noreply.github.com> Date: Fri, 25 Sep 2026 08:30:39 +0300 Subject: [PATCH 2/2] The roadmap describes the slots line and Online only as merged (#943, #948) Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01AP6LSxkr4n9dMFNSNMogmW --- docs/FUTURE.md | 2 +- docs/UPDATES.md | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/FUTURE.md b/docs/FUTURE.md index 3475c55..e9323af 100644 --- a/docs/FUTURE.md +++ b/docs/FUTURE.md @@ -72,7 +72,7 @@ Two pieces were worth doing carefully rather than quickly, and one of them turne - **Their answers are understood** — what OmeNServE, SDFind, SpR, BWI and DCCore itself reply to a request. A queued request shows its place in their queue on the Downloads page and waits for its turn (`FETCH_QUEUED_TIMEOUT`, 12 hours by default); "I don't have that file" or "queue full" ends it at once, in the server's own words. - **The download queue looks after itself.** A file queued from a bot that is offline waits, and is asked for a minute after the bot comes back. Only `FETCH_MAX_PER_BOT` (3) are asked of one bot at a time, the next going when one arrives, so a big selection does not earn "queue full". A "busy" answer is asked again, three times, ten minutes apart. Unfinished downloads survive a restart. - **A bot that cannot be reached is paused**, after three failed connections in a row, with a **Resume** button on the Downloads page, instead of failing file after file. When the drive fetched files go to has less than 200 MB free, downloads wait and carry on by themselves once there is room. -- **The List Browser** — every list you have fetched, one row per bot, with a tab for each list its archive holds (music, RAR, video) rather than only the largest. A light says whether the bot's advert shows a newer list than yours; a list you have not opened yet says *New*; each online bot's advertised free slots, queue and speed are shown, as that bot advertises them. One filter searches every held list at once, from an index built as each list arrives, and can be limited to bots that are online now. A bot that never advertises can be added by hand, and a list can be fetched again, or removed, from its own row. +- **The List Browser** — every list you have fetched, one row per bot, with a tab for each list its archive holds (music, RAR, video) rather than only the largest. A light says whether the bot's advert shows a newer list than yours; a list you have not opened yet says *New*; click an online bot and its advertised free slots, queue and speed appear under it. One filter searches every held list at once, from an index built as each list arrives, and **Online only** takes the bots that are not here off both the sidebar and the search. A bot that never advertises can be added by hand, and a list can be fetched again, or removed, from its own row. - **Held lists can keep themselves up to date** (`AUTO_REFETCH_LISTS`, off by default): a list is fetched again when its bot's advert shows a different date or file count, or - for a bot whose advert shows no date - once it is 14 days old. Never more often than `AUTO_REFETCH_INTERVAL_HOURS`, and at most `AUTO_REFETCH_MAX_PER_RUN` at a time. - **Lists can be grabbed by themselves** (`AUTO_GRAB_LISTS`, off by default), on AutoGet's rules: the list of a bot you have none from is asked for one at a time, at most one every 10 minutes, after a random 5-360 second wait, not at all if someone else just asked that bot, and at most three times per bot, 30 minutes apart. Small, slow and "servers only" bots can be skipped, and a list you removed is not grabbed back. diff --git a/docs/UPDATES.md b/docs/UPDATES.md index c4e510f..eea8e8e 100644 --- a/docs/UPDATES.md +++ b/docs/UPDATES.md @@ -11,7 +11,8 @@ does" is a summary that links to it - and it had fallen behind: the #926 PRs, am but not the roadmap, which that file's own header asks for in the same commit. Now listed: under *The file list*, quoted `@find` phrases, the rebuild that keeps answering (#923), the parallel scan (#922) and audio length/quality (#914); under *Receiving files from other bots*, understood replies (#927), the self-running queue (#928), paused bots and the low-disk wait (#929), the List Browser as a whole -(per-archive tabs #399, freshness, *New*, slots/queue/speed, the cross-list filter and Online only, hand-entered bots), +(per-archive tabs #399, freshness, *New*, slots/queue/speed under a clicked bot #943, the cross-list filter and +Online only #931/#948, hand-entered bots), `AUTO_REFETCH_LISTS` with the 14-day rule and `AUTO_GRAB_LISTS`; under *Operating it*, the update check (#572). "The 117 settings" is now "every setting" (131 today, and a count there goes stale with every one added). *A fetched list keeps only the peer's master* leaves *Planned*: every list in the archive has been kept since #399. The README's