The List Browser's search can ask only the bots that are online (#926)

docs/UPDATES-PUBLIC.md

@@ -2,6 +2,7 @@
 
 ## Unreleased
 
+- **Added: "Online only" in the List Browser's search.** Tick it to search only the lists of bots that are in the channel right now - the ones you can actually download from today.
 - **Added: a "Responsible use" note in the README.** DCCore is for sharing files you have the right to share; you decide what your bot offers and are responsible for it; the contributors do not host, control or endorse anything any bot shares.
 - **Added: a bot that keeps failing is paused, and a full disk makes downloads wait.** If DCCore cannot connect to a bot three times in a row, that bot is paused - its downloads stay queued with a **Resume** button on the Downloads page - instead of failing file after file. And when the drive fetched files go to has less than 200 MB free, downloads wait instead of failing, and carry on by themselves once there is space.
 - **Added: the Downloads queue looks after itself.** Files you queue from another bot wait for it if it is offline and are asked for a minute after it comes back; only three are asked of one bot at a time (*Files asked of one bot at once*), the next going when one arrives, so a big selection no longer earns "queue full"; a "busy" answer is asked again later, three times; and unfinished downloads survive a restart. The Downloads page says why each one is waiting.

docs/UPDATES.md

@@ -4,6 +4,16 @@ All version changes, optimizations, and bug fixes made over time in the DCCore p
 
 ## 🟨 Unreleased
 
+### 🟢 The List Browser's search can ask only the bots that are online (#926)
+
+Item 7 of #926, AutoGet's "Online Only". The cross-list filter searched every list held, including bots that left
+days ago - so the best match was often a file nobody could send. An **Online only** switch beside the filter asks
+`/api/filelists/search?online=1`, and `build_crosslist_search_payload(online_only=True)` keeps only the lists whose
+bot is in one of our channels right now (`dcc.user_is_present_in_ram()`, the same presence every request is checked
+against); the others are reported with the lists that had no match, so the sidebar dims them. Off, it is exactly as
+before. `tests/test_the_search_can_ask_only_bots_that_are_online.py` (5): off, on, nobody online, and the route and
+page passing the switch; removing the presence check fails two of them.
+
 ### ⚖️ The README says what DCCore is for, and who is responsible for what it shares
 
 The only legal text was the GPL, which covers the code and says nothing about use. A **Responsible use** section,

tests/test_the_search_can_ask_only_bots_that_are_online.py

@@ -0,0 +1,66 @@
+"""#926 item 7: the cross-list search can be limited to bots that are online.
+
+AutoGet's "Online Only": of every list held, only the ones whose bot is in one
+of our channels right now - the ones a request can reach today. The others are
+reported with the lists that had no match, so the sidebar dims them.
+"""
+
+import io
+import os
+import sys
+import unittest
+
+REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+if REPO_ROOT not in sys.path:
+    sys.path.insert(0, REPO_ROOT)
+
+import defaults as config  # noqa: E402
+import webserver  # noqa: E402
+
+from tests.test_crosslist_search import IndexCase  # noqa: E402
+
+
+class OnlineOnly(IndexCase):
+    def setUp(self):
+        super().setUp()
+        self.index("ServerOne", "01 - Opening Song.flac")
+        self.index("ServerTwo", "02 - Opening Song.flac")
+        self.hold("ServerOne", "ServerTwo")
+        config.channel_users["#chan"] = {"serverone", "someuser"}
+
+    def bots(self, payload):
+        return sorted({group["bot"] for group in payload["folders"]})
+
+    def test_off_every_list_is_searched(self):
+        payload = webserver.build_crosslist_search_payload("opening song")
+        self.assertEqual(self.bots(payload), ["ServerOne", "ServerTwo"])
+
+    def test_on_only_bots_in_a_channel_are(self):
+        payload = webserver.build_crosslist_search_payload("opening song", online_only=True)
+        self.assertEqual(self.bots(payload), ["ServerOne"])
+        self.assertIn("servertwo", payload["empty"], "the sidebar dims it")
+        self.assertNotIn("servertwo", payload["matched"])
+
+    def test_with_nobody_online_nothing_matches_and_everything_is_dimmed(self):
+        config.channel_users["#chan"] = {"someuser"}
+        payload = webserver.build_crosslist_search_payload("opening song", online_only=True)
+        self.assertEqual(payload["folders"], [])
+        self.assertEqual(sorted(payload["empty"]), ["serverone", "servertwo"])
+
+
+class TheRouteAndThePage(unittest.TestCase):
+    def read(self, *parts):
+        with io.open(os.path.join(REPO_ROOT, *parts), encoding="utf-8") as handle:
+            return handle.read()
+
+    def test_the_route_passes_the_switch(self):
+        self.assertIn('online_only=request.args.get("online", "") in ("1", "true")',
+                      self.read("webserver.py"))
+
+    def test_the_page_sends_it(self):
+        self.assertIn('(state.filelistsOnlineOnly ? "&online=1" : "")', self.read("web", "app.js"))
+        self.assertIn('id="filelists-online-only"', self.read("web", "index.html"))
+
+
+if __name__ == "__main__":
+    unittest.main()

web/app.js

@@ -139,6 +139,7 @@
     filelistsBody:document.getElementById("filelists-body"),
     filelistsFilterInput: document.getElementById("filelists-filter-input"),
     filelistsFilterClear: document.getElementById("filelists-filter-clear"),
+    filelistsOnlineOnly: document.getElementById("filelists-online-only"),
     filelistsFilterStatus: document.getElementById("filelists-filter-status"),
     filelistsFilterActions: document.getElementById("filelists-filter-actions"),
     filelistsFilterAll: document.getElementById("filelists-filter-all"),
@@ -1554,6 +1555,14 @@
     rerenderFromFilterPayload();
   });
 
+  // #926: search only the lists of bots that are in a channel right now.
+  if (el.filelistsOnlineOnly) {
+    el.filelistsOnlineOnly.addEventListener("change", function () {
+      state.filelistsOnlineOnly = el.filelistsOnlineOnly.checked;
+      runFilelistsFilter();
+    });
+  }
+
   el.filelistsFilterClear.addEventListener("click", function () {
     el.filelistsFilterInput.value = "";
     state.filelistsFilter = "";
@@ -2982,7 +2991,8 @@
         // spans every list held, so "which bot am I looking at" stops being
         // the question while a term is set. The sidebar still shows which
         // bots have matches - see applyFilterHighlight().
-        url = "/api/filelists/search?q=" + encodeURIComponent(filter);
+        url = "/api/filelists/search?q=" + encodeURIComponent(filter) +
+          (state.filelistsOnlineOnly ? "&online=1" : "");
       } else {
         var base;
         if (isOwnSource(source)) {

web/index.html

@@ -257,6 +257,11 @@ <h2 class="panel-title" data-i18n="download.panelTitle">Downloads (fetched from
                    autocomplete="off" aria-describedby="filelists-filter-status">
             <button type="button" class="btn btn-small" id="filelists-filter-clear"
                     data-i18n="common.clear" hidden>Clear</button>
+            <!-- #926: only the lists of bots in a channel right now. -->
+            <label class="filelists-online-only">
+              <input type="checkbox" id="filelists-online-only">
+              <span data-i18n="filelists.onlineOnly">Online only</span>
+            </label>
           </div>
           <p class="filelists-filter-status" id="filelists-filter-status" hidden></p>
           <div class="filelists-filter-actions" id="filelists-filter-actions" hidden>

web/lang/en.json

@@ -105,6 +105,7 @@
   "filelists.fetch": "Fetch",
   "filelists.fetchPlaceholder": "Fetch a bot's list… (nick)",
   "filelists.filterPlaceholder": "Filter every list you hold…",
+  "filelists.onlineOnly": "Online only",
   "filelists.legendCannotTell": "cannot tell",
   "filelists.legendHereNow": "here now",
   "filelists.legendListCurrent": "list current",

web/lang/es.json

@@ -105,6 +105,7 @@
   "filelists.fetch": "Obtener",
   "filelists.fetchPlaceholder": "Obtener la lista de un bot… (apodo)",
   "filelists.filterPlaceholder": "Filtrar todas las listas que tiene…",
+  "filelists.onlineOnly": "Solo en línea",
   "filelists.legendCannotTell": "no se puede saber",
   "filelists.legendHereNow": "presente ahora",
   "filelists.legendListCurrent": "lista actualizada",

web/lang/fr.json

@@ -105,6 +105,7 @@
   "filelists.fetch": "Récupérer",
   "filelists.fetchPlaceholder": "Récupérer la liste d'un bot… (pseudo)",
   "filelists.filterPlaceholder": "Filtrer toutes les listes que vous détenez…",
+  "filelists.onlineOnly": "En ligne seulement",
   "filelists.legendCannotTell": "impossible à dire",
   "filelists.legendHereNow": "présent maintenant",
   "filelists.legendListCurrent": "liste à jour",

web/style.css

@@ -1141,6 +1141,16 @@ html, body {
   margin-bottom: 6px;
 }
 
+/* #926: the "Online only" switch beside the filter. */
+.filelists-online-only {
+  display: flex;
+  align-items: center;
+  gap: 6px;
+  white-space: nowrap;
+  font-size: 13px;
+  color: var(--text-dim);
+}
+
 .filelists-filter .search-input {
   flex: 1;
   padding: 9px 12px;

webserver.py

@@ -1065,7 +1065,7 @@ def mark_rows_with_fetch_state(rows, marks, bot=None):
     return rows
 
 
-def build_crosslist_search_payload(term, limit=None):
+def build_crosslist_search_payload(term, limit=None, online_only=False):
     """GET /api/filelists/search: one term against every list we hold.
 
     Returns the SAME shape as GET /api/filelists - {"folders": [...]} of
@@ -1090,6 +1090,10 @@ def build_crosslist_search_payload(term, limit=None):
     Scoped to the lists actually held, never to whatever is in the index. The
     two can drift - a list file removed by hand, a reset store - and a row for
     a list we no longer have offers a file that cannot be requested.
+
+    `online_only` (#926, AutoGet's "Online Only"): only the lists of bots in
+    one of our channels right now - the ones a request can reach today. The
+    others are reported with the empty ones, so the sidebar dims them.
     """
     import list as list_mod
     import list_index
@@ -1130,6 +1134,14 @@ def build_crosslist_search_payload(term, limit=None):
             source = list_fetch.index_key(name, marker)
             held[source.lower()] = source
 
+    offline = []
+    if online_only:
+        import dcc
+        for key, source in list(held.items()):
+            if not dcc.user_is_present_in_ram(source.split("/", 1)[0]):
+                offline.append(key)
+                del held[key]
+
     empty_payload = {
         "term": str(term or ""),
         "terms": terms,
@@ -1139,14 +1151,15 @@ def build_crosslist_search_payload(term, limit=None):
         "returned": 0,
         "truncated": False,
         "matched": [],
-        "empty": sorted(held),
+        "empty": sorted(list(held) + offline),
     }
     if not terms or not held:
         return empty_payload
 
     names = list(held.values())
     rows = list_index.search(terms, limit=limit, bots=names)
     matched, empty = list_index.bots_with_a_match(terms, names)
+    empty = list(empty) + offline
 
     # One group per (bot, folder), in the order the index returned them so
     # that a bot's own list order survives rather than being re-sorted into
@@ -4241,7 +4254,8 @@ def api_filelists_search():
             _offset, limit = parse_pagination_params(
                 None, request.args.get("limit"))
             return jsonify(build_crosslist_search_payload(
-                request.args.get("q", ""), limit))
+                request.args.get("q", ""), limit,
+                online_only=request.args.get("online", "") in ("1", "true")))
 
         @app.route("/api/filelists/bot/<nick>")
         def api_filelists_bot(nick):