20261016 - Push fleet snapshots to retina-server

README.md

@@ -53,6 +53,7 @@ Edit `.env`:
 | `MENDER_PAT` | Yes | Personal Access Token from Mender UI |
 | `MENDER_SERVER` | No | Default: `https://hosted.mender.io` |
 | `NODE_ID_PREFIX` | No | Only accept nodes with this prefix (e.g., `ret`) |
+| `RETINA_SYNC_TARGETS` | No | Comma-separated `url=token` pairs, one per retina-server environment. Unset means no snapshot is pushed. See below |
 
 ### How it works
 
@@ -61,6 +62,48 @@ Edit `.env`:
 3. Filters by `node_id` prefix (if configured)
 4. Accepts matching devices
 
+### Fleet snapshot to retina-server
+
+After every sweep, `auto_accept.py` sends retina-server a snapshot of the whole
+Mender fleet. It runs even if the deploy phase fails. The script reads every
+device from Mender (devauth v2 plus inventory v1, paged, no status filter) and
+POSTs one JSON snapshot to each target. `NODE_ID_PREFIX` only limits which
+devices get accepted. The snapshot is not filtered by it.
+
+`RETINA_SYNC_TARGETS` is a comma-separated list of `url=token` pairs, one per
+retina-server environment:
+
+```
+RETINA_SYNC_TARGETS=https://retina.example/api/internal/mender/snapshot=<prod token>,https://staging.retina.example/api/internal/mender/snapshot=<staging token>,https://test.retina.example/api/internal/mender/snapshot=<test token>
+```
+
+* The url is the full endpoint, `https://<host>/api/internal/mender/snapshot`.
+  Plain `http` is only accepted for localhost.
+* The token is that environment's `MENDER_SYNC_TOKEN`. Each pair is split at the
+  first `=`, so a token may contain `=`.
+* Prod and staging are the usual targets. Test is optional. The test server
+  stays on `MENDER_SOURCE=api`, so it doesn't need one.
+* If the variable is unset or empty, nothing is fetched or pushed.
+
+Each POST has a 10 second timeout. A failing target doesn't stop the others. A
+409 means that server already has a newer snapshot, and counts as success. The
+result for each target is printed with the accepted, rejected and absent
+counts the server returned. A non-zero rejected count means a device was dropped
+or stored with an issue, and the retina-server log names it.
+
+The POST carries a browser User-Agent, because Cloudflare answers a script's
+default one with 403 error 1010.
+
+There is no retry queue. The next sweep, 30 seconds later, is the retry. A
+target that stays unreachable shows up on the retina-server side as a stale
+mirror, and that is where the alerting lives.
+
+If any Mender fetch fails, nothing is pushed that sweep. A partial fleet would
+mark devices absent.
+
+The Mender token stays on this host and is never sent to retina-server. The sync
+tokens are separate, one per environment.
+
 
 ## Failed-deployment retry
 

mender-auto-accept/.env.example

@@ -63,3 +63,12 @@ NODE_ID_PREFIX=ret
 # StateDirectory= is picked up automatically. Set it only for a hand run that
 # needs the counts somewhere specific.
 # DEPLOY_RETRY_STATE_FILE=
+
+
+# --- Fleet snapshot to retina-server (mender_snapshot.py) ---
+
+# Comma-separated url=token pairs, one per retina-server environment. The url is
+# the full endpoint, the token is that environment's MENDER_SYNC_TOKEN. https is
+# required (http only for localhost). Test is optional. Unset or empty means no
+# snapshot is fetched or pushed.
+# RETINA_SYNC_TARGETS=https://retina.example/api/internal/mender/snapshot=<prod token>,https://staging.retina.example/api/internal/mender/snapshot=<staging token>,https://test.retina.example/api/internal/mender/snapshot=<test token>

mender-auto-accept/auto_accept.py

@@ -25,6 +25,7 @@
 import sys
 import time
 
+import mender_snapshot
 import requests
 
 MENDER_SERVER = os.environ.get("MENDER_SERVER", "https://hosted.mender.io")
@@ -420,4 +421,11 @@ def main() -> int:
 
 
 if __name__ == "__main__":
-    sys.exit(main())
+    # The snapshot goes out even when the pass fails or raises: it reads Mender
+    # itself, and the mirror goes stale without it. It never raises, so the
+    # exit status and any traceback are still main's.
+    try:
+        exit_code = main()
+    finally:
+        mender_snapshot.run()
+    sys.exit(exit_code)