Upload restricted URN-covered theses to the SNL server - set up a daily background job

[mmo]

Problem description

When a URN-covered thesis is deposited in SONAR, in case it contains files with restricted access, the latter must be manually uploaded to the Swiss National Library's FTP server, for long-term archiving. (Note: for public access theses, this step is not required, as the SNL can fetch the files directly from SONAR).

Missing feature

In order to avoid this manual operation, a daily background job should be set up that uploads all the files from newly created URN-covered theses to the SNL's FTP server.

Solution draft

Here is the code for fetching all URN-covered theses with resctricted access files created after a given timestamp:

from elasticsearch_dsl import Q
from sonar.modules.documents.api import DocumentSearch, DocumentRecord

filters = Q(
    "nested",
    path="identifiedBy",
    query=Q("term", identifiedBy__type="bf:Urn"),
)
filters = filters &Q(
    "range",
    _updated={"gte": '2026-05-09T08:03:48Z'}
)

s = DocumentSearch()
s = s.filter(filters)
print(s.count())

for hit in s.source(["pid", "identifiedBy"]).scan():
    doc = DocumentRecord.get_record_by_pid(hit.pid)
    if not doc.is_open_access():
        print(hit.pid, DocumentRecord.get_rero_urn_code(doc))