Update the minimal transfer scripts examples (#1419)

Author: sirosen

docs/examples/minimal_transfer_script/index.rst

@@ -10,7 +10,6 @@ The following is an extremely minimal script to demonstrate a file transfer
 using the :class:`TransferClient <globus_sdk.TransferClient>`.
 
 It uses the tutorial client ID from the :ref:`tutorials <tutorials>`.
-For simplicity, the script will prompt for login on each use.
 
 .. note::
     You will need to replace the values for ``source_collection_id`` and
@@ -20,47 +19,35 @@ For simplicity, the script will prompt for login on each use.
     :caption: ``transfer_minimal.py`` [:download:`download <transfer_minimal.py>`]
     :language: python
 
-
-Minimal File Transfer Script Handling ConsentRequired
------------------------------------------------------
-
-The above example works with certain endpoint types, but will fail if either
-the source or destination endpoint requires a ``data_access`` scope. This
-requirement will cause the Transfer submission to fail with a
-``ConsentRequired`` error.
-
-The example below catches the ``ConsentRequired`` error and retries the
-submission after a second login.
-
-This kind of "reactive" handling of ``ConsentRequired`` is the simplest
-strategy to design and implement.
-
-We'll also enhance the example to take endpoint IDs from the command line.
-
-.. literalinclude:: transfer_consent_required_reactive.py
-    :caption: ``transfer_consent_required_reactive.py`` [:download:`download <transfer_consent_required_reactive.py>`]
-    :language: python
-
-
 Best-Effort Proactive Handling of ConsentRequired
 -------------------------------------------------
 
 The above example works in most cases, and especially when there is a low cost
-to failing and retrying an activity.
+to failing and retrying an activity. The ``auto_redrive_gares`` flag enables a
+behavior which will prompt the user for a fresh login if they are missing
+consents for access to various collections.
 
-However, in some cases, responding to ``ConsentRequired`` errors when the task
-is submitted is not acceptable. For example, for scripts used in batch job
-systems, the user cannot respond to the error until the job is already
-executing. The user would rather handle such issues when submitting their job.
+However, in some cases, responding to missing consents when the task is
+submitted is not acceptable. For example, for scripts used in batch job systems,
+the user cannot respond to the error until the job is already executing. The
+user would rather handle such issues when submitting their job.
 
-``ConsentRequired`` errors in this case can be avoided on a best-effort basis.
-Note, however, that the process for consenting ahead of time is more error
-prone and complex.
+The service still relies on ``ConsentRequired`` errors to indicate that some
+additional user consent is needed. But we can intentionally trigger them early
+to control when the user is prompted to resolve them.
+
+The example below tries an ``ls`` operation before starting to build the task
+data. If the ``ls`` fails with ``ConsentRequired``, the user can be put through
+the relevant login flow. Other errors (e.g., bad permissions) are suppressed, as
+they probably aren't relevant to the user.
+
+.. note::
+    The ``UserApp`` object is instantiated a second time, later in the script, to
+    actually start the transfer. This loads the same tokens from the earlier login
+    via the default token storage in ``~/.globus/``.
 
-The example below enhances the previous reactive error handling to try an
-``ls`` operation before starting to build the task data. If the ``ls`` fails
-with ``ConsentRequired``, the user can be put through the relevant login flow.
-And if not, we can relatively safely assume that any errors are not relevant.
+    To manage tokens in another way, please see the documentation on
+    :ref:`Token Storages <token_storages>`.
 
 .. literalinclude:: transfer_consent_required_proactive.py
     :caption: ``transfer_consent_required_proactive.py`` [:download:`download <transfer_consent_required_proactive.py>`]

docs/examples/minimal_transfer_script/transfer_consent_required_proactive.py

@@ -3,58 +3,47 @@
 import globus_sdk
 from globus_sdk.scopes import TransferScopes
 
+# do basic argument parsing
 parser = argparse.ArgumentParser()
 parser.add_argument("SRC")
 parser.add_argument("DST")
 args = parser.parse_args()
 
+# tutorial client ID (we recommend replacing this with your own client)
 CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"
-auth_client = globus_sdk.NativeAppAuthClient(CLIENT_ID)
+APP_NAME = "proactive-transfer-consent-example"
 
 
-# we will need to do the login flow potentially twice, so define it as a
-# function
+# Try an ls on the source and destination to see if ConsentRequired errors are raised --
+# if they are, a fresh login flow will *not* be triggered.
 #
-# we default to using the Transfer "all" scope, but it is settable here
-# look at the ConsentRequired handler below for how this is used
-def login_and_get_transfer_client(*, scopes=TransferScopes.all):
-    # note that 'requested_scopes' can be a single scope or a list
-    # this did not matter in previous examples but will be leveraged in
-    # this one
-    auth_client.oauth2_start_flow(requested_scopes=scopes)
-    authorize_url = auth_client.oauth2_get_authorize_url()
-    print(f"Please go to this URL and login:\n\n{authorize_url}\n")
-
-    auth_code = input("Please enter the code here: ").strip()
-    tokens = auth_client.oauth2_exchange_code_for_tokens(auth_code)
-    transfer_tokens = tokens.by_resource_server["transfer.api.globus.org"]
-
-    # return the TransferClient object, as the result of doing a login
-    return globus_sdk.TransferClient(
-        authorizer=globus_sdk.AccessTokenAuthorizer(transfer_tokens["access_token"])
-    )
-
-
-# get an initial client to try with, which requires a login flow
-transfer_client = login_and_get_transfer_client()
-
-# now, try an ls on the source and destination to see if ConsentRequired
-# errors are raised
-consent_required_scopes = []
-
-
-def check_for_consent_required(target):
-    try:
-        transfer_client.operation_ls(target, path="/")
-    # catch all errors and discard those other than ConsentRequired
-    # e.g. ignore PermissionDenied errors as not relevant
-    except globus_sdk.TransferAPIError as err:
-        if err.info.consent_required:
-            consent_required_scopes.extend(err.info.consent_required.required_scopes)
-
-
-check_for_consent_required(args.SRC)
-check_for_consent_required(args.DST)
+# This is more sophisticated than handling with `redrive_gares=True` and makes
+# sure that the user is only prompted to login *one* extra time, even if both
+# collections require additional consent.
+def probe_for_consent_required(
+    transfer_client: globus_sdk.TransferClient, targets: list[str]
+) -> list[str]:
+    consent_required_scopes: list[str] = []
+
+    for target in targets:
+        try:
+            transfer_client.operation_ls(target, path="/")
+        # catch all errors and discard those other than ConsentRequired
+        # e.g. ignore PermissionDenied errors as not relevant
+        except globus_sdk.TransferAPIError as err:
+            if err.info.consent_required:
+                consent_required_scopes.extend(
+                    err.info.consent_required.required_scopes
+                )
+
+    return consent_required_scopes
+
+
+with globus_sdk.UserApp(APP_NAME, client_id=CLIENT_ID) as app:
+    with globus_sdk.TransferClient(app=app) as transfer_client:
+        consent_required_scopes = probe_for_consent_required(
+            transfer_client, [args.SRC, args.DST]
+        )
 
 # the block above may or may not populate this list
 # but if it does, handle ConsentRequired with a new login
@@ -63,15 +52,21 @@ def check_for_consent_required(target):
         "One of your endpoints requires consent in order to be used.\n"
         "You must login a second time to grant consents.\n\n"
     )
-    transfer_client = login_and_get_transfer_client(scopes=consent_required_scopes)
-
-# from this point onwards, the example is exactly the same as the reactive
-# case, including the behavior to retry on ConsentRequiredErrors. This is
-# not obvious, but there are cases in which it is necessary -- for example,
-# if a user consents at the start, but the process of building task_data is
-# slow, they could revoke their consent before the submission step
-#
-# in the common case, a single submission with no retry would suffice
+    with globus_sdk.UserApp(
+        APP_NAME,
+        client_id=CLIENT_ID,
+        scope_requirements={
+            TransferScopes.resource_server: consent_required_scopes
+            + [TransferScopes.all]
+        },
+    ) as app:
+        app.login()
+
+
+# From this point onwards, the example is exactly the same as the previous scripts.
+# We will *not* set `redrive_gares=True`, on the grounds that if you want to use this
+# in a context like a job submission system, a prompt for login is not helpful if the
+# consent was revoked or insufficient.
 
 task_data = globus_sdk.TransferData(
     source_endpoint=args.SRC, destination_endpoint=args.DST
@@ -81,23 +76,9 @@ def check_for_consent_required(target):
     "/~/example-transfer-script-destination.txt",  # dest
 )
 
+with globus_sdk.UserApp(APP_NAME, client_id=CLIENT_ID) as app:
+    with globus_sdk.TransferClient(app=app) as transfer_client:
+        task_doc = transfer_client.submit_transfer(task_data)
 
-def do_submit(client):
-    task_doc = client.submit_transfer(task_data)
-    task_id = task_doc["task_id"]
-    print(f"submitted transfer, task_id={task_id}")
-
-
-try:
-    do_submit(transfer_client)
-except globus_sdk.TransferAPIError as err:
-    if not err.info.consent_required:
-        raise
-    print(
-        "Encountered a ConsentRequired error.\n"
-        "You must login a second time to grant consents.\n\n"
-    )
-    transfer_client = login_and_get_transfer_client(
-        scopes=err.info.consent_required.required_scopes
-    )
-    do_submit(transfer_client)
+task_id = task_doc["task_id"]
+print(f"submitted transfer, task_id={task_id}")

docs/examples/minimal_transfer_script/transfer_consent_required_reactive.py

@@ -1,39 +1,30 @@
 import globus_sdk
-from globus_sdk.scopes import TransferScopes
 
+# tutorial client ID (we recommend replacing this with your own client)
 CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"
-auth_client = globus_sdk.NativeAppAuthClient(CLIENT_ID)
-
-# requested_scopes specifies a list of scopes to request
-# instead of the defaults, only request access to the Transfer API
-auth_client.oauth2_start_flow(requested_scopes=TransferScopes.all)
-authorize_url = auth_client.oauth2_get_authorize_url()
-print(f"Please go to this URL and login:\n\n{authorize_url}\n")
-
-auth_code = input("Please enter the code here: ").strip()
-tokens = auth_client.oauth2_exchange_code_for_tokens(auth_code)
-transfer_tokens = tokens.by_resource_server["transfer.api.globus.org"]
-
-# construct an AccessTokenAuthorizer and use it to construct the
-# TransferClient
-transfer_client = globus_sdk.TransferClient(
-    authorizer=globus_sdk.AccessTokenAuthorizer(transfer_tokens["access_token"])
-)
 
 # Replace these with your own collection UUIDs
-source_collection_id = "..."
-dest_collection_id = "..."
+SOURCE_COLLECTION_ID = "..."
+DEST_COLLECTION_ID = "..."
 
 # create a Transfer task consisting of one or more items
-task_data = globus_sdk.TransferData(
-    source_endpoint=source_collection_id, destination_endpoint=dest_collection_id
-)
+task_data = globus_sdk.TransferData(SOURCE_COLLECTION_ID, DEST_COLLECTION_ID)
 task_data.add_item(
     "/share/godata/file1.txt",  # source
     "/~/minimal-example-transfer-script-destination.txt",  # dest
 )
 
-# submit, getting back the task ID
-task_doc = transfer_client.submit_transfer(task_data)
+# create an app to manage login, use it to create a client, and submit,
+# getting back the task ID
+with globus_sdk.UserApp(
+    "minimal-transfer-example",
+    client_id=CLIENT_ID,
+    # we set the 'auto_redrive_gares' flag, which enables handling for missing
+    # auth requirements when the script is run against a changing set of collection IDs
+    config=globus_sdk.GlobusAppConfig(auto_redrive_gares=True),
+) as app:
+    with globus_sdk.TransferClient(app=app) as transfer_client:
+        task_doc = transfer_client.submit_transfer(task_data)
+
 task_id = task_doc["task_id"]
 print(f"submitted transfer, task_id={task_id}")