Ewoks install bootstrap

.github/workflows/test.yml

@@ -23,7 +23,7 @@ jobs:
       os: ${{ matrix.os }}
       python-version: ${{ matrix.python-version }}
       pytest-args: >-
-        -v -ra -W error
+        -v -ra -W error -m "not package_manager_install"
         -W ignore::DeprecationWarning:jupyter_client.session
         -W ignore::DeprecationWarning:orangecanvas.utils.localization
         -W ignore::DeprecationWarning:orangecanvas.utils.localization.si
@@ -79,6 +79,25 @@ jobs:
             enable-coverage: "false"
             extra-pytest-warnings: ""
 
+  # Run the tests that install package managers, on runners without them
+  tests-package-manager-install:
+    needs: build
+    uses: ewoks-kit/.github/.github/workflows/python-tests.yml@main
+    with:
+      os: ${{ matrix.os }}
+      python-version: "3.12"
+      pytest-args: >-
+        -v -ra -W error -m package_manager_install
+        -W ignore::DeprecationWarning:jupyter_client.session
+        -W ignore::DeprecationWarning:orangecanvas.utils.localization
+        -W ignore::DeprecationWarning:orangecanvas.utils.localization.si
+        -W ignore::DeprecationWarning:matplotlib._fontconfig_pattern
+        -W ignore::DeprecationWarning:matplotlib._mathtext
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest, windows-latest]
+
   # Run linter / checks
   checks:
     uses: ewoks-kit/.github/.github/workflows/python-check.yml@main

doc/conf.py

@@ -3,6 +3,7 @@
 
 
 import importlib.metadata
+import os
 
 release = importlib.metadata.version("ewoks")
 
@@ -48,6 +49,7 @@
 html_title = docstitle
 html_logo = "_static/logo.png"
 html_static_path = ["_static"]
+html_extra_path = ["../src/ewoks/_bootstrap"]
 html_template_path = ["_templates"]
 html_css_files = ["custom.css"]
 
@@ -76,3 +78,20 @@
     "footer_start": ["copyright"],
     "footer_end": ["footer_end"],
 }
+
+# Root of the documentation version that is built: on Read the Docs, for example
+# .../en/stable, or on GitLab Pages
+_DOCS_URL = (
+    os.environ.get("READTHEDOCS_CANONICAL_URL")
+    or os.environ.get("CI_PAGES_URL")
+    or "https://ewoks.readthedocs.io/en/latest"
+).rstrip("/")
+
+
+def _substitute_docs_url(app, docname, source):
+    # Substitutions are not supported in code blocks
+    source[0] = source[0].replace("|docs_url|", _DOCS_URL)
+
+
+def setup(app):
+    app.connect("source-read", _substitute_docs_url)

doc/explanations/task_input_priority.rst

@@ -7,7 +7,7 @@ Details
 A :term:`node <Nodes>` in a :term:`workflow` can get inputs from three different sources:
 
 1. Via the ``data_mapping`` :term:`link <Links>` attribute of an incoming :term:`link <Links>` (see `Link attributes <https://ewokscore.readthedocs.io/en/stable/definitions.html#link-attributes>`_)
-2. Via the ``parameters`` CLI argument (or ``inputs`` for Python) when executing/submitting the :term:`workflow` (see `ewoks execute reference <https://ewoks.readthedocs.io/en/stable/cli.html#ewoks-execute>`_)
+2. Via the ``parameters`` CLI argument (or ``inputs`` for Python) when executing/submitting the :term:`workflow` (see :ref:`ewoks execute reference <cli_execute>`)
 3. Via the ``default_inputs`` :term:`node <Nodes>` attribute of the :term:`node <Nodes>` itself (see `Node attributes <https://ewokscore.readthedocs.io/en/stable/definitions.html#node-attributes>`_)
 
 If the same input is specified by these different sources, :term:`Ewoks` applies the following priorities:

doc/howtoguides/requirements.rst

@@ -13,6 +13,9 @@ What is stored
 
 ``ewoks convert`` and ``ewoks execute -o convert_destination=...`` store
 
+* ``ewoks``: the version of :term:`ewoks` that generated the requirements and the engine
+  that saved the :term:`workflow`. The
+  :ref:`ewoks-install script <install_bootstrap>` installs both to run ``ewoks install``.
 * ``python`` and ``system``: the python interpreter and the operating system.
 * ``distributions``: every installed python package with its version and, when it was not
   installed from the python package index, the git commit or the archive it came from. Any

doc/reference/cli.rst

@@ -63,6 +63,8 @@ ewoks convert
 
     **ewoks convert** can also be used to store ``inputs`` inside the destination :term:`workflow`.
 
+.. _cli_execute:
+
 ewoks execute
 -------------
 

doc/tutorials/install.rst

@@ -40,6 +40,7 @@ Store the requirements
 .. code-block:: json
 
     {
+        "ewoks": {"version": "7.0.0", "engine": {"name": "ewokscore", "version": "5.1.0"}},
         "python": {"version": "3.12.11", "implementation": "CPython", "...": "..."},
         "system": {"system": "Linux", "machine": "x86_64", "...": "..."},
         "distributions": [
@@ -53,6 +54,9 @@ Store the requirements
         }
     }
 
+* ``ewoks`` is the version of :term:`ewoks` that generated the requirements and the engine that
+  saved the :term:`workflow`: ``ewokscore`` for a ``.json`` or ``.yaml`` file and
+  ``ewoksorange`` for an ``.ows`` file.
 * ``distributions`` are the installed python packages. Any :term:`package manager` can
   recreate the environment from this list.
 * ``manager`` is the :term:`package manager` that generated the requirements together with
@@ -63,19 +67,67 @@ Create the environment
 
 ``ewoks install`` creates a python environment for the :term:`workflow`. It prints the python
 interpreter of the environment it created, the command to execute the :term:`workflow` in it
-and the command to remove it again
+and the command to remove it again. These commands work from any terminal and directory
 
 .. code-block:: text
 
     Installed requirements for demo.json
-      Python : ewoks_envs/demo/bin/python
-      Execute: ewoks execute --env ewoks_envs/demo demo.json
-      Remove : rm -rf ewoks_envs/demo
+      Python : /home/user/ewoks_envs/demo/bin/python
+      Execute: /home/user/ewoks_envs/demo/bin/python -m ewoks execute /home/user/demo.json
+      Remove : rm -rf /home/user/ewoks_envs/demo
 
 Without ``--yes`` you are asked to confirm after the packages have been listed. The
 walk-throughs use ``--env-root`` to create the environment in the working directory. Without
 it the environment is created where the :term:`package manager` creates named environments.
 
+.. _install_bootstrap:
+
+Create the environment without ewoks
+++++++++++++++++++++++++++++++++++++
+
+``ewoks install`` needs a python environment with :term:`ewoks`. The ``ewoks-install`` script
+only needs a :term:`package manager`: it installs the version of :term:`ewoks` that generated
+the requirements, and the engine that saved the :term:`workflow`, in a bootstrap environment
+and runs ``ewoks install`` from there. All
+arguments are passed to ``ewoks install``, except for the options of the script itself
+
+.. tabs::
+
+    .. group-tab:: Linux
+
+        .. code-block:: bash
+
+            curl -LsSf |docs_url|/ewoks-install.sh | sh -s -- demo.json --package-manager-name uv
+
+    .. group-tab:: macOS
+
+        .. code-block:: bash
+
+            curl -LsSf |docs_url|/ewoks-install.sh | sh -s -- demo.json --package-manager-name uv
+
+    .. group-tab:: Windows
+
+        .. code-block:: powershell
+
+            & ([scriptblock]::Create((irm |docs_url|/ewoks-install.ps1))) demo.json --package-manager-name uv
+
+* ``--package-manager-name`` and ``--package-manager-command`` select the
+  :term:`package manager` that creates the bootstrap environment. Without them it is the first
+  one that is available of uv, pixi, conda, poetry and pip-venv.
+* ``--print-command`` prints the ``ewoks install`` command instead of running it.
+* ``--ewoks-requirement`` installs another version of :term:`ewoks`, for example
+  ``--ewoks-requirement "ewoks>=7"``. Repeat it to install an engine as well, for example
+  ``--ewoks-requirement ewoks --ewoks-requirement ewoksmyengine``. The latest version of
+  :term:`ewoks` is installed when the requirements do not provide one.
+* ``--bootstrap-dir`` is the directory of the bootstrap environments.
+* ``--install-package-manager`` installs the :term:`package manager` of
+  ``--package-manager-name`` in the bootstrap directory when it is not installed.
+
+The script keeps the bootstrap environment in ``~/.ewoks/bootstrap`` and uses it again when
+it installs another :term:`workflow` whose requirements were generated by the same version of
+:term:`ewoks`. The bootstrap environment only runs ``ewoks install``:
+it is not the environment of the :term:`workflow`.
+
 Execute the workflow
 --------------------
 

doc/tutorials/install/conda.rst

@@ -124,6 +124,33 @@ The environment of the :term:`workflow` is a conda environment in ``ewoks_envs/d
 ``--env-root`` it is created in the first environment directory of conda, where
 ``conda activate demo`` finds it.
 
+Alternatively, the :ref:`ewoks-install script <install_bootstrap>` recreates the
+environment without an environment with :term:`ewoks`
+
+.. tabs::
+
+    .. group-tab:: Linux
+
+        .. code-block:: bash
+
+            curl -LsSf |docs_url|/ewoks-install.sh | sh -s -- demo.json --package-manager-name conda --yes --env-root ewoks_envs
+
+    .. group-tab:: macOS
+
+        .. code-block:: bash
+
+            curl -LsSf |docs_url|/ewoks-install.sh | sh -s -- demo.json --package-manager-name conda --yes --env-root ewoks_envs
+
+    .. group-tab:: Windows
+
+        .. code-block:: powershell
+
+            & ([scriptblock]::Create((irm |docs_url|/ewoks-install.ps1))) demo.json --package-manager-name conda --yes --env-root ewoks_envs
+
+The script does not install ``ewoks`` in the current environment, so execute the
+workflow with the python interpreter of the environment of the :term:`workflow`, for example
+``ewoks_envs/demo/bin/python -m ewoks execute demo.json``.
+
 Execute the workflow
 --------------------
 

doc/tutorials/install/pip_venv.rst

@@ -83,6 +83,33 @@ Only ``ewoks`` itself is needed to recreate the environment of ``demo.json``
 
 The environment of the :term:`workflow` is a virtual environment in ``ewoks_envs/demo``.
 
+Alternatively, the :ref:`ewoks-install script <install_bootstrap>` recreates the
+environment without an environment with :term:`ewoks`
+
+.. tabs::
+
+    .. group-tab:: Linux
+
+        .. code-block:: bash
+
+            curl -LsSf |docs_url|/ewoks-install.sh | sh -s -- demo.json --package-manager-name pip-venv --yes --env-root ewoks_envs
+
+    .. group-tab:: macOS
+
+        .. code-block:: bash
+
+            curl -LsSf |docs_url|/ewoks-install.sh | sh -s -- demo.json --package-manager-name pip-venv --yes --env-root ewoks_envs
+
+    .. group-tab:: Windows
+
+        .. code-block:: powershell
+
+            & ([scriptblock]::Create((irm |docs_url|/ewoks-install.ps1))) demo.json --package-manager-name pip-venv --yes --env-root ewoks_envs
+
+The script does not install ``ewoks`` in the current environment, so execute the
+workflow with the python interpreter of the environment of the :term:`workflow`, for example
+``ewoks_envs/demo/bin/python -m ewoks execute demo.json``.
+
 Execute the workflow
 --------------------
 

doc/tutorials/install/pixi.rst

@@ -119,6 +119,33 @@ Only ``ewoks`` itself is needed to recreate the environment of ``demo.json``
 The environment of the :term:`workflow` is a pixi workspace in ``ewoks_envs/demo`` with the
 environment in its ``.pixi/envs/default`` directory.
 
+Alternatively, the :ref:`ewoks-install script <install_bootstrap>` recreates the
+environment without an environment with :term:`ewoks`
+
+.. tabs::
+
+    .. group-tab:: Linux
+
+        .. code-block:: bash
+
+            curl -LsSf |docs_url|/ewoks-install.sh | sh -s -- demo.json --package-manager-name pixi --yes --env-root ewoks_envs
+
+    .. group-tab:: macOS
+
+        .. code-block:: bash
+
+            curl -LsSf |docs_url|/ewoks-install.sh | sh -s -- demo.json --package-manager-name pixi --yes --env-root ewoks_envs
+
+    .. group-tab:: Windows
+
+        .. code-block:: powershell
+
+            & ([scriptblock]::Create((irm |docs_url|/ewoks-install.ps1))) demo.json --package-manager-name pixi --yes --env-root ewoks_envs
+
+The script does not install ``ewoks`` in the current environment, so execute the
+workflow with the python interpreter of the environment of the :term:`workflow`, for example
+``ewoks_envs/demo/.pixi/envs/default/bin/python -m ewoks execute demo.json``.
+
 Execute the workflow
 --------------------
 

doc/tutorials/install/poetry.rst

@@ -106,6 +106,33 @@ Only ``ewoks`` itself is needed to recreate the environment of ``demo.json``
 The environment of the :term:`workflow` is a poetry project in ``ewoks_envs/demo`` with the
 virtual environment in its ``.venv`` directory.
 
+Alternatively, the :ref:`ewoks-install script <install_bootstrap>` recreates the
+environment without an environment with :term:`ewoks`
+
+.. tabs::
+
+    .. group-tab:: Linux
+
+        .. code-block:: bash
+
+            curl -LsSf |docs_url|/ewoks-install.sh | sh -s -- demo.json --package-manager-name poetry --yes --env-root ewoks_envs
+
+    .. group-tab:: macOS
+
+        .. code-block:: bash
+
+            curl -LsSf |docs_url|/ewoks-install.sh | sh -s -- demo.json --package-manager-name poetry --yes --env-root ewoks_envs
+
+    .. group-tab:: Windows
+
+        .. code-block:: powershell
+
+            & ([scriptblock]::Create((irm |docs_url|/ewoks-install.ps1))) demo.json --package-manager-name poetry --yes --env-root ewoks_envs
+
+The script does not install ``ewoks`` in the current environment, so execute the
+workflow with the python interpreter of the environment of the :term:`workflow`, for example
+``ewoks_envs/demo/.venv/bin/python -m ewoks execute demo.json``.
+
 Execute the workflow
 --------------------
 

doc/tutorials/install/uv.rst

@@ -113,6 +113,33 @@ Only ``ewoks`` itself is needed to recreate the environment of ``demo.json``
 The environment of the :term:`workflow` is a uv project in ``ewoks_envs/demo`` with the
 virtual environment in its ``.venv`` directory.
 
+Alternatively, the :ref:`ewoks-install script <install_bootstrap>` recreates the
+environment without an environment with :term:`ewoks`
+
+.. tabs::
+
+    .. group-tab:: Linux
+
+        .. code-block:: bash
+
+            curl -LsSf |docs_url|/ewoks-install.sh | sh -s -- demo.json --package-manager-name uv --yes --env-root ewoks_envs
+
+    .. group-tab:: macOS
+
+        .. code-block:: bash
+
+            curl -LsSf |docs_url|/ewoks-install.sh | sh -s -- demo.json --package-manager-name uv --yes --env-root ewoks_envs
+
+    .. group-tab:: Windows
+
+        .. code-block:: powershell
+
+            & ([scriptblock]::Create((irm |docs_url|/ewoks-install.ps1))) demo.json --package-manager-name uv --yes --env-root ewoks_envs
+
+The script does not install ``ewoks`` in the current environment, so execute the
+workflow with the python interpreter of the environment of the :term:`workflow`, for example
+``ewoks_envs/demo/.venv/bin/python -m ewoks execute demo.json``.
+
 Execute the workflow
 --------------------
 

pyproject.toml

@@ -80,6 +80,7 @@ ewoks = "ewoks.__main__:main"
 
 [tool.setuptools.package-data]
 "ewoks.tests.notebooks" = ["*.ipynb"]
+"ewoks._bootstrap" = ["*.sh", "*.ps1"]
 
 [tool.ruff.lint]
 select = [