ewoks install: support the uv package manager

.github/workflows/test.yml

@@ -32,6 +32,7 @@ jobs:
         ${{ matrix.extra-pytest-warnings }}
       enable-coverage: ${{ matrix.enable-coverage }}
       jupyter-platform-dirs: "1"
+      package-managers: "uv"
       codecov-flags: "unit"
     strategy:
       fail-fast: false

doc/howtoguides/requirements.rst

@@ -18,7 +18,8 @@ What is stored
   installed from the python package index, the git commit or the archive it came from. Any
   :term:`package manager` can recreate the environment from this list.
 * ``manager``: the :term:`package manager` that generated the requirements, with the content
-  of the files it needs to recreate the environment: ``requirements.txt`` for pip-venv.
+  of the files it needs to recreate the environment: ``requirements.txt`` for pip-venv,
+  ``pyproject.toml`` and ``uv.lock`` for uv.
 
 Use ``--exclude-requirements`` to store nothing.
 
@@ -37,7 +38,7 @@ one explicitly
 
 .. code-block:: bash
 
-    ewoks convert demo demo.json --test --package-manager-name pip-venv
+    ewoks convert demo demo.json --test --package-manager-name uv
 
 ``ewoks install`` uses the :term:`package manager` that generated the requirements, unless it
 is not installed on the machine. It installs the stored files of that :term:`package manager`.
@@ -47,15 +48,15 @@ confirmed separately unless ``--yes`` is provided
 
 .. code-block:: bash
 
-    ewoks install demo.json --yes --package-manager-name pip-venv
+    ewoks install demo.json --yes --package-manager-name uv
 
 ``--package-manager-command`` provides the command that invokes the tool, for example when it
 is not on the ``PATH`` or when a faster implementation should be used
 
 .. code-block:: bash
 
-    ewoks install demo.json --yes --package-manager-name pip-venv \
-        --package-manager-command /path/to/python
+    ewoks install demo.json --yes --package-manager-name uv \
+        --package-manager-command /path/to/uv
 
 Choose the environment
 ----------------------
@@ -85,21 +86,28 @@ The default name is the identifier of the :term:`workflow`, which is the ``id``
 content.
 
 Without ``--env-root`` the :term:`package manager` decides where the environment goes. venv
-creates an environment wherever it is told to, so for pip-venv ewoks uses ``~/.ewoks/envs``.
+and uv create an environment wherever they are told to, so for pip-venv and uv ewoks uses
+``~/.ewoks/envs``.
 
 An environment that already exists is installed in, which adds the requirements to what is
 already there. Use ``--clean`` to remove it first. Only a directory that contains a python
 environment is removed.
 
 The environment of a :term:`workflow` is a directory: for pip-venv it is a virtual
-environment. ``ewoks execute --env`` takes that directory, not the python interpreter inside
-it.
+environment and for uv a project with the environment in ``.venv``. ``ewoks execute --env``
+takes that directory, not the python interpreter inside it.
 
 Limitations
 -----------
 
-* ``--python-version`` is a request: pip-venv can only use the version of the python
-  interpreter that creates the environment. A warning is emitted when the version cannot be
-  provided.
+* ``--python-version`` is a request: uv can provide any python version but pip-venv can only
+  use the version of the python interpreter that creates the environment. A warning is emitted
+  when the version cannot be provided.
+* uv resolves the ``distributions`` into a lock file, which requires access to the package
+  index. When that fails, a warning is emitted and the requirements are stored without files:
+  the environment is then recreated from the ``distributions`` list.
+* A lock file is only read by a :term:`package manager` that understands its format version, so
+  reproducing an environment can require a version of the tool that is at least as recent as
+  the one that generated the requirements.
 * A package installed from a local directory cannot be recreated elsewhere. This is reported as
   a warning when the requirements are generated.

doc/reference/glossary.rst

@@ -29,7 +29,7 @@ Glossary
         An execution engine is the underlying software used to execute the :term:`workflow`. :term:`Ewoks` supports multiple execution engines: pypushflow, orange, dask and the ewoks internal excution engine.
 
     Package manager
-        A package manager creates python environments and installs packages in them. :term:`Ewoks` uses package managers to store the environment in which a :term:`workflow` was created and to recreate it: pip with venv.
+        A package manager creates python environments and installs packages in them. :term:`Ewoks` uses package managers to store the environment in which a :term:`workflow` was created and to recreate it: pip with venv and uv.
 
     blissdata
         `Blissdata <https://bliss.gitlab-pages.esrf.fr/blissdata>`_ is an API for accessing data from BLISS in memory.

doc/tutorials/install.rst

@@ -14,12 +14,14 @@ There are two sides to this
 * the *re-producer* receives the :term:`workflow`, recreates the environment and executes the
   :term:`workflow` in it.
 
-Both sides use a :term:`package manager`. The walk-through below does both sides
+Both sides use a :term:`package manager`. The walk-throughs below do the same thing with each
+of the supported ones
 
 .. toctree::
     :maxdepth: 1
 
     install/pip_venv
+    install/uv
 
 The producer and the re-producer do not need the same :term:`package manager`: the
 requirements contain the installed python packages, which any :term:`package manager` can
@@ -68,7 +70,7 @@ and the command to remove it again
       Remove : rm -rf ewoks_envs/demo
 
 Without ``--yes`` you are asked to confirm after the packages have been listed. The
-walk-through uses ``--env-root`` to create the environment in the working directory. Without
+walk-throughs use ``--env-root`` to create the environment in the working directory. Without
 it the environment is created where the :term:`package manager` creates named environments.
 
 Execute the workflow

doc/tutorials/install/uv.rst

@@ -0,0 +1,162 @@
+.. _install_uv:
+
+uv
+==
+
+End-to-end walk-through of `workflow installation and execution <../install.rst>`_ with `uv <https://docs.astral.sh/uv/>`_.
+
+Install uv
+----------
+
+.. tabs::
+
+    .. group-tab:: Linux
+
+        .. code-block:: bash
+
+            export UV_INSTALL_DIR=$HOME/.local/uv/bin  # default location
+            curl -LsSf https://astral.sh/uv/install.sh | INSTALLER_NO_MODIFY_PATH=1 sh
+            export PATH=$UV_INSTALL_DIR:$PATH
+
+    .. group-tab:: macOS
+
+        .. code-block:: bash
+
+            export UV_INSTALL_DIR=$HOME/.local/uv/bin  # default location
+            curl -LsSf https://astral.sh/uv/install.sh | INSTALLER_NO_MODIFY_PATH=1 sh
+            export PATH=$UV_INSTALL_DIR:$PATH
+
+    .. group-tab:: Windows
+
+        .. code-block:: powershell
+
+            $env:UV_INSTALL_DIR = "$env:USERPROFILE\.local\uv\bin"  # default location
+            irm https://astral.sh/uv/install.ps1 | iex
+            $env:PATH = "$env:UV_INSTALL_DIR;$env:PATH"
+
+Producer side
+-------------
+
+Create an environment, install :term:`ewoks` in it and store a :term:`workflow` with the
+packages of that environment as its ``requirements``
+
+.. tabs::
+
+    .. group-tab:: Linux
+
+        .. code-block:: bash
+
+            uv venv ewoks_producer
+            source ewoks_producer/bin/activate
+            uv pip install ewoks
+            ewoks convert demo demo.json --test
+            deactivate
+
+    .. group-tab:: macOS
+
+        .. code-block:: bash
+
+            uv venv ewoks_producer
+            source ewoks_producer/bin/activate
+            uv pip install ewoks
+            ewoks convert demo demo.json --test
+            deactivate
+
+    .. group-tab:: Windows
+
+        .. code-block:: powershell
+
+            uv venv ewoks_producer
+            ewoks_producer\Scripts\activate.ps1
+            uv pip install ewoks
+            ewoks convert demo demo.json --test
+            deactivate
+
+Install any package that provides :term:`tasks <Task>` instead of
+``ewoks`` for a real :term:`workflow`.
+
+Re-producer side
+----------------
+
+Only ``ewoks`` itself is needed to recreate the environment of ``demo.json``
+
+.. tabs::
+
+    .. group-tab:: Linux
+
+        .. code-block:: bash
+
+            uv venv ewoks_reproducer
+            source ewoks_reproducer/bin/activate
+            uv pip install ewoks
+            ewoks install demo.json --yes --env-root ewoks_envs
+
+    .. group-tab:: macOS
+
+        .. code-block:: bash
+
+            uv venv ewoks_reproducer
+            source ewoks_reproducer/bin/activate
+            uv pip install ewoks
+            ewoks install demo.json --yes --env-root ewoks_envs
+
+    .. group-tab:: Windows
+
+        .. code-block:: powershell
+
+            uv venv ewoks_reproducer
+            ewoks_reproducer\Scripts\activate.ps1
+            uv pip install ewoks
+            ewoks install demo.json --yes --env-root ewoks_envs
+
+The environment of the :term:`workflow` is a uv project in ``ewoks_envs/demo`` with the
+virtual environment in its ``.venv`` directory.
+
+Execute the workflow
+--------------------
+
+.. tabs::
+
+    .. group-tab:: Linux
+
+        .. code-block:: bash
+
+            ewoks execute --env ewoks_envs/demo demo.json --outputs=all
+
+    .. group-tab:: macOS
+
+        .. code-block:: bash
+
+            ewoks execute --env ewoks_envs/demo demo.json --outputs=all
+
+    .. group-tab:: Windows
+
+        .. code-block:: powershell
+
+            ewoks execute --env ewoks_envs\demo demo.json --outputs=all
+
+Clean up
+--------
+
+.. tabs::
+
+    .. group-tab:: Linux
+
+        .. code-block:: bash
+
+            deactivate
+            rm -rf ewoks_producer ewoks_reproducer ewoks_envs demo.json
+
+    .. group-tab:: macOS
+
+        .. code-block:: bash
+
+            deactivate
+            rm -rf ewoks_producer ewoks_reproducer ewoks_envs demo.json
+
+    .. group-tab:: Windows
+
+        .. code-block:: powershell
+
+            deactivate
+            Remove-Item -Recurse -Force ewoks_producer, ewoks_reproducer, ewoks_envs, demo.json

src/ewoks/_requirements/utils/_supported.py

@@ -3,12 +3,14 @@
 from typing import Type
 
 from ..pip_venv import PipVenvManager
+from ..uv import UvManager
 from .base_manager import BaseManager
 
 
 @lru_cache(1)
 def get_supported_managers() -> Dict[str, Type[BaseManager]]:
     managers = [
         PipVenvManager,
+        UvManager,
     ]
     return {manager_cls.NAME: manager_cls for manager_cls in managers}

src/ewoks/_requirements/utils/base_manager.py

@@ -26,7 +26,7 @@
 class BaseManagerInfo(models.BaseModel):
     name: str = Field(
         description="Package manager that generated the requirements.",
-        examples=["pip-venv"],
+        examples=["pip-venv", "uv"],
     )
     version: str = Field(
         description="Version of the package manager.", examples=["25.0.1"]
@@ -35,8 +35,9 @@ class BaseManagerInfo(models.BaseModel):
         default_factory=dict,
         description=(
             "Content of the files the package manager needs to reproduce the "
-            "environment: 'requirements.txt' for pip-venv. Empty when the package "
-            "manager could not generate them."
+            "environment: 'requirements.txt' for pip-venv, 'pyproject.toml' and "
+            "'uv.lock' for uv. Empty when the package manager could not generate "
+            "them."
         ),
         examples=[{"requirements.txt": "ewoks==7.0.0\nnetworkx==3.4.2\n"}],
     )
@@ -184,8 +185,8 @@ def environment_location(
 
     def environments_root(self) -> Path:
         """Root directory in which the package manager creates named environments.
-        Package managers that create an environment wherever they are told to (venv)
-        use a directory of ewoks.
+        Package managers that create an environment wherever they are told to (venv
+        and uv) use a directory of ewoks.
         """
         return EWOKS_ENVIRONMENTS_ROOT
 

src/ewoks/_requirements/utils/environment.py

@@ -13,6 +13,7 @@
 
 ENVIRONMENT_SUBDIRS: Tuple[str, ...] = (
     "",  # environment prefix, for example created by `python -m venv`
+    ".venv",  # uv project
 )
 
 

src/ewoks/_requirements/utils/metadata/models.py

@@ -82,5 +82,5 @@ class Distribution(BaseModel):
     installer: Optional[str] = Field(
         default=None,
         description="Tool that installed the distribution.",
-        examples=["pip"],
+        examples=["pip", "uv"],
     )