|
| 1 | +--- |
| 2 | +title: Build Customization |
| 3 | +sidebar: user_docs |
| 4 | +permalink: build-environment |
| 5 | +folder: docs |
| 6 | +--- |
| 7 | + |
| 8 | +It's commonly the case that you want to customize your build environment, such as specifying a custom cache directory for layers, or sending your Docker Credentials to the registry endpoint. Here we will discuss those things |
| 9 | + |
| 10 | +{% include toc.html %} |
| 11 | + |
| 12 | + |
| 13 | +## Cache Folders |
| 14 | +To make download of layers for build and <a href="/docs-pull">pull</a> faster and less redundant, we use a caching strategy. By default, the Singularity software will create a set of folders in your `$HOME` directory for docker layers, Singularity Hub images, and Docker metadata, respectively: |
| 15 | + |
| 16 | +``` |
| 17 | +$HOME/.singularity |
| 18 | +$HOME/.singularity/docker |
| 19 | +$HOME/.singularity/shub |
| 20 | +$HOME/.singularity/metadata |
| 21 | +``` |
| 22 | + |
| 23 | +Fear not, you have control to customize this behavior! If you don't want the cache to be created (and a temporary directory will be used), set `SINGULARITY_DISABLE_CACHE` to True/yes, or if you want to move it elsewhere, set `SINGULARITY_CACHEDIR` to the full path where you want to cache. Remember that when you run commands as sudo this will use root's home at `/root` and not your user's home. |
| 24 | + |
| 25 | +## Pull Folder |
| 26 | +For details about customizing the output location of <a href="/docs-pull">pull</a>, see the<a href="/docs-pull">pull docs.</a> You have the similar ability to set it to be something different, or to customize the name of the pulled image. |
| 27 | + |
| 28 | + |
| 29 | +## Environment Variables |
| 30 | +All environmental variables are parsed by Singularity python helper functions, and specifically the file <a href="https://github.com/singularityware/singularity/blob/master/libexec/python/defaults.py" target="_blank">defaults.py</a> is a gateway between variables defined at runtime, and pre-defined defaults. By way of import from the file, variables set at runtime do not change if re-imported. This was done intentionally to prevent changes during the execution, and could be changed if needed. For all variables, the order of operations works as follows: |
| 31 | + |
| 32 | + 1. First preference goes to environment variable set at runtime |
| 33 | + 2. Second preference goes to default defined in this file |
| 34 | + 3. Then, if neither is found, null is returned except in the case that `required=True`. A `required=True` variable not found will system exit with an error. |
| 35 | + 4. Variables that should not be dispayed in debug logger are set with `silent=True`, and are only reported to be defined. |
| 36 | + |
| 37 | + |
| 38 | +For boolean variables, the following are acceptable for True, with any kind of capitalization or not: |
| 39 | + |
| 40 | +``` |
| 41 | +("yes", "true", "t", "1","y") |
| 42 | +``` |
| 43 | + |
| 44 | +## Cache |
| 45 | +The location and usage of the cache is also determined by environment variables. |
| 46 | + |
| 47 | +**SINGULARITY_DISABLE_CACHE** |
| 48 | +If you want to disable the cache, this means is that the layers are written to a temporary directory. Thus, if you want to disable cache and write to a temporary folder, simply set `SINGULARITY_DISABLE_CACHE` to any true/yes value. By default, the cache is not disabled. |
| 49 | + |
| 50 | +**SINGULARITY_CACHE** |
| 51 | +Is the base folder for caching layers and singularity hub images. If not defined, it uses default of `$HOME/.singularity`. If defined, the defined location is used instead. If `SINGULARITY_DISABLE_CACHE` is set to True, this value is ignored in favor of a temporary directory. For specific subtypes of things to cache, subdirectories are created (by python), including `$SINGULARITY_CACHE/docker` for docker layers and `$SINGULARITY_CACHE/shub` for Singularity Hub images. If the cache is not created, the Python script creates it. |
| 52 | + |
| 53 | +**SINGULARITY_PULLFOLDER** |
| 54 | +While this isn't relevant for build, since build is close to pull, we will include it here. By default, images are pulled to the present working directory. The user can change this variable to change that. |
| 55 | + |
| 56 | + |
| 57 | +### Defaults |
| 58 | +The following variables have defaults that can be customized by you via environment variables at runtime. |
| 59 | + |
| 60 | + |
| 61 | +#### Docker |
| 62 | + |
| 63 | +**DOCKER_API_BASE** |
| 64 | +Set as `index.docker.io`, which is the name of the registry. In the first version of Singularity we parsed the Registry argument from the build spec file, however now this is removed because it can be obtained directly from the image name (eg, `registry/namespace/repo:tag`). If you don't specify a registry name for your image, this default is used. If you have trouble with your registry being detected from the image URI, use this variable. |
| 65 | + |
| 66 | +**DOCKER_API_VERSION** |
| 67 | +Is the version of the Docker Registry API currently being used, by default now is `v2`. |
| 68 | + |
| 69 | +**DOCKER_OS** |
| 70 | +This is exposed via the exported environment variable `SINGULARITY_DOCKER_OS` and pertains to images that reveal a version 2 manifest with a [manifest list](https://docs.docker.com/registry/spec/manifest-v2-2/#manifest-list). In the case that the list is present, we must choose an operating system (this variable) and an architecture (below). The default is `linux`. |
| 71 | + |
| 72 | +**DOCKER_ARCHITECTURE** |
| 73 | +This is exposed via the exported environment variable `SINGULARITY_DOCKER_ARCHITECTURE` and the same applies as for the `DOCKER_OS` with regards to being used in context of a list of manifests. In the case that the list is present, we must choose an architecture (this variable) and an os (above). The default is `amd64`, and other common ones include `arm`, `arm64`, `ppc64le`, `386`, and `s390x`. |
| 74 | + |
| 75 | +**NAMESPACE** |
| 76 | +Is the default namespace, `library`. |
| 77 | + |
| 78 | +**RUNSCRIPT_COMMAND** |
| 79 | +Is not obtained from the environment, but is a hard coded default (`"/bin/bash"`). This is the fallback command used in the case that the docker image does not have a `CMD` or `ENTRYPOINT`. |
| 80 | + |
| 81 | +**TAG** |
| 82 | +Is the default tag, `latest`. |
| 83 | + |
| 84 | +**DISABLE_HTTPS** |
| 85 | +This is relevant if you want to use a registry that doesn't have https, and it speaks for itself. If you export the variable `SINGULARITY_NOHTTPS` you can force the software to not use https when interacting with a Docker registry. This use case is typically for use of a local registry. |
| 86 | + |
| 87 | + |
| 88 | +#### Singularity Hub |
| 89 | + |
| 90 | +**SHUB_API_BASE** |
| 91 | +The default base for the Singularity Hub API, which is `https://singularity-hub.org/api`. If you deploy your own registry, you don't need to change this, you can again specify the registry name in the `shub://` URI. |
| 92 | + |
| 93 | + |
| 94 | + |
| 95 | +### General |
| 96 | +**SINGULARITY_PYTHREADS** |
| 97 | +The Python modules use threads (workers) to download layer files for Docker, and change permissions. By default, we will use 9 workers, unless the environment variable `SINGULARITY_PYTHREADS` is defined. |
| 98 | + |
| 99 | + |
| 100 | +**SINGULARITY_COMMAND_ASIS** |
| 101 | +By default, we want to make sure the container running process gets passed forward as the current process, so we want to prefix whatever the Docker command or entrypoint is with `exec`. We also want to make sure that following arguments get passed, so we append `"$@"`. Thus, some entrypoint or cmd might look like this: |
| 102 | + |
| 103 | +``` |
| 104 | + /usr/bin/python |
| 105 | +``` |
| 106 | + |
| 107 | +and we would parse it into the runscript as: |
| 108 | +``` |
| 109 | + exec /usr/bin/python "$@" |
| 110 | +``` |
| 111 | +However, it might be the case that the user does not want this. For this reason, we have the environmental variable `RUNSCRIPT_COMMAND_ASIS`. If defined as yes/y/1/True/true, etc., then the runscript will remain as `/usr/bin/python`. |
0 commit comments