You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{% include callout.html type="note" content="Curious why Shelley automates this instead of running `shpc install` by hand? See [why Shelley over manual sHPC](https://github.com/Sydney-Informatics-Hub/shelley/blob/main/docs/explanation/why-shelley.md) in the Shelley docs." %}
90
90
91
-
## How-to bulk install a list of containers
91
+
## How-to bulk install a list of containers {#bulk-install}
92
+
93
+
If you need several modules built at once, for example, all the tools a pipeline
94
+
depends on, you can pass `shelley build` a plain-text file listing one tool spec per line
95
+
instead of building each tool individually:
96
+
97
+
```bash
98
+
shelley build tools.txt
99
+
```
100
+
101
+
Each line follows the same format as a single-tool `build` call — `<tool>`,
102
+
`<tool>/<version>`, or `<tool>:<version>--<hash>`. Blank lines and `#` comments are
103
+
ignored, so you can group and annotate the list:
92
104
93
-
TODO:
94
-
- To do this, pass in a list of text files
105
+
```
106
+
# Core alignment tools
107
+
samtools/1.21
108
+
bwa
109
+
bowtie2/2.5.1 # pinned for reproducibility
110
+
fastqc
111
+
```
95
112
96
-
## How-to find the full CVMFS path of containers
113
+
Shelley detects that the argument is a file rather than a tool name, and builds every
114
+
entry in sequence, showing a progress table and a results summary at the end.
97
115
98
-
TODO:
99
-
- Use cases where you need to reference the full container path e.g. nextflow.config
100
-
- Add the previous nextflow.config example from @tools.md here
101
-
- Use find -vv
102
-
- Leave placeholders for screenshots
116
+
## How-to find the full CVMFS path of containers {#find-cvmfs-path}
103
117
104
-
## How-to build modules without an unregistrered tool
118
+
Some workflows need the exact CVMFS container path rather than a loaded module. For
119
+
example, a Nextflow process configured to pull a container directly instead of using
120
+
`module load`. Use `find -v` to to list every individual build of a tool together
121
+
with its full CVMFS image path:
105
122
106
-
TODO:
107
-
- Containers require a registry file to build the module, often this doesn't require
108
-
- You will not know this, and it doesn't matter because shelley deals with this on the fly
109
-
- These include very new containers, or containers that were built prior to the registry
110
-
- Downside: May clutter with utility binaries that you don't need
123
+
```bash
124
+
shelley find fastqc -v
125
+
```
126
+
127
+
TODO: screenshot
128
+
129
+
Each row corresponds to one `--hash` build of a version, with its buildable/installed
130
+
status and the path you can copy into a config file, such as:
See [Using Nextflow with CVMFS](nextflow) for a worked example of pointing a Nextflow
137
+
process at a container path like this.
138
+
139
+
## How-to build a tool that isn't in the sHPC registry {#unregistered-tool}
140
+
141
+
Building an Lmod module requires a registry entry describing the container.
142
+
sHPC's [shpc-registry](https://github.com/singularityhub/shpc-registry) supplies these
143
+
for most tools, but it doesn't cover every version. Very new containers, and containers
144
+
built before a tool was added to the registry, often have no entry at all.
145
+
146
+
You don't need to know whether a version is registered as `shelley build` checks for you
147
+
and will build this for you on-the-fly:
148
+
149
+
```bash
150
+
shelley build <tool>/<version>#TODO later: add version
151
+
```
152
+
153
+
If the version is missing from the upstream registry, Shelley creates a local registry
154
+
entry from the CVMFS container itself and retries the install. This is transparent and
155
+
adds only a few extra minutes to the build. See [Why Shelley over manual sHPC](https://github.com/Sydney-Informatics-Hub/shelley/blob/main/docs/explanation/why-shelley.md)
156
+
for what this replaces manually, and [Build design](https://github.com/Sydney-Informatics-Hub/shelley/blob/main/docs/explanation/build-design.md)
157
+
for how the fallback works internally.
158
+
159
+
{% include callout.html type="note" content="Because the alias list for a locally-registered entry is generated straight from the container, it can include extra utility binaries bundled alongside the tool (for example, conda-infrastructure commands) rather than only the tool's own executables." %}
160
+
161
+
## [EXPERIMENTAL] How-to amend the aliases {#amend-aliases}
162
+
163
+
{% include callout.html type="important" content="Experimental feature - use with caution. Curating aliases changes which commands a module exposes; an alias you remove or rename may break other users' or pipelines' expectations of that module." %}
164
+
165
+
As noted [above](#unregistered-tool), a container can bundle utility binaries alongside
166
+
the tool itself, which can clutter the module's alias list with commands you don't
167
+
need. Add `-i` (or `--interactive`) to `build` to open a session where you can curate
168
+
the aliases a module exposes such as deselecting, renaming, or adding them; before the install
169
+
completes:
170
+
171
+
```bash
172
+
shelley build <tool>/<version> -i
173
+
```
113
174
114
-
TODO:
115
-
- use with caution
116
-
- For example, container may include a lot of utility binaries that are not required
117
-
- You can manually amend this using `build -i <container>`
0 commit comments