You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pages/docs/user-docs/docs-flow.md
+23-22Lines changed: 23 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,9 +3,13 @@ title: Singularity Flow
3
3
sidebar: user_docs
4
4
permalink: docs-flow
5
5
folder: docs
6
-
toc: true
6
+
toc: false
7
7
---
8
8
9
+
This document describes a suggested "best-practices" workflow for building, running, and managing your containers.
10
+
11
+
{% include toc.html %}
12
+
9
13
There are generally two ways to get images. You either want to pull an image file as is, or (more likely) build your own custom image. We will start with talking about build, and the many different use cases it affords.
10
14
11
15
@@ -15,26 +19,28 @@ If you read the [quick start](/quickstart), you probably remember that building
15
19
### The Singularity Flow
16
20
The diagram below is a visual depiction of how you can use Singularity to build images. The high level idea is that we have two environments:
17
21
18
-
- a development environment (where you have sudo privileges) to test and build your container
19
-
- a production environment where you run your container
22
+
- a **build** environment (where you have sudo privileges) to test and build your container
23
+
- a **production** environment where you run your container
Singularity production images are immutable. This is a feature added as of Singularity 2.4, and it ensures a higher level of reproducibility and verification of images. To read more about the details, check out the <ahref="/build">bulid</a>docs. However, immutability is not so great when you are testing, debugging, or otherwise want to quickly change your image. We will proceed by describing a typical workflow of developing first, building a final image, and using in production.
29
+
Singularity production images are immutable. This is a feature added as of Singularity 2.4, and it ensures a higher level of reproducibility and verification of images. To read more about the details, check out the [build](docs-build)docs. However, immutability is not so great when you are testing, debugging, or otherwise want to quickly change your image. We will proceed by describing a typical workflow of developing first, building a final image, and using in production.
26
30
27
31
### 1. Development Commands
28
32
If you want a writable image or folder for developing, you have two options:
29
33
30
-
1. build into a folder that has writable permissions with sudo
31
-
2. build into an ext3 image file, also that has writable permissions with sudo and the `--writable` flag
34
+
1. build into a directory that has writable permissions using the `--sandbox` option
35
+
2. build into an ext3 image file, that has writable permissions with the `--writable` option
36
+
37
+
In both cases you will need to execute your container with the `--writable` option at runtime for your changes to be persistant.
32
38
33
39
#### Sandbox Folder
34
40
To build into a folder (we call this a "sandbox") just ask for it:
You can make changes to the container (assuming you have the proper permissions to do so) but those changes will disappear as soon as you exit. To make your changes persistent across sessions, use the `--writable` option. It's also a good practice to shell into your container as root to ensure you have permissions to write where you like.
90
95
91
96
```
92
-
sudo singularity shell ubuntu
97
+
$ sudo singularity shell ubuntu
93
98
Singularity: Invoking an interactive shell within container...
If you don't want a folder, you can perform a similar development build and specify the `--writable`command.
104
+
If you prefer to work with a writable image file rather than a directory, you can perform a similar development build and specify the `--writable`option.
100
105
This will produce an image that is writable with an ext3 file system. Unlike the sandbox, it is a single image file.
The same is true as the above, you can use any commands like `shell`, `exec`, `run`, and if you want a writable image you must use sudo and the `--writable` flag.
128
+
You can use this image with commands like `shell`, `exec`, `run`, and if you want to change the image you must use the `--writable` flag. As before, it's a good idea to issue these commands as root to ensure you have the proper permissions to write.
124
129
125
130
```
126
-
sudo singularity shell --writable ubuntu.img
131
+
$ sudo singularity shell --writable ubuntu.img
127
132
```
128
133
129
134
>> Development Tip! When building containers, it often is the case that you will have a lot of
@@ -132,7 +137,7 @@ interactively write the build recipe with one of these writable containers, you
132
137
build the production (squashfs) container without worrying that it will error and need to be started again.
133
138
134
139
### 2. Production Commands
135
-
Let's set the scene - we just finished buliding our perfect hello world container. It does a fantastic hello-world analysis, and we have written a paper on it! We now want to build an immutable container - meaning that if someone obtained our container and tried to change it, they could not. They *could* easily use the same recipe that you used (it is provided as metadata inside the container), so your work can still be extended.
140
+
Let's set the scene - we just finished buliding our perfect hello world container. It does a fantastic hello-world analysis, and we have written a paper on it! We now want to build an immutable container - meaning that if someone obtained our container and tried to change it, they could not. They *could* easily use the same recipe that you used (it is provided as metadata inside the container), or convert your container to one of the writable formats above using `build`. So your work can still be extended.
136
141
137
142
#### Recommended Production Build
138
143
What we want for production is a build into a <ahref="https://en.wikipedia.org/wiki/SquashFS"target="_blank">squashfs image</a>. Squashfs is a read only, and compressed filesystem, and well suited for confident archive and re-use of your hello-world. To build a production image, just remove the extra options:
@@ -152,8 +157,6 @@ Building Singularity image...
152
157
Cleaning up...
153
158
Singularity container built: ubuntu.simg
154
159
```
155
-
You will also notice the extension `.simg`. This is the correct extension to designate a Singularity 2.4, production (squashfs) file.
156
-
157
160
#### Production Build from Sandbox
158
161
We understand that it might be wanted to build a Singularity (squashfs) from a previous development image. While we advocate for the first approach, we support this use case. To do this, given our folder called "ubuntu/" we made above:
It could be the case that a cluster maintains a "working" base of container folders (with writable) and then builds and provides production containers to its users.
164
167
165
168
166
-
That's it! You probably want to check out [interacting with images](/quickstart#interacting-with-images) to now use your container, sandbox, or new friend.
167
-
168
169
If you want to go through this entire process without having singularity installed locally, or without leaving your cluster, you can build images using <ahref="https://github.com/singularityhub/singularityhub.github.io/wiki"target="_blank">singularity hub.</a>
Copy file name to clipboardExpand all lines: pages/docs/user-docs/docs-quickstart.md
+6-20Lines changed: 6 additions & 20 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,12 +3,14 @@ title: Quick Start
3
3
sidebar: user_docs
4
4
permalink: quickstart
5
5
folder: docs
6
-
toc: true
6
+
toc: false
7
7
---
8
8
9
9
This guide is intended for running Singularity on a computer where you have root (administrative) privileges. But if you are learning about Singularity on a system where you lack root privileges you can still complete the steps that do not require the `sudo` command.
10
10
11
-
## Installation Quick Start
11
+
{% include toc.html %}
12
+
13
+
## Installation
12
14
There are many ways to [install Singularity](docs-installation) but this quick start guide will only cover one.
13
15
14
16
```bash
@@ -211,22 +213,12 @@ I am your father
211
213
```
212
214
213
215
## Build images from scratch
214
-
215
-
The diagram below shows how you can use Singularity to build images and run images. The high level idea is that we have two environments:
216
-
217
-
- a build environment (where you have root privileges) to test and build your container
218
-
- a production environment where you run your container (where you may or may not have root privileges)
In practice, your build system may or may not differ from your production system. If you want more details about the different build options, read about the [singularity flow](/docs-flow).
225
-
226
216
As of Singularity v2.4 by default `build` produces immutable images in the squashfs file format. This ensures reproducible and verifiable images.
227
217
228
218
However, during testing and debugging you may want an image format that is writable. This way you can `shell` into the image and install software and dependencies until you are satisfied that your container will fulfill your needs. For these scenarios, Singularity supports two other image formats: a `sandbox` format (which is really just a chroot directory), and a `writable` format (the ext3 file system that was used in Singularity versions less than 2.4).
229
219
220
+
For more details about the different build options and best practices, read about the [singularity flow](/docs-flow).
221
+
230
222
### Sandbox Directory
231
223
To build into a `sandbox` (container in a directory) use the `build --sandbox` command and option:
232
224
@@ -252,12 +244,6 @@ When you want to alter your image, you can use commands like `shell`, `exec`, `r
252
244
$ sudo singularity shell --writable ubuntu.img
253
245
```
254
246
255
-
>> Development Tip! When building containers, it often is the case that you will have a lot of
256
-
testing of installation commands, and if building a production image, one error will stop the entire build. If you
257
-
interactively write the build recipe with one of these writable formats, you can debug as you go, and then
258
-
build the production (squashfs) container without worrying that it will error and need to be started again.
259
-
260
-
261
247
### Converting images from one format to another
262
248
The `build` command allows you to build a container from an existing container. This means that you can use it to convert a container from one format to another. For instance, if you have already created a sandbox (directory) and want to convert it to the default immutable image format (squashfs) you can do so:
0 commit comments