Skip to content

Commit dee855c

Browse files
vsochvsoch
committed
updating docs with @GoDLovedc changes
1 parent af03d4e commit dee855c

5 files changed

Lines changed: 368 additions & 329 deletions

File tree

‎_data/sidebars/user_docs.yml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,14 @@ entries:
1919
url: /docs-installation
2020
output: web, pdf
2121

22+
- title: Quick Start
23+
url: /quickstart
24+
output: web, pdf
25+
26+
- title: Singularity Flow
27+
url: /docs-flow
28+
output: web, pdf
29+
2230
- title: Container Recipes
2331
url: /docs-recipes
2432
output: web, pdf

‎pages/docs/user-docs/docs-flow.md‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
---
2+
title: Singularity Flow
3+
sidebar: user_docs
4+
permalink: docs-flow
5+
folder: docs
6+
toc: true
7+
---
8+
9+
There are generally two ways to get images. You either want to pull an image file as is, or (more likely) build your own custom image. We will start with talking about build, and the many different use cases it affords.
10+
11+
12+
## Building Images
13+
If you read the [quick start](/quickstart), you probably remember that building images from a Docker base does not require a <a href="/docs-recipes">Singularity recipe</a>. However, if you *do* want to build and customize your image, you can create a <a href="/docs-recipes">Singularity recipe</a> text file, which is a simple text file that describes how the container should be made.
14+
15+
### The Singularity Flow
16+
The diagram below is a visual depiction of how you can use Singularity to build images. The high level idea is that we have two environments:
17+
18+
- a development environment (where you have sudo privileges) to test and build your container
19+
- a production environment where you run your container
20+
21+
<a href="/assets/img/diagram/singularity-2.4-flow.png" target="_blank" class="no-after">
22+
<img style="max-width:900px" src="/assets/img/diagram/singularity-2.4-flow.png">
23+
</a>
24+
25+
Singularity production images are immutable. This is a feature added as of Singularity 2.4, and it ensures a higher level of reproducibility and verification of images. To read more about the details, check out the <a href="/build">bulid</a>docs. However, immutability is not so great when you are testing, debugging, or otherwise want to quickly change your image. We will proceed by describing a typical workflow of developing first, building a final image, and using in production.
26+
27+
### 1. Development Commands
28+
If you want a writable image or folder for developing, you have two options:
29+
30+
1. build into a folder that has writable permissions with sudo
31+
2. build into an ext3 image file, also that has writable permissions with sudo and the `--writable` flag
32+
33+
#### Sandbox Folder
34+
To build into a folder (we call this a "sandbox") just ask for it:
35+
36+
```
37+
sudo singularity build --sandbox ubuntu/ docker://ubuntu
38+
Docker image path: index.docker.io/library/ubuntu:latest
39+
Cache folder set to /root/.singularity/docker
40+
Importing: base Singularity environment
41+
Importing: /root/.singularity/docker/sha256:9fb6c798fa41e509b58bccc5c29654c3ff4648b608f5daa67c1aab6a7d02c118.tar.gz
42+
Importing: /root/.singularity/docker/sha256:3b61febd4aefe982e0cb9c696d415137384d1a01052b50a85aae46439e15e49a.tar.gz
43+
Importing: /root/.singularity/docker/sha256:9d99b9777eb02b8943c0e72d7a7baec5c782f8fd976825c9d3fb48b3101aacc2.tar.gz
44+
Importing: /root/.singularity/docker/sha256:d010c8cf75d7eb5d2504d5ffa0d19696e8d745a457dd8d28ec6dd41d3763617e.tar.gz
45+
Importing: /root/.singularity/docker/sha256:7fac07fb303e0589b9c23e6f49d5dc1ff9d6f3c8c88cabe768b430bdb47f03a9.tar.gz
46+
Importing: /root/.singularity/metadata/sha256:22e289880847a9a2f32c62c237d2f7e3f4eae7259bf1d5c7ec7ffa19c1a483c8.tar.gz
47+
Building image from sandbox: ubuntu/
48+
Singularity container built: ubuntu/
49+
```
50+
51+
We now have a folder with the entire ubuntu OS, plus some Singularity metadata, plopped in our present working directory.
52+
53+
```
54+
tree -L 1 ubuntu
55+
ubuntu
56+
├── bin
57+
├── boot
58+
├── dev
59+
├── environment -> .singularity.d/env/90-environment.sh
60+
├── etc
61+
├── home
62+
├── lib
63+
├── lib64
64+
├── media
65+
├── mnt
66+
├── opt
67+
├── proc
68+
├── root
69+
├── run
70+
├── sbin
71+
├── singularity -> .singularity.d/runscript
72+
├── srv
73+
├── sys
74+
├── tmp
75+
├── usr
76+
└── var
77+
```
78+
79+
And you can shell into it just like a normal container. Without sudo, you don't have write permission:
80+
81+
```
82+
singularity shell ubuntu
83+
Singularity: Invoking an interactive shell within container...
84+
85+
Singularity ubuntu:~/Desktop> touch /hello.txt
86+
touch: cannot touch '/hello.txt': Permission denied
87+
```
88+
89+
With sudo, you do:
90+
91+
```
92+
sudo singularity shell ubuntu
93+
Singularity: Invoking an interactive shell within container...
94+
95+
Singularity ubuntu:/home/vanessa/Desktop> touch /hello.txt
96+
```
97+
98+
#### Writable Image
99+
If you don't want a folder, you can perform a similar development build and specify the `--writable` command.
100+
This will produce an image that is writable with an ext3 file system. Unlike the sandbox, it is a single image file.
101+
102+
```
103+
sudo singularity build --writable ubuntu.img docker://ubuntu
104+
Docker image path: index.docker.io/library/ubuntu:latest
105+
Cache folder set to /root/.singularity/docker
106+
Importing: base Singularity environment
107+
Importing: /root/.singularity/docker/sha256:9fb6c798fa41e509b58bccc5c29654c3ff4648b608f5daa67c1aab6a7d02c118.tar.gz
108+
Importing: /root/.singularity/docker/sha256:3b61febd4aefe982e0cb9c696d415137384d1a01052b50a85aae46439e15e49a.tar.gz
109+
Importing: /root/.singularity/docker/sha256:9d99b9777eb02b8943c0e72d7a7baec5c782f8fd976825c9d3fb48b3101aacc2.tar.gz
110+
Importing: /root/.singularity/docker/sha256:d010c8cf75d7eb5d2504d5ffa0d19696e8d745a457dd8d28ec6dd41d3763617e.tar.gz
111+
Importing: /root/.singularity/docker/sha256:7fac07fb303e0589b9c23e6f49d5dc1ff9d6f3c8c88cabe768b430bdb47f03a9.tar.gz
112+
Importing: /root/.singularity/metadata/sha256:22e289880847a9a2f32c62c237d2f7e3f4eae7259bf1d5c7ec7ffa19c1a483c8.tar.gz
113+
Building image from sandbox: /tmp/.singularity-build.VCHPpP
114+
Creating empty Singularity writable container 130MB
115+
Creating empty 162MiB image file: ubuntu.img
116+
Formatting image with ext3 file system
117+
Image is done: ubuntu.img
118+
Building Singularity image...
119+
Cleaning up...
120+
Singularity container built: ubuntu.img
121+
```
122+
123+
The same is true as the above, you can use any commands like `shell`, `exec`, `run`, and if you want a writable image you must use sudo and the `--writable` flag.
124+
125+
```
126+
sudo singularity shell --writable ubuntu.img
127+
```
128+
129+
>> Development Tip! When building containers, it often is the case that you will have a lot of
130+
testing of installation commands, and if building a production image, one error will stop the entire build. If you
131+
interactively write the build recipe with one of these writable containers, you can debug as you go, and then
132+
build the production (squashfs) container without worrying that it will error and need to be started again.
133+
134+
### 2. Production Commands
135+
Let's set the scene - we just finished buliding our perfect hello world container. It does a fantastic hello-world analysis, and we have written a paper on it! We now want to build an immutable container - meaning that if someone obtained our container and tried to change it, they could not. They *could* easily use the same recipe that you used (it is provided as metadata inside the container), so your work can still be extended.
136+
137+
#### Recommended Production Build
138+
What we want for production is a build into a <a href="https://en.wikipedia.org/wiki/SquashFS" target="_blank">squashfs image</a>. Squashfs is a read only, and compressed filesystem, and well suited for confident archive and re-use of your hello-world. To build a production image, just remove the extra options:
139+
140+
```
141+
sudo singularity build ubuntu.simg docker://ubuntu
142+
Docker image path: index.docker.io/library/ubuntu:latest
143+
Cache folder set to /root/.singularity/docker
144+
Importing: base Singularity environment
145+
Importing: /root/.singularity/docker/sha256:9fb6c798fa41e509b58bccc5c29654c3ff4648b608f5daa67c1aab6a7d02c118.tar.gz
146+
Importing: /root/.singularity/docker/sha256:3b61febd4aefe982e0cb9c696d415137384d1a01052b50a85aae46439e15e49a.tar.gz
147+
Importing: /root/.singularity/docker/sha256:9d99b9777eb02b8943c0e72d7a7baec5c782f8fd976825c9d3fb48b3101aacc2.tar.gz
148+
Importing: /root/.singularity/docker/sha256:d010c8cf75d7eb5d2504d5ffa0d19696e8d745a457dd8d28ec6dd41d3763617e.tar.gz
149+
Importing: /root/.singularity/docker/sha256:7fac07fb303e0589b9c23e6f49d5dc1ff9d6f3c8c88cabe768b430bdb47f03a9.tar.gz
150+
Importing: /root/.singularity/metadata/sha256:22e289880847a9a2f32c62c237d2f7e3f4eae7259bf1d5c7ec7ffa19c1a483c8.tar.gz
151+
Building Singularity image...
152+
Cleaning up...
153+
Singularity container built: ubuntu.simg
154+
```
155+
You will also notice the extension `.simg`. This is the correct extension to designate a Singularity 2.4, production (squashfs) file.
156+
157+
#### Production Build from Sandbox
158+
We understand that it might be wanted to build a Singularity (squashfs) from a previous development image. While we advocate for the first approach, we support this use case. To do this, given our folder called "ubuntu/" we made above:
159+
160+
```
161+
sudo singularity build ubuntu.simg ubuntu/
162+
```
163+
It could be the case that a cluster maintains a "working" base of container folders (with writable) and then builds and provides production containers to its users.
164+
165+
166+
That's it! You probably want to check out [interacting with images](/quickstart#interacting-with-images) to now use your container, sandbox, or new friend.
167+
168+
If you want to go through this entire process without having singularity installed locally, or without leaving your cluster, you can build images using <a href="https://github.com/singularityhub/singularityhub.github.io/wiki" target="_blank">singularity hub.</a>
169+
170+
{% include links.html %}

0 commit comments

Comments
 (0)