Container Building Guide¶
Build commands are tagged values in your container definition. For example:
containers:
ubuntu:
setup:
- !Ubuntu trusty
- !Install [python]
This contains two build commands !Ubuntu
and !Install
. They mostly
run sequentially, but some of them are interesting, for example
!BuildDeps
installs package right now, but also removes package at
the end of the build to keep container smaller and cleaner.
See Build Steps for additional details on specific commands. There is also an Index
Generic Installers¶
To run arbitrary shell command use !Sh
:
setup:
- !Ubuntu trusty
- !Sh "apt-get install -y python"
If you have more than one-liner you may use YAMLy literal syntax for it:
setup:
- !Ubuntu trusty
- !Sh |
wget somepackage.tar.gz
tar -xzf somepackage.tar.gz
cd somepackage
make && make install
Warning
The !Sh
command is run by /bin/sh -exc
. With the flags meaning
-e
– exit if any command fails, -x
– print command before executing,
-c
– execute command. You may undo -ex
by inserting set +ex
at the start of the script. But it’s not recommended.
To run !Sh
you need /bin/sh
. If you don’t have shell in container you
may use !Cmd
that runs command directly:
setup:
# ...
- !Cmd [/usr/bin/python, '-c', 'print "hello from build"']
To install a package of any (supported) linux distribution just use
!Install
command:
containers:
ubuntu:
setup:
- !Ubuntu trusty
- !Install [python]
ubuntu-precise:
setup:
- !Ubuntu precise
- !Install [python]
alpine:
setup:
- !Alpine v3.1
- !Install [python]
Occasionally you need some additional packages to use for container building,
but not on final machine. Use !BuildDeps
for them:
setup:
- !Ubuntu trusty
- !Install [python]
- !BuildDeps [python-dev, gcc]
- !Sh "make && make install"
The python-dev
and gcc
packages from above will be removed after
building whole container.
To add some environment arguments to subsequent build commands use !Env
:
setup:
# ...
- !Env
VAR1: value1
VAR2: value2
- !Sh "echo $VAR1 / $VAR2"
Note
The !Env
command doesn’t add environment variables for processes
run after build. Use environ
setting for that.
Sometimes you want to rebuild container when some file changes. For example
if you have used the file in the build. There is a !Depends
command which
does nothing per se, but add a dependency. The path must be relative to your
project directory (the dir where vagga.yaml
is). For example:
setup:
# ...
- !Depends requirements.txt
- !Sh "pip install -r requirements.txt"
To download and unpack tar archive use !Tar
command:
setup:
- !Tar
url: http://something.example.com/some-project-1.0.tar.gz
sha256: acd1234...
path: /
subdir: some-project-1.0
Only url
field is mandatory. If url
starts with dot .
it’s treated
as filename inside project directory. The path
is target path to unpack
into, and subdir
is a dir inside tar file. By default path
is root of
new filesystem. The subdir
is a dir inside the tar file, if omitted whole
tar archive will be unpacked.
You can use !Tar
command to download and unpack the root filesystem from
scratch.
There is a shortcut to download tar file and build and install from there,
which is !TarInstall
:
setup:
- !TarInstall
url: http://static.rust-lang.org/dist/rust-0.12.0-x86_64-unknown-linux-gnu.tar.gz
sha256: abcd1234...
subdir: rust-0.12.0-x86_64-unknown-linux-gnu
script: ./install.sh --prefix=/usr
Only the url
is mandatory here too. Similarly, if url
starts with dot
.
it’s treated as filename inside project directory. The script
is by
default ./configure --prefix=/usr; make; make install
. It’s run in
subdir
of unpacked archive. If subdir
is omitted it’s run in the only
subdirectory of the archive. If archive contains more than one directory and
subdir
is empty, it’s an error, however you may use .
as subdir
.
To remove some data from the image after building use !Remove
command:
setup:
# ...
- !Remove /var/cache/something
To clean directory but ensure that directory exists use !EmptyDir
command:
setup:
# ...
- !EmptyDir /tmp
Note
The /tmp
directory is declared as !EmptyDir
implicitly for
all containers.
To ensure that directory exists use !EnsureDir
command. It’s very often
used for future mount points:
setup:
# ...
- !EnsureDir /sys
- !EnsureDir /dev
- !EnsureDir /proc
Note
The /sys
, /dev
and /proc
directories are created
automatically for all containers.
Sometimes you want to keep some cache between builds of container or similar
containers. Use !CacheDirs
for that:
setup
# ...
- !CacheDirs { "/var/cache/apt": "apt-cache" }
Mutliple directories may be specified at once.
Warning
The “apt-cache” name is a name of the directory like
.vagga/.cache/apt-cache
. So the directory is shared both between
all the containers and all the different builders (not only same versions
of the single container). In case user enabled shared-cache
the folder
will be also shared between containers of different projects.
Sometimes you just want to write a file in target system:
setup:
# ...
- !Text
/etc/locale.conf: |
LANG=en_US.UTF-8
LC_TIME=uk_UA.UTF-8
Note
You can use any YAML’y syntax for file body just the “literal” one
which starts with a pipe |
character is the most handy one
Ubuntu¶
To install base ubuntu system use:
setup:
- !Ubuntu trusty
Potentially any ubuntu long term support release instead of trusty
should
work. To install a non LTS release, use:
setup:
- !UbuntuRelease { version: 14.10 }
To install any ubuntu package use generic !Install
command:
setup:
- !Ubuntu trusty
- !Install python
Many interesting ubuntu packages are in the “universe” repository, you may add
it by series of !UbuntuRepo
commands (see below), but there is shortcut
!UbuntuUniverse
:
setup:
- !Ubuntu trusty
- !UbuntuUniverse
- !Install [checkinstall]
The !UbuntuRepo
command adds additional repository. For example, to add
marathon repository you may write:
setup:
- !Ubuntu trusty
- !UbuntuRepo
url: http://repos.mesosphere.io/ubuntu
suite: trusty
components: [main]
- !Install [mesos, marathon]
This effectively adds repository and installs mesos
and marathon
packages.
Note
Probably the key for repository should be added to be able to install packages.
Alpine¶
To install base alpine system use:
setup:
- !Alpine v3.1
Potentially any alpine version instead of v3.1
should work.
To install any alpine package use generic !Install
command:
setup:
- !Alpine v3.1
- !Install [python]
Npm Installer¶
You can build somewhat default nodejs environment using !NpmInstall
command. For example:
setup:
- !Ubuntu trusty
- !NpmInstall [react-tools]
All node packages are installed as --global
which should be expected. If
no distribution is specified before the !NpmInstall
command, the implicit
!Alpine v3.1
(in fact the latest version) will be executed.
setup:
- !NpmInstall [react-tools]
So above should just work as expected if you don’t need any special needs. E.g. it’s usually perfectly ok if you only use node to build static scripts.
The following npm
features are supported:
- Specify
package@version
to install specific version (recommended) - Use
git://
url for the package. In this case git will be installed for the duration of the build automatically - Bare
package_name
(should be used only for one-off environments)
Other forms may work, but are unsupported for now.
Note
The npm
and additional utilities (like build-essential
and
git
) will be removed after end of container building. You must
!Install
them explicitly if you rely on them later.
Python Installer¶
There are two separate commands for installing packages for python2 and python3. Here is a brief example:
setup:
- !Ubuntu trusty
- !Py2Install [sphinx]
Currently packages are installed by system pip. We consider this an
implementation detail and will use latest pip in future. The python-dev
headers are installed for the time of the build too. Both python-dev
and
pip
are removed when installation is finished.
The following pip
package specification formats are supported:
- The
package_name==version
to install specific version (recommended) - Bare
package_name
(should be used only for one-off environments) - The
git+
andhg+
links (the git and mercurial are installed as build dependency automatically), since vagga 0.4git+https
andhg+https
are supported too (required installingca-ceritificates
manually before)
All other forms may work but not supported. Specifying command-line arguments
instead of package names is not supported. To configure pip use !PipConfig
directive. In the example there are full list of parameters:
setup:
- !Ubuntu trusty
- !PipConfig
index-urls: ["http://internal.pypi.local"]
find-links: ["http://internal.additional-packages.local"]
dependencies: true
- !Py2Install [sphinx]
They should be self-descriptive. Note unlike in pip command line we use single list both for primary and “extra” indexes. See pip documentation for more info about options
Note
By default dependencies
is false. Which means pip is run with
--no-deps
option. Which is recommended way for setting up isolated
environments anyway. Even setuptools
are not installed by default.
To see list of dependencies and their versions you may use
pip freeze
command.
Better way to specify python dependencies is to use “requirements.txt”:
setup:
- !Ubuntu trusty
- !Py3Requirements "requirements.txt"
This works the same as Py3Install
including auto-installing of version
control packages and changes tracking. I.e. It will rebuild container when
“requirements.txt” change. So ideally in python projects you may use two lines
above and that’s it.
The Py2Requirements
command exists too.
Note
The “requirements.txt” is checked semantically. I.e. empty lines and comments are ignored. In current implementation the order of items is significant but we might remove this restriction in the future.
Dependent Containers¶
Sometimes you want to build on top of another container. For example, container
for running tests might be based on production container, but it might add some
test utils. Use !Container
command for that:
container:
base:
setup:
- !Ubuntu trusty
- !Py3Install [django]
test:
setup:
- !Container base
- !Py3Install [nosetests]
It’s also sometimes useful to freeze some part of container and test next build steps on top of of it. For example:
container:
temporary:
setup:
- !Ubuntu trusty
- !TarInstall
url: http://download.zeromq.org/zeromq-4.1.0-rc1.tar.gz
web:
setup:
- !Container temporary
- !Py3Install [pyzmq]
In this case when you try multiple different versions of pyzmq, the zeromq
itself will not be rebuilt. When you’re done, you can append build steps and
remove the temporary
container.
Sometimes you need to generate (part of) vagga.yaml
itself. For some things
you may just use shell scripting. For example:
container:
setup:
- !Ubuntu trusty
- !Env { VERSION: 0.1.0 }
- !Sh "apt-get install somepackage==$VERSION"
Note
Environment of user building container is always ignored during build process (but may be used when running command).
In more complex scenarios you may want to generate real vagga.yaml
. You may
use that with ancillary container and !SubConfig
command. For example, here
is how we use a docker2vagga script to transform Dockerfile
to vagga
config:
docker-parser: ❶
setup:
- !Alpine v3.1
- !Install [python]
- !Depends Dockerfile ❷
- !Depends docker2vagga.py ❷
- !Sh 'python ./docker2vagga.py > /docker.yaml' ❸
somecontainer:
setup:
- !SubConfig
source: !Container docker-parser ❶
path: docker.yaml ❹
container: docker-smart ❺
Few comments:
- ❶ – container used for build, it’s rebuilt automatically as a dependency for “somecontainer”
- ❷ – normal dependency rules apply, so you must add external files that are used to generate the container and vagga file in it
- ❸ – put generated vagga file inside a container
- ❹ – the “path” is relative to the source if the latter is set
- ❺ – name of the container used inside a “docker.yaml”
Warning
The functionality of !SubConfig
is experimental and is a
subject to change in future. In particular currently the /work
mount
point and current directory used to build container are those of initial
vagga.yaml
file. It may change in future.
The !SubConfig
command may be used to include some commands from another
file without building container. Just omit generator
command:
subdir:
setup:
- !SubConfig
path: subdir/vagga.yaml
container: containername
The YAML file used may be a partial container, i.e. it may contain just few commands, installing needed packages. The other things (including the name of the base distribution) can be set by original container:
# vagga.yaml
containers:
ubuntu:
setup:
- !Ubuntu trusty
- !SubConfig
path: packages.yaml
container: packages
alpine:
setup:
- !Alpine v3.1
- !SubConfig
path: packages.yaml
container: packages
# packages.yaml
containers:
packages:
setup:
- !Install [redis, bash, make]