Cacher is the code snippet organizer for pro developers

This is a simple list of commands for load and manage docker image and containers.

How to build a docker image

Go to the folder where the dockerfile is istored. Call build command.

$ docker build -t 'name:tag' .

Sometimes, the buils can crash because there is not more space in disk. To solve this issue, stop and delete first the active container, and then remove the image.

How to run an image as container

Run the image with a bash interface:

$ docker run -it gcr.io/tensorflow/tensorflow:latest-gpu bash  

Run the image with a notebook interface:

# With nvidia-docker v1
$ nvidia-docker run -it -p 8888:8888 gcr.io/tensorflow/tensorflow:latest-gpu
# With nvidia-docker v2
$ docker run --runtime=nvidia -it -p 8888:8888 gcr.io/tensorflow/tensorflow:latest-gpu
# With docker > 19.03
$ docker run --gpus=all -it -p 8888:8888 gcr.io/tensorflow/tensorflow:latest-gpu

Docker run configuration parameters

Most common docker run options (more in docker docs).

1. --detach, -d: By default a Docker container is run attached to local standard input, output, and error streams. The -d, --detach option runs the container in the background of your terminal session so its output is not displayed. This option is covered in more detail in Docker’s detached mode for beginners: How to run containers in the background of your terminal
2. --entrypoint: Set or overwrite the default entrypoint command for the image. The entrypoint sets the command and parameters that will be executed first when a container is run. Any commands and arguments passed at the end of the docker run command will be appended to the entrypoint. To learn more about using Entrypoint, check out Docker ENTRYPOINT & CMD: Dockerfile best practices
3. --env, -e: Set an environment variable using a KEY=VALUE pair. If you have environment variables in a file, you can pass in the file path to the option --env-file.
4. --ip: Declare an IP address, for example --ip=10.10.9.75
5. --name: Assign a name to the container, --name my_container
6. --publish, -p | --publish-all, -P: These publish port mappings between the container and host that are defined in an image’s Dockerfile or by using the expose option, --expose. The option --publish, -p publishes a container’s port(s) to the host, while --publish-all , -P publishes all exposed ports. You can learn more about exposing and defining ports in Expose vs publish: Docker port commands explained simply
7. --rm: Automatically remove the container when it exits. The alternative would be to manually stop it and then remove it, for more on how to do this see: How to delete Docker containers from the command line
8. --tty, -t: Allocate a virtual terminal session within the container. This is commonly used with the option --interactive, -i, which keeps STDIN open even if running in detached mode. One of the most common uses of -i -t is to run a command, such as bash, in a container, which you can read more about in my post Run bash or any command in a Docker container.
9. --interactive , -i: Keep STDIN open even if not attached.
10. --volume, -v: Mount a volume -v /my_volume. If you are new to volumes then find out more in Docker’s guide to Volumes.
11. --workdir , -w: State the working directory inside the container. For example, if you copy your files into an app folder within your container then you may want to set the working directory to app.
12. --restart: Restart policy to apply when a container exits (in conflict with --rm). Posible values: no, on-failure[:max-retries], unless-stopped amd always
13. --gpus: Defines the GPU devices available from the container. You can set as --gpus=all to include all instaled GPUs or define a subset of them by writing --gpus='"device=0,2"' for devices with index 0 and 2.

This is an example of the previous options:

$ docker run \ 
  --rm \ 
  --detach \ 
  --env KEY=VALUE \
  --ip 10.10.9.75 \
  --publish 3000:3000 \
  --volume my_volume \
  --name my_container \
  --tty --interactive \
  --volume /my_volume \
  --workdir /app \ 
  --entrypoint /app/superApp.sh \
  IMAGE bash

Possible entry errors

In some cases, there are entrypoint erros. A known issue is related with the bash prompt:

/bin/bash: /bin/bash: cannot execute binary file

To fix this, simply remove the entry point defined in the dockerfile used for the image generation, or in runtime replace the entrypoint with the option:

--rm --entrypoint  /bin/bash

How to run docker with a system user

You can specify the active user inside the container (which by default is root) to be used during the execution. This is specially useful if you need to access to elements restricted to certain users.

In the run command parameters, you need to add the -u option. To set the same user you are using to run the container:

-u `id -u $USER`

An example of the full call:

$ docker run -it gcr.io/tensorflow/tensorflow:latest-gpu -u `id -u $USER` bash 

How to check instances and images

Show the downloaded images (use the option --digests to see the has of the image):

$ docker images

Show the running instances:

$ docker ps -a

Show the stopped instances:

$ docker ps --filter "status=exited"

If you include the option -q (quiet) returns only the essential information, and could be useful for listing elements to inject in other commands. For example:

$ docker stop $(docker ps -qa)

Filter the existing containers

You can also filter the output of the docker ps command for example to show:

• Filter and show only the containers which name contains the word mongo.

  docker ps --filter name=mongo
  

• Show only the containers with an exit code 0.

  docker ps -a --filter exited=0
  

• Show only the containers which are in a specific status (e.g. running, created,

  docker ps --filter status=running #exited, restarting, etc.
  

You can combine this with with -q to show all containers that exited successfully. Try to create a command that will remove all such containers.

How to remove images and containers

Remove an image on disk:

$ docker rmi <Image Id>

Remove all stopped images:

$ docker rmi $(docker ps -q)

Remove all containers:

$ docker container prune

Stop a container:

$ docker stop <Container id>

Remove a container:

$ docker rm <Container id>

Remove all stopped containers:

$ docker rm $(docker ps --filter "status=exited" -q)

How to check the logs of a stopped container

We can check the logs (console outputs) of a stopped container using the logs option.

$ docker logs <docker container name or id>

Play with the logs command

You can see the logs command has some useful flags docker logs --help.

Start a new new Mongo container in detached mode and:

• Show the current logs.

  $ docker logs my-mongo
  

• Follow the logs (shown in real time).

  $ docker logs my-mongo -f
  

  • You can exit just typing Control+C
• Show the logs with a timestamp at the beginning of each line.

  $ docker logs my-mongo --timestamps
  

• Show the logs generated during the last 5 minutes.

  $ docker logs --since 5m my-mongo
  

• Show the last 10 lines of logs.

  $ docker logs --tail 10 my-mongo
  

This shows the logs of the container printed during the execution.

The same command works with compose:

# docker compose logs <container name> 
docker compose logs --follow kafka-ui 

How to make the changes in a container presistent

Include the changes in the image:

$ docker commit <container id> [repository[:tag]]

Change image name or tag:

$ docker tag <Image id> <image name>:<tag name>

Include the changes in the repo:

$ docker push <image name>:<tag name>

Revert to previous version/commit

First, show the list of commits to select the reverting point.

$ docker history imagename

IMAGE               CREATED             CREATED BY                SIZE
f770fc671f11        12 seconds ago      apt-get install -y curl   21.3 MB
28445c70c2b3        39 seconds ago      apt-get install ping      11.57 MB
8dbd9e392a96        7 months ago                                  131.5 MB

Then, select the desired commit,and copty the first part of the image hash.

$ docker tag 2844 imagename   # <-- that's the secret right there

You can show the final status checking the commit history again:

$ docker history imagename
IMAGE               CREATED             CREATED BY             SIZE
28445c70c2b3        56 seconds ago      apt-get install ping   11.57 MB
8dbd9e392a96        7 months ago  

Generate compressed docker image

To generate the zip file:

$ docker save -o <path for generated tar file> <image name>

To restore in the target machine:

$ docker load -i <path to image tar file>

Other considerations

How to set docker hub user and pass:

$ docker login

How to add network access to the container:

# add '--net=host' in the run call
$ docker run --net=host -it gcr.io/tensorflow/tensorflow:latest-gpu bas

How to connect with a running container:

$ docker exec -it <container id> bash

How to mount a host folder in the guest container:

$ docker run -it -v <host folder>:<guest folder> ...

How to share gui applications through ssh (source: https://dzone.com/articles/docker-x11-client-via-ssh):

$ xhost +
$ docker run --rm -ti \
  -e DISPLAY=$DISPLAY \
  -v /tmp/.X11-unix:/tmp/.X11-unix \
  --net=host --volume="$HOME/.Xauthority:/root/.Xauthority:rw" \
  tensorflow/tensorflow:1.5.0-devel-gpu bash

Example runs

Run an image with internet connection, a shared folder, the x11 forwarding and sound forwarding.

$ sudo docker run --net=host -it  -v /home/VICOMTECH/jgoenetxea/Downloads:/mnt/host_folder   -v /tmp/.X11-unix:/tmp/.X11-unix     -e DISPLAY=unix$DISPLAY     --device /dev/snd     --name tensorflowlab     jgoenetxea/tensorflowlab:gpu bash

# remove exited containers:

docker ps --filter status=dead --filter status=exited -aq | xargs -r docker rm -v

Remove unused images:

$ docker images --no-trunc | grep '<none>' | awk '{ print $3 }' | xargs -r docker rmi

Remove unused volumes:

$ find '/var/lib/docker/volumes/' -mindepth 1 -maxdepth 1 -type d | grep -vFf <(
  docker ps -aq | xargs docker inspect | jq -r '.[] | .Mounts | .[] | .Name | select(.)'
) | xargs -r rm -fr

Change the docker root folder

Docker uses the root folder (i.e. '/') as root folder. In some setups, the size of root folder ('/') is constrained for several reasons, but there is another partition with plenty of space. In those cases, we can configure docker to use a folder in that partition to work with.

First, check the current docker root folder:

$ sudo docker info | grep "Docker Root"

The default configuration shold be something similar to '/var/lib/docker'. Then, define a configuration file for docker daemon:

$ sudo mkdir /etc/systemd/system/docker.service.d/
$ sudo vim /etc/systemd/system/docker.service.d/docker.root.conf

and fill it with the new root path:

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -g /your/custom/docker/root -H fd://

Once the file is saved, reload the daemon:

$ sudo systemctl daemon-reload
$ sudo systemctl restart docker
$ docker info | grep "Docker Root"

If you want to maintain the configuration of previous ocker elements, you have to copy all the content of the original root fodler in the new path:

$ rsync -a /var/lib/docker/* /your/custom/docker/root

Check docker disk usage

To check the disk usage from docker:

$ docker system df

Cleaning

To free all the un-used elements from the disk, including the volumes:

$ docker system prune -a --volumes

We also can filter the elements to remove using the filter parameter. As example, the following example removes the system elements older than a month (720h = 24h * 30days):

$ docker system prune -a --filter "until=720h"

NOTE: By default, you are prompted to continue. To bypass the prompt, use the -f or --force flag.

The most part of the trash comes from the build cache. To remove all the caché:

$ docker buildx prune -f

And you can also remove dangling images by:

$ docker rmi $(docker images -f "dangling=true" -q)

Check docker and docker compose network configuration

This command shows the network addresses used internaly and externally by all the containers in the composition:

$ docker inspect -f '{{.Name}} - {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $(docker ps -aq)

If you need to define a specific ip range, you can do it by including this in the end of the file or where the definition of the network exists:

More info in: https://docs.docker.com/engine/reference/commandline/system_prune/