This is a simple list of commands for load and manage docker image and containers.
Go to the folder where the dockerfile is istored. Call build command.
$ docker build -t 'name:tag' .
Sometimes, the buils can crash because there is not more space in disk. To solve this issue, stop and delete first the active container, and then remove the image.
Run the image with a bash interface:
$ docker run -it gcr.io/tensorflow/tensorflow:latest-gpu bash
Run the image with a notebook interface:
# With nvidia-docker v1
$ nvidia-docker run -it -p 8888:8888 gcr.io/tensorflow/tensorflow:latest-gpu
# With nvidia-docker v2
$ docker run --runtime=nvidia -it -p 8888:8888 gcr.io/tensorflow/tensorflow:latest-gpu
# With docker > 19.03
$ docker run --gpus=all -it -p 8888:8888 gcr.io/tensorflow/tensorflow:latest-gpu
Most common docker run options (more in docker docs).
This is an example of the previous options:
$ docker run \
--rm \
--detach \
--env KEY=VALUE \
--ip 10.10.9.75 \
--publish 3000:3000 \
--volume my_volume \
--name my_container \
--tty --interactive \
--volume /my_volume \
--workdir /app \
--entrypoint /app/superApp.sh \
IMAGE bash
In some cases, there are entrypoint erros. A known issue is related with the bash prompt:
/bin/bash: /bin/bash: cannot execute binary file
To fix this, simply remove the entry point defined in the dockerfile used for the image generation, or in runtime replace the entrypoint with the option:
--rm --entrypoint /bin/bash
You can specify the active user inside the container (which by default is root) to be used during the execution. This is specially useful if you need to access to elements restricted to certain users.
In the run command parameters, you need to add the -u option. To set the same user you are using to run the container:
-u `id -u $USER`
An example of the full call:
$ docker run -it gcr.io/tensorflow/tensorflow:latest-gpu -u `id -u $USER` bash
Show the downloaded images (use the option --digests to see the has of the image):
$ docker images
Show the running instances:
$ docker ps -a
Show the stopped instances:
$ docker ps --filter "status=exited"
If you include the option -q (quiet) returns only the essential information, and could be useful for listing elements to inject in other commands. For example:
$ docker stop $(docker ps -qa)
You can also filter the output of the docker ps
command for example to show:
docker ps --filter name=mongo
0
.
docker ps -a --filter exited=0
docker ps --filter status=running #exited, restarting, etc.
You can combine this with with -q
to show all containers that exited successfully. Try to create a command that will remove all such containers.
Remove an image on disk:
$ docker rmi <Image Id>
Remove all stopped images:
$ docker rmi $(docker ps -q)
Remove all containers:
$ docker container prune
Stop a container:
$ docker stop <Container id>
Remove a container:
$ docker rm <Container id>
Remove all stopped containers:
$ docker rm $(docker ps --filter "status=exited" -q)
We can check the logs (console outputs) of a stopped container using the logs option.
$ docker logs <docker container name or id>
You can see the logs
command has some useful flags docker logs --help
.
Start a new new Mongo container in detached mode and:
$ docker logs my-mongo
$ docker logs my-mongo -f
$ docker logs my-mongo --timestamps
$ docker logs --since 5m my-mongo
$ docker logs --tail 10 my-mongo
This shows the logs of the container printed during the execution.
The same command works with compose:
# docker compose logs <container name>
docker compose logs --follow kafka-ui
Include the changes in the image:
$ docker commit <container id> [repository[:tag]]
Change image name or tag:
$ docker tag <Image id> <image name>:<tag name>
Include the changes in the repo:
$ docker push <image name>:<tag name>
First, show the list of commits to select the reverting point.
$ docker history imagename
IMAGE CREATED CREATED BY SIZE
f770fc671f11 12 seconds ago apt-get install -y curl 21.3 MB
28445c70c2b3 39 seconds ago apt-get install ping 11.57 MB
8dbd9e392a96 7 months ago 131.5 MB
Then, select the desired commit,and copty the first part of the image hash.
$ docker tag 2844 imagename # <-- that's the secret right there
You can show the final status checking the commit history again:
$ docker history imagename
IMAGE CREATED CREATED BY SIZE
28445c70c2b3 56 seconds ago apt-get install ping 11.57 MB
8dbd9e392a96 7 months ago
To generate the zip file:
$ docker save -o <path for generated tar file> <image name>
To restore in the target machine:
$ docker load -i <path to image tar file>
How to set docker hub user and pass:
$ docker login
How to add network access to the container:
# add '--net=host' in the run call
$ docker run --net=host -it gcr.io/tensorflow/tensorflow:latest-gpu bas
How to connect with a running container:
$ docker exec -it <container id> bash
How to mount a host folder in the guest container:
$ docker run -it -v <host folder>:<guest folder> ...
How to share gui applications through ssh (source: https://dzone.com/articles/docker-x11-client-via-ssh):
$ xhost +
$ docker run --rm -ti \
-e DISPLAY=$DISPLAY \
-v /tmp/.X11-unix:/tmp/.X11-unix \
--net=host --volume="$HOME/.Xauthority:/root/.Xauthority:rw" \
tensorflow/tensorflow:1.5.0-devel-gpu bash
Run an image with internet connection, a shared folder, the x11 forwarding and sound forwarding.
$ sudo docker run --net=host -it -v /home/VICOMTECH/jgoenetxea/Downloads:/mnt/host_folder -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=unix$DISPLAY --device /dev/snd --name tensorflowlab jgoenetxea/tensorflowlab:gpu bash
# remove exited containers:
docker ps --filter status=dead --filter status=exited -aq | xargs -r docker rm -v
Remove unused images:
$ docker images --no-trunc | grep '<none>' | awk '{ print $3 }' | xargs -r docker rmi
Remove unused volumes:
$ find '/var/lib/docker/volumes/' -mindepth 1 -maxdepth 1 -type d | grep -vFf <(
docker ps -aq | xargs docker inspect | jq -r '.[] | .Mounts | .[] | .Name | select(.)'
) | xargs -r rm -fr
Docker uses the root folder (i.e. '/') as root folder. In some setups, the size of root folder ('/') is constrained for several reasons, but there is another partition with plenty of space. In those cases, we can configure docker to use a folder in that partition to work with.
First, check the current docker root folder:
$ sudo docker info | grep "Docker Root"
The default configuration shold be something similar to '/var/lib/docker'. Then, define a configuration file for docker daemon:
$ sudo mkdir /etc/systemd/system/docker.service.d/
$ sudo vim /etc/systemd/system/docker.service.d/docker.root.conf
and fill it with the new root path:
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -g /your/custom/docker/root -H fd://
Once the file is saved, reload the daemon:
$ sudo systemctl daemon-reload
$ sudo systemctl restart docker
$ docker info | grep "Docker Root"
If you want to maintain the configuration of previous ocker elements, you have to copy all the content of the original root fodler in the new path:
$ rsync -a /var/lib/docker/* /your/custom/docker/root
To check the disk usage from docker:
$ docker system df
To free all the un-used elements from the disk, including the volumes:
$ docker system prune -a --volumes
We also can filter the elements to remove using the filter parameter. As example, the following example removes the system elements older than a month (720h = 24h * 30days):
$ docker system prune -a --filter "until=720h"
NOTE: By default, you are prompted to continue. To bypass the prompt, use the -f or --force flag.
The most part of the trash comes from the build cache. To remove all the caché:
$ docker buildx prune -f
And you can also remove dangling images by:
$ docker rmi $(docker images -f "dangling=true" -q)
This command shows the network addresses used internaly and externally by all the containers in the composition:
$ docker inspect -f '{{.Name}} - {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $(docker ps -aq)
If you need to define a specific ip range, you can do it by including this in the end of the file or where the definition of the network exists:
More info in: https://docs.docker.com/engine/reference/commandline/system_prune/