Docker is an amazing tool for developers. It allows us to create and replicate images on any host, eliminating inconsistencies in development environments and greatly reducing integration time.
To provide an example of how you can move to containerized development, I've created a simple
allAPI comPiton,Django REST Frameworkand PostgreSQL usingComposition of the dockerto development, testing, and finally into my CI/CD pipeline.
In a two-part series, I'll cover the steps of developing and creating pipelines. In this post, I'll cover the first part: developing and testing with Docker Compose.
Requirements for this tutorial
This tutorial requires you to have a few items before beginning.
install on pcCommunity Edition of Docker
Instalar en pcComposition of the docker
DescargarTodo app sample: non docker branch
The to-do app here is essentially a plugin, and you can replace it with your own app. Some of the settings here are specific to this app. It may not cover your application needs, but it should be a good starting point for getting the concepts you need to dockerize your own applications.
After setting everything up, you can move on to the next section.
Creating the Dockerfile
At the base of any docked app you will find a
Dockerfilecontains all the instructions used to create the application image. You can configure this by installing Python and all of its dependencies. However, the Docker ecosystem has an image repository with aPitonimage already created and ready to use.
In the application root directory, create a new
/> toque Dockerfile
Open the newly created
Dockerfilein your favorite editor. The first instruction
SINCE, it will tell Docker to use the prebuilt Python image. There are several options, but this project uses the
python: 3.6.1-alpinePicture. For more details on why I'm using
alpinehere about the other options you can readthis post.
FROM python: 3.6.1-alpine
if you run
docker construction., you will see something similar to the following:
Sending build context to Docker daemon 10.53MBStep 1/1 : FROM python:3.6.1-alpine3.6.1-alpine: Pulling from library/python90f4dba627d6: Pull complete19bc0bb0be9f: Pull completee05eff433916: Pull completee70196200a87: Pull completea6d780959950: Pull completeDigest: sha256:0945574465b917d524ce9b748479a286c2ed3c5a97311ac5950464907d4d8b53Status: Imagen más reciente bajada para python:3.6.1-alpine ---> ddd6300d05a3Construído con éxito ddd6300d05a3
With just one declaration in the Dockerfile, this doesn't do much, but it shows the build process without much happening. At this point, you already have an image created and working.
dockers imagesIt will show you the images you have available:
REPOSITOR LABEL IMAGE ID SIZE CREATEDpython 3.6.1-alpine ddd6300d05a3 5 weeks ago 88.7 MB
Dockerfileyou need more instructions to build the app. Currently it's just building an image with python installed, but we still need our application code to run inside the container. Let's add some more instructions for doing this and build this image again.
This specific Dockerfile usesCORRE,COPY OF, miWORK DIRECTOR🇧🇷 You can read more about them on the Docker reference page to get a deeper understanding.
Let's add the instructions to the
DE python:3.6.1-alpineRUN apk actualizar \ && apk agregar \ build-base \ postgresql \ postgresql-dev \ libpqRUN mkdir /usr/src/appWORKDIR /usr/src/appCOPY ./requirements.txt .RUN pip install -r requisitos.txtENV PYTHONUNBUFERED 1COPY . .
This is what is happening:
apkpackages and then install some additional requirements
set working directory
requirements.txtto working directory
Run the pip installation with the
Set the environment variable
SEARCHLESS PYTHONfor 1(Video) How to run Python scripts in Dokcer using Docker Compose
Copy all the files from the project root to the working directory
now you can run
docker construction.again and see the results:
Send build context to Docker Daemon 10.53MB Step 1/8: FROM python:3.6.1-alpine ---> ddd6300d05a3 Step 2/8: RUN apk update && apk add build base postgresql postgresql-dev libpq -- -> Running on 22bfdcf8a0ddfetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/alpine/v3.4/ community/x86_64/APKINDEX .tar.gzv3.4.6-165-g6b9a79f [http://dl-cdn.alpinelinux.org/alpine/v3.4/main]v3.4.6-160-g14ad2a3 [http://dl- cdn.alpinelinux.org /alpine/v3.4/community]OK: 5974 different packages available## APK packages installed ##Running busybox-1.24.2-r13.triggerOK: 215 MiB in 65 packages ---> 782357cafeceRemoving the container intermediate 22bfdcf8a0ddStep 3/8: ENV PYTHONUNBUFFERED 1 ---> Running at 0d6de64f5f8b ---> 609106526013 Removing intermediate container 0d6de64f5f8b Step 4/8: RUN mkdir /usr/src/app ---> Running at b30ac1098156 17def88f6c5f Removing the interm container edio 0d6de64f5f8b /usr/src/application ---> 158d43ac2a47Deleting the intermediate container int fef446ea4ed0Step 6/8: COPY ./requirements.txt . ---> 2751d1f4c313Removing the intermediate container a49b090cb8e3Step 7/8: RUN pip install -r requirements.txt ---> Running on eaf9912e4810## Installed PIP requirements ##Installing collected packages: pytz, django, djangorestframework, psycopg2, django -cors - headers, dj-database-url, gunicorn, virtualenvSuccessfully installed django-1.11.3 dj-database-url-0.4.2 django-cors-headers-2.1.0 djangorestframework-3.6.3 gunicorn-19.7.1 psycopg2 -2.7 .2 pytz-2017.2 virtualenv-15.1.0 ---> dfcca42a45b0Removing the intermediary container eaf9912e4810Step 8/8: COPY. 🇧🇷 ---> 61f356fba3eaRemoving the intermediate container fda87c3db8cfSuccessfully built 61f356fba3ea
You have now successfully created the application image using Docker. However, currently our app won't do much as we still need a database and want to connect everything. This is where Docker Compose will help us.
Docker Composition Services
Now that you know how to create an image with a
Dockerfile, we are going to create an application as a service and connect it to a database. We can then run some configuration commands and start creating this new task list.
create the file
/> toque docker-compose.yml
The Docker Compose file will define and run the containers based on a configuration file. We are usingcompose file version 3syntax, and you can read it on the Docker website.
An important concept to understand is that Docker Compose works at "runtime". So far, we have built images using
docker construction.-- this is the "compile time". This is especially important when we add things like
domainbecause they will overwrite what is configured in "buildtime".
For example, him
usr/src/applicationThe directory will be created during compilation. We then assign that directory to the host machine and the application code from the host machine will be used during "runtime". This allows us to make changes locally and they are accessible within the container.
docker-compose.ymlin your editor and copy and paste the following lines:
version: '3' services: web: build: . command: gunicorn -b 0.0.0.0:8000 allapp.wsgi:application depends on: - postgres volumes: - .:/usr/src/app ports: - environment "8000:8000": DATABASE_URL: postgres://todoapp @postgres/ all postgres: image: postgres: 9.6.2-alpine environment: POSTGRES_USER: allapp POSTGRES_DB: all
It'll take a while to unpack, but we'll break it down by service.
the web service
The first directive in the web service is
build upthe image based on our
Dockerfile🇧🇷 This will recreate the image we used before, but now it will have the name of the project we are in,
Name🇧🇷 After that, we'll give the service some specific instructions on how it should work:
command: gunicorn -b 0.0.0.0:8000 todosapp.wsgi:application- Once the image is created and the container is running, this command will start the application.(Video) How to create a great dev environment with Docker
depends on:- This will tell Docker Compose to start the
postgresservice when the
the netthe service is executed.
volumes:- This section will establish routes between the host and the container.
.:/usr/src/app- This will mount the root directory to our working directory in the container.
environment:- The application itself expects the environment variable.
ports:- This will publish the port of the container, in this case
8000, to house as a port
DATABASE_URLis the connection string.
postgres://todoapp@postgres/todosconnect using the
All applicationsuser, on host
postgres, using the database
The Postgres service
Like the Python image we used, Docker Store has a pre-built image forPostgreSQLName🇧🇷 Instead of using a
build updirective, we can use the image name and Docker will take that image for us and use it. In this case, we are using
postgres:9.6.2-alpino🇧🇷 We could leave it like that, but it has
environmentvariables so we can customize it a bit.
environment:- This particular image accepts some environment variables so we can customize things to our needs.
POSTGRES_USER: allapplication- This creates the user
All applicationsas the default PostgreSQL user.
POSTGRES_DB: all- This will create the default database as
running the application
Now that we have our services defined, we can build the application using
docker-compose🇧🇷 This will show the images being built and finally started. After the initial build, you will see the names of the containers being built.
Pulling postgres (postgres:9.6.2-alpine)...9.6.2-alpine: Pulling from library/postgrescfc728c1c558: Pull completeb749e72b24f9: Pull complete0abdb8c9c36b: Pull complete90ca3848ef7e: Pull complete3ecf037a5034: Pull complete9327e3c5554c: Pull complete3133782bad17: Pull complete143bac6c8910: Pull completed6da9f4bd18e: Pull completeDigest: sha256:f88000211e3c682e7419ac6e6cbd3a7a4980b483ac416a3b5d5ee81d4f831cc9Status: Downloaded newer image for postgres:9.6.2-alpineBuilding web...Creating pythondjangotodoapp_postgres_1 ...Creating pythondjangotodoapp_postgres_1 ... doneCreating pythondjangotodoapp_web_1 ...Creating pythondjangotodoapp_web_1 ... done...web_1 | [2017-08-03 13:23:18 +0000] () [INFO] Iniciando gunicorn 19.7.1web_1 | [2017-08-03 13:23:18 +0000] () [INFO] Ouvindo em: http://0.0.0.0:8000 (1)
At this point, the application is running and you will see the log output in the console. You can also run the services as a background process using
docker-compose up -d🇧🇷 During development, I prefer to run without
-dand create a second terminal window to run other commands. If you want to run it as a background process and see the logs, you can run
logs do docker-compose.
In a new command prompt you can run
docker-compose psto see your running containers. You should see something like the following:
Name Command Status Ports------------------------------------------- --- ----------------------------------------------- --- pythondjangotodoapp_postgres_1 docker-entrypoint.sh postgres Up to 5432/tc pythondjangotodoapp_web_1 gunicorn -b 0.0.0.0:8000 t ... Up to 0.0.0.0:8000->8000/tcp
This will tell you the name of the services, the command used to start them, their current state, and the ports. Notice
pythondjangotodoapp_web_1listed the port as
0.0.0.0:8000->8000/tcp🇧🇷 This tells us that you can access the app using
local host: 8000/allon the host machine.
Migrate the database schema
One small but important step that should not be overlooked is migrating the schema to the database. Composing comes with a
executivecommand that will execute a single command in a running container. The typical function for migrating schemas is
move python manage.py🇧🇷 We can run it in the web service using
/> docker-compose exec web python manage.py migrationOperations to perform: Apply all migrations: admin, auth, contenttypes, sessions, allRun migrations: Apply contenttypes.0001_initial... OK Apply auth.0001_initial... OK Apply admin. 0001_initial ... OK Aplicando admin.0002_logentry_remove_auto_add... OK Aplicando contenttypes.0002_remove_content_type_name... OK Aplicando auth.0002_alter_permission_name_max_length... OK Aplicando auth.0003_alter_user_email_max_length... OK Aplicando auth.0004_alter_user_log_in_last_username_opts... OK0 aplicando auth. OK Applying auth.0006_require_contenttypes_0002... OK Applying auth.0007_alter_validators_add_error_messages... OK Applying auth.0008_alter_user_username_max_length... OK Applying sessions.0001_initial... OK Applying all.0001_OK_initial
Now we can test the API:
/> curl localhost:8000/all
The schema and all data in the container will persist as long as the
postgres:9.6.2-alpinothe image is not removed. Eventually though, it would be nice to see how your app will develop with a clean setup. You can run
docker-compose down, which will clean things up and let you see what happens with a fresh start.
Feel free to check out the source code, play around a bit, and see how things work out for you.
The application itself includes some integration tests. There are several ways to test with Docker, including creating
docker-compose.test.ymlfiles specific to the test environment. This is a bit beyond the current scope of this article, but I want to show you how to run the tests with the current configuration.
Current containers run under project name
pythondjangotodoapp🇧🇷 This is a directory name pattern. If we try to run commands it will use the same project and the containers will be restarted. This is what we don't want.
Instead, we'll use a different project name to run the app, isolating the tests in their own environment. Since containers are ephemeral (short-lived), running your tests in a separate set of containers ensures that your application behaves exactly as it should in a clean environment.
In your terminal, run the following command:
/> docker-compose -p tests run -p 8000 --rm web python manage.py testStarting tests_postgres_1 ... doneCreating database test for alias 'default'... System check found no issues (0 muted ) ............................................... ......................... ------ ----- -------------- ----------Run 8 tests on 0.086sOK Destroying test database for alias 'default'...
docker-composecommand accept multipleoptions, followed by a command. In this case you are using
-p testiclesto perform the services under the
testiclesProject's name. The command used is
corre, which will run a single command in a service.
docker-compose.ymlfile specifies a port, we use
-at 8000to create a random port to avoid port collision. EITHER
--rmThe option will remove the containers when we stop them. Finally, we're running on the
Try Python Manage.py.
At this point, you should have a solid start using Docker Compose for local application development. In the next part of this series on using Docker Compose for Python development, I'll cover integrating and deploying this app via Codeship.
Does your team use Docker in their development workflow? If so, I'd love to hear what you're doing and what benefits you see as a result.
Is it OK to use Docker compose in production? ›
Using docker-compose is fine if you are working on a single machine or don't need to distribute containers across multiple inter-connected machines. If you would be alright with just using Docker by itself, you can use docker-compose as well.
- Step 1: Define the application dependencies. ...
- Step 2: Create a Dockerfile. ...
- Step 3: Define services in a Compose file. ...
- Step 4: Build and run your app with Compose. ...
- Step 5: Edit the Compose file to add a bind mount.
Note: Docker Compose requires Python 3.6 or later.How to write Dockerfile for Python application? ›
- FROM python:3. ...
- ADD my_script.py / ...
- RUN pip install pystrich. ...
- CMD [ "python", "./my_script.py" ] ...
- FROM python:3 ADD my_script.py / RUN pip install pystrich CMD [ "python", "./my_script.py" ]
- Manual installation and running of the Docker Compose takes time and is complex.
- There is no health check in the production with the Docker Compose.
- Impossible to change a container without downtime and there are no rolling updates.
Disadvantages of Docker Compose
Unfortunately, docker run and docker-compose won't re-create containers that failed a built-in health check. You can't replace a container without downtime. No rolling updates are available. Docker Compose fails to prove itself on reboots.
If you want the absolute latest bugfix version of Python, or a wide variety of versions, the official Docker Python image is your best bet. If you want the absolute latest system packages, you'll want Ubuntu 22.04; RedHat 9 is somewhat more conservative, for example including Python 3.9 rather than 3.10.Is Docker compose still free? ›
Still have questions? *Docker Desktop is free to use, as part of the Docker Personal subscription, for individuals, non-commercial open source developers, students and educators, and small businesses of less than 250 employees AND less than $10 million in revenue.What is difference between Dockerfile and Docker compose? ›
A Dockerfile is a simple text file that contains the commands a user could call to assemble an image whereas Docker Compose is a tool for defining and running multi-container Docker applications. Docker Compose define the services that make up your app in docker-compose.Is Docker compose deprecated? ›
Compose V1 support will no longer be provided after June 2023 and will be removed from all future Docker Desktop versions.
What is Docker compose good for? ›
Docker Compose is a tool that was developed to help define and share multi-container applications. With Compose, we can create a YAML file to define the services and with a single command, can spin everything up or tear it all down.When should I use Docker compose? ›
Docker Compose is used for configuring and starting multiple Docker containers on the same host–so you don't have to start each container separately. Docker swarm is a container orchestration tool that allows you to run and connect containers on multiple hosts.How do you Containerize a Python code? ›
- Create a simple static web application using flask.
- Create a Dockerfile.
- Build image out of Dockerfile.
- Run the image in the container.
Faster development process
There is no need to install 3rd-party apps like PostgreSQL, Redis, Elasticsearch on the system – you can run it in containers. Docker in Python also gives you the ability to run different versions of the same application simultaneously.
- Avoid installing unnecessary packages. If you install unnecessary packages in your dockerfile, it will increase the build time and the size of the image. ...
- Chain all RUN commands. ...
- Use a . ...
- Use the best order of statements. ...
- Avoid installing unnecessary package dependencies. ...
- Using a minimal base image:
Both the Dockerfile and docker-compose are important resources in the development and deployment of cloud-native applications. But knowing the difference between docker-compose and the Dockerfile is important. The Dockerfile is used to build images, while docker-compose helps you run them as containers.Why Docker is not used for production? ›
In small organizations, a reticence to use Docker in production can be due to be a skills gap both in the use of Docker itself and in the orchestrators. When using Docker, applications are "stateless" because of the microservices architecture used, while most applications, even n-tier, are "stateful".What is the difference between Docker Compose and Kubernetes? ›
In simple words, both Kubernetes & Docker compose are frameworks for container orchestration. The main difference is that Kubernetes runs containers across multiple computers, whether virtual or physical, whereas Docker Compose runs containers on a single host machine.Should I use docker or docker-compose? ›
Furthermore, when a cloud-native application requires more the support of more than one running container, docker-compose is a better option than docker run. The docker-compose command is preferable to 'docker run' whenever a Docker container has complex dependencies and configuration requirements.Why is docker being deprecated? ›
Instead, the Docker runtime is being deprecated in favor of an industry standard in Kubernetes, the open source container orchestration system that's increasingly used for general automation of software deployment, scaling and management.
Do I have to run Docker-compose up every time? ›
Typically, you want docker-compose up . Use up to start or restart all the services defined in a docker-compose. yml . In the default “attached” mode, you see all the logs from all the containers.What should you not use Docker for? ›
Docker is great for developing web applications, but if your end-product is a desktop application, then we would suggest you not to use Docker. As it doesn't provide the environment for running the software with a graphical interface, you would need to perform additional workarounds.Is Docker still relevant 2022? ›
Is Docker Still Relevant In 2022? Docker remains relevant to most container projects, applications, and developers today thanks to its modern tools, compatibility, large community, and ease of use. However, Docker Inc has undergone changes recently, among them changes to Docker Desktop licensing.Which frontend is best for Python backend? ›
Django [Frontend + Backend + Python]
Django is the most popular Python framework for web development. It's a full-stack framework and includes all the necessary features by default instead of offering them as separate libraries.
You can use Compose to deploy an app to a remote Docker host by setting the DOCKER_HOST , DOCKER_TLS_VERIFY , and DOCKER_CERT_PATH environment variables appropriately.Can I use Docker Compose without Docker Desktop? ›
You can also run docker-compose up as per normal but without needing Docker Desktop.What are the three main steps of Docker-Compose? ›
- Define your app's environment with a Dockerfile so it can be reproduced anywhere.
- Define the services that make up your app in docker-compose. yml so they can be run together in an isolated environment.
- Run docker-compose up and Compose starts and runs your entire app.
Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application's services. Then, with a single command, you create and start all the services from your configuration.What is the difference between build and image in Docker-Compose? ›
build: expects dockerfile path as an argument, it will build an image first and then use the image to create a container. image: expects existing image name as argument , it will launch container using this image.Is compose faster than XML? ›
Compose-based layout nearly took 2.5x the time than the XML-based layout.
Is compose better than XML? ›
Compose allows you to do more with less code compared to XML. Compose is Intuitive. This means that you just need to tell Compose what you want to show the user. Compose is compatible with all your existing code: you can call Compose code from Views and Views from Compose.Why is compose better? ›
Compose enables you to create beautiful apps with direct access to the Android platform APIs and built-in support for Material Design, Dark theme, animations, and more: “Compose has also solved more than declarative UI -- accessibility apis, layout, all kinds of stuff have been improved.What is the difference between Docker Compose and docker stack? ›
Docker stack does not support docker-compose. yml files which are written according to the version 2 specification. It has to be the most recent one, which is 3 at the time of writing, while Docker Compose still can handle versions 2 and 3 without problems.Does docker compose up build images? ›
What does docker-compose up -d ? That starts the containers in detached mode so will not show any output from them in the console and run them in the background. @thanks_in_advance --build forces the (re-)build of the images. If you omit the flag and the image exists already, it won't get rebuilt.Is docker deprecated in Kubernetes? ›
A while ago, Kubernetes announced that it was deprecating Docker. Actually, it was deprecating something called dockershim, and Docker alongside it. Roughly one year after the announcement, Docker was completely removed from Kubernetes. Now, Kubernetes is a container orchestrator.Does docker compose rebuild images? ›
If the container requires to build an image, the run command also builds the container but it doesn't have to rebuild the image. docker run does not build either. docker-compose up does everything so you have the option to rebuild. When you run docker-compose down it doesn't delete the image.Can I run Python in Docker? ›
Create your python program (skip if you already have a Python program code) Create a docker file. Build the docker file into an image. Run the docker image in a container.How to package Python code with Docker? ›
- Sample application.
- Test the application.
- Create a Dockerfile for Python. Directory structure.
- Build an image.
- View local images.
- Tag images.
So, one example of when not to use containers is if a high level of security is critical. They can require more work upfront: If you're using containers right, you will have decomposed your application into its various constituent services, which, while beneficial, isn't necessary if you are using VMs.Is Docker good for machine learning? ›
Docker allows to easily reproduce the working environment that is used to train and run the machine learning model anywhere. Docker allows packaging the code and dependencies into containers that can be ported to different servers even if it's a different hardware or operating system.
Is Docker useful for machine learning? ›
The use of Docker simplifies the process of deploying machine learning models. It's a matter of whether or not you want to share your model with others. It's as simple as wrapping your model in an API and putting it in a container utilizing Kubernetes technology.What is the most popular use of Docker? ›
Docker makes it easy for developers to quickly create, deploy and manage apps on any platform. Moreover, it optimizes resources at their core allowing you to deploy more apps on the same hardware than with any of its counterparts.What is the best OS to run Docker? ›
Any Linux OS can be used for docker but we prefer Boot2Docker or RancherOS. There are many reasons why you would want to choose one over the other. Boot2Docker is a better choice for Windows and Mac OS X users because it provides an easy way to get started with Docker.What is the difference between Dockerfile and Docker image? ›
These commands and guidelines we run act on the base image configured to create a new Docker image. A Dockerfile is the Docker image's source code. A Dockerfile is a text file containing various instructions and configurations. The FROM command in a Dockerfile identifies the base image from which you are constructing.Do programs run slower in Docker? ›
One of the more common problems for Developers that use Windows is that the projects with Docker configuration work really slowly, to a point when sometimes a single browser request needs to wait 30-60 seconds to be completed.Is Docker good for production database? ›
Docker is great for running databases in a development environment! You can even use it for databases of small, non-critical projects which run on a single server. Just make sure to have regular backups (as you should in any case), and you'll be fine.
Do Not Use Docker if You Have a Lot of Valuable Data to Store. By design, all Docker files are created inside a container and stored on a writable container layer. It may be difficult to retrieve the data out of the container if a different process needs it.Is Docker compose free for commercial use? ›
Docker Desktop License Agreement
Docker Desktop is free for small businesses (fewer than 250 employees AND less than $10 million in annual revenue), personal use, education, and non-commercial open source projects. Otherwise, it requires a paid subscription for professional use.
But now with modern containerisation tools and container orchestration services in place (such as Kubernetes and OpenShift ) docker provides too much then it's needed to get things running. In this article we will see briefly what is containerisation, how does docker came into place and why it's becoming obsolete.Is Docker good for Python? ›
The “official” Docker image has the benefit of providing every version of Python you might want, and tends to be very up-to-date with bugfix releases.
Is Docker being deprecated? ›
Instead, the Docker runtime is being deprecated in favor of an industry standard in Kubernetes, the open source container orchestration system that's increasingly used for general automation of software deployment, scaling and management. [Click on image for larger view.] Azure Kubernetes Service (source: Microosft).