Using Docker Compose for Python development (2023)

Docker is an amazing tool for developers. It allows us to create and replicate images on any host, eliminating inconsistencies in development environments and greatly reducing integration time.

To provide an example of how you can move to containerized development, I've created a simpleallAPI comPiton,Django REST Frameworkand PostgreSQL usingComposition of the dockerto development, testing, and finally into my CI/CD pipeline.

In a two-part series, I'll cover the steps of developing and creating pipelines. In this post, I'll cover the first part: developing and testing with Docker Compose.

Requirements for this tutorial

This tutorial requires you to have a few items before beginning.

• install on pcCommunity Edition of Docker

• Instalar en pcComposition of the docker

• DescargarTodo app sample: non docker branch

The to-do app here is essentially a plugin, and you can replace it with your own app. Some of the settings here are specific to this app. It may not cover your application needs, but it should be a good starting point for getting the concepts you need to dockerize your own applications.

After setting everything up, you can move on to the next section.

Creating the Dockerfile

At the base of any docked app you will find aDockerfile. oDockerfilecontains all the instructions used to create the application image. You can configure this by installing Python and all of its dependencies. However, the Docker ecosystem has an image repository with aPitonimage already created and ready to use.

In the application root directory, create a newDockerfile.

/> toque Dockerfile

Open the newly createdDockerfilein your favorite editor. The first instructionSINCE, it will tell Docker to use the prebuilt Python image. There are several options, but this project uses thepython: 3.6.1-alpinePicture. For more details on why I'm usingalpinehere about the other options you can readthis post.

FROM python: 3.6.1-alpine

if you rundocker construction., you will see something similar to the following:

Sending build context to Docker daemon 10.53MBStep 1/1 : FROM python:3.6.1-alpine3.6.1-alpine: Pulling from library/python90f4dba627d6: Pull complete19bc0bb0be9f: Pull completee05eff433916: Pull completee70196200a87: Pull completea6d780959950: Pull completeDigest: sha256:0945574465b917d524ce9b748479a286c2ed3c5a97311ac5950464907d4d8b53Status: Imagen más reciente bajada para python:3.6.1-alpine ---> ddd6300d05a3Construído con éxito ddd6300d05a3

With just one declaration in the Dockerfile, this doesn't do much, but it shows the build process without much happening. At this point, you already have an image created and working.dockers imagesIt will show you the images you have available:

REPOSITOR LABEL IMAGE ID SIZE CREATEDpython 3.6.1-alpine ddd6300d05a3 5 weeks ago 88.7 MB

oDockerfileyou need more instructions to build the app. Currently it's just building an image with python installed, but we still need our application code to run inside the container. Let's add some more instructions for doing this and build this image again.

This specific Dockerfile usesCORRE,COPY OF, miWORK DIRECTOR🇧🇷 You can read more about them on the Docker reference page to get a deeper understanding.

Let's add the instructions to theDockerfilenow:

DE python:3.6.1-alpineRUN apk actualizar \ && apk agregar \ build-base \ postgresql \ postgresql-dev \ libpqRUN mkdir /usr/src/appWORKDIR /usr/src/appCOPY ./requirements.txt .RUN pip install -r requisitos.txtENV PYTHONUNBUFERED 1COPY . .

This is what is happening:

• Updateapkpackages and then install some additional requirements

• make directory/usr/src/application

• set working directory/usr/src/application

• copy ofrequirements.txtto working directory

• Run the pip installation with therequirements.txtFile, Archive

• Set the environment variableSEARCHLESS PYTHONfor 1

  (Video) How to run Python scripts in Dokcer using Docker Compose

• Copy all the files from the project root to the working directory

now you can rundocker construction.again and see the results:

Send build context to Docker Daemon 10.53MB Step 1/8: FROM python:3.6.1-alpine ---> ddd6300d05a3 Step 2/8: RUN apk update && apk add build base postgresql postgresql-dev libpq -- -> Running on 22bfdcf8a0ddfetch http://dl-cdn.alpinelinux.org/alpine/v3.4/main/x86_64/APKINDEX.tar.gzfetch http://dl-cdn.alpinelinux.org/alpine/v3.4/ community/x86_64/APKINDEX .tar.gzv3.4.6-165-g6b9a79f [http://dl-cdn.alpinelinux.org/alpine/v3.4/main]v3.4.6-160-g14ad2a3 [http://dl- cdn.alpinelinux.org /alpine/v3.4/community]OK: 5974 different packages available## APK packages installed ##Running busybox-1.24.2-r13.triggerOK: 215 MiB in 65 packages ---> 782357cafeceRemoving the container intermediate 22bfdcf8a0ddStep 3/8: ENV PYTHONUNBUFFERED 1 ---> Running at 0d6de64f5f8b ---> 609106526013 Removing intermediate container 0d6de64f5f8b Step 4/8: RUN mkdir /usr/src/app ---> Running at b30ac1098156 17def88f6c5f Removing the interm container edio 0d6de64f5f8b /usr/src/application ---> 158d43ac2a47Deleting the intermediate container int fef446ea4ed0Step 6/8: COPY ./requirements.txt . ---> 2751d1f4c313Removing the intermediate container a49b090cb8e3Step 7/8: RUN pip install -r requirements.txt ---> Running on eaf9912e4810## Installed PIP requirements ##Installing collected packages: pytz, django, djangorestframework, psycopg2, django -cors - headers, dj-database-url, gunicorn, virtualenvSuccessfully installed django-1.11.3 dj-database-url-0.4.2 django-cors-headers-2.1.0 djangorestframework-3.6.3 gunicorn-19.7.1 psycopg2 -2.7 .2 pytz-2017.2 virtualenv-15.1.0 ---> dfcca42a45b0Removing the intermediary container eaf9912e4810Step 8/8: COPY. 🇧🇷 ---> 61f356fba3eaRemoving the intermediate container fda87c3db8cfSuccessfully built 61f356fba3ea

You have now successfully created the application image using Docker. However, currently our app won't do much as we still need a database and want to connect everything. This is where Docker Compose will help us.

Docker Composition Services

Now that you know how to create an image with aDockerfile, we are going to create an application as a service and connect it to a database. We can then run some configuration commands and start creating this new task list.

create the filedocker-compose.yml:

/> toque docker-compose.yml

The Docker Compose file will define and run the containers based on a configuration file. We are usingcompose file version 3syntax, and you can read it on the Docker website.

An important concept to understand is that Docker Compose works at "runtime". So far, we have built images usingdocker construction.-- this is the "compile time". This is especially important when we add things likevolumesmidomainbecause they will overwrite what is configured in "buildtime".

For example, himusr/src/applicationThe directory will be created during compilation. We then assign that directory to the host machine and the application code from the host machine will be used during "runtime". This allows us to make changes locally and they are accessible within the container.

open yourdocker-compose.ymlin your editor and copy and paste the following lines:

version: '3' services: web: build: . command: gunicorn -b 0.0.0.0:8000 allapp.wsgi:application depends on: - postgres volumes: - .:/usr/src/app ports: - environment "8000:8000": DATABASE_URL: postgres://todoapp @postgres/ all postgres: image: postgres: 9.6.2-alpine environment: POSTGRES_USER: allapp POSTGRES_DB: all

It'll take a while to unpack, but we'll break it down by service.

the web service

The first directive in the web service isbuild upthe image based on ourDockerfile🇧🇷 This will recreate the image we used before, but now it will have the name of the project we are in,Name🇧🇷 After that, we'll give the service some specific instructions on how it should work:

• command: gunicorn -b 0.0.0.0:8000 todosapp.wsgi:application- Once the image is created and the container is running, this command will start the application.

  (Video) How to create a great dev environment with Docker

• depends on:- This will tell Docker Compose to start thepostgresservice when thethe netthe service is executed.

• volumes:- This section will establish routes between the host and the container.

• .:/usr/src/app- This will mount the root directory to our working directory in the container.

• environment:- The application itself expects the environment variable.DATABASE_URLrun.

• ports:- This will publish the port of the container, in this case8000, to house as a port8000.

oDATABASE_URLis the connection string.postgres://todoapp@postgres/todosconnect using theAll applicationsuser, on hostpostgres, using the databaseall.

The Postgres service

Like the Python image we used, Docker Store has a pre-built image forPostgreSQLName🇧🇷 Instead of using abuild updirective, we can use the image name and Docker will take that image for us and use it. In this case, we are usingpostgres:9.6.2-alpino🇧🇷 We could leave it like that, but it hasenvironmentvariables so we can customize it a bit.

• environment:- This particular image accepts some environment variables so we can customize things to our needs.

• POSTGRES_USER: allapplication- This creates the userAll applicationsas the default PostgreSQL user.

  See Also

  How to build and run a Python application in a container – Docker Python Tutorial – CollabnixBest Practices for Containerizing Python Applications with Docker | SnykDocker for Data ScienceHow to choose a base Docker image for Python

• POSTGRES_DB: all- This will create the default database asall.

running the application

Now that we have our services defined, we can build the application usingdocker-compose🇧🇷 This will show the images being built and finally started. After the initial build, you will see the names of the containers being built.

Pulling postgres (postgres:9.6.2-alpine)...9.6.2-alpine: Pulling from library/postgrescfc728c1c558: Pull completeb749e72b24f9: Pull complete0abdb8c9c36b: Pull complete90ca3848ef7e: Pull complete3ecf037a5034: Pull complete9327e3c5554c: Pull complete3133782bad17: Pull complete143bac6c8910: Pull completed6da9f4bd18e: Pull completeDigest: sha256:f88000211e3c682e7419ac6e6cbd3a7a4980b483ac416a3b5d5ee81d4f831cc9Status: Downloaded newer image for postgres:9.6.2-alpineBuilding web...Creating pythondjangotodoapp_postgres_1 ...Creating pythondjangotodoapp_postgres_1 ... doneCreating pythondjangotodoapp_web_1 ...Creating pythondjangotodoapp_web_1 ... done...web_1 | [2017-08-03 13:23:18 +0000] () [INFO] Iniciando gunicorn 19.7.1web_1 | [2017-08-03 13:23:18 +0000] () [INFO] Ouvindo em: http://0.0.0.0:8000 (1)

At this point, the application is running and you will see the log output in the console. You can also run the services as a background process usingdocker-compose up -d🇧🇷 During development, I prefer to run without-dand create a second terminal window to run other commands. If you want to run it as a background process and see the logs, you can runlogs do docker-compose.

In a new command prompt you can rundocker-compose psto see your running containers. You should see something like the following:

Name Command Status Ports------------------------------------------- --- ----------------------------------------------- --- pythondjangotodoapp_postgres_1 docker-entrypoint.sh postgres Up to 5432/tc pythondjangotodoapp_web_1 gunicorn -b 0.0.0.0:8000 t ... Up to 0.0.0.0:8000->8000/tcp

This will tell you the name of the services, the command used to start them, their current state, and the ports. Noticepythondjangotodoapp_web_1listed the port as0.0.0.0:8000->8000/tcp🇧🇷 This tells us that you can access the app usinglocal host: 8000/allon the host machine.

Migrate the database schema

One small but important step that should not be overlooked is migrating the schema to the database. Composing comes with aexecutivecommand that will execute a single command in a running container. The typical function for migrating schemas ismove python manage.py🇧🇷 We can run it in the web service usingdocker-compose executive.

/> docker-compose exec web python manage.py migrationOperations to perform: Apply all migrations: admin, auth, contenttypes, sessions, allRun migrations: Apply contenttypes.0001_initial... OK Apply auth.0001_initial... OK Apply admin. 0001_initial ... OK Aplicando admin.0002_logentry_remove_auto_add... OK Aplicando contenttypes.0002_remove_content_type_name... OK Aplicando auth.0002_alter_permission_name_max_length... OK Aplicando auth.0003_alter_user_email_max_length... OK Aplicando auth.0004_alter_user_log_in_last_username_opts... OK0 aplicando auth. OK Applying auth.0006_require_contenttypes_0002... OK Applying auth.0007_alter_validators_add_error_messages... OK Applying auth.0008_alter_user_username_max_length... OK Applying sessions.0001_initial... OK Applying all.0001_OK_initial

Now we can test the API:

/> curl localhost:8000/all[]

The schema and all data in the container will persist as long as thepostgres:9.6.2-alpinothe image is not removed. Eventually though, it would be nice to see how your app will develop with a clean setup. You can rundocker-compose down, which will clean things up and let you see what happens with a fresh start.

Feel free to check out the source code, play around a bit, and see how things work out for you.

application test

The application itself includes some integration tests. There are several ways to test with Docker, including creatingdockerfile.testmidocker-compose.test.ymlfiles specific to the test environment. This is a bit beyond the current scope of this article, but I want to show you how to run the tests with the current configuration.

Current containers run under project namepythondjangotodoapp🇧🇷 This is a directory name pattern. If we try to run commands it will use the same project and the containers will be restarted. This is what we don't want.

Instead, we'll use a different project name to run the app, isolating the tests in their own environment. Since containers are ephemeral (short-lived), running your tests in a separate set of containers ensures that your application behaves exactly as it should in a clean environment.

In your terminal, run the following command:

/> docker-compose -p tests run -p 8000 --rm web python manage.py testStarting tests_postgres_1 ... doneCreating database test for alias 'default'... System check found no issues (0 muted ) ............................................... ......................... ------ ----- -------------- ----------Run 8 tests on 0.086sOK Destroying test database for alias 'default'...

odocker-composecommand accept multipleoptions, followed by a command. In this case you are using-p testiclesto perform the services under thetesticlesProject's name. The command used iscorre, which will run a single command in a service.

From thedocker-compose.ymlfile specifies a port, we use-at 8000to create a random port to avoid port collision. EITHER--rmThe option will remove the containers when we stop them. Finally, we're running on thethe netServiceTry Python Manage.py.

Conclution

At this point, you should have a solid start using Docker Compose for local application development. In the next part of this series on using Docker Compose for Python development, I'll cover integrating and deploying this app via Codeship.

Does your team use Docker in their development workflow? If so, I'd love to hear what you're doing and what benefits you see as a result.