Continuous Delivery and Continuous Deployment for Kubernetes microservices

THIS IS A DRAFT VERSION OF POST TO COME, PLEASE DO NOT SHARE

Starting Point

Over last years we’ve been adopting several concepts for our project, straggling to make them work together.

The first one is the Microservice Architecture. We did not start it clean and by the book, rather applied it to the already existing project: splitting big services into smaller and breaking excessive coupling. The refactoring work is not finished yet. New services, we are building, starts looking more like “microservices”, while there are still few that, I would call “micro-monoliths”. I have a feeling that this is a typical situation for an already existing project, that tries to adopt this new architecture pattern: you are almost there, but there is always a work to be done.

What follows is the text of my presentation, Chaos Testing for Docker Containers that I gave at ContainerCamp in London this year. I’ve also decided to turn the presentation into an article. I edited the text slightly for readability and added some links for more context. You can find the original video recording and slides at the end of this post.

Intro

Software development is about building software services that support business needs. More complex businesses processes we want to automate and integrate with. the more complex software system we are building. And solution complexity is tend to grown over time and scope.

Teaser

Suppose you want to debug a Node.js application already running on a remote machine inside Docker container. And would like to do it without modifying command arguments (enabling debug mode) and opening remote Node.js debugger agent port to the whole world.

I bet you didn’t know that it’s possible and also have no idea how to do it.

I encourage you to continue reading this post if you are eager to learn some new cool stuff.

TL;DR

Starting from Docker 17.05+, you can create a single Dockerfile that can build multiple helper images with compilers, tools, and tests and use files from above images to produce the final Docker image.

The “core principle” of Dockerfile

Docker can build images by reading the instructions from a Dockerfile. A Dockerfile is a text file that contains a list of all the commands needed to build a new Docker image. The syntax of Dockerfile is pretty simple and the Docker team tries to keep it intact between Docker engine releases.

TL;DR

What is the bare minimum you need to build, test and run my Java application in Docker container?

The recipe: Create a separate Docker image for each step and optimize the way you are running it.

Introduction

I started working with Java in 1998, and for a long time, it was my main programming language. It was a long love–hate relationship.

DDuring my work career, I wrote a lot of code in Java. Despite that fact, I don’t think Java is usually the right choice for writing microservices running in Docker containers.

In this post, I’ve decided to share with you some useful commands and tools, I’m frequently using, working with amazing Docker technology. There is no particular order or “coolness level” for every “hack”. I will try to present the use case and how does specific command or tool help me with my work.

Cleaning up

Working with Docker for some time, you start to accumulate development junk: unused volumes, networks, exited containers and unused images.

Disclaimer: all code snippets bellow are working only with Docker 1.13+

TL;DR

Docker 1.13 simplifies deployment of composed application to a swarm (mode) cluster. And you can do it without creating a new dab (Distribution Application Bundle) file, but just using familiar and well-known docker-compose.yml syntax (with some additions) and --compose-file option.

Swarm cluster

Docker Engine 1.12 introduced a new swarm mode for natively managing a cluster of Docker Engines called a swarm. Docker swarm mode implements Raft Consensus Algorithm and does not require using external key value store anymore, such as Consul or etcd.

TL;DR

Tip: Consider to define and use .dockerignore file for every Docker image you are building. It can help you to reduce Docker image size, speedup docker build and avoid unintended secret exposure.

Docker build context

The docker build command is used to build a new Docker image. There is one argument you can pass to the build command build context.

So, what is the Docker build context?

TL;DR

Docker-in-Docker dind can help you to run Docker Swarm cluster on your Macbook only with Docker for Mac (v1.12+). No virtualbox, docker-machine, vagrant or other app is required.

The Beginning

One day, I’ve decided to try running Docker 1.12 Swarm cluster on my MacBook Pro. Docker team did a great job releasing Docker for Mac, and from that time I forgot all problems I used to have with boot2docker. I really like Docker for Mac: it’s fast, lightweight, tightly integrated with MacOS and significantly simplifies my life when working in changing network environment. The only missing thing is that it’s possible to create and work with single Docker daemon running inside xhyve VM. Shit! I want a cluster.

TL;DR

Pumba netem delay and netem loss commands can emulate network delay and packet loss between Docker containers, even on single host. Give it a try!

Introduction

Microservice architecture has been adopted by software teams as a way to deliver business value faster. Container technology enables delivery of microservices into any environment. Docker has accelerated this by providing an easy to use toolset for development teams to build, ship, and run distributed applications. These applications can be composed of hundreds of microservices packaged in Docker containers.