Continuous Delivery with Docker and Jenkins

Continuous Delivery with Docker and Jenkins

by Rafał Leszko
Released August 2017
Publisher(s): Packt Publishing
ISBN: 9781787125230

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Unleash the combination of Docker and Jenkins in order to enhance the DevOps workflow

About This Book

  • Build reliable and secure applications using Docker containers.
  • Create a complete Continuous Delivery pipeline using Docker, Jenkins, and Ansible.
  • Deliver your applications directly on the Docker Swarm cluster.
  • Create more complex solutions using multi-containers and database migrations.

Who This Book Is For

This book is indented to provide a full overview of deep learning. From the beginner in deep learning and artificial intelligence to the data scientist who wants to become familiar with Theano and its supporting libraries, or have an extended understanding of deep neural nets.

Some basic skills in Python programming and computer science will help, as well as skills in elementary algebra and calculus.

What You Will Learn

  • Get to grips with docker fundamentals and how to dockerize an application for the Continuous Delivery process
  • Configure Jenkins and scale it using Docker-based agents
  • Understand the principles and the technical aspects of a successful Continuous Delivery pipeline
  • Create a complete Continuous Delivery process using modern tools: Docker, Jenkins, and Ansible
  • Write acceptance tests using Cucumber and run them in the Docker ecosystem using Jenkins
  • Create multi-container applications using Docker Compose
  • Managing database changes inside the Continuous Delivery process and understand effective frameworks such as Cucumber and Flyweight
  • Build clustering applications with Jenkins using Docker Swarm
  • Publish a built Docker image to a Docker Registry and deploy cycles of Jenkins pipelines using community best practices

In Detail

The combination of Docker and Jenkins improves your Continuous Delivery pipeline using fewer resources. It also helps you scale up your builds, automate tasks and speed up Jenkins performance with the benefits of Docker containerization.

This book will explain the advantages of combining Jenkins and Docker to improve the continuous integration and delivery process of app development. It will start with setting up a Docker server and configuring Jenkins on it. It will then provide steps to build applications on Docker files and integrate them with Jenkins using continuous delivery processes such as continuous integration, automated acceptance testing, and configuration management.

Moving on you will learn how to ensure quick application deployment with Docker containers along with scaling Jenkins using Docker Swarm. Next, you will get to know how to deploy applications using Docker images and testing them with Jenkins.

By the end of the book, you will be enhancing the DevOps workflow by integrating the functionalities of Docker and Jenkins.

Style and approach

The book is aimed at DevOps Engineers, developers and IT Operations who want to enhance the DevOps culture using Docker and Jenkins.

Table of contents

  1. Preface
    1. What this book covers
    2. What you need for this book
    3. Who this book is for
    4. Conventions
    5. Reader feedback
    6. Customer support
      1. Downloading the example code
      2. Downloading the color images of this book
      3. Errata
    7. Piracy
    8. Questions
  2. Introducing Continuous Delivery
    1. What is Continuous Delivery?
      1. The traditional delivery process
        1. Introducing the traditional delivery process
        2. Shortcomings of the traditional delivery process
      2. Benefits of Continuous Delivery
      3. Success stories
    2. The automated deployment pipeline
      1. Continuous Integration
      2. Automated acceptance testing
        1. The Agile testing matrix
        2. The testing pyramid
      3. Configuration management
    3. Prerequisites to Continuous Delivery
      1. Organizational prerequisites
        1. DevOps culture
        2. Client in the process
        3. Business decisions
      2. Technical and development prerequisites
    4. Building the Continuous Delivery process
      1. Introducing tools
        1. Docker ecosystem
        2. Jenkins
        3. Ansible
        4. GitHub
        5. Java/Spring Boot/Gradle
        6. The other tools
      2. Creating a complete Continuous Delivery system
        1. Introducing Docker
        2. Configuring Jenkins
        3. Continuous Integration Pipeline
        4. Automated acceptance testing
        5. Configuration management with Ansible/Continuous Delivery pipeline
        6. Clustering with Docker Swarm/Advanced Continuous Delivery
    5. Summary
  3. Introducing Docker
    1. What is Docker?
      1. Containerization versus virtualization
      2. The need for Docker
        1. Environment
        2. Isolation
        3. Organizing applications
        4. Portability
      3. Kittens and cattle
      4. Alternative containerization technologies
    2. Docker installation
      1. Prerequisites for Docker
      2. Installing on a local machine
        1. Docker for Ubuntu
        2. Docker for Linux
        3. Docker for Mac
        4. Docker for Windows
        5. Testing Docker installation
      3. Installing on a server
        1. Dedicated server
    3. Running Docker hello world>
      1. Docker components
        1. Docker client and server
        2. Docker images and containers
    4. Docker applications
    5. Building images
      1. Docker commit
      2. Dockerfile
      3. Complete Docker application
        1. Write the application
        2. Prepare the environment
        3. Build the image
        4. Run the application
      4. Environment variables
    6. Docker container states
    7. Docker networking
      1. Running services
      2. Container networks
      3. Exposing container ports
      4. Automatic port assignment
    8. Using Docker volumes
    9. Using names in Docker
      1. Naming containers
      2. Tagging images
    10. Docker cleanup
      1. Cleaning up containers
      2. Cleaning up images
    11. Docker commands overview
    12. Exercises
    13. Summary
  4. Configuring Jenkins
    1. What is Jenkins?
    2. Jenkins installation
      1. Requirements for installation
      2. Installing on Docker
      3. Installing without Docker
      4. Initial configuration
    3. Jenkins hello world
    4. Jenkins architecture
      1. Master and slaves
      2. Scalability
        1. Vertical scaling
        2. Horizontal scaling
      3. Test and production instances
      4. Sample architecture
    5. Configuring agents
      1. Communication protocols
      2. Setting agents
        1. Permanent agents
          1. Configuring permanent agents
          2. Understanding permanent agents
        2. Permanent Docker agents
          1. Configuring permanent Docker agents
          2. Understanding permanent Docker agents
        3. Jenkins Swarm agents
          1. Configuring Jenkins Swarm agents
          2. Understanding Jenkins Swarm agents
        4. Dynamically provisioned Docker agents
          1. Configuring dynamically provisioned Docker agents
        5. Understanding dynamically provisioned Docker agents
      3. Testing agents
    6. Custom Jenkins images
      1. Building Jenkins slave
      2. Building Jenkins master
    7. Configuration and management
      1. Plugins
      2. Security
      3. Backup
      4. Blue Ocean UI
    8. Exercises
    9. Summary
  5. Continuous Integration Pipeline
    1. Introducing pipelines
      1. Pipeline structure
      2. Multi-stage Hello World
      3. Pipeline syntax
        1. Sections
        2. Directives
        3. Steps
    2. Commit pipeline
      1. Checkout
        1. Creating a GitHub repository
        2. Creating a checkout stage
      2. Compile
        1. Creating a Java Spring Boot project
        2. Pushing code to GitHub
        3. Creating a compile stage
      3. Unit test
        1. Creating business logic
        2. Writing a unit test
        3. Creating a unit test stage
      4. Jenkinsfile
        1. Creating Jenkinsfile
        2. Running pipeline from Jenkinsfile
    3. Code quality stages
      1. Code coverage
        1. Adding JaCoCo to Gradle
        2. Adding a code coverage stage
        3. Publishing the code coverage report
      2. Static code analysis
        1. Adding the Checkstyle configuration
        2. Adding a static code analysis stage
        3. Publishing static code analysis reports
      3. SonarQube
    4. Triggers and notifications
      1. Triggers
        1. External
        2. Polling SCM
        3. Scheduled build
      2. Notifications
        1. Email
        2. Group chat
        3. Team space
    5. Team development strategies
      1. Development workflows
        1. Trunk-based workflow
        2. Branching workflow
        3. Forking workflow
      2. Adopting Continuous Integration
        1. Branching strategies
        2. Feature toggles
      3. Jenkins Multibranch
      4. Non-technical requirements
    6. Exercises
    7. Summary
  6. Automated Acceptance Testing
    1. Introducing acceptance testing
    2. Docker registry
      1. Artifact repository
      2. Installing Docker registry
        1. Docker Hub
        2. Private Docker registry
          1. Installing the Docker registry application
          2. Adding a domain certificate
          3. Adding an access restriction
        3. Other Docker registries
      3. Using Docker registry
        1. Building an image
        2. Pushing the image
        3. Pulling the image
    3. Acceptance test in pipeline
      1. The Docker build stage
        1. Adding Dockerfile
        2. Adding the Docker build to the pipeline
      2. The Docker push stage
      3. Acceptance testing stage
        1. Adding a staging deployment to the pipeline
        2. Adding an acceptance test to the pipeline
        3. Adding a cleaning stage environment
    4. Docker Compose
      1. What is Docker Compose?
      2. Installing Docker Compose
      3. Defining docker-compose.yml
      4. Using the docker-compose command
      5. Building images
      6. Scaling services
    5. Acceptance testing with Docker Compose
      1. Using a multi-container environment
        1. Adding a Redis client library to Gradle
        2. Adding a Redis cache configuration
        3. Adding Spring Boot caching
        4. Checking the caching environment
      2. Method 1 – Jenkins-first acceptance testing
        1. Changing the staging deployment stage
        2. Changing the acceptance test stage
      3. Method 2 – Docker-first acceptance testing
        1. Creating a Dockerfile for acceptance test
        2. Creating docker-compose.yml for acceptance test
        3. Creating an acceptance test script
        4. Running the acceptance test
        5. Changing the acceptance test stage
      4. Comparing method 1 and method 2
    6. Writing acceptance tests
      1. Writing user-facing tests
      2. Using the acceptance testing framework
        1. Creating acceptance criteria
        2. Creating step definitions
        3. Running an automated acceptance test
      3. Acceptance test-driven development
    7. Exercises
    8. Summary
  7. Configuration Management with Ansible
    1. Introducing configuration management
      1. Traits of good configuration management
      2. Overview of configuration management tools
    2. Installing Ansible
      1. Ansible server requirements
      2. Ansible installation
      3. Docker-based Ansible client
    3. Using Ansible
      1. Creating inventory
      2. Ad hoc commands
      3. Playbooks
        1. Defining a playbook
        2. Executing the playbook
        3. Playbook's idempotency
        4. Handlers
        5. Variables
      4. Roles
        1. Understanding roles
        2. Ansible Galaxy
    4. Deployment with Ansible
      1. Installing Redis
      2. Deploying a web service
        1. Configuring a project to be executable
        2. Changing the Redis host address
        3. Adding calculator deployment to the playbook
      3. Running deployment
    5. Ansible with Docker
      1. Benefits of Ansible
      2. Ansible Docker playbook
        1. Installing Docker
        2. Running Docker containers
        3. Using Docker Compose
    6. Exercises
    7. Summary
  8. Continuous Delivery Pipeline
    1. Environments and infrastructure
      1. Types of environment
        1. Production
        2. Staging
        3. QA
        4. Development
      2. Environments in Continuous Delivery
      3. Securing environments
    2. Nonfunctional testing
      1. Types of nonfunctional test
        1. Performance testing
        2. Load testing
        3. Stress testing
        4. Scalability testing
        5. Endurance testing
        6. Security testing
        7. Maintainability testing
        8. Recovery testing
      2. Nonfunctional challenges
    3. Application versioning
      1. Versioning strategies
      2. Versioning in the Jenkins pipeline
    4. Complete Continuous Delivery pipeline
      1. Inventory
      2. Acceptance testing environment
      3. Release
      4. Smoke testing
      5. Complete Jenkinsfile
    5. Exercises
    6. Summary
  9. Clustering with Docker Swarm
    1. Server clustering
      1. Introducing server clustering
      2. Introducing Docker Swarm
      3. Docker Swarm features overview
    2. Docker Swarm in practice
      1. Setting up a Swarm
      2. Adding worker nodes
      3. Deploying a service
      4. Scaling service
      5. Publishing ports
    3. Advanced Docker Swarm
      1. Rolling updates
      2. Draining nodes
      3. Multiple manager nodes
      4. Scheduling strategy
    4. Docker Compose with Docker Swarm
      1. Introducing Docker Stack
      2. Using Docker Stack
        1. Specifying docker-compose.yml
        2. Running the docker stack command
        3. Verifying the services and containers
        4. Removing the stack
    5. Alternative cluster management systems
      1. Kubernetes
      2. Apache Mesos
      3. Comparing features
    6. Scaling Jenkins
      1. Dynamic slave provisioning
      2. Jenkins Swarm
      3. Comparison of dynamic slave provisioning and Jenkins Swarm
    7. Exercises
    8. Summary
  10. Advanced Continuous Delivery
    1. Managing database changes
      1. Understanding schema updates
        1. Introducing database migrations
        2. Using Flyway
          1. Configuring Flyway
          2. Defining  the SQL migration script
          3. Accessing database
      2. Changing database in Continuous Delivery
        1. Backwards-compatible changes
        2. Non-backwards-compatible changes
          1. Adding a new column to the database
          2. Changing the code to use both columns
          3. Merging the data in both columns
          4. Removing the old column from the code
          5. Dropping the old column from the database
          6. Separating database updates from code changes
      3. Avoiding shared database
      4. Preparing test data
        1. Unit testing
        2. Integration/acceptance testing
        3. Performance testing
    2. Pipeline patterns
      1. Parallelizing pipelines
      2. Reusing pipeline components
        1. Build parameters
        2. Shared libraries
          1. Creating a shared library project
          2. Configure the shared library in Jenkins
          3. Use shared library in Jenkinsfile
      3. Rolling back deployments
      4. Adding manual steps
    3. Release patterns
      1. Blue-green deployment
      2. Canary release
    4. Working with legacy systems
      1. Automating build and deployment
      2. Automating tests
      3. Refactoring and introducing new features
      4. Understanding the human element
    5. Exercises
    6. Summary
    7. Best practices
      1. Practice 1 – own process within the team!
      2. Practice 2 – automate everything!
      3. Practice 3 – version everything!
      4. Practice 4 – use business language for acceptance tests!
      5. Practice 5 – be ready to roll back!
      6. Practice 6 – don't underestimate the impact of people
      7. Practice 7 – build in traceability!
      8. Practice 8 – integrate often!
      9. Practice 9 – build binaries only once!
      10. Practice 10 – release often!

Product information

  • Title: Continuous Delivery with Docker and Jenkins
  • Author(s): Rafał Leszko
  • Release date: August 2017
  • Publisher(s): Packt Publishing
  • ISBN: 9781787125230