Posts Tagged Gradle
Eventual Consistency with Spring for Apache Kafka: Part 1 of 2
Posted by Gary A. Stafford in Bash Scripting, Java Development, Kubernetes, Software Development on May 22, 2021
Using Spring for Apache Kafka to manage a Distributed Data Model in MongoDB across multiple microservices
Given a modern distributed system composed of multiple microservices, each possessing a sub-set of a domain’s aggregate data, the system will almost assuredly have some data duplication. Given this duplication, how do we maintain data consistency? In this two-part post, we will explore one possible solution to this challenge — Apache Kafka and the model of eventual consistency.
Introduction
Apache Kafka is an open-source distributed event streaming platform capable of handling trillions of messages. According to Confluent, initially conceived as a messaging queue, Kafka is based on an abstraction of a distributed commit log. Since being created and open-sourced by LinkedIn in 2011, Kafka has quickly evolved from a messaging queue to a full-fledged event streaming platform.
Eventual consistency, according to Wikipedia, is a consistency model used in distributed computing to achieve high availability that informally guarantees that if no new updates are made to a given data item, eventually all accesses to that item will return the last updated value. I previously covered the topic of eventual consistency in a distributed system using RabbitMQ in the May 2017 post, Eventual Consistency: Decoupling Microservices with Spring AMQP and RabbitMQ. The post was featured on Pivotal’s RabbitMQ website.
Domain-driven Design
To ground the discussion, let’s examine a common example — an online storefront. Using a domain-driven design (DDD) approach, we would expect our problem domain, the online storefront, to be composed of multiple bounded contexts. Bounded contexts would likely include Shopping, Customer Service, Marketing, Security, Fulfillment, Accounting, and so forth, as shown in the context map, below.
Given this problem domain, we can assume we have the concept of a Customer. Further, we can assume the unique properties that define a Customer are likely to be spread across several bounded contexts. A complete view of the Customer will require you to aggregate data from multiple contexts. For example, the Accounting context may be the system of record for primary customer information, such as the customer’s name, contact information, contact preferences, and billing and shipping addresses. Marketing may possess additional information about the customer’s use of the store’s loyalty program and online shopping activity. Fulfillment may maintain a record of all orders being shipped to the customer. Security likely holds the customer’s access credentials, account access history, and privacy settings.
Below are the Customer data objects are shown in yellow. Orange represents the logical divisions of responsibility within each bounded context. These divisions will manifest themselves as individual microservices in our online storefront example.
Distributed Data Consistency
If we agree that the architecture of our domain’s data model requires some duplication of data across bounded contexts or even between services within the same context, then we must ensure data consistency. Take, for example, the case where a customer changes their home address or email. Let us assume that the Accounting context is the system of record for these data fields. However, to fulfill orders, the Shipping context might also need to maintain the customer’s current home address. Likewise, the Marketing context, responsible for opt-in email advertising, also needs to be aware of the email change and update its customer records.
If a piece of shared data is changed, then the party making the change should be responsible for communicating the change without expecting a response. They are stating a fact, not asking a question. Interested parties can choose if and how to act upon the change notification. This decoupled communication model is often described as Event-Carried State Transfer, defined by Martin Fowler of ThoughtWorks in his insightful post, What do you mean by “Event-Driven”?. Changes to a piece of data can be thought of as a state change event — events that contain details of the data that changed. Coincidentally, Fowler uses a customer’s address change as an example of Event-Carried State Transfer in the post. Fellow former ThoughtWorker Graham Brooks also detailed the concept in his post, Event-Carried State Transfer Pattern.
Consistency Strategies
Multiple architectural approaches can be taken to solve for data consistency in a distributed system. For example, you could use a single relational database with shared schemas to persist data, avoiding the distributed data model altogether. However, it could be argued that using a single database just turned your distributed system back into a monolith.
You could use Change Data Capture (CDC) to track changes to each database and send a record of those changes to Kafka topics for consumption by interested parties. Kafka Connect is an excellent choice for this, as explained in the article, No More Silos: How to Integrate your Databases with Apache Kafka and CDC, by Robin Moffatt of Confluent.
Alternately, we could use a separate data service, independent of the domain’s other business services, whose sole role is to ensure data consistency across domains. If messages persist in Kafka, the service has the added ability to provide data auditability through message replay. Of course, another set of services adds additional operational complexity to the system.
In this post’s somewhat simplistic architecture, the business microservices will maintain consistency across their respective domains by producing and consuming messages from multiple Kafka topics to which they are subscribed. Kafka Producers may also be Consumers within our domain.
Storefront Example
In this post, our online storefront API will be built in Java using Spring Boot and OpenJDK 16. We will ensure the uniformity of distributed data by using a publish/subscribe model with Spring for Apache Kafka Project. When a piece of data is changed by one Spring Boot microservice, if appropriate, that state change will trigger a state change event, which will be shared with other microservices using Kafka topics.
We will explore different methods of leveraging Spring Kafka to communicate state change events, as they relate to the specific use case of a customer placing an order through the online storefront. An abridged view of the storefront ordering process is shown in the diagram below. The arrows represent the exchange of data. Kafka will serve as a means of decoupling services from one another while still ensuring the data is distributed.
Given the use case of placing an order, we will examine the interactions of three services that compose our storefront API: the Accounts service within the Accounting bounded context, the Fulfillment service within the Fulfillment context, and the Orders service within the Order Management context. We will examine how the three services use Kafka to communicate state changes (changes to their data) to each other in a completely decoupled manner.
The diagram below shows the event flows between sub-systems discussed in the post. The numbering below corresponds to the numbering in the ordering process above. We will look at three event flows 2, 5, and 6. We will simulate event flow 3, the order being created by the Shopping Cart service.
Below is a view of the online storefront through the lens of the major sub-systems involved. Although the diagram is overly simplified, it should give you an idea of where Kafka and Zookeeper, Kafka’s current cluster manager, might sit in a typical, highly-available, microservice-based, distributed application platform.
This post will focus on the storefront’s backend API — its services, databases, and messaging sub-systems.
Storefront Microservices
We will explore the functionality of each of the three microservices and how they share state change events using Kafka 2.8. Each storefront API service is built using Spring Boot 2.0 and Gradle. Each Spring Boot service includes Spring Data REST, Spring Data MongoDB, Spring for Apache Kafka, Spring Cloud Sleuth, SpringFox, and Spring Boot Actuator. For simplicity, Kafka Streams and the use of Spring Cloud Stream are not part of this post.
Source Code
The storefront’s microservices source code is publicly available on GitHub. The four GitHub projects can be cloned using the following commands:
git clone --branch 2021-istio \
--single-branch --depth 1 \
https://github.com/garystafford/storefront-demo-accounts.git
git clone --branch 2021-istio \
--single-branch --depth 1 \
https://github.com/garystafford/storefront-demo-orders.git
git clone --branch 2021-istio \
--single-branch --depth 1 \
https://github.com/garystafford/storefront-demo-fulfillment.git
git clone --branch 2021-istio \
--single-branch --depth 1 \
https://github.com/garystafford/storefront-demo.git
Code samples in this post are displayed as Gists, which may not display correctly on some mobile and social media browsers. Links to gists are also provided.
Accounts Service
The Accounts service is responsible for managing basic customer information, such as name, contact information, addresses, and credit cards for purchases. A partial view of the data model for the Accounts service is shown below. This cluster of domain objects represents the Customer Account Aggregate.
The Customer class, the Accounts service’s primary data entity, is persisted in the Accounts MongoDB database. Below we see the representation of a Customer, as a BSON document in the customer.accounts MongoDB database collection.
Along with the primary Customer entity, the Accounts service contains a CustomerChangeEvent class. As a Kafka producer, the Accounts service uses the CustomerChangeEvent domain event object to carry state information about the client the Accounts service wishes to share when a new customer is added or a change is made to an existing customer. The CustomerChangeEvent object is not an exact duplicate of the Customer object. For example, the CustomerChangeEvent object does not share sensitive credit card information with other message Consumers (the CreditCard data object).
Since the CustomerChangeEvent domain event object does not persist in MongoDB, we can look at its JSON message payload in Kafka to examine its structure. Note the differences in the data structure (schema) between the Customer document in MongoDB and the Kafka CustomerChangeEvent message payload.
For simplicity, we will assume that other services do not make changes to the customer’s name, contact information, or addresses — this is the sole responsibility of the Accounts service.
Source code for the Accounts service is available on GitHub. Use the latest 2021-istio branch of the project.
Orders Service
The Orders service is responsible for managing a customer’s past and current orders; it is the system of record for the customer’s order history. A partial view of the data model for the Orders service is shown below. This cluster of domain objects represents the Customer Orders Aggregate.
The CustomerOrders class, the Order service’s primary data entity, is persisted in MongoDB. This entity contains a history of all the customer’s orders (Order data objects), along with the customer’s name, contact information, and addresses. In the Orders MongoDB database, a CustomerOrders, represented as a BSON document in the customer.orders database collection, looks as follows:
Along with the primary CustomerOrders entity, the Orders service contains the FulfillmentRequestEvent class. As a Kafka producer, the Orders service uses the FulfillmentRequestEvent domain event object to carry state information about an approved order, ready for fulfillment, which it sends to Kafka for consumption by the Fulfillment service. The FulfillmentRequestEvent object only contains the information it needs to share. Our example shares a single Order, along with the customer’s name, contact information, and shipping address.
Since the FulfillmentRequestEvent domain event object is not persisted in MongoDB, we can look at its JSON message payload in Kafka. Again, note the schema differences between the CustomerOrders document in MongoDB and the FulfillmentRequestEvent message payload in Kafka.
Source code for the Orders service is available on GitHub. Use the latest 2021-istio branch of the project.
Fulfillment Service
Lastly, the Fulfillment service is responsible for fulfilling orders. A partial view of the data model for the Fulfillment service is shown below. This cluster of domain objects represents the Fulfillment Aggregate.
The Fulfillment service’s primary entity, the Fulfillment class, is persisted in MongoDB. This entity contains a single Order data object, along with the customer’s name, contact information, and shipping address. The Fulfillment service also uses the Fulfillment entity to store the latest shipping status, such as ‘Shipped’, ‘In Transit’, and ‘Received’. The customer’s name, contact information, and shipping address are managed by the Accounts service, replicated to the Orders service, and passed to the Fulfillment service via Kafka, using the FulfillmentRequestEvent entity.
In the Fulfillment MongoDB database, a Fulfillment object represented as a BSON document in the fulfillment.requests database collection looks as follows:
Along with the primary Fulfillment entity, the Fulfillment service has an OrderStatusChangeEvent class. As a Kafka producer, the Fulfillment service uses the OrderStatusChangeEvent domain event object to carry state information about an order’s fulfillment statuses. The OrderStatusChangeEvent object contains the order’s UUID, a timestamp, shipping status, and an option for order status notes.
Since the OrderStatusChangeEvent domain event object is not persisted in MongoDB, again, we can again look at its JSON message payload in Kafka.
Source code for the Fulfillment service is available on GitHub. Use the latest 2021-istio branch of the project.
State Change Event Messaging Flows
There are three state change event messaging flows illustrated in this post.
- Changes to a Customer triggers an event message produced by the Accounts service, which is published on the
accounts.customer.changeKafka topic and consumed by the Orders service; - Order Approved triggers an event message produced by the Orders service, which is published on the
orders.order.fulfillKafka topic, and is consumed by the Fulfillment service; - Changes to the status of an Order triggers an event message produced by the Fulfillment Service, which is published on the
fulfillment.order.changeKafka topic, and is consumed by the Orders service;
Each of these state change event messaging flows follows the same architectural pattern on both the Kafka topic’s producer and consumer sides.
Let us examine each state change event messaging flow and the code behind it.
Customer State Change
When a new Customer entity is created or updated by the Accounts service, a CustomerChangeEvent message is produced and sent to the accounts.customer.change Kafka topic. This message is retrieved and consumed by the Orders service. This is how the Orders service eventually has a record of all customers who may place an order. By way of Kafka, it can be said that the Order’s Customer contact information is eventually consistent with the Account’s Customer contact information.
There are different methods to trigger a message to be sent to Kafka. For this particular state change, the Accounts service uses a listener. The listener class, which extends AbstractMongoEventListener, listens for an onAfterSave event for a Customer entity.
The listener handles the event by instantiating a new CustomerChangeEvent with the Customer’s information and passes it to the Sender class.
The SenderConfig class handles the configuration of the Sender. This Spring Kafka producer configuration class uses Spring Kafka’s JsonSerializer class to serialize the CustomerChangeEvent object into a JSON message payload.
The Sender uses a KafkaTemplate to send the message to the accounts.customer.change Kafka topic, as shown below. Since message order is critical to ensure changes to a Customer’s information are processed in order, all messages are sent to a single topic with a single partition.
The Orders service’s Receiver class consumes the CustomerChangeEvent messages produced by the Accounts service.
The Orders service’s Receiver class is configured differently compared to the Fulfillment service. The Orders service receives messages from multiple topics, each containing messages with different payload structures. Each type of message must be deserialized into different object types. To accomplish this, the ReceiverConfig class uses Apache Kafka’s StringDeserializer. The Orders service’s ReceiverConfig references Spring Kafka’s AbstractKafkaListenerContainerFactory classes setMessageConverter method, which allows for dynamic object type matching.
Each Kafka topic the Orders service consumes messages from is associated with a method in the Receiver class (shown above). This method accepts a specific object type as input, denoting the object type into which the message payload needs to be deserialized. This way, we can receive multiple message payloads, serialized from multiple object types, and successfully deserialize each type into the correct data object. In the case of a CustomerChangeEvent, the Orders service calls the receiveCustomerOrder method to consume the message and properly deserialize it.
For all services, a Spring application.yaml properties file in each service’s resources directory contains the Kafka configuration (lines 11–19).
Order Approved for Fulfillment
When the status of the Order in a CustomerOrders entity is changed to ‘Approved’ from ‘Created’, a FulfillmentRequestEvent message is produced and sent to the orders.order.fulfill Kafka topic. This message is retrieved and consumed by the Fulfillment service. This is how the Fulfillment service has a record of what Orders are ready for fulfillment.
Since we did not create the Shopping Cart service for this post, the Orders service simulates an order approval event, containing an approved order, being received, through Kafka, from the Shopping Cart Service. To simulate order creation and approval, the Orders service can create a random order history for each customer. Further, the Orders service can scan all customer orders for orders that contain both a ‘Created’ and ‘Approved’ order status. This state is communicated as an event message to Kafka for all orders matching those criteria. A FulfillmentRequestEvent is produced, which contains the order to be fulfilled, and the customer’s contact and shipping information. The FulfillmentRequestEvent is passed to the Sender class.
The SenderConfig class handles the configuration of the Sender class. This Spring Kafka producer configuration class uses Spring Kafka’s JsonSerializer class to serialize the FulfillmentRequestEvent object into a JSON message payload.
The Sender class uses a KafkaTemplate to send the message to the orders.order.fulfill Kafka topic, as shown below. Since message order is not critical, messages can be sent to a topic with multiple partitions if the volume of messages required it.
The Fulfillment service’s Receiver class consumes the FulfillmentRequestEvent from the Kafka topic and instantiates a Fulfillment object, containing the data passed in the FulfillmentRequestEvent message payload. The Fulfillment object includes the order to be fulfilled and the customer’s contact and shipping information.
The Fulfillment service’s ReceiverConfig class defines the DefaultKafkaConsumerFactory and ConcurrentKafkaListenerContainerFactory, responsible for deserializing the message payload from JSON into a FulfillmentRequestEvent object.
Fulfillment Order Status State Change
When the Order status in a Fulfillment entity is changed to anything other than Approved, an OrderStatusChangeEvent message is produced by the Fulfillment service and sent to the fulfillment.order.change Kafka topic. This message is retrieved and consumed by the Orders service. This is how the Orders service tracks all CustomerOrder lifecycle events from the initial Created status to the final Received status.
The Fulfillment service exposes several endpoints via the FulfillmentController class, which simulates a change in order status. They allow an order’s status to be changed from Approved to Processing, to Shipped, to In Transit, and finally to Received. This change applies to all orders that meet the criteria.
Each of these state changes triggers a change to the Fulfillment document in MongoDB. Each change also generates a Kafka message, containing the OrderStatusChangeEvent in the message payload. The Fulfillment service’s Sender class handles this.
Note in this example that these two events are not handled in an atomic transaction. Either updating the database or sending the message could fail independently, which would cause a loss of data consistency. In the real world, we must ensure that both these independent actions succeed or fail as a single transaction to ensure data consistency, using any of a handful of common architectural patterns.
The SenderConfig class handles the configuration of the Sender class. This Spring Kafka producer configuration class uses Spring Kafka’s JsonSerializer class to serialize the OrderStatusChangeEvent object into a JSON message payload. This class is almost identical to the SenderConfig class in the Orders and Accounts services.
The Sender class uses a KafkaTemplate to send the message to the fulfillment.order.change Kafka topic, as shown below. Message order is not critical since a timestamp is recorded, which ensures the proper sequence of order status events can be maintained. Messages can be sent to a topic with multiple partitions if the volume of messages requires it.
The Orders service’s Receiver class is responsible for consuming the OrderStatusChangeEvent message produced by the Fulfillment service.
As explained above, the Orders service is configured differently compared to the Fulfillment service, to receive messages from Kafka. The Orders service receives messages from more than one topic. The ReceiverConfig class deserializes all messages using the StringDeserializer. The Orders service’s ReceiverConfig class references the Spring Kafka AbstractKafkaListenerContainerFactory class’s setMessageConverter method, which allows for dynamic object type matching.
Each Kafka topic the Orders service consumes messages from is associated with a method in the Receiver class (shown above). This method accepts a specific object type as an input parameter, denoting the object type the message payload needs to be deserialized into. In the case of an OrderStatusChangeEvent message, the receiveOrderStatusChangeEvents method is called to consume a message from the fulfillment.order.change Kafka topic.
Part Two
In Part Two of this post, we will review how to deploy and run the storefront API components into a local development environment running on Kubernetes with Istio, using Minikube. To provide operational visibility, we will add observability tools, like Yahoo’s CMAK (Cluster Manager for Apache Kafka), Mongo Express, Kiali, Prometheus, and Grafana to our system.
This blog represents my own viewpoints and not of my employer, Amazon Web Services (AWS). All product names, logos, and brands are the property of their respective owners.
Updating and Maintaining Gradle Project Dependencies
Posted by Gary A. Stafford in Build Automation, Enterprise Software Development, Java Development, Software Development, Technology Consulting on May 5, 2018
As a DevOps Consultant, I often encounter codebases that have not been properly kept up-to-date. Likewise, I’ve authored many open-source projects on GitHub, which I use for training, presentations, and articles. Those projects often sit dormant for months at a time, #myabandonware.
Poorly maintained and dormant projects often become brittle or break, as their dependencies and indirect dependencies continue to be updated. However, blindly updating project dependencies is often the quickest way to break, or further break an application. Ask me, I’ve given in to temptation and broken my fair share of applications as a result. Nonetheless, it is helpful to be able to quickly analyze a project’s dependencies and discover available updates. Defects, performance issues, and most importantly, security vulnerabilities, are often fixed with dependency updates.
For Node.js projects, I prefer David to discover dependency updates. I have other favorites for Ruby, .NET, and Python, including OWASP Dependency-Check, great for vulnerabilities. In a similar vein, for Gradle-based Java Spring projects, I recently discovered Ben Manes’ Gradle Versions Plugin, gradle-versions-plugin. The plugin is described as a ‘Gradle plugin to discover dependency updates’. The plugin’s GitHub project has over 1,350 stars! According to the plugin project’s README file, this plugin is similar to the Versions Maven Plugin. The project further indicates there are similar Gradle plugins available, including gradle-use-latest-versions, gradle-libraries-plugin, and gradle-update-notifier.
To try the Gradle Versions Plugin, I chose a recent Gradle-based Java Spring Boot API project. I added the plugin to the gradle.build file with a single line of code.
plugins {
id 'com.github.ben-manes.versions' version '0.17.0'
}
By executing the single Gradle task, dependencyUpdates, the plugin generates a report detailing the status of all project’s dependencies, including plugins. The plugin includes a revision task property, which controls the resolution strategy of determining what constitutes the latest version of a dependency. The property supports three strategies: release, milestone (default), and integration (i.e. SNAPSHOT), which are detailed in the plugin project’s README file.
As expected, the plugin will properly resolve any variables. Using a variable is an efficient practice for setting the Spring Boot versions for multiple dependencies (i.e. springBootVersion).
ext {
springBootVersion = '2.0.1.RELEASE'
}
dependencies {
compile('com.h2database:h2:1.4.197')
compile("io.springfox:springfox-swagger-ui:2.8.0")
compile("io.springfox:springfox-swagger2:2.8.0")
compile("org.liquibase:liquibase-core:3.5.5")
compile("org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:2.6.2")
compile("org.springframework.boot:spring-boot-starter-actuator:${springBootVersion}")
compile("org.springframework.boot:spring-boot-starter-data-jpa:${springBootVersion}")
compile("org.springframework.boot:spring-boot-starter-data-rest:${springBootVersion}")
compile("org.springframework.boot:spring-boot-starter-hateoas:${springBootVersion}")
compile("org.springframework.boot:spring-boot-starter-web:${springBootVersion}")
compileOnly('org.projectlombok:lombok:1.16.20')
runtime("org.postgresql:postgresql:42.2.2")
testCompile("org.springframework.boot:spring-boot-starter-test:${springBootVersion}")
}
My first run, using the default revision level, resulted in the following output. The report indicated three of my project’s dependencies were slightly out of date:
> Configure project : Inferred project: spring-postgresql-demo, version: 4.3.0-dev.2.uncommitted+929c56e > Task :dependencyUpdates Failed to resolve ::apiElements Failed to resolve ::implementation Failed to resolve ::runtimeElements Failed to resolve ::runtimeOnly Failed to resolve ::testImplementation Failed to resolve ::testRuntimeOnly ------------------------------------------------------------ : Project Dependency Updates (report to plain text file) ------------------------------------------------------------ The following dependencies are using the latest milestone version: - com.github.ben-manes.versions:com.github.ben-manes.versions.gradle.plugin:0.17.0 - com.netflix.nebula:gradle-ospackage-plugin:4.9.0-rc.1 - com.h2database:h2:1.4.197 - io.spring.dependency-management:io.spring.dependency-management.gradle.plugin:1.0.5.RELEASE - org.projectlombok:lombok:1.16.20 - com.netflix.nebula:nebula-release-plugin:6.3.3 - org.sonarqube:org.sonarqube.gradle.plugin:2.6.2 - org.springframework.boot:org.springframework.boot.gradle.plugin:2.0.1.RELEASE - org.postgresql:postgresql:42.2.2 - org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:2.6.2 - org.springframework.boot:spring-boot-starter-actuator:2.0.1.RELEASE - org.springframework.boot:spring-boot-starter-data-jpa:2.0.1.RELEASE - org.springframework.boot:spring-boot-starter-data-rest:2.0.1.RELEASE - org.springframework.boot:spring-boot-starter-hateoas:2.0.1.RELEASE - org.springframework.boot:spring-boot-starter-test:2.0.1.RELEASE - org.springframework.boot:spring-boot-starter-web:2.0.1.RELEASE The following dependencies have later milestone versions: - org.liquibase:liquibase-core [3.5.5 -> 3.6.1] - io.springfox:springfox-swagger-ui [2.8.0 -> 2.9.0] - io.springfox:springfox-swagger2 [2.8.0 -> 2.9.0] Generated report file build/dependencyUpdates/report.txt
After reading the release notes for the three available updates, and confident I had sufficient unit, smoke, and integration tests to validate any project changes, I manually updated the dependencies. Re-running the Gradle task generated the following abridged output.
------------------------------------------------------------ : Project Dependency Updates (report to plain text file) ------------------------------------------------------------ The following dependencies are using the latest milestone version: - com.github.ben-manes.versions:com.github.ben-manes.versions.gradle.plugin:0.17.0 - com.netflix.nebula:gradle-ospackage-plugin:4.9.0-rc.1 - com.h2database:h2:1.4.197 - io.spring.dependency-management:io.spring.dependency-management.gradle.plugin:1.0.5.RELEASE - org.liquibase:liquibase-core:3.6.1 - org.projectlombok:lombok:1.16.20 - com.netflix.nebula:nebula-release-plugin:6.3.3 - org.sonarqube:org.sonarqube.gradle.plugin:2.6.2 - org.springframework.boot:org.springframework.boot.gradle.plugin:2.0.1.RELEASE - org.postgresql:postgresql:42.2.2 - org.sonarsource.scanner.gradle:sonarqube-gradle-plugin:2.6.2 - org.springframework.boot:spring-boot-starter-actuator:2.0.1.RELEASE - org.springframework.boot:spring-boot-starter-data-jpa:2.0.1.RELEASE - org.springframework.boot:spring-boot-starter-data-rest:2.0.1.RELEASE - org.springframework.boot:spring-boot-starter-hateoas:2.0.1.RELEASE - org.springframework.boot:spring-boot-starter-test:2.0.1.RELEASE - org.springframework.boot:spring-boot-starter-web:2.0.1.RELEASE - io.springfox:springfox-swagger-ui:2.9.0 - io.springfox:springfox-swagger2:2.9.0 Generated report file build/dependencyUpdates/report.txt BUILD SUCCESSFUL in 3s 1 actionable task: 1 executed
After running a series of automated unit, smoke, and integration tests, to confirm no conflicts with the updates, I committed my changes to GitHub. The Gradle Versions Plugin is a simple and effective solution to Gradle dependency management.
All opinions expressed in this post are my own, and not necessarily the views of my current or past employers, or their clients.
Gradle logo courtesy Gradle.org, © Gradle Inc.
Spring Music Revisited: Java-Spring-MongoDB Web App with Docker 1.12
Posted by Gary A. Stafford in Build Automation, Continuous Delivery, DevOps, Enterprise Software Development, Java Development, Software Development on August 7, 2016
Build, test, deploy, and monitor a multi-container, MongoDB-backed, Java Spring web application, using the new Docker 1.12.
Introduction
This post and the post’s example project represent an update to a previous post, Build and Deploy a Java-Spring-MongoDB Application using Docker. This new post incorporates many improvements made in Docker 1.12, including the use of the new Docker Compose v2 YAML format. The post’s project was also updated to use Filebeat with ELK, as opposed to Logspout, which was used previously.
In this post, we will demonstrate how to build, test, deploy, and manage a Java Spring web application, hosted on Apache Tomcat, load-balanced by NGINX, monitored by ELK with Filebeat, and all containerized with Docker.
We will use a sample Java Spring application, Spring Music, available on GitHub from Cloud Foundry. The Spring Music sample record album collection application was originally designed to demonstrate the use of database services on Cloud Foundry, using the Spring Framework. Instead of Cloud Foundry, we will host the Spring Music application locally, using Docker on VirtualBox, and optionally on AWS.
All files necessary to build this project are stored on the docker_v2 branch of the garystafford/spring-music-docker repository on GitHub. The Spring Music source code is stored on the springmusic_v2 branch of the garystafford/spring-music repository, also on GitHub.
Application Architecture
The Java Spring Music application stack contains the following technologies: Java, Spring Framework, AngularJS, Bootstrap, jQuery, NGINX, Apache Tomcat, MongoDB, the ELK Stack, and Filebeat. Testing frameworks include the Spring MVC Test Framework, Mockito, Hamcrest, and JUnit.
A few changes were made to the original Spring Music application to make it work for this demonstration, including:
- Move from Java 1.7 to 1.8 (including newer Tomcat version)
- Add unit tests for Continuous Integration demonstration purposes
- Modify MongoDB configuration class to work with non-local, containerized MongoDB instances
- Add Gradle
warNoStatictask to build WAR without static assets - Add Gradle
zipStatictask to ZIP up the application’s static assets for deployment to NGINX - Add Gradle
zipGetVersiontask with a versioning scheme for build artifacts - Add
context.xmlfile andMANIFEST.MFfile to the WAR file - Add Log4j
RollingFileAppenderappender to send log entries to Filebeat - Update versions of several dependencies, including Gradle, Spring, and Tomcat
We will use the following technologies to build, publish, deploy, and host the Java Spring Music application: Gradle, git, GitHub, Travis CI, Oracle VirtualBox, Docker, Docker Compose, Docker Machine, Docker Hub, and optionally, Amazon Web Services (AWS).
NGINX
To increase performance, the Spring Music web application’s static content will be hosted by NGINX. The application’s WAR file will be hosted by Apache Tomcat 8.5.4. Requests for non-static content will be proxied through NGINX on the front-end, to a set of three load-balanced Tomcat instances on the back-end. To further increase application performance, NGINX will also be configured for browser caching of the static content. In many enterprise environments, the use of a Java EE application server, like Tomcat, is still not uncommon.
Reverse proxying and caching are configured thought NGINX’s default.conf file, in the server configuration section:
| server { | |
| listen 80; | |
| server_name proxy; | |
| location ~* \/assets\/(css|images|js|template)\/* { | |
| root /usr/share/nginx/; | |
| expires max; | |
| add_header Pragma public; | |
| add_header Cache-Control "public, must-revalidate, proxy-revalidate"; | |
| add_header Vary Accept-Encoding; | |
| access_log off; | |
| } |
The three Tomcat instances will be manually configured for load-balancing using NGINX’s default round-robin load-balancing algorithm. This is configured through the default.conf file, in the upstream configuration section:
| upstream backend { | |
| server music_app_1:8080; | |
| server music_app_2:8080; | |
| server music_app_3:8080; | |
| } |
Client requests are received through port 80 on the NGINX server. NGINX redirects requests, which are not for non-static assets, to one of the three Tomcat instances on port 8080.
MongoDB
The Spring Music application was designed to work with a number of data stores, including MySQL, Postgres, Oracle, MongoDB, Redis, and H2, an in-memory Java SQL database. Given the choice of both SQL and NoSQL databases, we will select MongoDB.
The Spring Music application, hosted by Tomcat, will store and modify record album data in a single instance of MongoDB. MongoDB will be populated with a collection of album data from a JSON file, when the Spring Music application first creates the MongoDB database instance.
ELK
Lastly, the ELK Stack with Filebeat, will aggregate NGINX, Tomcat, and Java Log4j log entries, providing debugging and analytics to our demonstration. A similar method for aggregating logs, using Logspout instead of Filebeat, can be found in this previous post.
Continuous Integration
In this post’s example, two build artifacts, a WAR file for the application and ZIP file for the static web content, are built automatically by Travis CI, whenever source code changes are pushed to the springmusic_v2 branch of the garystafford/spring-music repository on GitHub.
Following a successful build and a small number of unit tests, Travis CI pushes the build artifacts to the build-artifacts branch on the same GitHub project. The build-artifacts branch acts as a pseudo binary repository for the project, much like JFrog’s Artifactory. These artifacts are used later by Docker to build the project’s immutable Docker images and containers.
Build Notifications
Travis CI pushes build notifications to a Slack channel, which eliminates the need to actively monitor Travis CI.
Automation Scripting
The .travis.yaml file, custom gradle.build Gradle tasks, and the deploy_travisci.sh script handles the Travis CI automation described, above.
Travis CI .travis.yaml file:
| language: java | |
| jdk: oraclejdk8 | |
| before_install: | |
| - chmod +x gradlew | |
| before_deploy: | |
| - chmod ugo+x deploy_travisci.sh | |
| script: | |
| - "./gradlew clean build" | |
| - "./gradlew warNoStatic warCopy zipGetVersion zipStatic" | |
| - sh ./deploy_travisci.sh | |
| env: | |
| global: | |
| - GH_REF: github.com/garystafford/spring-music.git | |
| - secure: <GH_TOKEN_secure_hash_here> | |
| - secure: <COMMIT_AUTHOR_EMAIL_secure_hash_here> | |
| notifications: | |
| slack: | |
| - secure: <SLACK_secure_hash_here> |
Custom gradle.build tasks:
| // new Gradle build tasks | |
| task warNoStatic(type: War) { | |
| // omit the version from the war file name | |
| version = '' | |
| exclude '**/assets/**' | |
| manifest { | |
| attributes | |
| 'Manifest-Version': '1.0', | |
| 'Created-By': currentJvm, | |
| 'Gradle-Version': GradleVersion.current().getVersion(), | |
| 'Implementation-Title': archivesBaseName + '.war', | |
| 'Implementation-Version': artifact_version, | |
| 'Implementation-Vendor': 'Gary A. Stafford' | |
| } | |
| } | |
| task warCopy(type: Copy) { | |
| from 'build/libs' | |
| into 'build/distributions' | |
| include '**/*.war' | |
| } | |
| task zipGetVersion (type: Task) { | |
| ext.versionfile = | |
| new File("${projectDir}/src/main/webapp/assets/buildinfo.properties") | |
| versionfile.text = 'build.version=' + artifact_version | |
| } | |
| task zipStatic(type: Zip) { | |
| from 'src/main/webapp/assets' | |
| appendix = 'static' | |
| version = '' | |
| } |
The deploy.sh file:
| #!/bin/bash | |
| set -e | |
| cd build/distributions | |
| git init | |
| git config user.name "travis-ci" | |
| git config user.email "${COMMIT_AUTHOR_EMAIL}" | |
| git add . | |
| git commit -m "Deploy Travis CI Build #${TRAVIS_BUILD_NUMBER} artifacts to GitHub" | |
| git push --force --quiet "https://${GH_TOKEN}@${GH_REF}" master:build-artifacts > /dev/null 2>&1 |
You can easily replicate the project’s continuous integration automation using your choice of toolchains. GitHub or BitBucket are good choices for distributed version control. For continuous integration and deployment, I recommend Travis CI, Semaphore, Codeship, or Jenkins. Couple those with a good persistent chat application, such as Glider Labs’ Slack or Atlassian’s HipChat.
Building the Docker Environment
Make sure VirtualBox, Docker, Docker Compose, and Docker Machine, are installed and running. At the time of this post, I have the following versions of software installed on my Mac:
- Mac OS X 10.11.6
- VirtualBox 5.0.26
- Docker 1.12.1
- Docker Compose 1.8.0
- Docker Machine 0.8.1
To build the project’s VirtualBox VM, Docker images, and Docker containers, execute the build script, using the following command: sh ./build_project.sh. A build script is useful when working with CI/CD automation tools, such as Jenkins CI or ThoughtWorks go. However, to understand the build process, I suggest first running the individual commands, locally.
| #!/bin/sh | |
| set -ex | |
| # clone project | |
| git clone -b docker_v2 --single-branch \ | |
| https://github.com/garystafford/spring-music-docker.git music \ | |
| && cd "$_" | |
| # provision VirtualBox VM | |
| docker-machine create --driver virtualbox springmusic | |
| # set new environment | |
| docker-machine env springmusic \ | |
| && eval "$(docker-machine env springmusic)" | |
| # mount a named volume on host to store mongo and elk data | |
| # ** assumes your project folder is 'music' ** | |
| docker volume create --name music_data | |
| docker volume create --name music_elk | |
| # create bridge network for project | |
| # ** assumes your project folder is 'music' ** | |
| docker network create -d bridge music_net | |
| # build images and orchestrate start-up of containers (in this order) | |
| docker-compose -p music up -d elk && sleep 15 \ | |
| && docker-compose -p music up -d mongodb && sleep 15 \ | |
| && docker-compose -p music up -d app \ | |
| && docker-compose scale app=3 && sleep 15 \ | |
| && docker-compose -p music up -d proxy && sleep 15 | |
| # optional: configure local DNS resolution for application URL | |
| #echo "$(docker-machine ip springmusic) springmusic.com" | sudo tee --append /etc/hosts | |
| # run a simple connectivity test of application | |
| for i in {1..9}; do curl -I $(docker-machine ip springmusic); done |
Deploying to AWS
By simply changing the Docker Machine driver to AWS EC2 from VirtualBox, and providing your AWS credentials, the springmusic environment may also be built on AWS.
Build Process
Docker Machine provisions a single VirtualBox springmusic VM on which host the project’s containers. VirtualBox provides a quick and easy solution that can be run locally for initial development and testing of the application.
Next, the script creates a Docker data volume and project-specific Docker bridge network.
Next, using the project’s individual Dockerfiles, Docker Compose pulls base Docker images from Docker Hub for NGINX, Tomcat, ELK, and MongoDB. Project-specific immutable Docker images are then built for NGINX, Tomcat, and MongoDB. While constructing the project-specific Docker images for NGINX and Tomcat, the latest Spring Music build artifacts are pulled and installed into the corresponding Docker images.
Docker Compose builds and deploys (6) containers onto the VirtualBox VM: (1) NGINX, (3) Tomcat, (1) MongoDB, and (1) ELK.
The NGINX Dockerfile:
| # NGINX image with build artifact | |
| FROM nginx:latest | |
| MAINTAINER Gary A. Stafford <garystafford@rochester.rr.com> | |
| ENV REFRESHED_AT 2016-09-17 | |
| ENV GITHUB_REPO https://github.com/garystafford/spring-music/raw/build-artifacts | |
| ENV STATIC_FILE spring-music-static.zip | |
| RUN apt-get update -qq \ | |
| && apt-get install -qqy curl wget unzip nano \ | |
| && apt-get clean \ | |
| \ | |
| && wget -O /tmp/${STATIC_FILE} ${GITHUB_REPO}/${STATIC_FILE} \ | |
| && unzip /tmp/${STATIC_FILE} -d /usr/share/nginx/assets/ | |
| COPY default.conf /etc/nginx/conf.d/default.conf | |
| # tweak nginx image set-up, remove log symlinks | |
| RUN rm /var/log/nginx/access.log /var/log/nginx/error.log | |
| # install Filebeat | |
| ENV FILEBEAT_VERSION=filebeat_1.2.3_amd64.deb | |
| RUN curl -L -O https://download.elastic.co/beats/filebeat/${FILEBEAT_VERSION} \ | |
| && dpkg -i ${FILEBEAT_VERSION} \ | |
| && rm ${FILEBEAT_VERSION} | |
| # configure Filebeat | |
| ADD filebeat.yml /etc/filebeat/filebeat.yml | |
| # CA cert | |
| RUN mkdir -p /etc/pki/tls/certs | |
| ADD logstash-beats.crt /etc/pki/tls/certs/logstash-beats.crt | |
| # start Filebeat | |
| ADD ./start.sh /usr/local/bin/start.sh | |
| RUN chmod +x /usr/local/bin/start.sh | |
| CMD [ "/usr/local/bin/start.sh" ] |
The Tomcat Dockerfile:
| # Apache Tomcat image with build artifact | |
| FROM tomcat:8.5.4-jre8 | |
| MAINTAINER Gary A. Stafford <garystafford@rochester.rr.com> | |
| ENV REFRESHED_AT 2016-09-17 | |
| ENV GITHUB_REPO https://github.com/garystafford/spring-music/raw/build-artifacts | |
| ENV APP_FILE spring-music.war | |
| ENV TERM xterm | |
| ENV JAVA_OPTS -Djava.security.egd=file:/dev/./urandom | |
| RUN apt-get update -qq \ | |
| && apt-get install -qqy curl wget \ | |
| && apt-get clean \ | |
| \ | |
| && touch /var/log/spring-music.log \ | |
| && chmod 666 /var/log/spring-music.log \ | |
| \ | |
| && wget -q -O /usr/local/tomcat/webapps/ROOT.war ${GITHUB_REPO}/${APP_FILE} \ | |
| && mv /usr/local/tomcat/webapps/ROOT /usr/local/tomcat/webapps/_ROOT | |
| COPY tomcat-users.xml /usr/local/tomcat/conf/tomcat-users.xml | |
| # install Filebeat | |
| ENV FILEBEAT_VERSION=filebeat_1.2.3_amd64.deb | |
| RUN curl -L -O https://download.elastic.co/beats/filebeat/${FILEBEAT_VERSION} \ | |
| && dpkg -i ${FILEBEAT_VERSION} \ | |
| && rm ${FILEBEAT_VERSION} | |
| # configure Filebeat | |
| ADD filebeat.yml /etc/filebeat/filebeat.yml | |
| # CA cert | |
| RUN mkdir -p /etc/pki/tls/certs | |
| ADD logstash-beats.crt /etc/pki/tls/certs/logstash-beats.crt | |
| # start Filebeat | |
| ADD ./start.sh /usr/local/bin/start.sh | |
| RUN chmod +x /usr/local/bin/start.sh | |
| CMD [ "/usr/local/bin/start.sh" ] |
Docker Compose v2 YAML
This post was recently updated for Docker 1.12, and to use Docker Compose v2 YAML file format. The post’s docker-compose.yml takes advantage of improvements in Docker 1.12 and Docker Compose v2 YAML. Improvements to the YAML file include eliminating the need to link containers and expose ports, and the addition of named networks and volumes.
| version: '2' | |
| services: | |
| proxy: | |
| build: nginx/ | |
| ports: | |
| - 80:80 | |
| networks: | |
| - net | |
| depends_on: | |
| - app | |
| hostname: proxy | |
| container_name: proxy | |
| app: | |
| build: tomcat/ | |
| ports: | |
| - 8080 | |
| networks: | |
| - net | |
| depends_on: | |
| - mongodb | |
| hostname: app | |
| mongodb: | |
| build: mongodb/ | |
| ports: | |
| - 27017:27017 | |
| networks: | |
| - net | |
| depends_on: | |
| - elk | |
| hostname: mongodb | |
| container_name: mongodb | |
| volumes: | |
| - music_data:/data/db | |
| - music_data:/data/configdb | |
| elk: | |
| image: sebp/elk:latest | |
| ports: | |
| - 5601:5601 | |
| - 9200:9200 | |
| - 5044:5044 | |
| - 5000:5000 | |
| networks: | |
| - net | |
| volumes: | |
| - music_elk:/var/lib/elasticsearch | |
| hostname: elk | |
| container_name: elk | |
| volumes: | |
| music_data: | |
| external: true | |
| music_elk: | |
| external: true | |
| networks: | |
| net: | |
| driver: bridge |
The Results
Below are the results of building the project.
| # Resulting Docker Machine VirtualBox VM: | |
| $ docker-machine ls | |
| NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS | |
| springmusic * virtualbox Running tcp://192.168.99.100:2376 v1.12.1 | |
| # Resulting external volume: | |
| $ docker volume ls | |
| DRIVER VOLUME NAME | |
| local music_data | |
| local music_elk | |
| # Resulting bridge network: | |
| $ docker network ls | |
| NETWORK ID NAME DRIVER SCOPE | |
| f564dfa1b440 music_net bridge local | |
| # Resulting Docker images - (4) base images and (3) project images: | |
| $ docker images | |
| REPOSITORY TAG IMAGE ID CREATED SIZE | |
| music_proxy latest 7a8dd90bcf32 About an hour ago 250.2 MB | |
| music_app latest c93c713d03b8 About an hour ago 393 MB | |
| music_mongodb latest fbcbbe9d4485 25 hours ago 366.4 MB | |
| tomcat 8.5.4-jre8 98cc750770ba 2 days ago 334.5 MB | |
| mongo latest 48b8b08dca4d 2 days ago 366.4 MB | |
| nginx latest 4efb2fcdb1ab 10 days ago 183.4 MB | |
| sebp/elk latest 07a3e78b01f5 13 days ago 884.5 MB | |
| # Resulting (6) Docker containers | |
| $ docker ps | |
| CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES | |
| b33922767517 music_proxy "/usr/local/bin/start" 3 hours ago Up 13 minutes 0.0.0.0:80->80/tcp, 443/tcp proxy | |
| e16d2372f2df music_app "/usr/local/bin/start" 3 hours ago Up About an hour 0.0.0.0:32770->8080/tcp music_app_3 | |
| 6b7accea7156 music_app "/usr/local/bin/start" 3 hours ago Up About an hour 0.0.0.0:32769->8080/tcp music_app_2 | |
| 2e94f766df1b music_app "/usr/local/bin/start" 3 hours ago Up About an hour 0.0.0.0:32768->8080/tcp music_app_1 | |
| 71f8dc574148 sebp/elk:latest "/usr/local/bin/start" 3 hours ago Up About an hour 0.0.0.0:5000->5000/tcp, 0.0.0.0:5044->5044/tcp, 0.0.0.0:5601->5601/tcp, 0.0.0.0:9200->9200/tcp, 9300/tcp elk | |
| f7e7d1af7cca music_mongodb "/entrypoint.sh mongo" 20 hours ago Up About an hour 0.0.0.0:27017->27017/tcp mongodb |
Testing the Application
Below are partial results of the curl test, hitting the NGINX endpoint. Note the different IP addresses in the Upstream-Address field between requests. This test proves NGINX’s round-robin load-balancing is working across the three Tomcat application instances: music_app_1, music_app_2, and music_app_3.
Also, note the sharp decrease in the Request-Time between the first three requests and subsequent three requests. The Upstream-Response-Time to the Tomcat instances doesn’t change, yet the total Request-Time is much shorter, due to caching of the application’s static assets by NGINX.
| for i in {1..6}; do curl -I $(docker-machine ip springmusic);done | |
| HTTP/1.1 200 | |
| Server: nginx/1.11.4 | |
| Date: Sat, 17 Sep 2016 18:33:50 GMT | |
| Content-Type: text/html;charset=ISO-8859-1 | |
| Content-Length: 2094 | |
| Connection: keep-alive | |
| Accept-Ranges: bytes | |
| ETag: W/"2094-1473924940000" | |
| Last-Modified: Thu, 15 Sep 2016 07:35:40 GMT | |
| Content-Language: en | |
| Request-Time: 0.575 | |
| Upstream-Address: 172.18.0.4:8080 | |
| Upstream-Response-Time: 1474137230.048 | |
| HTTP/1.1 200 | |
| Server: nginx/1.11.4 | |
| Date: Sat, 17 Sep 2016 18:33:51 GMT | |
| Content-Type: text/html;charset=ISO-8859-1 | |
| Content-Length: 2094 | |
| Connection: keep-alive | |
| Accept-Ranges: bytes | |
| ETag: W/"2094-1473924940000" | |
| Last-Modified: Thu, 15 Sep 2016 07:35:40 GMT | |
| Content-Language: en | |
| Request-Time: 0.711 | |
| Upstream-Address: 172.18.0.5:8080 | |
| Upstream-Response-Time: 1474137230.865 | |
| HTTP/1.1 200 | |
| Server: nginx/1.11.4 | |
| Date: Sat, 17 Sep 2016 18:33:52 GMT | |
| Content-Type: text/html;charset=ISO-8859-1 | |
| Content-Length: 2094 | |
| Connection: keep-alive | |
| Accept-Ranges: bytes | |
| ETag: W/"2094-1473924940000" | |
| Last-Modified: Thu, 15 Sep 2016 07:35:40 GMT | |
| Content-Language: en | |
| Request-Time: 0.326 | |
| Upstream-Address: 172.18.0.6:8080 | |
| Upstream-Response-Time: 1474137231.812 | |
| # assets now cached... | |
| HTTP/1.1 200 | |
| Server: nginx/1.11.4 | |
| Date: Sat, 17 Sep 2016 18:33:53 GMT | |
| Content-Type: text/html;charset=ISO-8859-1 | |
| Content-Length: 2094 | |
| Connection: keep-alive | |
| Accept-Ranges: bytes | |
| ETag: W/"2094-1473924940000" | |
| Last-Modified: Thu, 15 Sep 2016 07:35:40 GMT | |
| Content-Language: en | |
| Request-Time: 0.012 | |
| Upstream-Address: 172.18.0.4:8080 | |
| Upstream-Response-Time: 1474137233.111 | |
| HTTP/1.1 200 | |
| Server: nginx/1.11.4 | |
| Date: Sat, 17 Sep 2016 18:33:53 GMT | |
| Content-Type: text/html;charset=ISO-8859-1 | |
| Content-Length: 2094 | |
| Connection: keep-alive | |
| Accept-Ranges: bytes | |
| ETag: W/"2094-1473924940000" | |
| Last-Modified: Thu, 15 Sep 2016 07:35:40 GMT | |
| Content-Language: en | |
| Request-Time: 0.017 | |
| Upstream-Address: 172.18.0.5:8080 | |
| Upstream-Response-Time: 1474137233.350 | |
| HTTP/1.1 200 | |
| Server: nginx/1.11.4 | |
| Date: Sat, 17 Sep 2016 18:33:53 GMT | |
| Content-Type: text/html;charset=ISO-8859-1 | |
| Content-Length: 2094 | |
| Connection: keep-alive | |
| Accept-Ranges: bytes | |
| ETag: W/"2094-1473924940000" | |
| Last-Modified: Thu, 15 Sep 2016 07:35:40 GMT | |
| Content-Language: en | |
| Request-Time: 0.013 | |
| Upstream-Address: 172.18.0.6:8080 | |
| Upstream-Response-Time: 1474137233.594 |
Spring Music Application Links
Assuming the springmusic VM is running at 192.168.99.100, the following links can be used to access various project endpoints. Note the (3) Tomcat instances each map to randomly exposed ports. These ports are not required by NGINX, which maps to port 8080 for each instance. The port is only required if you want access to the Tomcat Web Console. The port, shown below, 32771, is merely used as an example.
- Spring Music Application: 192.168.99.100
- NGINX Status: 192.168.99.100/nginx_status
- Tomcat Web Console – music_app_1*: 192.168.99.100:32771/manager
- Environment Variables – music_app_1: 192.168.99.100:32771/env
- Album List (RESTful endpoint) – music_app_1: 192.168.99.100:32771/albums
- Elasticsearch Info: 192.168.99.100:9200
- Elasticsearch Status: 192.168.99.100:9200/_status?pretty
- Kibana Web Console: 192.168.99.100:5601
* The Tomcat user name is admin and the password is t0mcat53rv3r.
Helpful Links
- Cloud Foundry’s Spring Music Example
- Getting Started with Gradle for Java
- Introduction to Gradle
- Spring Framework
- Understanding Nginx HTTP Proxying, Load Balancing, Buffering, and Caching
- Common conversion patterns for log4j’s PatternLayout
- Spring @PropertySource example
- Java log4j logging
TODOs
- Automate the Docker image build and publish processes
- Automate the Docker container build and deploy processes
- Automate post-deployment verification testing of project infrastructure
- Add Docker Swarm multi-host capabilities with overlay networking
- Update Spring Music with latest CF project revisions
- Include scripting example to stand-up project on AWS
- Add Consul and Consul Template for NGINX configuration
Build and Deploy a Java-Spring-MongoDB Application using Docker
Posted by Gary A. Stafford in Bash Scripting, Build Automation, Continuous Delivery, DevOps, Enterprise Software Development, Software Development on September 7, 2015
Build a multi-container, MongoDB-backed, Java Spring web application, and deploy to a test environment using Docker.
Introduction
Application Architecture
Spring Music Environment
Building the Environment
Spring Music Application Links
Helpful Links
Introduction
In this post, we will demonstrate how to build, deploy, and host a multi-tier Java application using Docker. For the demonstration, we will use a sample Java Spring application, available on GitHub from Cloud Foundry. Cloud Foundry’s Spring Music sample record album collection application was originally designed to demonstrate the use of database services on Cloud Foundry and Spring Framework. Instead of Cloud Foundry, we will host the Spring Music application using Docker with VirtualBox and optionally, AWS.
All files required to build this post’s demonstration are located in the master branch of this GitHub repository. Instructions to clone the repository are below. The Java Spring Music application’s source code, used in this post’s demonstration, is located in the master branch of this GitHub repository.
A few changes were necessary to the original Spring Music application to make it work for the this demonstration. At a high-level, the changes included:
- Modify MongoDB configuration class to work with non-local MongoDB instances
- Add Gradle
warNoStatictask to build WAR file without the static assets, which will be host separately in NGINX - Create new Gradle task,
zipStatic, to ZIP up the application’s static assets for deployment to NGINX - Add versioning scheme for build artifacts
- Add
context.xmlfile andMANIFEST.MFfile to the WAR file - Add log4j
syslogappender to send log entries to Logstash - Update versions of several dependencies, including Gradle to 2.6
Application Architecture
The Java Spring Music application stack contains the following technologies:
The Spring Music web application’s static content will be hosted by NGINX for increased performance. The application’s WAR file will be hosted by Apache Tomcat. Requests for non-static content will be proxied through a single instance of NGINX on the front-end, to one of two load-balanced Tomcat instances on the back-end. NGINX will also be configured to allow for browser caching of the static content, to further increase application performance. Reverse proxying and caching are configured thought NGINX’s default.conf file’s server configuration section:
server {
listen 80;
server_name localhost;
location ~* \/assets\/(css|images|js|template)\/* {
root /usr/share/nginx/;
expires max;
add_header Pragma public;
add_header Cache-Control "public, must-revalidate, proxy-revalidate";
add_header Vary Accept-Encoding;
access_log off;
}
The two Tomcat instances will be configured on NGINX, in a load-balancing pool, using NGINX’s default round-robin load-balancing algorithm. This is configured through NGINX’s default.conf file’s upstream configuration section:
upstream backend {
server app01:8080;
server app02:8080;
}
The Spring Music application can be run with MySQL, Postgres, Oracle, MongoDB, Redis, or H2, an in-memory Java SQL database. Given the choice of both SQL and NoSQL databases available for use with the Spring Music application, we will select MongoDB.
The Spring Music application, hosted by Tomcat, will store and modify record album data in a single instance of MongoDB. MongoDB will be populated with a collection of album data when the Spring Music application first creates the MongoDB database instance.
Lastly, the ELK Stack with Logspout, will aggregate both Docker and Java Log4j log entries, providing debugging and analytics to our demonstration. I’ve used the same method for Docker and Java Log4j log entries, as detailed in this previous post.
Spring Music Environment
To build, deploy, and host the Java Spring Music application, we will use the following technologies:
- Gradle
- GitHub
- Travis CI
- git
- Oracle VirtualBox
- Docker
- Docker Compose
- Docker Machine
- Docker Hub
- optional: Amazon Web Services (AWS)
All files necessary to build this project are stored in the garystafford/spring-music-docker repository on GitHub. The Spring Music source code and build artifacts are stored in a seperate garystafford/spring-music repository, also on GitHub.
Build artifacts are automatically built by Travis CI when changes are checked into the garystafford/spring-music repository on GitHub. Travis CI then overwrites the build artifacts back to a build artifact branch of that same project. The build artifact branch acts as a pseudo binary repository for the project. The .travis.yaml file, gradle.build file, and deploy.sh script handles these functions.
.travis.yaml file:
language: java jdk: oraclejdk7 before_install: - chmod +x gradlew before_deploy: - chmod ugo+x deploy.sh script: - bash ./gradlew clean warNoStatic warCopy zipGetVersion zipStatic - bash ./deploy.sh env: global: - GH_REF: github.com/garystafford/spring-music.git - secure: <secure hash here>
gradle.build file snippet:
// new Gradle build tasks
task warNoStatic(type: War) {
// omit the version from the war file name
version = ''
exclude '**/assets/**'
manifest {
attributes
'Manifest-Version': '1.0',
'Created-By': currentJvm,
'Gradle-Version': GradleVersion.current().getVersion(),
'Implementation-Title': archivesBaseName + '.war',
'Implementation-Version': artifact_version,
'Implementation-Vendor': 'Gary A. Stafford'
}
}
task warCopy(type: Copy) {
from 'build/libs'
into 'build/distributions'
include '**/*.war'
}
task zipGetVersion (type: Task) {
ext.versionfile =
new File("${projectDir}/src/main/webapp/assets/buildinfo.properties")
versionfile.text = 'build.version=' + artifact_version
}
task zipStatic(type: Zip) {
from 'src/main/webapp/assets'
appendix = 'static'
version = ''
}
deploy.sh file:
#!/bin/bash
# reference: https://gist.github.com/domenic/ec8b0fc8ab45f39403dd
set -e # exit with nonzero exit code if anything fails
# go to the distributions directory and create a *new* Git repo
cd build/distributions && git init
# inside this git repo we'll pretend to be a new user
git config user.name "travis-ci"
git config user.email "auto-deploy@travis-ci.com"
# The first and only commit to this new Git repo contains all the
# files present with the commit message.
git add .
git commit -m "Deploy Travis CI build #${TRAVIS_BUILD_NUMBER} artifacts to GitHub"
# Force push from the current repo's master branch to the remote
# repo's build-artifacts branch. (All previous history on the gh-pages branch
# will be lost, since we are overwriting it.) We redirect any output to
# /dev/null to hide any sensitive credential data that might otherwise be exposed. Environment variables pre-configured on Travis CI.
git push --force --quiet "https://${GH_TOKEN}@${GH_REF}" master:build-artifacts > /dev/null 2>&1
Base Docker images, such as NGINX, Tomcat, and MongoDB, used to build the project’s images and subsequently the containers, are all pulled from Docker Hub.
This NGINX and Tomcat Dockerfiles pull the latest build artifacts down to build the project-specific versions of the NGINX and Tomcat Docker images used for this project. For example, the NGINX Dockerfile looks like:
# NGINX image with build artifact
FROM nginx:latest
MAINTAINER Gary A. Stafford <garystafford@rochester.rr.com>
ENV REFRESHED_AT 2015-09-20
ENV GITHUB_REPO https://github.com/garystafford/spring-music/raw/build-artifacts
ENV STATIC_FILE spring-music-static.zip
RUN apt-get update -y &&
apt-get install wget unzip nano -y &&
wget -O /tmp/${STATIC_FILE} ${GITHUB_REPO}/${STATIC_FILE} &&
unzip /tmp/${STATIC_FILE} -d /usr/share/nginx/assets/
COPY default.conf /etc/nginx/conf.d/default.conf
Docker Machine builds a single VirtualBox VM. After building the VM, Docker Compose then builds and deploys (1) NGINX container, (2) load-balanced Tomcat containers, (1) MongoDB container, (1) ELK container, and (1) Logspout container, onto the VM. Docker Machine’s VirtualBox driver provides a basic solution that can be run locally for testing and development. The docker-compose.yml for the project is as follows:
proxy: build: nginx/ ports: "80:80" links: - app01 - app02 hostname: "proxy" app01: build: tomcat/ expose: "8080" ports: "8180:8080" links: - nosqldb - elk hostname: "app01" app02: build: tomcat/ expose: "8080" ports: "8280:8080" links: - nosqldb - elk hostname: "app01" nosqldb: build: mongo/ hostname: "nosqldb" volumes: "/opt/mongodb:/data/db" elk: build: elk/ ports: - "8081:80" - "8082:9200" expose: "5000/upd" logspout: build: logspout/ volumes: "/var/run/docker.sock:/tmp/docker.sock" links: elk ports: "8083:80" environment: ROUTE_URIS=logstash://elk:5000
Building the Environment
Before continuing, ensure you have nothing running on ports 80, 8080, 8081, 8082, and 8083. Also, make sure VirtualBox, Docker, Docker Compose, Docker Machine, VirtualBox, cURL, and git are all pre-installed and running.
docker --version && docker-compose --version && docker-machine --version && echo "VirtualBox $(vboxmanage --version)" && curl --version && git --version
All of the below commands may be executed with the following single command (sh ./build_project.sh). This is useful for working with Jenkins CI, ThoughtWorks go, or similar CI tools. However, I suggest building the project step-by-step, as shown below, to better understand the process.
# clone project
git clone -b master
--single-branch https://github.com/garystafford/spring-music-docker.git &&
cd spring-music-docker
# build VM
docker-machine create --driver virtualbox springmusic --debug
# create directory to store mongo data on host
docker-machine ssh springmusic mkdir /opt/mongodb
# set new environment
docker-machine env springmusic &&
eval "$(docker-machine env springmusic)"
# build images and containers
docker-compose -f docker-compose.yml -p music up -d
# wait for container apps to start
sleep 15
# run quick test of project
for i in {1..10}
do
curl -I --url $(docker-machine ip springmusic)
done
By simply changing the driver to AWS EC2 and providing your AWS credentials, the same environment can be built on AWS within a single EC2 instance. The ‘springmusic’ environment has been fully tested both locally with VirtualBox, as well as on AWS.
Results
Resulting Docker images and containers:
gstafford@gstafford-X555LA:$ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE music_proxy latest 46af4c1ffee0 52 seconds ago 144.5 MB music_logspout latest fe64597ab0c4 About a minute ago 24.36 MB music_app02 latest d935211139f6 2 minutes ago 370.1 MB music_app01 latest d935211139f6 2 minutes ago 370.1 MB music_elk latest b03731595114 2 minutes ago 1.05 GB gliderlabs/logspout master 40a52d6ca462 14 hours ago 14.75 MB willdurand/elk latest 04cd7334eb5d 9 days ago 1.05 GB tomcat latest 6fe1972e6b08 10 days ago 347.7 MB mongo latest 5c9464760d54 10 days ago 260.8 MB nginx latest cd3cf76a61ee 10 days ago 132.9 MB gstafford@gstafford-X555LA:$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES facb6eddfb96 music_proxy "nginx -g 'daemon off" 46 seconds ago Up 46 seconds 0.0.0.0:80->80/tcp, 443/tcp music_proxy_1 abf9bb0821e8 music_app01 "catalina.sh run" About a minute ago Up About a minute 0.0.0.0:8180->8080/tcp music_app01_1 e4c43ed84bed music_logspout "/bin/logspout" About a minute ago Up About a minute 8000/tcp, 0.0.0.0:8083->80/tcp music_logspout_1 eca9a3cec52f music_app02 "catalina.sh run" 2 minutes ago Up 2 minutes 0.0.0.0:8280->8080/tcp music_app02_1 b7a7fd54575f mongo:latest "/entrypoint.sh mongo" 2 minutes ago Up 2 minutes 27017/tcp music_nosqldb_1 cbfe43800f3e music_elk "/usr/bin/supervisord" 2 minutes ago Up 2 minutes 5000/0, 0.0.0.0:8081->80/tcp, 0.0.0.0:8082->9200/tcp music_elk_1
Partial result of the curl test, calling NGINX. Note the two different upstream addresses for Tomcat. Also, note the sharp decrease in request times, due to caching.
HTTP/1.1 200 OK Server: nginx/1.9.4 Date: Mon, 07 Sep 2015 17:56:11 GMT Content-Type: text/html;charset=ISO-8859-1 Content-Length: 2090 Connection: keep-alive Accept-Ranges: bytes ETag: W/"2090-1441648256000" Last-Modified: Mon, 07 Sep 2015 17:50:56 GMT Content-Language: en Request-Time: 0.521 Upstream-Address: 172.17.0.121:8080 Upstream-Response-Time: 1441648570.774 HTTP/1.1 200 OK Server: nginx/1.9.4 Date: Mon, 07 Sep 2015 17:56:11 GMT Content-Type: text/html;charset=ISO-8859-1 Content-Length: 2090 Connection: keep-alive Accept-Ranges: bytes ETag: W/"2090-1441648256000" Last-Modified: Mon, 07 Sep 2015 17:50:56 GMT Content-Language: en Request-Time: 0.326 Upstream-Address: 172.17.0.123:8080 Upstream-Response-Time: 1441648571.506 HTTP/1.1 200 OK Server: nginx/1.9.4 Date: Mon, 07 Sep 2015 17:56:12 GMT Content-Type: text/html;charset=ISO-8859-1 Content-Length: 2090 Connection: keep-alive Accept-Ranges: bytes ETag: W/"2090-1441648256000" Last-Modified: Mon, 07 Sep 2015 17:50:56 GMT Content-Language: en Request-Time: 0.006 Upstream-Address: 172.17.0.121:8080 Upstream-Response-Time: 1441648572.050 HTTP/1.1 200 OK Server: nginx/1.9.4 Date: Mon, 07 Sep 2015 17:56:12 GMT Content-Type: text/html;charset=ISO-8859-1 Content-Length: 2090 Connection: keep-alive Accept-Ranges: bytes ETag: W/"2090-1441648256000" Last-Modified: Mon, 07 Sep 2015 17:50:56 GMT Content-Language: en Request-Time: 0.006 Upstream-Address: 172.17.0.123:8080 Upstream-Response-Time: 1441648572.266
Spring Music Application Links
Assuming springmusic VM is running at 192.168.99.100:
- Spring Music: 192.168.99.100
- NGINX: 192.168.99.100/nginx_status
- Tomcat Node 1*: 192.168.99.100:8180/manager
- Tomcat Node 2*: 192.168.99.100:8280/manager
- Kibana: 192.168.99.100:8081
- Elasticsearch: 192.168.99.100:8082
- Elasticsearch: 192.168.99.100:8082/_status?pretty
- Logspout: 192.168.99.100:8083/logs
* The Tomcat user name is admin and the password is t0mcat53rv3r.
Helpful Links
Gary Stafford
AWS Senior Solutions Architect | 8x AWS Certified Pro | DevOps | Data/ML | Serverless | Polyglot Developer | Former ThoughtWorks and Accenture
Programmatic Ponderings 