Learn how to develop Cloud-native, RESTful Java services that query data in an AWS-based data lake using Amazon Athena’s API

Introduction

AWS provides a collection of fully-managed services that makes building and managing secure data lakes faster and easier, including AWS Lake Formation, AWS Glue, and Amazon S3. Additional analytics services such as Amazon EMR, AWS Glue Studio, and Amazon Redshift allow Data Scientists and Analysts to run high-performance queries on large volumes of semi-structured and structured data quickly and economically.

What is not always as obvious is how teams develop internal and external customer-facing analytics applications built on top of data lakes. For example, imagine sellers on an eCommerce platform, the scenario used in this post, want to make better marketing decisions regarding their products by analyzing sales trends and buyer preferences. Further, suppose the data required for the analysis must be aggregated from multiple systems and data sources; the ideal use case for a data lake.

In this post, we will explore an example Java Spring Boot RESTful Web Service that allows end-users to query data stored in a data lake on AWS. The RESTful Web Service will access data stored as Apache Parquet in Amazon S3 through an AWS Glue Data Catalog using Amazon Athena. The service will use Spring Boot and the AWS SDK for Java to expose a secure, RESTful Application Programming Interface (API).

Amazon Athena is a serverless, interactive query service based on Presto, used to query data and analyze big data in Amazon S3 using standard SQL. Using Athena functionality exposed by the AWS SDK for Java and Athena API, the Spring Boot service will demonstrate how to access tables, views, prepared statements, and saved queries (aka named queries).

TL;DR

Do you want to explore the source code for this post’s Spring Boot service or deploy it to Kubernetes before reading the full article? All the source code, Docker, and Kubernetes resources are open-source and available on GitHub.

git clone --depth 1 -b main \
    https://github.com/garystafford/athena-spring-app.git

A Docker image for the Spring Boot service is also available on Docker Hub.

Data Lake Data Source

There are endless data sources to build a demonstration data lake on AWS. This post uses the TICKIT sample database provided by AWS and designed for Amazon Redshift, AWS’s cloud data warehousing service. The database consists of seven tables. Two previous posts and associated videos, Building a Data Lake on AWS with Apache Airflow and Building a Data Lake on AWS, detail the setup of the data lake used in this post using AWS Glue and optionally Apache Airflow with Amazon MWAA.

Those two posts use the data lake pattern of segmenting data as bronze (aka raw), silver (aka refined), and gold (aka aggregated), popularized by Databricks. The data lake simulates a typical scenario where data originates from multiple sources, including an e-commerce platform, a CRM system, and a SaaS provider must be aggregated and analyzed.

Spring Projects with IntelliJ IDE

Although not a requirement, I used JetBrains IntelliJ IDEA 2022 (Ultimate Edition) to develop and test the post’s Spring Boot service. Bootstrapping Spring projects with IntelliJ is easy. Developers can quickly create a Spring project using the Spring Initializr plugin bundled with the IntelliJ.

The Spring Initializr plugin’s new project creation wizard is based on start.spring.io. The plugin allows you to quickly select the Spring dependencies you want to incorporate into your project.

Visual Studio Code

There are also several Spring extensions for the popular Visual Studio Code IDE, including Microsoft’s Spring Initializr Java Support extension.

Gradle

This post uses Gradle instead of Maven to develop, test, build, package, and deploy the Spring service. Based on the packages selected in the new project setup shown above, the Spring Initializr plugin’s new project creation wizard creates a build.gradle file. Additional packages, such as Lombak, Micrometer, and Rest Assured, were added separately.

	plugins {
	id 'org.springframework.boot' version '2.7.1'
	id 'io.spring.dependency-management' version '1.0.11.RELEASE'
	id 'java'
	id 'io.freefair.lombok' version '6.5.0-rc1'
	}
	group = 'aws.example'
	version = '1.0.0'
	sourceCompatibility = '17'
	def awsSdkVersion = '2.17.225'
	def springBootVersion = '2.7.1'
	def restAssuredVersion = '5.1.1'
	repositories {
	mavenCentral()
	}
	dependencies {
	// aws sdk
	runtimeOnly "software.amazon.awssdk:bom:${awsSdkVersion}"
	implementation "software.amazon.awssdk:athena:${awsSdkVersion}"
	// spring
	annotationProcessor "org.springframework.boot:spring-boot-configuration-processor:${springBootVersion}"
	implementation "org.springframework.boot:spring-boot-starter-web:${springBootVersion}"
	implementation "org.springframework.boot:spring-boot-starter-web:${springBootVersion}"
	implementation "org.springframework.boot:spring-boot-starter-actuator:${springBootVersion}"
	developmentOnly "org.springframework.boot:spring-boot-devtools:${springBootVersion}"
	implementation 'org.springdoc:springdoc-openapi-ui:1.6.9'
	implementation 'org.springframework:spring-context:5.3.20'
	// testings
	testImplementation "org.springframework.boot:spring-boot-starter-test:${springBootVersion}"
	testImplementation "io.rest-assured:rest-assured:${restAssuredVersion}"
	testImplementation "io.rest-assured:json-path:${restAssuredVersion}"
	testImplementation "io.rest-assured:xml-path:${restAssuredVersion}"
	testImplementation "io.rest-assured:json-schema-validator:${restAssuredVersion}"
	// monitoring
	implementation 'io.micrometer:micrometer-registry-prometheus:1.9.1'
	}
	tasks.named('test') {
	useJUnitPlatform()
	}

view raw build.gradle hosted with ❤ by GitHub

Amazon Corretto

The Spring boot service is developed for and compiled with the most recent version of Amazon Corretto 17. According to AWS, Amazon Corretto is a no-cost, multiplatform, production-ready distribution of the Open Java Development Kit (OpenJDK). Corretto comes with long-term support that includes performance enhancements and security fixes. Corretto is certified as compatible with the Java SE standard and is used internally at Amazon for many production services.

Source Code

Each API endpoint in the Spring Boot RESTful Web Service has a corresponding POJO data model class, service interface and service implementation class, and controller class. In addition, there are also common classes such as configuration, a client factory, and Athena-specific request/response methods. Lastly, there are additional class dependencies for views and prepared statements.

The project’s source code is arranged in a logical hierarchy by package and class type.

	.
	└── com
	└── example
	└── athena
	├── AthenaApplication.java
	├── common
	│   ├── AthenaClientFactory.java
	│   ├── AthenaClientFactoryImp.java
	│   ├── AthenaCommon.java
	│   ├── NamedQuery.java
	│   ├── PreparedStatement.java
	│   └── View.java
	├── config
	│   └── ConfigProperties.java
	└── tickit
	├── controller
	│   ├── BuyerLikesByCategoryController.java
	│   ├── CategoryController.java
	│   ├── DateDetailController.java
	│   ├── EventController.java
	│   ├── ListingController.java
	│   ├── SaleBySellerController.java
	│   ├── SaleController.java
	│   ├── SalesByCategoryController.java
	│   ├── UserController.java
	│   └── VenueController.java
	├── model
	│   ├── crm
	│   │   └── User.java
	│   ├── ecomm
	│   │   ├── DateDetail.java
	│   │   ├── Listing.java
	│   │   └── Sale.java
	│   ├── resultsets
	│   │   ├── BuyerLikesByCategory.java
	│   │   ├── SaleBySeller.java
	│   │   └── SalesByCategory.java
	│   └── saas
	│   ├── Category.java
	│   ├── Event.java
	│   └── Venue.java
	└── service
	├── BuyerLikesByCategoryServiceImp.java
	├── BuyersLikesByCategoryService.java
	├── CategoryService.java
	├── CategoryServiceImp.java
	├── DateDetailService.java
	├── DateDetailServiceImp.java
	├── EventService.java
	├── EventServiceImp.java
	├── ListingService.java
	├── ListingServiceImp.java
	├── SaleBySellerService.java
	├── SaleBySellerServiceImp.java
	├── SaleService.java
	├── SaleServiceImp.java
	├── SalesByCategoryService.java
	├── SalesByCategoryServiceImp.java
	├── UserService.java
	├── UserServiceImp.java
	├── VenueService.java
	└── VenueServiceImp.java

view raw athena_spring_project_structure.txt hosted with ❤ by GitHub

Amazon Athena Access

There are three standard methods for accessing Amazon Athena with the AWS SDK for Java: 1) the AthenaClient service client, 2) the AthenaAsyncClient service client for accessing Athena asynchronously, and 3) using the JDBC driver with the AWS SDK. The AthenaClient and AthenaAsyncClient service clients are both parts of the software.amazon.awssdk.services.athena package. For simplicity, this post’s Spring Boot service uses the AthenaClient service client instead of Java’s asynchronously programming model. AWS supplies basic code samples as part of their documentation as a starting point for writing Athena applications using the SDK. The code samples also use the AthenaClient service client.

POJO-based Data Model Class

For each API endpoint in the Spring Boot RESTful Web Service, there is a corresponding Plain Old Java Object (POJO). According to Wikipedia, a POGO is an ordinary Java object, not bound by any particular restriction. The POJO class is similar to a JPA Entity, representing persistent data stored in a relational database. In this case, the POJO uses Lombok’s @Data annotation. According to the documentation, this annotation generates getters for all fields, a useful toString method, and hashCode and equals implementations that check all non-transient fields. It also generates setters for all non-final fields and a constructor.

	package com.example.athena.tickit.model.saas;
	import lombok.AllArgsConstructor;
	import lombok.Data;
	import lombok.NoArgsConstructor;
	import java.time.LocalDateTime;
	@Data
	@NoArgsConstructor
	@AllArgsConstructor
	public class Event {
	private int id;
	private int venueId;
	private int catId;
	private int dateId;
	private String name;
	private LocalDateTime startTime;
	}

view raw Event.java hosted with ❤ by GitHub

Each POJO corresponds directly to a ‘silver’ table in the AWS Glue Data Catalog. For example, the Event POJO corresponds to the refined_tickit_public_event table in the tickit_demo Data Catalog database. The POJO defines the Spring Boot service’s data model for data read from the corresponding AWS Glue Data Catalog table.

The Glue Data Catalog table is the interface between the Athena query and the underlying data stored in Amazon S3 object storage. The Athena query targets the table, which returns the underlying data from S3.

Service Class

Retrieving data from the data lake via AWS Glue, using Athena, is handled by a service class. For each API endpoint in the Spring Boot RESTful Web Service, there is a corresponding Service Interface and implementation class. The service implementation class uses Spring Framework’s @Service annotation. According to the documentation, it indicates that an annotated class is a “Service,” initially defined by Domain-Driven Design (Evans, 2003) as “an operation offered as an interface that stands alone in the model, with no encapsulated state.” Most importantly for the Spring Boot service, this annotation serves as a specialization of @Component, allowing for implementation classes to be autodetected through classpath scanning.

	package com.example.athena.tickit.service;
	import com.example.athena.common.AthenaClientFactory;
	import com.example.athena.common.AthenaCommon;
	import com.example.athena.config.ConfigProperties;
	import com.example.athena.tickit.model.saas.Event;
	import org.slf4j.Logger;
	import org.slf4j.LoggerFactory;
	import org.springframework.beans.factory.annotation.Autowired;
	import org.springframework.stereotype.Service;
	import software.amazon.awssdk.services.athena.AthenaClient;
	import software.amazon.awssdk.services.athena.model.*;
	import software.amazon.awssdk.services.athena.paginators.GetQueryResultsIterable;
	import java.time.LocalDateTime;
	import java.time.format.DateTimeFormatter;
	import java.util.ArrayList;
	import java.util.List;
	import static java.lang.Integer.parseInt;
	@Service
	public class EventServiceImp implements EventService {
	private static final Logger logger = LoggerFactory.getLogger(EventServiceImp.class);
	private final ConfigProperties configProperties;
	private final AthenaClientFactory athenaClientFactory;
	private final AthenaCommon athenaCommon;
	@Autowired
	public EventServiceImp(ConfigProperties configProperties, AthenaClientFactory athenaClientFactory, AthenaCommon athenaCommon) {
	this.configProperties = configProperties;
	this.athenaClientFactory = athenaClientFactory;
	this.athenaCommon = athenaCommon;
	}
	public List<Event> findAll(Integer limit, Integer offset) {
	if (limit == null || limit < 1 || limit > configProperties.getLimit()) {
	limit = configProperties.getLimit();
	}
	if (offset == null || offset < 1) {
	offset = 0;
	}
	String whereClause = "WHERE eventid IS NOT NULL";
	String query = String.format("""
	SELECT *
	FROM refined_tickit_public_event
	%s
	ORDER BY eventid
	OFFSET %s
	LIMIT %s;""", whereClause, offset, limit);
	return startQuery(query);
	}
	public Event findById(int id) {
	String query = String.format("""
	SELECT DISTINCT *
	FROM refined_tickit_public_event
	WHERE eventid=%s""", id);
	Event event;
	try {
	event = startQuery(query).get(0);
	} catch (IndexOutOfBoundsException e) {
	logger.error(e.getMessage());
	return null;
	}
	return event;
	}
	private List<Event> startQuery(String query) {
	logger.debug(String.format("Query: %s", query.replace("\n", " ")));
	AthenaClient athenaClient = athenaClientFactory.createClient();
	String queryExecutionId = athenaCommon.submitAthenaQuery(athenaClient, query);
	athenaCommon.waitForQueryToComplete(athenaClient, queryExecutionId);
	List<Event> events = processResultRows(athenaClient, queryExecutionId);
	athenaClient.close();
	return events;
	}
	private List<Event> processResultRows(AthenaClient athenaClient, String queryExecutionId) {
	List<Event> events = new ArrayList<>();
	try {
	// Max Results can be set but if it's not set,
	// it will choose the maximum page size
	GetQueryResultsRequest getQueryResultsRequest = GetQueryResultsRequest.builder()
	.queryExecutionId(queryExecutionId).build();
	GetQueryResultsIterable getQueryResultsResults = athenaClient.getQueryResultsPaginator(getQueryResultsRequest);
	List<Row> rows;
	DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.S");
	for (GetQueryResultsResponse result : getQueryResultsResults) {
	rows = result.resultSet().rows();
	for (Row myRow : rows.subList(1, rows.size())) { // skip first row – column names
	List<Datum> allData = myRow.data();
	Event event = new Event();
	event.setId(parseInt(allData.get(0).varCharValue()));
	event.setVenueId(parseInt(allData.get(1).varCharValue()));
	event.setCatId(parseInt(allData.get(2).varCharValue()));
	event.setDateId(parseInt(allData.get(3).varCharValue()));
	event.setName(allData.get(4).varCharValue());
	event.setStartTime(LocalDateTime.parse(allData.get(5).varCharValue(), formatter));
	events.add(event);
	}
	}
	} catch (AthenaException e) {
	logger.error(e.getMessage());
	}
	return events;
	}
	}

view raw EventServiceImp.java hosted with ❤ by GitHub

Using Spring’s common constructor-based Dependency Injection (DI) method (aka constructor injection), the service auto-wires an instance of the AthenaClientFactory interface. Note that we are auto-wiring the service interface, not the service implementation, allowing us to wire in a different implementation if desired, such as for testing.

The service calls the AthenaClientFactoryclass’s createClient() method, which returns a connection to Amazon Athena using one of several available authentication methods. The authentication scheme will depend on where the service is deployed and how you want to securely connect to AWS. Some options include environment variables, local AWS profile, EC2 instance profile, or token from the web identity provider.

	return AthenaClient.builder()
	.credentialsProvider(EnvironmentVariableCredentialsProvider.create())
	.build();

view raw AthenaClient.builder.java hosted with ❤ by GitHub

The service class transforms the payload returned by an instance of GetQueryResultsResponse into an ordered collection (also known as a sequence), List<E>, where E represents a POJO. For example, with the data lake’srefined_tickit_public_event table, the service returns a List<Event>. This pattern repeats itself for tables, views, prepared statements, and named queries. Column data types can be transformed and formatted on the fly, new columns added, and existing columns skipped.

	List<Row> rows;
	DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.S");
	for (GetQueryResultsResponse result : getQueryResultsResults) {
	rows = result.resultSet().rows();
	for (Row myRow : rows.subList(1, rows.size())) { // skip first row – column names
	List<Datum> allData = myRow.data();
	Event event = new Event();
	event.setId(parseInt(allData.get(0).varCharValue()));
	event.setVenueId(parseInt(allData.get(1).varCharValue()));
	event.setCatId(parseInt(allData.get(2).varCharValue()));
	event.setDateId(parseInt(allData.get(3).varCharValue()));
	event.setName(allData.get(4).varCharValue());
	event.setStartTime(LocalDateTime.parse(allData.get(5).varCharValue(), formatter));
	events.add(event);
	}
	}

view raw EventServiceImpSnippet.java hosted with ❤ by GitHub

For each endpoint defined in the Controller class, for example, get(), findAll(), and FindById(), there is a corresponding method in the Service class. Below, we see an example of the findAll() method in the SalesByCategoryServiceImp service class. This method corresponds to the identically named method in the SalesByCategoryController controller class. Each of these service methods follows a similar pattern of constructing a dynamic Athena SQL query based on input parameters, which is passed to Athena through the AthenaClient service client using an instance of GetQueryResultsRequest.

	public List<SalesByCategory> findAll(String calendarDate, Integer limit, Integer offset) {
	if (limit == null || limit < 1 || limit > configProperties.getLimit()) {
	limit = configProperties.getLimit();
	}
	if (offset == null || offset < 1) {
	offset = 0;
	}
	String whereClause = "WHERE caldate IS NOT NULL";
	if (calendarDate != null) {
	whereClause = whereClause + " AND caldate=date('" + calendarDate + "')";
	}
	String query = String.format("""
	SELECT *
	FROM tickit_sales_by_day_and_category
	%s
	OFFSET %s
	LIMIT %s;""", whereClause, offset, limit);
	return startQuery(query);
	}

view raw SalesByCategoryServiceImpSnippet.java hosted with ❤ by GitHub

Controller Class

Lastly, there is a corresponding Controller class for each API endpoint in the Spring Boot RESTful Web Service. The controller class uses Spring Framework’s @RestController annotation. According to the documentation, this annotation is a convenience annotation that is itself annotated with @Controller and @ResponseBody. Types that carry this annotation are treated as controllers where @RequestMapping methods assume @ResponseBody semantics by default.

The controller class takes a dependency on the corresponding service class application component using constructor-based Dependency Injection (DI). Like the service example above, we are auto-wiring the service interface, not the service implementation.

	package com.example.athena.tickit.controller;
	import com.example.athena.tickit.model.saas.Event;
	import com.example.athena.tickit.service.EventService;
	import org.springframework.beans.factory.annotation.Autowired;
	import org.springframework.http.HttpStatus;
	import org.springframework.http.ResponseEntity;
	import org.springframework.web.bind.annotation.*;
	import java.util.List;
	@RestController
	@RequestMapping(value = "/events")
	public class EventController {
	private final EventService service;
	@Autowired
	public EventController(EventService service) {
	this.service = service;
	}
	@RequestMapping(method = RequestMethod.GET)
	public ResponseEntity<List<Event>> findAll(
	@RequestParam(required = false) Integer limit,
	@RequestParam(required = false) Integer offset
	) {
	List<Event> events = service.findAll(limit, offset);
	if (events.size() == 0) {
	return ResponseEntity.status(HttpStatus.NOT_FOUND).body(null);
	}
	return ResponseEntity.status(HttpStatus.OK).body(events);
	}
	@RequestMapping(value = "/{id}", method = RequestMethod.GET)
	public ResponseEntity<Event> findById(@PathVariable("id") int id) {
	Event event = service.findById(id);
	if (event == null) {
	return ResponseEntity.status(HttpStatus.NOT_FOUND).body(null);
	}
	return ResponseEntity.status(HttpStatus.OK).body(event);
	}
	}

view raw EventController.java hosted with ❤ by GitHub

The controller is responsible for serializing the ordered collection of POJOs into JSON and returning that JSON payload in the body of the HTTP response to the initial HTTP request.

Querying Views

In addition to querying AWS Glue Data Catalog tables (aka Athena tables), we also query views. According to the documentation, a view in Amazon Athena is a logical table, not a physical table. Therefore, the query that defines a view runs each time the view is referenced in a query.

For convenience, each time the Spring Boot service starts, the main AthenaApplication class calls the View.java class’s CreateView() method to check for the existence of the view, view_tickit_sales_by_day_and_category. If the view does not exist, it is created and becomes accessible to all application end-users. The view is queried through the service’s /salesbycategory endpoint.

This confirm-or-create pattern is repeated for the prepared statement in the main AthenaApplication class (detailed in the next section).

	package com.example.athena;
	import com.example.athena.common.PreparedStatement;
	import com.example.athena.common.View;
	import com.example.athena.config.ConfigProperties;
	import org.springframework.beans.factory.annotation.Autowired;
	import org.springframework.boot.SpringApplication;
	import org.springframework.boot.autoconfigure.SpringBootApplication;
	import org.springframework.boot.context.properties.EnableConfigurationProperties;
	import org.springframework.context.annotation.Bean;
	import org.springframework.web.servlet.config.annotation.CorsRegistry;
	import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
	@SpringBootApplication
	@EnableConfigurationProperties(ConfigProperties.class)
	public class AthenaApplication {
	private final PreparedStatement preparedStatement;
	private final View view;
	@Autowired
	public AthenaApplication(PreparedStatement preparedStatement, View view) {
	this.preparedStatement = preparedStatement;
	this.view = view;
	}
	public static void main(String[] args) {
	SpringApplication.run(AthenaApplication.class, args);
	}
	@Bean
	void CreatePreparedStatement() {
	preparedStatement.CreatePreparedStatement();
	}
	@Bean
	void createView() {
	view.CreateView();
	}
	@Bean
	public WebMvcConfigurer corsConfigurer() {
	return new WebMvcConfigurer() {
	@Override
	public void addCorsMappings(CorsRegistry registry) {
	registry.addMapping("/**").allowedOrigins("*");
	}
	};
	}
	}

view raw AthenaApplication.java hosted with ❤ by GitHub

Below, we see the View class called by the service at startup.

	package com.example.athena.common;
	import com.example.athena.config.ConfigProperties;
	import org.slf4j.Logger;
	import org.slf4j.LoggerFactory;
	import org.springframework.beans.factory.annotation.Autowired;
	import org.springframework.stereotype.Component;
	import software.amazon.awssdk.services.athena.AthenaClient;
	import software.amazon.awssdk.services.athena.model.GetTableMetadataRequest;
	import software.amazon.awssdk.services.athena.model.GetTableMetadataResponse;
	import software.amazon.awssdk.services.athena.model.MetadataException;
	@Component
	public class View {
	private static final Logger logger = LoggerFactory.getLogger(View.class);
	private final AthenaClientFactory athenaClientFactoryImp;
	private final ConfigProperties configProperties;
	private final AthenaCommon athenaCommon;
	@Autowired
	public View(AthenaClientFactory athenaClientFactoryImp,
	ConfigProperties configProperties,
	AthenaCommon athenaCommon) {
	this.athenaClientFactoryImp = athenaClientFactoryImp;
	this.configProperties = configProperties;
	this.athenaCommon = athenaCommon;
	}
	public void CreateView() {
	String viewName = "view_tickit_sales_by_day_and_category";
	String createViewSqlStatement = String.format("""
	CREATE VIEW %s AS
	SELECT cast(d.caldate AS DATE) AS caldate,
	c.catgroup,
	c.catname,
	sum(round(cast(s.pricepaid AS DECIMAL(8,2)) * s.qtysold, 2)) AS saleamount,
	sum(cast(s.commission AS DECIMAL(8,2))) AS commission
	FROM refined_tickit_public_sales AS s
	LEFT JOIN refined_tickit_public_event AS e ON e.eventid = s.eventid
	LEFT JOIN refined_tickit_public_date AS d ON d.dateid = s.dateid
	LEFT JOIN refined_tickit_public_category AS c ON c.catid = e.catid
	GROUP BY caldate,
	catgroup,
	catname
	ORDER BY caldate,
	catgroup,
	catname;""", viewName);
	try (AthenaClient athenaClient = athenaClientFactoryImp.createClient()) {
	try {
	GetTableMetadataResponse getPreparedStatementRequest = getGetTableMetadataResponse(viewName, athenaClient);
	logger.debug(String.format("View already exists: %s", getPreparedStatementRequest.tableMetadata().name()));
	} catch (MetadataException e) { // View does not exist
	String queryExecutionId = athenaCommon.submitAthenaQuery(athenaClient, createViewSqlStatement);
	athenaCommon.waitForQueryToComplete(athenaClient, queryExecutionId);
	// Confirm View was created
	GetTableMetadataResponse getPreparedStatementRequest = getGetTableMetadataResponse(viewName, athenaClient);
	logger.debug(String.format("View created successfully: %s", getPreparedStatementRequest.tableMetadata().name()));
	}
	}
	}
	private GetTableMetadataResponse getGetTableMetadataResponse(String viewName, AthenaClient athenaClient) {
	GetTableMetadataRequest getTableMetadataRequest = GetTableMetadataRequest.builder()
	.catalogName(configProperties.getCatalog())
	.databaseName(configProperties.getDatabase())
	.tableName(viewName)
	.build();
	return athenaClient.getTableMetadata(getTableMetadataRequest);
	}
	}

view raw View.java hosted with ❤ by GitHub

Aside from the fact the /salesbycategory endpoint queries a view, everything else is identical to querying a table. This endpoint uses the same model-service-controller pattern.

Executing Prepared Statements

According to the documentation, you can use the Athena parameterized query feature to prepare statements for repeated execution of the same query with different query parameters. The prepared statement used by the service, tickit_sales_by_seller, accepts a single parameter, the ID of the seller (sellerid). The prepared statement is executed using the /salesbyseller endpoint. This scenario simulates an end-user of the analytics application who wants to retrieve enriched sales information about their sales.

	package com.example.athena.common;
	import com.example.athena.config.ConfigProperties;
	import org.slf4j.Logger;
	import org.slf4j.LoggerFactory;
	import org.springframework.beans.factory.annotation.Autowired;
	import org.springframework.stereotype.Component;
	import software.amazon.awssdk.services.athena.AthenaClient;
	import software.amazon.awssdk.services.athena.model.CreatePreparedStatementRequest;
	import software.amazon.awssdk.services.athena.model.GetPreparedStatementRequest;
	import software.amazon.awssdk.services.athena.model.GetPreparedStatementResponse;
	import software.amazon.awssdk.services.athena.model.ResourceNotFoundException;
	@Component
	public class PreparedStatement {
	private static final Logger logger = LoggerFactory.getLogger(PreparedStatement.class);
	private final AthenaClientFactory athenaClientFactoryImp;
	private final ConfigProperties configProperties;
	@Autowired
	public PreparedStatement(AthenaClientFactory athenaClientFactoryImp, ConfigProperties configProperties) {
	this.athenaClientFactoryImp = athenaClientFactoryImp;
	this.configProperties = configProperties;
	}
	public void CreatePreparedStatement() {
	String preparedStatementName = "tickit_sales_by_seller";
	String preparedStatementSql = """
	SELECT cast(d.caldate AS DATE) AS caldate,
	s.pricepaid,
	s.qtysold,
	round(cast(s.pricepaid AS DECIMAL(8,2)) * s.qtysold, 2) AS saleamount,
	cast(s.commission AS DECIMAL(8,2)) AS commission,
	round((cast(s.commission AS DECIMAL(8,2)) / (cast(s.pricepaid AS DECIMAL(8,2)) * s.qtysold)) * 100, 2) AS commissionprcnt,
	e.eventname,
	concat(u1.firstname, ' ', u1.lastname) AS seller,
	concat(u2.firstname, ' ', u2.lastname) AS buyer,
	c.catgroup,
	c.catname
	FROM refined_tickit_public_sales AS s
	LEFT JOIN refined_tickit_public_listing AS l ON l.listid = s.listid
	LEFT JOIN refined_tickit_public_users AS u1 ON u1.userid = s.sellerid
	LEFT JOIN refined_tickit_public_users AS u2 ON u2.userid = s.buyerid
	LEFT JOIN refined_tickit_public_event AS e ON e.eventid = s.eventid
	LEFT JOIN refined_tickit_public_date AS d ON d.dateid = s.dateid
	LEFT JOIN refined_tickit_public_category AS c ON c.catid = e.catid
	WHERE s.sellerid = ?
	ORDER BY caldate,
	eventname;""";
	try (AthenaClient athenaClient = athenaClientFactoryImp.createClient()) {
	try {
	GetPreparedStatementResponse getPreparedStatementResponse = getGetPreparedStatementResponse(preparedStatementName, athenaClient);
	logger.debug(String.format("Prepared statement already exists: %s", getPreparedStatementResponse.preparedStatement().statementName()));
	} catch (ResourceNotFoundException e) { // PreparedStatement does not exist
	CreatePreparedStatementRequest createPreparedStatementRequest = CreatePreparedStatementRequest.builder()
	.statementName(preparedStatementName)
	.description("Returns all sales by seller based on the seller's userid")
	.workGroup(configProperties.getWorkGroup())
	.queryStatement(preparedStatementSql).build();
	athenaClient.createPreparedStatement(createPreparedStatementRequest);
	// Confirm PreparedStatement was created
	GetPreparedStatementResponse getPreparedStatementResponse = getGetPreparedStatementResponse(preparedStatementName, athenaClient);
	logger.debug(String.format("Prepared statement created successfully: %s", getPreparedStatementResponse.preparedStatement().statementName()));
	}
	}
	}
	private GetPreparedStatementResponse getGetPreparedStatementResponse(String preparedStatementName, AthenaClient athenaClient) {
	GetPreparedStatementRequest getPreparedStatementRequest = GetPreparedStatementRequest.builder()
	.statementName(preparedStatementName)
	.workGroup(configProperties.getWorkGroup()).build();
	return athenaClient.getPreparedStatement(getPreparedStatementRequest);
	}
	}

view raw PreparedStatement.java hosted with ❤ by GitHub

The pattern of querying data is similar to tables and views, except instead of using the common SELECT...FROM...WHERE SQL query pattern, we use the EXECUTE...USING pattern.

	package com.example.athena.tickit.service;
	import com.example.athena.common.AthenaClientFactory;
	import com.example.athena.common.AthenaCommon;
	import com.example.athena.tickit.model.resultsets.SaleBySeller;
	import org.slf4j.Logger;
	import org.slf4j.LoggerFactory;
	import org.springframework.beans.factory.annotation.Autowired;
	import org.springframework.stereotype.Service;
	import software.amazon.awssdk.services.athena.AthenaClient;
	import software.amazon.awssdk.services.athena.model.*;
	import software.amazon.awssdk.services.athena.paginators.GetQueryResultsIterable;
	import java.math.BigDecimal;
	import java.time.LocalDate;
	import java.util.ArrayList;
	import java.util.List;
	@Service
	public class SaleBySellerServiceImp implements SaleBySellerService {
	private static final Logger logger = LoggerFactory.getLogger(SaleBySellerServiceImp.class);
	private final AthenaClientFactory athenaClientFactory;
	private final AthenaCommon athenaCommon;
	@Autowired
	public SaleBySellerServiceImp(AthenaClientFactory athenaClientFactory, AthenaCommon athenaCommon) {
	this.athenaClientFactory = athenaClientFactory;
	this.athenaCommon = athenaCommon;
	}
	public List<SaleBySeller> find(int id) {
	String query = String.format("""
	EXECUTE tickit_sales_by_seller USING %s;""", id);
	return startQuery(query);
	}
	private List<SaleBySeller> startQuery(String query) {
	logger.debug(String.format("Query: %s", query.replace("\n", " ")));
	AthenaClient athenaClient = athenaClientFactory.createClient();
	String queryExecutionId = athenaCommon.submitAthenaQuery(athenaClient, query);
	athenaCommon.waitForQueryToComplete(athenaClient, queryExecutionId);
	List<SaleBySeller> saleBySellers = processResultRows(athenaClient, queryExecutionId);
	athenaClient.close();
	return saleBySellers;
	}
	private List<SaleBySeller> processResultRows(AthenaClient athenaClient, String queryExecutionId) {
	List<SaleBySeller> saleBySellers = new ArrayList<>();
	try {
	GetQueryResultsRequest getQueryResultsRequest = GetQueryResultsRequest.builder()
	.queryExecutionId(queryExecutionId).build();
	GetQueryResultsIterable getQueryResultsResults = athenaClient.getQueryResultsPaginator(getQueryResultsRequest);
	List<Row> rows;
	for (GetQueryResultsResponse result : getQueryResultsResults) {
	rows = result.resultSet().rows();
	for (Row myRow : rows.subList(1, rows.size())) { // skip first row – column names
	List<Datum> allData = myRow.data();
	SaleBySeller saleBySeller = new SaleBySeller();
	saleBySeller.setCalDate(LocalDate.parse(allData.get(0).varCharValue()));
	saleBySeller.setPricePaid(new BigDecimal(allData.get(1).varCharValue()));
	saleBySeller.setQtySold(Integer.parseInt(allData.get(2).varCharValue()));
	saleBySeller.setSaleAmount(new BigDecimal(allData.get(3).varCharValue()));
	saleBySeller.setCommission(new BigDecimal(allData.get(4).varCharValue()));
	saleBySeller.setCommissionPrcnt(Double.valueOf(allData.get(5).varCharValue()));
	saleBySeller.setEventName(allData.get(6).varCharValue());
	saleBySeller.setSeller(allData.get(7).varCharValue());
	saleBySeller.setBuyer(allData.get(8).varCharValue());
	saleBySeller.setCatGroup(allData.get(9).varCharValue());
	saleBySeller.setCatName(allData.get(10).varCharValue());
	saleBySellers.add(saleBySeller);
	}
	}
	} catch (AthenaException e) {
	logger.error(e.getMessage());
	}
	return saleBySellers;
	}
	}

view raw SaleBySellerServiceImp.java hosted with ❤ by GitHub

For example, to execute the prepared statement for a seller with an ID of 3, we would use EXECUTE tickit_sales_by_seller USING 3;. We pass the seller’s ID of 3 as a path parameter similar to other endpoints exposed by the service: /v1/salesbyseller/3.

Again, aside from the fact the /salesbyseller endpoint executes a prepared statement and passes a parameter; everything else is identical to querying a table or a view, using the same model-service-controller pattern.

Working with Named Queries

In addition to tables, views, and prepared statements, Athena has the concept of saved queries, referred to as named queries in the Athena API and when using AWS CloudFormation. You can use the Athena console or API to save, edit, run, rename, and delete queries. The queries are persisted using a NamedQueryId, a unique identifier (UUID) of the query. You must reference the NamedQueryId when working with existing named queries.

There are multiple ways to use and reuse existing named queries programmatically. For this demonstration, I created the named query, buyer_likes_by_category, in advance and then stored the resulting NamedQueryId as an application property, injected at runtime or kubernetes deployment time through a local environment variable.

	package com.example.athena.tickit.service;
	import com.example.athena.common.AthenaClientFactory;
	import com.example.athena.common.AthenaCommon;
	import com.example.athena.config.ConfigProperties;
	import com.example.athena.tickit.model.resultsets.BuyerLikesByCategory;
	import org.slf4j.Logger;
	import org.slf4j.LoggerFactory;
	import org.springframework.beans.factory.annotation.Autowired;
	import org.springframework.stereotype.Service;
	import software.amazon.awssdk.services.athena.AthenaClient;
	import software.amazon.awssdk.services.athena.model.*;
	import software.amazon.awssdk.services.athena.paginators.GetQueryResultsIterable;
	import java.util.ArrayList;
	import java.util.List;
	import static java.lang.Integer.parseInt;
	@Service
	public class BuyerLikesByCategoryServiceImp implements BuyersLikesByCategoryService {
	private static final Logger logger = LoggerFactory.getLogger(BuyerLikesByCategoryServiceImp.class);
	private final ConfigProperties configProperties;
	private final AthenaClientFactory athenaClientFactory;
	private final AthenaCommon athenaCommon;
	@Autowired
	public BuyerLikesByCategoryServiceImp(ConfigProperties configProperties, AthenaClientFactory athenaClientFactory, AthenaCommon athenaCommon) {
	this.configProperties = configProperties;
	this.athenaClientFactory = athenaClientFactory;
	this.athenaCommon = athenaCommon;
	}
	public List<BuyerLikesByCategory> get() {
	return getNamedQueryResults(configProperties.getNamedQueryId());
	}
	private List<BuyerLikesByCategory> getNamedQueryResults(String queryId) {
	logger.debug(String.format("NamedQueryId: %s", queryId));
	AthenaClient athenaClient = athenaClientFactory.createClient();
	GetNamedQueryRequest getNamedQueryRequest = GetNamedQueryRequest.builder()
	.namedQueryId(queryId)
	.build();
	GetNamedQueryResponse getNamedQueryResponse = athenaClient.getNamedQuery(getNamedQueryRequest);
	String queryExecutionId = athenaCommon.submitAthenaQuery(athenaClient, getNamedQueryResponse.namedQuery().queryString());
	athenaCommon.waitForQueryToComplete(athenaClient, queryExecutionId);
	List<BuyerLikesByCategory> buyerLikesByCategories = processResultRows(athenaClient, queryExecutionId);
	athenaClient.close();
	return buyerLikesByCategories;
	}
	private List<BuyerLikesByCategory> processResultRows(AthenaClient athenaClient, String queryExecutionId) {
	List<BuyerLikesByCategory> buyerLikesByCategories = new ArrayList<>();
	try {
	// Max Results can be set but if it's not set,
	// it will choose the maximum page size
	GetQueryResultsRequest getQueryResultsRequest = GetQueryResultsRequest.builder()
	.queryExecutionId(queryExecutionId).build();
	GetQueryResultsIterable getQueryResultsResults = athenaClient.getQueryResultsPaginator(getQueryResultsRequest);
	List<Row> rows;
	for (GetQueryResultsResponse result : getQueryResultsResults) {
	rows = result.resultSet().rows();
	for (Row myRow : rows.subList(1, rows.size())) { // skip first row – column names
	List<Datum> allData = myRow.data();
	BuyerLikesByCategory buyerLikesByCategory = new BuyerLikesByCategory();
	buyerLikesByCategory.setSports(parseInt(allData.get(0).varCharValue()));
	buyerLikesByCategory.setTheatre(parseInt(allData.get(1).varCharValue()));
	buyerLikesByCategory.setConcerts(parseInt(allData.get(2).varCharValue()));
	buyerLikesByCategory.setJazz(parseInt(allData.get(3).varCharValue()));
	buyerLikesByCategory.setClassical(parseInt(allData.get(4).varCharValue()));
	buyerLikesByCategory.setOpera(parseInt(allData.get(5).varCharValue()));
	buyerLikesByCategory.setRock(parseInt(allData.get(6).varCharValue()));
	buyerLikesByCategory.setVegas(parseInt(allData.get(7).varCharValue()));
	buyerLikesByCategory.setBroadway(parseInt(allData.get(8).varCharValue()));
	buyerLikesByCategory.setMusicals(parseInt(allData.get(9).varCharValue()));
	buyerLikesByCategories.add(buyerLikesByCategory);
	}
	}
	} catch (AthenaException e) {
	logger.error(e.getMessage());
	}
	return buyerLikesByCategories;
	}
	}

view raw BuyerLikesByCategoryServiceImp.java hosted with ❤ by GitHub

Alternately, you might iterate through a list of named queries to find one that matches the name at startup. However, this method would undoubtedly impact service performance, startup time, and cost. Lastly, you could use a method like NamedQuery() included in the unused NamedQuery class at startup, similar to the view and prepared statement. That named query’s unique NamedQueryId would be persisted as a system property, referencable by the service class. The downside is that you would create a duplicate of the named query each time you start the service. Therefore, this method is also not recommended.

Configuration

Two components responsible for persisting configuration for the Spring Boot service are the application.yml properties file and ConfigProperties class. The class uses Spring Framework’s @ConfigurationProperties annotation. According to the documentation, this annotation is used for externalized configuration. Add this to a class definition or a @Bean method in a @Configuration class if you want to bind and validate some external Properties (e.g., from a .properties or .yml file). Binding is performed by calling setters on the annotated class or, if @ConstructorBinding in use, by binding to the constructor parameters.

The @ConfigurationProperties annotation includes the prefix of athena. This value corresponds to the athena prefix in the the application.yml properties file. The fields in the ConfigProperties class are bound to the properties in the the application.yml. For example, the property, namedQueryId, is bound to the property, athena.named.query.id. Further, that property is bound to an external environment variable, NAMED_QUERY_ID. These values could be supplied from an external configuration system, a Kubernetes secret, or external secrets management system.

	spring:
	profiles:
	active: dev
	server:
	port: 8080
	servlet:
	contextPath: /v1
	athena:
	region: us-east-1
	workgroup: primary
	catalog: AwsDataCatalog
	database: tickit_demo
	limit: 25
	client-execution-timeout: 100000
	retry-sleep: 1000
	results-bucket: ${RESULTS_BUCKET}
	named-query-id: ${NAMED_QUERY_ID}
	—
	spring:
	config:
	activate:
	on-profile: dev
	logging:
	level:
	root: DEBUG
	management:
	endpoints:
	web:
	exposure:
	include: '*'
	jmx:
	exposure:
	include: '*'
	—
	spring:
	config:
	activate:
	on-profile: prod
	logging:
	level:
	root: INFO
	management:
	endpoints:
	web:
	exposure:
	include: health, prometheus
	jmx:
	exposure:
	include: health

view raw application.yml hosted with ❤ by GitHub

AWS IAM: Authentication and Authorization

For the Spring Boot service to interact with Amazon Athena, AWS Glue, and Amazon S3, you need to establish an AWS IAM Role, which the service assumes once authenticated. The Role must be associated with an attached IAM Policy containing the requisite Athena, Glue, and S3 permissions. For development, the service uses a policy similar to the one shown below. Please note this policy is broader than recommended for Production; it does not represent the security best practice of least privilege. In particular, the use of the overly-broad * for Resources should be strictly avoided when creating policies.

	{
	"Version": "2012-10-17",
	"Statement": [
	{
	"Effect": "Allow",
	"Action": [
	"athena:StartQueryExecution",
	"athena:CreatePreparedStatement",
	"athena:ListPreparedStatements",
	"glue:CreateTable",
	"athena:CreateNamedQuery",
	"athena:ListNamedQueries",
	"athena:GetTableMetadata",
	"athena:GetPreparedStatement",
	"athena:GetQueryResults",
	"athena:GetQueryExecution",
	"athena:GetNamedQuery"
	],
	"Resource": [
	"*"
	]
	},
	{
	"Effect": "Allow",
	"Action": [
	"glue:BatchGetPartition",
	"glue:GetTable"
	],
	"Resource": [
	"*"
	]
	},
	{
	"Effect": "Allow",
	"Action": [
	"s3:GetBucketLocation",
	"s3:GetObject",
	"s3:ListBucket",
	"s3:ListBucketMultipartUploads",
	"s3:ListMultipartUploadParts",
	"s3:AbortMultipartUpload",
	"s3:CreateBucket",
	"s3:PutObject",
	"s3:PutBucketPublicAccessBlock"
	],
	"Resource": [
	"arn:aws:s3:::aws-athena-query-results-*"
	]
	},
	{
	"Effect": "Allow",
	"Action": [
	"s3:GetObject",
	"s3:ListBucket"
	],
	"Resource": [
	"arn:aws:s3:::date-lake-demo-*"
	]
	},
	{
	"Effect": "Allow",
	"Action": [
	"s3:ListBucket",
	"s3:GetBucketLocation",
	"s3:ListAllMyBuckets"
	],
	"Resource": [
	"*"
	]
	}
	]
	}

view raw AthenaSpringAppPolicy.json hosted with ❤ by GitHub

In addition to the authorization granted by the IAM Policy, AWS Lake Formation can be used with Amazon S3, AWS Glue, and Amazon Athena to grant fine-grained database-, table-, column-, and row-level access to datasets.

Swagger UI and the OpenAPI Specification

The easiest way to view and experiment with all the endpoints available through the controller classes is using the Swagger UI, included in the example Spring Boot service, by way of the springdoc-openapi Java library. The Swagger UI is accessed at /v1/swagger-ui/index.html.

The OpenAPI Specification (formerly Swagger Specification) is an API description format for REST APIs. The /v1/v3/api-docs endpoint allows you to generate an OpenAPI v3 specification file. The OpenAPI file describes the entire API.

The OpenAPI v3 specification can be saved as a file and imported into applications like Postman, the API platform for building and using APIs.

Integration Tests

Included in the Spring Boot service’s source code is a limited number of example integration tests, not to be confused with unit tests. Each test class uses Spring Framework’s @SpringBootTest annotation. According to the documentation, this annotation can be specified on a test class that runs Spring Boot-based tests. It provides several features over and above the regular Spring TestContext Framework.

	package com.example.athena;
	import io.restassured.http.ContentType;
	import org.hamcrest.Matchers;
	import org.junit.jupiter.api.Test;
	import org.springframework.boot.test.context.SpringBootTest;
	import static io.restassured.RestAssured.get;
	import static io.restassured.RestAssured.given;
	import static io.restassured.http.ContentType.JSON;
	@SpringBootTest
	class CategoriesResourceTests {
	private static final String ResourcePath = "/v1/categories";
	private static final int resultsetLimit = 25;
	@Test
	void findAll() {
	given()
	.contentType(ContentType.JSON)
	.accept(ContentType.JSON)
	.when()
	.get(ResourcePath)
	.then()
	.assertThat()
	.contentType(JSON)
	.statusCode(200)
	.body("$.size()", Matchers.greaterThanOrEqualTo(1))
	.body("$.size()", Matchers.lessThanOrEqualTo(resultsetLimit));
	}
	@Test
	void findAllWithLimit() {
	given()
	.contentType(ContentType.JSON)
	.accept(ContentType.JSON)
	.queryParam("limit", 3)
	.when()
	.get(ResourcePath)
	.then()
	.assertThat()
	.contentType(JSON)
	.statusCode(200)
	.body("$.size()", Matchers.equalTo(3));
	}
	@Test
	void findAllWithOffset() {
	given()
	.contentType(ContentType.JSON)
	.accept(ContentType.JSON)
	.queryParam("offset", 2)
	.when()
	.get(ResourcePath)
	.then()
	.assertThat()
	.contentType(JSON)
	.statusCode(200)
	.body("$.size()", Matchers.greaterThanOrEqualTo(1))
	.body("$.size()", Matchers.lessThanOrEqualTo(resultsetLimit));
	}
	@Test
	void findAllWithLimitAndOffset() {
	given()
	.contentType(ContentType.JSON)
	.accept(ContentType.JSON)
	.queryParam("limit", 3)
	.queryParam("offset", 2)
	.when()
	.get(ResourcePath)
	.then()
	.assertThat()
	.contentType(JSON)
	.statusCode(200)
	.body("$.size()", Matchers.equalTo(3));
	}
	@Test
	void findById() {
	// Get the first 'id' available
	int id = get(ResourcePath + "?limit=1")
	.then()
	.extract()
	.path("[0].id");
	given()
	.contentType(ContentType.JSON)
	.accept(ContentType.JSON)
	.when()
	.get(ResourcePath + "/{id}", id)
	.then()
	.assertThat()
	.contentType(JSON)
	.statusCode(200)
	.body("id", Matchers.equalTo(id));
	}
	}

view raw CategoriesResourceTests.java hosted with ❤ by GitHub

The integration tests use Rest Assured’s given-when-then pattern of testing, made popular as part of Behavior-Driven Development (BDD). In addition, each test uses the JUnit’s @Test annotation. According to the documentation, this annotation signals that the annotated method is a test method. Therefore, methods using this annotation must not be private or static and must not return a value.

	@Test
	void findById() {
	// Get the first 'id' available
	int id = get(ResourcePath + "?limit=1")
	.then()
	.extract()
	.path("[0].id");
	given()
	.contentType(ContentType.JSON)
	.accept(ContentType.JSON)
	.when()
	.get(ResourcePath + "/{id}", id)
	.then()
	.assertThat()
	.contentType(JSON)
	.statusCode(200)
	.body("id", Matchers.equalTo(id));
	}

view raw CategoriesResourceTestsSnippet.java hosted with ❤ by GitHub

Run the integration tests using Gradle from the project’s root: ./gradlew clean build test. A detailed ‘Test Summary’ is produced in the project’s build directory as HTML for easy review.

Load Testing the Service

In Production, the Spring Boot service will need to handle multiple concurrent users executing queries against Amazon Athena.

We could use various load testing tools to evaluate the service’s ability to handle multiple concurrent users. One of the simplest is my favorite go-based utility, hey, which sends load to a URL using a provided number of requests in the provided concurrency level and prints stats. It also supports HTTP2 endpoints. So, for example, we could execute 500 HTTP requests with a concurrency level of 25 against the Spring Boot service’s /users endpoint using hey. The post’s integration tests were run against three Kubernetes replica pods of the service deployed to Amazon EKS.

hey -n 500 -c 25 -T "application/json;charset=UTF-8" \
  -h2 https://athena.example-api.com/v1/users

From Athena’s Recent Queries console, we see many simultaneous queries being queued and executed by a hey through the Spring Boot service’s endpoint.

Metrics

The Spring Boot service implements the micrometer-registry-prometheus extension. The Micrometer metrics library exposes runtime and application metrics. Micrometer defines a core library, providing a registration mechanism for metrics and core metric types. These metrics are exposed by the service’s /v1/actuator/prometheus endpoint.

Using the Micrometer extension, metrics exposed by the /v1/actuator/prometheus endpoint can be scraped and visualized by tools such as Prometheus. Conveniently, AWS offers the fully-managed Amazon Managed Service for Prometheus (AMP), which easily integrates with Amazon EKS.

Using Prometheus as a datasource, we can build dashboards in Grafana to observe the service’s metrics. Like AMP, AWS also offers the fully-managed Amazon Managed Grafana (AMG).

Conclusion

This post taught us how to create a Spring Boot RESTful Web Service, allowing end-user applications to securely query data stored in a data lake on AWS. The service used AWS SDK for Java to access data stored in Amazon S3 through an AWS Glue Data Catalog using Amazon Athena.

---

This blog represents my own viewpoints and not of my employer, Amazon Web Services (AWS). All product names, logos, and brands are the property of their respective owners. All diagrams and illustrations are property of the author unless otherwise noted.