1. OVERVIEW

If your organization is working, or plans to work with large amount of unstructured data, Amazon DynamoDB would be an option to consider, especially if your organization is already invested in AWS.

DynamoDB is a schemaless NoSQL (key-value), performant, and scalable database solution.

This blog post shows you how to get started with Spring Boot 2 or 3, DynamoDB, Spring Cloud AWS, and AWS Java SDK version 2.

And continuing with the dynamic queries trend I have covered in the past when the data comes from a relational database:

as well as when the data comes from Cosmos DB:

Besides getting started, this blog post also helps you to write dynamic DynamoDB queries with DynamoDbTemplate.

Spring Boot, Spring Cloud AWS, and DynamoDB Spring Boot, Spring Cloud AWS, and DynamoDB

2. DATA MODEL

Think of the DynamoDB Data Model as the equivalent to the RDBMS Entity Relationship Diagram (ERD).

I’ll also use a single-table design to store and retrieve Customer DynamoDB entities.

Even though this blog post doesn’t cover modeling DynamoDB data in depth, it’s worth mentioning a couple of points to build a useful and accurate DynamoDB data model.

  1. The list of entities to store
    This tutorial deals with the Customer entity only.

  2. The list of read/write access patterns
    As for the Customer Spring Boot RESTful example application, these are the access patterns:

    • Addition of a new Customer entity with attributes: id, firstName, lastName, etc..
    • Retrieval of an existing Customer using its unique identifier, aided by the customer table.
    • Search for Customers by first name, last name or email address, aided by the customer table’s global secondary index (GSI) named AllCustomersFirstLastEmailIndex.

3. DYNAMODB AND DOCKER

This blog post uses Amazon’s dynamodb-local Docker image to run a DynamoDB instance.

Let’s first add this specific AWS profile to your local AWS credentials file:

~/.aws/credentials:

[dynamodb-localdev]
aws_access_key_id = testAccessKey
aws_secret_access_key = testSecretKey
region = localhost

These settings are good enough for local development purposes.

Let’s run a DynamoDB Docker container:

docker run -d -p 8000:8000 --name dynamodb amazon/dynamodb-local:latest

You can now interact with the DynamoDB instance using AWS CLI:

aws --profile=dynamodb-localdev dynamodb list-tables --endpoint-url http://localhost:8000
{
  "TableNames": []
}

Notice the command uses the AWS profile just added.

This is a new DynamoDB instance.
Let’s create a new table customer a RESTful Spring Boot application will store/retrieve Customer items to/from.

aws --profile=dynamodb-localdev dynamodb create-table --table-name customer \  
--attribute-definitions AttributeName=id,AttributeType=S AttributeName=gsiAllCustomersFirstLastEmail_PK,AttributeType=S \
--key-schema AttributeName=id,KeyType=HASH \ 
--provisioned-throughput ReadCapacityUnits=10,WriteCapacityUnits=1 \  
--global-secondary-indexes "[{ \"IndexName\": \"AllCustomersFirstLastEmailIndex\",\"KeySchema\": [{\"AttributeName\": \"gsiAllCustomersFirstLastEmail_PK\",\"KeyType\": \"HASH\"}],\"Projection\": {\"ProjectionType\":\"ALL\"},\"ProvisionedThroughput\": {\"ReadCapacityUnits\": 10, \"WriteCapacityUnits\": 1}}]" \
--table-class STANDARD --endpoint-url http://localhost:8000
{
  "TableDescription": {
    "AttributeDefinitions": [
      {
        "AttributeName": "id",
        "AttributeType": "S"
      },
      {
        "AttributeName": "gsiAllCustomersFirstLastEmail_PK",
        "AttributeType": "S"
      }
    ],
    "TableName": "customer",
    "KeySchema": [
      {
        "AttributeName": "id",
        "KeyType": "HASH"
      }
    ],
    "TableStatus": "ACTIVE",
    "CreationDateTime": "2024-10-21T18:18:24.699000-04:00",
    "ProvisionedThroughput": {
      "LastIncreaseDateTime": "1969-12-31T19:00:00-05:00",
      "LastDecreaseDateTime": "1969-12-31T19:00:00-05:00",
      "NumberOfDecreasesToday": 0,
      "ReadCapacityUnits": 10,
      "WriteCapacityUnits": 1
    },
    "TableSizeBytes": 0,
    "ItemCount": 0,
    "TableArn": "arn:aws:dynamodb:ddblocal:000000000000:table/customer",
    "GlobalSecondaryIndexes": [
      {
        "IndexName": "AllCustomersFirstLastEmailIndex",
        "KeySchema": [
          {
            "AttributeName": "gsiAllCustomersFirstLastEmail_PK",
            "KeyType": "HASH"
          }
        ],
        "Projection": {
          "ProjectionType": "ALL"
        },
        "IndexStatus": "ACTIVE",
        "ProvisionedThroughput": {
          "ReadCapacityUnits": 10,
          "WriteCapacityUnits": 1
        },
        "IndexSizeBytes": 0,
        "ItemCount": 0,
        "IndexArn": "arn:aws:dynamodb:ddblocal:000000000000:table/customer/index/AllCustomersFirstLastEmailIndex"
      }
    ],
    "DeletionProtectionEnabled": false
  }
}

DynamoDB is a NoSQL schemaless database, you only need to define the partition and sorting keys.

I added two attributes (id, gsiAllCustomersFirstLastEmail_PK), and a global secondary index (AllCustomersFirstLastEmailIndex) to the customer table.

The id attribute is partition key for the table, and gsiAllCustomersFirstLastEmail_PK is the partition key for the GSI.

DynamoDB customer table DynamoDB customer table

A DynamoDB customer table's Global Secondary Index A DynamoDB customer table’s Global Secondary Index

The purpose of the AllCustomersFirstLastEmailIndex GSI is to prevent a full table scan when searching for Customer entities by firstName, lastName or emailAddress. This scenario corresponds to the third access pattern defined in the data model section.

Projecting all the customer table attributes ("ProjectionType": "ALL") to the AllCustomersFirstLastEmailIndex GSI is OK for the purposes of this blog post.
In case this table had more attributes or complex items like Orders, Shopping Cart, etc.; you would need to project at least the attributes you plan to use in the dynamic queries.

If the table already exists, you’ll get same DynamoDB table details output when you run this command:

aws --profile=dynamodb-localdev dynamodb describe-table --table-name customer --endpoint-url http://localhost:8000

4. MAVEN DEPENDENCIES

pom.xml:

...
<parent>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-parent</artifactId>
  <version>3.2.11</version>
  <relativePath />
</parent>

<properties>
  <java.version>21</java.version>
  <spring-cloud-aws.version>3.2.1</spring-cloud-aws.version>
</properties>

<dependencies>
  <dependency>
    <groupId>io.awspring.cloud</groupId>
    <artifactId>spring-cloud-aws-starter-dynamodb</artifactId>
  </dependency>
</dependencies>

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>io.awspring.cloud</groupId>
      <artifactId>spring-cloud-aws-dependencies</artifactId>
      <version>${spring-cloud-aws.version}</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>
  • This Spring Boot application uses version 3.2.11.

  • AWS BOM’s spring-cloud-aws-dependencies is set to 3.2.1.
    This is the most recent version you can use with Spring Boot 3.2.x according to the compatibility table below.

  • spring-cloud-aws-starter-dynamodb is one of the libraries managed by spring-cloud-aws-dependencies.
    I’ll use it to implement CRUD against a DynamoDB table.

Spring Boot, Spring Cloud AWS, and AWS Java SDK compatibility table

Spring Cloud AWS Spring Boot Spring Cloud AWS Java SDK
3.0.x 3.0.x, 3.1.x 2022.0.x (4.0/Kilburn) 2.x
3.1.x 3.2.x 2023.0.x (4.0/Kilburn) 2.x
3.2.0, 3.2.1 3.2.x, 3.3.x 2023.0.x (4.0/Kilburn) 2.x

Souce: https://github.com/awspring/spring-cloud-aws

5. DYNAMODB ENTITY/BEAN

Customer.java:

@DynamoDbBean
// ...
public class Customer {

  public static final String GSI_ALL_CUSTOMERS_FIRST_LAST_EMAIL_INDEX_NAME = "AllCustomersFirstLastEmailIndex";
  public static final String GSI_ALL_CUSTOMERS_FIRST_LAST_EMAIL_ID_PK = "all-customers-first-last-email";

  @Getter(onMethod = @__({@DynamoDbPartitionKey, @DynamoDbAttribute("id")}))
  private String id;

  @Getter(onMethod = @__({@DynamoDbAttribute("firstName")}))
  private String firstName;

  @Getter(onMethod = @__({@DynamoDbAttribute("lastName")}))
  private String lastName;

  @Getter(onMethod = @__({@DynamoDbAttribute("emailAddress")}))
  private String emailAddress;

  @Getter(onMethod = @__({@DynamoDbAttribute("phone")}))
  private Phone phone;

  @Getter(onMethod = @__({@DynamoDbAttribute("mailingAddress")}))
  private Address mailingAddress;

  @Getter(onMethod = @__({
    @DynamoDbSecondaryPartitionKey(indexNames = GSI_ALL_CUSTOMERS_FIRST_LAST_EMAIL_INDEX_NAME),
    @DynamoDbAttribute("gsiAllCustomersFirstLastEmail_PK")})
  )
  private String gsiAllCustomersFirstLastEmailId;
}

Customer’s attributes are annotated with @Getter(onMethod = @__({....})) because this Spring Boot application uses Lombok.

If you are not using Lombok, you would place the @DynamoDbAttribute, @DynamoDbPartitionKey, @DynamoDbSortKey, @DynamoDbSecondaryPartitionKey, etc. annotations on the getter methods.

@DynamoDbBean annotation identifies Customer as a DynamoDB entity.

@DynamoDbPartitionKey annotation indicates id is the partition key for this table.

@DynamoDbSecondaryPartitionKey annotation indicates gsiAllCustomersFirstLastEmailId is the partition key for the GSI named AllCustomersFirstLastEmailIndex.

@DynamoDbAttribute annotation indicates which DynamoDB attribute this property maps to.

Address and Phone classes include a subset of the annotations described above.

6. DYNAMODB CUSTOMER REPOSITORY

There is no spring data dynamodb implementation that integrates with AWS Java SDK 2.x.

There is a community module with GAV coordinate:

<dependency>
  <groupId>io.github.boostchicken</groupId>
  <artifactId>spring-data-dynamodb</artifactId>
  <version>5.2.5</span>
</dependency>

but it’s outdated, it doesn’t support for AWS Java SDK 2.x, just 1.x, and it’s compatible with Spring Boot up to 2.2.x, which is legacy as it reached its end-of-life in early 2021.

Instead, let’s write our own CRUD interfaces and DynamoDB repository implementation classes.

CrudRepository.java:

public interface CrudRepository<T> {

  <S extends T> S save(S entity);

  Optional<T> findById(Key primaryKey);

}

Key is the primary key class found in AWS Java SDK. It includes the partitionValue and the sortValue attributes to help retrieving DynamoDB entities.

The CrudRepository interface implementation looks like:

SimpleDynamoDbCrudRepository.java:

public class SimpleDynamoDbCrudRepository<T> implements CrudRepository<T> {

  private final DynamoDbOperations dynamoDbTemplate;
  private final Class<T> entityClass;

// ...

  @Override
  public <S extends T> S save(S entity) {
      return this.dynamoDbTemplate.save(entity);
  }

  @Override
  public Optional<T> findById(Key primaryKey) {
      T result = this.dynamoDbTemplate.load(primaryKey, this.entityClass);
      return Optional.ofNullable(result);
  }

}

A generic DynamoDB repository implementation with two methods to cover the first two access patterns included when defining the DynamoDB data model.

  • The save() method is straightforward. It delegates saving a DynamoDB entity to dynamoDbTemplate.
  • The findById() method is also simple. It retrieves a DynamoDB entity using the primary key.

CustomerRepository.java:

public interface CustomerRepository  extends CrudRepository<Customer> {

  Iterable<Customer> findAll(CustomerSearchCriteria searchCriteria);

}

CustomerRepository is an example of interface segregation from the SOLID principles.
Had I included the findAll(CustomerSearchCriteria) method in the CrudRepository interface, and the application had another entity repository, say for instance ShoppingCartRepository, I would be forcing ShoppingCartRepository to implement findAll(CustomerSearchCriteria), which is a method unrelated to the ShoppingCart entity, and the service tier classes won’t use.

CustomerDynamoDbRepository.java:

@Repository
public class CustomerDynamoDbRepository extends SimpleDynamoDbCrudRepository<Customer> implements CustomerRepository {

  private final DynamoDbOperations dynamoDbTemplate;

// ...

  @Override
  public Iterable<Customer> findAll(CustomerSearchCriteria searchCriteria) {
    QueryEnhancedRequest query = this.buildQuery(searchCriteria);
    PageIterable<Customer> customers = this.dynamoDbTemplate.query(query, Customer.class, Customer.GSI_ALL_CUSTOMERS_FIRST_LAST_EMAIL_INDEX_NAME);
    List<Customer> result = customers.stream()
      .flatMap(page -> page.items().stream())
      .collect(Collectors.toList());
    return result;
  }

  private QueryEnhancedRequest buildQuery(CustomerSearchCriteria searchCriteria) {
    Map<String, AttributeValue> expressionAttributeValues = new HashMap<>();
    MutableObject<String> filterExpressionHolder = new MutableObject<>();
    this.buildQueryParts(searchCriteria, expressionAttributeValues, filterExpressionHolder);

    QueryConditional allCustomersQueryCondition = QueryConditional
      .keyEqualTo(Key.builder()
        .partitionValue(Customer.GSI_ALL_CUSTOMERS_FIRST_LAST_EMAIL_ID_PK)
        .build()
      );

    QueryEnhancedRequest.Builder queryBuilder = QueryEnhancedRequest.builder()
      .queryConditional(allCustomersQueryCondition);

    String filterExpression = filterExpressionHolder.getValue();
    if (filterExpression != null) {
      Expression expression = Expression.builder()
        .expression(filterExpression)
        .expressionValues(expressionAttributeValues)
        .build();
      queryBuilder.filterExpression(expression);
    }

    return queryBuilder.build();
  }

  private void buildQueryParts(CustomerSearchCriteria searchCriteria,
      Map<String, AttributeValue> expressionAttributeValues,
      MutableObject<String> filterExpressionHolder) {

    String filterExpression = null;

    if (StringUtils.isNotEmpty(searchCriteria.getFirstName())) {
      expressionAttributeValues.put(":firstName", AttributeValue.fromS(searchCriteria.getFirstName()));
      filterExpression = "firstName = :firstName";
    }

    if (StringUtils.isNotEmpty(searchCriteria.getLastName())) {
      expressionAttributeValues.put(":lastName", AttributeValue.fromS(searchCriteria.getLastName()));
      filterExpression = (filterExpression == null) ? "" : filterExpression + " or ";
      filterExpression += "lastName = :lastName";
    }

    if (StringUtils.isNotEmpty(searchCriteria.getEmail())) {
      expressionAttributeValues.put(":emailAddress", AttributeValue.fromS(searchCriteria.getEmail()));
      filterExpression = (filterExpression == null) ? "" : filterExpression + " or ";
      filterExpression += "emailAddress = :emailAddress";
    }

    filterExpressionHolder.setValue(filterExpression);
  }

}
  • CustomerDynamoDbRepository class extends from SimpleDynamoDbCrudRepository base class to reuse save() and findById() methods for Customer entities.
  • Spring scans for the @Repository stereotype-annotated classes and manages beans’ instantiation.
  • spring-cloud-aws-starter-dynamodb dependency brings in DynamoDbClient, DynamoDbEnhancedClient, and DynamoDbTemplate classes transitively.
    DynamoDbAutoConfiguration finds them in the classpath and auto-configures dynamoDbTemplate among other beans, which is passed to the CustomerDynamoDbRepository constructor.
  • The findAll() method implementation is more interesting.
    • It builds a dynamic query based on the presence of CustomerSearchCriteria class attributes, and the AllCustomersFirstLastEmailIndex GSI’s partition value, which is hard-coded to all-customers-first-last-email for all Customer entities.
    • It then executes the dynamic query on the AllCustomersFirstLastEmailIndex GSI to retrieve matching Customer entities.
    • Setting up and querying the AllCustomersFirstLastEmailIndex GSI this way avoids a full table scan.

7. SERVICE CLASS

DefaultProvisioningService.java:

@Service
public class DefaultProvisioningService implements ProvisioningService {

  private final CustomerRepository customerRepository;

  @Override
  public Customer addCustomer(Customer customer) {
    return this.customerRepository.save(customer);
  }

  @Override
  public Optional<Customer> findCustomerById(String id) {
    Key primaryKey = Key.builder()
      .partitionValue(id)
      .build();
    return this.customerRepository.findById(primaryKey);
  }

  @Override
  public Iterable<Customer> retrieveCustomers(CustomerSearchCriteria searchCriteria) {
    return this.customerRepository.findAll(searchCriteria);
  }

}

Easy business logic implementation. The service delegates the operations to the CustomerDynamoDbRepository.

Notice the findCustomerById() method instantiates the primary key with the partition key, ignoring the sorting key because the Customer entity doesn’t have it.

The searchCriteria argument instantiated in the REST Controller is passed to the findAll() repository method implementation.

CustomerSearchCriteria.java:

public class CustomerSearchCriteria {

  private String firstName;
  private String lastName;
  private String email;

}

CustomerSearchCriteria is a wrapper class to hold the request parameters passed to the corresponding GET endpoint implementation.

8. REST CONTROLLER CLASS

@RestController
@RequestMapping(value = "/api/customers", produces = MediaType.APPLICATION_JSON_VALUE)
public class CustomerController {

  private final ProvisioningService provisioningService;

  @GetMapping(path = "")
  public ResponseEntity<List<CustomerDto>> retrieveCustomers(

      @RequestParam(required = false)
      String firstName,

      @RequestParam(required = false)
      String lastName,

      @RequestParam(required = false)
      String email) {

    CustomerSearchCriteria searchCriteria = CustomerSearchCriteria.builder()
      .firstName(firstName)
      .lastName(lastName)
      .email(email)
      .build();

    Iterable<Customer> customers = this.provisioningService.retrieveCustomers(searchCriteria);
    List<CustomerDto> result = CustomerDtoMapper.INSTANCE.map(customers);
    return new ResponseEntity<>(result, HttpStatus.OK);
  }
// ...
}

The retrieveCustomer() method (left out of the code snippet) delegates to the Service implementation to retrieve a Customer by the partition key, then maps the Customer entity to the CustomerDto external model.

The addCustomer() method (left out of the code snippet) maps the EditCustomerDto external interface to the Customer internal model before saving it, and from the Customer entity to the CustomerDto external model after it is created.

The retrieveCustomers() method implementation instantiates a CustomerSearchCriteria object with the request parameters’ values, delegates to the Service implementation to search for matching Customer entities, then maps from the internal domain to the CustomerDto external model.

I used MapStruct to map to/from the internal domain and the external model objects to prevent exposing, or leaking critical and unneeded data to the clients.
Most-likely the internal and external models evolve at a different pace, and you don’t want to break API compatibility.

9. CONFIGURATION

application.yml:

aws:
  dynamodb:
    accessKey: testAccessKey
    secretKey: testSecretKey
    endpoint: http://localhost:8000/
    region: localhost

Note the aws.dynamodb properties match those added in the AWS credentials dynamodb-localdev profile.

10. SAMPLE REQUESTS / RESPONSES

As a final exercise, let’s send some API requests based on the access patterns included when defining the DynamoDB data model.

Request:

curl -H "Content-Type: application/json" -H "Accept: application/json" -X POST http://localhost:8080/api/customers -d '
{
  "firstName": "Orlando",
  "lastName": "Otero",
  "emailAddress": "invalid.1@asimiotech.com",
  "phoneNumber": "123-456-7890",
  "phoneType": "MOBILE",
  "mailingAddress": {
    "street": "123 Main St",
    "city": "Orlando",
    "state": "FL",
    "zipcode": "32801"
  }
}' | json_pp

Response:

{
  "id" : "4b073d5e-0616-444e-9e2b-0f5460e210d2",
  "emailAddress" : "invalid.1@asimiotech.com",
  "firstName" : "Orlando",
  "lastName" : "Otero",
  "mailingAddress" : {
    "city" : "Orlando",
    "state" : "FL",
    "street" : "123 Main St",
    "zipcode" : "32801"
  },
  "phoneNumber" : "123-456-7890",
  "phoneType" : "MOBILE"
}
  • Returns all Customer entities. No request parameter to filter from:
curl http://localhost:8080/api/customers | json_pp
[
  {
    "emailAddress" : "invalid.2@asimiotech.com",
    "firstName" : "Blah",
    "id" : "e67d6d19-fae1-4932-ada9-7732298b10e6",
    "lastName" : "Meh",
    "mailingAddress" : {
      "city" : "Kissimmee",
      "state" : "FL",
      "street" : "234 Main Ave",
      "zipcode" : "34741"
    },
    "phoneNumber" : "234-567-8901",
    "phoneType" : "LANDLINE"
  },
  {
    "emailAddress" : "invalid.1@asimiotech.com",
    "firstName" : "Orlando",
    "id" : "4b073d5e-0616-444e-9e2b-0f5460e210d2",
    "lastName" : "Otero",
    "mailingAddress" : {
      "city" : "Orlando",
      "state" : "FL",
      "street" : "123 Main St",
      "zipcode" : "32801"
    },
    "phoneNumber" : "123-456-7890",
    "phoneType" : "MOBILE"
  }
]
  • Filters Customer entities by the firstName request parameter:
curl http://localhost:8080/api/customers?firstName=Orlando | json_pp
[
  {
    "emailAddress" : "invalid.1@asimiotech.com",
    "firstName" : "Orlando",
    "id" : "4b073d5e-0616-444e-9e2b-0f5460e210d2",
    "lastName" : "Otero",
    "mailingAddress" : {
      "city" : "Orlando",
      "state" : "FL",
      "street" : "123 Main St",
      "zipcode" : "32801"
    },
    "phoneNumber" : "123-456-7890",
    "phoneType" : "MOBILE"
  }
]
  • Filters Customer entities by firstName OR lastName request parameters:
curl "http://localhost:8080/api/customers?firstName=Orlando&lastName=Meh" | json_pp
[
  {
    "emailAddress" : "invalid.2@asimiotech.com",
    "firstName" : "Blah",
    "id" : "e67d6d19-fae1-4932-ada9-7732298b10e6",
    "lastName" : "Meh",
    "mailingAddress" : {
      "city" : "Kissimmee",
      "state" : "FL",
      "street" : "234 Main Ave",
      "zipcode" : "34741"
    },
    "phoneNumber" : "234-567-8901",
    "phoneType" : "LANDLINE"
  },
  {
    "emailAddress" : "invalid.1@asimiotech.com",
    "firstName" : "Orlando",
    "id" : "4b073d5e-0616-444e-9e2b-0f5460e210d2",
    "lastName" : "Otero",
    "mailingAddress" : {
      "city" : "Orlando",
      "state" : "FL",
      "street" : "123 Main St",
      "zipcode" : "32801"
    },
    "phoneNumber" : "123-456-7890",
    "phoneType" : "MOBILE"
  }
]
curl http://localhost:8080/api/customers/4b073d5e-0616-444e-9e2b-0f5460e210d2 | json_pp
{
  "emailAddress" : "invalid.1@asimiotech.com",
  "firstName" : "Orlando",
  "id" : "4b073d5e-0616-444e-9e2b-0f5460e210d2",
  "lastName" : "Otero",
  "mailingAddress" : {
    "city" : "Orlando",
    "state" : "FL",
    "street" : "123 Main St",
    "zipcode" : "32801"
  },
  "phoneNumber" : "123-456-7890",
  "phoneType" : "MOBILE"
}

11. CONCLUSION

This blog post covered how to get started with Spring Boot, DynamoDB, spring-cloud-aws-starter-dynamodb, and AWS Java SDK version 2.

It explained in a nutshell the requirements to model DynamoDB data, such as listing the DynamoDB entities and access patterns.

AWS CLI was also used to provision a DynamoDB table, attributes, a global secondary index, and partition keys hosted in a DynamoDB Docker container.

And lastly, it showed you how to write dynamic DynamoDB queries with DynamoDbTemplate based on optional HTTP request parameters.

Thanks for reading and as always, feedback is very much appreciated. If you found this post helpful and would like to receive updates when content like this gets published, sign up to the newsletter.

12. SOURCE CODE

Your organization can now save time and costs by purchasing a working code base, clean implementation, with support for future revisions.

13. REFERENCES