Amazon DynamoDB, a NoSQL database service, offers unparalleled scalability and flexibility for modern applications. But efficiently retrieving data requires understanding its core read operations: queries and scans.
This tutorial dives deep into using Spring Cloud AWS to seamlessly interact with DynamoDB and harness the power of both queries and scans.
If you're new to Spring Cloud AWS and DynamoDB, I recommend reading our (getButton) #text=(Spring Boot, DynamoDB, and Spring Cloud AWS: A Step-by-Step Tutorial) #icon=(link) #color=(#35a576), which will help you build up your project and get deep into the CRUD (Create, Read, Update, and Delete) operations.
The complete code for this project is available on my GitHub repository (getButton) #text=(GitHub) #icon=(share) #color=(#000000)
Mastering Scans: Unleashing DynamoDB's Power for Comprehensive Data Retrieval
Amazon DynamoDB's Scan operation reads all items from a table or secondary index. By default, a scan operation retrieves all data attributes for each item in a table or index.
You can use the ProjectionExpression parameter to restrict Scan from returning all attributes. Scan always produces a set of results. If no matching items are discovered, the result set is blank.
DynamoDB's Scan operation fetches data in chunks of 1 MB at most. To refine the results and avoid unnecessary data transfer, you can leverage filter expressions that narrow down the items after they're retrieved but before they're delivered.
Scanning DynamoDB with Spring Cloud AWS: Code Example
Ready to unleash the full power of your DynamoDB table? We'll begin by exploring Spring Cloud AWS's Scan operation to extract all its data.
public PageIterable<Movie> scanMovieTable() {
PageIterable<Movie> moviePageIterable = dynamoDbTemplate.scanAll(Movie.class);
return moviePageIterable;
}Spring Cloud AWS provides a convenient method called scanAll within its DynamoDbTemplate class. This method takes a single argument - the name of the DynamoDB table you want to scan. In our example, we'll use the Movie class, which represents our data model, to retrieve all items from the specified table.
@Test
void scanTable()
{
PageIterable<Movie> moviePageIterable = movieService.scanMovieTable();
if (Objects.nonNull(moviePageIterable))
{
SdkIterable<Movie> items = moviePageIterable.items();
if (Objects.nonNull(items))
items.forEach(System.out :: println);
}
}Targeting Your Data: Filtering DynamoDB Scans with Expressions
Filter expressions act as a powerful tool to streamline your scans by specifying conditions that retrieved items must meet. This significantly reduces the data volume returned, improving efficiency and focusing on relevant information.
public PageIterable<Movie> scanMovieTableWithFilterExpression(String movieTitleBeginWith) {
// Filter expression is nothing but, It's just like a where clause of SQL. Here we are only selecting whose
// movie_title begin with the given-movie-title(movieTitleBeginWith) like a LIKE operation of SQL.
Expression expression = Expression.builder()
.expression("begins_with(movie_title, :movie_title)")
.putExpressionValue(":movie_title", AttributeValue.fromS(movieTitleBeginWith))
.build();
ScanEnhancedRequest scanEnhancedRequest = ScanEnhancedRequest.builder()
.filterExpression(expression) // filter-expression
.build();
PageIterable<Movie> scan = dynamoDbTemplate.scan(scanEnhancedRequest, Movie.class);
return scan;
}DynamoDbTemplate offers a scan method that takes two arguments: a ScanEnhancedRequest object and the class representing your DynamoDB table (in this case, the Movie class).
This approach allows you to specify filtering criteria within the ScanEnhancedRequest for targeted data retrieval.
Expressions Explained: A Powerful Tool for Data Control
Expressions in DynamoDB act as a blueprint for interacting with your data. They can be used for filtering, sorting, and other operations
The expression method of the Expression class allows you to specify a filter expression as a string. This string can contain placeholders for the actual values you want to filter on. You then use the putExpressionValue method to define these placeholder values
Streamlining Scans: Retrieving Only Relevant Data with Projection Expressions
For those familiar with SQL, retrieving only specific columns from a table is common practice.
DynamoDB offers a similar functionality called projection expressions. This allows you to select the exact attributes you need during a scan operation, reducing data transfer and improving efficiency.
public PageIterable<Movie> scanMovieTableWithProjectionExpression() {
// ProjectionExpression, If we want to select only selected attributes from the dynamoDB then we can utilize
// the projection expression, Since we are using the Spring cloud AWS, we can utilize the NestedAttributeName
// to set the desire attributes name just like below.
List<NestedAttributeName> nestedAttributeNameList = new ArrayList<>();
// retrieve characters attribute in a result
nestedAttributeNameList.add(NestedAttributeName.builder()
.elements("characters")
.build());
// data type of attribute directors is a map, here we are only selecting key = dir2 from the directors attribute.
nestedAttributeNameList.add(NestedAttributeName.builder()
.elements("directors", "Dir2")
.build());
ScanEnhancedRequest scanEnhancedRequest = ScanEnhancedRequest.builder()
.addNestedAttributesToProject(nestedAttributeNameList) // projection-expression
.build();
PageIterable<Movie> scan = dynamoDbTemplate.scan(scanEnhancedRequest, Movie.class);
return scan;
}• NestedAttributeName: In DynamoDB, nested attribute names let you specify how to access data within nested structures. These names are built using a list of strings, where each string represents a level deeper in the nesting. For top-level attributes (those not nested), you can simply create a list with a single string element
In this code example, we're focusing on retrieving just two attributes from the DynamoDB table: characters and directors.
Since 'directors' is stored as a map (key-value pairs), we can be even more specific. We can use the projection expression to target only the value associated with a specific key within the 'directors' map. Here, we're interested in retrieving directors where the key is 'Dir2'.
Leveraging Queries for Efficient Data Retrieval in DynamoDB
DynamoDB's Query operation lets you pinpoint items using their primary key values
You must specify the partition key attribute's name as well as a single value. Query returns all objects with the specified partition key value. You may also specify a sort key attribute and use a comparison operator to filter the search results.
Querying DynamoDB with the Partition Key
public PageIterable<Movie> findMovieByQuery(Long movieId) {
Key key = Key.builder().partitionValue(movieId).build();
QueryConditional queryConditional = QueryConditional.keyEqualTo(key);
QueryEnhancedRequest queryEnhancedRequest = QueryEnhancedRequest.builder()
.queryConditional(queryConditional)
.build();
PageIterable<Movie> moviePageIterable = dynamoDbTemplate.query(queryEnhancedRequest, Movie.class);
return moviePageIterable;
}• QueryConditional : Defines the parameters that determine which items are returned when querying DynamoDB tables or indexes with the Query operation. These parameters specify the key conditions and any additional filtering criteria.
• QueryEnhancedRequest : Specifies conditions for finding items in DynamoDB tables or indexes with the Query operation.
@Test
public void findMovieByQueryUsingMovieId()
{
PageIterable<Movie> movieByQuery = movieService.findMovieByQuery(MOVIE_ID);
if (Objects.nonNull(movieByQuery))
{
SdkIterable<Movie> items = movieByQuery.items();
if (Objects.nonNull(items))
items.forEach(System.out::println);
}
}Querying DynamoDB with the Partition Key and Sort key
Just like with the partition key, you can define the sort key for your DynamoDB query by setting it in a Key object and passing that object to the QueryConditional class.
public PageIterable<Movie> findMovieByQuery(Long movieId, String movieTitle) {
Key key = Key.builder().partitionValue(movieId).sortValue(movieTitle).build();
QueryConditional queryConditional = QueryConditional.keyEqualTo(key);
QueryEnhancedRequest queryEnhancedRequest = QueryEnhancedRequest.builder()
.queryConditional(queryConditional).build();
PageIterable<Movie> moviePageIterable = dynamoDbTemplate.query(queryEnhancedRequest, Movie.class);
return moviePageIterable;
}DynamoDB Query Power: Partition Keys and Filter Expressions
While partition and sort keys are powerful for efficient queries, DynamoDB also allows you to search by other attributes when needed.
Many situations require querying DynamoDB based on attributes beyond the partition and sort keys. These queries leverage filter expressions, similar to how Scan operations utilize them. Filter expressions refine the results of a Query operation after the initial search using keys.
public PageIterable<Movie> findMovieByQuery(Long movieId, Double rating) {
Key key = Key.builder().partitionValue(movieId).build();
QueryConditional queryConditional = QueryConditional.keyEqualTo(key);
Expression expression = Expression.builder()
.expression("rating >= :rating_value")
.putExpressionValue(":rating_value", AttributeValue.fromN(rating.toString()))
.build();
QueryEnhancedRequest queryEnhancedRequest = QueryEnhancedRequest.builder()
.queryConditional(queryConditional) // query-condition with partition-key
.filterExpression(expression) // filter-expression
.build();
PageIterable<Movie> moviePageIterable = dynamoDbTemplate.query(queryEnhancedRequest, Movie.class);
return moviePageIterable;
}Streamlining Results: Optimizing Data Retrieval with Projection Expressions
For situations where you only need a subset of data, DynamoDB offers projection expressions. These allow you to fetch only the relevant attributes from the table, reducing data transfer and improving performance.
In the example below, we'll demonstrate how projection expressions work. We're only interested in retrieving two specific attributes: movie_id and movie_title.
Similarly, in the following example, I demonstrate how we may use both filter expression and projection expressions.
public PageIterable<Movie> findMovieTitle(String genreContain, Long movieId) {
// it's mandatory to use the key with the filter expression
Key key = Key.builder().partitionValue(movieId).build();
QueryConditional queryConditional = QueryConditional.keyEqualTo(key);
// filter expression, finding only those movie which genre's contain genreContain
Expression expression = Expression.builder()
.expression("contains(genre, :genre_contain_value)")
.putExpressionValue(":genre_contain_value", AttributeValue.fromS(genreContain))
.build();
// Selecting only movie_id and movie title attributes only.
List<NestedAttributeName> nestedAttributeNameList = new ArrayList<>();
nestedAttributeNameList.add(NestedAttributeName.builder()
.elements("movie_id")
.build());
nestedAttributeNameList.add(NestedAttributeName.builder()
.elements("movie_title")
.build());
QueryEnhancedRequest queryEnhancedRequest = QueryEnhancedRequest.builder()
.queryConditional(queryConditional)
.filterExpression(expression) // filter-expression
.addNestedAttributesToProject(nestedAttributeNameList) // projection-expression
.build();
PageIterable<Movie> moviePageIterable = dynamoDbTemplate.query(queryEnhancedRequest, Movie.class);
return moviePageIterable;
}Conclusion
Spring Cloud AWS provides a powerful and convenient way to integrate DynamoDB with your Spring Boot applications. By leveraging the Query and Scan operations, you can efficiently retrieve data based on specific criteria or scan the entire table as needed.
We've explored using partition and sort keys for targeted queries, filter expressions for narrowing down results based on additional attributes, and projection expressions for optimizing data retrieval.
With this knowledge, you can unlock the full potential of DynamoDB within your Spring Boot projects, ensuring efficient data access and streamlined application performance.
Optimize your DynamoDB interactions with BatchWriteItem and BatchGetItem, See how: (getButton) #text=(How to Use AWS DynamoDB BatchWriteItem and BatchGetItem in Java Spring Boot Applications) #icon=(link) #color=(#35a576)
The complete code for this project is available on my GitHub repository (getButton) #text=(GitHub) #icon=(share) #color=(#000000)
Keep learning and keep growing.
