In this blog post, we'll delve into the world of Spring Boot and AWS S3 bucket management. We'll explore how to seamlessly integrate AWS S3 into your Spring Boot applications using Spring Cloud AWS, a powerful framework that simplifies the process of interacting with AWS services.
By the end of this tutorial, you'll have a solid understanding of how to list, create, and delete S3 buckets using Spring Cloud AWS, enabling you to efficiently manage your cloud storage needs.
The complete code for this project is available on my GitHub repository (getButton) #text=(GitHub) #icon=(share) #color=(#000000)
Dependencies
To get started with this tutorial, you'll need to add the following dependencies to your Spring Boot project's. This dependency provides the necessary components for interacting with AWS services, including S3.
Gradle
ext {
springCloudAwsVersion = '3.0.0'
}
dependencies {
// spring cloud aws BOM(Bill of materials)
implementation platform("io.awspring.cloud:spring-cloud-aws-dependencies:${springCloudAwsVersion}")
implementation 'io.awspring.cloud:spring-cloud-aws-starter-s3'
// AWS launched a high level file transfer utility, called Transfer Manager and a CRT based S3 client.
// The starter automatically configures and registers a software.amazon.awssdk.transfer.s3.S3TransferManager bean if the following dependency is added to the project:
implementation 'software.amazon.awssdk:s3-transfer-manager'
//Transfer Manager works the best with CRT S3 Client. To auto-configure CRT based S3AsyncClient add following dependency to your project:
implementation 'software.amazon.awssdk.crt:aws-crt'
}Maven
<dependencies>
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-starter-s3</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3-transfer-manager</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk.crt</groupId>
<artifactId>aws-crt</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-dependencies</artifactId>
<version>{project-version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>Configuration and Credentials
Before diving into bucket operations, let's configure our Spring Boot application to interact with AWS S3.
Spring Cloud AWS offers auto-configuration for S3Client, S3TransferManager, and S3Template, making setup a breeze. Here's what you need to add to your application.properties file:
# s3 Configuration
spring.cloud.aws.credentials.access-key=
spring.cloud.aws.credentials.secret-key=
# Configures endpoint used by S3Client, I'm woorking in the Asia Pacific (Mumbai) hence, I've configure ap-south1 region and endpoint
spring.cloud.aws.s3.endpoint=https://s3.ap-south-1.amazonaws.com
spring.cloud.aws.region.static=ap-south-1IAM Permissions for Bucket Management
You must apply this policy to your IAM user or role to enable listing, creation, and deletion of buckets.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListCreateDeleteBucket",
"Effect": "Allow",
"Action": [
"s3:DeleteBucketWebsite",
"s3:CreateBucket",
"s3:ListBucket",
"s3:DeleteBucket",
"s3:GetObject",
"s3:DeleteObjectVersion",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::*"
}
]
}Creating an S3 Bucket with S3Client
S3Client is a Java interface that interacts with Amazon Simple Storage Service (S3). An S3 bucket is like a virtual folder where you can store any type of file. Buckets have unique names globally. Let's create your first bucket on Amazon S3.
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.*;
public void createBucket(String bucketName) {
try {
CreateBucketRequest createBucketRequest = CreateBucketRequest.builder()
.bucket(bucketName)
.build();
CreateBucketResponse createBucketResponse = s3Client.createBucket(createBucketRequest);
boolean successful = createBucketResponse.sdkHttpResponse()
.isSuccessful();
if (successful) {
String location = createBucketResponse.location();
System.out.println("Bucket created successfully at : " + location);
} else System.out.println("Unable to create bucket");
} catch (S3Exception s3Exception) {
s3Exception.printStackTrace();
}
}Listing All Buckets Using S3Client
Using the S3Client, you can retrieve a list of all buckets in your configured region. The listBuckets() method provides this functionality.
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.*;
public void listBucketObjects() {
try {
ListBucketsResponse listBucketsResponse = s3Client.listBuckets();
listBucketsResponse.buckets()
.stream()
.forEach(bucket -> System.out.println("bucket : " + bucket.name()));
} catch (S3Exception s3Exception) {
s3Exception.printStackTrace();
}
}Deleting Empty Buckets with S3Client
To delete a bucket on Amazon S3, use the deleteBucket() method of the S3Client. However, the bucket must be empty before deletion; otherwise, the AWS SDK will throw an exception
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.*;
public boolean deleteEmptyBucket(String bucketName) {
try {
DeleteBucketRequest deleteBucketRequest = DeleteBucketRequest.builder()
.bucket(bucketName)
.build();
DeleteBucketResponse deleteBucketResponse = s3Client.deleteBucket(deleteBucketRequest);
boolean successful = deleteBucketResponse.sdkHttpResponse()
.isSuccessful();
System.out.println("is bucket deleted : " + successful);
return successful;
} catch (S3Exception s3Exception) {
s3Exception.printStackTrace();
return false;
}
}When deleting an S3 bucket using the AWS SDK for Java, a software.amazon.awssdk.services.s3.model.S3Exception is thrown if the bucket is not empty. This exception signifies a validation error and prevents accidental deletion of buckets with content.(alert-error)Deleting Unversioned, Non-Empty S3 Buckets with S3Client
If you have a non-empty bucket without versioning enabled, you'll need to delete its contents before deleting the bucket itself. To do this, list all the objects in the bucket and then delete them individually.
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.*;
public boolean deleteUnversionedBucket(String bucketName) {
try {
// Before you can delete an Amazon S3 bucket, you must ensure that the bucket is empty or an error will result.
// If you have a versioned bucket, you must also delete any versioned objects associated with the bucket.
ListObjectsRequest listObjectsRequest = ListObjectsRequest.builder()
.bucket(bucketName)
.build();
ListObjectsResponse listObjectsResponse = s3Client.listObjects(listObjectsRequest);
listObjectsResponse.contents()
.stream()
// create a DeleteObjectRequest
.map(s3Object -> DeleteObjectRequest.builder()
.bucket(bucketName)
.key(s3Object.key())
.build())
// delete objects
.forEach(deleteObjectRequest -> s3Client.deleteObject(deleteObjectRequest));
// once all the objects in a bucket deleted successfully, we can delete empty bucket.
return deleteEmptyBucket(bucketName);
} catch (S3Exception s3Exception) {
s3Exception.printStackTrace();
return false;
}
}Deleting a Versioned Non-Empty S3 Bucket with S3Client
If versioning is enabled for your bucket, you must delete all versioned objects individually before deleting the bucket itself. Attempting to delete a versioned bucket directly will result in an S3Exception.
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.*;
public boolean deleteVersionedBucket(String bucketName) {
try {
// Before you can delete an Amazon S3 bucket, you must ensure that the bucket is empty or an error will result.
// If you have a versioned bucket, you must also delete any versioned objects associated with the bucket.
ListObjectVersionsRequest listObjectVersionsRequest = ListObjectVersionsRequest.builder()
.bucket(bucketName)
.build();
ListObjectVersionsResponse listObjectVersionsResponse = s3Client.listObjectVersions(listObjectVersionsRequest);
listObjectVersionsResponse.versions()
.stream()
.peek(objectVersion -> System.out.println("versionid: " + objectVersion.versionId() +
" key: " + objectVersion.key()))
// create a DeleteObjectRequest
.map(objectVersion -> DeleteObjectRequest.builder()
.bucket(bucketName)
.key(objectVersion.key())
.versionId(objectVersion.versionId())
.build())
// delete objets one by one
.forEach(deleteObjectRequest -> s3Client.deleteObject(deleteObjectRequest));
// once all the objects in a bucket deleted successfully, we can delete empty bucket.
return deleteEmptyBucket(bucketName);
} catch (S3Exception s3Exception) {
s3Exception.printStackTrace();
return false;
}
}The AWS SDK for Java throws a software.amazon.awssdk.services.s3.model.S3Exception when deleting a bucket that contains versioned objects. This exception highlights the requirement to delete all versions within the bucket before deletion can be successful.(alert-error)Removing S3 Website Bucket Configuration Using S3Client
Amazon S3 can serve as a static website host. While you cannot directly delete a bucket hosting a static website, you can remove its static website configuration.
If the specified bucket doesn't exist, Amazon S3 returns a 404 error code. Upon successfully deleting the bucket's static website configuration, it returns a 200 success code.
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.*;
public boolean deleteBucketWebsite(String bucketName) {
try {
DeleteBucketWebsiteRequest deleteBucketWebsiteRequest = DeleteBucketWebsiteRequest.builder()
.bucket(bucketName)
.build();
DeleteBucketWebsiteResponse deleteBucketWebsiteResponse = s3Client.deleteBucketWebsite(deleteBucketWebsiteRequest);
boolean successful = deleteBucketWebsiteResponse.sdkHttpResponse()
.isSuccessful();
System.out.println("Is S3 Website Bucket Deleted : " + successful);
return successful;
} catch (S3Exception s3Exception) {
s3Exception.printStackTrace();
return false;
}
}Conclusion
This tutorial has provided a foundational understanding of Amazon S3 bucket operations using the Spring Cloud AWS library. We've explored essential tasks like creating buckets, listing their contents, and effectively deleting them, both with and without versioning enabled.
Additionally, we've delved into the process of removing static website configurations from buckets and have become familiar with common S3Exception scenarios that may arise during development.
Want to know how to efficiently store your files in the cloud? Explore our guide on (getButton) #text=(uploading files to Amazon S3) #icon=(link) #color=(#35a576)
The complete code for this project is available on my GitHub repository (getButton) #text=(GitHub) #icon=(share) #color=(#000000)
Keep learning and keep growing.
