Setting up the AWS CDK with Java & Gradle

AWS CDK with Java and Gradle

Using the CDK with Java is not necessarily a natural choice for most projects - if you’re starting from scratch then I’d strongly suggest you use Typescript, but the previous post described why much of my CDK work is in Java (it’s around enabling my colleagues to contribute and maintain our codebase). This post is about how to set up a basic AWS CDK project with Java. We’re going to cover the Gradle build tool too, as the very essence of building for the cloud is in automation, and the first part of automation is being able to run every part of your build by script.

Pre-requisites

It’s assumed that you have the following already installed on your computer:

JRE or JDK, version 1.8 or better.
a Gradle distribution: you need to be able to run gradle command line tool.

Project structure

We’re going to use Gradle as our build tool - in my opinion, it’s a lot more user-friendly than Maven, plus it’s got a rich ecosystem of plugins that can add functionality easily. We’re going to create a multi-module Gradle project, where the first module is our CDK infrastructure.

1> mkdir blog
2> cd blog
3> gradle init

The init command will create a basic Gradle project which we can then modify.

1├── build.gradle  
2├── gradle
3│   └── wrapper
4│       ├── gradle-wrapper.jar  
5│       └── gradle-wrapper.properties  
6├── gradlew  
7├── gradlew.bat  
8└── settings.gradle

Gradle Wrapper

The Gradle Wrapper is a script which can be used to invoke a defined version of Gradle. If necessary, that version will be downloaded. It’s a neat way of ensuring that your build is repeatable and will always work on your colleagues’ computers, irrespective of which version of Gradle they have installed. The Gradle wrapper is the gradlew script in the root directory (or gradlew.bat if you’re developing on Windows).

First, let’s edit our build.gradle file to define which version of Gradle we will use:

1// build.gradle
2wrapper {
3    gradleVersion = "6.3"
4    distributionType = Wrapper.DistributionType.ALL
5}

Now run the wrapper task to install the version of Gradle we’ve just defined (the version that was previously there will defined by whichever version of Gradle you have installed on your computer).

1> ./gradlew wrapper
2> ./gradlew --version
3./gradlew --version
4
5------------------------------------------------------------
6Gradle 6.3
7------------------------------------------------------------
8
9Build time:   2020-03-24 19:52:07 UTC
10Revision:     bacd40b727b0130eeac8855ae3f9fd9a0b207c60
11
12Kotlin:       1.3.70
13Groovy:       2.5.10
14Ant:          Apache Ant(TM) version 1.10.7 compiled on September 1 2019
15JVM:          11.0.7 (Amazon.com Inc. 11.0.7+10-LTS)
16OS:           Mac OS X 10.15.3 x86_64

Let’s also set up our project to pull in dependencies as required from maven Central. Here’s our build.gradle file in full:

1plugins {
2  id 'java-library'
3}
4
5group 'org.johntipper'
6version = "0.1.0-SNAPSHOT"
7
8wrapper {
9    gradleVersion = "6.3"
10    distributionType = Wrapper.DistributionType.ALL
11}
12
13allprojects {
14    repositories {
15        mavenCentral()
16    }
17
18    group = rootProject.group
19    version = rootProject.version
20}

We define the group and version of the project and all sub-modules that we will define in the future.

Gradle sub-modules

The top level directory is just a container for sub-modules, we’re not going to build anything there. All our code will be in one of several sub-modules we will create; these are described in detail in the Gradle documentation.

1// settings.gradle
2rootProject.name = 'blog'
3
4include "infrastructure"

Now create a sub-module called infrastructure where we will define our CDK infrastructure:

1> mkdir infrastructure

We need to create another file called build.gradle within that newly-created directory where we will define how we will build that sub-module.

1├── ...  
2├── infrastructure
3│   └── build.gradle
4│   └── src
5│       └── main
6│           └── java
7└── ...

Here is the build.gradle file for our infrastructure project in full:

1// infrastructure/build.gradle
2plugins {
3    id 'java-library'
4    id 'application'
5    id 'com.github.johnrengelman.shadow' version '5.2.0'
6
7}
8
9def CDK_VERSION = "1.60.0"
10
11dependencies {
12    implementation "software.amazon.awscdk:core:${CDK_VERSION}"
13    implementation "software.amazon.awscdk:route53:${CDK_VERSION}"
14    implementation "software.amazon.awscdk:route53-targets:${CDK_VERSION}"
15    implementation "software.amazon.awscdk:route53-patterns:${CDK_VERSION}"
16    implementation "software.amazon.awscdk:ses:${CDK_VERSION}"
17    implementation "software.amazon.awscdk:certificatemanager:${CDK_VERSION}"
18    implementation "software.amazon.awscdk:s3:${CDK_VERSION}"
19    implementation "software.amazon.awscdk:s3-deployment:${CDK_VERSION}"
20    implementation "software.amazon.awscdk:cloudfront:${CDK_VERSION}"
21    implementation "software.amazon.awscdk:apigateway:${CDK_VERSION}"
22    implementation "software.amazon.awscdk:dynamodb:${CDK_VERSION}"
23    implementation "software.amazon.awscdk:events:${CDK_VERSION}"
24    implementation "software.amazon.awscdk:events-targets:${CDK_VERSION}"
25    implementation "software.amazon.awscdk:lambda:${CDK_VERSION}"
26    implementation "software.amazon.awscdk:lambda-event-sources:${CDK_VERSION}"
27
28    implementation 'commons-cli:commons-cli:1.4'
29    implementation 'org.slf4j:slf4j-log4j12:1.7.28'
30    implementation 'com.github.spullara.mustache.java:compiler:0.9.6'
31    implementation 'com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.10.4'
32
33    testCompile group: 'junit', name: 'junit', version: '4.12'
34}
35
36application {
37    mainClassName = 'org.johntipper.blog.aws.cdk.webapp.WebBackendApp'
38}

From the top, we are declaring this project to be a Java one via the java-library plugin, we wish to use the application plugin which will build an executable jar for us and we also use th shadow plugin which will ensure that our executable jar has all necessary dependencies bundled up into it.

We then define a whole bunch of CDK dependencies - you don’t need all of them now for this particular blog post, but you will do by the end. We define a couple of additional dependencies, whose purpose will become apparent shortly, plus include JUnit for unit-testing.

We define the class of our application that will be called when we execute our jar. Let’s create a do-nothing main class:

1package org.johntipper.blog.aws.cdk.webapp;
2
3public class WebBackendApp {
4    public static void main(String[] args) {
5        System.out.println("Hello world!");
6    }
7}

then build and execute it:

1>./gradlew shadowJar
2> ./infrastructure/build/libs/infrastructure-0.1.0-SNAPSHOT-all.jar
3Hello World!

We now have a working toolchain, so let’s introduce the CDK.

CDK Setup

Installation

Detailed installation instructions for the CDK are here.

1>npm install -g aws-cdk
2>cdk --version

AWS Credentials with multiple AWS accounts

UPDATE: ignore this section, as things have come a long way with cross-account deployments using CDK. Use cdk-assume-role-credential-plugin, details here, and see a later blog post by me for setting it up with CDK Pipelines.

We are going to interact with the CDK command line tool in two ways: firstly, to synthesize a CloudFormation template that describes the infrastructure that we wish to create and secondly to initiate the deployment. Those tasks are accomplished by means of the cdk synth and cdk deploy commands.

The deploy command obviously requires AWS credentials to work. Depending on your infrastructure, you may need AWS credentials for the synth command to work too: this will be the case if your code performs a lookup inside AWS, e.g. it determines which VPCs exist to deploy a resource into.

You should never deploy with your AWS root credentials - always create a user or role which has whatever permissions you require (and no more) and use AWS credentials for that user. If you lose control of tightly-scoped credentials it’s less of a security disaster than if you lose control of your account root credentials: someone malicious can run up a very large bill in your name in a short period of time otherwise.

If your credentials are in an AWS credentials file (e.g. ~/.aws/credentials) then you may have several roles that you wish to switch between. Each role will have a profile in the credentials file, where each profile will have separate credentials. You can tell CDK to use a particular profile’s credentials by means of the --profile <profile name> command line argument, if you don’t wish to use the default one.

It may be the case that you have several AWS accounts: AWS Organizations is a nice way of defining many accounts, all tied back to one root account. You can then deploy resources to one account without having to worry about wht else might be running in that account. Equally, you may wish to deploy to multiple AWS accounts at the same time, in which case CDK will need credentials for different accounts. You can’t specify a single AWS profile for CDK to use, as that means CDK will only have one set of credentials.

To solve this problem, we use a plugin mechanism for CDK. CDK will call our plugin when it runs and our plugin will be responsible for telling CDK which profile to use for any particular target account.

If you do not have multiple AWS accounts or do not wish to deploy to multiple accounts simultaneously, then skip to the next section.

Create the following plugin file anywhere on your computer and modify the logic as required to return the name of an appropriate profile:

1// cdk-profile-plugin.js
2const { CredentialProviderChain } = require('aws-sdk');
3const AWS = require('aws-sdk');
4
5let getEnv = function(accountId) {
6    // TODO: insert logic to get your desired profile name
7    console.log("getEnv called with acc id: " + accountId);
8    let profileName = '';
9    if (accountId == "111111111111") {
10        profileName "my-profile-for-account-1";
11    }
12    // further logic here
13
14    return profileName;
15};
16
17let getProvider = async (accountId, mode) => {
18
19    console.log("getProvider called with acc id: " + accountId);
20
21    let profile = getEnv(accountId);
22    let chain = new CredentialProviderChain([
23        new AWS.SharedIniFileCredentials({
24            profile: profile,
25        }),
26    ]);
27    let credentials = await chain.resolvePromise();
28    return Promise.resolve({
29        accessKeyId: credentials.accessKeyId,
30        secretAccessKey: credentials.secretAccessKey,
31        sessionToken: credentials.sessionToken,
32    });
33};
34
35module.exports = {
36    version: '1',
37    init: host => {
38        console.log('Init loading cdk profile plugin', host);
39        host.registerCredentialProviderSource({
40            name: 'cdk-profile-plugin',
41            canProvideCredentials(accountId) {
42                canProvide = true; // TODO: your logic to determine whether should use the code in this file or not (optional)
43                return Promise.resolve(canProvide);
44            },
45            getProvider,
46            isAvailable() {
47                return Promise.resolve(true);
48            },
49        });
50    },
51};

When you execute CDK, call it with the following flag:

1--plugin /path/to/cdk-profile-plugin.js

CDK will then call the plugin for each account being targeted for deployment and the plugin will return the name of the profile to use. Provided that your credentials file has credentials for that profile, CDK will then use those credentials for that profile to interact with AWS for those particular resources.

I can’t claim credit for this mechanism and I can’t remember where I first saw this as a solution, but the code above is heavily taken from Thomas de Ruiter’s Binx blog here. It’s a pretty neat solution, I think.

CDK synthesis

The cdk synth command will execute our code and output a CloudFormation template into a directory (strictly speaking, it’s called a CloudAssembly). There are some parameters that are important that we need to pass it. We will call CDK like this:

1> cdk synth --output cdk-system/build/cdk.out \
2      --app 'java -jar /path/to/our/fat-jar.jar <params>'

The --output <path> parameter determines where our CloudAssembly code will be written.

The --app <command> is the command CDK needs to execute in order to synthesize our CloudASsembly definition.

The main class defines a variable of type software.amazon.awscdk.core.App, which has a method called synth(), which is supposed to output our code. However, the assembly process requires JSII and so we’re forced to call cdk to call our app, rather than being able to execute our app directly. Frustrating, not-intuitive and it’s where the notion of coding in your own loanguage starts to break down.

If we want to pass parameters to our synthesis, then we have two options: we can either pass via --c var=val parameters to the cdk command, or we can pass parameters to our app directly. Personally, I find it easier to unit-test and provide better error messages to users if the parameters aren’t correct by the second method (I found the CDK client a bit wanting in this regard), so that’s what we’re going to do here. This is why our dependencies included Apache Commons CLI, as we’re going to treat input parameters as just another command line argument passed to a standard Java application.

CDK do-nothing

Let’s create a basic do-nothing app that is integrated with CDK.

First, let’s create a class which will hold some config that we want to pass to our app:

1public class WebBackendStackConfig {
2    public static final String API_LAMBDA_PATH_KEY = "apiLambdaPath";
3    public static final String TARGET_ACCOUNT_KEY = "targetAccount";
4    public static final String REGION_KEY = "region";
5    public static final String DOMAIN_NAME_KEY = "domainName";
6
7    private final String domainName;
8
9    private final String apiLambdaPath;
10
11    private final String targetAccount;
12
13    private final String region;
14
15
16    public WebBackendStackConfig(String domainName, String apiLambdaPath, String targetAccount, String region) {
17        this.domainName = domainName;
18        this.apiLambdaPath = apiLambdaPath;
19        this.targetAccount = targetAccount;
20        this.region = region;
21    }
22
23    public String getDomainName() {
24        return domainName;
25    }
26
27    public String getApiLambdaPath() {
28        return apiLambdaPath;
29    }
30
31    public String getTargetAccount() {
32        return targetAccount;
33    }
34
35    public String getRegion() {
36        return region;
37    }
38
39
40    public static WebBackendStackConfig fromCommandLine(CommandLine cmd) {
41        return new WebBackendStackConfig(cmd.getOptionValue(DOMAIN_NAME_KEY), cmd.getOptionValue(API_LAMBDA_PATH_KEY), cmd.getOptionValue(TARGET_ACCOUNT_KEY),cmd.getOptionValue(REGION_KEY));
42    }
43}

Now let’s define our main app:

1public class WebBackendApp {
2
3    private static final Logger LOG = LoggerFactory.getLogger(WebBackendApp.class);
4
5    final App app;
6
7    public WebBackendApp(CommandLine cmd) throws IOException {
8
9        this.app = new App();
10
11        WebBackendStackConfig webStackConfig = WebBackendStackConfig.fromCommandLine(cmd);
12
13        WebBackendStack webBackendStack = new WebBackendStack(
14                                              app,
15                                              "WebBackendStack",
16                                              StackProps.builder()
17                                                        .env(Environment.builder()
18                                                                        .account(webStackConfig.getTargetAccount())
19                                                                        .region(webStackConfig.getRegion())
20                                                                        .build())
21                                                        .stackName("WebBackendStack")
22                                                        .tags(Map.of("cdk", Boolean.toString(true)))
23                                                        .build(),
24                                              webStackConfig);
25
26    }
27
28    public void synth() {
29
30        app.synth();
31    }
32
33    public static void main(String[] args) {
34        Options options = new Options();
35
36        options.addOption(Option.builder(REGION_KEY)
37                                .argName(REGION_KEY)
38                                .desc("AWS region of the deployment.")
39                                .hasArg()
40                                .required(true)
41                                .build());
42
43        options.addOption(Option.builder(TARGET_ACCOUNT_KEY)
44                                .argName(TARGET_ACCOUNT_KEY)
45                                .desc("AWS target account.")
46                                .hasArg()
47                                .required(true)
48                                .build());
49
50        options.addOption(Option.builder(API_LAMBDA_PATH_KEY)
51                                .argName(API_LAMBDA_PATH_KEY)
52                                .desc("Path to the cognito lambda bundle.")
53                                .hasArg()
54                                .required(true)
55                                .build());
56
57        options.addOption(Option.builder(DOMAIN_NAME_KEY)
58                                .argName(DOMAIN_NAME_KEY)
59                                .desc("Domain name of the website.")
60                                .hasArg()
61                                .required(true)
62                                .build());
63
64        CommandLineParser parser = new DefaultParser();
65
66        try {
67
68            CommandLine cmd = parser.parse(options, args);
69
70            WebBackendApp cdkApp = new WebBackendApp(cmd);
71
72            cdkApp.synth();
73
74        } catch (MissingArgumentException | MissingOptionException | UnrecognizedOptionException e) {
75            System.err.println(e.getMessage());
76            HelpFormatter formatter = new HelpFormatter();
77
78            formatter.printHelp(120, "java -cp /path/to/jar org.johntipper.blog.aws.cdk.webapp.WebBackendApp ARGS", "Args:", options, "", false);
79
80            System.exit(1);
81
82        } catch (Exception e) {
83
84            LOG.error("Error when attempting to synthesize CDK: {}", e.getMessage());
85
86            e.printStackTrace();
87
88            System.exit(-1);
89        }
90    }
91}

Basically, we define a CDK stack which can be configured with a target account and target AWS region and the act of running our main() means we parse input parameters and error if they are incorrect, otherwise we call for the synth() of our stack. Ignore the apiLambdaPath and domainName parameters for the moment, we’ll explain those later.

Finally, let’s create our do-nothing CDK stack, which creates no resources:

1public class WebBackendStack extends Stack {
2    public WebBackendStack(Construct scope, String id, StackProps props, WebBackendStackConfig stackConfig) throws IOException {
3        super(scope, id, props);
4    }
5
6}

We can test this:

1> ./gradlew shadowJar
2> cdk synth --output infrastructure/build/cdk.out \
3      --app 'java -jar infrastructure/libs/infrastructure-0.1.0-SNAPSHOT-all.jar -apiLambdaPath /any/path/here -targetAccount <your AWS account number> -region <AWS target region> -domainName example.com'

Summary

All non-trivial projects have a certain amount of scaffolding that is required to get them up and running, but we’ve successfully built an AWS CDK application that uses Java, is built by Gradle (so is capable of automation) and which can be integrated into an arbitrarily complex AWS environment involving simultaneous deployment to multiple AWS accounts. The next post will be about how we define the infrastructure to support a static website by means of CDK.