How to set up an AWS environment

This is part 1 of a series.

Table of contents

1. Background
2. Overview
3. Account management
4. User management
5. Account policy management
6. Review
7. Next steps

Background

Setting up a well-designed, scalable, secure AWS environment is difficult. Making it more challenging, most people have inherited large AWS environments in various states, typically ranging from a single AWS account for dev and prod to hundreds of AWS accounts, with a multitude of IAM users, and varying degrees of success with different tagging strategies.

In this series of how-to posts, ClearVector outlines our approach for designing an AWS environment with the following desired outcomes:

1. The developer experience is excellent
   1. Fast onboarding of new developers
   2. Quick feedback loops for dev/test
   3. Dedicated, not shared, environments
   4. Easy to ship
2. Security is designed up front, to ease instrumentation, risk mitigation, and future improvements
3. Simplicity and understanding of how things work under the hood is important
4. Manageability is feasible long term with reasonable effort, enables a foundation to build on as growth occurs, and enables stakeholders such as finance and security with the right data

There are many existing options, including AWS native choices. We've outlined five below:

1. AWS "native"
2. AWS Landing Zone
3. AWS Control Tower
4. AWS Secure Environment Accelerator
5. Commercial solutions

Overview

We reviewed the five solutions, incorporated many of the best practices, and decided on #1. Future posts may dive into the details of each of the other options. Figure 1 shows a high-level diagram of the solution.

Before diving into specifics, Figure 1 shows that we have:

1. One AWS organization
2. Six AWS accounts covering management, logs, security services, build (CI/CD), staging, and production
3. One AWS account for each developer
4. Four organizational units (OUs) covering security, development, build, and production

You might think, "Wow! That is a lot to manage," and we agree. However, prior to starting this journey, there are a few things that make scaling this manageable.

1. Plus addressing
2. AWS SSO for all user accounts, groups, and permission sets
3. OUs and SCPs to manage policies for AWS accounts

We'll examine each of these items in more detail.

Account Management

One challenge with using multiple AWS accounts is the overhead involved in managing communication for each AWS account. Further, AWS requires a unique email address for each AWS account.  Plus addressing or subaddressing as it’s sometimes known, (you may need to turn this on if you are on O365) makes this easy to manage.

Plus addressing enables you to add a tag within the local-part of an email address, for example: [email protected], where ‘awsblog’ is the tag. This enables you to have a single shared mailbox for all AWS accounts, such as ‘aws@’. Then, for each of the six root user accounts, add an account-specific tag, such as ‘mgmt’, ‘security’, ‘prod-cicd’, and ‘prod’. For each development account, choose a user-specific identifier, such as first name and ‘dev’, to make a tag such as ‘alice-dev’.

Table 1 shows a summary of the AWS account setup using plus addressing.

Next, we'll look at using AWS SSO.

User Management

AWS SSO enables each user to have a single set of credentials to access the environment and switch between accounts using the right set of permissions. This environment further separates admin and dev user accounts, and uses three SSO groups: “Administrators”, “Developers”, and “Production-ReadOnly".

Table 2 shows a summary of the AWS user account and group setup, excluding the root users.

Now that we have user accounts and groups, we'll examine permissions and policies.

Table 3 shows three permission sets, the use of AWS managed policies, and the use of inline policies.

The “AdministratorAccess” and “ReadOnlyAccess” permission sets map directly to the corresponding AWS managed policies.

The Development permission set enables users in this group to do most things except manage IAM users. This helps to reduce the risk of accidentally creating roles, users, or access keys that could become compromised.

For this development environment, we make heavy use of the AWS CDK, which requires developers to create and manage specific roles as part of the CDK bootstrapping process. We allow this by name, through a custom policy, as shown in Figure 2.

{
    "Version": "2012-10-17",
    "Statement": [
    	{
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "iam:CreateRole",
                "iam:DeleteRole",
                "iam:AttachRolePolicy",
                "iam:DetachRolePolicy",
                "iam:GetRolePolicy",
                "iam:PutRolePolicy",
                "iam:DeleteRolePolicy"
            ],
            "Resource": "arn:aws:iam::*:role/cdk-*"
        }
    ]
}

Note that these permission sets are meant to prevent accidental mistakes and help control drift, not prevent malicious use of the development environments.

Next, we’ll examine using policies for AWS accounts.

Account Policy Management

Using OUs and SCPs to manage policies for accounts enables easy and scalable control of environments. For this environment, Table 1 above shows four OUs, “Security”, “Development”, “Prod CI/CD”, and “Production”, and uses an SCP to restrict usable regions. Figure 3 shows an SCP that restricts to the us-east-1 region.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Deny",
            "NotAction": [
                "organizations:*"
            ],
            "Resource": [
                "*"
            ],
            "Condition": {
                "StringNotEqualsIgnoreCase": {
                    "aws:RequestedRegion": [
                        "us-east-1"
                    ]
                }
            }
        }
    ]
}

Now, let's review how this environment supports our outcomes from the first section.

Review

The developer experience is excellent

In this model, developers are provisioned an entire AWS account, can quickly deploy the entire product within their account, and rapidly commit to GitHub and deploy code to staging or production. This enables a rapid dev/test cycle, without any shared or locally mocked resources.

Security is designed up front to ease instrumentation, risk mitigation, and future improvements

From the beginning, we’ve incorporated security in a way that scales and can expand to new requirements over time. Each AWS account has a specific reason for existing, clear accountability, and a base set of policies, within an AWS organization. Users can access multiple AWS accounts using the same set of credentials via AWS SSO. Security services are separated with limited access, and we’ve separated dev and production, along with separating out our build infrastructure.

Simplicity and understanding of how things work under the hood is important

Everything in the environment uses well-supported, well-documented, and tested base AWS services – which enables future adoption of new AWS or non-AWS security controls or services in a timely manner. The ability to reason about the environment and explain to auditors, new developers, or other stakeholders how everything works is as close to the metal as you can be in the cloud-native world.

Manageability is feasible long term with reasonable effort, enables a foundation to build on as growth occurs, and enables stakeholders such as finance and security with the right data

This environment minimizes overhead for small teams and enables scaling – with robust requirements – as teams grow. The environment’s design also enables finance teams in non-production environments to quickly understand where costs might be optimized.

Next Steps

In this post, we examined how to build a foundational AWS environment that enables a great developer experience, builds a secure foundation, is easy to understand and reason about, and is manageable for growth. In future blogposts, we’ll dive into more specifics about services inside each account, the development environment, where to go from here, and how you can use ClearVector to understand and mitigate risk in your environment.

Have feedback? Contact us on Twitter or here. Can’t wait? Have a simpler environment? Have a more complex environment? Talk with us!