How to separate your business logic with abstraction layers.
Following on from my previous post, Making Terraform Work a Bit Harder, another important factor in making any code scalable is to create abstraction layers in your code to reduce risks associated with change, and to just make life easier with layers that limits the scope of knowledge of implementation. This is just as important with Terraform as it is for any ‘regular’ programming language. There are any number of articles and posts on the importance of adding layers and keeping business logic isolated from the technology implementing it so I’m not going to go into the reasoning here.
Terraform only really recommends using Modules, Root Modules seem to be mentioned in passing and the actual arrangement of Modules in directories is recommended to have a relatively shallow Module tree.
The only mechanism available for setting up the abstraction layers is therefore going to be using Modules.
Business logic has a habit of creeping its tentacles into everything, to make this work, it needs a bit of discipline to make sure it stays where it is supposed to be because Terraform doesn’t really supply the tools to do it for you.
So first of all I’ll consider the bottom layer. I am going to use AWS again as this is the one I am most familiar with and this time I’m going to look at IAM policies and roles. This is a basic piece of TF to create an IAM role:
resource "aws_iam_role" "my_role" {
name = var.name
description = var.description
path = var.path
assume_role_policy = var.assume_role_policy
tags = {
Name = var.name
}
}output name {
value = aws_iam_role.my_role.name
}variable name {
type = string
}
variable path {
type = string
}
variable description {
type = string
}
variable assume_role_policy {
type = string
}
Everything is a variable so making this a module is easy, we can call the folder for this one ‘modules/resource_iam_role’ to keep things simple, and we’ll put this code in the main.tf file in that folder. All good so far.
We’re going to probably have a few roles that we need to create so we need this code to be as generic as possible. There are more attributes available for this resource but I’m ignoring them to try and keep it simple. Whilst it is important to make it generic, don’t add stuff in that you will never use.
IAM roles typically have one or more policies associated with them. This can be done using the policy attachment resource, so if I add that into the module, it might look something like this
resource "aws_iam_role" "my_role" {
name = var.name
description = var.description
path = var.path
assume_role_policy = var.assume_role_policy
tags = {
Name = var.name
}
}resource "aws_iam_role_policy_attachment" "policy" {
role = aws_iam_role.my_role.name
policy_arn = var.policy_arn
}output name {
value = aws_iam_role.my_role.name
}variable name {
type = string
}
variable path {
type = string
}
variable description {
type = string
}
variable assume_role_policy {
type = string
}variable policy_arn {
type = string
}
This terraform resource takes just two arguments, the policy you’re attaching, and the role you’re attaching it to. I’ve also added a variable so that the policy arn can be passed in to the module.
I mentioned before that one or more policies can be associated with the role, so ideally our module would take a list of policies rather than just one. So using a count loop to iterate over the list, we could modify the module like this to take a list of policies rather than just the one.
resource "aws_iam_role" "my_role" {
name = var.name
description = var.description
path = var.path
assume_role_policy = var.assume_role_policy
tags = {
Name = var.name
}
}resource "aws_iam_role_policy_attachment" "policies" {
count = var.policy_arns
role = aws_iam_role.my_role.name
policy_arn = var.policy_arns[count.index]
}output name {
value = aws_iam_role.my_role.name
}variable name {
type = string
}
variable path {
type = string
}
variable description {
type = string
}
variable assume_role_policy {
type = string
}variable policy_arns {
type = list(string)
}
This will loop over the policy_arns list and add each policy in the list to the role.
The Business Logic (BL) Module
I’m going to introduce the idea of a BL module now, these type of modules should sit between the resource module, and the root module.
Starting nice and simple, the main.tf file in our BL module might look something like this, calling the IAM role resource module we have already created.
module "my_role" {
source = "../resource_iam_role"
name = local.name
path = local.path
description = local.description
assume_role_policy = local.assume_role_policy
policy_arns = var.policies
}With an associated variables.tf that might look something like this
variable application {
type = string
}
variable owner {
type = string
}
variable environment {
type = string
}variable policies {
type = list(string)
}locals {
name = "${var.application}-${var.owner}-${var.environment}"
path = "/service-role/"
description = "EC2 role to do stuff"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": ["ecs.amazonaws.com", "ec2.amazonaws.com"]
},
"Effect": "Allow"
}
]
}
EOF
}
What I’ve done here is to assume there are some global names, like the application and the team or owner creating the items and an environment, Dev, Test, Prod or whatever.
The Name of the role is built joining these together, this might be the ‘ec2_role’ module so in this case, the assume role policy will always be the same so all we need to pass into this module are generic application, owner and environment settings, plus any policies we want.
Ideally we don’t want to have to pass in the policies either, so how could we fix that?
Well, to start with, we need a module to create policies. Something like this maybe?
resource "aws_iam_policy" "policy" {
name = var.name
path = var.path
description = var.description
policy = var.policy
}
output "arn" {
value = aws_iam_policy.policy.arn
}variable name {
type = string
}variable path {
type = string
}variable description {
type = string
}variable policy {
type = string
}
A nice touch here might be to separate the policy data from the terraform, maybe json files somewhere that can be changed without having to worry about any Terraform. Separating data from code is another good pattern to adhere to — I’m pretending I didn’t embed the assume role policy data in the code above.
So lets change the policy variable to one for a path to the json file, and add processing to read the file . . .
resource "aws_iam_policy" "policy" {
name = var.name
path = var.path
description = var.description
policy = local.policy
}
output "arn" {
value = aws_iam_policy.policy.arn
}variable name {
type = string
}variable path {
type = string
default = "/"
}variable description {
type = string
}variable policy_file {
type = string
}locals {
policy = file("${path.module}/${var.policy_file}")
}
So, we can have files stored with a relative file path to the location of this module, pass in that relative path and using the path.module variable, it’ll find it. This is separate to the path variable, this is an IAM attribute which I have set to a default value and which can be overridden if necessary.
The json file could just be some like this for the S3 policy . . .
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListAllMyBuckets"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListBucketByTags",
"s3:ListBucket",
"s3:GetObject"
],
"Resource": "arn:aws:s3:::*"
}
]
}Now, going back to the main.tf in the BL module we could create the policies for our role now as well . . .
module "ssh_policy" {
source = "../../modules/resource_iam_policy"
policy_file = "../../iam_docs/access_for_ssh.json"
description = "Policy to allow access to ssh access"
name = local.ssh_name
}module "s3_policy" {
source = "../../modules/resource_iam_policy"
policy_file = "../../iam_docs/access_to_s3.json"
description = "Policy to allow access to S3 bucket"
name = local.s3_name
}module "my_role" {
source = "../resource_iam_role"
name = local.role_name
path = local.role_path
description = local.description
assume_role_policy = local.assume_role_policy
policy_arns = local.policies
}
and the variables.tf file might now look like this
variable application {
type = string
}
variable owner {
type = string
}
variable environment {
type = string
}locals {
name = "${var.application}-${var.owner}-${var.environment}"
role_name = "${local.name}-role"
ssh_name = "${local.name}-ssh_policy"
s3_name = "${local.name}-s3_policy"
role_path = "/service-role/"
role_description = "EC2 role to do stuff"
policies = [module.ssh_policy.arn, module.s3_policy.arn]
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": ["ecs.amazonaws.com", "ec2.amazonaws.com"]
},
"Effect": "Allow"
}
]
}
EOF
}
In the locals block, the three ‘_name’ variables use a common base variable local.name, and the separate policies that get created in the main.tf file are pulled together into a list to pass to the role in the local.policies variable. Now all of the IAM specific detail is contained in the BL and resource modules. The benefit of that is now the only thing the BL module needs to be told are generic application, owner and environment settings, so the root module would just need this, no need for any IAM detail at all.
module "iams" {
source = "../modules/bl_iams"
application = var.application
owner = var.owner
environment = var.env
}We have separated the json policies from the Terraform, wrapped all of the IAM details in their own module but still kept that detail separated from the resource modules, and at the same time kept the root module clean. Not too shabby so far.
What else could we do?
Most organisations have mandatory policies that have to be attached to every role that is created, as well as the policies that are needed for the task. We could put these in the BL module, but then if we need to create other IAM BL modules, we’d have to put it in there as well. The best place for these is probably the resource module. That way the people writing the BL and root modules don’t need to think about them, and they’re all applied in a single place.
Going back to the Role Module then …
resource "aws_iam_role" "my_role" {
name = var.name
description = var.description
path = var.path
assume_role_policy = var.assume_role_policy
tags = {
Name = var.name
}
}resource "aws_iam_role_policy_attachment" "policies" {
count = var.policy_arns
role = aws_iam_role.my_role.name
policy_arn = var.policy_arns[count.index]
}resource "aws_iam_role_policy_attachment" "mandatory" {
count = length(local.mandatory_policies)
role = aws_iam_role.my_role.name
policy_arn = local.mandatory_policies[count.index]
}output name {
value = aws_iam_role.my_role.name
}variable name {
type = string
}
variable path {
type = string
}
variable description {
type = string
}
variable assume_role_policy {
type = string
}variable policy_arns {
type = list(string)
}locals {
account_id = data.aws_caller_identity.current.account_id
mandatory_policies = [
"arn:aws:iam::${local.account_id}:policy/global-policy-1",
"arn:aws:iam::${local.account_id}:policy/global-policy-2"
]
The mandatory policies could be merged with the policies passed in and then use just one policy attachment block. I like to keep them separate though, mainly because it just makes it easier to read and see what is going on.
One last piece then. Lets say there were sometimes sub environments used to distinguish between say different types of integration testing, but that the sub environment variable is not always populated. If it’s there, we want to add it to our resource name.
Revisiting the BL module variables.tf file, we could implement that doing something like this, using a ternary expression to test whether sub_env is populated and then change the env variable accordingly, adding sub_env in or not. Then use the local.env variable in the local.name variable
variable application {
type = string
}
variable owner {
type = string
}
variable environment {
type = string
}
variable sub_env {
type = string
}locals {
env = length(var.sub_env) > 0 ? "${var.environment}-${var.sub_env} : "${var.environment}
name = "${var.application}-${var.owner}-${local.env}"
role_name = "${local.name}-role"
ssh_name = "${local.name}-ssh_policy"
s3_name = "${local.name}-s3_policy"
role_path = "/service-role/"
role_description = "EC2 role to do stuff"
policies = [module.ssh_policy.arn, module.s3_policy.arn]
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": ["ecs.amazonaws.com", "ec2.amazonaws.com"]
},
"Effect": "Allow"
}
]
}
EOF
}
This is just a example of how a ternary might be used and whilst this example is a little bit contrived, it shows what could be done.
My original Making Terraform Work a Bit Harder post was mostly about leveraging the syntax capabilities that Terraform provides, this was more about organising the code, hopefully both of these posts will prove useful to those people looking to solve problems associated with Terraform code bloat.
