Introduction

AWS Lambda is the most popular and easy-to-use serverless service offered by AWS. It allows developers to run code without the hassle of managing compute instances. With seamless integration with other AWS services and support for multiple runtime environments, Lambda is favored for its simplicity, scalability, and efficiency in building modern applications.

On the other hand, Terraform is a game-changer in the world of infrastructure as code (IaC). It allows us to defining infrastructures using simple directives, enabling us to provision, manage, and update cloud resources with ease. With its declarative syntax, Terraform automates the deployment process, ensures consistency across environments, and provides visibility into your infrastructure changes.

In this article, we will see how we can create Serverless Scheduled Lambda Functions (Cron Jobs), using Terraform.

Prerequesites

In this guide, we will be using the aws-cli, terraform and python 3.10. Make sure you have the required binaries by running:

# aws-cli version
aws --version

# terraform version
terraform -version

# python version
python -V

In case you don't have the required binaries configured, checkout the following guides to set them up.

Project Setup

The project we'll be working with is structured in the following way:

/
.terraform  /
src /
    | lambda  /
              | main.mjs  <- Lambda function file
    | glue
    | redshift
main.tf
README.md

I personally like to keep the terraform manifest on the project root, with the all the source code in the src directory, which is organized by service. In the example above we can see Lambda, Glue and Redshift. The file containing the lambda function is main.mjs, and the terraform configuration will be in main.tf.

For the lambda function, we will be working with a basic NodeJS example:

export const handler = async (event) => {
    // TODO implement
    const response = {
      statusCode: 200,
      body: JSON.stringify('Hello from Lambda!'),
    };
    return response;
};

Feel free to implement your specific logic inside the lambda function.

Terraform Configuration

IAM Roles and Policies

In order to create any resource on AWS, it's important to determine to access level that we want to attach to it and the services that it can and cannot invoke. Once that's clear, creating the right Polices and Role is the first step in our terraform manifest files, without that, our resources can't execute properly.

Let's start by creating a Role (tf_lambda_role) for our Lambda Function, and allow the Lambda Service as well as EventBridge to Assume it.

resource "aws_iam_role" "tf_lambda_role" {
  name = "tf_lambda_role"
  assume_role_policy = <<EOF
{qawdf 
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": [
          "lambda.amazonaws.com",
          "events.amazonaws.com",
          "scheduler.amazonaws.com"
        ]
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
EOF
}

Then we need to create the policie(s) that we want to attach to the Role, and then attach them. Policies are the smallest unit in AWS IAM, in our case for our example, we want our Lambda function to be able to write Logs and LogEvents to CloudWatch, which we will need for monitoring. To create such a policy, we do the following.

resource "aws_iam_policy" "tf_policy" {
  name = "tf_policy"
  path = "/"
  description = "TF lambda policy"
  policy = <<EOF
{
	"Statement": [
		{
			"Action": [
				"logs:CreateLogGroup",
				"logs:CreateLogStream",
				"logs:PutLogEvents",
				"lambda:*"
			],
			"Effect": "Allow",
			"Resource": [
				"arn:aws:logs:*:*:*",
				"*"
			]
		}
	],
	"Version": "2012-10-17"
}
EOF
}

After that, we need to attach the Policy to the Role using an aws_iam_role_policy_attachment resource. To do so, we create the following resource in our Terraform manifest.

resource "aws_iam_role_policy_attachment" "tf_role_policy_attachement" {
  role = aws_iam_role.tf_lambda_role.name
  policy_arn = aws_iam_policy.tf_policy.arn
}

Now that our Role is ready, we can move to creating our Lambda fucntion.

The Lambda Function

As mentioned in the introduction, our the lambda function, we will be working with a basic NodeJS example:

export const handler = async (event) => {
    // TODO implement
    const response = {
      statusCode: 200,
      body: JSON.stringify('Hello from Lambda!'),
    };
    return response;
};

In order to deploy the new function to AWS, a Lambda function has to be zipped, then uploaded. Terraform offers an Archiver Data directive. We define it as follows:

data "archive_file" "zip_js_file" {
  type = "zip"
  source_dir = "${path.module}/src/lambda/hello/"
  output_path = "${path.module}/src/lambda/hello.zip"
}

Then we define the function itself using the zip file created.

resource "aws_lambda_function" "tf_function" {
  filename = "${path.module}/src/lambda/hello.zip"
  function_name = "tf_function"
  role = aws_iam_role.tf_lambda_role.arn
  handler = "hello.handler"
  runtime = "nodejs20.x"
  depends_on = [ aws_iam_role_policy_attachment.tf_role_policy_attachement ]
}

No that our function is created, we can move to implementing the EventBridge Scheduler.

EventBridge Scheduler

To implement Scheduled Service-Level Events, AWS offers EventBridge Scheduling. The service accepts either Cron syntaxes, or Rate syntaxes (every 1 hour/minute/etc..). To implement a Scheduler using Terraform, we use the following directive.

resource "aws_scheduler_schedule" "lambda_schedule" {
  name = "tf_scheduler"
  group_name = "default"

  flexible_time_window {
    mode = "OFF"
  }

  schedule_expression = "rate(1 hours)"

  target {
    arn = aws_lambda_function.tf_function.arn
    role_arn = aws_iam_role.tf_lambda_role.arn
    input = jsonencode({
      MessageBody = "Triggering Lambda..."
    })
  }
}

In this example, we are running the scheduler every 1 hour, and the target in our Lambda Function. In case you need to pass parameters to your targer, the trigger event accept a payload (input) which you can customize.

As you can see, another required parameter is flexible_time_window. This parameter adds a random duration shorter than flexible_time_window after the scheduler trigger time.

Conculsion

In this article we took a look to how to implement scheduled Lambda Functions using Terraform and AWS EventBridge. It's the recommended way for Service Level scheduling, however, in case of code level scheduling, services such as Airflow offer intensive scheduling features that are worth taking a look at.