Home Page

Polymer Framework

Blog Series

Build a Simple Api with AWS

Build a Frontend Site with AWS

Terraform Documentation

Terraform Module Documentation

Introduction

Welcome to the Polymer Framework!

The Polymer Framework comprises a collection of pre-built frameworks designed to empower novice developers in utilizing Infrastructure as Code (IaC) and Continuous Integration/Continuous Deployment (CI/CD) to deploy resilient AWS resources. By introducing this framework, my aim is to cultivate a DevOps culture and promote a 'shift-left' approach, making these practices more accessible and integral from the start of development.

Infrastructure as Code: utilizes imperative code to automate the provisioning, configuration, and management of computing infrastructure components, promoting consistency and reproducibility.

Continuous Integration/Continuous Deployment: automating the process of integrating code changes (CI) and the subsequent deployment to production environments (CD), streamlining development workflows and ensuring rapid and reliable software delivery.

Target Audience

I customized the blog series to consider the viewpoint of a developer without any familiarity with DevOps concepts and tools. My focus is on a 'do first, think later' approach, where I hope that by showing the possibilities these technologies offer, I would spark interest and encourage further exploration into the technologies by developers.

Therefore, in these blogs, I will focus on guiding you to implement a simple POC first before explaining the technologies behind it.

Features

In order to achieve the objective, the framework uses these features

• Well-Documented Terraform Modules: Created extensively documented Terraform modules battle tested across numerous personal projects to ensure reliability and reproducibility.
• Tailored Examples: Provides complete examples covering a spectrum of use cases, from full-stack development to simple APIs, and supporting both serverless and containerized architectures. This allows developers to apply this framework to a plethora of projects.
• Automated Deployment Integration: Fussless integration of automated deployment by providing both GitHub Actions and GitLab CI/CD scripts. This ensures a deployment workflow can be built on both Git ecosystems.
• Use-Case Specific Blogs: Series of use-case specific blogs that offer step-by-step instructions to lower the learning curve for developers and foster faster adoption for projects.

Prerequisites

While you can still use the application code to build successful POC projects without setting up these prerequisites, I strongly suggest setting up the following prerequisites to harness the full potential of this framework.

1. An AWS Account:

This framework harnesses the power of AWS to deploy resilient and highly available projects. If you don't have an AWS account yet, create one on the official AWS website.

This framework is crafted with compatibility for the Free Tier in mind. Any instances where usage surpasses the free tier limits will be explicitly documented.

2. Terraform (>=1.5.0):

To run Terraform locally, ensure it is installed on your local machine. You can download the latest version here.

While by using CD, you do not need Terraform locally Terraform is executed on GitLab/GitHub runners, I recommend running terraform plan locally before commits as a practice to ensure the integrity of your infrastructure configuration.

Introduction to Technologies used

Amazon Web Services (AWS)

• AWS is a comprehensive and broadly adopted cloud platform, offering over 200 fully featured services including computing, storage, databases, machine learning, content delivery, from data centers globally.
• IMO, AWS prides themselves with excellent customer support and high stability in its global infrastructure compared to its counterparts, making it the first choice for many enterprise level development.

I understand if you feel apprehensive to the concept of AWS as it may feel alien to you, similar to how I've felt initially too. In my experience, overcoming this hurdle is the most defining moment of my journey as it opens a whole new paradigm of cloud computing. Whether you are a Developer or DevOps, you will surely benefit from it.

Terraform

• Terraform is an open-source infrastructure as code tool that enables users to define and provision infrastructure using a declarative configuration language (HCL).
• By utilizing declarative code to automate the provisioning, configuration, and management of computing infrastructure components, it promotes versioning, collaboration, and repeatability in infrastructure deployment.
• IMO, Terraform distinguishes itself from other IaC competitors by its wide adoption and by using regularly maintained provider plugins with good documentation in AWS, GCP, Azure etc.

resource "aws_instance" "instances" {
  count         = 10 # how many servers to provision
  ami           = data.aws_ami.ubuntu.id # what "template" to use for the server (ubuntu)
  instance_type = "t3.micro" # size of server
}

<aside> ⌨️ How to use Terraform terraform init initializes Terraform in a working directory by installing required plugins/modules and initializing local or remote state.

terraform plan generates an execution plan by comparing the current infrastructure state and the desired state defined in code, and outlines a list of actions to achieve the desired state.

terraform apply applies the changes to the infrastructure using the generated plan.

</aside>

Continuous Integration/Continuous Deployment (CI/CD)

• CI/CD is a set of practices aimed at automating the process of software delivery.
• Continuous Integration involves integrating frequently code changes into a shared repository and running automated build and tests to detect integration errors early.
• Continuous Deployment automates the deployment process, allowing software changes to be delivered to production environments quickly and reliably.
• This framework places a stronger emphasis on Continuous Deployment over Continuous Integration as CI involves more code-specific integration. My preferred approach to implementing CI/CD is through GitLab.

A minimal example of a GitLab Pipeline execution. This will run on every commit pushes.

Setup Instructions

AWS Account

1. Creating a user

By default, after creating an account, you will be signed in as the root user. It is an IAM best practice to not use root user for day-to-day interactions.

1. Create a user through the AWS Console. You can refer to the documentation here for detailed step-by-step instructions on how to do so.
2. Sign in to the AWS Console using the credentials of your new user here. Remember to sign out first. You may need to type in your

2. Generating and Configuring Access Key

An access key is require for programmatic access to your account e.g. to use Terraform to provision resources.

1. Create an Access Key under the user you created just now. You can refer to the documentation here for detailed step-by-step instructions on how to do so.

2. Download Access Key: Once the access key is generated, download the CSV file containing the access key ID and secret access key. Keep this file secure.

3. Enable Access Locally: To access your AWS account locally, you can either set up Environment Variables for temporary development or configure credential files (recommended) for longer term usages. An example use case is to access DynamoDB for local development, or execute Terraform locally.

   .aws/credentials file:

   [default]
   aws_access_key_id=AKIAIOSFODNN7EXAMPLE
   aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
   

   .aws/config file:

   [default]
   aws_region=ap-southeast-1 # Singapore region
   

   If you choose to use credential files, you can create the following files in your root folder to provide access globally. If this method does not work, you can set up Environment Variables if you require a temporary fast solution.

3. Using Singapore region

On your first login, AWS may use us-east-1 or ap-southeast-2 region by default. Remember to change to the Singapore region if you want to provision resources closer to Singapore users.

You can do so by selecting region on the top left navbar in the console, and choosing ap-southeast-1

Terraform State

4. Creating an S3 Bucket

Amazon Simple Storage Service (Amazon S3) is an object storage service offering industry-leading scalability, data availability, security, and performance. We can create an S3 bucket to store the Terraform state files.

1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/
2. Click on Create bucket and enter a unique name for the bucket name field. We can use the default configuration by not specifying any other fields. Bucket names must be unique even across region or account.

5. Terraform Configuration to use Remote State

To use a remote state, configure a backend block within the terraform.tf file. Remember to replace the values with information about the newly created bucket.

terraform {
  ... 
  ...

  backend "s3" {
    bucket               = <your bucket name>
    key                  = "<a unique name for you state file>.tfstate"
    workspace_key_prefix = "tf-state" # optional
    region               = "<region of your bucket>" # ap-southeast-1
  }
}

Terraform must store state about your managed infrastructure and configuration. This state is used by Terraform to map real world resources to your configuration.

There are two ways to store state in Terraform:

1. Local State stores state within a local terraform.tfstate in the project directory
2. Remote State stores state in a remote location e.g. an S3 Bucket which can store simple files

On terraform plan, Terraform updates the latest infrastructure state by pulling information about a resource using its unique id (e.g. arn) before generating an execution plan on how to achieve the desired state. If there is a loss in state e.g. due to a deletion of state file, there is a loss in metadata mapping an existing resource to a unique id, thus forcing a recreation of the resource that may result in a ResourceAlreadyExist exception.

Therefore, it is recommended to use a remote state to store Terraform state instead of having a local state prone to deletion. If Terraform is executed on GitLab/GitHub runners, it is necessary to use a remote state as the local directory is not preserved between runs.

Table of Contents