Terraform

tags : Infrastructure, Ansible

FAQ §

Ansible vs Terraform §

Ansible can accomplish the same type of infrastructure provisioning as Terraform. Usually these can co-exist as they serve different purposes.

• Ansible
  • Is an imperative script execution engine that will run tasks in the order listed in the code
• Terraform
  • Is a declarative execution engine that will parallelize tasks that don’t directly depend on each other. (So can be faster)
  • graph-based nature also makes it possible to destroy infrastructure with the exact same code you used to create it. (Useful for environments such as staging/dev etc. which need infra re-provisioningg)

TF module? §

• It’s just a set of files

backend vs provider §

• backend: where the state is
  • for s3 that’s dynamodb table and s3 bucket
  • DynamoDB: dynamodb for the lock table so that 2 entities cannot do a terraform apply at the same time.
• provider: where the resources will be created

Auth for backend vs Auth for provider §

Auth for both are done separately

• Your backend (s3+dynamo) can live in another separate (acc,region) and the provider(resources to be provisioned) can be elsewhere
• The configuration blocks for both backend and aws provider allow for setting custom aws profile, region etc.

Reading the plan §

• + create
• - destroy
• -/+ replace (destroy and then create, or vice-versa if create-before-destroy is used)
• ~ update in-place i.e. change without destroying
• <= read

File names? File name convention? §

• In Terraform, filenames do not matter as long as they have the .tf extension. Terraform automatically loads all files in the current directory with the .tf extension when you run commands like terraform init, terraform plan, or terraform apply. The purpose of breaking your configuration into multiple files is mainly for organization and readability.
• Some conventions (for each tf module)
  • main.tf: Contains the main set of configurations for the project. (what’s the main set? it depends on what you consider the primary components to be)
  • provider.tf: Terraform relies on plugins called “providers” to interact with remote systems, we put those here. In many cases you’ll see these directly in the main.tf aswell.
  • backend.tf: Each Terraform configuration can specify a backend, which defines where the state file of the current infrastructure will be stored. This state can be stored locally or remotely. Configuring a remote backend allows multiple people to work on the same infrastructure.
  • variables.tf: Contain the inputs the tf config. Any variables that are not given a default value will become required argument to the config.
  • <file_name>.tfvars: Allows you to set the actual value of the variables. This essentially specifies the “content” of the inputs that we define in variables.tf. There are other ways to specify this other than this file.

AWS §

IAM Roles in Terraform (AWS) §

See AWS

• For carrying out Terraform operations, we can use any AWS IAM user which has sufficient permissions to carry out the changes that terraform apply would make. The aws region is set per environment tf module, so we don’t need to manually switch regions either.
• In the file tf/environments/bootstrap/main.tf we have the resource which creates the tf_role, ideally we’d want to do all our TF deployment/provisioning using this role and keep granting it more permissions as required etc.
• If we use terraform from CI/CD like github action, tf_role can be combined with oidc<>aws
• But in non-ideal, it’s okay to deploy with IAM user token directly. We’re using remote state so it’ll be locked and we don’t have too many people making changes to the infra at the moment so running terrform code from your local machine is safe.

Auth for backend vs Auth for provider (for AWS) §

Auth for both are done separately, it’s valid both ways (i.e they can be the same or they can be different)

• Your backend (s3+dynamo) can live in another separate (acc,region) and the provider(resources to be provisioned) can be elsewhere. i.e Your TF state file can live in one region but it can contain the state of resources in another region. In other words, the TF state file will have data of your entire infra across regions and accounts and that state file can exist in any acc/region that the IAM user operating terraform has access to.
• The configuration blocks for both backend and aws provider allow for setting custom aws profile, region etc.

What happens when §

• Does terraform apply affect existing resources in the system/provider?
  • No. terraform apply only operates on resources which are managed by terraform. i.e You’ve explicitly defined it as a resource in your terraform files/or used terraform import to bring existing resources to be managed under terraform. Terraform has no ability to automatically go discover resources in your infra.
• What happens if the terraform state file goes missing?
  • The state file can be local or it can be in a remote storage
  • Infra will be gone if terraform apply is run after it, so we don’t want to do this
  • We’d need to re-manage the infra using terraform import blocks
• Does the terraform statefile store secrets in plain text?
  • Yes. It’s a guarantee from vendors such as AWS that if you use encryption in your buckets then it’s safe to keep secrets.
    • But this would require using tools such as sops, and even after using them we’ll have the secrets in plain text in the statefile anyway
  • We instead
• What happens when someone edits a resource managed by terraform?
  • The new state will not be known to terraform, if you run terraform apply again it’ll reset the state to what it thinks the state should be
  • See resource drift

Running terraform in a team setting §

• We want to prevent people from running TF apply from their local machines, but it’s okay to do so till we don’t have the CI setup. With this, we’ll be able to review plan before apply in a PR.
• We can add cronjob to check for resource drifts

Intro §

• Useful for stateful infrastructure as code.

Usecase §

• TF should be infra provisioning only
• TF not to be used for config provisioning/management, for config management we got Ansible. Both can co-exist.
• We can use cloud-init (See LXC & LXD) for initial provisioning, that needs to be done before Ansible, eg. SSH setup etc.
• Usually you run Terraform outside of the CI/CD process. Unless integration tests etc, then you might need TF in CI/CD.

Practices/Best Practices §

• See Evolving Your Infrastructure with Terraform - YouTube

Others §

• Consider building AMIs (AWS) that need to do minimal post-boot reconfiguration instead of booting a stock image and then bringing it into desired state from scratch.

Stages and Modules §

Think of the infra in layers of lifetime(you could also structure it the same way) (from reddit)

Eg.

• Stage 1: VPC/networking/DNS (will almost never change)
• Stage 2: Stateful services (things that will rarely change)
• Stage 3: Everything else.

Eg. (Not super sure about following example, should Terraform be handling app deploy logic? (Don’t think so)) Split your configuration into modules/stages:

• base (repositories, network, accounts)
• app (docker deployment)

Then run those in different stages:

• deploy base (creates repo needed to continue)
• build image (docker build … && docker push … now that we have a destination)
• deploy app (deploy image from ECR you just uploaded)

We can then use TF datasources to grab resources from lower layers. It’s easy to get your vpc id and subnet ids from layer 3