CDK for Terraform in Production: Learning from Pocket

Kelvin Yeboah and Mathijs Miermans from Pocket’s engineering team joined us in a recent Terraform Community Office Hours to share a live demo of how they define and provision infrastructure using CDK for Terraform. Check out the recording to watch the live demo, and read on for a summary of how their team is using CDK for Terraform.

»CDK for Terraform

The Cloud Development Kit for Terraform (CDKTF) allows you to use familiar programming languages to define cloud infrastructure and provision it through Terraform. This gives you access to the entire Terraform ecosystem while allowing you to leverage the power of your existing toolchain.

CDKTF is under active development and is not officially recommended for production use cases. However, some early adopters, like Pocket, are already using CDK for Terraform in production, and we are excited to work with them to validate and improve workflows.

»Provisioning with Terraform Using the Familiarity of TypeScript

Pocket — a Mozilla product — is a website and app that finds the most interesting, thought-provoking and entertaining articles from trusted sources around the internet and puts them all in one place. Pocket also lets you save articles — as well as anything else you find online (videos, recipes, shopping pages, etc.) — to your personal Pocket for digging into later.

To support the user experience, the Pocket engineering team is tasked with building, maintaining, and deploying a wide variety of services, with the same type of infrastructure being deployed frequently. Over time, the team transitioned from managing infrastructure in a UI to working with Terraform, where they loved having version control and being able to manage state. But HCL was a new syntax and framework for the team to learn.

Transitioning to writing infrastructure code in their preferred programming language— TypeScript — using CDK for Terraform has allowed them to spin up new services faster and get new features in front of users with less delay. Managing infrastructure in a familiar language has also empowered their developers to engage closely with the infrastructure, giving them a better understanding of the services they are building and removing blockers.

“When CDKTF came out, that was like a game changer for us. We are mostly application developers, we are not an SRE team … so if we could write our infrastructure in code in the language that we already use, that was a game changer for us. Because then we can empower ourselves to work faster and more efficiently and build services faster.” — Kelvin Yeboah, Senior Software Engineer, Mozilla, Pocket

Another practical improvement that they’ve noticed is that they have fewer files to manage and they are able to use their existing code pipeline with CircleCI to automate the plan/apply deployment workflow. For those interested in seeing a CDK for Terraform project in action, the team’s open source GitHub repos are an excellent reference for structuring and writing code with CDKTF: https://github.com/Pocket.

»The Demo

During the live demo, Mathijs and Kelvin walked us through how to deploy a “Hello world” Apache web server in a production-ready environment using Amazon ECS in fewer than 150 lines of readable TypeScript code. In addition to provisioning infrastructure for the web server, this demo application also includes other production-ready necessities, including an ECS cluster, application load balancer, defined security groups, alarms via PagerDuty, and snapshot testing of the output JSON configuration file.

You can reference the code used for this demo in the repos below, and even follow along as you watch the recording:

• Demo source code
• Reusable constructs

»Simplifying Application Code with Reusable Constructs

CDK for Terraform allows you to manage complexity and reduce code duplication by creating custom abstraction layers, referred to as constructs. During the demo, Kelvin references the custom construct libraries that their team has built for their CDK for Terraform TypeScript applications. These are opinionated libraries that their team has built to customize the CDKTF interface to the developer team’s needs. For example, in the demo, Kelvin uses two of these custom packages to add a production-ready ECS cluster, a load balancer, and to create alarms using PagerDuty. You can find these libraries in an open source repo of Node.js packages that define their infrastructure patterns in CDKTF.

For more examples of how the Pocket team uses custom constructs to simplify their development process and ensure best practices, check out these open source repos:

• Pocket/firefox-android-home-recommendations: TypeScript REST API
• Pocket/collection-api: TypeScript GraphQL API with a relational database
• Pocket/recommendation-api: Python GraphQL API with a key-value store and AWS Lambda functions

»Get Started with CDK for Terraform

If you’re new to CDK for Terraform, the best place to get started is with the hands-on tutorials on HashiCorp Learn.

Stay tuned for a follow-up Community Office Hours on October 27, where the Pocket team will do a more thorough walkthrough of their codebase, recommended architecture, and best practices.

Source: HashiCorp Blog

Announcing HashiCorp Nomad 1.2 Beta

We are excited to announce that the beta release of HashiCorp Nomad 1.2 is now available. Nomad is a simple and flexible orchestrator used to deploy and manage containers and non-containerized applications. Nomad works across on-premises and cloud environments. It is widely adopted and used in production by organizations such as Cloudflare, Roblox, Q2, Pandora, and GitHub.

Let’s take a look at what’s new in Nomad and in the Nomad ecosystem, including:

• System Batch jobs
• User interface upgrades
• Nomad Pack

»System Batch Jobs

Like System jobs, System Batch jobs work without an update stanza and will run on any node in the cluster that is not excluded via constraints. Unlike System jobs, System Batch jobs will run only on clients that are ready at the time the job was submitted to Nomad.

Like Batch jobs, System Batch jobs are meant to run to completion, can be run on a scheduled basis, and support dispatch execution with per-run parameters.

If you want to run a simple sysbatch job, the job specification might look something like this:

job "sysbatchjob" {
  datacenters = ["dc1"]

  type = "sysbatch"

  constraint {
    attribute = "${attr.kernel.name}"
    value     = "linux"
  }

  group "sysbatch_job_group" {
    count = 1

    task "sysbatch_task" {
      driver = "docker"

      config {
        image = "busybox:1"

        command = "/bin/sh"
        args    = ["-c", "echo hi; sleep 1"]
      }
    }
  }
}

This will run a short-lived Docker task on every client node in the cluster that is running Linux.

To run this job at regular intervals, you would add a periodic stanza:

periodic {
  cron             = "0 0 */2 ? * *"
  prohibit_overlap = true
}

For instance, the stanza above instructs Nomad to re-run the sysbatch job every hour.

Additionally, sysbatch jobs can be parameterized and then invoked later using the dispatch command. These specialized jobs act less like regular Nomad jobs and more like cluster-wide functions.

Adding a parameterized stanza defines the arguments that can be passed into the job. For example, a sysbatch job that upgrades Consul to a different version might have a parameterized stanza that looks like this:

parameterized {
  payload   	= "forbidden"
  meta_required = ["consul_version"]
  meta_optional = ["retry_count"]
}

This sysbatch job could then be registered using the run command, and executed using the dispatch command:

$ nomad job run upgrade_consul
$ nomad job dispatch upgrade_consul -meta consul_version=1.11.0

»User Interface Upgrades

»Nomad Pack (Tech Preview)

We are excited to announce the tech preview of Nomad Pack, a package manager for Nomad. Nomad Pack makes it easy to define reusable application deployments. This lets you quickly spin up popular open source applications, define deployment patterns that can be reused across teams within your organization, and discover job specifications from the Nomad community. Need a quick Traefik load balancer? There’s a Pack for that.

Each Pack is a group of resources that are meant to be deployed to Nomad together. In the Tech Preview, these resources must be Nomad jobs, but we expect to add volumes and ACL policies in a future release.

Let’s take a look at Nomad Pack, using the Nomad Autoscaler as an example.

Traditionally, users deploying the Nomad Autoscaler often need to deploy or configure multiple jobs within Nomad, usually Grafana, Loki, the autoscaler itself, an APM, and a load balancer.

With Nomad Pack you can run a single command to deploy all the necessary autoscaler resources to Nomad. Optionally, the deployment can be customized by passing in a variable value:

This allows you to spend less time learning and writing Nomad job specs for each app you deploy. See the Nomad Pack repository for more details on basic usage.

By default, Nomad Pack uses the Nomad Pack Community Registry as its source for Packs. This registry provides a location for the Nomad community to share their Nomad configuration files, learn app-specific best practices, and get feedback and contributions from the broader community. Alternative registries and internal repositories can also be used with Nomad Pack. To view available packs, run the registry list command:

You can easily write and customize Packs for your specific organization’s needs using Go Template, a common templating language that is simple to write but can also contain complex logic. Templates can be composed and re-used across multiple packs, which allows organizations to more easily standardize Nomad configurations, codify best practices, and make changes across multiple jobs at once.

To learn more about writing your own packs and registries, see the Writing Custom Packs guide in the repository.

A Tech Preview release of Nomad Pack will be available in the coming weeks. The Nomad team is still validating the design and specifications around the tool and packs. While we don’t expect changes to the user flows that Nomad Pack enables, some details may change based on user feedback. Until the release, to use Nomad Pack you can build from the source code. Details can be found in the repository’s contributing guide.

As you use Nomad Pack and write your own packs, please don’t hesitate to provide feedback. Issues and pull requests are welcome on the GitHub repository and Pack suggestions and votes are encouraged via Community Pack Registry issues.

»What’s Next?

We encourage you to experiment with the new features in Nomad 1.2 and Nomad Pack, but we recommend against using Nomad 1.2 in a production environment until the official GA release. We are eager to see how the new features and projects enhance your Nomad experience. If you encounter an issue, please file a new bug report in GitHub and we’ll take a look.

Finally, on behalf of the Nomad team, I’d like to thank our amazing community. Your dedication, feature requests, pull requests, and bug reports help us make Nomad better. We are deeply grateful for your time, passion, and support.

Source: HashiCorp Blog