CDK for Terraform in Production: Learning from Pocket

CDK for Terraform in Production: Learning from Pocket

Kelvin Yeboah and Mathijs Miermans from Pocket’s engineering team joined us in a recent Terraform Community Office Hours to share a live demo of how they define and provision infrastructure using CDK for Terraform. Check out the recording to watch the live demo, and read on for a summary of how their team is using CDK for Terraform.

»CDK for Terraform

The Cloud Development Kit for Terraform (CDKTF) allows you to use familiar programming languages to define cloud infrastructure and provision it through Terraform. This gives you access to the entire Terraform ecosystem while allowing you to leverage the power of your existing toolchain.

CDKTF is under active development and is not officially recommended for production use cases. However, some early adopters, like Pocket, are already using CDK for Terraform in production, and we are excited to work with them to validate and improve workflows.

»Provisioning with Terraform Using the Familiarity of TypeScript

Pocket — a Mozilla product — is a website and app that finds the most interesting, thought-provoking and entertaining articles from trusted sources around the internet and puts them all in one place. Pocket also lets you save articles — as well as anything else you find online (videos, recipes, shopping pages, etc.) — to your personal Pocket for digging into later.

To support the user experience, the Pocket engineering team is tasked with building, maintaining, and deploying a wide variety of services, with the same type of infrastructure being deployed frequently. Over time, the team transitioned from managing infrastructure in a UI to working with Terraform, where they loved having version control and being able to manage state. But HCL was a new syntax and framework for the team to learn.

Transitioning to writing infrastructure code in their preferred programming language— TypeScript — using CDK for Terraform has allowed them to spin up new services faster and get new features in front of users with less delay. Managing infrastructure in a familiar language has also empowered their developers to engage closely with the infrastructure, giving them a better understanding of the services they are building and removing blockers.

“When CDKTF came out, that was like a game changer for us. We are mostly application developers, we are not an SRE team … so if we could write our infrastructure in code in the language that we already use, that was a game changer for us. Because then we can empower ourselves to work faster and more efficiently and build services faster.” — Kelvin Yeboah, Senior Software Engineer, Mozilla, Pocket

Another practical improvement that they’ve noticed is that they have fewer files to manage and they are able to use their existing code pipeline with CircleCI to automate the plan/apply deployment workflow. For those interested in seeing a CDK for Terraform project in action, the team’s open source GitHub repos are an excellent reference for structuring and writing code with CDKTF: https://github.com/Pocket.

»The Demo

During the live demo, Mathijs and Kelvin walked us through how to deploy a “Hello world” Apache web server in a production-ready environment using Amazon ECS in fewer than 150 lines of readable TypeScript code. In addition to provisioning infrastructure for the web server, this demo application also includes other production-ready necessities, including an ECS cluster, application load balancer, defined security groups, alarms via PagerDuty, and snapshot testing of the output JSON configuration file.

A

You can reference the code used for this demo in the repos below, and even follow along as you watch the recording:

»Simplifying Application Code with Reusable Constructs

CDK for Terraform allows you to manage complexity and reduce code duplication by creating custom abstraction layers, referred to as constructs. During the demo, Kelvin references the custom construct libraries that their team has built for their CDK for Terraform TypeScript applications. These are opinionated libraries that their team has built to customize the CDKTF interface to the developer team’s needs. For example, in the demo, Kelvin uses two of these custom packages to add a production-ready ECS cluster, a load balancer, and to create alarms using PagerDuty. You can find these libraries in an open source repo of Node.js packages that define their infrastructure patterns in CDKTF.

Pocket

For more examples of how the Pocket team uses custom constructs to simplify their development process and ensure best practices, check out these open source repos:

»Get Started with CDK for Terraform

If you’re new to CDK for Terraform, the best place to get started is with the hands-on tutorials on HashiCorp Learn.

Stay tuned for a follow-up Community Office Hours on October 27, where the Pocket team will do a more thorough walkthrough of their codebase, recommended architecture, and best practices.


Source: HashiCorp Blog

HashiCorp Learning Resources Reference Guide

HashiCorp Learning Resources Reference Guide

Continuing technical education is highly valued here at HashiCorp, and it’s something we want to help practitioners and organizations practice. Over the years, we have created many learning resources and will continue to do so. This article contains a curated list of learning resources we have made available to help practitioners and organizations better understand the cloud operating model and our products.

We break out the learning resources into three categories:

  1. Self-paced learning
  2. Workshops
  3. Technical examples

»Self-Paced Learning

The majority of the resources listed below may be accessed at any time. Previously held conferences are available as recordings.

»Documentation

The HashiCorp product documentation pages contain information related to product configuration, architecture, extensions, integrations, and much more:

»HashiCorp Learn

HashiCorp Learn is the primary learning platform for all our products. The education material is maintained and developed by the HashiCorp Education team. The HashiCorp Learn platform contains beginner to advanced tutorials for each product. Several tutorials contain interactive, hands-on lab environments that allow the reader to learn without having to install any software. The platform is a supplemental resource to the HashiCorp documentation pages.

»YouTube HashiCorp

The HashiCorp YouTube Channel is filled with content that explains products and technical topics at a high level in a user-friendly way. The channel is a great starting point for learning and understanding how our products fit in the cloud operating model.

»HashiCast

HashiCast is a podcast about the world of cloud infrastructure. This podcast highlights people and technology from companies in the technology community, as well as insight and news from HashiCorp.

»HashiCorp Resource Library

The HashiCorp resource library aggregates a wide variety of HashiCorp content. The resource library’s integrated search functionality allows you to filter content by product, resource type, language, industry, infrastructure provider, and enterprise type. Below is a list of resource types available in the library. To stay up to date on new additions, subscribe to the resource library’s RSS feed.

»Community Office Hours

The Community Office Hours are live sessions that allow practitioners to ask questions and learn about various topics. Each HashiCorp product has a dedicated community office-hours session led by the HashiCorp Developer Advocate team. Practitioners may also request private office-hours sessions.

»HashiCorp Blog

The HashiCorp Blog contains announcements, technical articles, topic deep dives, and many more resources. The HashiCorp blog contains content for each product, as well as general information about HashiCorp.

»HashiCorp Medium Blog

Focusing on technical topics, the HashiCorp Medium blog is developed and maintained by the HashiCorp Sales Engineering team.

»HashiCorp Certifications

The HashiCorp Certification program offers certifications for HashiCorp Consul, Terraform, and Vault. You can use the HashiCorp Certification program to earn formal, industry-accepted credentials that validate your technical knowledge. Each certification program tests conceptual knowledge and real-world experience using HashiCorp’s multi-cloud tools (Terraform, Vault, Consul, Nomad). Passing the exam allows you to easily communicate your proficiency, and employers can quickly verify your results.

»HashiConf Global and HashiConf Europe

HashiConf is an annual conference held in Europe and the U.S. HashiConf Europe is held during the early part of the year, and HashiConf Global is held in the latter part of the year. HashiConf is an interactive digital experience organized for the HashiCorp community. Hear keynotes and product updates, dive deep with hands-on labs and technical sessions, and make connections around the world. Most of the sessions are recorded and made available on-demand on the HashiCorp YouTube channel.

»HashiTalks

HashiTalks are presentations featuring our community members. The topics range from creative solutions to common challenges, to technical deep dives, to tips and tricks from our practitioners.

»Workshops

We offer interactive workshops at many times during the year. These instructor-led workshops allow attendees to ask questions and explore topics in greater detail.

»HashiCorp Enterprise Academy

The HashiCorp Enterprise Academy offers instructor-led courses taught by HashiCorp Certified Instructors. These workshops help students gain product proficiency through hands-on learning while offering the opportunity to ask instructors questions throughout the course.

»HashiConf Hands-on Labs

The HashiConf Hands-on Labs are based on existing tutorials available at learn.hashicorp.com. They leverage interactive learning environments that do not require attendees to install any software. Attendees can follow along with the instructor, as well as access learning environments from the HashiCorp Learn tutorial page. The Hands-on Labs offer the opportunity to learn more about a product topic and ask questions.

»Technical Examples

The resources listed below were all authored by HashiCorp. We omitted technical examples from the community due to size and scope constraints.

Boundary

»Boundary Repository Collection

The Boundary repository collection contains example GitHub repositories. Each repository addresses a specific scenario.

»Boundary Reference Architecture

The Boundary reference architecture repository contains reference architectures for deploying Boundary on the major cloud platforms.

Consul

»Consul Terraform Repository

The Consul Terraform repository is an example repository that showcases how to deploy Consul through Terraform.

»Consul Docker Compose Repository

The Consul Docker Compose repository contains several Consul architectures that you can deploy through Docker Compose.

»Consul Repository Collection

The Consul repository collection is a collection of Consul Learn repositories from the HashiCorp Education team. Each repository addresses a specific scenario.

Nomad

»Nomad Repository Collection

The Nomad repository collection is an aggregation of Nomad example repositories. Each repository addresses a specific scenario.

»Nomad Autoscaler Demos

The Nomad Autoscaler Demos repository contains a collection of demos for the Nomad Autoscaler.

»Nomad 1 Million Container Challenge

The Nomad 1 Million Container Challenge repository contains the infrastructure code necessary to run the Million Container Challenge using HashiCorp Nomad on Google’s Compute Engine Cloud or Amazon Web Services.

»Nomad 2 Million Container Challenge

The Nomad 2 Million Container Challenge repository contains the code to run scalability tests on Nomad for the second iteration of the C1M challenge.

Packer

»Packer Repository Collection

The Packer repository collection is a collection of Packer Learn repositories from the HashiCorp Education team. Each repository addresses a specific scenario.

HashiCorp

»Sentinel GitHub Repository

The Sentinel GitHub repository is an example repository of various Sentinel use cases. The project repository contains Sentinel policies for common governance scenarios.

Terraform

»Terraform Repository Collection

The Terraform repository collection is a collection of Terraform Learn repositories from the HashiCorp Education team. Each repository addresses a specific scenario.

»Terraform Foundational Sentinel Policies

The Terraform Foundational Sentinel Policies repository contains a library of policies that can be used within Terraform Cloud and Terraform Enterprise to accelerate adoption of policy as code.

»Terraform Enterprise Install Module

The official Terraform module for installing Terraform Enterprise (private install) contains examples for installing Terraform Enterprise in AWS, Microsoft Azure, and Google Cloud.

NOTE: The modules are currently in beta status (10/11/2021)

»Terraform Kubernetes Collection

This collection holds Terraform- and Kubernetes-related example repositories for the major cloud platforms.

Vault

»Vault Install Module

The official HashiCorp Terraform module for installing Vault in accordance with best practices includes a repository and module for each of the major cloud platforms.

»Vault Enterprise Install Module

The official HashiCorp Terraform module for installing Vault Enterprise in accordance with best practices includes a repository and module for each of the major cloud platforms.

»Vault Repository Collection

The Vault repository collection is an aggregation of Vault Learn repositories from the HashiCorp Education team. Each repository addresses a specific scenario.

»Vault Programmatic Usage Repository

The Vault Programmatic Usage repository is an example repository that showcases how to use the Vault SDK from inside an application.

Vagrant

»Vagrant Consul Nomad Repository

The Vagrant Consul Nomad repository
is a collection of Vagrant scripts for starting up a Consul datacenter and a Nomad datacenter.

Waypoint

»Waypoint Examples

The Waypoint examples repository showcases how to use Waypoint with different types of programming languages and applications.

»Next Steps

If you are ready to take your learning to the next level, we encourage you to visit the HashiCorp Certification page and start preparing for the HashiCorp Certification exam of your choice.

Additionally, registration is now open for HashiConf Global (Tuesday – Wednesday, October 19 – 20). HashiConf offers many fantastic learning opportunities and announcements that you will not want to miss out on. Go to HashiConf.com to learn more.

Lastly, make sure to visit (and bookmark) HashiCorp Learn. HashiCorp Learn is our primary education resource to help you along the infrastructure journey. As mentioned earlier, it’s packed with amazing content and interactive lab environments. Let us know what you think about these learning resources and how we may improve at the HashiCorp Community Forum.


Source: HashiCorp Blog