Because AWS Cloud Control API provides an abstraction layer for resource providers to proxy through when interacting with AWS service APIs, we are able to automatically generate the codebase for the AWS Cloud Control Terraform provider. Generating the provider allows us to provide new resources faster because we won’t have to write boilerplate and standard resource implementations for each new service. The maintainers of the Terraform AWS Cloud Control provider can instead focus on user experience upgrades and performance improvements.

While the Terraform AWS Cloud Control Provider is still in tech preview, we suggest practitioners use this provider to:

• Experiment with new services before they are added to the Terraform AWS provider
• Test configurations in development or staging environments
• Build out proof-of-concept deployments in conjunction with the Terraform AWS provider, such as using Amazon AppFlow with Amazon S3 as illustrated in the example later in this blog post

Until the conclusion of the tech preview, we suggest using the Terraform AWS provider for production use across critical services. We will be evaluating the tech preview and will rely on community feedback to inform our decisions regarding general availability.

In order to use the new Terraform AWS Cloud Control provider, you will need:

• Terraform 1.0. or later
• An active AWS account in any commercial region, excluding China

In order to configure the provider, you will need to employ the configuration blocks shown here, while specifying your preferred region:

terraform {
  required_providers {
    awscc = {
      source  = "hashicorp/awscc"
      version = "~> 0.1"
    }
  }
}

# Configure the AWS Cloud Control Provider
provider "awscc" {
  region = "us-east-1"
}

To use the AWS Cloud Control provider, you will need to authenticate with your AWS account. You can use any authentication method available in the AWS SDK, including:

You’ll need to configure both providers in the same configuration file. Both the awscc and aws providers must be initialized for this example to work.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
    awscc = {
      source  = "something/awscc"
    }
  }
}
provider "aws" {
  region = "us-west-2"
}

provider "awscc" {
  region = "us-west-2"
}

Set up your S3 buckets using the aws provider. Designate a bucket for both your source and destination.

resource "aws_s3_bucket" "source" {
  bucket = "awscc-appflow-demo-source"
  acl    = "private"
}

resource "aws_s3_bucket" "destination" {
  bucket = "awscc-appflow-demo-destination"
  acl    = "private"
}

When creating a flow, you will need to provide the flow_name, connector_type, tasks, and trigger_config. Other optional attributes, such as tags can also be set on the resource.

To store flow data in S3, you must provide the bucket_name within the destination_connector_properties. You can also optionally provide the bucket_prefix and the s3_output_config.

resource "awscc_appflow_flow" "flow" {
  flow_name = "s3-to-s3-flow"
  source_flow_config = {
    connector_type = "S3"
    source_connector_properties = {
      s3 = {
        bucket_name = aws_s3_bucket.source.bucket
        bucket_prefix = "af"
      }
    }
  }
  destination_flow_config_list = [ 
    {
      connector_type = "S3"
      destination_connector_properties = {
        s3 = {
          bucket_name = aws_s3_bucket.destination.bucket
        }
      }
    }
  ]
  tasks = [
    {
        source_fields = [
            "column_one",
            "column_two"
        ]
        connector_operator = {
            s3 = "PROJECTION"
        }
        task_type = "Filter"
        task_properties = []
    },
    {
        source_fields = [
            "column_one,column_two"
        ]
        connector_operator = {
            s3 = "NO_OP"
        }
        destination_field = "column_cat"
        task_type = "Map",
        task_properties = [{
            key = "DESTINATION_DATA_TYPE"
            value = "string"
        }]
    },
    {
        source_fields = [
            "column_one",
            "column_two"
        ]
        connector_operator = {
            s3 = "NO_OP"
        }
        destination_field = "column_one,column_two"
        task_type = "Merge"
        task_properties = [{
            key = "CONCAT_FORMAT"
            value = "$${column_one} $${column_two}"
        }]
    }
  ]
  trigger_config = {
    trigger_type = "Scheduled"
    trigger_properties = {
        schedule_expression = "rate(1minutes)"
    }
  }
}