Categorygithub.com/ionosphere-io/manage-aws-prefix-lists
modulepackage
0.0.0-20240722054134-70e08b841ea9
Repository: https://github.com/ionosphere-io/manage-aws-prefix-lists.git
Documentation: pkg.go.dev

# README

Manage AWS Prefix Lists

This is a tool for automatically keeping one or more AWS prefix lists in sync with ip-ranges.json. This allows you to create security group rules that allow traffic from a specific AWS service or region without having to specify those ranges manually.

Architecture

The central piece of this is a Lambda function that manages the prefix lists.

The Lambda function is invoked whenever:

  • (1a) AWS updates the ip-ranges.json document, triggering a (1b) notification to an AWS-managed SNS topic that we subscribe to. This causes an (2a) AWS Event Bridge rule to invoke the Lambda function.
  • (2b) A time-based AWS Event Bridge rule (e.g., daily, every 4 hours, etc.) fires and invokes the Lambda function.

When the Lambda function runs, it:

  • (3) Reads ip-ranges.json,
  • (4) Creates or updates a set of prefix lists in your account
  • (5) Optionally updates AWS Systems Manager parameters with the prefix lists, and
  • (6) Optionally notifies an AWS Simple Notification Service (SNS) topic when changes are made.

Examples

Allowing security group access for CloudFront would look like the following:

CloudFormation

CloudFormation requires some way of setting the prefix list ids. We assume that the prefix list ids have been saved to AWS Systems Manager Parameter Store under the names /ManagedPrefixLists/CloudFront/IPv4 and /ManagedPrefixLists/CloudFront/IPv6.

VpcId:
  Type: AWS::EC2::VPC::Id
  Description: The VPC to create the security groups in.
CloudFrontPrefixListsIpv4:
  Type: AWS::SSM::Parameter::Value<List<String>>
  Description: The AWS SSM parameter holding the prefix list IDs for CloudFront's IPv4 ranges.
  Default: /ManagedPrefixLists/CloudFront/IPv4
CloudFrontPrefixListsIpv4:
  Type: AWS::SSM::Parameter::Value<List<String>>
  Description: The AWS SSM parameter holding the prefix list IDs for CloudFront's IPv6 ranges.
  Default: /ManagedPrefixLists/CloudFront/IPv6
Resources:
  # Ideally you should have a CloudFormation macro here that expands into the required number of security groups instead of
  # hard-coding the number directly. This is an advanced CloudFormation setup.
  AllowCloudFront0:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: CloudFront-HTTPS (Group 0)
      GroupDescription: Allow CloudFront to access HTTPS ports
      VpcId: ...
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          SourcePrefixListId: !Select [0, !Ref CloudFrontPrefixListsIpv4]
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          SourcePrefixListId: !Select [0, !Ref CloudFrontPrefixListsIpv6]
  AllowCloudFront1:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: CloudFront-HTTPS (Group 1)
      GroupDescription: Allow CloudFront to access HTTPS ports
      VpcId: ...
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 443
          ToPort: 443
          SourcePrefixListId: !Select [1, !Ref CloudFrontPrefixListsIpv4]

Terraform

As of this writing, Terraform's aws_prefix_list data source cannot query customer-managed prefix lists properly. Here, we assume that the prefix list ids have been saved to AWS Systems Manager Parameter Store under the names /ManagedPrefixLists/CloudFront/IPv4 and /ManagedPrefixLists/CloudFront/IPv6.

variable "region" {
    type = string
    description = "The AWS region to deploy to."
}

variable "aws_vpc_id" {
    type = string
    description = "The VPC to create the security groups in."
}

variable "prefix_list_ssm_parameter_ipv4" {
    type = string
    description = "The AWS SSM parameter holding the prefix list IDs for CloudFront's IPv4 ranges."
    default = "/ManagedPrefixLists/CloudFront/IPv4"
}

variable "prefix_list_ssm_parameter_ipv4" {
    type = string
    description = "The AWS SSM parameter holding the prefix list IDs for CloudFront's IPv6 ranges."
    default = "/ManagedPrefixLists/CloudFront/IPv6"
}

provider "aws" {
    region = var.region
}

data "aws_ssm_parameter" "prefix_list_ipv4" {
    name = var.prefix_list_ssm_parameter.ipv4
}

data "aws_ssm_parameter" "prefix_list_ipv6" {
    name = var.prefix_list_ssm_parameter.ipv6
}

resource "aws_security_group" "cloudfront" {
    # Create as many groups as needed according to the number of prefix lists seen.
    count = max(length(data.aws_ssm_parameter.prefix_list.ipv4.value, data.aws_ssm_parameter.prefix_list.ipv6.value))
    name = "CloudFrfront-HTTPS (Group ${count.index})"
    description = "Allow CloudFront to access HTTPS ports"
    vpc_id = var.aws_vpc_id
}

resource "aws_security_group_rule" "cloudfront_ipv4" {
    count = length(data.aws_ssm_parameter.prefix_list.ipv4.value)
    security_group_id = aws_security_group.cloudfront[count.index].id
    type = "ingress"
    protocol = "tcp"
    from_port = 443
    to_port = 443
    prefix_list_ids = [data.aws.ssm_parameter.prefix_list_ipv4.value[count.index]]
}

resource "aws_security_group_rule" "cloudfront_ipv6" {
    count = length(data.aws_ssm_parameter.prefix_list.ipv6.value)
    security_group_id = aws_security_group.cloudfront[count.index].id
    type = "ingress"
    protocol = "tcp"
    from_port = 443
    to_port = 443
    prefix_list_ids = [data.aws.ssm_parameter.prefix_list_ipv6.value[count.index]]
}

module "managed_prefix_list_core" {
    source  = "ionosphere-io/managed-prefix-list-core/aws"
    version = "v0.2.0"
    region = var.region
}

module "cloudfront_prefix_list" {
    source  = "ionosphere-io/managed-prefix-list/aws"
    version = "v0.2.0"
    region = var.region
    lambda_function_arn = module.managed_prefix_list_core.lambda_function_arn

    filter_service = "CLOUDFRONT"
}

Caution!

Security groups have a default limit of 60 IP rules per group -- this cannot be increased without consequences.

Each customer-managed prefix list has a "maximum number of entries" setting. If you assign a customer-managed prefix list to a security group, the value of this setting counts against your limit for that security group. The actual number of CIDR blocks in the prefix list is irrelevant here.

By default, this tool splits the security group into

# Packages

No description provided by the author

# Functions

AddMetric records a metric on any type that implmenets MetricRecorder.
AggregateNetworks finds the smallest set of prefixes that encompasses a slice of IPNets.
CompareIPNets compares two IP networks, ordering them first by IP address, then by prefix.
CopyAWSInt64 copies an AWS-style int64 from one pointer to another.
CopyAWSString copies an AWS-style string from one pointer to another.
DimAddressFamily creates a CloudWatch metrics dimension for the address family.
DimGroupID creates a CloudWatch metrics dimension for the group ID.
DimPrefixListNameBase create a CloudWatch metrics dimension for the prefix list name base.
GetSupernet returns the network one-mask-size larger than the specified network.
HandleIPRangesUpdated handles SNS notifications when ip-ranges.json is updated.
HandleLambdaRequest is the main Lambda entrypoint for updating a prefix list from ip-ranges.json.
HandleManageRequest handles the core logic of updating a set of prefix lists from ip-ranges.json.
MakeEC2Filter creates an EC2 filter specification with the specified key and values.
MakeEC2Tags converts a TagMap to an array of EC2 Tags.
MakeEC2TagSpec converts a TagMap to a tag specification applied to a single resource.
NewPrefixListAddressFamilyManager creates a new PrefixListAddressFamilyManager object with the specified parameters.
NewPrefixListManagerFromRequest creates a new PrefixListManager object using the data from a ManageAWSPrefixListsRequest event.
SortIPNets sorts a slice of net.IPNet objects and removes any duplicates found.
Time starts a time metric with the specified metric name.

# Constants

AddressFamilyAll indicates all types of IP addresses (IPv4, IPv6) should be queried.
AddressFamilyIPv4 indicates only IPv4 addresses should be queried.
AddressFamilyIPv6 indicates only IPv6 addresses should be queried.
DefaultGroupSize is the default group size used when the group size is not specified.
DefaultIPRangesURL is the default URL used to retrieve ip-ranges.json.
DefaultPrefixListNameTemplate is the default template used when a template is not specified.
DefaultSNSSubject is the default subject used when an SNS subject is not specified.
DimNameAddressFamily is the CloudWatch metrics dimension name for the AddressFamily dimension.
DimNameGroupID is the CloudWatch metrics dimension name for the GroupId dimension.
DimNamePrefixListNameBase is the CloudWatch metrics dimension name for the PrefixListNameBase dimension.
FilterOwnerID is the EC2 filter name for filtering based on owner account numbers.
FilterPrefixListName is the EC2 filter name for filtering based on prefix list names.
FilterTagGroupID is the EC2 filter name for filtering based on GroupId tag values.
MaxRetries is the maximum number of times to retry an API call.
MetricAggregatedPrefixes is the CloudWatch metrics name for the AggregatedPrefixes metric.
MetricCreatePrefixListGroup is the CloudWatch metrics name for the CreatePrefixListGroup metric.
MetricCreatePrefixListGroupSuccess is the CloudWatch metrics name for the CreatePrefixListGroup:Success metric.
MetricDescribeSecurityGroups is the CloudWatch Metrics name for the DescribeSecurityGroups metric.
MetricExaminePrefixListGroup is the CloudWatch Metrics name for the ExaminePrefixListGroup metric.
MetricGetManagedPrefixListEntries is the CloudWatch Metrics name for the GetManagedPrefixListEntries metric.
MetricGetManagedPrefixListEntriesSuccess is the CloudWatch Metrics name for the GetManagedPrefixListEntries:Success metric.
MetricGetPrefixListAssociations is the CloudWatch Metrics name for the GetPrefixListAssociations metric.
MetricGetPrefixListAssociationsSuccess is the CloudWatch Metrics name for the GetPrefixListAssociations:Success metric.
MetricGroups is the CloudWatch metrics name for the Groups metric.
MetricPrefixes is the CloudWatch metrics name for the Prefixes metric.
MetricProcessRuns is the CloudWatch Metrics name for the ProcessRuns metric.
MetricProcessRunsSuccess is the CloudWatch Metrics name for the ProcessRuns:Success metric.
MetricReplacePrefixListGroup is the CloudWatch metrics name for the ReplacePrefixListGroup metric.
MetricsBatchSize is the number of metrics to write to CloudWatch in a single PutMetrics call.
MetricUpdatePrefixListGroup is the CloudWatch metrics name for the UpdatePrefixListGroup metric.
MetricUpdatePrefixListGroupSuccess is the CloudWatch metrics name for the UpdatePrefixListGroup:Success metric.
SleepDuration is the amount of time to sleep before refreshing state.
TierAdvanced represents the advanced SSM tier.
TierIntelligentTiering represents the intelligent tiering SSM tier.
TierStandard represents the standard SSM tier.
UnitCount is the CloudWatch metrics unit name for counted metrics.
UnitMilliseconds is the CloudWatch metrics unit name for millisecond-based metrics.

# Variables

CloudWatchClientKey is a context key to use for retrieving a cloudwatchiface.CloudWatchAPI value from a context.
EC2ClientKey is a context key to use for retrieving an ec2iface.EC2API value from a context.
MatchAllRegex is a regular expression that matches everything.
SNSClientKey is a context key to use for retrieving an snsiface.SNSAPI value from a context.
SSMClientKey is a context key to use for retrieving an ssmiface.STSAPI value from a context.
STSClientKey is a context key to use for retrieving an stsiface.STSAPI value from a context.
TemplateFuncs is a map of template functions.

# Structs

CloudWatchClientKeyType is a context key structure identifying an cloudwatchiface.CloudWatchAPI to use when making API calls (for testing).
EC2ClientKeyType is a context key structure identifying an ec2iface.EC2API to use when making API calls (for testing).
Invoke is the union of types allowed by the Lambda function.
IPRanges is the structure of the ip-ranges.json document.
IPRangesFilter is a filter for the ip-ranges.json file.
IPRangesUpdatedRequest is the outer layer of an SNS notification received by Lambda.
IPv4Prefix is the structure of an IPv4 prefix in the ip-ranges.json document.
IPv6Prefix is the structure of an IPv6 prefix in the ip-ranges.json document.
ManageAWSPrefixListsRequest is the structure an incoming event is expected to adhere to.
ManageAWSPrefixListsResponse is the response message returned by the handler.
MetricTimer represents a time metric (either in-progress or completed).
PrefixListAddressFamilyManager handles the data related to a specific address family.
PrefixListAddressFamilyNotification is an SNS notification sub-structure containing the details about a given address family.
PrefixListManager is the main structure for holding the state of the prefix list manager application.
PrefixListMetrics contains information about where and what type of metrics to write.
PrefixListNotification is the notification we send to SNS when needed.
PrefixListReplacement is an SNS notification sub-struction describing the old prefix list and new prefix list it was replaced by.
PrefixListTemplateVars is a structure holding the variables needed to render the prefix list name from a template.
SNSClientKeyType is a context key structure identifying an snsiface.SNSAPI to use when making API calls (for testing).
SNSNotification is the structure of the SNS message received by Lambda.
SNSNotification is the structure of an SNS notification received by Lambda.
SSMClientKeyType is a context key structure identifying an ssmiface.SSMAPI to use when making API calls (for testing).
SSMParameters contains information about the SSM parameters to write to.
STSClientKeyType is a context key structure identifying an stsiface.STSAPI to use when making API calls (for testing).

# Interfaces

IPPrefix is a common interface for IPv4Prefix and IPv6Prefix.
MetricRecorder is an interface implemented by types that can hold or report metric datums to CloudWatch.

# Type aliases

AddressFamily is an enumeration of the possible types of IP addresses to filter on.
TagMap is a mapping of key-value pairs.
TierEnum is an enumeration of the possible SSM tiers.