Commit 8836d0f1 authored by Melissa Greenbaum's avatar Melissa Greenbaum Committed by GitHub

feat: Add inventory config support (#192)

Co-authored-by: magreenbaum <magreenbaum>
parent 4ee2d93b
......@@ -139,6 +139,7 @@ No modules.
| [aws_s3_bucket_acl.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) | resource |
| [aws_s3_bucket_cors_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_cors_configuration) | resource |
| [aws_s3_bucket_intelligent_tiering_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_intelligent_tiering_configuration) | resource |
| [aws_s3_bucket_inventory.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_inventory) | resource |
| [aws_s3_bucket_lifecycle_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_lifecycle_configuration) | resource |
| [aws_s3_bucket_logging.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_logging) | resource |
| [aws_s3_bucket_metric.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_metric) | resource |
......@@ -151,11 +152,13 @@ No modules.
| [aws_s3_bucket_server_side_encryption_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration) | resource |
| [aws_s3_bucket_versioning.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_versioning) | resource |
| [aws_s3_bucket_website_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_website_configuration) | resource |
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
| [aws_canonical_user_id.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/canonical_user_id) | data source |
| [aws_elb_service_account.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/elb_service_account) | data source |
| [aws_iam_policy_document.combined](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.deny_insecure_transport](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.elb_log_delivery](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.inventory_destination_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.lb_log_delivery](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.require_latest_tls](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
......@@ -167,6 +170,7 @@ No modules.
| <a name="input_acl"></a> [acl](#input\_acl) | (Optional) The canned ACL to apply. Conflicts with `grant` | `string` | `null` | no |
| <a name="input_attach_deny_insecure_transport_policy"></a> [attach\_deny\_insecure\_transport\_policy](#input\_attach\_deny\_insecure\_transport\_policy) | Controls if S3 bucket should have deny non-SSL transport policy attached | `bool` | `false` | no |
| <a name="input_attach_elb_log_delivery_policy"></a> [attach\_elb\_log\_delivery\_policy](#input\_attach\_elb\_log\_delivery\_policy) | Controls if S3 bucket should have ELB log delivery policy attached | `bool` | `false` | no |
| <a name="input_attach_inventory_destination_policy"></a> [attach\_inventory\_destination\_policy](#input\_attach\_inventory\_destination\_policy) | Controls if S3 bucket should have bucket inventory destination policy attached. | `bool` | `false` | no |
| <a name="input_attach_lb_log_delivery_policy"></a> [attach\_lb\_log\_delivery\_policy](#input\_attach\_lb\_log\_delivery\_policy) | Controls if S3 bucket should have ALB/NLB log delivery policy attached | `bool` | `false` | no |
| <a name="input_attach_policy"></a> [attach\_policy](#input\_attach\_policy) | Controls if S3 bucket should have bucket policy attached (set to `true` to use value of `policy` as bucket policy) | `bool` | `false` | no |
| <a name="input_attach_public_policy"></a> [attach\_public\_policy](#input\_attach\_public\_policy) | Controls if a user defined public bucket policy will be attached (set to `false` to allow upstream to apply defaults to the bucket) | `bool` | `true` | no |
......@@ -183,6 +187,10 @@ No modules.
| <a name="input_grant"></a> [grant](#input\_grant) | An ACL policy grant. Conflicts with `acl` | `any` | `[]` | no |
| <a name="input_ignore_public_acls"></a> [ignore\_public\_acls](#input\_ignore\_public\_acls) | Whether Amazon S3 should ignore public ACLs for this bucket. | `bool` | `false` | no |
| <a name="input_intelligent_tiering"></a> [intelligent\_tiering](#input\_intelligent\_tiering) | Map containing intelligent tiering configuration. | `any` | `{}` | no |
| <a name="input_inventory_configuration"></a> [inventory\_configuration](#input\_inventory\_configuration) | Map containing S3 inventory configuration. | `any` | `{}` | no |
| <a name="input_inventory_self_source_destination"></a> [inventory\_self\_source\_destination](#input\_inventory\_self\_source\_destination) | Whether or not the inventory source bucket is also the destination bucket. | `bool` | `false` | no |
| <a name="input_inventory_source_account_id"></a> [inventory\_source\_account\_id](#input\_inventory\_source\_account\_id) | The inventory source account id. | `string` | `null` | no |
| <a name="input_inventory_source_bucket_arn"></a> [inventory\_source\_bucket\_arn](#input\_inventory\_source\_bucket\_arn) | The inventory source bucket ARN. | `string` | `null` | no |
| <a name="input_lifecycle_rule"></a> [lifecycle\_rule](#input\_lifecycle\_rule) | List of maps containing configuration of object lifecycle management. | `any` | `[]` | no |
| <a name="input_logging"></a> [logging](#input\_logging) | Map containing access bucket logging configuration. | `map(string)` | `{}` | no |
| <a name="input_metric_configuration"></a> [metric\_configuration](#input\_metric\_configuration) | Map containing bucket metric configuration. | `any` | `[]` | no |
......
......@@ -347,5 +347,4 @@ module "s3_bucket" {
name = "all"
}
]
}
# S3 bucket with Inventory Configurations
Configuration in this directory creates an S3 bucket with several inventory configurations including a different source and destination for inventory reports generated.
Please check [complete example](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/complete) to see all other features supported by this module.
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
## Requirements
| Name | Version |
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.13.1 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.9 |
| <a name="requirement_random"></a> [random](#requirement\_random) | >= 2.0 |
## Providers
| Name | Version |
|------|---------|
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.9 |
| <a name="provider_random"></a> [random](#provider\_random) | >= 2.0 |
## Modules
| Name | Source | Version |
|------|--------|---------|
| <a name="module_inventory_destination_bucket"></a> [inventory\_destination\_bucket](#module\_inventory\_destination\_bucket) | ../../ | n/a |
| <a name="module_inventory_source_bucket"></a> [inventory\_source\_bucket](#module\_inventory\_source\_bucket) | ../../ | n/a |
| <a name="module_kms"></a> [kms](#module\_kms) | terraform-aws-modules/kms/aws | n/a |
| <a name="module_multi_inventory_configurations_bucket"></a> [multi\_inventory\_configurations\_bucket](#module\_multi\_inventory\_configurations\_bucket) | ../../ | n/a |
## Resources
| Name | Type |
|------|------|
| [random_pet.this](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) | resource |
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
## Inputs
No inputs.
## Outputs
| Name | Description |
|------|-------------|
| <a name="output_s3_bucket_arn"></a> [s3\_bucket\_arn](#output\_s3\_bucket\_arn) | The ARN of the bucket. Will be of format arn:aws:s3:::bucketname. |
| <a name="output_s3_bucket_bucket_domain_name"></a> [s3\_bucket\_bucket\_domain\_name](#output\_s3\_bucket\_bucket\_domain\_name) | The bucket domain name. Will be of format bucketname.s3.amazonaws.com. |
| <a name="output_s3_bucket_bucket_regional_domain_name"></a> [s3\_bucket\_bucket\_regional\_domain\_name](#output\_s3\_bucket\_bucket\_regional\_domain\_name) | The bucket region-specific domain name. The bucket domain name including the region name, please refer here for format. Note: The AWS CloudFront allows specifying S3 region-specific endpoint when creating S3 origin, it will prevent redirect issues from CloudFront to S3 Origin URL. |
| <a name="output_s3_bucket_hosted_zone_id"></a> [s3\_bucket\_hosted\_zone\_id](#output\_s3\_bucket\_hosted\_zone\_id) | The Route 53 Hosted Zone ID for this bucket's region. |
| <a name="output_s3_bucket_id"></a> [s3\_bucket\_id](#output\_s3\_bucket\_id) | The name of the bucket. |
| <a name="output_s3_bucket_region"></a> [s3\_bucket\_region](#output\_s3\_bucket\_region) | The AWS region this bucket resides in. |
| <a name="output_s3_bucket_website_domain"></a> [s3\_bucket\_website\_domain](#output\_s3\_bucket\_website\_domain) | The domain of the website endpoint, if the bucket is configured with a website. If not, this will be an empty string. This is used to create Route 53 alias records. |
| <a name="output_s3_bucket_website_endpoint"></a> [s3\_bucket\_website\_endpoint](#output\_s3\_bucket\_website\_endpoint) | The website endpoint, if the bucket is configured with a website. If not, this will be an empty string. |
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
locals {
bucket_name = "s3-bucket-${random_pet.this.id}"
region = "eu-west-1"
}
provider "aws" {
region = local.region
# Make it faster by skipping something
skip_get_ec2_platforms = true
skip_metadata_api_check = true
skip_region_validation = true
skip_credentials_validation = true
skip_requesting_account_id = true
}
data "aws_caller_identity" "current" {}
module "multi_inventory_configurations_bucket" {
source = "../../"
bucket = local.bucket_name
force_destroy = true
attach_policy = true
attach_inventory_destination_policy = true
inventory_self_source_destination = true
acl = "private" # "acl" conflicts with "grant" and "owner"
versioning = {
status = true
mfa_delete = false
}
inventory_configuration = {
# Same source and destination buckets
daily = {
included_object_versions = "Current"
destination = {
format = "CSV"
encryption = {
encryption_type = "sse_kms"
kms_key_id = module.kms.key_arn
}
}
filter = {
prefix = "documents/"
}
frequency = "Daily"
}
weekly = {
included_object_versions = "All"
destination = {
format = "CSV"
}
frequency = "Weekly"
}
# Different destination bucket
destination_other = {
included_object_versions = "All"
destination = {
bucket_arn = module.inventory_destination_bucket.s3_bucket_arn
format = "Parquet"
encryption = {
encryption_type = "sse_s3"
}
}
frequency = "Weekly"
optional_fields = ["Size", "EncryptionStatus", "StorageClass", "ChecksumAlgorithm"]
}
# Different source bucket
source_other = {
included_object_versions = "Current"
bucket = module.inventory_source_bucket.s3_bucket_id
destination = {
format = "ORC"
encryption = {
encryption_type = "sse_s3"
}
}
frequency = "Daily"
}
}
}
resource "random_pet" "this" {
length = 2
}
# https://docs.aws.amazon.com/AmazonS3/latest/userguide/configure-inventory.html#configure-inventory-kms-key-policy
module "kms" {
source = "terraform-aws-modules/kms/aws"
description = "Key example for Inventory S3 destination encyrption"
deletion_window_in_days = 7
key_statements = [
{
sid = "s3InventoryPolicy"
actions = [
"kms:GenerateDataKey",
]
resources = ["*"]
principals = [
{
type = "Service"
identifiers = ["s3.amazonaws.com"]
}
]
conditions = [
{
test = "StringEquals"
variable = "aws:SourceAccount"
values = [
data.aws_caller_identity.current.id,
]
},
{
test = "ArnLike"
variable = "aws:SourceARN"
values = [
module.inventory_source_bucket.s3_bucket_arn,
module.multi_inventory_configurations_bucket.s3_bucket_arn
]
}
]
}
]
}
module "inventory_destination_bucket" {
source = "../../"
bucket = "inventory-destination-${random_pet.this.id}"
acl = "private" # "acl" conflicts with "grant" and "owner"
force_destroy = true
attach_policy = true
attach_inventory_destination_policy = true
inventory_source_bucket_arn = module.multi_inventory_configurations_bucket.s3_bucket_arn
inventory_source_account_id = data.aws_caller_identity.current.id
}
module "inventory_source_bucket" {
source = "../../"
bucket = "inventory-source-${random_pet.this.id}"
acl = "private" # "acl" conflicts with "grant" and "owner"
force_destroy = true
}
output "s3_bucket_id" {
description = "The name of the bucket."
value = module.multi_inventory_configurations_bucket.s3_bucket_id
}
output "s3_bucket_arn" {
description = "The ARN of the bucket. Will be of format arn:aws:s3:::bucketname."
value = module.multi_inventory_configurations_bucket.s3_bucket_arn
}
output "s3_bucket_bucket_domain_name" {
description = "The bucket domain name. Will be of format bucketname.s3.amazonaws.com."
value = module.multi_inventory_configurations_bucket.s3_bucket_bucket_domain_name
}
output "s3_bucket_bucket_regional_domain_name" {
description = "The bucket region-specific domain name. The bucket domain name including the region name, please refer here for format. Note: The AWS CloudFront allows specifying S3 region-specific endpoint when creating S3 origin, it will prevent redirect issues from CloudFront to S3 Origin URL."
value = module.multi_inventory_configurations_bucket.s3_bucket_bucket_regional_domain_name
}
output "s3_bucket_hosted_zone_id" {
description = "The Route 53 Hosted Zone ID for this bucket's region."
value = module.multi_inventory_configurations_bucket.s3_bucket_hosted_zone_id
}
output "s3_bucket_region" {
description = "The AWS region this bucket resides in."
value = module.multi_inventory_configurations_bucket.s3_bucket_region
}
output "s3_bucket_website_endpoint" {
description = "The website endpoint, if the bucket is configured with a website. If not, this will be an empty string."
value = module.multi_inventory_configurations_bucket.s3_bucket_website_endpoint
}
output "s3_bucket_website_domain" {
description = "The domain of the website endpoint, if the bucket is configured with a website. If not, this will be an empty string. This is used to create Route 53 alias records. "
value = module.multi_inventory_configurations_bucket.s3_bucket_website_domain
}
terraform {
required_version = ">= 0.13.1"
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 4.9"
}
random = {
source = "hashicorp/random"
version = ">= 2.0"
}
}
}
data "aws_canonical_user_id" "this" {}
data "aws_caller_identity" "current" {}
locals {
create_bucket = var.create_bucket && var.putin_khuylo
attach_policy = var.attach_require_latest_tls_policy || var.attach_elb_log_delivery_policy || var.attach_lb_log_delivery_policy || var.attach_deny_insecure_transport_policy || var.attach_policy
attach_policy = var.attach_require_latest_tls_policy || var.attach_elb_log_delivery_policy || var.attach_lb_log_delivery_policy || var.attach_deny_insecure_transport_policy || var.attach_inventory_destination_policy || var.attach_policy
# Variables with type `any` should be jsonencode()'d when value is coming from Terragrunt
grants = try(jsondecode(var.grant), var.grant)
......@@ -516,6 +518,7 @@ data "aws_iam_policy_document" "combined" {
var.attach_lb_log_delivery_policy ? data.aws_iam_policy_document.lb_log_delivery[0].json : "",
var.attach_require_latest_tls_policy ? data.aws_iam_policy_document.require_latest_tls[0].json : "",
var.attach_deny_insecure_transport_policy ? data.aws_iam_policy_document.deny_insecure_transport[0].json : "",
var.attach_inventory_destination_policy ? data.aws_iam_policy_document.inventory_destination_policy[0].json : "",
var.attach_policy ? var.policy : ""
])
}
......@@ -735,3 +738,102 @@ resource "aws_s3_bucket_metric" "this" {
}
}
}
resource "aws_s3_bucket_inventory" "this" {
for_each = { for k, v in var.inventory_configuration : k => v if local.create_bucket }
name = each.key
bucket = try(each.value.bucket, aws_s3_bucket.this[0].id)
included_object_versions = each.value.included_object_versions
enabled = try(each.value.enabled, true)
optional_fields = try(each.value.optional_fields, null)
destination {
bucket {
bucket_arn = try(each.value.destination.bucket_arn, aws_s3_bucket.this[0].arn)
format = try(each.value.destination.format, null)
account_id = try(each.value.destination.account_id, null)
prefix = try(each.value.destination.prefix, null)
dynamic "encryption" {
for_each = length(try(flatten([each.value.destination.encryption]), [])) == 0 ? [] : [true]
content {
dynamic "sse_kms" {
for_each = each.value.destination.encryption.encryption_type == "sse_kms" ? [true] : []
content {
key_id = try(each.value.destination.encryption.kms_key_id, null)
}
}
dynamic "sse_s3" {
for_each = each.value.destination.encryption.encryption_type == "sse_s3" ? [true] : []
content {
}
}
}
}
}
}
schedule {
frequency = each.value.frequency
}
dynamic "filter" {
for_each = length(try(flatten([each.value.filter]), [])) == 0 ? [] : [true]
content {
prefix = try(each.value.filter.prefix, null)
}
}
}
# Inventory destination bucket requires a bucket policy to allow source to PutObjects
# https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html#example-bucket-policies-use-case-9
data "aws_iam_policy_document" "inventory_destination_policy" {
count = local.create_bucket && var.attach_inventory_destination_policy ? 1 : 0
statement {
sid = "destinationInventoryPolicy"
effect = "Allow"
actions = [
"s3:PutObject",
]
resources = [
"${aws_s3_bucket.this[0].arn}/*",
]
principals {
type = "Service"
identifiers = ["s3.amazonaws.com"]
}
condition {
test = "ArnLike"
variable = "aws:SourceArn"
values = [
var.inventory_self_source_destination ? aws_s3_bucket.this[0].arn : var.inventory_source_bucket_arn
]
}
condition {
test = "StringEquals"
values = [
var.inventory_self_source_destination ? data.aws_caller_identity.current.id : var.inventory_source_account_id
]
variable = "aws:SourceAccount"
}
condition {
test = "StringEquals"
values = ["bucket-owner-full-control"]
variable = "s3:x-amz-acl"
}
}
}
......@@ -40,6 +40,12 @@ variable "attach_public_policy" {
default = true
}
variable "attach_inventory_destination_policy" {
description = "Controls if S3 bucket should have bucket inventory destination policy attached."
type = bool
default = false
}
variable "bucket" {
description = "(Optional, Forces new resource) The name of the bucket. If omitted, Terraform will assign a random, unique name."
type = string
......@@ -166,6 +172,30 @@ variable "metric_configuration" {
default = []
}
variable "inventory_configuration" {
description = "Map containing S3 inventory configuration."
type = any
default = {}
}
variable "inventory_source_account_id" {
description = "The inventory source account id."
type = string
default = null
}
variable "inventory_source_bucket_arn" {
description = "The inventory source bucket ARN."
type = string
default = null
}
variable "inventory_self_source_destination" {
description = "Whether or not the inventory source bucket is also the destination bucket."
type = bool
default = false
}
variable "object_lock_enabled" {
description = "Whether S3 bucket should have an Object Lock configuration enabled."
type = bool
......
......@@ -10,6 +10,7 @@ module "wrapper" {
attach_require_latest_tls_policy = try(each.value.attach_require_latest_tls_policy, var.defaults.attach_require_latest_tls_policy, false)
attach_policy = try(each.value.attach_policy, var.defaults.attach_policy, false)
attach_public_policy = try(each.value.attach_public_policy, var.defaults.attach_public_policy, true)
attach_inventory_destination_policy = try(each.value.attach_inventory_destination_policy, var.defaults.attach_inventory_destination_policy, false)
bucket = try(each.value.bucket, var.defaults.bucket, null)
bucket_prefix = try(each.value.bucket_prefix, var.defaults.bucket_prefix, null)
acl = try(each.value.acl, var.defaults.acl, null)
......@@ -31,6 +32,10 @@ module "wrapper" {
intelligent_tiering = try(each.value.intelligent_tiering, var.defaults.intelligent_tiering, {})
object_lock_configuration = try(each.value.object_lock_configuration, var.defaults.object_lock_configuration, {})
metric_configuration = try(each.value.metric_configuration, var.defaults.metric_configuration, [])
inventory_configuration = try(each.value.inventory_configuration, var.defaults.inventory_configuration, {})
inventory_source_account_id = try(each.value.inventory_source_account_id, var.defaults.inventory_source_account_id, null)
inventory_source_bucket_arn = try(each.value.inventory_source_bucket_arn, var.defaults.inventory_source_bucket_arn, null)
inventory_self_source_destination = try(each.value.inventory_self_source_destination, var.defaults.inventory_self_source_destination, false)
object_lock_enabled = try(each.value.object_lock_enabled, var.defaults.object_lock_enabled, false)
block_public_acls = try(each.value.block_public_acls, var.defaults.block_public_acls, false)
block_public_policy = try(each.value.block_public_policy, var.defaults.block_public_policy, false)
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment