AWS guidance documentation change
Summary
Updated documentation for AWS Data Exports solution to change from cross-account S3 replication to direct cross-account delivery architecture, restructured content, updated authors section, and clarified deployment steps.
Security assessment
The changes primarily involve architectural improvements (direct S3 delivery vs. replication) and documentation restructuring. There is no evidence of addressing a specific security vulnerability, incident, or weakness. The changes improve clarity and operational procedures but don't introduce new security features or address security flaws.
Diff
diff --git a/guidance/latest/cloud-intelligence-dashboards/data-exports.md b/guidance/latest/cloud-intelligence-dashboards/data-exports.md index 66de99490..7ba1d782f 100644 --- a//guidance/latest/cloud-intelligence-dashboards/data-exports.md +++ b//guidance/latest/cloud-intelligence-dashboards/data-exports.md @@ -5 +5 @@ -AuthorsIntroductionCommon Use CasesSupported AWS Data Export typesArchitectureCostsBefore you startPrerequisitesDeploymentUpdating StackUsageTeardownFeedback & Support +IntroductionCommon Use CasesSupported AWS Data Export typesArchitectureCostsBefore you startPrerequisitesDeploymentUpdating StackUsageTeardownAuthorsFeedback & Support @@ -9,13 +8,0 @@ AuthorsIntroductionCommon Use CasesSupported AWS Data Export typesArchitectureCo -## Authors - - * Zach Erdman, Senior Product Manager, AWS - - * Petro Kashlikov, Senior Solutions Architect, AWS - - * Iakov Gan, Senior Solutions Architect, AWS - - * Yuriy Prykhodko, Principal Technical Account Manager, AWS - - - - @@ -24 +11 @@ AuthorsIntroductionCommon Use CasesSupported AWS Data Export typesArchitectureCo -The [AWS Data Exports](https://docs.aws.amazon.com/cur/latest/userguide/what-is-data-exports.html) service allows you to create exports of various types of billing and cost management data to an Amazon S3 bucket in your local AWS account. This page introduces a simple [Cloud Formation Template](https://github.com/aws-solutions-library-samples/cloud-intelligence-dashboards-data-collection/blob/main/data-exports/deploy/data-exports-aggregation.yaml) that can automate the following tasks: +The [AWS Data Exports](https://docs.aws.amazon.com/cur/latest/userguide/what-is-data-exports.html) service allows you to create exports of various types of billing and cost management data. This page introduces a simple [CloudFormation Template](https://github.com/aws-solutions-library-samples/cloud-intelligence-dashboards-data-collection/blob/main/data-exports/deploy/data-exports-aggregation.yaml) that can automate the following tasks: @@ -26,3 +13 @@ The [AWS Data Exports](https://docs.aws.amazon.com/cur/latest/userguide/what-is- - * **Create AWS Data Exports** : Set up exports for billing and cost management data to be delivered to an Amazon S3 bucket in your account. - - * **Cross-Account Replication** : Replicate the exported data to dedicated Data Collection AWS account for centralized analysis or auditing purposes. + * **Create AWS Data Exports with cross-account delivery** : Set up exports for billing and cost management data with direct delivery to an Amazon S3 bucket in a centralized Data Collection account. @@ -65 +50 @@ This solution covers the following use cases: -The same stack must be installed the Destination (Data Collection) account and also in one or many Source account(s). +The CloudFormation stack must be installed in the Destination (Data Collection) account and also in one or many Source account(s). @@ -67 +52 @@ The same stack must be installed the Destination (Data Collection) account and a - * `Destination (Data Collection) Account` is used to consolidate, analyze or/and visualize the data. Typically FinOps or CCoE team owns this account. + * `Destination (Data Collection) Account` is used to consolidate, analyze, and visualize the data. Typically the FinOps or CCoE team owns this account. @@ -69 +54 @@ The same stack must be installed the Destination (Data Collection) account and a - * `Source Account` is the account where stack will create AWS Data Exports. Typically it is one or many Management (Payer) Accounts, but can be also one or many Linked Accounts. + * `Source Account` is the account where the stack will create AWS Data Exports. Typically it is one or many Management (Payer) Accounts, but can also be one or many Linked Accounts. @@ -76 +61,9 @@ The same stack must be installed the Destination (Data Collection) account and a -The CloudFormation template installed in 2 accounts does the following: +The CloudFormation template installed in each account does the following: + +**In the Destination (Data Collection) Account:** + + 1. Creates an Amazon S3 bucket with an S3 bucket policy allowing write access for the AWS Data Exports service from Source account. + + 2. Creates AWS Glue Database, Amazon Athena Tables, and AWS Glue Crawlers. + + @@ -78 +70,0 @@ The CloudFormation template installed in 2 accounts does the following: - 1. Creates data exports for one or more supported export types, and a local Amazon S3 bucket in the Source Account. @@ -80 +72 @@ The CloudFormation template installed in 2 accounts does the following: - 2. Sets up a replication from a local Amazon S3 Bucket in a Source Account to another Aggregation bucket in a Data Collection Account. +**In the Source (Management or Linked) Account:** @@ -82 +74 @@ The CloudFormation template installed in 2 accounts does the following: - 3. Creates AWS Glue Database, Amazon Athena Tables and AWS Glue Crawlers in the Data Collection Account. + 1. Creates data exports for one or more supported export types with remote delivery directly to the S3 bucket in the Destination (Data Collection) Account. @@ -119 +111 @@ If you plan to activate Data Export for Cost Optimization Hub you need to activa -The deployment process consists of 2 steps. The first is in Destination/Data Collection account, and the 2nd in one or multiple Source Accounts. +The deployment process consists of 2 required steps and 1 optional step. First, create the destination infrastructure in the Data Collection account. Then, create the data exports in one or multiple Source Accounts. Step 3 is only required if you already have Cloud Intelligence Dashboards deployed and want to migrate them to use the new Data Exports as a data source. @@ -123 +115 @@ The deployment process consists of 2 steps. The first is in Destination/Data Col -### Step 1 of 3. (In Destination/Data Collection Account) Create Destination for Data Exports aggregation +### Step 1 of 3. (In Destination/Data Collection Account) Create Destination for Data Exports @@ -132 +124 @@ The deployment process consists of 2 steps. The first is in Destination/Data Col -[  ](https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?&templateURL=https://aws-managed-cost-intelligence-dashboards.s3.amazonaws.com/cfn/data-exports-aggregation.yaml&stackName=CID-DataExports-Destination¶m_ManageCUR2=yes¶m_ManageCOH=yes¶m_ManageCarbon=yes¶m_DestinationAccountId=REPLACE%20WITH%20DATA%20COLLECTION%20ACCOUNT%20ID¶m_SourceAccountIds=PUT%20HERE%20PAYER%20ACCOUNT%20ID) +[  ](https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?&templateURL=https://aws-managed-cost-intelligence-dashboards.s3.amazonaws.com/cfn/data-exports-aggregation.yaml&stackName=CID-DataExports-Destination¶m_ManageCUR2=yes¶m_ManageCOH=yes¶m_ManageCarbon=yes¶m_LegacyLocalBucket=no¶m_DestinationAccountId=REPLACE%20WITH%20DATA%20COLLECTION%20ACCOUNT%20ID¶m_SourceAccountIds=PUT%20HERE%20PAYER%20ACCOUNT%20ID) @@ -138 +130 @@ The deployment process consists of 2 steps. The first is in Destination/Data Col - 3. Choose the exports to manage. For selected types in Destination account the stack will create Athena Tables and allow replication from Management Accounts. + 3. Choose the exports to manage. For selected types, the Destination account stack will create the S3 bucket, Athena Tables, and allow delivery from Source Accounts. @@ -140 +132 @@ The deployment process consists of 2 steps. The first is in Destination/Data Col - 4. Enter your **Source Account IDs** parameter as a comma separated list of all accounts that must deliver AWS Data Exports. In a rare case if you need the Data Collection account to produce AWS Data Export(for testing or mono account deploy), you need to specify this Account Id first in the list, and skip the "Step 2 of 3". + 4. Enter your **Source Account IDs** parameter as a comma separated list of all accounts that must deliver AWS Data Exports. In a rare case if you need the Data Collection account to produce AWS Data Export (for testing or single account deployment), you need to specify this Account Id first in the list, and skip "Step 2 of 3". @@ -149 +141 @@ The deployment process consists of 2 steps. The first is in Destination/Data Col -### Step 2 of 3. (In Management/Payer/Source Account) Create AWS Data Exports and Amazon S3 Replication Rules +### Step 2 of 3. (In Management/Payer/Source Account) Create AWS Data Exports @@ -156 +148 @@ The deployment process consists of 2 steps. The first is in Destination/Data Col -[  ](https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?&templateURL=https://aws-managed-cost-intelligence-dashboards.s3.amazonaws.com/cfn/data-exports-aggregation.yaml&stackName=CID-DataExports-Source¶m_ManageCUR2=yes¶m_ManageCOH=yes¶m_ManageCarbon=yes¶m_DestinationAccountId=REPLACE%20WITH%20DATA%20COLLECTION%20ACCOUNT%20IDs¶m_SourceAccountIds=) +[  ](https://console.aws.amazon.com/cloudformation/home#/stacks/create/review?&templateURL=https://aws-managed-cost-intelligence-dashboards.s3.amazonaws.com/cfn/data-exports-aggregation.yaml&stackName=CID-DataExports-Source¶m_ManageCUR2=yes¶m_ManageCOH=yes¶m_ManageCarbon=yes¶m_LegacyLocalBucket=no¶m_DestinationAccountId=REPLACE%20WITH%20DATA%20COLLECTION%20ACCOUNT%20IDs¶m_SourceAccountIds=) @@ -162 +154 @@ The deployment process consists of 2 steps. The first is in Destination/Data Col - 3. Choose the exports to manage. For selected types in Source account the stack will create exports set up replication to the Data Collection Accounts. Please keep the choice consistent with the same configuration in the Data Collection Account. + 3. Choose the exports to manage. For selected types, the Source account stack will create data exports with direct delivery to the Destination Account S3 bucket. Please keep the choice consistent with the same configuration in the Data Collection Account. @@ -173 +165 @@ The deployment process consists of 2 steps. The first is in Destination/Data Col -It will typically take about 24 hours for the first delivery of AWS Data Exports replication to the Destination Account, but it might take up to 72 hours (3 days). You can continue with the dashboards deployment however data will appear on the dashboards the next day after the first data delivery. +It will typically take about 24 hours for the first delivery of AWS Data Exports to the Destination Account, but it might take up to 72 hours (3 days). You can continue with the dashboards deployment however data will appear on the dashboards the next day after the first data delivery. @@ -205 +197 @@ Make sure you create the case from your Source Accounts (Typically Management/Pa -### Step 3 of 3. (In Destination/Data Collection Account) Allow Quick Sight access to Database and Bucket +### Step 3 of 3. (Optional) (In Destination/Data Collection Account) Allow QuickSight access to Database and Bucket @@ -207 +199,5 @@ Make sure you create the case from your Source Accounts (Typically Management/Pa -In order to allow Amazon Quick Sight access to Bucket and Database table you need to extend the role that Quick Sight uses. +###### Note + +This step is only required if you already have Cloud Intelligence Dashboards deployed and want to migrate them to use the new Data Exports as a data source. If you are deploying dashboards for the first time, you can skip this step — the necessary permissions will be configured automatically during the dashboard deployment. + +In order to allow Amazon QuickSight access to the Data Exports bucket and Glue database, you need to extend the role that QuickSight uses. @@ -328 +324,7 @@ Follow [this guide](https://docs.aws.amazon.com/quicksight/latest/user/troublesh -### Adding Exports +### Update stack or add new exports + +To update the stack or enable additional export types, follow these steps: + + 1. Download [the latest version](https://aws-managed-cost-intelligence-dashboards.s3.amazonaws.com/cfn/data-exports-aggregation.yaml) of the CloudFormation template. + + 2. Update the stack in the **Destination (Data Collection) Account** first. If you want to add new export types, set the corresponding parameters to `yes`. @@ -330 +332,8 @@ Follow [this guide](https://docs.aws.amazon.com/quicksight/latest/user/troublesh -Update Stack in Data Collection Account as well as in all Source Account with [the latest version](https://aws-managed-cost-intelligence-dashboards.s3.amazonaws.com/cfn/data-exports-aggregation.yaml). Set parameter that corresponds to the needed Data Collection to 'yes'. + 3. Then update the stack in each **Source Account** with the same template and matching parameter configuration. + + + + +###### Important + +Always update the Destination Account stack before updating Source Account stacks. @@ -334 +343,8 @@ Update Stack in Data Collection Account as well as in all Source Account with [t -If you need to add Source Account, you need first to login to your Data Collection account and update Source Account Ids list. Then you can create a stack in the new Source Account as per deployment guide. +To add a new Source Account: + + 1. Login to your **Data Collection Account** and update the existing Destination stack to include the new Source Account ID in the **Source Account IDs** parameter. + + 2. Then create a new stack in the Source Account following Step 2 of 3. + + + @@ -347 +363,14 @@ If you need to add Source Account, you need first to login to your Data Collecti -If you need to delete the stack you can do that by deleting all stacks in all Source accounts and Destination one. Before deleting stacks you might need to empty Amazon S3 buckets. +If you need to delete the stack, you can do that by deleting all stacks in all Source accounts and the Destination account. Before deleting the Destination stack you might need to empty the Amazon S3 bucket. + +## Authors + + * Yuriy Prykhodko, Principal Technical Account Manager, AWS + + * Petro Kashlikov, Senior Solutions Architect, AWS + + * Zach Erdman, Senior Product Manager, AWS + + * Iakov Gan, Ex-Amazonian + + +