AWS AmazonS3 documentation change
Summary
Restructured server access logging documentation to focus on delivery options (CloudWatch Logs vs S3 bucket), added pricing details, removed legacy configuration steps, and introduced comparison table of capabilities.
Security assessment
The changes document CloudWatch Logs' KMS encryption capability (security feature) but show no evidence of addressing a specific vulnerability. The removal of ACL/permission details doesn't indicate a security fix, and the rewording ('understand security posture') is descriptive rather than incident-related.
Diff
diff --git a/AmazonS3/latest/userguide/ServerLogs.md b/AmazonS3/latest/userguide/ServerLogs.md index f6fc8422c..0276d440b 100644 --- a//AmazonS3/latest/userguide/ServerLogs.md +++ b//AmazonS3/latest/userguide/ServerLogs.md @@ -7 +7 @@ -How do I enable log delivery?Log object key formatHow are logs delivered?Best-effort server log deliveryBucket logging status changes take effect over time +Delivery optionsBest-effort server log deliveryBucket logging status changes take effect over timePricing @@ -11 +11,3 @@ How do I enable log delivery?Log object key formatHow are logs delivered?Best-ef -Server access logging provides detailed records for the requests that are made to a bucket. Server access logs are useful for many applications. For example, access log information can be useful in security and access audits. This information can also help you learn about your customer base and understand your Amazon S3 bill. +Server access logging provides detailed records for the requests that are made to a bucket. Server access logs can be used for operational analysis, to understand your security posture, and to charge back users. + +Amazon S3 periodically collects access log records, consolidates them, and delivers them to your chosen destination. You can deliver logs to Amazon CloudWatch Logs or to an Amazon S3 general purpose bucket. @@ -17,49 +19 @@ Server access logs don't record information about wrong-Region redirect errors f -## How do I enable log delivery? - -To enable log delivery, perform the following basic steps. For details, see [Enabling Amazon S3 server access logging](./enable-server-access-logging.html). - - 1. **Provide the name of the destination bucket** (also known as a _target bucket_). This bucket is where you want Amazon S3 to save the access logs as objects. Both the source and destination buckets must be in the same AWS Region and owned by the same account. The destination bucket must not have an S3 Object Lock default retention period configuration. The destination bucket must also not have Requester Pays enabled. - -You can have logs delivered to any bucket that you own that is in the same Region as the source bucket, including the source bucket itself. But for simpler log management, we recommend that you save access logs in a different bucket. - -When your source bucket and destination bucket are the same bucket, additional logs are created for the logs that are written to the bucket, which creates an infinite loop of logs. We do not recommend doing this because it could result in a small increase in your storage billing. In addition, the extra logs about logs might make it harder to find the log that you are looking for. - -If you choose to save access logs in the source bucket, we recommend that you specify a destination prefix (also known as a _target prefix_) for all log object keys. When you specify a prefix, all the log object names begin with a common string, which makes the log objects easier to identify. - - 2. **(Optional) Assign a destination prefix to all Amazon S3 log object keys.** The destination prefix (also known as a _target prefix_) makes it simpler for you to locate the log objects. For example, if you specify the prefix value `logs/`, each log object that Amazon S3 creates begins with the `logs/` prefix in its key, for example: - - logs/2013-11-01-21-32-16-E568B2907131C0C0 - -If you specify the prefix value `logs`, the log object appears as follows: - - logs2013-11-01-21-32-16-E568B2907131C0C0 - -[Prefixes](https://docs.aws.amazon.com/general/latest/gr/glos-chap.html#keyprefix) are also useful to distinguish between source buckets when multiple buckets log to the same destination bucket. - -This prefix can also help when you delete the logs. For example, you can set a lifecycle configuration rule for Amazon S3 to delete objects with a specific prefix. For more information, see [Deleting Amazon S3 log files](./deleting-log-files-lifecycle.html). - - 3. **(Optional) Set permissions __so that others can access the generated logs.** By default, only the bucket owner always has full access to the log objects. If your destination bucket uses the Bucket owner enforced setting for S3 Object Ownership to disable access control lists (ACLs), you can't grant permissions in destination grants (also known as _target grants_) that use ACLs. However, you can update your bucket policy for the destination bucket to grant access to others. For more information, see [Identity and Access Management for Amazon S3](./security-iam.html) and [Permissions for log delivery](./enable-server-access-logging.html#grant-log-delivery-permissions-general). - - 4. **(Optional) Set a log object key format for the log files.** You have two options for the log object key format (also known as the _target object key format_): - - * **Non-date-based partitioning** – This is the original log object key format. If you choose this format, the log file key format appears as follows: - - [DestinationPrefix][YYYY]-[MM]-[DD]-[hh]-[mm]-[ss]-[UniqueString] - -For example, if you specify `logs/` as the prefix, your log objects are named like this: - - logs/2013-11-01-21-32-16-E568B2907131C0C0 - - * **Date-based partitioning** – If you choose date-based partitioning, you can choose the event time or delivery time for the log file as the date source used in the log format. This format makes it easier to query the logs. - -If you choose date-based partitioning, the log file key format appears as follows: - - [DestinationPrefix][SourceAccountId]/[SourceRegion]/[SourceBucket]/[YYYY]/[MM]/[DD]/[YYYY]-[MM]-[DD]-[hh]-[mm]-[ss]-[UniqueString] - -For example, if you specify `logs/` as the target prefix, your log objects are named like this: - - logs/123456789012/us-west-2/amzn-s3-demo-source-bucket/2023/03/01/2023-03-01-21-32-16-E568B2907131C0C0 - -For delivery time delivery, the time in the log file names corresponds to the delivery time for the log files. - -For event time delivery, the year, month, and day correspond to the day on which the event occurred, and the hour, minutes and seconds are set to `00` in the key. The logs delivered in these log files are for a specific day only. +## Delivery options @@ -67 +21 @@ For event time delivery, the year, month, and day correspond to the day on which -If you're configuring your logs through the AWS Command Line Interface (AWS CLI), AWS SDKs, or Amazon S3 REST API, use `TargetObjectKeyFormat` to specify the log object key format. To specify non-date-based partitioning, use `SimplePrefix`. To specify data-based partitioning, use `PartitionedPrefix`. If you use `PartitionedPrefix`, use `PartitionDateSource` to specify either `EventTime` or `DeliveryTime`. +You can deliver server access logs to two destinations. You can use one or both. @@ -69 +23 @@ If you're configuring your logs through the AWS Command Line Interface (AWS CLI) -For `SimplePrefix`, the log file key format appears as follows: + * **Amazon CloudWatch Logs** – Delivers logs in structured format to a CloudWatch Logs log group. You can query logs with CloudWatch Logs Insights, aggregate logs across accounts and Regions, and encrypt logs with AWS KMS. From the log group, you can also deliver logs to Amazon S3 in JSON or Apache Parquet format, or mirror logs to S3 Tables in Apache Iceberg format for SQL analytics. CloudWatch vended logs ingestion rates apply. @@ -71 +25 @@ For `SimplePrefix`, the log file key format appears as follows: - [TargetPrefix][YYYY]-[MM]-[DD]-[hh]-[mm]-[ss]-[UniqueString] + * **Amazon S3 general purpose bucket** – Delivers logs in space-delimited text format to an Amazon S3 bucket you specify. There is no charge for log delivery. You pay only for the storage of the log files. The destination bucket must be in the same AWS Region and AWS account as the source bucket. @@ -73 +26,0 @@ For `SimplePrefix`, the log file key format appears as follows: -For `PartitionedPrefix` with event time or delivery time, the log file key format appears as follows: @@ -75 +27,0 @@ For `PartitionedPrefix` with event time or delivery time, the log file key forma - [TargetPrefix][SourceAccountId]/[SourceRegion]/[SourceBucket]/[YYYY]/[MM]/[DD]/[YYYY]-[MM]-[DD]-[hh]-[mm]-[ss]-[UniqueString] @@ -77,0 +30 @@ For `PartitionedPrefix` with event time or delivery time, the log file key forma +The following table compares the two delivery options. @@ -79,39 +32,11 @@ For `PartitionedPrefix` with event time or delivery time, the log file key forma - -## Log object key format - -Amazon S3 uses the following object key formats for the log objects that it uploads in the destination bucket: - - * **Non-date-based partitioning** – This is the original log object key format. If you choose this format, the log file key format appears as follows: - - [DestinationPrefix][YYYY]-[MM]-[DD]-[hh]-[mm]-[ss]-[UniqueString] - - * **Date-based partitioning** – If you choose date-based partitioning, you can choose the event time or delivery time for the log file as the date source used in the log format. This format makes it easier to query the logs. - -If you choose date-based partitioning, the log file key format appears as follows: - - [DestinationPrefix][SourceAccountId]/[SourceRegion]/[SourceBucket]/[YYYY]/[MM]/[DD]/[YYYY]-[MM]-[DD]-[hh]-[mm]-[ss]-[UniqueString] - - - - -In the log object key, `YYYY`, `MM`, `DD`, `hh`, `mm`, and `ss` are the digits of the year, month, day, hour, minute, and seconds (respectively). These dates and times are in Coordinated Universal Time (UTC). - -A log file delivered at a specific time can contain records written at any point before that time. There is no way to know whether all log records for a certain time interval have been delivered or not. - -The `UniqueString` component of the key is there to prevent overwriting of files. It has no meaning, and log processing software should ignore it. - -## How are logs delivered? - -Amazon S3 periodically collects access log records, consolidates the records in log files, and then uploads log files to your destination bucket as log objects. If you enable logging on multiple source buckets that identify the same destination bucket, the destination bucket will have access logs for all those source buckets. However, each log object reports access log records for a specific source bucket. - -Amazon S3 uses a special log delivery account to write server access logs. These writes are subject to the usual access control restrictions. We recommend that you update the bucket policy on the destination bucket to grant access to the logging service principal (`logging.s3.amazonaws.com`) for access log delivery. You can also grant access for access log delivery to the S3 log delivery group through your bucket access control list (ACL). However, granting access to the S3 log delivery group by using your bucket ACL is not recommended. - -When you enable server access logging and grant access for access log delivery through your destination bucket policy, you must update the policy to allow `s3:PutObject` access for the logging service principal. If you use the Amazon S3 console to enable server access logging, the console automatically updates the destination bucket policy to grant these permissions to the logging service principal. For more information about granting permissions for server access log delivery, see [Permissions for log delivery](./enable-server-access-logging.html#grant-log-delivery-permissions-general). - -###### Note - -S3 does not support delivery of CloudTrail logs or server access logs to the requester or the bucket owner for VPC endpoint requests when the VPC endpoint policy denies them or for requests that fail before the VPC policy is evaluated. - -###### Bucket owner enforced setting for S3 Object Ownership - -If the destination bucket uses the Bucket owner enforced setting for Object Ownership, ACLs are disabled and no longer affect permissions. You must update the bucket policy on the destination bucket to grant access to the logging service principal. For more information about Object Ownership, see [Grant access to the S3 log delivery group for server access logging](./object-ownership-migrating-acls-prerequisites.html#object-ownership-server-access-logs). +Comparing server access log delivery options Capability | Amazon CloudWatch Logs | Amazon S3 general purpose bucket +---|---|--- +Destinations | CloudWatch Logs log group, Amazon S3 (JSON or Parquet), or Amazon Data Firehose. The CloudWatch Logs log group delivery can be extended to mirror logs to S3 Tables in Iceberg format. | Amazon S3 bucket (space-delimited text) +Log format | Structured JSON (log group), JSON or Apache Parquet (Amazon S3), Apache Iceberg (S3 Tables) | Space-delimited text +Querying | CloudWatch Logs Insights (log group), Amazon Athena or Spark (S3 Tables) | Amazon Athena +SQL analytics via S3 Tables | Yes (automatic Iceberg mirror) | No (requires ETL) +Cross-account aggregation | Yes (Amazon S3 and Firehose destinations support cross-account delivery) | No (same account only) +Cross-Region aggregation | Yes (Amazon S3 and Firehose destinations support cross-Region delivery) | No (same Region only) +AWS KMS encryption | Yes | No (SSE-S3 only) +Setup | CloudWatch Logs APIs or Amazon S3 console | Amazon S3 API (`PutBucketLogging`) or console +Delivery latency | Within a few hours | Within a few hours @@ -129,5 +54 @@ Because of the best-effort nature of server logging, your usage reports might in -Changes to the logging status of a bucket take time to actually affect the delivery of log files. For example, if you enable logging for a bucket, some requests made in the following hour might be logged, and others might not. Suppose that you change the destination bucket for logging from bucket A to bucket B. For the next hour, some logs might continue to be delivered to bucket A, whereas others might be delivered to the new destination bucket B. In all cases, the new settings eventually take effect without any further action on your part. - -For more information about logging and log files, see the following sections: - -###### Topics +Changes to the logging status of a bucket take time to actually affect the delivery of log files. For example, if you enable logging for a bucket, some requests made in the following hour might be logged, and others might not. Suppose that you change the destination bucket for logging from bucket A to bucket B. For the next hour, some logs might continue to be delivered to bucket A, whereas others might be delivered to the new destination bucket B. Similarly, if you change the CloudWatch Logs log group used for delivery, some logs might continue to be delivered to the previous log group during the transition. In all cases, the new settings eventually take effect without any further action on your part. @@ -135 +56 @@ For more information about logging and log files, see the following sections: - * [Enabling Amazon S3 server access logging](./enable-server-access-logging.html) +## Pricing @@ -137 +58 @@ For more information about logging and log files, see the following sections: - * [Amazon S3 server access log format](./LogFormat.html) +The cost of server access logging depends on which delivery option you use. @@ -139 +60 @@ For more information about logging and log files, see the following sections: - * [Deleting Amazon S3 log files](./deleting-log-files-lifecycle.html) + * **Amazon CloudWatch Logs** – You pay for ingestion at CloudWatch vended logs rates, with volume-based tiered pricing. CloudWatch Logs compresses logs before storage, so stored volume is significantly less than ingested volume. If you enable the S3 Tables mirror, there is no additional charge for storage or table maintenance. You pay only for query requests at S3 Tables pricing. For current rates, see [CloudWatch pricing](https://aws.amazon.com/cloudwatch/pricing/). @@ -141 +62 @@ For more information about logging and log files, see the following sections: - * [Using Amazon S3 server access logs to identify requests](./using-s3-access-logs-to-identify-requests.html) + * **Amazon S3 general purpose bucket** – There is no charge for log delivery. You pay only for the storage of the log files at standard Amazon S3 storage rates. We do not assess data-transfer charges for log file delivery, but we do charge the normal data-transfer rate for accessing the log files. For current rates, see [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/). @@ -143 +63,0 @@ For more information about logging and log files, see the following sections: - * [Troubleshoot server access logging](./troubleshooting-server-access-logging.html) @@ -146,0 +67 @@ For more information about logging and log files, see the following sections: +For more information about delivering and using server access logs, see the following sections. @@ -156 +77 @@ Identifying S3 requests -Enabling server access logging +Delivering to CloudWatch Logs