AWS athena documentation change
Summary
Updated WAF log table schema and partition projection configuration, including simplified struct syntax, changed partitioning column from date/region to log_time with minute granularity, added new fields like ja4fingerprint, and updated example paths
Security assessment
Changes focus on schema updates, syntax improvements, and partition configuration changes. No evidence of addressing security vulnerabilities or weaknesses. New fields like ja4fingerprint relate to TLS fingerprinting but are schema additions rather than security documentation.
Diff
diff --git a/athena/latest/ug/create-waf-table-partition-projection.md b/athena/latest/ug/create-waf-table-partition-projection.md index 40d4f3593..be4339a06 100644 --- a//athena/latest/ug/create-waf-table-partition-projection.md +++ b//athena/latest/ug/create-waf-table-partition-projection.md @@ -9 +9 @@ Because AWS WAF logs have a known structure whose partition scheme you can speci -The following example `CREATE TABLE` statement automatically uses partition projection on AWS WAF logs from a specified date until the present for four different AWS regions. The `PARTITION BY` clause in this example partitions by region and by date, but you can modify this according to your requirements. Modify the fields as necessary to match your log output. In the `LOCATION` and `storage.location.template` clauses, replace the `bucket` and `accountID` placeholders with values that identify the Amazon S3 bucket location of your AWS WAF logs. For `projection.day.range`, replace `2021`/`01`/`01` with the starting date that you want to use. After you run the query successfully, you can query the table. You do not have to run `ALTER TABLE ADD PARTITION` to load the partitions. +The following example `CREATE TABLE` statement automatically uses partition projection on AWS WAF logs from a specified date until the present for four different AWS regions. The `PARTITION BY` clause in this example partitions by region and by date, but you can modify this according to your requirements. Modify the fields as necessary to match your log output. In the `LOCATION` and `storage.location.template` clauses, replace the `amzn-s3-demo-bucket` and `AWS_ACCOUNT_NUMBER` placeholders with values that identify the Amazon S3 bucket location of your AWS WAF logs. For `projection.day.range`, replace `2021`/`01`/`01` with the starting date that you want to use. After you run the query successfully, you can query the table. You do not have to run `ALTER TABLE ADD PARTITION` to load the partitions. @@ -12 +12 @@ The following example `CREATE TABLE` statement automatically uses partition proj - CREATE EXTERNAL TABLE `waf_logs`( + CREATE EXTERNAL TABLE `waf_logs_partition_projection`( @@ -19,8 +19 @@ The following example `CREATE TABLE` statement automatically uses partition proj - `terminatingrulematchdetails` array < - struct < - conditiontype: string, - sensitivitylevel: string, - location: string, - matcheddata: array < string > - > - >, + `terminatingrulematchdetails` array<struct<conditiontype:string,sensitivitylevel:string,location:string,matcheddata:array<string>>>, @@ -29,76 +22,4 @@ The following example `CREATE TABLE` statement automatically uses partition proj - `rulegrouplist` array < - struct < - rulegroupid: string, - terminatingrule: struct < - ruleid: string, - action: string, - rulematchdetails: array < - struct < - conditiontype: string, - sensitivitylevel: string, - location: string, - matcheddata: array < string > - > - > - >, - nonterminatingmatchingrules: array < - struct < - ruleid: string, - action: string, - overriddenaction: string, - rulematchdetails: array < - struct < - conditiontype: string, - sensitivitylevel: string, - location: string, - matcheddata: array < string > - > - >, - challengeresponse: struct < - responsecode: string, - solvetimestamp: string - >, - captcharesponse: struct < - responsecode: string, - solvetimestamp: string - > - > - >, - excludedrules: string - > - >, - `ratebasedrulelist` array < - struct < - ratebasedruleid: string, - limitkey: string, - maxrateallowed: int - > - >, - `nonterminatingmatchingrules` array < - struct < - ruleid: string, - action: string, - rulematchdetails: array < - struct < - conditiontype: string, - sensitivitylevel: string, - location: string, - matcheddata: array < string > - > - >, - challengeresponse: struct < - responsecode: string, - solvetimestamp: string - >, - captcharesponse: struct < - responsecode: string, - solvetimestamp: string - > - > - >, - `requestheadersinserted` array < - struct < - name: string, - value: string - > - >, + `rulegrouplist` array<struct<rulegroupid:string,terminatingrule:struct<ruleid:string,action:string,rulematchdetails:array<struct<conditiontype:string,sensitivitylevel:string,location:string,matcheddata:array<string>>>>,nonterminatingmatchingrules:array<struct<ruleid:string,action:string,overriddenaction:string,rulematchdetails:array<struct<conditiontype:string,sensitivitylevel:string,location:string,matcheddata:array<string>>>,challengeresponse:struct<responsecode:string,solvetimestamp:string>,captcharesponse:struct<responsecode:string,solvetimestamp:string>>>,excludedrules:string>>, + `ratebasedrulelist` array<struct<ratebasedruleid:string,limitkey:string,maxrateallowed:int>>, + `nonterminatingmatchingrules` array<struct<ruleid:string,action:string,rulematchdetails:array<struct<conditiontype:string,sensitivitylevel:string,location:string,matcheddata:array<string>>>,challengeresponse:struct<responsecode:string,solvetimestamp:string>,captcharesponse:struct<responsecode:string,solvetimestamp:string>>>, + `requestheadersinserted` array<struct<name:string,value:string>>, @@ -106,31 +27,6 @@ The following example `CREATE TABLE` statement automatically uses partition proj - `httprequest` struct < - clientip: string, - country: string, - headers: array < - struct < - name: string, - value: string - > - >, - uri: string, - args: string, - httpversion: string, - httpmethod: string, - requestid: string - >, - `labels` array < - struct < - name: string - > - >, - `captcharesponse` struct < - responsecode: string, - solvetimestamp: string, - failureReason: string - >, - `challengeresponse` struct < - responsecode: string, - solvetimestamp: string, - failureReason: string - >, - `ja3Fingerprint` string, + `httprequest` struct<clientip:string,country:string,headers:array<struct<name:string,value:string>>,uri:string,args:string,httpversion:string,httpmethod:string,requestid:string,fragment:string,scheme:string,host:string>, + `labels` array<struct<name:string>>, + `captcharesponse` struct<responsecode:string,solvetimestamp:string,failurereason:string>, + `challengeresponse` struct<responsecode:string,solvetimestamp:string,failurereason:string>, + `ja3fingerprint` string, + `ja4fingerprint` string, @@ -139,2 +35 @@ The following example `CREATE TABLE` statement automatically uses partition proj - `requestbodysizeinspectedbywaf` int - ) + `requestbodysizeinspectedbywaf` int) @@ -142,2 +37 @@ The following example `CREATE TABLE` statement automatically uses partition proj - `region` string, - `date` string) + `log_time` string) @@ -151 +45 @@ The following example `CREATE TABLE` statement automatically uses partition proj - 's3://amzn-s3-demo-bucket/AWSLogs/accountID/WAFLogs/region/DOC-EXAMPLE-WEBACL/' + 's3://amzn-s3-demo-bucket/AWSLogs/AWS_ACCOUNT_NUMBER/WAFLogs/cloudfront/testui/' @@ -154,8 +48,6 @@ The following example `CREATE TABLE` statement automatically uses partition proj - 'projection.region.type' = 'enum', - 'projection.region.values' = 'us-east-1,us-west-2,eu-central-1,eu-west-1', - 'projection.date.type' = 'date', - 'projection.date.range' = '2021/01/01,NOW', - 'projection.date.format' = 'yyyy/MM/dd', - 'projection.date.interval' = '1', - 'projection.date.interval.unit' = 'DAYS', - 'storage.location.template' = 's3://amzn-s3-demo-bucket/AWSLogs/accountID/WAFLogs/${region}/DOC-EXAMPLE-WEBACL/${date}/') + 'projection.log_time.format'='yyyy/MM/dd/HH/mm', + 'projection.log_time.interval'='1', + 'projection.log_time.interval.unit'='minutes', + 'projection.log_time.range'='2025/01/01/00/00,NOW', + 'projection.log_time.type'='date', + 'storage.location.template'='s3://amzn-s3-demo-bucket/AWSLogs/AWS_ACCOUNT_NUMBER/WAFLogs/cloudfront/testui/${log_time}') @@ -168 +60 @@ The format of the path in the `LOCATION` clause in the example is standard but c - s3://amzn-s3-demo-bucket/AWSLogs/12345678910/WAFLogs/cloudfront/cloudfronyt/2022/08/08/17/55/ + s3://amzn-s3-demo-bucket/AWSLogs/AWS_ACCOUNT_NUMBER/WAFLogs/cloudfront/cloudfronyt/2025/01/01/00/00/ @@ -182 +74 @@ AWS WAF -Create a table without partitioning +Use manual partition