AWS code-library documentation change
Summary
The file was completely rewritten from a single S3 bucket creation tutorial to a comprehensive collection of multiple AWS service tutorials including Amazon Athena, Amazon EMR, Amazon S3 (updated), Amazon SageMaker Feature Store, Amazon Textract, and AWS Config. Each tutorial includes complete bash scripts with setup, execution, and cleanup steps.
Security assessment
The change does not address a specific security vulnerability or incident. It is a major content expansion adding multiple new tutorials. However, the new S3 tutorial (third example) explicitly adds security documentation by including steps to configure default encryption (SSE-S3) and block all public access, which are security best practices. The SageMaker Feature Store tutorial also includes blocking public access to S3 buckets. These additions provide security guidance but are not in response to a security issue.
Diff
diff --git a/code-library/latest/ug/bash_2_s3_code_examples.md b/code-library/latest/ug/bash_2_s3_code_examples.md index eb724d49e..6520c3a2b 100644 --- a//code-library/latest/ug/bash_2_s3_code_examples.md +++ b//code-library/latest/ug/bash_2_s3_code_examples.md @@ -1050 +1050 @@ The following code example shows how to: - * Create an S3 bucket with unique naming and regional configuration + * Create an S3 bucket for query results @@ -1052 +1052 @@ The following code example shows how to: - * Configure bucket security settings including public access blocking + * Create a database @@ -1054 +1054 @@ The following code example shows how to: - * Enable versioning and default encryption for data protection + * Create a table @@ -1056 +1056 @@ The following code example shows how to: - * Upload objects with and without custom metadata + * Run a query @@ -1058 +1058 @@ The following code example shows how to: - * Download objects from the bucket to local storage + * Create and use named queries @@ -1060 +1060 @@ The following code example shows how to: - * Copy objects within the bucket to organize data in folders + * Clean up resources @@ -1062 +1061,0 @@ The following code example shows how to: - * List bucket contents and objects with specific prefixes @@ -1064 +1062,0 @@ The following code example shows how to: - * Add tags to buckets for resource management @@ -1066 +1064,1731 @@ The following code example shows how to: - * Clean up all resources including versioned objects + +**AWS CLI with Bash script** + + +###### Note + +There's more on GitHub. Find the complete example and learn how to set up and run in the [Sample developer tutorials](https://github.com/aws-samples/sample-developer-tutorials/tree/main/tuts/061-amazon-athena-gs) repository. + + + #!/bin/bash + + # Amazon Athena Getting Started Script + # This script demonstrates how to use Amazon Athena with AWS CLI + # It creates a database, table, runs queries, and manages named queries + + # Set up logging + LOG_FILE="athena-tutorial.log" + exec > >(tee -a "$LOG_FILE") 2>&1 + + echo "Starting Amazon Athena Getting Started Tutorial..." + echo "Logging to $LOG_FILE" + + # Function to handle errors + handle_error() { + echo "ERROR: $1" + echo "Resources created:" + if [ -n "$NAMED_QUERY_ID" ]; then + echo "- Named Query: $NAMED_QUERY_ID" + fi + if [ -n "$DATABASE_NAME" ]; then + echo "- Database: $DATABASE_NAME" + if [ -n "$TABLE_NAME" ]; then + echo "- Table: $TABLE_NAME in $DATABASE_NAME" + fi + fi + if [ -n "$S3_BUCKET" ]; then + echo "- S3 Bucket: $S3_BUCKET" + fi + + echo "Exiting..." + exit 1 + } + + # Generate a random identifier for S3 bucket + RANDOM_ID=$(openssl rand -hex 6) + S3_BUCKET="athena-${RANDOM_ID}" + DATABASE_NAME="mydatabase" + TABLE_NAME="cloudfront_logs" + + # Get the current AWS region + AWS_REGION=$(aws configure get region) + if [ -z "$AWS_REGION" ]; then + AWS_REGION="us-east-1" + echo "No AWS region found in configuration, defaulting to $AWS_REGION" + fi + + echo "Using AWS Region: $AWS_REGION" + + # Create S3 bucket for Athena query results + echo "Creating S3 bucket for Athena query results: $S3_BUCKET" + CREATE_BUCKET_RESULT=$(aws s3 mb "s3://$S3_BUCKET" 2>&1) + if echo "$CREATE_BUCKET_RESULT" | grep -i "error"; then + handle_error "Failed to create S3 bucket: $CREATE_BUCKET_RESULT" + fi + echo "$CREATE_BUCKET_RESULT" + + # Step 1: Create a database + echo "Step 1: Creating Athena database: $DATABASE_NAME" + CREATE_DB_RESULT=$(aws athena start-query-execution \ + --query-string "CREATE DATABASE IF NOT EXISTS $DATABASE_NAME" \ + --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" 2>&1) + + if echo "$CREATE_DB_RESULT" | grep -i "error"; then + handle_error "Failed to create database: $CREATE_DB_RESULT" + fi + + QUERY_ID=$(echo "$CREATE_DB_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + echo "Database creation query ID: $QUERY_ID" + + # Wait for database creation to complete + echo "Waiting for database creation to complete..." + while true; do + QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" --query "QueryExecution.Status.State" --output text 2>&1) + if [ "$QUERY_STATUS" = "SUCCEEDED" ]; then + echo "Database creation completed successfully." + break + elif [ "$QUERY_STATUS" = "FAILED" ] || [ "$QUERY_STATUS" = "CANCELLED" ]; then + handle_error "Database creation failed with status: $QUERY_STATUS" + fi + echo "Database creation in progress, status: $QUERY_STATUS" + sleep 2 + done + + # Verify the database was created + echo "Verifying database creation..." + LIST_DB_RESULT=$(aws athena list-databases --catalog-name AwsDataCatalog 2>&1) + if echo "$LIST_DB_RESULT" | grep -i "error"; then + handle_error "Failed to list databases: $LIST_DB_RESULT" + fi + echo "$LIST_DB_RESULT" + + # Step 2: Create a table + echo "Step 2: Creating Athena table: $TABLE_NAME" + # Replace the region placeholder in the S3 location + CREATE_TABLE_QUERY="CREATE EXTERNAL TABLE IF NOT EXISTS $DATABASE_NAME.$TABLE_NAME ( + \`Date\` DATE, + Time STRING, + Location STRING, + Bytes INT, + RequestIP STRING, + Method STRING, + Host STRING, + Uri STRING, + Status INT, + Referrer STRING, + os STRING, + Browser STRING, + BrowserVersion STRING + ) + ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe' + WITH SERDEPROPERTIES ( + \"input.regex\" = \"^(?!#)([^ ]+)\\\\s+([^ ]+)\\\\s+([^ ]+)\\\\s+([^ ]+)\\\\s+([^ ]+)\\\\s+([^ ]+)\\\\s+([^ ]+)\\\\s+([^ ]+)\\\\s+([^ ]+)\\\\s+([^ ]+)\\\\s+[^\\\\(]+[\\\\(]([^\\\\;]+).*\\\\%20([^\\\\/]+)[\\\\/](.*)$\" + ) LOCATION 's3://athena-examples-us-east-1/cloudfront/plaintext/';" + + CREATE_TABLE_RESULT=$(aws athena start-query-execution \ + --query-string "$CREATE_TABLE_QUERY" \ + --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" 2>&1) + + if echo "$CREATE_TABLE_RESULT" | grep -i "error"; then + handle_error "Failed to create table: $CREATE_TABLE_RESULT" + fi + + QUERY_ID=$(echo "$CREATE_TABLE_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + echo "Table creation query ID: $QUERY_ID" + + # Wait for table creation to complete + echo "Waiting for table creation to complete..." + while true; do + QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" --query "QueryExecution.Status.State" --output text 2>&1) + if [ "$QUERY_STATUS" = "SUCCEEDED" ]; then + echo "Table creation completed successfully." + break + elif [ "$QUERY_STATUS" = "FAILED" ] || [ "$QUERY_STATUS" = "CANCELLED" ]; then + handle_error "Table creation failed with status: $QUERY_STATUS" + fi + echo "Table creation in progress, status: $QUERY_STATUS" + sleep 2 + done + + # Verify the table was created + echo "Verifying table creation..." + LIST_TABLE_RESULT=$(aws athena list-table-metadata \ + --catalog-name AwsDataCatalog \ + --database-name "$DATABASE_NAME" 2>&1) + if echo "$LIST_TABLE_RESULT" | grep -i "error"; then + handle_error "Failed to list tables: $LIST_TABLE_RESULT" + fi + echo "$LIST_TABLE_RESULT" + + # Step 3: Query data + echo "Step 3: Running a query on the table..." + QUERY="SELECT os, COUNT(*) count + FROM $DATABASE_NAME.$TABLE_NAME + WHERE date BETWEEN date '2014-07-05' AND date '2014-08-05' + GROUP BY os" + + QUERY_RESULT=$(aws athena start-query-execution \ + --query-string "$QUERY" \ + --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" 2>&1) + + if echo "$QUERY_RESULT" | grep -i "error"; then + handle_error "Failed to run query: $QUERY_RESULT"