AWS code-library high security documentation change
Summary
Added multiple security enhancements to the Athena/S3 tutorial script including AWS credential validation, file permission restrictions, input validation, S3 bucket encryption, public access blocking, secure random ID generation, timeout handling, and secure file cleanup.
Security assessment
The changes explicitly add security measures like credential validation (preventing unauthorized operations), umask restrictions (file permission hardening), input validation (preventing injection attacks), S3 encryption/access controls (data protection), and secure file handling. These directly address potential security weaknesses in the example implementation.
Diff
diff --git a/code-library/latest/ug/s3_example_athena_GettingStarted_061_section.md b/code-library/latest/ug/s3_example_athena_GettingStarted_061_section.md index bc2847169..62a37b48a 100644 --- a//code-library/latest/ug/s3_example_athena_GettingStarted_061_section.md +++ b//code-library/latest/ug/s3_example_athena_GettingStarted_061_section.md @@ -45 +45,12 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Set up logging + set -euo pipefail + + # Security: Validate AWS credentials are configured + if ! aws sts get-caller-identity &>/dev/null; then + echo "ERROR: AWS credentials not configured or invalid" + exit 1 + fi + + # Security: Restrict umask to prevent world-readable files + umask 0077 + + # Set up logging with restricted permissions @@ -46,0 +58,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + touch "$LOG_FILE" + chmod 600 "$LOG_FILE" @@ -56 +69 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ -n "$NAMED_QUERY_ID" ]; then + if [ -n "${NAMED_QUERY_ID:-}" ]; then @@ -59 +72 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ -n "$DATABASE_NAME" ]; then + if [ -n "${DATABASE_NAME:-}" ]; then @@ -61 +74 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ -n "$TABLE_NAME" ]; then + if [ -n "${TABLE_NAME:-}" ]; then @@ -65 +78 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ -n "$S3_BUCKET" ]; then + if [ -n "${S3_BUCKET:-}" ]; then @@ -73 +86,22 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Generate a random identifier for S3 bucket + # Security: Validate bucket name format + validate_bucket_name() { + local bucket_name="$1" + if [[ ! "$bucket_name" =~ ^[a-z0-9][a-z0-9.-]*[a-z0-9]$ ]] || [ ${#bucket_name} -lt 3 ] || [ ${#bucket_name} -gt 63 ]; then + return 1 + fi + return 0 + } + + # Security: Validate database and table names + validate_identifier() { + local identifier="$1" + if [[ ! "$identifier" =~ ^[a-zA-Z_][a-zA-Z0-9_]*$ ]]; then + return 1 + fi + return 0 + } + + # Security: Safely generate random identifier + if ! command -v openssl &>/dev/null; then + RANDOM_ID=$(head -c 6 /dev/urandom | od -An -tx1 | tr -d ' ') + else @@ -75 +109,11 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Check for shared prereq bucket + fi + + # Security: Validate random ID format + if [[ ! "$RANDOM_ID" =~ ^[a-f0-9]{12}$ ]]; then + handle_error "Failed to generate valid random ID" + fi + + # Check for shared prereq bucket with proper error handling + PREREQ_BUCKET="" + if aws cloudformation describe-stacks --stack-name tutorial-prereqs-bucket \ + --query 'Stacks[0].Outputs[?OutputKey==`BucketName`].OutputValue' --output text 2>/dev/null | grep -qv "^$"; then @@ -77,0 +122,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + fi + @@ -85,0 +132,5 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + + if ! validate_bucket_name "$S3_BUCKET"; then + handle_error "Invalid S3 bucket name: $S3_BUCKET" + fi + @@ -89,2 +140,10 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Get the current AWS region - AWS_REGION=$(aws configure get region) + if ! validate_identifier "$DATABASE_NAME"; then + handle_error "Invalid database name: $DATABASE_NAME" + fi + + if ! validate_identifier "$TABLE_NAME"; then + handle_error "Invalid table name: $TABLE_NAME" + fi + + # Get the current AWS region with validation + AWS_REGION=$(aws configure get region 2>/dev/null || echo "") @@ -95,0 +155,5 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + # Security: Validate region format - expanded regex for newer regions + if [[ ! "$AWS_REGION" =~ ^[a-z]{2}-[a-z]+-[0-9]{1}$ ]] && [[ ! "$AWS_REGION" =~ ^[a-z]+-[a-z]+-[0-9]{1}$ ]]; then + echo "WARNING: Region format may be invalid: $AWS_REGION" + fi + @@ -100,2 +164,3 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - CREATE_BUCKET_RESULT=$(aws s3 mb "s3://$S3_BUCKET" 2>&1) - if echo "$CREATE_BUCKET_RESULT" | grep -i "error"; then + if [ "$BUCKET_IS_SHARED" = false ]; then + CREATE_BUCKET_RESULT=$(aws s3 mb "s3://$S3_BUCKET" --region "$AWS_REGION" 2>&1) + if echo "$CREATE_BUCKET_RESULT" | grep -qi "error\|failed"; then @@ -104 +169,34 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - echo "$CREATE_BUCKET_RESULT" + + # Security: Enable S3 bucket encryption with KMS validation + echo "Enabling default encryption on S3 bucket..." + if ! aws s3api put-bucket-encryption \ + --bucket "$S3_BUCKET" \ + --server-side-encryption-configuration '{ + "Rules": [{ + "ApplyServerSideEncryptionByDefault": { + "SSEAlgorithm": "AES256" + } + }] + }' 2>&1; then + echo "Warning: Could not enable encryption on bucket" + fi + + # Security: Block public access + echo "Blocking public access to S3 bucket..." + if ! aws s3api put-public-access-block \ + --bucket "$S3_BUCKET" \ + --public-access-block-configuration \ + "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true" 2>&1; then + echo "Warning: Could not block public access on bucket" + fi + + # Security: Enable versioning for data protection + echo "Enabling versioning on S3 bucket..." + if ! aws s3api put-bucket-versioning \ + --bucket "$S3_BUCKET" \ + --versioning-configuration Status=Enabled 2>&1; then + echo "Warning: Could not enable versioning on bucket" + fi + + echo "S3 bucket created successfully: $S3_BUCKET" + fi @@ -110 +208,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" 2>&1) + --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" \ + --region "$AWS_REGION" 2>&1) @@ -112 +211 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if echo "$CREATE_DB_RESULT" | grep -i "error"; then + if echo "$CREATE_DB_RESULT" | grep -qi "error\|failed"; then @@ -116 +215,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - QUERY_ID=$(echo "$CREATE_DB_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + QUERY_ID=$(echo "$CREATE_DB_RESULT" | jq -r '.QueryExecutionId // empty' 2>/dev/null || echo "$CREATE_DB_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + if [ -z "$QUERY_ID" ]; then + handle_error "Failed to extract Query ID from database creation response" + fi @@ -121,2 +223,5 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - while true; do - QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" --query "QueryExecution.Status.State" --output text 2>&1) + WAIT_TIMEOUT=60 + ELAPSED=0 + while [ $ELAPSED -lt $WAIT_TIMEOUT ]; do + QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" \ + --query "QueryExecution.Status.State" --output text --region "$AWS_REGION" 2>&1) @@ -130,0 +236 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + ((ELAPSED+=2)) @@ -132,0 +239,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + if [ $ELAPSED -ge $WAIT_TIMEOUT ]; then + handle_error "Database creation timed out" + fi + @@ -135,2 +245,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - LIST_DB_RESULT=$(aws athena list-databases --catalog-name AwsDataCatalog 2>&1) - if echo "$LIST_DB_RESULT" | grep -i "error"; then + LIST_DB_RESULT=$(aws athena list-databases --catalog-name AwsDataCatalog --region "$AWS_REGION" 2>&1) + if echo "$LIST_DB_RESULT" | grep -qi "error\|failed"; then @@ -166 +276,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" 2>&1) + --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" \ + --region "$AWS_REGION" 2>&1) @@ -168 +279 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if echo "$CREATE_TABLE_RESULT" | grep -i "error"; then + if echo "$CREATE_TABLE_RESULT" | grep -qi "error\|failed"; then @@ -172 +283,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - QUERY_ID=$(echo "$CREATE_TABLE_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + QUERY_ID=$(echo "$CREATE_TABLE_RESULT" | jq -r '.QueryExecutionId // empty' 2>/dev/null || echo "$CREATE_TABLE_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + if [ -z "$QUERY_ID" ]; then + handle_error "Failed to extract Query ID from table creation response" + fi @@ -177,2 +291,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - while true; do - QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" --query "QueryExecution.Status.State" --output text 2>&1) + ELAPSED=0 + while [ $ELAPSED -lt $WAIT_TIMEOUT ]; do + QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" \ + --query "QueryExecution.Status.State" --output text --region "$AWS_REGION" 2>&1) @@ -186,0 +303 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + ((ELAPSED+=2)) @@ -188,0 +306,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + if [ $ELAPSED -ge $WAIT_TIMEOUT ]; then + handle_error "Table creation timed out" + fi +