AWS code-library documentation change
Summary
Added comprehensive security hardening to multiple AWS service examples (Athena, EMR, S3, Textract) including credential validation, input sanitization, file permission controls, S3 bucket security features (encryption, public access blocking, versioning), secure random ID generation, and secure cleanup procedures.
Security assessment
The changes add security best practices and hardening measures to existing code examples but do not address a specific disclosed vulnerability or security incident. Changes include: 1) AWS credential validation before execution, 2) Input validation for bucket names and identifiers, 3) File permission controls (umask 0077, chmod 600), 4) S3 bucket security features (server-side encryption, public access blocking, versioning), 5) Secure random ID generation, 6) Sensitive data redaction in logs, 7) Secure file deletion using shred, 8) Timeout controls for operations. These are proactive security improvements rather than fixes for specific security issues.
Diff
diff --git a/code-library/latest/ug/bash_2_s3_code_examples.md b/code-library/latest/ug/bash_2_s3_code_examples.md index 002570bde..b72f52c02 100644 --- a//code-library/latest/ug/bash_2_s3_code_examples.md +++ b//code-library/latest/ug/bash_2_s3_code_examples.md @@ -1079 +1079,12 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Set up logging + set -euo pipefail + + # Security: Validate AWS credentials are configured + if ! aws sts get-caller-identity &>/dev/null; then + echo "ERROR: AWS credentials not configured or invalid" + exit 1 + fi + + # Security: Restrict umask to prevent world-readable files + umask 0077 + + # Set up logging with restricted permissions @@ -1080,0 +1092,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + touch "$LOG_FILE" + chmod 600 "$LOG_FILE" @@ -1090 +1103 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ -n "$NAMED_QUERY_ID" ]; then + if [ -n "${NAMED_QUERY_ID:-}" ]; then @@ -1093 +1106 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ -n "$DATABASE_NAME" ]; then + if [ -n "${DATABASE_NAME:-}" ]; then @@ -1095 +1108 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ -n "$TABLE_NAME" ]; then + if [ -n "${TABLE_NAME:-}" ]; then @@ -1099 +1112 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ -n "$S3_BUCKET" ]; then + if [ -n "${S3_BUCKET:-}" ]; then @@ -1107 +1120,22 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Generate a random identifier for S3 bucket + # Security: Validate bucket name format + validate_bucket_name() { + local bucket_name="$1" + if [[ ! "$bucket_name" =~ ^[a-z0-9][a-z0-9.-]*[a-z0-9]$ ]] || [ ${#bucket_name} -lt 3 ] || [ ${#bucket_name} -gt 63 ]; then + return 1 + fi + return 0 + } + + # Security: Validate database and table names + validate_identifier() { + local identifier="$1" + if [[ ! "$identifier" =~ ^[a-zA-Z_][a-zA-Z0-9_]*$ ]]; then + return 1 + fi + return 0 + } + + # Security: Safely generate random identifier + if ! command -v openssl &>/dev/null; then + RANDOM_ID=$(head -c 6 /dev/urandom | od -An -tx1 | tr -d ' ') + else @@ -1109 +1143,11 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Check for shared prereq bucket + fi + + # Security: Validate random ID format + if [[ ! "$RANDOM_ID" =~ ^[a-f0-9]{12}$ ]]; then + handle_error "Failed to generate valid random ID" + fi + + # Check for shared prereq bucket with proper error handling + PREREQ_BUCKET="" + if aws cloudformation describe-stacks --stack-name tutorial-prereqs-bucket \ + --query 'Stacks[0].Outputs[?OutputKey==`BucketName`].OutputValue' --output text 2>/dev/null | grep -qv "^$"; then @@ -1111,0 +1156,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + fi + @@ -1119,0 +1166,5 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + + if ! validate_bucket_name "$S3_BUCKET"; then + handle_error "Invalid S3 bucket name: $S3_BUCKET" + fi + @@ -1123,2 +1174,10 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Get the current AWS region - AWS_REGION=$(aws configure get region) + if ! validate_identifier "$DATABASE_NAME"; then + handle_error "Invalid database name: $DATABASE_NAME" + fi + + if ! validate_identifier "$TABLE_NAME"; then + handle_error "Invalid table name: $TABLE_NAME" + fi + + # Get the current AWS region with validation + AWS_REGION=$(aws configure get region 2>/dev/null || echo "") @@ -1129,0 +1189,5 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + # Security: Validate region format - expanded regex for newer regions + if [[ ! "$AWS_REGION" =~ ^[a-z]{2}-[a-z]+-[0-9]{1}$ ]] && [[ ! "$AWS_REGION" =~ ^[a-z]+-[a-z]+-[0-9]{1}$ ]]; then + echo "WARNING: Region format may be invalid: $AWS_REGION" + fi + @@ -1134,2 +1198,3 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - CREATE_BUCKET_RESULT=$(aws s3 mb "s3://$S3_BUCKET" 2>&1) - if echo "$CREATE_BUCKET_RESULT" | grep -i "error"; then + if [ "$BUCKET_IS_SHARED" = false ]; then + CREATE_BUCKET_RESULT=$(aws s3 mb "s3://$S3_BUCKET" --region "$AWS_REGION" 2>&1) + if echo "$CREATE_BUCKET_RESULT" | grep -qi "error\|failed"; then @@ -1138 +1203,34 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - echo "$CREATE_BUCKET_RESULT" + + # Security: Enable S3 bucket encryption with KMS validation + echo "Enabling default encryption on S3 bucket..." + if ! aws s3api put-bucket-encryption \ + --bucket "$S3_BUCKET" \ + --server-side-encryption-configuration '{ + "Rules": [{ + "ApplyServerSideEncryptionByDefault": { + "SSEAlgorithm": "AES256" + } + }] + }' 2>&1; then + echo "Warning: Could not enable encryption on bucket" + fi + + # Security: Block public access + echo "Blocking public access to S3 bucket..." + if ! aws s3api put-public-access-block \ + --bucket "$S3_BUCKET" \ + --public-access-block-configuration \ + "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true" 2>&1; then + echo "Warning: Could not block public access on bucket" + fi + + # Security: Enable versioning for data protection + echo "Enabling versioning on S3 bucket..." + if ! aws s3api put-bucket-versioning \ + --bucket "$S3_BUCKET" \ + --versioning-configuration Status=Enabled 2>&1; then + echo "Warning: Could not enable versioning on bucket" + fi + + echo "S3 bucket created successfully: $S3_BUCKET" + fi @@ -1144 +1242,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" 2>&1) + --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" \ + --region "$AWS_REGION" 2>&1) @@ -1146 +1245 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if echo "$CREATE_DB_RESULT" | grep -i "error"; then + if echo "$CREATE_DB_RESULT" | grep -qi "error\|failed"; then @@ -1150 +1249,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - QUERY_ID=$(echo "$CREATE_DB_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + QUERY_ID=$(echo "$CREATE_DB_RESULT" | jq -r '.QueryExecutionId // empty' 2>/dev/null || echo "$CREATE_DB_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + if [ -z "$QUERY_ID" ]; then + handle_error "Failed to extract Query ID from database creation response" + fi @@ -1155,2 +1257,5 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - while true; do - QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" --query "QueryExecution.Status.State" --output text 2>&1) + WAIT_TIMEOUT=60 + ELAPSED=0 + while [ $ELAPSED -lt $WAIT_TIMEOUT ]; do + QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" \ + --query "QueryExecution.Status.State" --output text --region "$AWS_REGION" 2>&1) @@ -1164,0 +1270 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + ((ELAPSED+=2)) @@ -1166,0 +1273,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + if [ $ELAPSED -ge $WAIT_TIMEOUT ]; then + handle_error "Database creation timed out" + fi + @@ -1169,2 +1279,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - LIST_DB_RESULT=$(aws athena list-databases --catalog-name AwsDataCatalog 2>&1) - if echo "$LIST_DB_RESULT" | grep -i "error"; then + LIST_DB_RESULT=$(aws athena list-databases --catalog-name AwsDataCatalog --region "$AWS_REGION" 2>&1) + if echo "$LIST_DB_RESULT" | grep -qi "error\|failed"; then @@ -1200 +1310,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" 2>&1) + --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" \ + --region "$AWS_REGION" 2>&1) @@ -1202 +1313 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if echo "$CREATE_TABLE_RESULT" | grep -i "error"; then + if echo "$CREATE_TABLE_RESULT" | grep -qi "error\|failed"; then @@ -1206 +1317,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - QUERY_ID=$(echo "$CREATE_TABLE_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + QUERY_ID=$(echo "$CREATE_TABLE_RESULT" | jq -r '.QueryExecutionId // empty' 2>/dev/null || echo "$CREATE_TABLE_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + if [ -z "$QUERY_ID" ]; then + handle_error "Failed to extract Query ID from table creation response" + fi @@ -1211,2 +1325,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - while true; do - QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" --query "QueryExecution.Status.State" --output text 2>&1) + ELAPSED=0 + while [ $ELAPSED -lt $WAIT_TIMEOUT ]; do + QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" \ + --query "QueryExecution.Status.State" --output text --region "$AWS_REGION" 2>&1) @@ -1220,0 +1337 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + ((ELAPSED+=2)) @@ -1222,0 +1340,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + if [ $ELAPSED -ge $WAIT_TIMEOUT ]; then + handle_error "Table creation timed out" + fi +