AWS code-library high security documentation change
Summary
Added multiple security hardening measures to AWS Athena bash script including AWS credentials verification, strict file permissions, input validation, S3 bucket security features (encryption, public access blocking, versioning), secure cleanup, and timeout handling
Security assessment
Changes explicitly implement security controls: validates credentials (prevents unintended actions), restricts file permissions (protects sensitive data), validates inputs (prevents injection attacks), enables S3 encryption and blocking public access (protects data at rest), securely erases files, and adds timeouts (prevents hanging processes).
Diff
diff --git a/code-library/latest/ug/bash_2_athena_code_examples.md b/code-library/latest/ug/bash_2_athena_code_examples.md index ed93a834b..a8cef1ee1 100644 --- a//code-library/latest/ug/bash_2_athena_code_examples.md +++ b//code-library/latest/ug/bash_2_athena_code_examples.md @@ -59 +59,12 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Set up logging + set -euo pipefail + + # Security: Validate AWS credentials are configured + if ! aws sts get-caller-identity &>/dev/null; then + echo "ERROR: AWS credentials not configured or invalid" + exit 1 + fi + + # Security: Restrict umask to prevent world-readable files + umask 0077 + + # Set up logging with restricted permissions @@ -60,0 +72,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + touch "$LOG_FILE" + chmod 600 "$LOG_FILE" @@ -70 +83 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ -n "$NAMED_QUERY_ID" ]; then + if [ -n "${NAMED_QUERY_ID:-}" ]; then @@ -73 +86 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ -n "$DATABASE_NAME" ]; then + if [ -n "${DATABASE_NAME:-}" ]; then @@ -75 +88 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ -n "$TABLE_NAME" ]; then + if [ -n "${TABLE_NAME:-}" ]; then @@ -79 +92 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ -n "$S3_BUCKET" ]; then + if [ -n "${S3_BUCKET:-}" ]; then @@ -87 +100,22 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Generate a random identifier for S3 bucket + # Security: Validate bucket name format + validate_bucket_name() { + local bucket_name="$1" + if [[ ! "$bucket_name" =~ ^[a-z0-9][a-z0-9.-]*[a-z0-9]$ ]] || [ ${#bucket_name} -lt 3 ] || [ ${#bucket_name} -gt 63 ]; then + return 1 + fi + return 0 + } + + # Security: Validate database and table names + validate_identifier() { + local identifier="$1" + if [[ ! "$identifier" =~ ^[a-zA-Z_][a-zA-Z0-9_]*$ ]]; then + return 1 + fi + return 0 + } + + # Security: Safely generate random identifier + if ! command -v openssl &>/dev/null; then + RANDOM_ID=$(head -c 6 /dev/urandom | od -An -tx1 | tr -d ' ') + else @@ -89 +123,11 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Check for shared prereq bucket + fi + + # Security: Validate random ID format + if [[ ! "$RANDOM_ID" =~ ^[a-f0-9]{12}$ ]]; then + handle_error "Failed to generate valid random ID" + fi + + # Check for shared prereq bucket with proper error handling + PREREQ_BUCKET="" + if aws cloudformation describe-stacks --stack-name tutorial-prereqs-bucket \ + --query 'Stacks[0].Outputs[?OutputKey==`BucketName`].OutputValue' --output text 2>/dev/null | grep -qv "^$"; then @@ -91,0 +136,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + fi + @@ -99,0 +146,5 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + + if ! validate_bucket_name "$S3_BUCKET"; then + handle_error "Invalid S3 bucket name: $S3_BUCKET" + fi + @@ -103,2 +154,10 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Get the current AWS region - AWS_REGION=$(aws configure get region) + if ! validate_identifier "$DATABASE_NAME"; then + handle_error "Invalid database name: $DATABASE_NAME" + fi + + if ! validate_identifier "$TABLE_NAME"; then + handle_error "Invalid table name: $TABLE_NAME" + fi + + # Get the current AWS region with validation + AWS_REGION=$(aws configure get region 2>/dev/null || echo "") @@ -109,0 +169,5 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + # Security: Validate region format - expanded regex for newer regions + if [[ ! "$AWS_REGION" =~ ^[a-z]{2}-[a-z]+-[0-9]{1}$ ]] && [[ ! "$AWS_REGION" =~ ^[a-z]+-[a-z]+-[0-9]{1}$ ]]; then + echo "WARNING: Region format may be invalid: $AWS_REGION" + fi + @@ -114,2 +178,3 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - CREATE_BUCKET_RESULT=$(aws s3 mb "s3://$S3_BUCKET" 2>&1) - if echo "$CREATE_BUCKET_RESULT" | grep -i "error"; then + if [ "$BUCKET_IS_SHARED" = false ]; then + CREATE_BUCKET_RESULT=$(aws s3 mb "s3://$S3_BUCKET" --region "$AWS_REGION" 2>&1) + if echo "$CREATE_BUCKET_RESULT" | grep -qi "error\|failed"; then @@ -118 +183,34 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - echo "$CREATE_BUCKET_RESULT" + + # Security: Enable S3 bucket encryption with KMS validation + echo "Enabling default encryption on S3 bucket..." + if ! aws s3api put-bucket-encryption \ + --bucket "$S3_BUCKET" \ + --server-side-encryption-configuration '{ + "Rules": [{ + "ApplyServerSideEncryptionByDefault": { + "SSEAlgorithm": "AES256" + } + }] + }' 2>&1; then + echo "Warning: Could not enable encryption on bucket" + fi + + # Security: Block public access + echo "Blocking public access to S3 bucket..." + if ! aws s3api put-public-access-block \ + --bucket "$S3_BUCKET" \ + --public-access-block-configuration \ + "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true" 2>&1; then + echo "Warning: Could not block public access on bucket" + fi + + # Security: Enable versioning for data protection + echo "Enabling versioning on S3 bucket..." + if ! aws s3api put-bucket-versioning \ + --bucket "$S3_BUCKET" \ + --versioning-configuration Status=Enabled 2>&1; then + echo "Warning: Could not enable versioning on bucket" + fi + + echo "S3 bucket created successfully: $S3_BUCKET" + fi @@ -124 +222,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" 2>&1) + --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" \ + --region "$AWS_REGION" 2>&1) @@ -126 +225 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if echo "$CREATE_DB_RESULT" | grep -i "error"; then + if echo "$CREATE_DB_RESULT" | grep -qi "error\|failed"; then @@ -130 +229,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - QUERY_ID=$(echo "$CREATE_DB_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + QUERY_ID=$(echo "$CREATE_DB_RESULT" | jq -r '.QueryExecutionId // empty' 2>/dev/null || echo "$CREATE_DB_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + if [ -z "$QUERY_ID" ]; then + handle_error "Failed to extract Query ID from database creation response" + fi @@ -135,2 +237,5 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - while true; do - QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" --query "QueryExecution.Status.State" --output text 2>&1) + WAIT_TIMEOUT=60 + ELAPSED=0 + while [ $ELAPSED -lt $WAIT_TIMEOUT ]; do + QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" \ + --query "QueryExecution.Status.State" --output text --region "$AWS_REGION" 2>&1) @@ -144,0 +250 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + ((ELAPSED+=2)) @@ -146,0 +253,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + if [ $ELAPSED -ge $WAIT_TIMEOUT ]; then + handle_error "Database creation timed out" + fi + @@ -149,2 +259,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - LIST_DB_RESULT=$(aws athena list-databases --catalog-name AwsDataCatalog 2>&1) - if echo "$LIST_DB_RESULT" | grep -i "error"; then + LIST_DB_RESULT=$(aws athena list-databases --catalog-name AwsDataCatalog --region "$AWS_REGION" 2>&1) + if echo "$LIST_DB_RESULT" | grep -qi "error\|failed"; then @@ -180 +290,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" 2>&1) + --result-configuration "OutputLocation=s3://$S3_BUCKET/output/" \ + --region "$AWS_REGION" 2>&1) @@ -182 +293 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if echo "$CREATE_TABLE_RESULT" | grep -i "error"; then + if echo "$CREATE_TABLE_RESULT" | grep -qi "error\|failed"; then @@ -186 +297,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - QUERY_ID=$(echo "$CREATE_TABLE_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + QUERY_ID=$(echo "$CREATE_TABLE_RESULT" | jq -r '.QueryExecutionId // empty' 2>/dev/null || echo "$CREATE_TABLE_RESULT" | grep -o '"QueryExecutionId": "[^"]*' | cut -d'"' -f4) + if [ -z "$QUERY_ID" ]; then + handle_error "Failed to extract Query ID from table creation response" + fi @@ -191,2 +305,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - while true; do - QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" --query "QueryExecution.Status.State" --output text 2>&1) + ELAPSED=0 + while [ $ELAPSED -lt $WAIT_TIMEOUT ]; do + QUERY_STATUS=$(aws athena get-query-execution --query-execution-id "$QUERY_ID" \ + --query "QueryExecution.Status.State" --output text --region "$AWS_REGION" 2>&1) @@ -200,0 +317 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + ((ELAPSED+=2)) @@ -202,0 +320,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + if [ $ELAPSED -ge $WAIT_TIMEOUT ]; then + handle_error "Table creation timed out" + fi +