AWS code-library medium security documentation change
Summary
Enhanced AWS Glue bash script example with improved security practices including input validation, secure resource cleanup, error handling, temporary file management, and AWS credential verification
Security assessment
The changes include concrete security improvements: 1) Added input validation and AWS CLI prerequisite checks to prevent misconfiguration risks 2) Implemented secure temporary file handling with chmod 600 and shred for sensitive data 3) Added AWS credential verification using STS to prevent unauthorized operations 4) Enhanced resource cleanup to avoid orphaned resources 5) Replaced insecure random generation with cryptographic RNG (openssl rand). These directly address security weaknesses in script execution and resource management.
Diff
diff --git a/code-library/latest/ug/bash_2_glue_code_examples.md b/code-library/latest/ug/bash_2_glue_code_examples.md index a10e3ec12..54562d84d 100644 --- a//code-library/latest/ug/bash_2_glue_code_examples.md +++ b//code-library/latest/ug/bash_2_glue_code_examples.md @@ -50,0 +51,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + # Cost improvements: Reduced API calls, optimized queries, eliminated redundant operations + # Reliability improvements: Enhanced error handling, input validation, resource tracking + + set -euo pipefail @@ -60 +64 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - UNIQUE_ID=$(LC_ALL=C tr -dc 'a-z0-9' < /dev/urandom | head -c 8) + UNIQUE_ID=$(openssl rand -hex 4) @@ -62,0 +67 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + TABLE_INPUT_FILE="table-input-${UNIQUE_ID}.json" @@ -65 +70,10 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - CREATED_RESOURCES=() + declare -a CREATED_RESOURCES=() + + # Set default region if not provided + AWS_REGION="${AWS_REGION:-us-east-1}" + + # Flag to track if database was successfully created + DATABASE_CREATED=false + + # Trap to ensure cleanup on exit + trap cleanup_resources EXIT @@ -70,2 +84 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - echo "ERROR: $1 failed. Exiting." - cleanup_resources + echo "ERROR: $1 failed." >&2 @@ -77,0 +91 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + local exit_code=$? @@ -90 +104,7 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - aws glue delete-table --database-name "$DB_NAME" --name "$resource_name" + if [ "$DATABASE_CREATED" = true ]; then + aws glue delete-table \ + --database-name "$DB_NAME" \ + --name "$resource_name" \ + --region "$AWS_REGION" \ + 2>/dev/null || echo "Warning: Failed to delete table $resource_name" + fi @@ -93 +113,4 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - aws glue delete-database --name "$resource_name" + aws glue delete-database \ + --name "$resource_name" \ + --region "$AWS_REGION" \ + 2>/dev/null || echo "Warning: Failed to delete database $resource_name" @@ -96 +119 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - echo "Unknown resource type: $resource_type" + echo "Unknown resource type: $resource_type" >&2 @@ -100,0 +124,9 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + # Clean up temporary files securely + if [ -f "$TABLE_INPUT_FILE" ]; then + if command -v shred &> /dev/null; then + shred -vfz -n 3 "$TABLE_INPUT_FILE" 2>/dev/null || rm -f "$TABLE_INPUT_FILE" + else + rm -f "$TABLE_INPUT_FILE" + fi + fi + @@ -101,0 +134,38 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + exit $exit_code + } + + # Function to validate prerequisites + validate_prerequisites() { + # Validate AWS CLI is available + if ! command -v aws &> /dev/null; then + echo "ERROR: AWS CLI is not installed or not in PATH" >&2 + exit 1 + fi + + # Validate AWS CLI version + local AWS_CLI_VERSION + AWS_CLI_VERSION=$(aws --version 2>&1 | cut -d' ' -f1 | cut -d'/' -f2 | cut -d'.' -f1) + if [ "$AWS_CLI_VERSION" -lt 1 ]; then + echo "ERROR: AWS CLI is required" >&2 + exit 1 + fi + + # Validate jq is available for JSON validation + if ! command -v jq &> /dev/null; then + echo "ERROR: jq is not installed or not in PATH" >&2 + exit 1 + fi + + # Validate AWS credentials and get account identity in single call (cost optimization) + local CALLER_IDENTITY + CALLER_IDENTITY=$(aws sts get-caller-identity --region "$AWS_REGION" --query 'Account' --output text 2>/dev/null) || { + echo "ERROR: Failed to get AWS caller identity. Check credentials and permissions." >&2 + exit 1 + } + + if [ -z "$CALLER_IDENTITY" ] || [ "$CALLER_IDENTITY" == "None" ]; then + echo "ERROR: Unable to determine AWS account identity" >&2 + exit 1 + fi + echo "Using AWS Account: $CALLER_IDENTITY" + echo "Using Region: $AWS_REGION" @@ -104 +174,2 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Step 1: Create a database + # Function to create database with verification + create_database() { @@ -106,2 +177,10 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - aws glue create-database --database-input "{\"Name\":\"$DB_NAME\",\"Description\":\"Database for AWS Glue tutorial\"}" - check_status "Creating database" + + if ! aws glue create-database \ + --database-input "Name=$DB_NAME,Description=Database for AWS Glue tutorial" \ + --region "$AWS_REGION" \ + --output json > /dev/null 2>&1; then + echo "ERROR: Failed to create database $DB_NAME" >&2 + exit 1 + fi + + DATABASE_CREATED=true @@ -109,0 +189 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + } @@ -111,8 +191,5 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Verify the database was created - echo "Verifying database creation..." - DB_VERIFY=$(aws glue get-database --name "$DB_NAME" --query 'Database.Name' --output text) - check_status "Verifying database" - - if [ "$DB_VERIFY" != "$DB_NAME" ]; then - echo "ERROR: Database verification failed. Expected $DB_NAME but got $DB_VERIFY" - cleanup_resources + # Function to prepare table input JSON + prepare_table_input() { + # Create a temporary JSON file for table input with restricted permissions + if ! touch "$TABLE_INPUT_FILE" 2>/dev/null; then + echo "ERROR: Failed to create temporary file $TABLE_INPUT_FILE" >&2 @@ -121 +197,0 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - echo "Database verification successful." @@ -123,2 +199,5 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Step 2: Create a table - echo "Step 2: Creating a table named $TABLE_NAME in database $DB_NAME" + if ! chmod 600 "$TABLE_INPUT_FILE" 2>/dev/null; then + echo "ERROR: Failed to set permissions on $TABLE_INPUT_FILE" >&2 + rm -f "$TABLE_INPUT_FILE" + exit 1 + fi @@ -126,3 +205 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Create a temporary JSON file for table input - TABLE_INPUT_FILE="table-input-${UNIQUE_ID}.json" - cat > "$TABLE_INPUT_FILE" << EOF + cat > "$TABLE_INPUT_FILE" << 'EOF' @@ -130 +207 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - "Name": "$TABLE_NAME", + "Name": "TABLE_NAME_PLACEHOLDER", @@ -174,4 +251,6 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - aws glue create-table --database-name "$DB_NAME" --table-input file://"$TABLE_INPUT_FILE" - check_status "Creating table" - CREATED_RESOURCES+=("table:$TABLE_NAME") - echo "Table $TABLE_NAME created successfully." + # Replace placeholder with actual table name + if ! sed -i "s/TABLE_NAME_PLACEHOLDER/$TABLE_NAME/g" "$TABLE_INPUT_FILE" 2>/dev/null; then + echo "ERROR: Failed to substitute table name in JSON file" >&2 + rm -f "$TABLE_INPUT_FILE" + exit 1 + fi @@ -179 +258,3 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Clean up the temporary file + # Validate JSON syntax before using it + if ! jq empty "$TABLE_INPUT_FILE" 2>/dev/null; then + echo "ERROR: Invalid JSON in table input file" >&2 @@ -180,0 +262,7 @@ There's more on GitHub. Find the complete example and learn how to set up and ru + exit 1 + fi + } + + # Function to create table + create_table() { + echo "Step 2: Creating a table named $TABLE_NAME in database $DB_NAME" @@ -182,4 +270 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Verify the table was created - echo "Verifying table creation..." - TABLE_VERIFY=$(aws glue get-table --database-name "$DB_NAME" --name "$TABLE_NAME" --query 'Table.Name' --output text) - check_status "Verifying table" + prepare_table_input @@ -187,3 +272,7 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - if [ "$TABLE_VERIFY" != "$TABLE_NAME" ]; then - echo "ERROR: Table verification failed. Expected $TABLE_NAME but got $TABLE_VERIFY" - cleanup_resources + if ! aws glue create-table \ + --database-name "$DB_NAME" \ + --table-input "file://${TABLE_INPUT_FILE}" \ + --region "$AWS_REGION" \ + --output json > /dev/null 2>&1; then + echo "ERROR: Failed to create table $TABLE_NAME" >&2 + rm -f "$TABLE_INPUT_FILE" @@ -192 +280,0 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - echo "Table verification successful." @@ -194 +282,6 @@ There's more on GitHub. Find the complete example and learn how to set up and ru - # Step 3: Get table details + CREATED_RESOURCES+=("table:$TABLE_NAME") + echo "Table $TABLE_NAME created successfully." + } + + # Function to get and display table details