AWS AmazonS3 documentation change
Summary
Comprehensive restructuring of S3 Batch Operations troubleshooting guide with expanded error explanations, added best practices, detailed troubleshooting steps, and removal of redundant error types from initial list
Security assessment
While the changes significantly expand security-related documentation (IAM permissions, role configuration, access controls), there's no evidence of addressing a specific disclosed vulnerability. The additions focus on improving security posture through better documentation of existing security features like job execution roles, cross-account access controls, and permission best practices.
Diff
diff --git a/AmazonS3/latest/userguide/troubleshooting-batch-operations.md b/AmazonS3/latest/userguide/troubleshooting-batch-operations.md index 2e5305e28..0081c38e5 100644 --- a//AmazonS3/latest/userguide/troubleshooting-batch-operations.md +++ b//AmazonS3/latest/userguide/troubleshooting-batch-operations.md @@ -5 +5 @@ -NoSuchJobExceptionAccessDeniedExceptionSlowDownErrorInvalidManifestContent +NoSuchJobExceptionInvalidManifestContent @@ -7 +7 @@ NoSuchJobExceptionAccessDeniedExceptionSlowDownErrorInvalidManifestContent -# Troubleshooting Batch Operations +# Troubleshooting S3 Batch Operations @@ -9 +9 @@ NoSuchJobExceptionAccessDeniedExceptionSlowDownErrorInvalidManifestContent -The following topics cover common errors to help you troubleshoot issues that you might encounter while working with Amazon S3 Batch Operations. +Amazon S3 Batch Operations allows you to perform large-scale operations on Amazon S3 objects. This guide helps you troubleshoot common issues you might encounter. @@ -11 +11 @@ The following topics cover common errors to help you troubleshoot issues that yo -To troubleshoot issues with S3 Batch Replication, see [Batch Replication errors](./replication-troubleshoot.html#troubleshoot-batch-replication-errors). +To troubleshoot issues with S3 Batch Replication, see [Troubleshooting replication](./replication-troubleshoot.html). @@ -15 +15 @@ There are two primary types of failures that result in Batch operation errors: - * **API Failure** – The requested API (such as `CreateJob`) has failed to execute. + 1. **API Failure** – The requested API (such as `CreateJob`) has failed to execute. @@ -17 +17 @@ There are two primary types of failures that result in Batch operation errors: - * **Job Failure** – The initial API request succeeded but the job failed, for example, due to issues with the manifest or permissions to objects specified in the manifest. + 2. **Job Failure** – The initial API request succeeded but the job failed, for example, due to issues with the manifest or permissions to objects specified in the manifest. @@ -24 +24 @@ There are two primary types of failures that result in Batch operation errors: -**Type:** API failure +**Type** : API failure @@ -26 +26 @@ There are two primary types of failures that result in Batch operation errors: -This can occur if the job has expired, or when the ID used in the `CreateJob` request isn't the same one used in the `DescribeJob` or `UpdateJobStatus` request. +The `NoSuchJobException` occurs when S3 Batch Operations cannot locate the specified job. This error can happen in several scenarios beyond simple job expiration. Common causes include the following. @@ -28 +28 @@ This can occur if the job has expired, or when the ID used in the `CreateJob` re -Jobs expire after 90 days in a terminal state (`Complete`, `Cancelled`, or `Failed`) For more information, see [Tracking job status and completion reports](./batch-ops-job-status.html). + 1. **Job expiration** – Jobs are automatically deleted 90 days after reaching a terminal state (`Complete`, `Cancelled`, or `Failed`). @@ -30 +30 @@ Jobs expire after 90 days in a terminal state (`Complete`, `Cancelled`, or `Fail -**Related error messages** + 2. **Incorrect job ID** – The job ID used in `DescribeJob` or `UpdateJobStatus` doesn't match the ID returned by `CreateJob`. @@ -32 +32 @@ Jobs expire after 90 days in a terminal state (`Complete`, `Cancelled`, or `Fail -`No such job` + 3. **Wrong region** – Attempting to access a job in a different region than where it was created. @@ -34 +34 @@ Jobs expire after 90 days in a terminal state (`Complete`, `Cancelled`, or `Fail -**Recommended actions** + 4. **Wrong account** – Using a job ID from a different AWS account. @@ -36 +36 @@ Jobs expire after 90 days in a terminal state (`Complete`, `Cancelled`, or `Fail -To troubleshoot `NoSuchJobException` you can try the following: + 5. **Job ID format errors** – Typos, extra characters, or incorrect formatting in the job ID. @@ -38 +38 @@ To troubleshoot `NoSuchJobException` you can try the following: - 1. Verify that the job exists and is in your account. You can use the following AWS CLI command: `aws s3control list-jobs --account-id 111122223333` + 6. **Timing issues** – Checking job status immediately after creation before the job is fully registered. @@ -40 +39,0 @@ To troubleshoot `NoSuchJobException` you can try the following: - 2. Verify that the ID received in the `CreateJob` request is the same one used in the `DescribeJob` or `UpdateJobStatus` request. @@ -43,0 +43 @@ To troubleshoot `NoSuchJobException` you can try the following: +Related error messages include the following. @@ -45 +45 @@ To troubleshoot `NoSuchJobException` you can try the following: -## AccessDeniedException + 1. `No such job` @@ -47 +47 @@ To troubleshoot `NoSuchJobException` you can try the following: -**Type:** API failure + 2. `The specified job does not exist` @@ -49 +48,0 @@ To troubleshoot `NoSuchJobException` you can try the following: -The `AccessDeniedException` occurs when an S3 Batch Operations request is blocked because of unsupported operations or the IAM identity making the request doesn't have sufficient permissions to perform an action. @@ -51 +49,0 @@ The `AccessDeniedException` occurs when an S3 Batch Operations request is blocke -**Related error messages** @@ -53 +50,0 @@ The `AccessDeniedException` occurs when an S3 Batch Operations request is blocke -`Access Denied` @@ -55 +52 @@ The `AccessDeniedException` occurs when an S3 Batch Operations request is blocke -**Recommended actions** +### Best practices to prevent `NoSuchJobException` API failures @@ -57 +54 @@ The `AccessDeniedException` occurs when an S3 Batch Operations request is blocke -To troubleshoot AccessDeniedException you can try the following: + 1. **Store job IDs immediately** – Save the job ID from the `CreateJob` response before making subsequent API calls. @@ -59 +56 @@ To troubleshoot AccessDeniedException you can try the following: - 1. Make sure that S3 Batch Operations supports the operation or feature in the Region. For a list of supported operations, see [Operations supported by S3 Batch Operations](./batch-ops-operations.html). Batch operations are supported for General purpose buckets in all Regions, but operations for Directory buckets are only available in Regional and Zonal endpoints for directory buckets. + 2. **Implement retry logic** – Add exponential backoff when checking job status immediately after creation. @@ -61 +58 @@ To troubleshoot AccessDeniedException you can try the following: - 2. Make sure the IAM identity making the request has permissions to create and manage batch operations, for a list of permissions, see [Granting permissions for Batch Operations](./batch-ops-iam-role-policies.html). + 3. **Set up monitoring** – Create CloudWatch alarms to track job completion before the 90-day expiration. For details, see [Using CloudWatch alarms](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html) in the Amazon CloudWatch User Guide. @@ -62,0 +60 @@ To troubleshoot AccessDeniedException you can try the following: + 4. **Use consistent regions** – Ensure all job operations use the same region as job creation. @@ -63,0 +62 @@ To troubleshoot AccessDeniedException you can try the following: + 5. **Validate input** – Check job ID format before making API calls. @@ -66 +64,0 @@ To troubleshoot AccessDeniedException you can try the following: -## SlowDownError @@ -68 +65,0 @@ To troubleshoot AccessDeniedException you can try the following: -**Type:** API failure @@ -70 +67 @@ To troubleshoot AccessDeniedException you can try the following: -The `SlowDownError` exception occurs when your account has exceeded the request rate limit for S3 Batch Operations. +### When jobs expire @@ -72 +69 @@ The `SlowDownError` exception occurs when your account has exceeded the request -**Recommended actions** +Jobs in terminal states are automatically deleted after 90 days. To avoid losing job information, consider the following. @@ -74 +71,272 @@ The `SlowDownError` exception occurs when your account has exceeded the request -To resolve `SlowDownError` exceptions you can try the following: + 1. **Download completion reports before expiration** – For instructions on retrieving and storing job results, see [](./batch-ops-job-status.html#batch-ops-completion-report.title). + + 2. **Archive job metadata in your own systems** – Store critical job information in your databases or monitoring systems. + + 3. **Set up automated notifications before the 90-day deadline** – Use Amazon EventBridge to create rules that trigger notifications when jobs complete. For more information, see [Amazon S3 Event Notifications](./EventNotifications.html). + + + + +### `NoSuchJobException` troubleshooting + + 1. Use the following command to verify the job exists in your account and region. + + aws s3control list-jobs --account-id 111122223333 --region us-east-1 + + + 2. Use the following comamand to search across all job statuses. Possible job statuses include `Active`, `Cancelled`, `Cancelling`, `Complete`, `Completing`, `Failed`, `Failing`, `New`, `Paused`, `Pausing`, `Preparing`, `Ready`, and `Suspended`. + + aws s3control list-jobs --account-id 111122223333 --job-statuses your-job-status + + + 3. Use the following command to check if the job exists in other regions where you commonly create jobs. + + aws s3control list-jobs --account-id 111122223333 --region job-region-1 aws s3control list-jobs --account-id 111122223333 --region job-region-2 + + + 4. Validate the job ID format. Job IDs typically contain 36 character, such as `12345678-1234-1234-1234-123456789012`. Check for extra spaces, missing characters, or case sensitivity issues and verify you're using the complete job ID returned by the `CreateJob` command. + + 5. Use the following command to check CloudTrail logs for job creation events. + + aws logs filter-log-events --log-group-name CloudTrail/S3BatchOperations \ --filter-pattern "{ $.eventName = CreateJob }" \ --start-time timestamp + + + + + +### AccessDeniedException + +**Type** : API failure + +The `AccessDeniedException` occurs when an S3 Batch Operations request is blocked due to insufficient permissions, unsupported operations, or policy restrictions. This is one of the most common errors in Batch Operations. It has the following common causes: + + 1. **Missing IAM permissions** – The IAM identity lacks required permissions for Batch Operations APIs. + + 2. **Insufficient S3 permissions** – Missing permissions to access source or destination buckets and objects. + + 3. **Job execution role issues** – The job execution role lacks permissions to perform the specified operation. + + 4. **Unsupported operations** – Attempting to use operations not supported in the current region or bucket type. + + 5. **Cross-account access issues** – Missing permissions for cross-account bucket or object access. + + 6. **Resource-based policy restrictions** – Bucket policies or object ACLs blocking the operation. + + 7. **Service Control Policy (SCP) restrictions** – Organization-level policies preventing the operation. + + + + +Related error messages: + + 1. `Access Denied` + + 2. `User: arn:aws:iam::account:user/username is not authorized to perform: s3:operation` + + 3. `Cross-account pass role is not allowed` + + 4. `The bucket policy does not allow the specified operation` + + + + +#### Best practices to prevent AccessDeniedException API failures + + 1. **Use least privilege principle** – Grant only the minimum permissions required for your specific operations. + + 2. **Test permissions before large jobs** – Run small test jobs to validate permissions before processing thousands of objects. + + 3. **Use IAM policy simulator** – Test policies before deployment using the IAM policy simulator. For more information, see [IAM policy testing with the IAM policy simulator](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_testing-policies.html) in the IAM User Guide. + + 4. **Implement proper cross-account setup** – Check your cross-account access configuration for cross-account job configurations. For more information, see [IAM tutorial: Delegate access across AWS accounts using IAM roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html) in the IAM User Guide. + + 5. **Monitor permission changes** – Set up CloudTrail alerts for IAM policy modifications that might affect Batch Operations. + + 6. **Document role requirements** – Maintain clear documentation of required permissions for each job type. + + 7. **Use common permission templates** \- Use the permission examples and polcy templates: + + 1. [Granting permissions for Batch Operations](./batch-ops-iam-role-policies.html) + + 2. [Cross account resources in IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies-cross-account-resource-access.html) in the IAM User Guide. + + 3. [Control access to VPC endpoints using endpoint policies](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-access.html) in the AWS PrivateLink Guide. + + + + +#### AccessDeniedException troubleshooting + +Follow these steps systematically to identify and resolve permission issues. + + 1. Check [Operations supported by S3 Batch Operations](./batch-ops-operations.html) for supported operations by region. Confirm directory bucket operations are only available at Regional and Zonal endpoints. Verify the operation is supported for your bucket's storage class. + + 2. Use the following command to determine if you can list jobs. + + aws s3control list-jobs --account-id 111122223333 + + + 3. Use the following command to check IAM permissions for the requesting identity. The account running the job needs the following permissions: `s3:CreateJob`, `s3:DescribeJob`, `s3:ListJobs-s3:UpdateJobPriority`, `s3:UpdateJobStatus-iam:PassRole`. +