AWS sns medium security documentation change
Summary
Updated documentation for Amazon SNS Extended Client Library with improved code examples, added 'use_legacy_attribute' configuration, clarified S3 client usage, and demonstrated policy attachment for SQS access control
Security assessment
The change adds explicit SQS policy configuration requiring messages to come from a specific SNS topic ARN (Condition.ArnEquals.'aws:SourceArn'), which demonstrates security best practices for access control. This qualifies as security documentation addition as it shows implementation of resource-based policies to restrict message sources.
Diff
diff --git a/sns/latest/dg/extended-client-library-python.md index 6c87794f0..c687bbb4c 100644 --- a/sns/latest/dg/extended-client-library-python.md +++ b/sns/latest/dg/extended-client-library-python.md @@ -13 +13 @@ The following are the prerequisites for using the [Amazon SNS Extended Client Li - * An AWS SDK. + * An AWS SDK. The example on this page uses AWS Python SDK Boto3. To install and set up the SDK, see the [_AWS SDK for Python_](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html) documentation. @@ -15,5 +15 @@ The following are the prerequisites for using the [Amazon SNS Extended Client Li -The example on this page uses AWS Python SDK Boto3. To install and set up the SDK, see the [_AWS SDK for Python_](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html) documentation. - - * An AWS account with the proper credentials. - -To create an AWS account, navigate to the [AWS home page](https://aws.amazon.com/), and then choose **Create an AWS Account**. Follow the instructions. + * An AWS account with the proper credentials. To create an AWS account, navigate to the [AWS home page](https://aws.amazon.com/), and then choose **Create an AWS Account**. Follow the instructions. @@ -34 +30,3 @@ The below attributes are available on Boto3 Amazon SNS [Client](https://boto3.am - * **`large_payload_support`** – The Amazon S3 bucket name to store large messages. + * **`large_payload_support`** – The Amazon S3 bucket name that will store large messages. + + * **`use_legacy_attribute`** – If `True`, then all published messages use the Legacy reserved message attribute (`SQSLargePayloadSize`) instead of the current reserved message attribute (`ExtendedPayloadSize`). @@ -36 +34 @@ The below attributes are available on Boto3 Amazon SNS [Client](https://boto3.am - * **`message_size_threshold`** – The threshold for storing the message in the large messages bucket. Cannot be less than 0, or greater than 262144. The default is 262144. + * **`message_size_threshold`** – The threshold for storing the message in the large messages bucket. Cannot be less than `0`, or greater than `262144`. The default is `262144`. @@ -40 +38 @@ The below attributes are available on Boto3 Amazon SNS [Client](https://boto3.am - * **`s3`** – The Boto3 Amazon S3 `resource` object used to store objects in Amazon S3. Use this if you want to control the Amazon S3 resource (for example, a custom Amazon S3 config or credentials). If not previously set on first use, the default is ` boto3.resource("s3")`. + * **`s3_client`** – The Boto3 Amazon S3 `client` object to use to store objects to Amazon S3. Use this if you want to control the Amazon S3 client (for example, custom Amazon S3 config or credentials). Defaults to `boto3.client("s3")` on first use if not previously set. @@ -50,0 +49,2 @@ The following code example shows how to: + * Attach the policy to the Amazon SQS queue to receive the message from Amazon SNS topic. + @@ -53 +53 @@ The following code example shows how to: - * Publish a test message. + * Publish a test message using the Amazon SNS extended client, Topic resource, and PlatformEndpoint resource. @@ -66 +66 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - import sns_extended_client + from sns_extended_client import SNSExtendedClientSession @@ -69,3 +69,24 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - s3_extended_payload_bucket = "extended-client-bucket-store" - TOPIC_NAME = "TOPIC-NAME" - QUEUE_NAME = "QUEUE-NAME" + s3_extended_payload_bucket = "extended-client-bucket-store" # S3 bucket with the given bucket name is a resource which is created and accessible with the given AWS credentials + TOPIC_NAME = "---TOPIC-NAME---" + QUEUE_NAME = "---QUEUE-NAME---" + + def allow_sns_to_write_to_sqs(topicarn, queuearn): + policy_document = """{{ + "Version":"2012-10-17", + "Statement":[ + {{ + "Sid":"MyPolicy", + "Effect":"Allow", + "Principal" : {{"AWS" : "*"}}, + "Action":"SQS:SendMessage", + "Resource": "{}", + "Condition":{{ + "ArnEquals":{{ + "aws:SourceArn": "{}" + }} + }} + }} + ] + }}""".format(queuearn, topicarn) + + return policy_document @@ -73,2 +94,2 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - # Create an helper to fetch message from S3 - def get_msg_from_s3(body): + def get_msg_from_s3(body,sns_extended_client): + """Handy Helper to fetch message from S3""" @@ -76,2 +97 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - s3_client = boto3.client("s3") - s3_object = s3_client.get_object( + s3_object = sns_extended_client.s3_client.get_object( @@ -83,5 +103,9 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - # Create an helper to fetch and print message SQS queue and S3 - def fetch_and_print_from_sqs(sqs, queue_url): - """Handy Helper to fetch and print message from SQS queue and S3""" - message = sqs.receive_message( - QueueUrl=queue_url, MessageAttributeNames=["All"], MaxNumberOfMessages=1 + + def fetch_and_print_from_sqs(sqs, queue_url,sns_extended_client): + sqs_msg = sqs.receive_message( + QueueUrl=queue_url, + AttributeNames=['All'], + MessageAttributeNames=['All'], + VisibilityTimeout=0, + WaitTimeSeconds=0, + MaxNumberOfMessages=1 @@ -89 +113,2 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - message_body = message.get("Body") + + message_body = sqs_msg.get("Body") @@ -91 +116,8 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - print("Message Stored in S3 Bucket is: {}\n".format(get_msg_from_s3(message_body))) + print("Message Stored in S3 Bucket is: {}\n".format(get_msg_from_s3(message_body,sns_extended_client))) + + # Delete the Processed Message + sqs.delete_message( + QueueUrl=queue_url, + ReceiptHandle=sqs_msg['ReceiptHandle'] + ) + @@ -93 +124,0 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - # Initialize the SNS client and create SNS Topic @@ -96 +127 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - demo_topic_arn = create_topic_response.get("TopicArn") + sns_topic_arn = create_topic_response.get("TopicArn") @@ -98,2 +129,2 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - # Create and subscribe an SQS queue to the SNS client - sqs = boto3.client("sqs") + # create and subscribe an sqs queue to the sns client + sqs = boto3.client("sqs",region_name="us-east-1") @@ -101,5 +132,19 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - demo_queue_arn = sqs.get_queue_attributes(QueueUrl=demo_queue_url, - AttributeNames=["QueueArn"])["Attributes"].get("QueueArn") - # Set the RawMessageDelivery subscription attribute to TRUE - sns_extended_client.subscribe(TopicArn=demo_topic_arn, Protocol="sqs", - Endpoint=demo_queue_arn, Attributes={"RawMessageDelivery":"true"}) + sqs_queue_arn = sqs.get_queue_attributes( + QueueUrl=demo_queue_url, AttributeNames=["QueueArn"] + )["Attributes"].get("QueueArn") + + # Adding policy to SQS queue such that SNS topic can send msg to SQS queue + policy_json = allow_sns_to_write_to_sqs(sns_topic_arn, sqs_queue_arn) + response = sqs.set_queue_attributes( + QueueUrl = demo_queue_url, + Attributes = { + 'Policy' : policy_json + } + ) + + # Set the RawMessageDelivery subscription attribute to TRUE if you want to use + # SQSExtendedClient to help with retrieving msg from S3 + sns_extended_client.subscribe(TopicArn=sns_topic_arn, Protocol="sqs", + Endpoint=sqs_queue_arn + , Attributes={"RawMessageDelivery":"true"} + ) @@ -109,3 +154,16 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - # To store all messages content in S3, set always_through_s3 to True - # In the example, we set message size threshold as 32 bytes, adjust this threshold as per your usecase - # Message will only be uploaded to S3 when its payload size exceeded threshold + # Change default s3_client attribute of sns_extended_client to use 'us-east-1' region + sns_extended_client.s3_client = boto3.client("s3", region_name="us-east-1") + + + # Below is the example that all the messages will be sent to the S3 bucket + sns_extended_client.always_through_s3 = True + sns_extended_client.publish( + TopicArn=sns_topic_arn, Message="This message should be published to S3" + ) + print("\n\nPublished using SNS extended client:") + fetch_and_print_from_sqs(sqs, demo_queue_url,sns_extended_client) # Prints message stored in s3 + + # Below is the example that all the messages larger than 32 bytes will be sent to the S3 bucket + print("\nUsing decreased message size threshold:") + + sns_extended_client.always_through_s3 = False @@ -114,2 +172,2 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - TopicArn=demo_topic_arn, - Message="This message should be published to S3 as it exceeds the message_size_threshold limit", + TopicArn=sns_topic_arn, + Message="This message should be published to S3 as it exceeds the limit of the 32 bytes", @@ -117,2 +174,0 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - # Print message stored in s3 - fetch_and_print_from_sqs(sqs, demo_queue_url) @@ -119,0 +176 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho + fetch_and_print_from_sqs(sqs, demo_queue_url,sns_extended_client) # Prints message stored in s3 @@ -121 +177,0 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho -**Output** @@ -122,0 +179,7 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho + # Below is the example to publish message using the SNS.Topic resource + sns_extended_client_resource = SNSExtendedClientSession().resource( + "sns", region_name="us-east-1" + ) + + topic = sns_extended_client_resource.Topic(sns_topic_arn) + topic.large_payload_support = s3_extended_payload_bucket @@ -124,6 +187,11 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - Published Message: - [ - "software.amazon.payloadoffloading.PayloadS3Pointer", - { - "s3BucketName": "extended-client-bucket-store", - "s3Key": "xxxx-xxxxx-xxxxx-xxxxxx" + # Change default s3_client attribute of topic to use 'us-east-1' region + topic.s3_client = boto3.client("s3", region_name="us-east-1") + + topic.always_through_s3 = True + # Can Set custom S3 Keys to be used to store objects in S3 + topic.publish( + Message="This message should be published to S3 using the topic resource", + MessageAttributes={ + "S3Key": { + "DataType": "String", + "StringValue": "347c11c4-a22c-42e4-a6a2-9b5af5b76587", @@ -131,2 +199,48 @@ To publish a large message, use the Amazon SNS Extended Client Library for Pytho - ] - Message Stored in S3 Bucket is: This message should be published to S3 as it exceeds the message_size_threshold limit + }, + ) + print("\nPublished using Topic Resource:") + fetch_and_print_from_sqs(sqs, demo_queue_url,topic)