AWS prescriptive-guidance documentation change
Summary
Updated documentation to use consistent product naming (Amazon S3 instead of S3 bucket), fixed documentation links formatting, and corrected whitespace in code samples
Security assessment
Changes are primarily stylistic improvements and documentation formatting. No security-related content was added or modified. The updates focus on naming consistency (Amazon S3), link formatting (removing parentheses from references), and code indentation fixes. No changes to security controls, permissions, or vulnerability mitigations are present in the diff.
Diff
diff --git a/prescriptive-guidance/latest/patterns/extract-and-query-aws-iot-sitewise-metadata-attributes-in-a-data-lake.md b/prescriptive-guidance/latest/patterns/extract-and-query-aws-iot-sitewise-metadata-attributes-in-a-data-lake.md index f80b36d2b..f9c91fc79 100644 --- a//prescriptive-guidance/latest/patterns/extract-and-query-aws-iot-sitewise-metadata-attributes-in-a-data-lake.md +++ b//prescriptive-guidance/latest/patterns/extract-and-query-aws-iot-sitewise-metadata-attributes-in-a-data-lake.md @@ -15 +15 @@ AWS IoT SiteWise uses asset models and hierarchies to represent your industrial -However, metadata attributes can’t be queried directly from the AWS IoT SiteWise service. To make the attributes queryable, you must extract and ingest them into a data lake. This pattern uses a Python script to extract the attributes for all AWS IoT SiteWise assets and ingest them into a data lake in an Amazon Simple Storage Service (Amazon S3) bucket. When you have completed this process, you can use SQL queries in Amazon Athena to access the AWS IoT SiteWise metadata attributes and other datasets, such as measurement datasets. The metadata attribute information is also useful when working with AWS IoT SiteWise monitors or dashboards. You can also build an AWS QuickSight dashboard by using the extracted attributes in the S3 bucket. +However, metadata attributes can’t be queried directly from the AWS IoT SiteWise service. To make the attributes queryable, you must extract and ingest them into a data lake. This pattern uses a Python script to extract the attributes for all AWS IoT SiteWise assets and ingest them into a data lake in an Amazon Simple Storage Service (Amazon S3) bucket. When you have completed this process, you can use SQL queries in Amazon Athena to access the AWS IoT SiteWise metadata attributes and other datasets, such as measurement datasets. The metadata attribute information is also useful when working with AWS IoT SiteWise monitors or dashboards. You can also build an Amazon QuickSight dashboard by using the extracted attributes in the Amazon S3 bucket. @@ -29 +29 @@ The pattern has reference code, and you can you can implement the code by using - * The asset models and hierarchies are set up in AWS IoT SiteWise. For more information, see [Creating asset models](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/create-asset-models.html) (AWS IoT SiteWise documentation). + * The asset models and hierarchies are set up in AWS IoT SiteWise. For more information, see [Creating asset models](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/create-asset-models.html) in the AWS IoT SiteWise documentation. @@ -42 +42 @@ The solution architecture and workflow are shown in the following diagram. - 1. The scheduled AWS Glue job or Lambda function runs. It extracts the asset metadata attributes from AWS IoT SiteWise and ingests them into an S3 bucket. + 1. The scheduled AWS Glue job or Lambda function runs. It extracts the asset metadata attributes from AWS IoT SiteWise and ingests them into an Amazon S3 bucket. @@ -44 +44 @@ The solution architecture and workflow are shown in the following diagram. - 2. An AWS Glue crawler crawls the extracted data in the S3 bucket and creates tables in an AWS Glue Data Catalog. + 2. An AWS Glue crawler crawls the extracted data in the Amazon S3 bucket and creates tables in an AWS Glue Data Catalog. @@ -59 +59 @@ There is no limit to the number of AWS IoT SiteWise assets that the sample code - * [Amazon Athena](https://docs.aws.amazon.com/athena/latest/ug/what-is.html) is an interactive query service that helps you analyze data directly in Amazon Simple Storage Service (Amazon S3) by using standard SQL. + * [Amazon Athena](https://docs.aws.amazon.com/athena/latest/ug/what-is.html) is an interactive query service that helps you analyze data directly in Amazon S3 by using standard SQL. @@ -83 +83 @@ Configure permissions in IAM.| In the IAM console, grant permissions to the IAM - * Write to the S3 bucket + * Write to the Amazon S3 bucket @@ -85,3 +85,3 @@ Configure permissions in IAM.| In the IAM console, grant permissions to the IAM -For more information, see [Creating a role for an AWS service](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html#roles-creatingrole-service-console) (IAM documentation).| General AWS -Create the Lambda function or AWS Glue job.| If you are using Lambda, create a new Lambda function. For **Runtime** , choose **Python**. For more information, see [Building Lambda functions with Python](https://docs.aws.amazon.com/lambda/latest/dg/lambda-python.html) (Lambda documentation).If you are using AWS Glue, create a new Python shell job in the AWS Glue console. For more information, see [Adding Python shell jobs](https://docs.aws.amazon.com/glue/latest/dg/add-job-python.html#create-job-python-properties) (AWS Glue documentation). | General AWS -Update the Lambda function or AWS Glue job.| Modify the new Lambda function or AWS Glue job, and enter the code sample in the Additional information section. Modify the code as needed for your use case. For more information, see [Edit code using the console editor](https://docs.aws.amazon.com/lambda/latest/dg/foundation-console.html#code-editor) (Lambda documentation) and [Working with scripts](https://docs.aws.amazon.com/glue/latest/dg/console-edit-script.html) (AWS Glue documentation).| General AWS +For more information, see [Creating a role for an AWS service](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html#roles-creatingrole-service-console) in the IAM documentation.| General AWS +Create the Lambda function or AWS Glue job.| If you are using Lambda, create a new Lambda function. For **Runtime** , choose **Python**. For more information, see [Building Lambda functions with Python](https://docs.aws.amazon.com/lambda/latest/dg/lambda-python.html) in the Lambda documentation.If you are using AWS Glue, create a new Python shell job in the AWS Glue console. For more information, see [Adding Python shell jobs](https://docs.aws.amazon.com/glue/latest/dg/add-job-python.html#create-job-python-properties) in the AWS Glue documentation. | General AWS +Update the Lambda function or AWS Glue job.| Modify the new Lambda function or AWS Glue job, and enter the code sample in the Additional information section. Modify the code as needed for your use case. For more information, see [Edit code using the console editor](https://docs.aws.amazon.com/lambda/latest/dg/foundation-console.html#code-editor) in the Lambda documentation and see [Working with scripts](https://docs.aws.amazon.com/glue/latest/dg/console-edit-script.html) in theAWS Glue documentation.| General AWS @@ -91,4 +91,4 @@ Task| Description| Skills required -Run the Lambda function or AWS Glue job.| Run the Lambda function or AWS Glue job. For more information, see [Invoke the Lambda function](https://docs.aws.amazon.com/lambda/latest/dg/getting-started.html#get-started-invoke-manually) (Lambda documentation) or [Starting jobs using triggers](https://docs.aws.amazon.com/glue/latest/dg/trigger-job.html) (AWS Glue documentation). This extracts the metadata attributes for the assets and models in the AWS IoT SiteWise hierarchy and stores them in the specified S3 bucket.| General AWS -Set up an AWS Glue crawler.| Set up an AWS Glue crawler with the necessary format classifier for a CSV-formatted file. Use the S3 bucket and prefix details used in the Lambda function or AWS Glue job. For more information, see [Defining crawlers](https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html) (AWS Glue documentation).| General AWS -Run the AWS Glue crawler.| Run the crawler to process the data file created by the Lambda function or AWS Glue job. The crawler creates a table in the specified AWS Glue Data Catalog. For more information, see or [Starting crawlers using triggers](https://docs.aws.amazon.com/glue/latest/dg/trigger-job.html) (AWS Glue documentation).| General AWS -Query the metadata attributes.| Using Amazon Athena, use standard SQL to query the AWS Glue Data Catalog as needed for your use case. You can join the metadata attribute table with other databases and tables. For more information, see [Getting Started](https://docs.aws.amazon.com/athena/latest/ug/getting-started.html) (Amazon Athena documentation).| General AWS +Run the Lambda function or AWS Glue job.| Run the Lambda function or AWS Glue job. For more information, see [Invoke the Lambda function](https://docs.aws.amazon.com/lambda/latest/dg/getting-started.html#get-started-invoke-manually) in the Lambda documentation or see [Starting jobs using triggers](https://docs.aws.amazon.com/glue/latest/dg/trigger-job.html) in the AWS Glue documentation. This extracts the metadata attributes for the assets and models in the AWS IoT SiteWise hierarchy and stores them in the specified Amazon S3 bucket.| General AWS +Set up an AWS Glue crawler.| Set up an AWS Glue crawler with the necessary format classifier for a CSV-formatted file. Use the Amazon S3 bucket and prefix details used in the Lambda function or AWS Glue job. For more information, see [Defining crawlers](https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html) in the AWS Glue documentation.| General AWS +Run the AWS Glue crawler.| Run the crawler to process the data file created by the Lambda function or AWS Glue job. The crawler creates a table in the specified AWS Glue Data Catalog. For more information, see or [Starting crawlers using triggers](https://docs.aws.amazon.com/glue/latest/dg/trigger-job.html) in the AWS Glue documentation.| General AWS +Query the metadata attributes.| Using Amazon Athena, use standard SQL to query the AWS Glue Data Catalog as needed for your use case. You can join the metadata attribute table with other databases and tables. For more information, see [Getting Started](https://docs.aws.amazon.com/athena/latest/ug/getting-started.html) in the Amazon Athena documentation.| General AWS @@ -130 +130 @@ The sample code provided is for reference, and you can customize this code as ne - # Following code can be used in an AWS Lambda function or in an AWS Glue Python shell job. + # Following code can be used in an AWS Lambda function or in an AWS Glue Python shell job. @@ -135 +135 @@ The sample code provided is for reference, and you can customize this code as ne - + @@ -140 +140 @@ The sample code provided is for reference, and you can customize this code as ne - + @@ -143,28 +143,28 @@ The sample code provided is for reference, and you can customize this code as ne - model_id = m_rec['id'] - model_name = m_rec['name'] - - attribute_list.clear() - dam_response = sw_client.describe_asset_model(assetModelId=model_id) - for rec in dam_response['assetModelProperties']: - if 'attribute' in rec['type']: - attribute_list.append(rec['name']) - - response = sw_client.list_assets(assetModelId=model_id, filter='ALL') - for asset in response['assetSummaries']: - asset_id = asset['id'] - asset_name = asset['name'] - resp = sw_client.describe_asset(assetId=asset_id) - for rec in resp['assetProperties']: - if rec['name'] in attribute_list: - p_resp = sw_client.get_asset_property_value(assetId=asset_id, propertyId=rec['id']) - if 'propertyValue' in p_resp: - if p_resp['propertyValue']['value']: - if 'stringValue' in p_resp['propertyValue']['value']: - output.write(model_id + "," + model_name + "," + asset_id + "," + asset_name + "," + rec['id'] + "," + rec['name'] + "," + str(p_resp['propertyValue']['value']['stringValue']) + "\n") - if 'doubleValue' in p_resp['propertyValue']['value']: - output.write(model_id + "," + model_name + "," + asset_id + "," + asset_name + "," + rec['id'] + "," + rec['name'] + "," + str(p_resp['propertyValue']['value']['doubleValue']) + "\n") - if 'integerValue' in p_resp['propertyValue']['value']: - output.write(model_id + "," + model_name + "," + asset_id + "," + asset_name + "," + rec['id'] + "," + rec['name'] + "," + str(p_resp['propertyValue']['value']['integerValue']) + "\n") - if 'booleanValue' in p_resp['propertyValue']['value']: - output.write(model_id + "," + model_name + "," + asset_id + "," + asset_name + "," + rec['id'] + "," + rec['name'] + "," + str(p_resp['propertyValue']['value']['booleanValue']) + "\n") - + model_id = m_rec['id'] + model_name = m_rec['name'] + + attribute_list.clear() + dam_response = sw_client.describe_asset_model(assetModelId=model_id) + for rec in dam_response['assetModelProperties']: + if 'attribute' in rec['type']: + attribute_list.append(rec['name']) + + response = sw_client.list_assets(assetModelId=model_id, filter='ALL') + for asset in response['assetSummaries']: + asset_id = asset['id'] + asset_name = asset['name'] + resp = sw_client.describe_asset(assetId=asset_id) + for rec in resp['assetProperties']: + if rec['name'] in attribute_list: + p_resp = sw_client.get_asset_property_value(assetId=asset_id, propertyId=rec['id']) + if 'propertyValue' in p_resp: + if p_resp['propertyValue']['value']: + if 'stringValue' in p_resp['propertyValue']['value']: + output.write(model_id + "," + model_name + "," + asset_id + "," + asset_name + "," + rec['id'] + "," + rec['name'] + "," + str(p_resp['propertyValue']['value']['stringValue']) + "\n") + if 'doubleValue' in p_resp['propertyValue']['value']: + output.write(model_id + "," + model_name + "," + asset_id + "," + asset_name + "," + rec['id'] + "," + rec['name'] + "," + str(p_resp['propertyValue']['value']['doubleValue']) + "\n") + if 'integerValue' in p_resp['propertyValue']['value']: + output.write(model_id + "," + model_name + "," + asset_id + "," + asset_name + "," + rec['id'] + "," + rec['name'] + "," + str(p_resp['propertyValue']['value']['integerValue']) + "\n") + if 'booleanValue' in p_resp['propertyValue']['value']: + output.write(model_id + "," + model_name + "," + asset_id + "," + asset_name + "," + rec['id'] + "," + rec['name'] + "," + str(p_resp['propertyValue']['value']['booleanValue']) + "\n") +