AWS wellarchitected documentation change
Summary
Restructured IoT operations guidance with updated best practice numbering (IOTOPS09/IOTOPS10), added operational metrics analysis recommendations, and emphasized dynamic thing groups for OTA updates
Security assessment
Changes focus on operational improvements like firmware deployment monitoring and team training rather than addressing specific vulnerabilities. While references to AWS IoT Device Defender exist, they reinforce existing security monitoring practices rather than documenting new security features or patching vulnerabilities.
Diff
diff --git a/wellarchitected/latest/iot-lens/evolve.md b/wellarchitected/latest/iot-lens/evolve.md index d6033b7d6..1c9ef3ce6 100644 --- a//wellarchitected/latest/iot-lens/evolve.md +++ b//wellarchitected/latest/iot-lens/evolve.md @@ -3,277 +3 @@ -[Documentation](/index.html)[AWS Well-Architected](https://aws.amazon.com/architecture/well-architected/)[AWS Well-Architected Framework](abstract-and-introduction.html) - -# Evolve - -IOTOPS 3. How do you evolve your IoT application with minimum impact to downstream IoT devices? ---- - -IoT solutions frequently involve a combination of low-power devices, remote locations, low bandwidth, and intermittent network connectivity. Each of those factors poses communications challenges, including upgrading firmware and edge applications. Therefore, it's important for you to incorporate and implement an IoT update process that minimizes the impact to downstream devices and operations. In addition to reducing downstream impact, devices must be resilient to common challenges that exist in local environments, such as intermittent network connectivity and power loss. Use a combination of grouping IoT devices for deployment and staggering firmware upgrades over a period of time. Monitor the behavior of devices as they are updated in the field, and proceed only after a percentage of devices have upgraded successfully. - -Use AWS IoT Device Management for creating deployment groups of devices and delivering over-the-air (OTA) updates to specific device groups. During upgrades, continue to collect all of the CloudWatch Logs, telemetry, and IoT device job messages and combine that information with the KPIs used to measure overall application health and the performance of any long-running canaries. - -Before and after firmware updates, perform a retrospective analysis of operations metrics with participants spanning the business to determine opportunities and methods for improvement. Services such as AWS IoT Analytics and AWS IoT Device Defender are used to track anomalies in overall device behavior, and to measure deviations in performance that may indicate an issue in the updated firmware. - -IOTOPS 4. How do you ensure that you are ready to support the operations of devices in your IoT workload? ---- - -Operating IoT workloads at scale is different than testing and running prototypes. You need to ensure that your team is prepared and trained to operate a widely distributed IoT data collection application. IoT workloads require your teams to learn new skills and competencies to deliver edge-to-cloud outcomes. Your team needs to be able to pinpoint key operational thresholds that indicate a high level of readiness. - -**Best practice IOTOPS_4.1 – Train team members supporting your IoT workloads on the lifecycle of IoT applications and your business objectives** - -Key team members responsible for IoT workloads are trained on major IoT lifecycle events: onboarding, command and control, security, data ingestion, integration, and analytics services. Team members should be able to identify key operational metrics and know how to apply incident response measures. Training team members on the basics of IoT lifecycles and how these align with business objectives provides actionable context on failure scenarios, mitigation strategies, and defining lasting processes that effectively contribute to fewer operational events and less severe impact during events. - -**Recommendation IOTOPS_4.1.1** – _Build IoT operational expertise by having team members and architects’ complete reviews of common IoT architectural patterns, best practices, and educational courses_ - - * Introduce new team members to IoT lifecycles with onboarding checklists that include at least one educational course. - - * Introduce new team members with onboarding checklists that include a step to review, validate, and submit updates to your IoT application architecture documentation and operational monitoring plan. - - - - -**Recommendation IOTOPS_4.1.2** – _Author runbooks for each component of the architecture and train team members on their use_ - - * Include guidance for a response procedure for remote devices that are no longer online. - - * Apply recovery commands for troubleshooting remote devices that are faulty but still online. - - - - -IOTOPS 5. How do you assess whether your IoT application meets your operational goals? ---- - -Evaluating your operational goals enables you to fine-tune and identify improvements throughout the lifecycle of your IoT application. Measuring and extracting operational and business value from your IoT application allows you to effectively drive high-value initiatives. - -**Best practice IOTOPS_5.1** – **Enable appropriate responses to events** - -Key operational data elements are those data points that convey some notion of operational health of your application by classifying events. Detecting operational events early can uncover unforeseen risks in your application and give your operations team a head start to prevent or reduce significant business interruption. By defining a minimum set of logs, metrics, and alarms, your operations team can provide a first line of defense against significant business interruption. - -**Recommendation IOTOPS_5.1.1** – _Configure logging to capture and store at least error-level events_ - - * [Use AWS IoT service logging options to capture error events in CloudWatch Logs](https://docs.aws.amazon.com/iot/latest/developerguide/cloud-watch-logs.html) - - - - -**Recommendation IOTOPS_5.1.2** – _Create a dashboard for your responders to use in investigations of operational events to rapidly pinpoint the period of time when errors are logged_ - - * Group clusters of error events into buckets of time to quickly identify when surges of errors were captured. - - * Drill down into clusters of errors to identify any patterns to signal for triage response. - - - - -**Recommendation IOTOPS_5.1.3** – _Review the default metrics emitted by your IoT services and configure alarms for metrics that might indicate business interruption_ - - * For example: - - * Your business deploys a thousand sensors across manufacturing plants and your operations team wants to be alerted if sensors can no longer connect to the cloud and send telemetry. - - * Your IT team administering the AWS account reviews the AWS IoT Core metrics and notes the following metrics to monitor: `Connect.AuthError`, `Connect.ClientError`, `Connect.ClientIDThrottle`, `Connect.ServerError`, `Connect.Throttle`. Activity in any of these metrics constitutes alerting and investigation. - - * Your IT team uses CloudWatch to configure new alarms on these metrics when for any period the metrics’ SUM of Count is greater than zero. - - * Your IT team configures an Amazon SNS topic to notify their paging tool when any of the new CloudWatch alarms changes status. - - * For more information: - - * [Monitor AWS IoT alarms and metrics using Amazon CloudWatch](https://docs.aws.amazon.com/iot/latest/developerguide/monitoring-cloudwatch.html) - - - - -**Recommendation IOTOPS_5.1.4** – _Configure an automated monitoring and alerting tool to detect common symptoms and warnings of operational impact_ - - * For example: - - * Configure AWS IoT Device Defender to run a daily audit on at least the high and critical checks. - - * Configure an Amazon SNS topic to notify a team email list, paging tool, or operations channel when AWS IoT Device Defender reports non-compliant resources in an audit. - - * For more information: - - * [AWS IoT Device Defender Audit](https://docs.aws.amazon.com/iot/latest/developerguide/device-defender-audit.html) - - - - -IOTOPS 6. How do you govern device fleet provisioning process? ---- - -IoT solutions can scale to millions of devices and this requires device fleets to be well planned from the perspectives of provisioning processes and metadata organization. Defining how devices are provisioned must include how the devices are manufactured and how they are registered. Maintain a full chain of security controls over who or what processes can start device provisioning to decrease the likelihood of inviting unintended, or rogue, devices into your fleet. - -**Best practice IOTOPS_6.1** – **Document how devices join your fleet from manufacturing to provisioning** - -Document the whole device provisioning process to clearly define the responsibilities of different actors at different stages. The end-to-end device provisioning process involves multiple stages owned by different actors. Documenting the plan and processes by which devices onboard and join the fleet affords the appropriate amount of review for potential gaps. - -**Recommendation IOTOPS_6.1.1** – _Document each step (manual and programmatic) of all the stages for the corresponding actors of that stage and clearly define the sequence_ - - * Identify the steps at each stage and the corresponding actors. - - * Device assembly by hardware manufacturer. - - * Device registration by service and solution provider. - - * Device activation by the end user of the service or solution provider. - - * Clearly define and document the dependencies and specific steps for each actor from device manufacturer to the end user. - - * Document whether devices can self-provision or are user-provisioned and how you can ensure that newly provisioned devices are yours. - - - - -**Recommendation IOTOPS_6.1.2** – _Assign device metadata to enable easy grouping and classification of devices in a fleet_ - - * The metadata can be used to group the devices in groups to search and force common actions and behaviors. - - * For example, you can assign the following metadata at the time of manufacturing: - - * Unique ID - - * Manufacturer details - - * Model number - - * Version or generation - - * Manufacturing date - - * If a particular model of a device requires a security patch, then you can easily target the patch to all the devices that are part of the corresponding model number group. Similarly, you can apply the patches to devices manufactured in a specific time frame or belonging to a particular version or generation. - - - - -**Best practice IOTOPS_6.2** – **Use programmatic techniques to provision devices at scale** - -Scaling the onboarding and provisioning of a large device fleet can be a bottleneck if there is even one manual step per device. Programmatic techniques define patterns of behavior for automating the provisioning process such that authenticated and authorized devices can onboard at any time. This practice ensures a well-documented, reliable, and programmatic provisioning mechanism that is consistent across all devices devoid of any human errors. - -**Recommendation IOTOPS_6.2.1** – _Embed provisioning claims into the devices that are mapped to approval authorities recognized by the provisioning service_ - - * Generate a provisioning claim and embed it into the device at the time of manufacturing. - - * AWS IoT Core can generate and securely deliver certificates and private keys to your devices when they connect to AWS IoT for the first time, using AWS IoT fleet provisioning. - - - - -**Recommendation IOTOPS_6.2.2** – _Use programmatic bootstrapping mechanisms if you are bringing your own certificates_ - - * Determine if you will or won’t have device information beforehand - - * If you don’t have device information beforehand, use just-in-time provisioning (JITP). - - * Enable automatic registration and associate a provisioning template with the CA certificate used to sign the device certificate. - - * For example, when a device attempts to connect to AWS IoT by using a certificate signed by a registered CA certificate, AWS IoT loads the template from the certificate and initiates the JITP workflow. - - * If you have device information beforehand, use bulk registration. - - * Specify a list of single-thing provisioning template values that are stored in a file in an S3 bucket. - - * Run the start-thing-registration-task command to register things in bulk. Provide provisioning template, S3 bucket name, a key name, and a role ARN to the command. - - - - -**Best practice IOTOPS_6.3 – Use device level features to enable re-provisioning** - -A birth or bootstrap certificate is a low-privilege unique certificate that is associated with each device during the manufacturing process. The certificate should have a policy to restrict devices to only allow connecting to specific endpoints to initiate provisioning process and fetch the final certificate. Before a device is provisioned, it should be limited in functionality to prevent its misuse. Only after a provisioning process is invoked and approved, should the device be allowed to operate on the system as designed. - -**Recommendation IOTOPS_6.3.1** – _Use a certificate bootstrapping process to establish processes for device assembly, registration, and activation_ - - * For example, AWS IoT Core offers a fleet provisioning interface to devices for upgrading a birth certificate to long-lived credentials that enable normal runtime operations. - - - - -**Recommendation IOTOPS_6.3.2** – _Obtain a list of allowed devices from the device manufacturer_ - - * Check the allow list file to validate that the device has been fully vetted by the supplier. - - * Ensure that the list is encrypted, securely stored, and can only be accessed by necessary services and users. Even if the list changes, keep the original list securely stored. -