AWS prescriptive-guidance documentation change
Summary
Removed 'Topics' section, updated navigation links, fixed apostrophes in text, reformatted code examples with consistent indentation, updated SageMaker service names ('AI Experiments' to 'Experiments', 'AI Debugger' to 'Debugger'), and corrected capitalization of 'Feature Store'.
Security assessment
Changes are editorial and structural: navigation updates, typo fixes, and service name corrections. No security vulnerabilities, mitigations, or security features are mentioned. The content focuses on general ML best practices without security implications.
Diff
diff --git a/prescriptive-guidance/latest/ml-operations-planning/training.md b/prescriptive-guidance/latest/ml-operations-planning/training.md index 18f583bc5..9b884119c 100644 --- a//prescriptive-guidance/latest/ml-operations-planning/training.md +++ b//prescriptive-guidance/latest/ml-operations-planning/training.md @@ -5 +5 @@ -[Documentation](/index.html)[AWS Prescriptive Guidance](https://aws.amazon.com/prescriptive-guidance/)[Planning for successful MLOps](welcome.html) +[Documentation](/index.html)[AWS Prescriptive Guidance](https://aws.amazon.com/prescriptive-guidance/)[Planning for successful MLOps](introduction.html) @@ -15,15 +14,0 @@ Follow the best practices in this section to help address model training challen -###### Topics - - * Create a baseline model - - * Use a data-centric approach and error analysis - - * Architect your model for fast iteration - - * Track your ML experiments - - * Troubleshoot training jobs - - - - @@ -32 +17 @@ Follow the best practices in this section to help address model training challen -When practitioners face a business problem with an ML solution, typically their first inclination is to use the state-of-the-art algorithm. This practice is risky, because it’s likely that the state-of-the-art algorithm hasn’t been time-tested. Moreover, the state-of-the-art algorithm is often more complex and not well understood, so it might result in only marginal improvements over simpler, alternative models. A better practice is to create a baseline model that is relatively quick to validate and deploy, and can earn the trust of the project stakeholders. +When practitioners face a business problem with an ML solution, typically their first inclination is to use the state-of-the-art algorithm. This practice is risky, because it's likely that the state-of-the-art algorithm hasn't been time-tested. Moreover, the state-of-the-art algorithm is often more complex and not well understood, so it might result in only marginal improvements over simpler, alternative models. A better practice is to create a baseline model that is relatively quick to validate and deploy, and can earn the trust of the project stakeholders. @@ -34 +19 @@ When practitioners face a business problem with an ML solution, typically their -When you create a baseline, we recommend that you evaluate its metric performance whenever possible. Compare the baseline model’s performance with other automated or manual systems to guarantee its success and to make sure that the model implementation or project can be delivered in the medium and long term. +When you create a baseline, we recommend that you evaluate its metric performance whenever possible. Compare the baseline model's performance with other automated or manual systems to guarantee its success and to make sure that the model implementation or project can be delivered in the medium and long term. @@ -36 +21 @@ When you create a baseline, we recommend that you evaluate its metric performanc -The baseline model should be further validated with ML engineers to confirm that the model can deliver the non-functional requirements that have been established for the project, such as inference time, how often the data is expected to shift distribution, if the model can be easily retrained in these cases, and how it will be deployed, which will affect the cost of the solution. Get multi-disciplinary viewpoints on these questions to increase the chance that you’re developing a successful and long-running model. +The baseline model should be further validated with ML engineers to confirm that the model can deliver the non-functional requirements that have been established for the project, such as inference time, how often the data is expected to shift distribution, if the model can be easily retrained in these cases, and how it will be deployed, which will affect the cost of the solution. Get multi-disciplinary viewpoints on these questions to increase the chance that you're developing a successful and long-running model. @@ -40 +25 @@ Data scientists might be inclined to add as many features as possible to a basel -Consider a model in which two input features are highly correlated, but only one feature has causality. For example, a model that predicts whether a loan will default might have input features such as customer age and income, which might be highly correlated, but only income should be used to give or deny a loan. A model that was trained on these two features might be relying on the feature that doesn’t have causality, such as age, to generate the prediction output. If, after going to production, the model receives inference requests for customers older or younger than the average age included in the training set, it might start to perform poorly. +Consider a model in which two input features are highly correlated, but only one feature has causality. For example, a model that predicts whether a loan will default might have input features such as customer age and income, which might be highly correlated, but only income should be used to give or deny a loan. A model that was trained on these two features might be relying on the feature that doesn't have causality, such as age, to generate the prediction output. If, after going to production, the model receives inference requests for customers older or younger than the average age included in the training set, it might start to perform poorly. @@ -72,2 +57,2 @@ For example, in the following code, you can add new experiments by just adding a - def __init__(self, *experiments): - self.experiments = experiments + def __init__(self, *experiments): + self.experiments = experiments @@ -75,4 +60,4 @@ For example, in the following code, you can add new experiments by just adding a - def run(self, df: DataFrame) -> None: - for experiment in self.experiments: - result = experiment.run(df) - print(f'Experiment "{experiment.name}" gave result {result}') + def run(self, df: DataFrame) -> None: + for experiment in self.experiments: + result = experiment.run(df) + print(f'Experiment "{experiment.name}" gave result {result}') @@ -83,3 +68,3 @@ For example, in the following code, you can add new experiments by just adding a - @abstractmethod - def run(self, df: DataFrame) -> float: - pass + @abstractmethod + def run(self, df: DataFrame) -> float: + pass @@ -87,4 +72,4 @@ For example, in the following code, you can add new experiments by just adding a - @property - @abstractmethod - def name(self) -> str: - pass + @property + @abstractmethod + def name(self) -> str: + pass @@ -95,3 +80,3 @@ For example, in the following code, you can add new experiments by just adding a - def run(self, df: DataFrame) -> float: - print('performing experiment 1') - return 0 + def run(self, df: DataFrame) -> float: + print('performing experiment 1') + return 0 @@ -99,2 +84,2 @@ For example, in the following code, you can add new experiments by just adding a - def name(self) -> str: - return 'experiment 1' + def name(self) -> str: + return 'experiment 1' @@ -105,3 +90,3 @@ For example, in the following code, you can add new experiments by just adding a - def run(self, df: DataFrame) -> float: - print('performing experiment 2') - return 0 + def run(self, df: DataFrame) -> float: + print('performing experiment 2') + return 0 @@ -109,2 +94,2 @@ For example, in the following code, you can add new experiments by just adding a - def name(self) -> str: - return 'experiment 2' + def name(self) -> str: + return 'experiment 2' @@ -115,3 +100,3 @@ For example, in the following code, you can add new experiments by just adding a - def run(self, df: DataFrame) -> float: - print('performing experiment 3') - return 0 + def run(self, df: DataFrame) -> float: + print('performing experiment 3') + return 0 @@ -119,2 +104,2 @@ For example, in the following code, you can add new experiments by just adding a - def name(self) -> str: - return 'experiment 3' + def name(self) -> str: + return 'experiment 3' @@ -124,8 +109,7 @@ For example, in the following code, you can add new experiments by just adding a - runner = ExperimentRunner(*[ - Experiment1(), - Experiment2(), - Experiment3() - ]) - df = ... - runner.run(df) - + runner = ExperimentRunner(*[ + Experiment1(), + Experiment2(), + Experiment3() + ]) + df = ... + runner.run(df) @@ -135 +119 @@ For example, in the following code, you can add new experiments by just adding a -When you work with a large number of experiments, it's important to gauge whether the improvements you observe are a product of implemented changes or chance. You can use [Amazon SageMaker AI Experiments](https://aws.amazon.com/blogs/aws/amazon-sagemaker-experiments-organize-track-and-compare-your-machine-learning-trainings/) to easily create experiments and associate metadata with them for tracking, comparison, and evaluation. +When you work with a large number of experiments, it's important to gauge whether the improvements you observe are a product of implemented changes or chance. You can use [Amazon SageMaker Experiments](https://aws.amazon.com/blogs/aws/amazon-sagemaker-experiments-organize-track-and-compare-your-machine-learning-trainings/) to easily create experiments and associate metadata with them for tracking, comparison, and evaluation. @@ -145 +129 @@ In some cases, it might be difficult for data scientists to fit even a very simp -Another helpful tool for debugging complex scenarios is [Amazon SageMaker AI Debugger](https://docs.aws.amazon.com/sagemaker/latest/dg/train-debugger.html), which can capture issues related to algorithmic correctness and infrastructure, such as optimal compute usage. +Another helpful tool for debugging complex scenarios is [Amazon SageMaker Debugger](https://docs.aws.amazon.com/sagemaker/latest/dg/train-debugger.html), which can capture issues related to algorithmic correctness and infrastructure, such as optimal compute usage. @@ -153 +137 @@ To use the Amazon Web Services Documentation, Javascript must be enabled. Please -Feature store +Feature Store