AWS solutions documentation change
Summary
Removed detailed documentation about using Amazon SageMaker AI as an LLM provider including endpoint creation, input schema configuration, and advanced parameters. Retained general security recommendation about using AWS services for sensitive data.
Security assessment
The removed content was operational documentation about SageMaker integration, not security-specific. The retained security statement about data residency is a general best practice recommendation without evidence of addressing a specific vulnerability.
Diff
diff --git a/solutions/latest/generative-ai-application-builder-on-aws/configuring-an-llm.md index d712cc2ca..b013460d7 100644 --- a/solutions/latest/generative-ai-application-builder-on-aws/configuring-an-llm.md +++ b/solutions/latest/generative-ai-application-builder-on-aws/configuring-an-llm.md @@ -5,2 +4,0 @@ -Using Amazon SageMaker AI as an LLM Provider - @@ -15,58 +13 @@ The AI-generated space is evolving rapidly, so it is incumbent on you to keep up -If you're working with non-public or sensitive data, then be sure to select an LLM option using AWS services (such as Amazon Bedrock or Amazon SageMaker AI). This improves the overall security posture of your deployment by keeping data within your Region and on the AWS network when compared to using an LLM hosted by a third-party provider. - -## Using Amazon SageMaker AI as an LLM Provider - -As of v1.3.0, [Amazon SageMaker AI](https://aws.amazon.com/sagemaker/) is available as a model provider for Text use cases. This feature allows you to use a SageMaker AI inference endpoint already existing within the AWS account in the solution. Here are some ways to get started. - -###### Important - -The solution does not manage the lifecycle of your SageMaker AI endpoints. You are responsible for deleting the SageMaker AI endpoints once they are no longer needed to stop incurring additional charges. - -### Creating a SageMaker AI endpoint - -You can use [Amazon SageMaker AI JumpStart](https://aws.amazon.com/sagemaker/jumpstart/) to quickly deploy an endpoint. - -You can also use a text-generation based SageMaker AI endpoint and deploy using the base SageMaker AI service. Refer to the [SageMaker AI JumpStart documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-jumpstart.html) for a step by step guide on [how to deploy a model](https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-deploy.html) for inference. - -###### Note - -Foundation models/LLMs are typically quite large and can often require the use of large accelerated compute instances. Many of these larger instances might not be available by default in your AWS account. Refer to the default [SageMaker AI quotas](https://docs.aws.amazon.com/general/latest/gr/sagemaker.html#limits_sagemaker) and be sure to [request a quota increase](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) before deploying to avoid common deployment failures. - -### Use SageMaker AI endpoint to create a Text use case deployment - -To deploy a new Text use case using a SageMaker AI endpoint for inference: - - 1. [Create a new use case](./step-2-deploy-use-case.html) through the Deployment dashboard wizard and complete the forms until you reach the Models selection page. - - 2. On the Models page, select **SageMaker AI** as the model provider. This will generate a custom form requiring three key pieces of user input: - - * The name of the SageMaker AI endpoint you want to use. DevOps users can obtain this from the AWS console. Note that the endpoint must be in the same account and Region as the solution is deployed in. - - - -_**Location of the endpoint name on the AWS console**_ - - * The schema of the input payload expected by the endpoint. To support the widest set of endpoints, admin users are required to tell the solution how their endpoint expects the input to be formatted. In the model selection wizard, provide the JSON schema for the solution to send to the endpoint. You can add placeholders to inject static and dynamic values into the request payload. The available options are: - - * Mandatory placeholders: <<prompt>> will be dynamically replaced with the full input (for example, history, context, and user input as per the prompt template) to be sent to the SageMaker AI endpoint at runtime. - - * Optional placeholders: <<temperature>>**,** as well as any parameters defined in advanced model parameters can be provided to the endpoint. Any string containing a placeholder enclosed in << and >> (for example, <<max_new_tokens>>) will be replaced by the value of the advanced model parameter of the same name. - - - -_**Example input schema - setting mandatory fields, prompt and temperature, along with a custom advanced parameter, max_new_tokens. Output path must be supplied as a valid JSONPath string**_ - - 3. The location of the LLMs generated string response within the output payload. This must be supplied as a JSONPath expression to indicate where the final text response shown to users is expected to be accessed from within the endpoint's return object and response. - - - -_**Example of adding Advanced model parameters to use within SageMaker AI input schema (see Figure 2 for previous options/settings)**_ - - - - -###### Note - -SageMaker AI now supports hosting multiple models behind the same endpoint, and this is the default configuration when deploying an endpoint in the current version of SageMaker AI Studio (not Studio Classic). - -If your endpoint is configured in this way, you will be required to add **InferenceComponentName** to the advanced model parameters section, with a value corresponding to the name of the model you want to use. +If you’re working with non-public or sensitive data, then be sure to select an LLM option using AWS services (such as Amazon Bedrock or Amazon SageMaker AI). This improves the overall security posture of your deployment by keeping data within your Region and on the AWS network when compared to using an LLM hosted by a third-party provider.