AWS solutions documentation change
Summary
Added documentation for using Amazon SageMaker AI as an LLM Provider, including endpoint creation, input schema configuration, and advanced parameters. Minor text formatting changes.
Security assessment
The changes primarily document integration with SageMaker AI endpoints and configuration details. While there is a note about endpoint lifecycle management to avoid charges, this relates to cost optimization rather than security. No evidence of addressing vulnerabilities or adding security controls.
Diff
diff --git a/solutions/latest/generative-ai-application-builder-on-aws/use-the-solution.md index 4a1366e99..6001bd35e 100644 --- a/solutions/latest/generative-ai-application-builder-on-aws/use-the-solution.md +++ b/solutions/latest/generative-ai-application-builder-on-aws/use-the-solution.md @@ -5 +5 @@ -Accessing the UIHow to update a deploymentHow to clone a deploymentHow to delete a deployment +Accessing the UIHow to update a deploymentHow to clone a deploymentHow to delete a deploymentUsing Amazon SageMaker AI as an LLM Provider @@ -11 +11 @@ Accessing the UIHow to update a deploymentHow to clone a deploymentHow to delete -During the stack deployment process (for both the Deployment dashboard and use cases) an email is sent to the configured email address. The email contains the user's temporary credentials they can use to sign up and access the web interface. +During the stack deployment process (for both the Deployment dashboard and use cases) an email is sent to the configured email address. The email contains the user’s temporary credentials they can use to sign up and access the web interface. @@ -50,0 +51,57 @@ Some resources are left behind during stack deletion and must be manually delete +## Using Amazon SageMaker AI as an LLM Provider + +As of v1.3.0, [Amazon SageMaker AI](https://aws.amazon.com/sagemaker/) is available as a model provider for Text use cases. This feature allows you to use a SageMaker AI inference endpoint already existing within the AWS account in the solution. Here are some ways to get started. + +###### Important + +The solution does not manage the lifecycle of your SageMaker AI endpoints. You are responsible for deleting the SageMaker AI endpoints once they are no longer needed to stop incurring additional charges. + +### Creating a SageMaker AI endpoint + +You can use [Amazon SageMaker AI JumpStart](https://aws.amazon.com/sagemaker/jumpstart/) to quickly deploy an endpoint. + +You can also use a text-generation based SageMaker AI endpoint and deploy using the base SageMaker AI service. Refer to the [SageMaker AI JumpStart documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-jumpstart.html) for a step by step guide on [how to deploy a model](https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-deploy.html) for inference. + +###### Note + +Foundation models/LLMs are typically quite large and can often require the use of large accelerated compute instances. Many of these larger instances might not be available by default in your AWS account. Refer to the default [SageMaker AI quotas](https://docs.aws.amazon.com/general/latest/gr/sagemaker.html#limits_sagemaker) and be sure to [request a quota increase](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html) before deploying to avoid common deployment failures. + +#### Use SageMaker AI endpoint to create a Text use case deployment + +To deploy a new Text use case using a SageMaker AI endpoint for inference: + + 1. [Create a new use case](./step-2-deploy-use-case.html) through the Deployment dashboard wizard and complete the forms until you reach the Models selection page. + + 2. On the Models page, select **SageMaker AI** as the model provider. This will generate a custom form requiring three key pieces of user input: + + * The name of the SageMaker AI endpoint you want to use. DevOps users can obtain this from the AWS console. Note that the endpoint must be in the same account and Region as the solution is deployed in. + +**Location of the endpoint name on the AWS console** + + + + * The schema of the input payload expected by the endpoint. To support the widest set of endpoints, admin users are required to tell the solution how their endpoint expects the input to be formatted. In the model selection wizard, provide the JSON schema for the solution to send to the endpoint. You can add placeholders to inject static and dynamic values into the request payload. The available options are: + + * Mandatory placeholders: \<\<prompt\>\> will be dynamically replaced with the full input (for example, history, context, and user input as per the prompt template) to be sent to the SageMaker AI endpoint at runtime. + + * Optional placeholders: \<\<temperature\>\> *,\\* as well as any parameters defined in advanced model parameters can be provided to the endpoint. Any string containing a placeholder enclosed in \<\< and \>\> (for example, \<\<max_new_tokens\>\>) will be replaced by the value of the advanced model parameter of the same name. + +**Example input schema - setting mandatory fields, prompt and temperature, along with a custom advanced parameter, max_new_tokens. Output path must be supplied as a valid JSONPath string** + + + + 3. The location of the LLMs generated string response within the output payload. This must be supplied as a JSONPath expression to indicate where the final text response shown to users is expected to be accessed from within the endpoint’s return object and response. + +**Example of adding Advanced model parameters to use within SageMaker AI input schema (see Figure 2 for previous options/settings)** + + + + + + +###### Note + +SageMaker AI now supports hosting multiple models behind the same endpoint, and this is the default configuration when deploying an endpoint in the current version of SageMaker AI Studio (not Studio Classic). + +If your endpoint is configured in this way, you will be required to add **InferenceComponentName** to the advanced model parameters section, with a value corresponding to the name of the model you want to use. +