AWS sagemaker documentation change
Summary
Updated vLLM image version, removed KV caching configuration section, and added reference to new documentation page
Security assessment
Routine dependency version update and documentation restructuring. Removed note about VPC encryption but didn't add new security guidance. No vulnerability fixes mentioned.
Diff
diff --git a/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-deploy-ftm.md b/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-deploy-ftm.md index 6ae3c77d5..4d545a6e7 100644 --- a//sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-deploy-ftm.md +++ b//sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-deploy-ftm.md @@ -7 +7 @@ -PrerequisitesSetup and configurationConfigure KV caching and intelligent routing for improved performanceDeploy your model from Amazon S3, Amazon FSx, or Hugging Face HubVerify the status of your deploymentManage your deployment +PrerequisitesSetup and configurationDeploy your model from Amazon S3, Amazon FSx, or Hugging Face HubVerify the status of your deploymentManage your deployment @@ -87 +87 @@ For clusters with GPU partitioning enabled, replace `nvidia.com/gpu` with the ap - image: vllm/vllm-openai:v0.10.1 + image: vllm/vllm-openai:v0.19.1 @@ -152 +152 @@ For clusters with GPU partitioning enabled, replace `nvidia.com/gpu` with the ap - image: vllm/vllm-openai:v0.10.1 + image: vllm/vllm-openai:v0.19.1 @@ -227 +227 @@ For clusters with GPU partitioning enabled, replace `nvidia.com/gpu` with the ap - image: "vllm/vllm-openai:v0.10.1" + image: "vllm/vllm-openai:v0.19.1" @@ -272,11 +271,0 @@ For clusters with GPU partitioning enabled, replace `nvidia.com/gpu` with the ap -## Configure KV caching and intelligent routing for improved performance - - 1. Enable KV caching by setting `enableL1Cache` and `enableL2Cache` to `true`.Then, set `l2CacheSpec` to `redis` and update `l2CacheLocalUrl` with the Redis cluster URL. - - kvCacheSpec: - enableL1Cache: true - enableL2Cache: true - l2CacheSpec: - l2CacheBackend: <redis | tieredstorage> - l2CacheLocalUrl: <redis cluster URL if l2CacheBackend is redis > - @@ -285,24 +274 @@ For clusters with GPU partitioning enabled, replace `nvidia.com/gpu` with the ap -If the redis cluster is not within the same Amazon VPC as the HyperPod cluster, encryption for the data in transit is not guaranteed. - -###### Note - -Do not need l2CacheLocalUrl if tieredstorage is selected. - - 2. Enable intelligent routing by setting `enabled` to `true` under `intelligentRoutingSpec`. You can specify which routing strategy to use under `routingStrategy`. If no routing strategy is specified, it defaults to `prefixaware`. - - intelligentRoutingSpec: - enabled: true - routingStrategy: <routing strategy to use> - - 3. Enable router metrics and caching metrics by setting `enabled` to `true` under `metrics`. The `port` value needs to be the same as the `containerPort` value under `modelInvocationPort`. - - metrics: - enabled: true - modelMetrics: - port: <port value> - ... - modelInvocationPort: - containerPort: <port value> - - - +To configure KV caching and intelligent routing for improved performance, see [Configure KV caching and intelligent routing](./sagemaker-hyperpod-model-deployment-caching-routing.html#sagemaker-hyperpod-model-deployment-deploy-ftm-cache-route).