AWS opensearch-service documentation change
Summary
Added warning about SLA exclusions for issues caused by user-developed plugin code and clarified security scanning during package validation
Security assessment
Mentions Amazon Inspector vulnerability scanning during package creation, but does not reference any specific CVE or security incident. The SLA exclusion notice is a liability clarification rather than a security fix.
Diff
diff --git a/opensearch-service/latest/developerguide/custom-plugins.md b/opensearch-service/latest/developerguide/custom-plugins.md index dfd7af290..c6e5f89e6 100644 --- a//opensearch-service/latest/developerguide/custom-plugins.md +++ b//opensearch-service/latest/developerguide/custom-plugins.md @@ -5 +5 @@ -Plugin limitsUsing custom plugins with OpenSearch Service +LimitationsPrerequisitesInstalling custom plugins @@ -7 +7 @@ Plugin limitsUsing custom plugins with OpenSearch Service -# Custom plugins +# Installing custom plugins in Amazon OpenSearch Service @@ -9 +9 @@ Plugin limitsUsing custom plugins with OpenSearch Service -Custom plugins for Amazon OpenSearch Service is a new plugin management option that extends OpenSearch functionality in areas like language analysis, custom filtering, and ranking, enabling you to craft personalized search experiences. Custom plugins for OpenSearch can be developed by extending the `org.opensearch.plugins.Plugin` class and then packaged as a .zip file. The following plugin extensions are currently supported by Amazon OpenSearch Service: +Custom plugins for Amazon OpenSearch Service extend the functionality of OpenSearch by allowing you to add new features, modify existing behavior, or integrate with external systems. @@ -11 +11 @@ Custom plugins for Amazon OpenSearch Service is a new plugin management option t - * Analysis plugin: Extends analysis functionality by adding custom analyzers, character tokenizers, or filters for text processing. +Custom plugins contain user-developed code. Any issues, including SLA breaches, caused by user developed code aren't eligible for SLA credits. For more information, see [Amazon OpenSearch Service - Service Level Agreement](https://aws.amazon.com/opensearch-service/sla/). @@ -13 +13 @@ Custom plugins for Amazon OpenSearch Service is a new plugin management option t - * Search plugin: Enhances search capabilities with custom query types, similarity algorithms, suggest options, and aggregations. +###### Topics @@ -14,0 +15 @@ Custom plugins for Amazon OpenSearch Service is a new plugin management option t + * Limitations @@ -15,0 +17 @@ Custom plugins for Amazon OpenSearch Service is a new plugin management option t + * Prerequisites @@ -16,0 +19 @@ Custom plugins for Amazon OpenSearch Service is a new plugin management option t + * Installing custom plugins @@ -18 +20,0 @@ Custom plugins for Amazon OpenSearch Service is a new plugin management option t -You can use the Amazon OpenSearch Service console or existing APIs for custom packages to upload and associate a plugin with your Amazon OpenSearch Service domain, for more information on custom packages, see [Custom packages for Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/custom-packages.html). Additonally, you can use `DescribePackages` to describe all the packages in your account to view details such as what OpenSearch version is currently in use or error details. Amazon OpenSearch Service validates plugin package for version compatibility, security vulnerabilities, and permitted plugin operations. @@ -20 +21,0 @@ You can use the Amazon OpenSearch Service console or existing APIs for custom pa -Custom plugins are supported on OpenSearch Service domains running OpenSearch version 2.15 or later, and are available in 14 regions globally: US West (Oregon), US East (Ohio), US East (N. Virginia), South America (Sao Paulo), Europe(Paris), Europe (London), Europe (Ireland), Europe (Frankfurt), Canada (Central), Asia Pacific (Tokyo), Asia Pacific (Sydney), Asia Pacific (Singapore), Asia Pacific (Seoul) and Asia Pacific (Mumbai). @@ -22 +22,0 @@ Custom plugins are supported on OpenSearch Service domains running OpenSearch ve -###### Note @@ -24 +24 @@ Custom plugins are supported on OpenSearch Service domains running OpenSearch ve -Custom plugins contain user developed code. Any issues, including SLA breaches, caused by user developed code will not be eligible for SLA credits. For more information, see Amazon OpenSearch Service SLA exclusions under Amazon OpenSearch Service - [Service Level Agreement](https://aws.amazon.com/opensearch-service/sla/). +## Limitations @@ -26 +26 @@ Custom plugins contain user developed code. Any issues, including SLA breaches, -## Plugin limits + * You can create up to 25 custom plugins per account. @@ -28 +28 @@ Custom plugins contain user developed code. Any issues, including SLA breaches, -You can create up to 25 custom plugins per account. Maximum number of plugins that can be associated with a single domain is 20, this number is inclusive of all plugin types i.e. optional, third party or custom. Maximum allowed uncompressed size for a plugin is 1 GB. + * The maximum allowed uncompressed size for a plugin is 1 GB. @@ -30 +30 @@ You can create up to 25 custom plugins per account. Maximum number of plugins th -The following table lists the features that are not available when using custom plugins: + * Custom plugins are supported on domains running OpenSearch version 2.15 or later. @@ -32,8 +32 @@ The following table lists the features that are not available when using custom -Amazon OpenSearch Service features | Custom plugins ----|--- -[Cross cluster search](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/cross-cluster-search.html) | Not supported. -[Cross cluster replication](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/replication.html) | Not supported -[Remote-reindex](https://docs.aws.amazon.com/prescriptive-guidance/latest/opensearch-service-migration/remote-reindexing.html) | Not supported -[Auto-tune](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/auto-tune.html) | Not supported -[Multi-AZ with Standby](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-multiaz.html#managedomains-za-standby) | Not supported -[Centralized OpenSearch user Interface](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/application.html) | Not supported + * The `descriptor.properties` file for your plugin must support an engine version similar to 2.15.0 or any 2.x.x version, where the patch version is set to zero. @@ -41 +34 @@ Amazon OpenSearch Service features | Custom plugins -## Using custom plugins with OpenSearch Service + * The following features are unavailable when your domain uses custom plugins: @@ -43 +36 @@ Amazon OpenSearch Service features | Custom plugins -### Prerequisites for using custom plugins with OpenSearch Service + * Cross-cluster search @@ -45 +38 @@ Amazon OpenSearch Service features | Custom plugins -Before you can use custom plugins with Amazon OpenSearch Service you will need to be sure you have the following set up: + * Cross-cluster replication @@ -47 +40 @@ Before you can use custom plugins with Amazon OpenSearch Service you will need t - * [Node to node encryption](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ntn.html) + * Remote reindex @@ -49 +42 @@ Before you can use custom plugins with Amazon OpenSearch Service you will need t - * [Encryption of data at rest](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/encryption-at-rest.html#enabling-ear) + * Auto-Tune @@ -51 +44 @@ Before you can use custom plugins with Amazon OpenSearch Service you will need t - * [EnforceHTTPS](https://docs.aws.amazon.com/config/latest/developerguide/opensearch-https-required.html) set to `true` + * UltraWarm @@ -53 +46 @@ Before you can use custom plugins with Amazon OpenSearch Service you will need t - * Clients must support TLSSecurityPolicy ‘`Policy-Min-TLS-1-2-PFS-2023-10'` , you can set this up with following command: + * Multi-AZ with Standby @@ -55 +47,0 @@ Before you can use custom plugins with Amazon OpenSearch Service you will need t - aws opensearch update-domain-config —domain-name domain-name —domain-endpoint-options '{"TLSSecurityPolicy":"Policy-Min-TLS-1-2-PFS-2023-10" }' @@ -57 +48,0 @@ Before you can use custom plugins with Amazon OpenSearch Service you will need t -For more information, see [DomainEndpointOptions](https://docs.aws.amazon.com/opensearch-service/latest/APIReference/API_DomainEndpointOptions.html). @@ -59 +49,0 @@ For more information, see [DomainEndpointOptions](https://docs.aws.amazon.com/op - * The descriptor.properties file for your plugin's supported engine version should be similar to 2.15.0 or the 2.x.0.i.e patch version should be zero. @@ -60,0 +51 @@ For more information, see [DomainEndpointOptions](https://docs.aws.amazon.com/op +## Prerequisites @@ -61,0 +53 @@ For more information, see [DomainEndpointOptions](https://docs.aws.amazon.com/op +Before you install a custom plugin and associate it to a domain, make sure you meet the following requirements: @@ -62,0 +55 @@ For more information, see [DomainEndpointOptions](https://docs.aws.amazon.com/op + * Your domain uses TLS policy **Policy-Min-TLS-1-2-PFS-2023-10**. For more information, see [DomainEndpointOptions](https://docs.aws.amazon.com/opensearch-service/latest/APIReference/API_DomainEndpointOptions.html). @@ -64 +57 @@ For more information, see [DomainEndpointOptions](https://docs.aws.amazon.com/op -### Installing custom plugins with AWS CLI + * You enabled the following features on your domain: @@ -66 +59 @@ For more information, see [DomainEndpointOptions](https://docs.aws.amazon.com/op -You can install custom plugins using the AWS CLI. Before you can associate a custom plugin with your domain, you must upload it to an Amazon S3 bucket. You must create the Amazon S3 bucket in the same region you intend to use the plugin. For instructions on how to do this, see [Uploading objects](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/upload-objects.html) in the [What is Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) guide. If your plugin contains sensitive information, select [server-side encryption with S3-managed keys](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) when you upload it. After you upload the file, make note of its Amazon S3 path. See the following example Amazon S3 path format: + * [Node-to-node encryption](./ntn.html) @@ -67,0 +61 @@ You can install custom plugins using the AWS CLI. Before you can associate a cus + * [Encryption at rest](./encryption-at-rest.html) @@ -69 +63 @@ You can install custom plugins using the AWS CLI. Before you can associate a cus - s3://bucket-name/file-path/file-name + * [Require HTTPS for all traffic to the domain](./createupdatedomains.html) @@ -71 +64,0 @@ You can install custom plugins using the AWS CLI. Before you can associate a cus -You will need to create a new package for your custom plugin. You can do this using the existing [CreatePackage](https://docs.aws.amazon.com/cli/latest/reference/opensearch/create-package.html) API. When creating your new package, please update the bucket and key location to point to your custom plugin's .zip file in the calling account’s Amazon S3 bucket. Note that, your Amazon S3 bucket must be in the same region as the package being created. Only .zip files are supported for `ZIP-PLUGIN` packages. The contents of the .zip file must follow directory structure as expected by the plugin. To create a package, see the following example: @@ -74 +66,0 @@ You will need to create a new package for your custom plugin. You can do this us - aws opensearch --region $REGION create-package --package-name <package-name> --package-type ZIP-PLUGIN --package-source S3BucketName=<bucket>,S3Key=<key> --engine-version OpenSearch_2.15 @@ -76 +68 @@ You will need to create a new package for your custom plugin. You can do this us -You can view the status of the create package operation, including any validation and security vulnerability findings errors by using the [describe-packages](https://docs.aws.amazon.com/cli/latest/reference/es/describe-packages.html?highlight=describepackages). To do this, see the following example: +## Installing custom plugins @@ -77,0 +70 @@ You can view the status of the create package operation, including any validatio +To associate a third-party plugin to a domain, first import the plugin license and configuration as packages. @@ -79 +72 @@ You can view the status of the create package operation, including any validatio - aws opensearch --region $REGION describe-packages --filters '[{"Name": "PackageType","Value": ["ZIP-PLUGIN"]}, {"Name": "PackageName","Value": ["<package-name>"]}]' +###### To install a custom plugin @@ -81 +74 @@ You can view the status of the create package operation, including any validatio -The following is a sample response of the [describe-packages](https://docs.aws.amazon.com/cli/latest/reference/es/describe-packages.html?highlight=describepackages) API: + 1. Sign in to the Amazon OpenSearch Service console at [https://console.aws.amazon.com/aos/home](https://console.aws.amazon.com/aos/home). @@ -82,0 +76 @@ The following is a sample response of the [describe-packages](https://docs.aws.a + 2. In the left navigation pane, choose **Packages**. @@ -84,13 +78 @@ The following is a sample response of the [describe-packages](https://docs.aws.a - { "PackageDetailsList": [ { - "PackageID": "pkg-identifier", - "PackageName": "custom-plugin-test", - "PackageType": "ZIP-PLUGIN", - "PackageStatus": "VALIDATION_FAILED", - "CreatedAt": "2024-11-11T13:07:18.297000-08:00", - "LastUpdatedAt": "2024-11-11T13:10:13.843000-08:00", - "ErrorDetails": - { "ErrorType": "", "ErrorMessage": - "PluginValidationFailureReason : Dependency Scan reported 3 vulnerabilities for the plugin: CVE-2022-23307, CVE-2019-17571, CVE-2022-23305" }, - "EngineVersion": "OpenSearch_2.15", - "AllowListedUserList": [], - "PackageOwner": "OWNER-XXXX" } ] } + 3. Choose **Import package**. @@ -98 +80 @@ The following is a sample response of the [describe-packages](https://docs.aws.a -###### Note + 4. Enter a descriptive name for the plugin. @@ -100 +82 @@ The following is a sample response of the [describe-packages](https://docs.aws.a -During the create package operation, Amazon OpenSearch Service checks the `ZIP-PLUGIN` for version compatibility, supported plugin extensions and security vulnerabilities. In particular the security vulnerabilities are scanned using the [Amazon Inspector service](https://aws.amazon.com/inspector/getting-started/). The results of these checks are shown in the `ErrorDetails` field of the API response. + 5. For **Package type** , choose **Plugin**. @@ -102 +84 @@ During the create package operation, Amazon OpenSearch Service checks the `ZIP-P -Use the [AssociatePackage](https://docs.aws.amazon.com/opensearch-service/latest/APIReference/API_AssociatePackage.html) API to associate the plugin with your Amazon OpenSearch Service domain using the package id of package created in the previous step. If you have multiple plugins, you can use the [AssociatePackages](https://docs.aws.amazon.com/opensearch-service/latest/APIReference/API_AssociatePackages.html) API to associate multiple packages to a domain in a single operation. To do this, see the following example: + 6. For **Package source** , enter the path to the plugin ZIP file in Amazon S3. @@ -103,0 +86 @@ Use the [AssociatePackage](https://docs.aws.amazon.com/opensearch-service/latest + 7. For **OpenSearch engine version** , choose the version of OpenSearch that the plugin supports. @@ -105 +88 @@ Use the [AssociatePackage](https://docs.aws.amazon.com/opensearch-service/latest - aws opensearch --region $REGION associate-package --domain-name <domain-name> --package-id <package-id> + 8. For **Package encryption** , choose whether to customize the enryption key for the package. By default, OpenSearch Service encrypts the plugin package with an AWS owned key. You can use a customer managed key instead. @@ -107 +90 @@ Use the [AssociatePackage](https://docs.aws.amazon.com/opensearch-service/latest -###### Note + 9. Choose **Import**. @@ -109 +91,0 @@ Use the [AssociatePackage](https://docs.aws.amazon.com/opensearch-service/latest -Plugins are installed and uninstalled using a [blue/green deployment process](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-configuration-changes.html). @@ -111 +92,0 @@ Plugins are installed and uninstalled using a [blue/green deployment process](ht -You can use the [ListPackagesForDomain](https://docs.aws.amazon.com/cli/latest/reference/opensearch/list-packages-for-domain.html) API to see the status of the association. The association status will change as the workflow progresses from `ASSOCIATING` to `ACTIVE`. The association status changes to `ACTIVE` once the plugin installation workflow is complete and your plugin is ready to be used. To do this, see the following example: @@ -114,2 +95 @@ You can use the [ListPackagesForDomain](https://docs.aws.amazon.com/cli/latest/r - aws opensearch --region $REGION list-packages-for-domain - --domain-name <domain-name> +After you import the plugin package, associate it to a domain. For instructions, see [Import and associate a package to a domain](./custom-packages.html#associate-console). @@ -117 +97 @@ You can use the [ListPackagesForDomain](https://docs.aws.amazon.com/cli/latest/r -### Updating custom plugins +To import a custom plugin using the AWS CLI, use the [create-package](https://docs.aws.amazon.com/cli/latest/reference/opensearch/create-package.html). Optionally, include the `package-encryption-options` parameter to specify a customer managed key. @@ -119 +98,0 @@ You can use the [ListPackagesForDomain](https://docs.aws.amazon.com/cli/latest/r -You can update custom plugins using the existing [UpdatePackage](https://docs.aws.amazon.com/cli/latest/reference/opensearch/update-package.html) API. You can use the following associate-packages API example to apply package updates to a domain. To do this, see the following example: @@ -121,27 +100,10 @@ You can update custom plugins using the existing [UpdatePackage](https://docs.aw - - aws opensearch --region $REGION update-package --package-id <package-id> --package-source S3BucketName=<bucket>,S3Key=<key> --package-description <description> - -###### Note - -You can audit create, update, associate and disassociate operations on your plugin using AWS CloudTrail. For more information refer to the documentation [Monitoring Amazon OpenSearch Service API calls with AWS CloudTrail](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-cloudtrailauditing.html). - -### Upgrading your domain with custom plugins - -To upgrade your Amazon OpenSearch Service domain that has associated custom plugins to a later version of OpenSearch, you can use the [CreatePackage](https://docs.aws.amazon.com/cli/latest/reference/opensearch/create-package.html) API to create a new package for your plugin. - -###### Note - -Please ensure that the package name is the same for the plugin for all engine versions. Changing the package name will cause the upgrade domain process to fail during the blue/green deployment. - -For instructions on upgrading your Amazon OpenSearch Service domain, see [Upgrading Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/version-migration.html) . Amazon OpenSearch Service will disassociate the previous version of your plugin package and install the new version of the plugin through a blue/green deployment. - -### Encrypting custom plugins - -When using the [CreatePackage](https://docs.aws.amazon.com/cli/latest/reference/opensearch/create-package.html) API , you can set the `PackageEncryptionOptions` to `true` and pass the KMS key ARN that you would like to use for encryption. To do this, see the following example: - - - aws opensearch --region $REGION create-package --package-name <package-name> --package-type ZIP-PLUGIN --package-source S3BucketName=<bucket>,S3Key=<key> --engine-version OpenSearch_2.15 - "PackageConfigOptions": { - ... - } - "PackageEncryptionOptions": { + aws opensearch create-package \ + --package-name my-package \ + --package-type ZIP-PLUGIN \ + --package-source S3BucketName=my-bucket,S3Key=my-key \ + --engine-version OpenSearch_2.15 \ + --package-config-options '{ + "Option1": "value1", + "Option2": "value2" + }' \ + --package-encryption-options '{ @@ -149,21 +111,2 @@ When using the [CreatePackage](https://docs.aws.amazon.com/cli/latest/reference/ - "KmsKeyId":"kms_key_arn" - }