AWS lambda documentation change
Summary
Restructured and updated Python layer documentation to focus on simplified packaging steps, added platform-specific installation guidance, and included a sample application. Removed detailed prerequisites and compatibility sections in favor of concise instructions.
Security assessment
The changes primarily improve documentation clarity and structure. No explicit security vulnerabilities, patches, or security feature additions are mentioned. Updates about platform compatibility (e.g., manylinux wheels) address runtime dependencies but do not directly mitigate security risks.
Diff
diff --git a/lambda/latest/dg/python-layers.md b/lambda/latest/dg/python-layers.md index b94e52b1f..3eac51f67 100644 --- a//lambda/latest/dg/python-layers.md +++ b//lambda/latest/dg/python-layers.md @@ -5 +5 @@ -PrerequisitesPython layer compatibility with Amazon LinuxLayer paths for Python runtimesPackaging the layer contentCreating the layerAdding the layer to your functionWorking with manylinux wheel distributions +Package your layer contentCreate the layer in LambdaAdd the layer to your functionSample app @@ -9 +9 @@ PrerequisitesPython layer compatibility with Amazon LinuxLayer paths for Python -A [Lambda layer](./chapter-layers.html) is a .zip file archive that contains supplementary code or data. Layers usually contain library dependencies, a [custom runtime](./runtimes-custom.html), or configuration files. Creating a layer involves three general steps: +Use [Lambda layers](./chapter-layers.html) to package code and dependencies that you want to reuse across multiple functions. Layers usually contain library dependencies, a [custom runtime](./runtimes-custom.html), or configuration files. Creating a layer involves three general steps: @@ -20 +20 @@ A [Lambda layer](./chapter-layers.html) is a .zip file archive that contains sup -This topic contains steps and guidance on how to properly package and create a Python Lambda layer with external library dependencies. +This topic explains how to create a Python layer and attach it to a Lambda function. @@ -24 +24 @@ This topic contains steps and guidance on how to properly package and create a P - * Prerequisites + * Package your layer content @@ -26 +26 @@ This topic contains steps and guidance on how to properly package and create a P - * Python layer compatibility with Amazon Linux + * Create the layer in Lambda @@ -28 +28 @@ This topic contains steps and guidance on how to properly package and create a P - * Layer paths for Python runtimes + * Add the layer to your function @@ -30 +30 @@ This topic contains steps and guidance on how to properly package and create a P - * Packaging the layer content + * Sample app @@ -32 +31,0 @@ This topic contains steps and guidance on how to properly package and create a P - * Creating the layer @@ -34 +32,0 @@ This topic contains steps and guidance on how to properly package and create a P - * Adding the layer to your function @@ -36 +33,0 @@ This topic contains steps and guidance on how to properly package and create a P - * Working with manylinux wheel distributions @@ -37,0 +35 @@ This topic contains steps and guidance on how to properly package and create a P +## Package your layer content @@ -38,0 +37 @@ This topic contains steps and guidance on how to properly package and create a P +To create a layer, bundle your packages into a .zip file archive that meets the following requirements: @@ -39,0 +39 @@ This topic contains steps and guidance on how to properly package and create a P + * Build the layer using the same Python version that you plan to use for the Lambda function. For example, if you build your layer using Python 3.13, use the Python 3.13 runtime for your function. @@ -41 +41 @@ This topic contains steps and guidance on how to properly package and create a P -## Prerequisites + * Your .zip file must include a `python` directory at the root level. @@ -43 +43 @@ This topic contains steps and guidance on how to properly package and create a P -To follow the steps in this section, you must have the following: + * The packages in your layer must be compatible with Linux. Lambda functions run on Amazon Linux. @@ -45 +44,0 @@ To follow the steps in this section, you must have the following: - * [Python 3.13](https://www.python.org/downloads/release/python-3130/) and the [pip](https://pip.pypa.io/en/stable/installation/) package installer @@ -47 +45,0 @@ To follow the steps in this section, you must have the following: - * [AWS CLI version 2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) @@ -49,0 +48 @@ To follow the steps in this section, you must have the following: +You can create layers that contain either third-party Python libraries installed with `pip` (such as `requests` or `pandas`) or your own Python modules and packages. @@ -50,0 +50 @@ To follow the steps in this section, you must have the following: +###### To create a layer using pip packages @@ -52 +52 @@ To follow the steps in this section, you must have the following: -Throughout this topic, we reference the [`layer-python`](https://github.com/awsdocs/aws-lambda-developer-guide/tree/main/sample-apps/layer-python) sample application on the **awsdocs** GitHub repository. This application contains scripts that download the dependencies and generate the layers. The application also contains corresponding functions that use dependencies from the layers. After creating a layer, you can deploy and invoke the corresponding function to verify that everything works properly. The runtime of the layer must be compatible with the runtime of the function. For example, if you use the Python 3.13 runtime for your function, your layer must also be compatible with Python 3.13. + 1. Choose one of the following methods to install `pip` packages into the required top-level directory (`python/`): @@ -54 +54 @@ Throughout this topic, we reference the [`layer-python`](https://github.com/awsd -In the `layer-python` sample application, there are two examples: +pip install @@ -56 +55,0 @@ In the `layer-python` sample application, there are two examples: - * The first example involves packaging the [`requests`](https://pypi.org/project/requests/) library into a Lambda layer. The `layer/` directory contains the scripts to generate the layer. The `function/` directory contains a sample function to help test that the layer works. The majority of this tutorial walks through how to create and package this layer. @@ -58 +57 @@ In the `layer-python` sample application, there are two examples: - * The second example involves packaging the [`numpy`](https://numpy.org/) library into a Lambda layer. The `layer-numpy/` directory contains the scripts to generate the layer. The `function-numpy/` directory contains a sample function to help test that the layer works. For an example of how to create and package this layer, see Working with manylinux wheel distributions. +For pure Python packages (like requests or boto3): @@ -59,0 +59 @@ In the `layer-python` sample application, there are two examples: + pip install requests -t python/ @@ -60,0 +61 @@ In the `layer-python` sample application, there are two examples: +Some Python packages, such as NumPy and Pandas, include compiled C components. If you're building a layer with these packages on macOS or Windows, you might need to use this command to install a Linux-compatible wheel: @@ -61,0 +63 @@ In the `layer-python` sample application, there are two examples: + pip install numpy --platform manylinux2014_x86_64 --only-binary=:all: -t python/ @@ -63 +65 @@ In the `layer-python` sample application, there are two examples: -## Python layer compatibility with Amazon Linux +For more information about working with Python packages that contain compiled components, see [Creating .zip deployment packages with native libraries](./python-package.html#python-package-native-libraries). @@ -65 +67 @@ In the `layer-python` sample application, there are two examples: -The first step to creating a layer is to bundle all of your layer content into a .zip file archive. Because Lambda functions run on [Amazon Linux](https://docs.aws.amazon.com/linux/al2023/ug/what-is-amazon-linux.html), your layer content must be able to compile and build in a Linux environment. +requirements.txt @@ -67 +68,0 @@ The first step to creating a layer is to bundle all of your layer content into a -In Python, most packages are available as [wheels](https://packaging.python.org/en/latest/glossary/#term-Wheel) (`.whl` files) in addition to the source distribution. Each wheel is a type of built distribution that supports a specific combination of Python versions, operating systems, and machine instruction sets. @@ -69,48 +70 @@ In Python, most packages are available as [wheels](https://packaging.python.org/ -Wheels are useful for ensuring that your layer is compatible with Amazon Linux. When you download your dependencies, download the _universal wheel_ if possible. (By default, `pip` installs the universal wheel if one is available.) The universal wheel contains `any` as the platform tag, indicating that it's compatible with all platforms, including Amazon Linux. - -In the example that follows, you package the `requests` library into a Lambda layer. The `requests` library is an example of a package that's available as a universal wheel. - -Not all Python packages are distributed as universal wheels. For example, [`numpy`](https://numpy.org/) has multiple wheel distributions, each supporting a different set of platforms. For such packages, download the `manylinux` distribution to ensure compatibility with Amazon Linux. For detailed instructions about how to package such layers, see Working with manylinux wheel distributions. - -In rare cases, a Python package might not be available as a wheel. If only the [source distribution](https://packaging.python.org/en/latest/overview/#python-source-distributions) (_sdist_) exists, then we recommend installing and packaging your dependencies in a [Docker](https://docs.docker.com/get-docker) environment based on the [Amazon Linux 2023 base container image](https://docs.aws.amazon.com/linux/al2023/ug/base-container.html). We also recommend this approach if you want to include your own custom libraries written in other languages such as C/C++. This approach mimics the Lambda execution environment in Docker, and ensures that your non-Python package dependencies are compatible with Amazon Linux. - -## Layer paths for Python runtimes - -When you add a layer to a function, Lambda loads the layer content into the `/opt` directory of that execution environment. For each Lambda runtime, the `PATH` variable already includes specific folder paths within the `/opt` directory. To ensure that Lambda picks up your layer content, your layer .zip file should have its dependencies in the following folder paths: - - * `python` - - * `python/lib/python3.`x`/site-packages` - - - - -For example, the resulting layer .zip file that you create in this tutorial has the following directory structure: - - - layer_content.zip - └ python - └ lib - └ python3.13 - └ site-packages - └ requests - └ <other_dependencies> (i.e. dependencies of the requests package) - └ ... - -The [`requests`](https://pypi.org/project/requests/) library is correctly located in the `python/lib/python3.13/site-packages` directory. This ensures that Lambda can locate the library during function invocations. - -## Packaging the layer content - -In this example, you package the Python `requests` library in a layer .zip file. Complete the following steps to install and package the layer content. - -###### To install and package your layer content - - 1. Clone the [`aws-lambda-developer-guide` GitHub repo](https://github.com/awsdocs/aws-lambda-developer-guide), which contains the sample code that you need in the `sample-apps/layer-python` directory. - - git clone https://github.com/awsdocs/aws-lambda-developer-guide.git - - 2. Navigate to the `layer` directory of the `layer-python` sample app. This directory contains the scripts that you use to create and package the layer properly. - - cd aws-lambda-developer-guide/sample-apps/layer-python/layer - - 3. Examine the [`requirements.txt`](https://github.com/awsdocs/aws-lambda-developer-guide/blob/main/sample-apps/layer-python/layer/requirements.txt) file. This file defines the dependencies that you want to include in the layer, namely the `requests` library. You can update this file to include any dependencies that you want to include in your own layer. +Using a `requirements.txt` file helps you manage package versions and ensure consistent installations. @@ -120,0 +75,2 @@ In this example, you package the Python `requests` library in a layer .zip file. + boto3==1.37.34 + numpy==1.26.4 @@ -122,3 +78 @@ In this example, you package the Python `requests` library in a layer .zip file. - 4. Ensure that you have permissions to run both scripts. - - chmod 744 1-install.sh && chmod 744 2-package.sh +If your `requirements.txt` file includes only pure Python packages (like requests or boto3): @@ -126 +80 @@ In this example, you package the Python `requests` library in a layer .zip file. - 5. Run the [`1-install.sh`](https://github.com/awsdocs/aws-lambda-developer-guide/blob/main/sample-apps/layer-python/layer/1-install.sh) script using the following command: + pip install -r requirements.txt -t python/ @@ -128 +82 @@ In this example, you package the Python `requests` library in a layer .zip file. - ./1-install.sh +Some Python packages, such as NumPy and Pandas, include compiled C components. If you're building a layer with these packages on macOS or Windows, you might need to use this command to install a Linux-compatible wheel: @@ -130 +84 @@ In this example, you package the Python `requests` library in a layer .zip file. -This script uses `venv` to create a Python virtual environment named `create_layer`. It then installs all required dependencies in the `create_layer/lib/python3.11/site-packages` directory. + pip install -r requirements.txt --platform manylinux2014_x86_64 --only-binary=:all: -t python/ @@ -132 +86 @@ This script uses `venv` to create a Python virtual environment named `create_lay -###### Example 1-install.sh +For more information about working with Python packages that contain compiled components, see [Creating .zip deployment packages with native libraries](./python-package.html#python-package-native-libraries). @@ -134,3 +88 @@ This script uses `venv` to create a Python virtual environment named `create_lay - python3.13 -m venv create_layer - source create_layer/bin/activate - pip install -r requirements.txt + 2. Zip the contents of the `python` directory. @@ -138 +90 @@ This script uses `venv` to create a Python virtual environment named `create_lay - 6. Run the [`2-package.sh`](https://github.com/awsdocs/aws-lambda-developer-guide/blob/main/sample-apps/layer-python/layer/2-package.sh) script using the following command: + zip -r layer.zip python/ @@ -140 +92 @@ This script uses `venv` to create a Python virtual environment named `create_lay - ./2-package.sh +The directory structure of your .zip file should look like this: @@ -142 +94,5 @@ This script uses `venv` to create a Python virtual environment named `create_lay -This script copies the contents from the `create_layer/lib` directory into a new directory named `python`. It then zips the contents of the `python` directory into a file named `layer_content.zip`. This is the .zip file for your layer. You can unzip the file and verify that it contains the correct file structure, as shown in the Layer paths for Python runtimes section. + python/ _# Required top-level directory_ + └── requests/ + └── boto3/ + └── numpy/ + └── (dependencies of the other packages) @@ -144 +100 @@ This script copies the contents from the `create_layer/lib` directory into a new -###### Example 2-package.sh +###### Note @@ -146,3 +102 @@ This script copies the contents from the `create_layer/lib` directory into a new - mkdir python - cp -r create_layer/lib python/ - zip -r layer_content.zip python +If you use a Python virtual environment (venv) to install packages, your directory structure will be different (for example, `python/lib/python3.`x`/site-packages`). As long as your .zip file includes the `python` directory at the root level, Lambda can locate and import your packages. @@ -153 +107 @@ This script copies the contents from the `create_layer/lib` directory into a new -## Creating the layer +###### To create a layer using your own code @@ -155 +109 @@ This script copies the contents from the `create_layer/lib` directory into a new -In this section, you take the `layer_content.zip` file that you generated in the previous section and upload it as a Lambda layer. You can upload a layer using the AWS Management Console or the Lambda API via the AWS Command Line Interface (AWS CLI). When you upload your layer .zip file, in the following [PublishLayerVersion](https://docs.aws.amazon.com/lambda/latest/api/API_PublishLayerVersion.html) AWS CLI command, specify `python3.13` as the compatible runtime and `arm64` as the compatible architecture. + 1. Create the required top-level directory for your layer: @@ -156,0 +111 @@ In this section, you take the `layer_content.zip` file that you generated in the + mkdir python @@ -158,27 +113 @@ In this section, you take the `layer_content.zip` file that you generated in the - aws lambda publish-layer-version --layer-name python-requests-layer \ - --zip-file fileb://layer_content.zip \ - --compatible-runtimes python3.13 \ - --compatible-architectures "arm64" - -From the response, note the `LayerVersionArn`, which looks like `arn:aws:lambda:us-east-1:`123456789012`:layer:python-requests-layer:1`. You'll need this Amazon Resource Name (ARN) in the next step of this tutorial, when you add the layer to your function. - -## Adding the layer to your function - -In this section, you deploy a sample Lambda function that uses the `requests` library in its function code, then you attach the layer. To deploy the function, you need an [execution role](./lambda-intro-execution-role.html). If you don't have an existing execution role, follow the steps in the collapsible section. - -###### To create an execution role - - 1. Open the [roles page](https://console.aws.amazon.com/iam/home#/roles) in the IAM console. - - 2. Choose **Create role**. - - 3. Create a role with the following properties. -