New serverless pattern: sqs to lambda managed instance with provisioned mode esm

sqs-lambda-lmi-esm-provisioned-sam/README.md

@@ -0,0 +1,111 @@
+# Amazon SQS + Lambda Managed Instances (LMI) with Provisioned Mode ESM
+
+This pattern deploys an Amazon SQS Standard Queue connected to an AWS Lambda function via an Event Source Mapping (ESM) configured with **Provisioned Mode** — an ESM feature that pre-allocates dedicated polling resources for predictable, high-throughput message processing.
+
+Learn more about this pattern at Serverless Land Patterns: [https://serverlessland.com/patterns/sqs-lambda-lmi-esm-provisioned-sam](https://serverlessland.com/patterns/sqs-lambda-lmi-esm-provisioned-sam)
+
+Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
+
+## Requirements
+
+* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
+* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
+* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+* [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed
+
+## Build Instructions
+
+1. From the command line, use AWS SAM to build the AWS resources for the pattern as specified in the template.yml file:
+    ```
+    sam build
+    ```
+
+## Deployment Instructions
+
+1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
+    ``` 
+    git clone https://github.com/aws-samples/serverless-patterns
+    ```
+2. Change directory to the pattern directory:
+    ```
+    cd sqs-lambda-lmi-esm-provisioned-sam
+    ```
+1. From the command line, use AWS SAM to deploy the AWS resources for the pattern as specified in the template.yml file:
+    ```
+    sam deploy --guided
+    ```
+1. During the prompts:
+    * Enter a stack name
+    * Enter the desired AWS Region
+    * Accept the default parameter values or tune them for your workload
+    * Allow SAM CLI to create IAM roles with the required permissions.
+
+    Once you have run `sam deploy --guided` mode once and saved arguments to a configuration file (samconfig.toml), you can use `sam deploy` in future to use these defaults.
+
+1. Note the outputs from the SAM deployment process. These contain the resource names and/or ARNs which are used for testing.
+
+## How it works
+
+1. A producer (any AWS service, SDK client, or CLI) sends messages to the SQS Standard Queue.
+2. The ESM's provisioned event pollers continuously long-poll the queue using up to 10 SQS API calls per second per poller.
+3. When messages are available, pollers assemble batches (up to `BatchSize` messages, waiting up to `MaxBatchingWindowInSeconds`) and invoke the Lambda function concurrently.
+4. Lambda scales the number of active pollers between `MinimumPollers` and `MaximumPollers` based on queue depth, adding up to **1,000 concurrent executions per minute**.
+5. If a message fails processing after 3 attempts (`maxReceiveCount`), it is moved to the Dead Letter Queue.
+6. A CloudWatch Alarm fires as soon as any message lands in the DLQ.
+
+## Testing
+
+1. Get the queue URL from the stack outputs:
+
+```bash
+aws cloudformation describe-stacks \
+  --stack-name <your-stack-name> \
+  --query "Stacks[0].Outputs"
+```
+
+2. Send test messages:
+
+```bash
+QUEUE_URL=<QueueUrl from outputs>
+
+# Send a single message
+aws sqs send-message \
+  --queue-url $QUEUE_URL \
+  --message-body '{"orderId": "123", "amount": 99.99}'
+
+# Send a batch of 10
+for i in $(seq 1 10); do
+  aws sqs send-message \
+    --queue-url $QUEUE_URL \
+    --message-body "{\"orderId\": \"$i\", \"amount\": $((RANDOM % 100))}"
+done
+```
+
+3. Check Lambda logs:
+
+```bash
+sam logs --stack-name <your-stack-name> --tail
+```
+
+4. Inspect the ESM status and poller count:
+
+```bash
+ESM_ID=<EventSourceMappingId from outputs>
+
+aws lambda get-event-source-mapping --uuid $ESM_ID
+```
+
+## Cleanup
+
+1. Delete the stack
+    ```bash
+    aws cloudformation delete-stack --stack-name STACK_NAME
+    ```
+2. Confirm the stack has been deleted
+    ```bash
+    aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus"
+    ```
+----
+Copyright 2025 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+SPDX-License-Identifier: MIT-0

sqs-lambda-lmi-esm-provisioned-sam/example-pattern.json

@@ -0,0 +1,68 @@
+{
+  "title": "Amazon SQS to AWS Lambda with Lambda Managed Instances (LMI) Provisioned Mode ESM",
+  "description": "This pattern shows how to process Amazon SQS messages using AWS Lambda Managed Instances (LMI) with a Provisioned Mode Event Source Mapping for faster scaling and higher throughput.",
+  "language": "Python",
+  "level": "300",
+  "framework": "AWS SAM",
+  "introBox": {
+    "headline": "How it works",
+    "text": [
+      "A producer sends messages to an Amazon SQS Standard Queue. The Event Source Mapping (ESM) is configured with Provisioned Mode, which pre-allocates dedicated event pollers between a configurable minimum and maximum. Each poller handles up to 1 MB/s throughput, 10 concurrent Lambda invocations, and 10 SQS API calls per second.",
+      "The Lambda function runs on Lambda Managed Instances (LMI) — an EC2-backed execution model that supports multiconcurrency, allowing each execution environment to handle multiple SQS batches simultaneously. LMI instances run inside a private VPC with outbound access via a NAT Gateway.",
+      "Partial batch failure reporting (ReportBatchItemFailures) ensures only failed messages are returned to the queue. After 3 failed attempts, messages are moved to a Dead Letter Queue. A CloudWatch Alarm fires as soon as any message lands in the DLQ.",
+      "This pattern deploys one SQS Standard Queue, one Dead Letter Queue, one Lambda function with an LMI Capacity Provider, one Event Source Mapping with Provisioned Mode, a VPC with private subnets and NAT Gateway, and one CloudWatch Alarm."
+    ]
+  },
+  "gitHub": {
+    "template": {
+      "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/sqs-lambda-lmi-esm-provisioned-sam",
+      "templateURL": "serverless-patterns/sqs-lambda-lmi-esm-provisioned-sam",
+      "projectFolder": "sqs-lambda-lmi-esm-provisioned-sam",
+      "templateFile": "template.yaml"
+    }
+  },
+  "resources": {
+    "bullets": [
+      {
+        "text": "Using Lambda with Amazon SQS",
+        "link": "https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html"
+      },
+      {
+        "text": "Lambda Managed Instances (LMI) overview",
+        "link": "https://docs.aws.amazon.com/lambda/latest/dg/managed-instances.html"
+      },
+      {
+        "text": "Event Source Mapping Provisioned Mode for SQS",
+        "link": "https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html#sqs-esm-provisioned-mode"
+      },
+      {
+        "text": "Reporting batch item failures for SQS",
+        "link": "https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html#services-sqs-batchfailurereporting"
+      }
+    ]
+  },
+  "deploy": {
+    "text": [
+      "sam build",
+      "sam deploy --guided"
+    ]
+  },
+  "testing": {
+    "text": [
+      "See the GitHub repo for detailed testing instructions."
+    ]
+  },
+  "cleanup": {
+    "text": [
+      "Delete the stack: <code>aws cloudformation delete-stack --stack-name STACK_NAME</code>."
+    ]
+  },
+  "authors": [
+    {
+      "name": "Serda Kasaci Yildirim",
+      "image": "https://drive.google.com/file/d/1rzVS1hrIMdqy6P9i7-o7OBLNc0xY0FVB/view?usp=sharing",
+      "bio": "Serda is a Solutions Architect at Amazon Web Services (AWS) based in Vienna with 3 years at AWS, specializing in serverless technologies and Event-Driven Architecture.",
+      "linkedin": "serdakasaci"
+    }
+  ]
+}

sqs-lambda-lmi-esm-provisioned-sam/src/handler.py

@@ -0,0 +1,19 @@
+import json
+import logging
+
+logger = logging.getLogger()
+logger.setLevel(logging.INFO)
+
+
+def lambda_handler(event, context):
+    batch_item_failures = []
+
+    for record in event.get("Records", []):
+        try:
+            body = json.loads(record["body"])
+            logger.info("Processing message", extra={"message_id": record["messageId"], "body": body})
+        except Exception as e:
+            logger.error("Failed to process record %s: %s", record["messageId"], e)
+            batch_item_failures.append({"itemIdentifier": record["messageId"]})
+
+    return {"batchItemFailures": batch_item_failures}