diff --git a/apigw-lambda-bedrock-sam-node/README.md b/apigw-lambda-bedrock-sam-node/README.md new file mode 100644 index 000000000..2c4ac90cf --- /dev/null +++ b/apigw-lambda-bedrock-sam-node/README.md @@ -0,0 +1,93 @@ +# Amazon API Gateway response streaming with AWS Lambda integration + +This sample project demonstrates how to deploy Amazon API Gateway REST API with response streaming for AWS Lambda backend. The Lambda function invokes an Amazon Bedrock model to get response to user queries. The API Gateway streams the response back to the client. + + +## Requirements + +- [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources. +- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured +- [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) +- [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed +- [Node 24 or above](https://nodejs.org/en/download) installed + + + +## Deployment Instructions + +1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository: + + ```bash + git clone https://github.com/aws-samples/serverless-patterns + ``` + +2. Change directory to the pattern directory: + + ```bash + cd serverless-patterns/apigw-lambda-bedrock-sam-node + ``` + +3. From the command line, run the following commands: + + ```bash + sam build + sam deploy --guided + ``` + +4. During the prompts: + + - Enter a stack name + - Enter the desired AWS Region e.g. `us-east-1`. + - Allow SAM CLI to create IAM roles with the required permissions. + - Keep default values to the rest of the parameters. + + Once you have run `sam deploy --guided` mode once and saved arguments to a configuration file (samconfig.toml), you can use `sam deploy` in future to use these defaults. + +5. Note the outputs from the SAM deployment process. These contain the resource names and/or ARNs which are used for next step as well as testing. + +## How it works + +This SAM project uses Amazon Bedrock API to generate content based on given user prompt. This is exposed through a serverless REST API. Please refer to the architecture diagram below: + +![End to End Architecture](diagram/response-streaming.png) + +Here's a breakdown of the steps: + +The SAM template deploys two APIs following the same architecture - one with response streaming and another with buffered response. This is to demonstrate the difference in user experience. + +1. **Amazon API Gateway**: Receives the HTTP POST request containing the user prompt. + +2. **AWS Lambda**: The API Gateway invokes the Lambda functions which use either `InvokeModelWithResponseStreamCommand` or `InvokeModelCommand` to call Bedrock for streaming and non-streaming use cases. + +3. **Amazon Bedrock**: Based on the given prompt, generates the content and streams/ returns the response to the respective Lambda functions. + +4. **Response**: The API Gateway either streams the responses back to the client (Fig. 1) or returns the whole response together. + +## Testing + +Use [curl](https://curl.se/) to send a HTTP POST request to the API. Make sure to replace `api-id` with the one from your `sam deploy --guided` output: + +```bash +curl -X POST https://[api-id].execute-api.us-west-1.amazonaws.com/Prod/ask \ + -H "Content-Type: application/json" \ + -d '{"message": "Explain quantum computing in simple terms"}' +``` + +Test with both `NonStreamingApiUrl` and `StreamingApiUrl` to compare the difference. The `StreamingApiUrl` streams the output to the console as received while `NonStreamingApiUrl` buffers the whole response and renders together. + + + +## Cleanup + +1. To delete the resources deployed to your AWS account via AWS SAM, run the following command: + +```bash +sam delete +``` + + +--- + +Copyright 2026 Amazon.com, Inc. or its affiliates. All Rights Reserved. + +SPDX-License-Identifier: MIT-0 diff --git a/apigw-lambda-bedrock-sam-node/api-gateway-non-streaming-config.yaml b/apigw-lambda-bedrock-sam-node/api-gateway-non-streaming-config.yaml new file mode 100644 index 000000000..d9ed82318 --- /dev/null +++ b/apigw-lambda-bedrock-sam-node/api-gateway-non-streaming-config.yaml @@ -0,0 +1,33 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT-0 + +openapi: "3.0.1" +info: + title: "non-streaming-api" + version: "1.0.0" +paths: + /ask: + post: + requestBody: + content: + application/json: + schema: + type: object + properties: + message: + type: string + description: "The message to send to the Bedrock model" + required: + - message + responses: + "200": + description: "Complete response from Bedrock model" + content: + text/plain: + schema: + type: string + x-amazon-apigateway-integration: + uri: + Fn::Sub: "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${NonStreamingResponseFunction.Arn}/invocations" + httpMethod: "POST" + type: "aws_proxy" \ No newline at end of file diff --git a/apigw-lambda-bedrock-sam-node/api-gateway-streaming-config.yaml b/apigw-lambda-bedrock-sam-node/api-gateway-streaming-config.yaml new file mode 100644 index 000000000..1eaad000d --- /dev/null +++ b/apigw-lambda-bedrock-sam-node/api-gateway-streaming-config.yaml @@ -0,0 +1,35 @@ +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT-0 + +openapi: "3.0.1" +info: + title: "streaming-api" + version: "1.0.0" +paths: + /ask: + post: + requestBody: + content: + application/json: + schema: + type: object + properties: + message: + type: string + description: "The message to send to the Bedrock model" + required: + - message + responses: + "200": + description: "Streaming response from Bedrock model" + content: + text/plain: + schema: + type: string + description: "Newline-delimited JSON objects representing streaming events" + x-amazon-apigateway-integration: + uri: + Fn::Sub: "arn:aws:apigateway:${AWS::Region}:lambda:path/2021-11-15/functions/${StreamingResponseFunction.Arn}/response-streaming-invocations" + httpMethod: "POST" + responseTransferMode: "STREAM" + type: "aws_proxy" \ No newline at end of file diff --git a/apigw-lambda-bedrock-sam-node/diagram/response-streaming.png b/apigw-lambda-bedrock-sam-node/diagram/response-streaming.png new file mode 100644 index 000000000..6f1dc1f4b Binary files /dev/null and b/apigw-lambda-bedrock-sam-node/diagram/response-streaming.png differ diff --git a/apigw-lambda-bedrock-sam-node/example-pattern.json b/apigw-lambda-bedrock-sam-node/example-pattern.json new file mode 100644 index 000000000..89987c24e --- /dev/null +++ b/apigw-lambda-bedrock-sam-node/example-pattern.json @@ -0,0 +1,66 @@ +{ + "title": "Amazon API Gateway response streaming with AWS Lambda integration", + "description": "This sample project demonstrates how to deploy Amazon API Gateway REST API with response streaming for AWS Lambda backend.", + "language": "Node.js", + "level": "200", + "framework": "AWS SAM", + "introBox": { + "headline": "How it works", + "text": [ + "Amazon API Gateway receives the HTTP POST request containing the prompt.", + "The API Gateway triggers the Lambda functions which use either InvokeModelWithResponseStreamCommand or InvokeModelCommand to call Bedrock for streaming and non-streaming use cases.", + "The foundation model in Bedrock generates content and streams or returns the response to the respective Lambda functions.", + "The API Gateway either streams the responses back to the client or returns the whole response together." + ] + }, + "gitHub": { + "template": { + "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/apigw-lambda-bedrock-sam-node", + "templateURL": "serverless-patterns/apigw-lambda-bedrock-sam-node", + "projectFolder": "apigw-lambda-bedrock-sam-node", + "templateFile": "template.yaml" + } + }, + "resources": { + "bullets": [ + { + "text": "Stream the integration response for your proxy integrations in API Gateway", + "link": "https://docs.aws.amazon.com/apigateway/latest/developerguide/response-transfer-mode.html" + }, + { + "text": "AWS Blog - Building responsive APIs with Amazon API Gateway response streaming", + "link": "https://aws.amazon.com/blogs/compute/building-responsive-apis-with-amazon-api-gateway-response-streaming/" + } + ] + }, + "deploy": { + "text": [ + "sam build", + "sam deploy --guided" + ] + }, + "testing": { + "text": [ + "See the GitHub repo for detailed testing instructions." + ] + }, + "cleanup": { + "text": [ + "Delete the stack: sam delete." + ] + }, + "authors": [ + { + "name": "Biswanath Mukherjee", + "image": "https://serverlessland.com/assets/images/resources/contributors/biswanath-mukherjee.jpg", + "bio": "I am a Senior Solutions Architect working at AWS India. I help strategic global enterprise customer to architect their workload to run on AWS.", + "linkedin": "biswanathmukherjee" + }, + { + "name": "Giedrius Praspaliauskas", + "image": "https://serverlessland.com/assets/images/resources/contributors/giedrius-praspaliauskas.jpg", + "bio": "I am a Senior Specialist Solutions Architect for serverless based in California. I work with customers to help them leverage serverless services to build scalable, fault-tolerant, high-performing, cost-effective applications.", + "linkedin": "gpraspaliauskas" + } + ] +} diff --git a/apigw-lambda-bedrock-sam-node/src/nonstreaming/index.js b/apigw-lambda-bedrock-sam-node/src/nonstreaming/index.js new file mode 100644 index 000000000..16c51c1b2 --- /dev/null +++ b/apigw-lambda-bedrock-sam-node/src/nonstreaming/index.js @@ -0,0 +1,66 @@ +import { BedrockRuntimeClient, InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime"; + +const bedrockClient = new BedrockRuntimeClient({ + region: process.env.BEDROCK_REGION || 'us-east-1' +}); + +export const handler = async (event) => { + console.log('Non-streaming Event:', JSON.stringify(event, null, 2)); + + try { + // Extract the message from the request body + const body = JSON.parse(event.body || '{}'); + const prompt = body.message || body.prompt || 'Hello, how can I help you?'; + + // Configure the model parameters + const modelId = 'global.anthropic.claude-sonnet-4-5-20250929-v1:0'; + const payload = { + anthropic_version: "bedrock-2023-05-31", + max_tokens: 1000, + messages: [ + { + role: "user", + content: prompt + } + ] + }; + + // Create the command for non-streaming response + const command = new InvokeModelCommand({ + modelId: modelId, + body: JSON.stringify(payload), + contentType: 'application/json', + accept: 'application/json' + }); + + // Invoke the model without streaming + const response = await bedrockClient.send(command); + + // Parse the response + const responseBody = JSON.parse(new TextDecoder().decode(response.body)); + const content = responseBody.content?.[0]?.text || 'No response generated'; + + return { + statusCode: 200, + headers: { + 'Content-Type': 'text/plain', + 'Access-Control-Allow-Origin': '*', + 'Access-Control-Allow-Headers': 'Content-Type', + 'Access-Control-Allow-Methods': 'POST, OPTIONS' + }, + body: content + }; + + } catch (error) { + console.error('Non-streaming Error:', error); + + return { + statusCode: 500, + headers: { + 'Content-Type': 'text/plain', + 'Access-Control-Allow-Origin': '*' + }, + body: `Error: ${error.message}` + }; + } +}; \ No newline at end of file diff --git a/apigw-lambda-bedrock-sam-node/src/nonstreaming/package.json b/apigw-lambda-bedrock-sam-node/src/nonstreaming/package.json new file mode 100644 index 000000000..26254449e --- /dev/null +++ b/apigw-lambda-bedrock-sam-node/src/nonstreaming/package.json @@ -0,0 +1,20 @@ +{ + "name": "lambda-streaming-response", + "version": "1.0.0", + "description": "AWS Lambda function that streams responses", + "main": "index.js", + "type": "module", + "scripts": { + "test": "echo \"Error: no test specified\" && exit 1" + }, + "keywords": [ + "aws", + "lambda", + "streaming" + ], + "author": "", + "license": "ISC", + "dependencies": { + "@aws-sdk/client-bedrock-runtime": "^3.485.0" + } +} diff --git a/apigw-lambda-bedrock-sam-node/src/package.json b/apigw-lambda-bedrock-sam-node/src/package.json new file mode 100644 index 000000000..26254449e --- /dev/null +++ b/apigw-lambda-bedrock-sam-node/src/package.json @@ -0,0 +1,20 @@ +{ + "name": "lambda-streaming-response", + "version": "1.0.0", + "description": "AWS Lambda function that streams responses", + "main": "index.js", + "type": "module", + "scripts": { + "test": "echo \"Error: no test specified\" && exit 1" + }, + "keywords": [ + "aws", + "lambda", + "streaming" + ], + "author": "", + "license": "ISC", + "dependencies": { + "@aws-sdk/client-bedrock-runtime": "^3.485.0" + } +} diff --git a/apigw-lambda-bedrock-sam-node/src/streaming/index.js b/apigw-lambda-bedrock-sam-node/src/streaming/index.js new file mode 100644 index 000000000..32122c131 --- /dev/null +++ b/apigw-lambda-bedrock-sam-node/src/streaming/index.js @@ -0,0 +1,94 @@ +import { BedrockRuntimeClient, InvokeModelWithResponseStreamCommand } from "@aws-sdk/client-bedrock-runtime"; + +const bedrockClient = new BedrockRuntimeClient({ + region: process.env.BEDROCK_REGION || 'us-east-1' +}); + +const streamifyResponse = globalThis.awslambda?.streamifyResponse; +const HttpResponseStream = globalThis.awslambda?.HttpResponseStream; + +if (!streamifyResponse || !HttpResponseStream) { + throw new Error('Lambda streaming functionality not available. Ensure you are using Node.js 20.x or later.'); +} + +export const handler = streamifyResponse(async (event, responseStream, context) => { + console.log('Event:', JSON.stringify(event, null, 2)); + + try { + // Set up HTTP response metadata + const httpResponseMetadata = { + statusCode: 200, + headers: { + 'Content-Type': 'text/plain', + 'Access-Control-Allow-Origin': '*', + 'Access-Control-Allow-Headers': 'Content-Type', + 'Access-Control-Allow-Methods': 'POST, OPTIONS' + } + }; + + // Create the response stream with metadata + responseStream = HttpResponseStream.from(responseStream, httpResponseMetadata); + + // Extract the message from the request body + const body = JSON.parse(event.body || '{}'); + const prompt = body.message || body.prompt || 'Hello, how can I help you?'; + + // Configure the model parameters + const modelId = 'global.anthropic.claude-sonnet-4-5-20250929-v1:0'; + const payload = { + anthropic_version: "bedrock-2023-05-31", + max_tokens: 1000, + messages: [ + { + role: "user", + content: prompt + } + ] + }; + + // Create the command for streaming response + const command = new InvokeModelWithResponseStreamCommand({ + modelId: modelId, + body: JSON.stringify(payload), + contentType: 'application/json', + accept: 'application/json' + }); + + // Invoke the model with streaming + const response = await bedrockClient.send(command); + + // Stream the response chunks as they arrive + if (response.body) { + for await (const chunk of response.body) { + if (chunk.chunk?.bytes) { + const chunkData = JSON.parse(new TextDecoder().decode(chunk.chunk.bytes)); + if (chunkData.type === 'content_block_delta' && chunkData.delta?.text) { + responseStream.write(chunkData.delta.text); + } + } + } + } + + // End the response stream + responseStream.end(); + + } catch (error) { + console.error('Error:', error); + + // Set error response metadata + const errorResponseMetadata = { + statusCode: 500, + headers: { + 'Content-Type': 'application/json', + 'Access-Control-Allow-Origin': '*' + } + }; + + responseStream = HttpResponseStream.from(responseStream, errorResponseMetadata); + responseStream.write(JSON.stringify({ + error: 'Internal server error', + message: error.message + })); + responseStream.end(); + } +}); \ No newline at end of file diff --git a/apigw-lambda-bedrock-sam-node/src/streaming/package.json b/apigw-lambda-bedrock-sam-node/src/streaming/package.json new file mode 100644 index 000000000..26254449e --- /dev/null +++ b/apigw-lambda-bedrock-sam-node/src/streaming/package.json @@ -0,0 +1,20 @@ +{ + "name": "lambda-streaming-response", + "version": "1.0.0", + "description": "AWS Lambda function that streams responses", + "main": "index.js", + "type": "module", + "scripts": { + "test": "echo \"Error: no test specified\" && exit 1" + }, + "keywords": [ + "aws", + "lambda", + "streaming" + ], + "author": "", + "license": "ISC", + "dependencies": { + "@aws-sdk/client-bedrock-runtime": "^3.485.0" + } +} diff --git a/apigw-lambda-bedrock-sam-node/template.yaml b/apigw-lambda-bedrock-sam-node/template.yaml new file mode 100644 index 000000000..653ad8efc --- /dev/null +++ b/apigw-lambda-bedrock-sam-node/template.yaml @@ -0,0 +1,160 @@ + +# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. +# SPDX-License-Identifier: MIT-0 + +AWSTemplateFormatVersion: "2010-09-09" +Transform: AWS::Serverless-2016-10-31 +Description: SAM Template for Amazon API Gateway response streaming with AWS Lambda proxy integration +Resources: + # CloudWatch Log Group for API Gateway Access Logs + ApiGatewayAccessLogs: + Type: AWS::Logs::LogGroup + Properties: + RetentionInDays: 30 + LogGroupName: !Sub "/aws/apigateway/${AWS::StackName}/access-logs" + + # IAM Role for API Gateway to write to CloudWatch Logs + ApiGatewayCloudWatchRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: apigateway.amazonaws.com + Action: sts:AssumeRole + ManagedPolicyArns: + - arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs + + # API Gateway Account Configuration + ApiGatewayAccount: + Type: AWS::ApiGateway::Account + Properties: + CloudWatchRoleArn: !GetAtt ApiGatewayCloudWatchRole.Arn + + StreamingApi: + Type: AWS::Serverless::Api + DependsOn: ApiGatewayAccount + Properties: + StageName: Prod + EndpointConfiguration: + Type: REGIONAL + AccessLogSetting: + DestinationArn: !GetAtt ApiGatewayAccessLogs.Arn + Format: '{"requestId":"$context.requestId","requestTime":"$context.requestTime","httpMethod":"$context.httpMethod","path":"$context.path","resourcePath":"$context.resourcePath","status":$context.status,"responseLength":$context.responseLength,"responseTime":$context.responseTime,"userAgent":"$context.identity.userAgent","sourceIp":"$context.identity.sourceIp","protocol":"$context.protocol","error.message":"$context.error.message","error.messageString":"$context.error.messageString"}' + MethodSettings: + - ResourcePath: "/*" + HttpMethod: "*" + LoggingLevel: INFO + DataTraceEnabled: true + MetricsEnabled: true + DefinitionBody: + Fn::Transform: + Name: AWS::Include + Parameters: + Location: api-gateway-streaming-config.yaml + + StreamingResponseFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: src/streaming + Handler: index.handler + Runtime: nodejs24.x + Architectures: + - x86_64 + MemorySize: 256 + Timeout: 300 + Environment: + Variables: + BEDROCK_REGION: !Ref AWS::Region + Policies: + - AWSLambdaBasicExecutionRole + - Version: "2012-10-17" + Statement: + - Effect: Allow + Action: + - bedrock:InvokeModel + - bedrock:InvokeModelWithResponseStream + Resource: "*" + - Effect: Allow + Action: + - bedrock-runtime:InvokeModel + - bedrock-runtime:InvokeModelWithResponseStream + Resource: "*" + + # Non-streaming API Gateway + NonStreamingApi: + Type: AWS::Serverless::Api + DependsOn: ApiGatewayAccount + Properties: + StageName: Prod + EndpointConfiguration: + Type: REGIONAL + AccessLogSetting: + DestinationArn: !GetAtt ApiGatewayAccessLogs.Arn + Format: '{"requestId":"$context.requestId","requestTime":"$context.requestTime","httpMethod":"$context.httpMethod","path":"$context.path","resourcePath":"$context.resourcePath","status":$context.status,"responseLength":$context.responseLength,"responseTime":$context.responseTime,"userAgent":"$context.identity.userAgent","sourceIp":"$context.identity.sourceIp","protocol":"$context.protocol","error.message":"$context.error.message","error.messageString":"$context.error.messageString"}' + MethodSettings: + - ResourcePath: "/*" + HttpMethod: "*" + LoggingLevel: INFO + DataTraceEnabled: true + MetricsEnabled: true + DefinitionBody: + Fn::Transform: + Name: AWS::Include + Parameters: + Location: api-gateway-non-streaming-config.yaml + + NonStreamingResponseFunction: + Type: AWS::Serverless::Function + Properties: + CodeUri: src/nonstreaming + Handler: index.handler + Runtime: nodejs24.x + Architectures: + - x86_64 + MemorySize: 256 + Timeout: 300 + Environment: + Variables: + BEDROCK_REGION: !Ref AWS::Region + Policies: + - AWSLambdaBasicExecutionRole + - Version: "2012-10-17" + Statement: + - Effect: Allow + Action: + - bedrock:InvokeModel + - bedrock:InvokeModelWithResponseStream + Resource: "*" + - Effect: Allow + Action: + - bedrock-runtime:InvokeModel + - bedrock-runtime:InvokeModelWithResponseStream + Resource: "*" + + ApiGatewayInvokePermission: + Type: AWS::Lambda::Permission + Properties: + FunctionName: !Ref StreamingResponseFunction + Action: lambda:InvokeFunction + Principal: apigateway.amazonaws.com + SourceArn: !Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${StreamingApi}/*/*" + + ApiGatewayNonStreamingInvokePermission: + Type: AWS::Lambda::Permission + Properties: + FunctionName: !Ref NonStreamingResponseFunction + Action: lambda:InvokeFunction + Principal: apigateway.amazonaws.com + SourceArn: !Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${NonStreamingApi}/*/*" + +Outputs: + StreamingApiUrl: + Description: "API Gateway streaming endpoint URL" + Value: !Sub "https://${StreamingApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/ask" + + NonStreamingApiUrl: + Description: "API Gateway non-streaming endpoint URL" + Value: !Sub "https://${NonStreamingApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/ask"