Skip to content

[GP2] - adds slack webhook to receive 5xx API errors #4729

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AimeurAmin wants to merge 4 commits into
base: master
Choose a base branch
from
Open

[GP2] - adds slack webhook to receive 5xx API errors #4729

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2e85103
adds slack webhook package
AimeurAmin Oct 29, 2025
5b340af
links slack webhook to alert 5xx erros on GP2 API
AimeurAmin Oct 29, 2025
06ea52a
adds slack alert test
AimeurAmin Oct 29, 2025
ed64b3e
better logs with test fix
AimeurAmin Nov 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .pnp.cjs
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions apps/gp2-server/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
"@graphql-typed-document-node/core": "3.2.0",
"@hapi/boom": "9.1.4",
"@sentry/serverless": "6.19.7",
"@slack/webhook": "^6.1.0",
"@types/yargs": "17.0.5",
"ajv": "8.11.0",
"algoliasearch": "4.22.1",
Expand Down
53 changes: 53 additions & 0 deletions apps/gp2-server/serverless.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ const contentfulSpaceId = process.env.CONTENTFUL_SPACE_ID!;
const contentfulWebhookAuthenticationToken =
process.env.CONTENTFUL_WEBHOOK_AUTHENTICATION_TOKEN!;
const openaiApiKey = process.env.OPENAI_API_KEY!;
const slackWebhook = process.env.SLACK_WEBHOOK || '';

if (stage === 'dev' || stage === 'production') {
['SENTRY_DSN_API', 'SENTRY_DSN_PUBLIC_API', 'SENTRY_DSN_HANDLERS'].forEach(
Expand Down Expand Up @@ -253,6 +254,8 @@ const serverlessConfig: AWS = {
bundle: true,
concurrency: 8,
},
apiGateway5xxTopic:
'${self:service}-${self:provider.stage}-topic-api-gateway-5xx',
s3Sync: [
{
bucketName: '${self:service}-${self:provider.stage}-gp2-frontend',
Expand Down Expand Up @@ -288,6 +291,21 @@ const serverlessConfig: AWS = {
SENTRY_DSN: sentryDsnPublicApi,
},
},
sendSlackAlert: {
handler: './src/handlers/send-slack-alert.handler',
events: [
{
sns: {
arn: { Ref: 'TopicCloudwatchAlarm' },
topicName: '${self:custom.apiGateway5xxTopic}',
},
},
],
environment: {
SLACK_WEBHOOK: slackWebhook,
ENVIRONMENT: '${env:SLS_STAGE}',
},
},
apiHandler: {
handler: 'src/handlers/api-handler.apiHandler',
events: [
Expand Down Expand Up @@ -1624,6 +1642,41 @@ const serverlessConfig: AWS = {
MessageRetentionPeriod: 1209600, // 14 days
},
},
ApiGatewayAlarm5xx: {
Type: 'AWS::CloudWatch::Alarm',
Properties: {
AlarmDescription: '5xx errors detected at API Gateway',
Namespace: 'AWS/ApiGateway',
MetricName: '5xx',
Statistic: 'Sum',
Threshold: 0,
ComparisonOperator: 'GreaterThanThreshold',
EvaluationPeriods: 1,
Period: 60,
AlarmActions: [{ Ref: 'TopicCloudwatchAlarm' }],
TreatMissingData: 'notBreaching',
Dimensions: [
{
Name: 'ApiId',
Value: {
Ref: 'HttpApi',
},
},
{
Name: 'Stage',
Value: {
Ref: 'HttpApiStage',
},
},
],
},
},
TopicCloudwatchAlarm: {
Type: 'AWS::SNS::Topic',
Properties: {
TopicName: '${self:custom.apiGateway5xxTopic}',
},
},
},
extensions: {
SubscribeCalendarLambdaFunction: {
Expand Down
31 changes: 31 additions & 0 deletions apps/gp2-server/src/handlers/send-slack-alert.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import { IncomingWebhook } from '@slack/webhook';
import logger from '../utils/logger';

export async function handler() {
const url = process.env.SLACK_WEBHOOK;
const appEnv = process.env.ENVIRONMENT;
const region = process.env.AWS_REGION;

if (!url) {
logger.warn('SLACK_WEBHOOK not configured, skipping alert');
return;
}

// Validate that the webhook URL is a proper URL, not localhost
if (!url.startsWith('https://hooks.slack.com/')) {
throw new Error(
`Invalid SLACK_WEBHOOK configuration: URL must start with 'https://hooks.slack.com/' - current value: ${url}`,
);
}

const webhook = new IncomingWebhook(url);

try {
await webhook.send({
text: `🚨 *GP2-${appEnv}*: API Gateway 5xx errors detected\n• Environment: ${appEnv}\n• Time: ${new Date().toISOString()}\n• [View CloudWatch Alarm](https://console.aws.amazon.com/cloudwatch/home?region=${region}#alarmsV2:alarm/API%20Gateway%205xx%20errors%20detected%20at%20API%20Gateway)\n• [View API Gateway](https://console.aws.amazon.com/apigateway/main/apis)`,
});
} catch (error) {
logger.warn('Failed to send Slack alert:', error);
throw error;
}
}
86 changes: 86 additions & 0 deletions apps/gp2-server/test/handlers/send-slack-alert.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
export {};

const mockSend = jest.fn();
const mockLogger = {
warn: jest.fn(),
};

jest.mock('@slack/webhook', () => {
return {
IncomingWebhook: jest.fn().mockImplementation((url) => {
return {
send: async (...args: any) => mockSend(...args),
};
}),
};
});

jest.mock('../../src/utils/logger', () => ({
__esModule: true,
default: mockLogger,
}));

describe('sendSlackAlert', () => {
const OLD_ENV = process.env;

beforeEach(() => {
jest.resetModules();
jest.resetAllMocks();
process.env = { ...OLD_ENV };
});

it('sends message to slack', async () => {
process.env.SLACK_WEBHOOK = 'https://hooks.slack.com/services/TEST/WEBHOOK';
process.env.ENVIRONMENT = 'test';
process.env.AWS_REGION = 'us-east-1';

const { handler } = await import('../../src/handlers/send-slack-alert');

await handler();

expect(mockSend).toBeCalledWith({
text: expect.stringContaining(
'🚨 *GP2-test*: API Gateway 5xx errors detected',
),
});
});

it('does not sends message to slack, when webhook url is missing', async () => {
const { handler } = await import('../../src/handlers/send-slack-alert');

await handler();

expect(mockSend).not.toBeCalled();
});

it('throws error when webhook URL is not a valid Slack URL', async () => {
process.env.SLACK_WEBHOOK = 'https://example.com/webhook';
process.env.ENVIRONMENT = 'test';
process.env.AWS_REGION = 'us-east-1';

const { handler } = await import('../../src/handlers/send-slack-alert');

await expect(handler()).rejects.toThrow(
"Invalid SLACK_WEBHOOK configuration: URL must start with 'https://hooks.slack.com/'",
);

expect(mockSend).not.toBeCalled();
});

it('logs and re-throws error when sending to Slack fails', async () => {
process.env.SLACK_WEBHOOK = 'https://hooks.slack.com/services/TEST/WEBHOOK';
process.env.ENVIRONMENT = 'test';
process.env.AWS_REGION = 'us-east-1';

const sendError = new Error('Network error');
mockSend.mockRejectedValueOnce(sendError);

const { handler } = await import('../../src/handlers/send-slack-alert');

await expect(handler()).rejects.toThrow('Network error');
expect(mockLogger.warn).toHaveBeenCalledWith(
'Failed to send Slack alert:',
sendError,
);
});
});
1 change: 1 addition & 0 deletions yarn.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1520,6 +1520,7 @@ __metadata:
"@hapi/boom": 9.1.4
"@sentry/serverless": 6.19.7
"@serverless/typescript": 3.38.0
"@slack/webhook": ^6.1.0
"@types/apicache": ^1
"@types/apr-intercept": "workspace:*"
"@types/aws-lambda": 8.10.157
Expand Down
Loading