-
Notifications
You must be signed in to change notification settings - Fork 810
Open
Description
Description:
Problem
Currently, when a consumer group removes topics from its subscription list, Burrow continues to track and report these removed topics as errors (stopped partitions) until either:
- The Burrow service is manually restarted, or
- The
/v3/kafka/(cluster)/consumer/(group)DELETE endpoint is manually called
This creates operational overhead and can lead to false alerts in monitoring systems.
Current Workaround:
As mentioned in #14, the current solutions require manual intervention:
- Restart Burrow to refresh consumer group configurations
- Use the DELETE API endpoint to manually drop the consumer group
Proposed Enhancement:
Implement automatic detection of topic removal from consumer groups by:
- Periodic reconciliation: Burrow should periodically compare the list of topics currently consumed by a group (from Kafka metadata) against the topics it's tracking internally
- Topic removal detection: When a topic is no longer present in the consumer group's subscription but still exists in Burrow's internal state, automatically remove it from tracking
- Configurable interval: Add a configuration parameter (e.g., topic-reconciliation-interval) to control how frequently this check occurs (default: 5-10 minutes)
Metadata
Metadata
Assignees
Labels
No labels