Better handle indexing race condition for OL9 by shriyanshk128T · Pull Request #80 · 128technology/telegraf

shriyanshk128T · 2025-04-29T22:29:45Z

Description

This change fixes a race condition where lastObservedIndex updates could be lost during shutdown, leading to duplicated message processing. We now reliably track and persist the highest observed index using a buffered channel and wait group, draining it safely before exit. We also added log clarity to aid debugging.

Testing

Manual Testing

[Ember] Starting a TCP to Ember Unix Proxy on port 31337
Validates a Standard Valid Config :: Validate completes with no er... | PASS |
------------------------------------------------------------------------------
Validates a Nonstandard Valid Config :: Validate completes with no... | PASS |
------------------------------------------------------------------------------
Generates a Standard Valid Config :: Generate has no errors and cr... | PASS |
------------------------------------------------------------------------------
Generates a Nonstandard Valid Config :: Generate has no errors and... | PASS |
------------------------------------------------------------------------------
Configures a Standard Valid Config :: Configure has no errors and ... | PASS |
------------------------------------------------------------------------------
Configures a Nonstandard Valid Config :: Configure has no errors a... | PASS |
------------------------------------------------------------------------------
Runs a Standard Test Input :: Test input runs with no errors and p... | PASS |
------------------------------------------------------------------------------
Runs a Nonstandard Test Input :: Test input runs with no errors an... | PASS |
------------------------------------------------------------------------------
Runs a Standard Run Once :: Run once executes with no errors and p... | PASS |
------------------------------------------------------------------------------
Runs a Nonstandard Run Once :: Run once executes with no errors an... | PASS |
------------------------------------------------------------------------------
Produces a Standard Sample Agent Config :: Produces a sample agent... | PASS |
------------------------------------------------------------------------------
Produces a Nonstandard Sample Agent Config :: Produces a sample ag... | PASS |
------------------------------------------------------------------------------
Produces a Standard List of Available Inputs And Outputs :: Lists ... | PASS |
------------------------------------------------------------------------------
Produces a Nonstandard List of Available Inputs And Outputs :: Lis... | PASS |
------------------------------------------------------------------------------
Produces an Example Config for a Standard Input :: Produces a samp... | PASS |
------------------------------------------------------------------------------
Produces an Example Config for a Nonstandard Input :: Produces a s... | PASS |
------------------------------------------------------------------------------
Stops Standard Running Services :: Running services are stopped by... | PASS |
------------------------------------------------------------------------------
Stops Nonstandard Running Services :: Running services are stopped... | PASS |
------------------------------------------------------------------------------
Stops Only Services for Specified Config :: Unrelated services are... | PASS |
------------------------------------------------------------------------------
MA Service Operates On Standard Config :: Unrelated services are l... | PASS |
------------------------------------------------------------------------------
MA Service Runs Agent Setup On Start :: Unrelated services are lef... | PASS |
------------------------------------------------------------------------------
Validates Old Config are Removed :: Validate completes with no errors | PASS |
------------------------------------------------------------------------------
Validates Old Config left With Flag :: Validate completes with no ... | PASS |
------------------------------------------------------------------------------
Verify t128_metrics Sample By Name :: Verifies that the t128_metri... | PASS |
------------------------------------------------------------------------------
Verify t128_metrics configured output :: Verifies the configured o... | PASS |
------------------------------------------------------------------------------
Verify t128_metrics file output :: Verify the t128_metrics configu... | PASS |
------------------------------------------------------------------------------
Verify t128_metrics kafka output :: Verifies the t128_metrics conf... | PASS |
------------------------------------------------------------------------------
Verify Nonstandard kafka output :: Verifies the t128_metrics confi...
| PASS |
------------------------------------------------------------------------------
Verify t128_device_state Executable :: Verifies the t128_device_st... | PASS |
------------------------------------------------------------------------------
Verify t128_device_state Sample By Name :: Verifies that the t128_... | PASS |
------------------------------------------------------------------------------
Verify t128_device_state configured output :: Verifies the configu... | PASS |
------------------------------------------------------------------------------
Verify t128_device_state file output :: Verify the t128_peer_path ... | PASS |
------------------------------------------------------------------------------
Verify t128_device_state kafka output :: Verifies the t128_device_... | PASS |
------------------------------------------------------------------------------
Verify t128_peer_path Executable :: Verifies the t128_peer_path py... | PASS |
------------------------------------------------------------------------------
Verify t128_peer_path Sample By Name :: Verifies that the t128_pee... | PASS |
------------------------------------------------------------------------------
Verify t128_peer_path configured output :: Verifies the configured... | PASS |
------------------------------------------------------------------------------
Verify t128_peer_path file output :: Verify the t128_peer_path con... | PASS |
------------------------------------------------------------------------------
Verify t128_peer_path kafka output :: Verifies the t128_peer_path ... | PASS |
------------------------------------------------------------------------------
Verify t128_top_analytics Executable :: Verifies the t128_top_anal... | PASS |
------------------------------------------------------------------------------
Verify t128_top_analytics Sample By Name :: Verifies that the t128... | PASS |
------------------------------------------------------------------------------
Verify t128_top_analytics configured output :: Verifies the config... | PASS |
------------------------------------------------------------------------------
Verify t128_top_analytics file output :: Verify the t128_top_analy... | PASS |
------------------------------------------------------------------------------
Verify t128_top_analytics kafka output :: Verifies the t128_top_an... | PASS |
------------------------------------------------------------------------------
Verify t128_arp_state Executable :: Verifies the lytics t128_arp_s... | PASS |
------------------------------------------------------------------------------
Verify t128_arp_state Sample By Name :: Verifies that the t128_arp... | PASS |
------------------------------------------------------------------------------
Verify t128_arp_state configured output :: Verifies the configured... | PASS |
------------------------------------------------------------------------------
Verify t128_arp_state file output :: Verify the t128_arp_state con... | PASS |
------------------------------------------------------------------------------
Verify t128_arp_state kafka output :: Verifies the t128_arp_state ... | PASS |
------------------------------------------------------------------------------
Verify t128_graphql Sample By Name :: Verifies that the t128_graph... | PASS |
------------------------------------------------------------------------------
Verify t128_graphql configured output :: Verifies the configured o... | PASS |
------------------------------------------------------------------------------
Verify t128_graphql file output :: Verify the t128_graphql configu... | PASS |
------------------------------------------------------------------------------
Verify t128_graphql kafka output :: Verifies the t128_graphql conf... | PASS |
------------------------------------------------------------------------------
Verify t128_http Sample By Name :: Verifies that the t128_http sam... | PASS |
------------------------------------------------------------------------------
Verify t128_http configured output :: Verifies the configured outp... | PASS |
------------------------------------------------------------------------------
Verify t128_http file output :: Verify the t128_http configured ou... | PASS |
------------------------------------------------------------------------------
Verify t128_http kafka output :: Verifies the t128_http configured... | PASS |
------------------------------------------------------------------------------
Verify t128_cpu Sample By Name :: Verifies that the t128_cpu sampl... | PASS |
------------------------------------------------------------------------------
Verify t128_cpu configured output :: Verifies the configured outpu... | PASS |
------------------------------------------------------------------------------
Verify t128_cpu file output :: Verify the t128_cpu configured outp... | PASS |
------------------------------------------------------------------------------
Verify t128_cpu kafka output :: Verifies the t128_cpu configured o... | PASS |
------------------------------------------------------------------------------
Verify t128_events Sample By Name :: Verifies that the t128_events... | PASS |
------------------------------------------------------------------------------
Verify t128_events file output :: Verify the t128_events configure... | PASS |
------------------------------------------------------------------------------
Verify t128_events kafka output :: Verifies the t128_events config... | PASS |
------------------------------------------------------------------------------
Verify t128_events indexed output :: Verify the t128_events correc... | PASS |
------------------------------------------------------------------------------
Verify t128_tank File Descriptior Leak :: Ensure that t128_tank do... | PASS |
------------------------------------------------------------------------------
Verify t128_events output produces events                             | PASS |
------------------------------------------------------------------------------

WAN-4112 #time 1d

WAN-4112 #time 2h

shriyanshk128T requested review from BenMatase and gregschrock April 29, 2025 22:29

better handle indexing race condition

fcdb0bc

WAN-4112 #time 1d

shriyanshk128T force-pushed the WAN-4112-update-telegraf-for-ol-9-to-handle-the-indexing-race-condition branch from 63116aa to fcdb0bc Compare April 30, 2025 14:57

shriyanshk128T requested a review from agrawalkaushik April 30, 2025 16:00

shriyanshk128T marked this pull request as ready for review April 30, 2025 16:00

remove buffer changes and update defer func

3f3bfe1

WAN-4112 #time 2h

shriyanshk128T changed the title ~~Better handle indexing race condition~~ Better handle indexing race condition for OL9 May 1, 2025

gregschrock approved these changes May 7, 2025

View reviewed changes

shriyanshk128T merged commit 0c56460 into release-128tech-1.22.100-multiarch-support May 7, 2025
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Better handle indexing race condition for OL9#80

Better handle indexing race condition for OL9#80
shriyanshk128T merged 2 commits intorelease-128tech-1.22.100-multiarch-supportfrom
WAN-4112-update-telegraf-for-ol-9-to-handle-the-indexing-race-condition

shriyanshk128T commented Apr 29, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

shriyanshk128T commented Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

shriyanshk128T commented Apr 29, 2025 •

edited

Loading