fix: Reduce the chance of CurlEventPublisher exhausting file descriptors on high throughput PHP-FPM servers#216
Merged
keelerm84 merged 3 commits intolaunchdarkly:mainfrom Mar 25, 2025
Conversation
See https://curl.se/docs/manpage.html - this helps make sure background processes aren't piling up.
CurlEventPublisher exhausting file descriptors on high throughput PHP-FPM serversCurlEventPublisher exhausting file descriptors on high throughput PHP-FPM servers
CurlEventPublisher exhausting file descriptors on high throughput PHP-FPM serversCurlEventPublisher exhausting file descriptors on high throughput PHP-FPM servers
keelerm84
reviewed
Mar 13, 2025
| $scheme = $this->_ssl ? "https://" : "http://"; | ||
| $args = " -X POST"; | ||
| $args.= " --connect-timeout " . $this->_connectTimeout; | ||
| $args.= " --max-time " . $this->_connectTimeout; |
Member
There was a problem hiding this comment.
The LDClient config already allows specifying both a connect_timeout and a max length timeout parameter. We use that timeout parameter in the existing Guzzle client.
Can you update this so that we still support the connect time parameter as before, and also add the max time flag hooked up to that value?
keelerm84
approved these changes
Mar 25, 2025
Member
|
Thank you @Watercycle for the contribution. I went ahead and made that minor modification. I appreciate your time and effort on this work. |
abarker-launchdarkly
pushed a commit
that referenced
this pull request
Mar 25, 2025
🤖 I have created a release *beep* *boop* --- ## [6.5.2](6.5.1...6.5.2) (2025-03-25) ### Bug Fixes * Honor `timeout` configuration for CurlEventPublisher ([#216](#216)) ([1c969be](1c969be)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requirements
Related issues
Describe the solution you've provided
TLDR:
--connect-timeoutvs--max-time(docs)Consider the following code:
+
However long the request takes is how long file descriptors hang open for.
Web apps like Laravel frequently use PHP-FPM as part of their server infrastructure where each request is a new process. As a result, on high throughput servers, EventProcessor.php::__destruct gets invoked quite frequently. Having many child worker processes seems to also mean new sub-processes consuming many file descriptors. While the Relay Proxy is great at allowing the PHP SDK to quickly read from data stores like Redis, the impressions event processing system ends up quickly bottle-necking if there's any slow-down in publishing events.
Describe alternatives you've considered
I think ideally:
CurlEventPublishershould use the actualcurlclient or something likeGuzzlefor making async requests instead of usingshell_exec.CurlEventPublishershould possibly explore batch requesting strategies that won't choke out technologies like PHP-FPM (e.g. using the file system + a separate worker thread). With more consideration for making sure these processes terminate - like logging warnings when it happens.I ultimately stuck with a simple change to minimize the chance of it being a breaking change.
Additional context
This is related to an issue I filed a bit ago: https://support.launchdarkly.com/hc/en-us/requests/87721