Skip to content

AXON-1600: Unable to stop response when "Rate limit exceeded" #1476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
jwang19-atlassian wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

AXON-1600: Unable to stop response when "Rate limit exceeded" #1476

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions src/rovo-dev/rovoDevChatProvider.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -301,6 +301,14 @@ export class RovoDevChatProvider {
const replayBuffer: RovoDevResponse[] = [];

while (replayInProgress) {
// Check for cancellation during replay
if (this.pendingCancellation) {
Copy link
Collaborator

@marcomura marcomura Jan 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about this logic.

You are setting pending cancellation to false when Rovo Dev didn't yet confirm the cancellation.
Which means, you will allow the user to send a new prompt, which is going to fail because there is another prompt being generated (since cancellation didn't happen yet).
The existing logic is blocking the user from sending a new prompt, which is intentional.

Also, even closing the socket with Rovo Dev doesn't result in a cancellation. Rovo Dev can still generate the response when the socket is disconnected.

Moreover, the cancellation API can return a failure. In a perfect world it shouldn't, but it's possible.
In that case we want to surface the error to the user and let them know that the stream is still happening.

await reader.cancel();
isDone = true;
this._pendingCancellation = false;
break;
}

const { done, value } = await reader.read();
if (done) {
isDone = true;
Expand All @@ -323,6 +331,14 @@ export class RovoDevChatProvider {
}

while (!isDone) {
// Check for cancellation before each read
if (this.pendingCancellation) {
await reader.cancel();
isDone = true;
this._pendingCancellation = false;
break;
}

const { done, value } = await reader.read();
if (done) {
isDone = true;
Expand All @@ -336,6 +352,14 @@ export class RovoDevChatProvider {

const data = decoder.decode(value, { stream: true });
for (const msg of parser.parse(data)) {
// Check for cancellation while processing messages
if (this.pendingCancellation) {
await reader.cancel();
isDone = true;
this._pendingCancellation = false;
break;
}

if (isFirstMessage) {
telemetryProvider?.perfLogger.promptFirstMessageReceived(this._currentPromptId);
isFirstMessage = false;
Expand Down Expand Up @@ -674,6 +698,10 @@ export class RovoDevChatProvider {
} catch (error) {
// the error is retriable only when it happens during the streaming of a 'chat' response
await this.processError(error, { isRetriable: sourceApi === 'chat' });
} finally {
if (this.pendingCancellation) {
this._pendingCancellation = false;
}
}
} else {
await this.processError(new Error('RovoDev client not initialized'));
Expand Down
Loading