feat(integration-tests): add OpenAI integration tests and configuration by zomux · Pull Request #252 · openagents-org/openagents

zomux · 2026-01-12T18:08:15Z

Introduced integration tests for the Hello World demo, verifying the Charlie agent's response to both channel and direct messages.
Added configuration and fixtures for dynamic network setup and agent lifecycle management in conftest.py.
Updated pytest.yml to include necessary API keys for testing.
Created initial documentation for integration tests in __init__.py.

- Introduced integration tests for the Hello World demo, verifying the Charlie agent's response to both channel and direct messages. - Added configuration and fixtures for dynamic network setup and agent lifecycle management in `conftest.py`. - Updated `pytest.yml` to include necessary API keys for testing. - Created initial documentation for integration tests in `__init__.py`.

vercel · 2026-01-12T18:08:19Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Review	Updated (UTC)
openagents-studio	Ready	Preview, Comment	Jan 12, 2026 6:09pm

Copilot

Pull request overview

This pull request introduces comprehensive integration tests for the Hello World demo, verifying end-to-end functionality of the Charlie agent's message handling capabilities with real LLM interactions. The tests validate channel messaging, direct messaging, conversation flows, and self-message loop prevention.

Changes:

Added integration test suite for Hello World demo with 5 test cases covering channel messages, direct messages, conversation flow, self-response prevention, and network connectivity
Implemented shared test infrastructure with fixtures for network setup, agent lifecycle management, and client connections using dynamic port allocation
Updated CI/CD pipeline to include OpenAI API key configuration for integration tests

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 28 comments.

File	Description
tests/integration/test_hello_world_demo.py	Comprehensive test suite validating Charlie agent's responses to various message types with LLM integration
tests/integration/conftest.py	Shared fixtures providing network setup, dynamic port allocation, and agent/client lifecycle management
tests/integration/init.py	Package documentation describing integration test scope
.github/workflows/pytest.yml	Added OpenAI API key environment variables for integration test execution

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-12T18:14:43Z

tests/integration/test_hello_world_demo.py

+    # Wait for second response
+    try:
+        await asyncio.wait_for(response_event.wait(), timeout=60)
+    except asyncio.TimeoutError:
+        pass


The response_event is not cleared before waiting for the second response, which could cause the test to immediately pass if Charlie sent multiple messages in response to the first question. Consider adding response_event.clear() after line 409 or restructuring to use a different synchronization mechanism for each expected response.

Copilot · 2026-01-12T18:14:44Z

tests/integration/test_hello_world_demo.py

+    first_response_payload = charlie_responses_1[0].payload
+    first_response = first_response_payload.get("text") or first_response_payload.get("content", {}).get("text", "")


If charlie_responses_1[0].payload is None, accessing .get() will raise an AttributeError. While the payload is checked for existence in earlier tests, this specific test doesn't validate that the payload exists before attempting to access it. Consider adding a null check: first_response_payload = charlie_responses_1[0].payload or {} to handle cases where payload might be None.

Copilot · 2026-01-12T18:14:44Z

tests/integration/test_hello_world_demo.py

+    second_response_payload = charlie_responses_2[-1].payload
+    second_response = second_response_payload.get("text") or second_response_payload.get("content", {}).get("text", "")


If charlie_responses_2[-1].payload is None, accessing .get() will raise an AttributeError. Consider adding a null check: second_response_payload = charlie_responses_2[-1].payload or {} to handle cases where payload might be None.

Copilot · 2026-01-12T18:14:44Z

tests/integration/conftest.py

+                except:
+                    pass
+        except:


Bare except clauses should specify exception types. Using bare except: can catch system-critical exceptions like KeyboardInterrupt and SystemExit, which should typically be allowed to propagate. Consider catching specific exceptions like Exception or more targeted exceptions like aiohttp.ClientError and asyncio.TimeoutError.

Suggested change

except:

pass

except:

except (aiohttp.ClientError, asyncio.TimeoutError):

pass

except Exception:

Copilot · 2026-01-12T18:14:45Z

tests/integration/conftest.py

+    max_retries = 10
+    for attempt in range(max_retries):
+        try:
+            import aiohttp


The aiohttp module is imported inside the retry loop on each attempt. This import should be moved to the top of the file to follow Python conventions and avoid repeated import overhead during retries.

Copilot · 2026-01-12T18:14:50Z

tests/integration/test_hello_world_demo.py

+
+import pytest
+import asyncio
+import os


Import of 'os' is not used.

Suggested change

import os

Copilot · 2026-01-12T18:14:50Z

tests/integration/test_hello_world_demo.py

+import pytest
+import asyncio
+import os
+from pathlib import Path


Import of 'Path' is not used.

Suggested change

from pathlib import Path

Copilot · 2026-01-12T18:14:51Z

tests/integration/test_hello_world_demo.py

+from openagents.agents.runner import AgentRunner
+from openagents.models.event import Event
+
+from tests.integration.conftest import check_llm_api_key, skip_without_api_key


Import of 'check_llm_api_key' is not used.

Suggested change

from tests.integration.conftest import check_llm_api_key, skip_without_api_key

from tests.integration.conftest import skip_without_api_key

Copilot · 2026-01-12T18:14:51Z

tests/integration/test_hello_world_demo.py

+    # Wait for second response
+    try:
+        await asyncio.wait_for(response_event.wait(), timeout=60)
+    except asyncio.TimeoutError:


'except' clause does nothing but pass and there is no explanatory comment.

Copilot · 2026-01-12T18:14:51Z

tests/integration/conftest.py

+    env_file = Path(__file__).parent.parent.parent / ".env"
+    if env_file.exists():
+        load_dotenv(env_file)
+        print(f"Loaded environment from {env_file}")


Print statement may execute during import.

Copilot AI review requested due to automatic review settings January 12, 2026 18:08

Copilot started reviewing on behalf of zomux January 12, 2026 18:08 View session

vercel bot deployed to Preview January 12, 2026 18:09 View deployment

Copilot AI reviewed Jan 12, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(integration-tests): add OpenAI integration tests and configuration#252

feat(integration-tests): add OpenAI integration tests and configuration#252
zomux wants to merge 1 commit intodevelopfrom
feature/integration_test

zomux commented Jan 12, 2026

Uh oh!

vercel bot commented Jan 12, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 12, 2026

Uh oh!

Copilot AI Jan 12, 2026

Uh oh!

Copilot AI Jan 12, 2026

Uh oh!

Copilot AI Jan 12, 2026

Uh oh!

Copilot AI Jan 12, 2026

Uh oh!

Copilot AI Jan 12, 2026

Uh oh!

Copilot AI Jan 12, 2026

Uh oh!

Copilot AI Jan 12, 2026

Uh oh!

Copilot AI Jan 12, 2026

Uh oh!

Copilot AI Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		first_response_payload = charlie_responses_1[0].payload
		first_response = first_response_payload.get("text") or first_response_payload.get("content", {}).get("text", "")

		second_response_payload = charlie_responses_2[-1].payload
		second_response = second_response_payload.get("text") or second_response_payload.get("content", {}).get("text", "")

-                except:
-                    pass
-        except:
+                except (aiohttp.ClientError, asyncio.TimeoutError):
+                    pass
+        except Exception:

	from tests.integration.conftest import check_llm_api_key, skip_without_api_key
	from tests.integration.conftest import skip_without_api_key

Conversation

zomux commented Jan 12, 2026

Uh oh!

vercel bot commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vercel bot commented Jan 12, 2026 •

edited

Loading