Skip to content

Feature: whatsapp bridge integration#20

Open
jeancsil wants to merge 16 commits intomainfrom
feature/whatsapp-bridge-integration
Open

Feature: whatsapp bridge integration#20
jeancsil wants to merge 16 commits intomainfrom
feature/whatsapp-bridge-integration

Conversation

@jeancsil
Copy link
Owner

@jeancsil jeancsil commented Mar 1, 2026

Integrate Whatsapp with agent-framework. Limiting messages being seen and sent only from me to myself :)

jeancsil and others added 14 commits March 1, 2026 12:25
- Add MCP resilience: load servers individually with graceful degradation
- Add SQLite-based message deduplication using SHA-256 hash
- Add --mcp-servers CLI option to customize/disable MCP servers
- Add processed_messages.db for persistent duplicate prevention
- Fix shutdown warning (debug instead of warning)
- Add whatsapp.yaml.example with MCP servers configuration
- Update README.md with Python 3.13 and test matrix
- Update CI workflow for Python 3.13

This prevents duplicate LLM API calls for same message content
and allows agent to continue working if some MCP servers fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove binary files from git tracking and update .gitignore
- Remove unused signal and Optional imports
- Add pydantic configuration validation for type-safe config parsing
- Implement database cleanup to remove old deduplication records
- Improve media content-type detection using HTTP response headers
- Add comprehensive tests for privacy, deduplication, and error handling
- Update README with WhatsApp setup instructions and privacy notes
- Set group_messages to false by default in config example
  This commit addresses issues found during code review:

  - Fix type hint in MockChannel (use Callable instead of lowercase callable)
  - Improve SQLite connection lifecycle with proper error handling on close
  - Add debug logging for signal handler setup failures
  - Implement typing indicators feature for WhatsAppChannel
  - Use context manager for safe working directory changes
  - Enhance _on_message_event docstring with sync callback details

  The typing indicators feature shows users when the agent is processing
  their message, providing better user experience.
Root Cause
After upgrading langchain-openai, the ChatOpenAI (and AzureChatOpenAI) constructors no longer accept a plain str for the api_key parameter — they now require SecretStr | Callable | None. The azure_openai branch had already been patched to wrap its key in SecretStr(...), but the OpenAI fallback at the bottom of

_create_model()
 was still passing a raw str | None directly, causing the mypy error:
Added an autouse=True fixture to the test class that monkeypatches _create_model
to a no-op — exactly the same pattern used in test_core_agents.py and test_developer_agent.py:
 4 mypy errors are now fixed. The issue was that cli.py was calling non-existent getter
  methods on WhatsAppAgentConfig:
- Add better error logging for WhatsApp session issues (401 errors)
- Add --reset-session CLI option to clear WhatsApp session
- Allow self-messages (is_from_me=True) to enable legitimate self-messaging
- Ensure self-message responses go back to original sender JID, not to allowed_contact
- Add comprehensive tests covering real-world scenarios:
  - Self-message from LID when allowed_contact is phone number
  - Response routing verification for self-messages
  - External message filtering
  - Group chat filtering
  - LID format contact support

This fixes the issue where users couldn't self-message due to LID vs phone
number mismatch, while maintaining security by ensuring responses echo back to
the original sender only.
- Fix contact filtering to check SenderAlt and RecipientAlt fields
- Only allow self-messages when phone number matches allowed contact
- Add SQLite schema migration for last_seen_at column
- Update tests to reflect new security behavior
- Add debug logging for alternative JID fields

This prevents processing messages from unknown LIDs while allowing
legitimate self-messages where the phone number is in alt fields.
@jeancsil
Copy link
Owner Author

jeancsil commented Mar 1, 2026

Code Review Summary

Overall assessment: Request Changes

This is a substantial feature addition for WhatsApp integration with a well-structured architecture, but it has several critical security, reliability, and quality issues that need to be addressed before merging.

Issues Found

🔴 Critical:

  • Security vulnerability in contact filtering: The _on_message_event method has a critical flaw in handling self-messages from LIDs. It incorrectly processes self-messages (is_from_me=True) from unknown LIDs, which could allow messages from other contacts (e.g., LinkedIn sync) to be processed. This violates the security model that only messages from allowed_contact should be processed.
  • Hardcoded test API key: CI configuration contains OPENAI_API_KEY: sk-test which appears to be a test key that should not be committed.
  • Missing dependency management: The pydantic dependency is added but not properly integrated into the existing codebase - imports are inconsistent.

🟡 Minor:

  • Incomplete error handling: Several async operations lack proper error handling, especially in whatsapp.py message handling.
  • Configuration validation issues: The configuration validation doesn't properly validate phone number format in some cases.
  • Missing documentation: Some new modules lack docstrings and proper documentation.
  • Test coverage gaps: While there are comprehensive tests, some edge cases in the message filtering logic need additional test coverage.

Positive Highlights

  • Well-structured architecture: The channel abstraction is clean and extensible.
  • Comprehensive test suite: Extensive test coverage for the WhatsApp channel, including security-focused tests.
  • Good separation of concerns: Configuration, channel logic, and agent logic are well-separated.
  • Nice typing indicators feature: The typing indicator implementation is sophisticated with duration control.
  • Security-conscious design: Strong privacy filtering and message deduplication features.

Next Steps

  1. Fix security vulnerability: Update the contact filtering logic in _on_message_event to properly handle self-messages from unknown LIDs. The fix should ensure that only messages from the allowed_contact phone number (regardless of LID format) are processed.

  2. Remove hardcoded credentials: Remove the test API key from the CI configuration.

  3. Fix dependency imports: Ensure consistent use of pydantic across the codebase and remove unused imports.

  4. Improve error handling: Add proper error handling for all async operations, especially in the WhatsApp channel's message handling.

  5. Add configuration validation: Improve phone number format validation in the configuration.

  6. Documentation: Add proper docstrings and usage documentation for new modules.

The implementation shows good engineering practices but needs these critical fixes before it can be safely merged and used.

mime_type = mime_types.get(media_type, "image/jpeg")

# Build and send media message based on type - run in thread pool
def _build_and_send() -> None:
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor Issue: Missing error handling for database operations

The _send_typing and _stop_typing methods call self._client.send_chat_presence() but don't handle potential exceptions from the Go backend. This could cause the agent to crash if there are network issues.

Fix: Add try-catch around the send_chat_presence calls and log errors gracefully without crashing the agent.

try:
    self._client.send_chat_presence(...)
except Exception as e:
    logger.warning(f"Failed to send typing indicator: {e}")
    # Don't crash the agent, just log and continue

def _run_client() -> None:
try:
assert self._client is not None

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor Issue: Database connection thread safety concern

The code uses thread-local SQLite connections, but there's a potential race condition when multiple threads are accessing the database concurrently.

Fix: The current approach is generally good for avoiding SQLite threading issues, but consider adding additional synchronization or connection pooling if the agent becomes multi-threaded.

For now, the current implementation should work for the single-threaded event loop model, but this is worth noting for future scaling.

successful_servers.append(server_name)
logger.info(f"✓ Connected to {server_name}: {len(tools)} tools loaded")
except MCPConnectionError as e:
failed_servers[server_name] = str(e)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor Issue: Error message could expose sensitive information

The error handling in _handle_message includes the raw error message in the response to users. This could potentially expose internal implementation details, API keys, or other sensitive information.

Fix: Use a generic error message instead of str(e) to avoid exposing sensitive information.

# Instead of: f"Sorry, something went wrong processing your message: {e}"
# Use:
error_message = OutgoingMessage(
    text="Sorry, something went wrong processing your message. Please try again.",
    recipient_id=incoming.sender_id,
)

bridge_timeout_sec: int = Field(default=180, ge=1, le=600, description="Max wait for bridge startup (seconds).")
poll_interval_sec: float = Field(
default=1.0, ge=0.1, le=60.0, description="Check for new messages interval (seconds)."
)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Practice: Excellent configuration validation with pydantic

The configuration validation using pydantic models is well-implemented and provides strong type safety. The custom validators for MCP servers and storage path expansion are particularly good.

The validation ensures that all required configuration fields are present and properly typed before runtime, which prevents many potential issues.

# SECURITY: Reject group chats entirely to prevent privacy leaks
# Group chats have JIDs ending with @g.us
if chat_jid.endswith("@g.us"):
if self.log_filtered_messages:
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Practice: Excellent security filtering

The group chat filtering logic (if chat_jid.endswith("@g.us")) is perfect - it absolutely prevents any group chat messages from being processed, which is critical for privacy.

This is exactly the right approach to prevent the agent from accidentally processing messages from group chats that could contain sensitive information or spam.

"""
try:
# Test by creating a connection in the current thread
_ = self._get_db_connection()
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Practice: Excellent deduplication implementation

The message deduplication using SQLite with time-based windows is very well-implemented. It:

  • Prevents processing of duplicate messages within the time window
  • Allows legitimate repeated messages after the window expires
  • Cleans up old records to prevent database bloat
  • Uses proper thread-safe operations

This is much better than simple hash-based deduplication as it handles the real-world case where users might legitimately send the same message multiple times.

wait_time = self._min_typing_duration - elapsed
logger.info(
f"Waiting {wait_time:.1f}s before stopping typing indicator for {jid} (elapsed: {elapsed:.1f}s)"
)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor Issue: Database initialization could fail silently

The _init_deduplication_db method calls _cleanup_old_deduplication_records() but if the database connection fails, it logs an error but continues running. This means the agent will run without deduplication.

Fix: Consider having a fallback mode or more graceful degradation when the database is unavailable, or at least make the error more visible to users.

if sqlite3.Error as e:
    logger.error(f"CRITICAL: Failed to initialize deduplication DB: {e}")
    logger.error("Agent will run without message deduplication - this may cause message processing issues")
    # Optionally raise an exception to prevent silent failure

This stops listening for messages and closes connections.
"""
logger.info("Shutting down WhatsApp channel...")

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor Issue: Media message error handling could be improved

The media message sending code in _send_media has good error handling for the HTTP request, but if the media building fails (e.g., invalid media type), the error might not be handled gracefully.

Fix: Add additional error handling for the media building and sending process to ensure users get appropriate feedback if media fails to send.

try:
    # Build and send media message
    await _build_and_send()
except Exception as media_error:
    logger.error(f"Failed to send media: {media_error}")
    # Send text fallback to user
    fallback_message = OutgoingMessage(
        text="Sorry, I couldn't send that media file. Please try sending it as text.",
        recipient_id=message.recipient_id,
    )
    await self.send(fallback_message)

@jeancsil jeancsil self-assigned this Mar 1, 2026
@jeancsil jeancsil added the enhancement New feature or request label Mar 1, 2026
jeancsil added 2 commits March 1, 2026 18:53
Add test to validate the real-world scenario where user sends
a message to themselves in WhatsApp:
- Message arrives from LID (not phone number)
- Phone number is in RecipientAlt field
- Message should be allowed and processed
- Response must go back to LID (not phone number)

This ensures users see agent responses in their own chat thread
when messaging themselves.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant