Skip to content

Integrate AG2's Custom Logging for Runtime Data Collection #2

@dwhan89

Description

@dwhan89

Description:
To collect more runtime data and to help us debug, I suggest that we use AG2's custom logging framework to collect runtime data. This implementation will allow us to monitor the CMBAgent's behavior and provide a full runtime record of where the agents got it right and wrong. Eventually, we may even be able to use the logs to fine-tune the agents.


Proposal

  1. Integrate AG2's custom logging system into the project.
  2. Configure the logging framework to capture runtime data, such as:
    • Full decision tree stack.
    • Whether the run was successful or not.
  3. Make sure that logs are structured, consistent, and saved to a designated folder.
  4. Provide configuration options for log verbosity and output destinations (e.g., console, files, or external logging systems).

Goals

  • Improves visibility into the model's runtime behavior.
  • Simplifies the debugging process by capturing detailed logs of errors and events.

Misc Notes

  1. We could implement this ourselves using the Python standard library, but it seems like AG2 team has already been implemented it (https://docs.ag2.ai/docs/reference/runtime_logging). So, it’s probably just a matter of using this feature throughout and also making it configurable at runtime.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions