Skip to content

Conversation

@slister1001
Copy link
Member

Description

Redacts user-provided content (queries, responses, tool definitions, exception messages) from log strings in azure-ai-evaluation that flow into Geneva telemetry (ViennaExp/Log). This prevents database connection strings and other credentials embedded in user payloads from being flagged by CredScan.

Resolves ICM 738457593

Changes

  • utils.py - Remove f-string interpolation of query, response, tool_definitions, and exception from log messages
  • Downgrade noisy agent-response fallback warnings to logger.debug
  • Sanitize upload error to emit only type(e).__name__ instead of full exception text
  • Chain original exception with from e for proper traceback

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes. (N/A - no user-facing behavior change)
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message.

Testing Guidelines

  • Pull request includes test coverage for the included changes. (Log message changes only - no behavioral change to test)

Copilot AI review requested due to automatic review settings February 12, 2026 21:10
@slister1001 slister1001 requested a review from a team as a code owner February 12, 2026 21:10
@github-actions github-actions bot added the Evaluation Issues related to the client library for Azure AI Evaluation label Feb 12, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses a critical security issue by redacting sensitive user-provided data from log messages in the azure-ai-evaluation SDK that flow into Geneva telemetry. The changes prevent database connection strings and other credentials embedded in user payloads from being exposed in logs and flagged by CredScan tools.

Changes:

  • Removed user-provided content (queries, responses, tool definitions) from log message strings
  • Downgraded noisy agent-response fallback warnings from logger.warning to logger.debug
  • Sanitized upload error messages to emit only exception type names instead of full exception text
  • Added proper exception chaining with from e for maintaining traceback while keeping logs clean

return response
return "\n".join(agent_response)
except:
# If the agent response cannot be parsed for whatever reason (e.g. the converter format changed), the original response is returned
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bare except: clause should specify Exception to avoid catching system-exiting exceptions like KeyboardInterrupt and SystemExit. This is inconsistent with the exception handling patterns used elsewhere in the file (see lines 700 and 786 which use except Exception as e:). Consider changing this to except Exception: for consistency and better practice.

Copilot uses AI. Check for mistakes.
@slister1001 slister1001 force-pushed the fix/credscan-redact-logs branch from b6281bd to 2092f7c Compare February 12, 2026 21:30
Remove user-provided content (queries, responses, tool definitions, exception
messages) from log strings that flow into Geneva telemetry. This prevents
database connection strings and other credentials embedded in user payloads
from being flagged by CredScan.

Changes:
- Remove f-string interpolation of query/response/tool_definitions in
  warning and debug log messages
- Downgrade noisy agent-response warnings to debug level
- Sanitize upload error messages to emit only exception type name
- Chain original exception with 'from e'

Resolves ICM 738457593
@slister1001 slister1001 force-pushed the fix/credscan-redact-logs branch from 2092f7c to b5b0674 Compare February 12, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Evaluation Issues related to the client library for Azure AI Evaluation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant