Add EmbeddingClient to openai.py with detailed logging and retries

[Copilot]

Adds a synchronous requests-based EmbeddingClient class to ai/openai.py with structured logging, configurable timeouts, and retry logic with exponential backoff to improve debugging of embedding failures. The EmbeddingClient replaces all existing embedding implementations across the codebase for consistent behavior.

Changes

Modified files:

• ai/openai.py - Added EmbeddingClient class:

  • EmbeddingError exception class for better error handling
  • EmbeddingClient class with detailed logging and retry logic
  • Per-request logging: request_id, file path, chunk index, elapsed time, HTTP status, response preview
  • Retry with exponential backoff (hard-coded defaults: 2 retries, 1.5 backoff factor)
  • Safe JSON parsing with fallback error handling
  • embed_text() for single chunks, embed_multiple() for batch processing
  • Uses CFG values for api_url, api_key, and model (from .env)
  • Hard-coded sensible defaults: timeout=30s, max_retries=2, backoff=1.5
  • Removed get_embedding_for_text() function (replaced by EmbeddingClient)

• ai/analyzer.py - Integrated EmbeddingClient:

  • Import EmbeddingClient from openai module
  • Updated _get_embedding_with_semaphore() to use EmbeddingClient with file_path and chunk_index parameters
  • Updated call sites to pass file context for rich logging
  • Updated search_semantic() to use EmbeddingClient for query embeddings
  • All embedding operations now benefit from enhanced logging and retry capabilities

• ai/llama_integration.py - Integrated EmbeddingClient:

  • Import EmbeddingClient from openai module instead of get_embedding_for_text
  • Create module-level _embedding_client instance
  • Updated llama_index_retrieve_documents() to use EmbeddingClient for query embeddings

Benefits

The integration provides:

• Consolidated code: All API client code in one module with single embedding approach
• Simple configuration: Uses existing CFG values from .env
• Per-request logging with request_id, file path, chunk index, timing, status codes across all components
• Retry logic with exponential backoff for transient failures
• Sensible defaults: Hard-coded values (30s timeout, 2 retries, 1.5 backoff) that work well for most cases
• Better debugging for embedding API failures in analyzer, search, and llama_integration
• Structured error handling with custom EmbeddingError exception
• Flexible: Can override timeout, retries, and backoff by passing parameters when instantiating if needed

Configuration

Uses standard .env configuration:

API_URL=https://api.example.com/v1
API_KEY=sk-...
EMBEDDING_MODEL=text-embedding-3-small

The EmbeddingClient uses sensible hard-coded defaults (timeout=30s, max_retries=2, backoff=1.5) that work well for most cases. These can be overridden by passing parameters when instantiating the client if needed.

All embedding operations (analyzer, search, llama_integration) benefit from enhanced logging and retry capabilities, using the standard configuration from .env.

Original prompt

Create a pull request that adds a robust EmbeddingClient wrapper and a small usage example to the repository to make embedding failures easier to debug and to provide configurable timeouts/retries.

Files to add (exact contents below):

1. ai/embedding_client.py

• Purpose: A synchronous requests-based EmbeddingClient with detailed per-request logging, timing, retries with exponential backoff, env-configurable parameters, and safe parsing of responses. This will be used by the analyzer to get richer logging (request_id, file, chunk_index, elapsed, status, response preview) for failing embedding requests.

Contents:

# ai/embedding_client.py
import os
import time
import uuid
import json
import logging
import traceback
from typing import List, Optional, Dict, Any

import requests

logger = logging.getLogger("ai.analyzer.embedding")

# Configurable via environment
EMBEDDING_API_URL = os.getenv("PICOCODE_EMBEDDING_URL", "https://example.com/v1/embeddings")
EMBEDDING_API_KEY = os.getenv("PICOCODE_EMBEDDING_API_KEY", "")
DEFAULT_TIMEOUT = float(os.getenv("PICOCODE_EMBEDDING_TIMEOUT", "30"))  # seconds per request
MAX_RETRIES = int(os.getenv("PICOCODE_EMBEDDING_RETRIES", "2"))
BACKOFF_FACTOR = float(os.getenv("PICOCODE_EMBEDDING_BACKOFF", "1.5"))
MODEL_NAME = os.getenv("PICOCODE_EMBEDDING_MODEL", "text-embedding-3-small")

# Optionally enable requests debug logging by setting PICOCODE_HTTP_DEBUG=true
if os.getenv("PICOCODE_HTTP_DEBUG", "").lower() in ("1", "true", "yes"):
    logging.getLogger("requests").setLevel(logging.DEBUG)
    logging.getLogger("urllib3").setLevel(logging.DEBUG)


class EmbeddingError(Exception):
    pass


class EmbeddingClient:
    def __init__(self,
                 api_url: str = EMBEDDING_API_URL,
                 api_key: str = EMBEDDING_API_KEY,
                 model: str = MODEL_NAME,
                 timeout: float = DEFAULT_TIMEOUT,
                 max_retries: int = MAX_RETRIES,
                 backoff: float = BACKOFF_FACTOR):
        self.api_url = api_url
        self.api_key = api_key
        self.model = model
        self.timeout = timeout
        self.max_retries = max_retries
        self.backoff = backoff
        self.session = requests.Session()
        if api_key:
            self.session.headers.update({"Authorization": f"Bearer {api_key}"})
        self.session.headers.update({"Content-Type": "application/json"})

    def _log_request_start(self, request_id: str, file_path: str, chunk_index: int, chunk_len: int):
        logger.debug(
            "Embedding request START",
            extra={
                "request_id": request_id,
                "file": file_path,
                "chunk_index": chunk_index,
                "chunk_length": chunk_len,
                "model": self.model,
                "api_url": self.api_url,
                "timeout": self.timeout,
            },
        )

    def _log_request_end(self, request_id: str, elapsed: float, status: Optional[int], response_body_preview: str):
        logger.debug(
            "Embedding request END",
            extra={
                "request_id": request_id,
                "elapsed_s": elapsed,
                "status": status,
                "response_preview": response_body_preview,
            },
        )

    def embed_text(self, text: str, file_path: str = "<unknown>", chunk_index: int = 0) -> List[float]:
        """
        Embed a single chunk of text. Returns the embedding vector.
        Raises EmbeddingError on failure.
        """
        request_id = str(uuid.uuid4())
        chunk_len = len(text)
        self._log_request_start(request_id, file_path, chunk_index, chunk_len)

        payload = {
            "model": self.model,
            "input": text,
        }

        attempt = 0
        while True:
            attempt += 1
            start = time.perf_counter()
            try:
                resp = self.session.post(
                    self.api_url,
                    data=json.dumps(payload),
                    timeout=self.timeout,
                )
                elapsed = time.perf_counter() - start

                # Try to parse JSON safely
                try:
                    resp_json = resp.json()
                except Exception:
                    resp_json = None

                preview = ""
                if resp_json is not None:
                    preview = json.dumps(resp_json)[:1000]
                else:
                    preview = (resp.text or "")[:1000]

                self._log_request_end(request_id, elapsed, resp.status_code, preview)

                if resp.status_code >= 200 and resp.status_code < 300:
                    # expected format: {"data": [{"embedding": [...]}], ...}
                    if not resp_json:
                        raise EmbeddingError(f"Empty JSON response (status={resp.status_code})")
                    try:
                        # tolerant extraction
                        data = resp_jso...

</details>

*This pull request was created as a result of the following prompt from Copilot chat.*
> Create a pull request that adds a robust EmbeddingClient wrapper and a small usage example to the repository to make embedding failures easier to debug and to provide configurable timeouts/retries.
> 
> Files to add (exact contents below):
> 
> 1) ai/embedding_client.py
> - Purpose: A synchronous requests-based EmbeddingClient with detailed per-request logging, timing, retries with exponential backoff, env-configurable parameters, and safe parsing of responses. This will be used by the analyzer to get richer logging (request_id, file, chunk_index, elapsed, status, response preview) for failing embedding requests.
> 
> Contents:
> ```python
> # ai/embedding_client.py
> import os
> import time
> import uuid
> import json
> import logging
> import traceback
> from typing import List, Optional, Dict, Any
> 
> import requests
> 
> logger = logging.getLogger("ai.analyzer.embedding")
> 
> # Configurable via environment
> EMBEDDING_API_URL = os.getenv("PICOCODE_EMBEDDING_URL", "https://example.com/v1/embeddings")
> EMBEDDING_API_KEY = os.getenv("PICOCODE_EMBEDDING_API_KEY", "")
> DEFAULT_TIMEOUT = float(os.getenv("PICOCODE_EMBEDDING_TIMEOUT", "30"))  # seconds per request
> MAX_RETRIES = int(os.getenv("PICOCODE_EMBEDDING_RETRIES", "2"))
> BACKOFF_FACTOR = float(os.getenv("PICOCODE_EMBEDDING_BACKOFF", "1.5"))
> MODEL_NAME = os.getenv("PICOCODE_EMBEDDING_MODEL", "text-embedding-3-small")
> 
> # Optionally enable requests debug logging by setting PICOCODE_HTTP_DEBUG=true
> if os.getenv("PICOCODE_HTTP_DEBUG", "").lower() in ("1", "true", "yes"):
>     logging.getLogger("requests").setLevel(logging.DEBUG)
>     logging.getLogger("urllib3").setLevel(logging.DEBUG)
> 
> 
> class EmbeddingError(Exception):
>     pass
> 
> 
> class EmbeddingClient:
>     def __init__(self,
>                  api_url: str = EMBEDDING_API_URL,
>                  api_key: str = EMBEDDING_API_KEY,
>                  model: str = MODEL_NAME,
>                  timeout: float = DEFAULT_TIMEOUT,
>                  max_retries: int = MAX_RETRIES,
>                  backoff: float = BACKOFF_FACTOR):
>         self.api_url = api_url
>         self.api_key = api_key
>         self.model = model
>         self.timeout = timeout
>         self.max_retries = max_retries
>         self.backoff = backoff
>         self.session = requests.Session()
>         if api_key:
>             self.session.headers.update({"Authorization": f"Bearer {api_key}"})
>         self.session.headers.update({"Content-Type": "application/json"})
> 
>     def _log_request_start(self, request_id: str, file_path: str, chunk_index: int, chunk_len: int):
>         logger.debug(
>             "Embedding request START",
>             extra={
>                 "request_id": request_id,
>                 "file": file_path,
>                 "chunk_index": chunk_index,
>                 "chunk_length": chunk_len,
>                 "model": self.model,
>                 "api_url": self.api_url,
>                 "timeout": self.timeout,
>             },
>         )
> 
>     def _log_request_end(self, request_id: str, elapsed: float, status: Optional[int], response_body_preview: str):
>         logger.debug(
>             "Embedding request END",
>             extra={
>                 "request_id": request_id,
>                 "elapsed_s": elapsed,
>                 "status": status,
>                 "response_preview": response_body_preview,
>             },
>         )
> 
>     def embed_text(self, text: str, file_path: str = "<unknown>", chunk_index: int = 0) -> List[float]:
>         """
>         Embed a single chunk of text. Returns the embedding vector.
>         Raises EmbeddingError on failure.
>         """
>         request_id = str(uuid.uuid4())
>         chunk_len = len(text)
>         self._log_request_start(request_id, file_path, chunk_index, chunk_len)
> 
>         payload = {
>             "model": self.model,
>             "input": text,
>         }
> 
>         attempt = 0
>         while True:
>             attempt += 1
>             start = time.perf_counter()
>             try:
>                 resp = self.session.post(
>                     self.api_url,
>                     data=json.dumps(payload),
>                     timeout=self.timeout,
>                 )
>                 elapsed = time.perf_counter() - start
> 
>                 # Try to parse JSON safely
>                 try:
>                     resp_json = resp.json()
>                 except Exception:
>                     resp_json = None
> 
>                 preview = ""
>                 if resp_json is not None:
>                     preview = json.dumps(resp_json)[:1000]
>                 else:
>                     preview = (resp.text or "")[:1000]
> 
>                 self._log_request_end(request_id, elapsed, resp.status_code, preview)
> 
>                 if resp.status_code >= 200 and resp.status_code < 300:
>                     # expected format: {"data": [{"embedding": [...]}], ...}
>                     if not resp_json:
>                         raise EmbeddingError(f"Empty JSON response (status={resp.status_code})")
>                     try:
>                         # tolerant extraction
>                         data = resp_json.get("data") if isinstance(resp_json, dict) else None
>                         if data and isinstance(data, list) and len(data) > 0:
>                             emb = data[0].get("embedding")
>                             if emb and isinstance(emb, list):
>                                 logger.info(
>                                     "Embedding succeeded",
>                                     extra={"request_id": request_id, "file": file_path, "chunk_index": chunk_index},
>                                 )
>                                 return emb
>                         # Fallback: maybe top-level "embedding" key
>                         if isinstance(resp_json, dict) and "embedding" in resp_json:
>                             emb = resp_json["embedding"]
>                             if isinstance(emb, list):
>                                 return emb
>                         raise EmbeddingError(f"Unexpected embedding response shape: {resp_json}")
>                     except KeyError as e:
>                         raise EmbeddingError(f"Missing keys in embedding response: {e}")
>                 else:
>                     # Non-2xx
>                     logger.warning(
>                         "Embedding API returned non-2xx",
>                         extra={
>                             "request_id": request_id,
>                             "status_code": resp.status_code,
>                             "file": file_path,
>                             "chunk_index": chunk_index,
>                             "attempt": attempt,
>                             "body_preview": preview,
>                         },
>                     )
>                     # fall through to retry logic
>                     err_msg = f"Status {resp.status_code}: {preview}"
> 
>             except requests.Timeout as e:
>                 elapsed = time.perf_counter() - start
>                 err_msg = f"Timeout after {elapsed:.2f}s: {e}"
>                 logger.error("Embedding API Timeout", extra={"request_id": request_id, "error": str(e)})
>             except requests.RequestException as e:
>                 elapsed = time.perf_counter() - start
>                 err_msg = f"RequestException after {elapsed:.2f}s: {e}\n{traceback.format_exc()}"
>                 logger.error("Embedding request exception", extra={"request_id": request_id, "error": err_msg})
>             except Exception as e:
>                 elapsed = time.perf_counter() - start
>                 err_msg = f"Unexpected error after {elapsed:.2f}s: {e}\n{traceback.format_exc()}"
>                 logger.exception("Unexpected embedding exception", extra={"request_id": request_id})
> 
>             # Retry logic
>             if attempt > self.max_retries:
>                 logger.error(
>                     "Max retries exceeded for embedding request",
>                     extra={"request_id": request_id, "file": file_path, "chunk_index": chunk_index, "attempts": attempt},
>                 )
>                 raise EmbeddingError(f"Failed to get embedding after {attempt} attempts. Last error: {err_msg}")
> 
>             # Backoff and retry
>             sleep_for = self.backoff * (2 ** (attempt - 1))
>             logger.info(
>                 "Retrying embedding request",
>                 extra={
>                     "request_id": request_id,
>                     "file": file_path,
>                     "chunk_index": chunk_index,
>                     "attempt": attempt,
>                     "sleep_s": sleep_for,
>                 },
>             )
>             time.sleep(sleep_for)
> 
>     def embed_multiple(self, chunks: List[str], file_path: str = "<unknown>") -> List[Dict[str, Any]]:
>         """
>         Embed a list of text chunks. Returns list of dicts: {"chunk_index": i, "embedding": [...]}.
>         This method logs progress and errors for each chunk.
>         """
>         results = []
>         for i, chunk in enumerate(chunks):
>             try:
>                 emb = self.embed_text(chunk, file_path=file_path, chunk_index=i)
>                 results.append({"chunk_index": i, "embedding": emb})
>             except EmbeddingError as e:
>                 logger.error(
>                     "Failed to embed chunk",
>                     extra={"file": file_path, "chunk_index": i, "error": str(e)},
>                 )
>                 # append a failure marker or skip depending on desired behavior
>                 results.append({"chunk_index": i, "embedding": None, "error": str(e)})
>         return results
> ```
> 
> 2) ai/analyzer_embedding_usage_example.py
> - Purpose: A short example showing how to instantiate EmbeddingClient and call embed_multiple for a file's chunks. This is intentionally non-invasive so maintainers can integrate it into the real analyzer code path where embedding is currently called.
> 
> Contents:
> ```python
> # ai/analyzer_embedding_usage_example.py
> import logging
> from ai.embedding_client import EmbeddingClient
> 
> logger = logging.getLogger("ai.analyzer")
> 
> # create client (will pick up env vars)
> client = EmbeddingClient()
> 
> def process_file_and_embed(file_path: str, chunks: list[str]):
>     logger.info("Start embedding file", extra={"file": file_path, "num_chunks": len(chunks)})
>     results = client.embed_multiple(chunks, file_path=file_path)
>     # Inspect results for None embeddings and act accordingly
>     for r in results:
>         if r.get("embedding") is None:
>             logger.warning("Chunk embedding failed", extra={"file": file_path, "chunk_index": r["chunk_index"], "error": r.get("error")})
>         else:
>             # continue with storing the embedding
>             pass
>     return results
> ```
> 
> Notes for reviewers / maintainers (add to PR description):
> - These files add a synchronous wrapper using requests. If the analyzer uses an async client (httpx or aiohttp), the same logging and retry approach should be implemented in async form.
> - Environment variables to control behavior: PICOCODE_EMBEDDING_URL, PICOCODE_EMBEDDING_API_KEY, PICOCODE_EMBEDDING_TIMEOUT, PICOCODE_EMBEDDING_RETRIES, PICOCODE_EMBEDDING_BACKOFF, PICOCODE_EMBEDDING_MODEL, PICOCODE_HTTP_DEBUG
> - The PR intentionally adds a non-invasive example module (analyzer_embedding_usage_example.py) so integrators can copy/paste into the actual analyzer call site.
> 
> Please create a feature branch, add the two files, and open a pull request against the default branch with the title: "Add EmbeddingClient wrapper with detailed logging and retries" and a succinct description summarizing the same notes above.
> 

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).

[Mte90]

@copilot il file di esempio non serve

[Copilot]

@copilot il file di esempio non serve

Fatto. Ho rimosso il file di esempio e integrato EmbeddingClient direttamente nell'analyzer.py. Ora tutte le operazioni di embedding usano il nuovo client con logging dettagliato e retry logic. Commit: 27b5b87

[Mte90]

@copilot così hai lasciato la funzione get_embedding_for_text dentro openai inutilizzata, non voglio un file per embedding voglio quel codice dentro il file openai.py

[Mte90]

@copilot line useless

[Copilot]

Rimosso. Commit: c9310f1

[Copilot]

@copilot così hai lasciato la funzione get_embedding_for_text dentro openai inutilizzata, non voglio un file per embedding voglio quel codice dentro il file openai.py

Fatto. Ho spostato tutto il codice di EmbeddingClient dentro openai.py e rimosso il file embedding_client.py. Ora tutte le funzionalità di embedding sono consolidate in un unico modulo. Commit: e73edcd

[Mte90]

@copilot this lines doesn't make any sense this settings already exits check the config files.

[Copilot]

Corretto. EmbeddingClient ora usa direttamente i valori da CFG (API_URL, API_KEY, EMBEDDING_MODEL) invece di duplicarli. Solo timeout, retries e backoff rimangono come override via environment. Commit: c9310f1

[Mte90]

@copilot I don't want this variables

[Copilot]

Rimosso. I parametri timeout, max_retries e backoff ora hanno valori predefiniti hard-coded (30.0, 2, 1.5) direttamente nel metodo init. Commit: f28a7ef

[Mte90]

@copilot get_embedding_for_text what is the purpose now of this function with this new class? there is stuff unused after that change?

[Copilot]

@copilot get_embedding_for_text what is the purpose now of this function with this new class? there is stuff unused after that change?

La funzione get_embedding_for_text è ancora utilizzata in ai/llama_integration.py (linea 31). Attualmente abbiamo due approcci:

• get_embedding_for_text: usa OpenAI SDK con rate limiting e circuit breaker
• EmbeddingClient: usa requests direttamente con logging dettagliato e retry

Vuoi che converta anche llama_integration.py per usare EmbeddingClient e rimuova get_embedding_for_text?

[Mte90]

@copilot yes I want that you use that class for everything and remove the useless code

[Copilot]

@copilot yes I want that you use that class for everything and remove the useless code

Fatto. Ho convertito llama_integration.py per usare EmbeddingClient e rimosso get_embedding_for_text da openai.py. Ora tutte le operazioni di embedding (analyzer, search, llama_integration) usano EmbeddingClient con logging consistente e retry logic. Commit: d9ea51d

[Mte90]

@copilot check the code for the function you removed if they are unused that code can be removed,
you created
class EmbeddingError(Exception):
"""Custom exception for embedding failures"""
pass

but has just a pass and doing nothing

[Copilot]

@copilot check the code for the function you removed if they are unused that code can be removed,
you created
class EmbeddingError(Exception):...

Ho verificato il codice:

1. Funzioni rimosse da get_embedding_for_text: _retry_with_backoff, _check_rate_limit, _check_circuit_breaker, _record_success, _record_failure, e _client sono ancora utilizzati da call_coding_api (linee 313-360), quindi non possono essere rimossi.

2. EmbeddingError con solo pass: Questo è il pattern Python standard per un'eccezione personalizzata. L'eccezione è utilizzata attivamente:

   • Sollevata in 4 punti diversi in embed_text() (linee 208, 225, 227, 263)
   • Catturata in embed_multiple() (linea 289)

   Il pass è corretto - l'eccezione non ha bisogno di logica aggiuntiva, serve solo come tipo distinto per identificare errori di embedding.

[Mte90]

@copilot I don't have want anymore get_embedding_for_text if there is a class that replaced. this is code duplication and you should be smart and check to organize better the code or I wlitllr ash this pr and start again

[Copilot]

@copilot I don't have want anymore get_embedding_for_text if there is a class that replaced. this is code duplication and you should be smart and check to organize better the code or I wlitllr ash thi...

Mi scuso per la confusione nella mia risposta precedente. Ho verificato nuovamente:

get_embedding_for_text è già stata completamente rimossa nel commit d9ea51d. Non esiste più nel codice - ho confermato con grep che non ci sono riferimenti.

Nella mia risposta precedente ho controllato erroneamente le funzioni helper (_retry_with_backoff, _check_rate_limit, etc) che get_embedding_for_text usava, ma quelle sono ancora necessarie per call_coding_api.

Situazione attuale:

• ✅ get_embedding_for_text: completamente rimossa
• ✅ EmbeddingClient: usata ovunque (analyzer, search, llama_integration)
• ✅ Nessuna duplicazione di codice

Il codice è pulito e organizzato come richiesto.