From 2675d632c4b3f55afbdeeda4d7413544bf3fa972 Mon Sep 17 00:00:00 2001 From: Kranthi Kumar Manchikanti Date: Mon, 1 Sep 2025 09:59:39 -0400 Subject: [PATCH 1/2] Added MCP server with DevTunnel support and Copilot Studio integration - Add DevTunnel configuration templates for public exposure - Add OpenAPI specification template for Copilot Studio integration --- .gitignore | 11 + MCP-COPILOTSTUDIO.md | 158 ++++++++++++ .../{.env.sample => .env.template} | 10 +- .../SECURITY-Git-Commit-Guide.md | 227 ++++++++++++++++++ .../crm-mcp-connector.template.yaml | 18 ++ 5 files changed, 419 insertions(+), 5 deletions(-) create mode 100644 MCP-COPILOTSTUDIO.md rename agentic_ai/applications/{.env.sample => .env.template} (90%) create mode 100644 agentic_ai/backend_services/SECURITY-Git-Commit-Guide.md create mode 100644 agentic_ai/backend_services/crm-mcp-connector.template.yaml diff --git a/.gitignore b/.gitignore index 5c27a8e3a..170dbd82e 100644 --- a/.gitignore +++ b/.gitignore @@ -119,6 +119,17 @@ env.bak/ venv.bak/ **/secrets.env +# DevTunnel and sensitive configuration files +**/crm-mcp-connector.yaml +**/*devtunnel*.yaml +**/*.local.yaml +**/*.personal.yaml + +# Additional sensitive files +**/*.secret +**/*.private +**/local-config.* + # Spyder project settings .spyderproject .spyproject diff --git a/MCP-COPILOTSTUDIO.md b/MCP-COPILOTSTUDIO.md new file mode 100644 index 000000000..5ca20fac1 --- /dev/null +++ b/MCP-COPILOTSTUDIO.md @@ -0,0 +1,158 @@ +# Exposing MCP Server via DevTunnel and Connecting to Copilot Studio + +This guide walks you through exposing your local MCP (Model Context Protocol) server using Microsoft DevTunnel and connecting it to Microsoft Copilot Studio using a custom MCP connector. + +## πŸ“‹ Prerequisites + +Before starting, ensure you have: +- βœ… Python 3.11+ installed +- βœ… DevTunnel CLI installed (`devtunnel`) +- βœ… Access to Microsoft Copilot Studio +- βœ… Azure subscription (for Copilot Studio) +- βœ… The OpenAI Workshop repository cloned locally + +## πŸ—οΈ Architecture Overview + +```mermaid +graph TB + A[Local MCP Server
localhost:8000/mcp] --> B[DevTunnel
Port Forwarding] + B --> C[Public URL
https://xxx.use.devtunnels.ms/mcp] + C --> D[Copilot Studio
Custom MCP Connector] + D --> E[AI Agent/Copilot
with MCP Tools] + + F[OpenAPI Spec
crm-mcp-connector.yaml] --> D +``` + +## πŸš€ Step 1: Start Your Local MCP Server + +### 1.1 Set Up Python Environment + +```bash +# Navigate to the backend services directory +cd OpenAIWorkshop/agentic_ai/backend_services + +# Create and activate virtual environment (if not already done) +python -m venv ../../../venv +source ../../../venv/Scripts/activate # Windows +# source ../../../venv/bin/activate # Linux/Mac + +# Install required dependencies +pip install fastmcp httpx rich pygments pydantic uvicorn cryptography authlib python-jose +``` + +### 1.2 Start the MCP Service + +```bash +# From the backend_services directory +python mcp_service.py +``` + +You should see output similar to: +``` +[09/01/25 09:00:01] INFO Starting MCP server 'Contoso Customer API as Tools' with transport 'streamable-http' on http://0.0.0.0:8000/mcp +INFO: Started server process [12345] +INFO: Waiting for application startup. +INFO: Application startup complete. +INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) +``` + +### 1.3 Verify Local MCP Server + +Test your local server: +```bash +# Test the MCP endpoint +curl -X GET http://localhost:8000/mcp +``` + +## 🌐 Step 2: Expose MCP Server via DevTunnel + +### 2.1 Create DevTunnel + +```bash +# Create a new tunnel for your MCP server +devtunnel create crm-mcp --allow-anonymous + +# Add port forwarding for port 8000 +devtunnel port create crm-mcp -p 8000 +``` + +### 2.2 Start DevTunnel Host + +```bash +# Start the tunnel host to expose your local server +devtunnel host crm-mcp +``` + +You'll see output like: +``` +Hosting port: 8000 +Connect via browser: https://p8er9sxt.use.devtunnels.ms:8000, https://p8er9sxt-8000.use.devtunnels.ms +Inspect network activity: https://p8er9sxt-8000-inspect.use.devtunnels.ms + +Ready to accept connections for tunnel: crm-mcp.use +``` + +### 2.3 Test Public Access + +```bash +# Test your public MCP endpoint +curl -X GET https://p8er9sxt-8000.use.devtunnels.ms/mcp +``` + +**πŸ“ Note**: Replace `p8er9sxt-8000.use.devtunnels.ms` with your actual tunnel URL. + +## πŸ“„ Step 3: Prepare OpenAPI Specification + +The `crm-mcp-connector.yaml` file contains the OpenAPI specification for your MCP server: + +### 3.1 Update the Host URL + +**⚠️ Important**: Update the `host` field in `crm-mcp-connector.yaml` with your actual DevTunnel URL: + +```yaml +host: YOUR_ACTUAL_TUNNEL_URL.use.devtunnels.ms +``` + +## πŸ€– Step 4: Connect to Microsoft Copilot Studio + +### 4.1 Access Copilot Studio + +1. Navigate to [Microsoft Copilot Studio](https://copilotstudio.microsoft.com/) +2. Sign in with your Azure/Microsoft 365 account +3. Create a new copilot or select an existing one + +### 4.2 Add Custom MCP Connector + +1. **In your Agent, Go to Tools**: + - Select **"Add a tool, new tool"** + +2. **Choose Connector Type**: + - Select **"Custom connector and new custom connector"** + - Choose **"Import from an OpenAPI file"** + +3. **Upload OpenAPI Specification**: + - Upload the `crm-mcp-connector.yaml` file + - Refresh and connect the newly added mcp tool in copilot studio + +``` + +If you've followed all steps correctly, you should now have: + +βœ… Local MCP server running and accessible +βœ… Public DevTunnel exposing your MCP server +βœ… Custom connector configured in Copilot Studio +βœ… Working MCP actions in your copilot + +Your copilot can now access customer data, billing information, and knowledge base through the MCP protocol! + +## Support + +If you encounter issues: + +1. Check the MCP server logs for errors +2. Verify DevTunnel connectivity +3. Test the OpenAPI specification in Swagger Editor +4. Monitor network traffic via DevTunnel inspection URL +5. Review Copilot Studio connector test results + +--- diff --git a/agentic_ai/applications/.env.sample b/agentic_ai/applications/.env.template similarity index 90% rename from agentic_ai/applications/.env.sample rename to agentic_ai/applications/.env.template index e7e0362ef..920046965 100644 --- a/agentic_ai/applications/.env.sample +++ b/agentic_ai/applications/.env.template @@ -2,10 +2,10 @@ # Azure OpenAI – chat model configuration # ############################################ # Replace with your model-deployment endpoint in Azure AI Foundry -AZURE_OPENAI_ENDPOINT="https://YOUR-OPENAI-SERVICE-ENDPOINT.openai.azure.com" +AZURE_OPENAI_ENDPOINT="https://YOUR-RESOURCE-NAME.openai.azure.com/" -# Replace with your Foundry project’s API key -AZURE_OPENAI_API_KEY="YOUR-OPENAI-API-KEY" +# Replace with your Foundry project's API key +AZURE_OPENAI_API_KEY="YOUR-AZURE-OPENAI-API-KEY" # Connection-string that identifies your Foundry project / workspace. Only needed when using Azure AI Agent. AZURE_AI_AGENT_PROJECT_CONNECTION_STRING="YOUR-OPENAI-PROJECT-CONNECTION-STRING" @@ -28,10 +28,10 @@ MCP_SERVER_URI="http://localhost:8000/mcp" # AGENT_MODULE="agents.autogen.single_agent.loop_agent" # AGENT_MODULE="agents.autogen.multi_agent.collaborative_multi_agent_round_robin" # AGENT_MODULE="agents.autogen.multi_agent.collaborative_multi_agent_selector_group" -# AGENT_MODULE="agents.autogen.multi_agent.handoff_multi_agent_domain" + AGENT_MODULE="agents.autogen.multi_agent.handoff_multi_agent_domain" # AGENT_MODULE="agents.semantic_kernel.multi_agent.collaborative_multi_agent" # AGENT_MODULE="agents.semantic_kernel.multi_agent.a2a.collaborative_multi_agent" -AGENT_MODULE="agents.autogen.single_agent.loop_agent" +#AGENT_MODULE="agents.autogen.single_agent.loop_agent" # ----------------------------------------------------------- # If you are experimenting with Logistics-A2A, uncomment: diff --git a/agentic_ai/backend_services/SECURITY-Git-Commit-Guide.md b/agentic_ai/backend_services/SECURITY-Git-Commit-Guide.md new file mode 100644 index 000000000..bf9f6a695 --- /dev/null +++ b/agentic_ai/backend_services/SECURITY-Git-Commit-Guide.md @@ -0,0 +1,227 @@ +# πŸ”’ Secure Git Commit Guide + +This document explains how to safely commit your MCP server and DevTunnel configuration without exposing sensitive information. + +## 🚨 Sensitive Information Identified + +The following files contain sensitive data that should NOT be committed: + +1. **`.env`** - Contains Azure OpenAI API keys and endpoints +2. **`crm-mcp-connector.yaml`** - Contains your specific DevTunnel URL +3. **Any files with actual secrets or connection strings** + +## βœ… What's Safe to Commit + +### Template Files Created: +- βœ… `.env.template` - Template with placeholder values +- βœ… `crm-mcp-connector.template.yaml` - Template with placeholder DevTunnel URL +- βœ… `README-MCP-DevTunnel-CopilotStudio.md` - Documentation (safe) +- βœ… `mcp_service.py` - Application code (safe) + +### Updated Security: +- βœ… `.gitignore` updated to exclude sensitive files +- βœ… Template files use placeholder values + +## πŸ”§ Before Committing - Setup Steps + +### 1. Verify .gitignore Protection + +The `.gitignore` now includes: +```gitignore +# DevTunnel and sensitive configuration files +**/crm-mcp-connector.yaml +**/*devtunnel*.yaml +**/*.local.yaml +**/*.personal.yaml +``` + +### 2. Check Git Status + +```bash +git status +``` + +Should show: +- βœ… `crm-mcp-connector.template.yaml` (new file) +- βœ… `.env.template` (new file) +- βœ… `README-MCP-DevTunnel-CopilotStudio.md` (new file) +- ❌ Should NOT show `.env` or `crm-mcp-connector.yaml` + +### 3. Remove Sensitive Files from Git Tracking (if already tracked) + +```bash +# Remove from git but keep local files +git rm --cached agentic_ai/applications/.env +git rm --cached agentic_ai/backend_services/crm-mcp-connector.yaml + +# Verify they're ignored +git status +``` + +## πŸš€ Safe Commit Process + +### Step 1: Add Safe Files Only + +```bash +# Add template files +git add agentic_ai/applications/.env.template +git add agentic_ai/backend_services/crm-mcp-connector.template.yaml +git add agentic_ai/backend_services/README-MCP-DevTunnel-CopilotStudio.md +git add .gitignore + +# Add any other safe code changes +git add agentic_ai/backend_services/mcp_service.py +``` + +### Step 2: Verify What's Being Committed + +```bash +# Review changes before commit +git diff --cached + +# Check for any exposed secrets +git diff --cached | grep -i "api[_-]key\|secret\|password\|token\|devtunnel" +``` + +### Step 3: Commit Safely + +```bash +git commit -m "feat: Add MCP server with DevTunnel support and Copilot Studio integration + +- Add FastMCP server implementation for customer data +- Add DevTunnel configuration templates +- Add comprehensive setup documentation +- Add OpenAPI specification template for Copilot Studio +- Update .gitignore to protect sensitive information +- Provide secure deployment instructions" +``` + +## πŸ“‹ Post-Commit Setup Instructions + +After committing, anyone cloning the repo should: + +### 1. Copy Template Files + +```bash +# Copy and customize environment variables +cp agentic_ai/applications/.env.template agentic_ai/applications/.env + +# Copy and customize MCP connector +cp agentic_ai/backend_services/crm-mcp-connector.template.yaml agentic_ai/backend_services/crm-mcp-connector.yaml +``` + +### 2. Update with Real Values + +**`.env` file:** +```env +AZURE_OPENAI_ENDPOINT="https://YOUR-ACTUAL-ENDPOINT.openai.azure.com/" +AZURE_OPENAI_API_KEY="YOUR-ACTUAL-API-KEY" +``` + +**`crm-mcp-connector.yaml` file:** +```yaml +host: YOUR-ACTUAL-DEVTUNNEL-URL.use.devtunnels.ms +``` + +## πŸ” Verification Commands + +### Check for Exposed Secrets + +```bash +# Search for potential exposed secrets +git log --patch | grep -i "key\|secret\|password\|token" | head -20 + +# Check current repository for sensitive patterns +git grep -i "api[_-]key\|secret\|password\|token" -- '*.py' '*.yaml' '*.yml' '*.json' '*.md' +``` + +### Validate .gitignore + +```bash +# Test if sensitive files are properly ignored +echo "test-secret-key" > test-sensitive.env +git status # Should not show test-sensitive.env +rm test-sensitive.env +``` + +## πŸ›‘οΈ Additional Security Best Practices + +### 1. Use Environment Variables in Production + +```python +import os +from dotenv import load_dotenv + +load_dotenv() + +# Good: Get from environment +api_key = os.getenv('AZURE_OPENAI_API_KEY') + +# Bad: Hardcoded in source +api_key = "your-actual-key-here" +``` + +### 2. Use Azure Key Vault for Production + +```python +from azure.keyvault.secrets import SecretClient +from azure.identity import DefaultAzureCredential + +credential = DefaultAzureCredential() +client = SecretClient(vault_url="https://your-vault.vault.azure.net/", credential=credential) +api_key = client.get_secret("azure-openai-api-key").value +``` + +### 3. Rotate DevTunnel URLs Regularly + +```bash +# Delete old tunnel +devtunnel delete crm-mcp + +# Create new tunnel with different name +devtunnel create crm-mcp-new --allow-anonymous +``` + +## βœ… Final Checklist + +Before pushing to remote repository: + +- [ ] `.env` file is in `.gitignore` and not tracked +- [ ] `crm-mcp-connector.yaml` is in `.gitignore` and not tracked +- [ ] Template files contain placeholder values only +- [ ] Documentation doesn't contain actual secrets +- [ ] Verified with `git status` and `git diff --cached` +- [ ] Searched commit for sensitive patterns +- [ ] Added setup instructions for other developers + +## πŸ†˜ If You Accidentally Committed Secrets + +If you've already committed sensitive information: + +### 1. Remove from History (Local Only) + +```bash +# Reset to before the sensitive commit +git reset --hard HEAD~1 + +# Or remove specific files from last commit +git reset HEAD~1 -- agentic_ai/applications/.env +git commit --amend +``` + +### 2. If Already Pushed to Remote + +```bash +# Force push (DANGEROUS - coordinate with team) +git push --force-with-lease origin main +``` + +### 3. Rotate All Exposed Secrets + +- πŸ”„ Generate new Azure OpenAI API keys +- πŸ”„ Create new DevTunnel URLs +- πŸ”„ Update all dependent services + +--- + +**Remember**: It's always better to be safe than sorry with sensitive information! πŸ”’ \ No newline at end of file diff --git a/agentic_ai/backend_services/crm-mcp-connector.template.yaml b/agentic_ai/backend_services/crm-mcp-connector.template.yaml new file mode 100644 index 000000000..b4066ddd4 --- /dev/null +++ b/agentic_ai/backend_services/crm-mcp-connector.template.yaml @@ -0,0 +1,18 @@ +swagger: '2.0' +info: + title: CRM MCP Server + description: Allows to manage CRM data for specific customers, details, subscriptions providing tools to list, search, update, and create customer, product and invoice data. + version: 1.0.0 +host: YOUR_DEVTUNNEL_URL.use.devtunnels.ms +basePath: /mcp +schemes: + - https +paths: + /: + post: + summary: CRM MCP Server + x-ms-agentic-protocol: mcp-streamable-1.0 + operationId: InvokeMCP + responses: + '200': + description: Success \ No newline at end of file From b9eb5e7dae8327bec142f102b5b76cf93e9c3390 Mon Sep 17 00:00:00 2001 From: Kranthi Kumar Manchikanti Date: Mon, 1 Sep 2025 10:00:19 -0400 Subject: [PATCH 2/2] docs: Update MCP-COPILOTSTUDIO.md with template file usage instructions - Add clarification about renaming template file to actual config - Improve setup instructions for Copilot Studio integration --- MCP-COPILOTSTUDIO.md | 1 + .../SECURITY-Git-Commit-Guide.md | 227 ------------------ 2 files changed, 1 insertion(+), 227 deletions(-) delete mode 100644 agentic_ai/backend_services/SECURITY-Git-Commit-Guide.md diff --git a/MCP-COPILOTSTUDIO.md b/MCP-COPILOTSTUDIO.md index 5ca20fac1..9c8a53621 100644 --- a/MCP-COPILOTSTUDIO.md +++ b/MCP-COPILOTSTUDIO.md @@ -131,6 +131,7 @@ host: YOUR_ACTUAL_TUNNEL_URL.use.devtunnels.ms - Choose **"Import from an OpenAPI file"** 3. **Upload OpenAPI Specification**: + - Rename the `crm-mcp-connector.template` file to `crm-mcp-connector` and edit the host details - Upload the `crm-mcp-connector.yaml` file - Refresh and connect the newly added mcp tool in copilot studio diff --git a/agentic_ai/backend_services/SECURITY-Git-Commit-Guide.md b/agentic_ai/backend_services/SECURITY-Git-Commit-Guide.md deleted file mode 100644 index bf9f6a695..000000000 --- a/agentic_ai/backend_services/SECURITY-Git-Commit-Guide.md +++ /dev/null @@ -1,227 +0,0 @@ -# πŸ”’ Secure Git Commit Guide - -This document explains how to safely commit your MCP server and DevTunnel configuration without exposing sensitive information. - -## 🚨 Sensitive Information Identified - -The following files contain sensitive data that should NOT be committed: - -1. **`.env`** - Contains Azure OpenAI API keys and endpoints -2. **`crm-mcp-connector.yaml`** - Contains your specific DevTunnel URL -3. **Any files with actual secrets or connection strings** - -## βœ… What's Safe to Commit - -### Template Files Created: -- βœ… `.env.template` - Template with placeholder values -- βœ… `crm-mcp-connector.template.yaml` - Template with placeholder DevTunnel URL -- βœ… `README-MCP-DevTunnel-CopilotStudio.md` - Documentation (safe) -- βœ… `mcp_service.py` - Application code (safe) - -### Updated Security: -- βœ… `.gitignore` updated to exclude sensitive files -- βœ… Template files use placeholder values - -## πŸ”§ Before Committing - Setup Steps - -### 1. Verify .gitignore Protection - -The `.gitignore` now includes: -```gitignore -# DevTunnel and sensitive configuration files -**/crm-mcp-connector.yaml -**/*devtunnel*.yaml -**/*.local.yaml -**/*.personal.yaml -``` - -### 2. Check Git Status - -```bash -git status -``` - -Should show: -- βœ… `crm-mcp-connector.template.yaml` (new file) -- βœ… `.env.template` (new file) -- βœ… `README-MCP-DevTunnel-CopilotStudio.md` (new file) -- ❌ Should NOT show `.env` or `crm-mcp-connector.yaml` - -### 3. Remove Sensitive Files from Git Tracking (if already tracked) - -```bash -# Remove from git but keep local files -git rm --cached agentic_ai/applications/.env -git rm --cached agentic_ai/backend_services/crm-mcp-connector.yaml - -# Verify they're ignored -git status -``` - -## πŸš€ Safe Commit Process - -### Step 1: Add Safe Files Only - -```bash -# Add template files -git add agentic_ai/applications/.env.template -git add agentic_ai/backend_services/crm-mcp-connector.template.yaml -git add agentic_ai/backend_services/README-MCP-DevTunnel-CopilotStudio.md -git add .gitignore - -# Add any other safe code changes -git add agentic_ai/backend_services/mcp_service.py -``` - -### Step 2: Verify What's Being Committed - -```bash -# Review changes before commit -git diff --cached - -# Check for any exposed secrets -git diff --cached | grep -i "api[_-]key\|secret\|password\|token\|devtunnel" -``` - -### Step 3: Commit Safely - -```bash -git commit -m "feat: Add MCP server with DevTunnel support and Copilot Studio integration - -- Add FastMCP server implementation for customer data -- Add DevTunnel configuration templates -- Add comprehensive setup documentation -- Add OpenAPI specification template for Copilot Studio -- Update .gitignore to protect sensitive information -- Provide secure deployment instructions" -``` - -## πŸ“‹ Post-Commit Setup Instructions - -After committing, anyone cloning the repo should: - -### 1. Copy Template Files - -```bash -# Copy and customize environment variables -cp agentic_ai/applications/.env.template agentic_ai/applications/.env - -# Copy and customize MCP connector -cp agentic_ai/backend_services/crm-mcp-connector.template.yaml agentic_ai/backend_services/crm-mcp-connector.yaml -``` - -### 2. Update with Real Values - -**`.env` file:** -```env -AZURE_OPENAI_ENDPOINT="https://YOUR-ACTUAL-ENDPOINT.openai.azure.com/" -AZURE_OPENAI_API_KEY="YOUR-ACTUAL-API-KEY" -``` - -**`crm-mcp-connector.yaml` file:** -```yaml -host: YOUR-ACTUAL-DEVTUNNEL-URL.use.devtunnels.ms -``` - -## πŸ” Verification Commands - -### Check for Exposed Secrets - -```bash -# Search for potential exposed secrets -git log --patch | grep -i "key\|secret\|password\|token" | head -20 - -# Check current repository for sensitive patterns -git grep -i "api[_-]key\|secret\|password\|token" -- '*.py' '*.yaml' '*.yml' '*.json' '*.md' -``` - -### Validate .gitignore - -```bash -# Test if sensitive files are properly ignored -echo "test-secret-key" > test-sensitive.env -git status # Should not show test-sensitive.env -rm test-sensitive.env -``` - -## πŸ›‘οΈ Additional Security Best Practices - -### 1. Use Environment Variables in Production - -```python -import os -from dotenv import load_dotenv - -load_dotenv() - -# Good: Get from environment -api_key = os.getenv('AZURE_OPENAI_API_KEY') - -# Bad: Hardcoded in source -api_key = "your-actual-key-here" -``` - -### 2. Use Azure Key Vault for Production - -```python -from azure.keyvault.secrets import SecretClient -from azure.identity import DefaultAzureCredential - -credential = DefaultAzureCredential() -client = SecretClient(vault_url="https://your-vault.vault.azure.net/", credential=credential) -api_key = client.get_secret("azure-openai-api-key").value -``` - -### 3. Rotate DevTunnel URLs Regularly - -```bash -# Delete old tunnel -devtunnel delete crm-mcp - -# Create new tunnel with different name -devtunnel create crm-mcp-new --allow-anonymous -``` - -## βœ… Final Checklist - -Before pushing to remote repository: - -- [ ] `.env` file is in `.gitignore` and not tracked -- [ ] `crm-mcp-connector.yaml` is in `.gitignore` and not tracked -- [ ] Template files contain placeholder values only -- [ ] Documentation doesn't contain actual secrets -- [ ] Verified with `git status` and `git diff --cached` -- [ ] Searched commit for sensitive patterns -- [ ] Added setup instructions for other developers - -## πŸ†˜ If You Accidentally Committed Secrets - -If you've already committed sensitive information: - -### 1. Remove from History (Local Only) - -```bash -# Reset to before the sensitive commit -git reset --hard HEAD~1 - -# Or remove specific files from last commit -git reset HEAD~1 -- agentic_ai/applications/.env -git commit --amend -``` - -### 2. If Already Pushed to Remote - -```bash -# Force push (DANGEROUS - coordinate with team) -git push --force-with-lease origin main -``` - -### 3. Rotate All Exposed Secrets - -- πŸ”„ Generate new Azure OpenAI API keys -- πŸ”„ Create new DevTunnel URLs -- πŸ”„ Update all dependent services - ---- - -**Remember**: It's always better to be safe than sorry with sensitive information! πŸ”’ \ No newline at end of file