From a02e89ec5be0fdbe2e293eaebbf024feb5305c2d Mon Sep 17 00:00:00 2001 From: Vinay Gera Date: Wed, 15 Oct 2025 00:05:57 -0700 Subject: [PATCH 01/10] add bugs bash docs --- docs/bug-bash/README.md | 139 ++++++ docs/bug-bash/installation-testing.md | 350 ++++++++++++++ docs/bug-bash/scenarios/agent-building.md | 213 +++++++++ .../bug-bash/scenarios/database-operations.md | 437 +++++++++++++++++ docs/bug-bash/scenarios/deployment.md | 324 +++++++++++++ docs/bug-bash/scenarios/full-stack-apps.md | 445 ++++++++++++++++++ docs/bug-bash/scenarios/infra-as-code.md | 171 +++++++ docs/bug-bash/scenarios/paas-services.md | 378 +++++++++++++++ docs/bug-bash/scenarios/storage-operations.md | 299 ++++++++++++ 9 files changed, 2756 insertions(+) create mode 100644 docs/bug-bash/README.md create mode 100644 docs/bug-bash/installation-testing.md create mode 100644 docs/bug-bash/scenarios/agent-building.md create mode 100644 docs/bug-bash/scenarios/database-operations.md create mode 100644 docs/bug-bash/scenarios/deployment.md create mode 100644 docs/bug-bash/scenarios/full-stack-apps.md create mode 100644 docs/bug-bash/scenarios/infra-as-code.md create mode 100644 docs/bug-bash/scenarios/paas-services.md create mode 100644 docs/bug-bash/scenarios/storage-operations.md diff --git a/docs/bug-bash/README.md b/docs/bug-bash/README.md new file mode 100644 index 0000000000..394e114075 --- /dev/null +++ b/docs/bug-bash/README.md @@ -0,0 +1,139 @@ +# Azure MCP Server - Bug Bash Welcome Guide + +Welcome to the Azure MCP Server Bug Bash! We're excited to have you help us improve the quality and reliability of Azure MCP Server across different platforms and scenarios. + +## Table of Contents + +- [Introduction](#introduction) +- [Bug Bash Goals](#bug-bash-goals) +- [What to Test](#what-to-test) +- [How to Report Issues](#how-to-report-issues) +- [Testing Scenarios](#testing-scenarios) +- [Resources](#resources) + +## Introduction + +The Azure MCP Server enables AI agents to interact with Azure services through natural language commands. As we continue to enhance the server, we need your help to identify issues across different platforms, IDEs, and usage scenarios. + +This bug bash focuses on: +- **Multi-platform compatibility** (Windows, macOS, Linux) +- **Installation and setup** across different IDEs +- **Performance and stability** under real-world usage +- **Authentication** across different environments +- **End-to-end scenarios** that developers commonly encounter + +## Bug Bash Goals + +The primary goals of this bug bash are to: + +1. **Validate cross-platform compatibility** - Ensure the server works reliably on Windows, macOS, and Linux +2. **Verify installation experience** - Test installation across VS Code, Visual Studio, and IntelliJ IDEA +3. **Assess performance** - Monitor memory consumption and CPU usage under typical workloads +4. **Validate authentication** - Ensure auth works consistently across all platforms +5. **Test server modes** - Verify single, namespace, and all modes work as expected +6. **Validate feature flags** - Test enabling/disabling server features +7. **Exercise real-world scenarios** - Run through common developer workflows + +## What to Test + +We encourage you to test the following areas: + +### Platform Testing +- [ ] **Windows** - Test on Windows 10/11 +- [ ] **macOS** - Test on macOS (Intel and Apple Silicon) +- [ ] **Linux** - Test on Ubuntu, Fedora, or other distributions + +### IDE Installation +- [ ] **VS Code** - Stable and Insiders versions +- [ ] **Visual Studio 2022** - Community, Professional, or Enterprise +- [ ] **IntelliJ IDEA** - Ultimate or Community editions + +### Performance Monitoring +- [ ] Monitor **memory consumption** during typical operations +- [ ] Monitor **CPU usage** during command execution +- [ ] Test with **multiple concurrent operations** +- [ ] Observe behavior during **long-running sessions** + +### Authentication Testing +- [ ] Test **Azure CLI authentication** (`az login`) +- [ ] Test **Azure PowerShell authentication** (`ConnectAzAccount`) +- [ ] Test **Interactive browser authentication** +- [ ] Test authentication across **multiple tenants** +- [ ] Test authentication with **service principals** + +### Server Mode Testing +- [ ] **All mode** - All tools exposed individually +- [ ] **Namespace mode** - Tools grouped by Azure service +- [ ] **Single mode** - Single dynamic proxy tool +- [ ] **Filtered namespaces** - Specific services only + +### Feature Flag Testing +- [ ] Enable/disable server +- [ ] Test read-only mode +- [ ] Test with different namespace configurations +- [ ] Test tool filtering + +## How to Report Issues + +When you find a bug or issue, please report it on GitHub: + +### Report Issues Here: [https://github.com/microsoft/mcp/issues](https://github.com/microsoft/mcp/issues) + +**Steps to Report:** + +1. Go to the [issues page](https://github.com/microsoft/mcp/issues) +2. Click **"New Issue"** +3. Select **"Azure MCP - Bug Bash Report"** template +4. Fill in the following information: + +**Required Information:** +- **Platform**: Windows/macOS/Linux (include version) +- **IDE**: VS Code/Visual Studio/IntelliJ (include version) +- **Azure MCP Server Version**: Found in extension details or `azmcp --version` +- **Node.js Version** (if using npm): Run `node --version` +- **Description**: Clear description of the issue +- **Steps to Reproduce**: Detailed steps to reproduce the problem +- **Expected Behavior**: What you expected to happen +- **Actual Behavior**: What actually happened +- **Logs**: Include relevant error messages or logs (see [Troubleshooting Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/TROUBLESHOOTING.md#logging-and-diagnostics)) +- **Screenshots**: If applicable, include screenshots + +## Testing Scenarios + +We've prepared detailed testing guides for common scenarios: + +### Scenario Guides + +1. **[Installation Testing](installation-testing.md)** - Test installation across different platforms and IDEs +2. **[Infrastructure as Code](scenarios/infra-as-code.md)** - Generate and deploy Azure infrastructure +3. **[PaaS Services](scenarios/paas-services.md)** - Work with App Service, Container Apps, and Functions +4. **[Storage Operations](scenarios/storage-operations.md)** - Test blob storage and file operations +5. **[Database Operations](scenarios/database-operations.md)** - Work with Cosmos DB, PostgreSQL, and Azure SQL +6. **[Deployment Scenarios](scenarios/deployment.md)** - Deploy resources and applications +7. **[Full Stack Applications](scenarios/full-stack-apps.md)** - Build complete apps with database backends +9. **[Agent Building](scenarios/agent-building.md)** - Create and deploy Azure Foundry agents + +### Quick Start Scenarios + +If you're short on time, try these quick scenarios: + +- **5 minutes**: Install Azure MCP extension and verify tools are loaded +- **10 minutes**: List your Azure resources (subscriptions, resource groups, storage accounts) +- **15 minutes**: Create a simple storage account and upload a file +- **30 minutes**: Create a basic web app and deploy it to Azure App Service + +## Resources + +### Documentation +- [Azure MCP Server Documentation](https://learn.microsoft.com/azure/developer/azure-mcp-server/) +- [Installation Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/README.md#installation) +- [Troubleshooting Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/TROUBLESHOOTING.md) +- [Authentication Guide](https://github.com/microsoft/mcp/blob/main/docs/Authentication.md) +- [Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) + +### Test Prompts +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - Sample prompts for testing + +### Support +- [GitHub Issues](https://github.com/microsoft/mcp/issues) - Report bugs and issues +- [Discussions](https://github.com/microsoft/mcp/discussions) - Ask questions and share feedback \ No newline at end of file diff --git a/docs/bug-bash/installation-testing.md b/docs/bug-bash/installation-testing.md new file mode 100644 index 0000000000..cc48088e05 --- /dev/null +++ b/docs/bug-bash/installation-testing.md @@ -0,0 +1,350 @@ +# Installation Testing Guide + +This guide covers testing Azure MCP Server installation across different platforms and IDEs. + +## Table of Contents + +- [Prerequisites](#prerequisites) +- [Platform-Specific Testing](#platform-specific-testing) +- [IDE Installation Testing](#ide-installation-testing) +- [Verification Steps](#verification-steps) + +## Prerequisites + +Before testing installation, ensure you have: + +- [ ] **Azure Subscription** - Access to an Azure subscription + +## Platform-Specific Testing + +### Windows Testing + +#### Installation Steps + +1. **Install Prerequisites** + ```powershell + # Check if Node.js is installed + node --version + + # If not installed, download from https://nodejs.org/ + ``` + +2. **Install via VS Code Extension** (Recommended) + - Open VS Code + - Go to Extensions (Ctrl+Shift+X) + - Search for "Azure MCP Server" + - Click Install + +3. **Install via NPM** (Alternative) + ```powershell + npm install -g @azure/mcp@latest + ``` + +4. **Install via NuGet** (Alternative) + ```powershell + dotnet tool install Azure.Mcp + ``` + +#### Verification +```powershell +# Check Azure MCP version +azmcp --version + +# Verify Azure CLI +az --version + +# Test basic command +azmcp server start --help +``` + +#### Things to Test +- [ ] Installation completes without errors +- [ ] Binary is accessible from command line +- [ ] Extension loads in VS Code +- [ ] Server starts successfully +- [ ] Memory usage after installation: _____ MB +- [ ] Installation time: _____ minutes + +### macOS Testing + +#### Installation Steps + +1. **Install Prerequisites** + ```bash + # Check if Node.js is installed + node --version + + # Install via Homebrew if needed + brew install node + ``` + +2. **Install via VS Code Extension** (Recommended) + - Open VS Code + - Go to Extensions (Cmd+Shift+X) + - Search for "Azure MCP Server" + - Click Install + +3. **Install via NPM** (Alternative) + ```bash + npm install -g @azure/mcp@latest + ``` + +4. **Install via .NET Tool** (Alternative) + ```bash + dotnet tool install Azure.Mcp + ``` + +#### Verification +```bash +# Check Azure MCP version +azmcp --version + +# Verify Azure CLI +az --version + +# Test basic command +azmcp server start --help +``` + +#### Platform-Specific Checks +- [ ] **Intel Macs**: Installation works on x64 architecture +- [ ] **Apple Silicon**: Installation works on ARM64 architecture + +#### Things to Test +- [ ] Installation completes without errors +- [ ] Binary has correct permissions (executable) +- [ ] Extension loads in VS Code +- [ ] Server starts successfully +- [ ] No "unidentified developer" warnings +- [ ] Memory usage after installation: _____ MB +- [ ] Installation time: _____ minutes + +--- + +### Linux Testing + +#### Installation Steps + +1. **Install Prerequisites** + ```bash + # Check if Node.js is installed + node --version + + # Ubuntu/Debian + sudo apt update + sudo apt install nodejs npm + + # Fedora + sudo dnf install nodejs npm + ``` + +2. **Install via VS Code Extension** (Recommended) + - Open VS Code + - Go to Extensions (Ctrl+Shift+X) + - Search for "Azure MCP Server" + - Click Install + +3. **Install via NPM** (Alternative) + ```bash + npm install -g @azure/mcp@latest + ``` + +4. **Install via .NET Tool** (Alternative) + ```bash + dotnet tool install Azure.Mcp + ``` + +#### Verification +```bash +# Check Azure MCP version +azmcp --version + +# Verify Azure CLI +az --version + +# Test basic command +azmcp server start --help +``` + +#### Things to Test +- [ ] Installation completes without errors +- [ ] Binary has correct permissions +- [ ] Extension loads in VS Code +- [ ] Server starts successfully +- [ ] No dependency conflicts +- [ ] Memory usage after installation: _____ MB +- [ ] Installation time: _____ minutes + +--- + +## IDE Installation Testing + +### VS Code + +#### Versions to Test +- [ ] **VS Code Stable** (latest) +- [ ] **VS Code Insiders** (latest) + +#### Installation Testing + +1. **Install Extension** + - Method 1: Via Extension Marketplace + - Method 2: Via Command Palette (`ext install ms-azuretools.vscode-azure-mcp-server`) + - Method 3: Via [Installation Link](https://vscode.dev/redirect?url=vscode:extension/ms-azuretools.vscode-azure-mcp-server) + +2. **Verify Installation** + ``` + 1. Open Command Palette (Ctrl+Shift+P / Cmd+Shift+P) + 2. Run "MCP: List Servers" + 3. Verify "Azure MCP Server" appears in the list + 4. Click "Start Server" + 5. Check Output window for startup logs + ``` + +3. **Test Configuration** + - Open `.vscode/mcp.json` (if it exists) + - Verify server configuration + - Test with custom settings + +#### Things to Test +- [ ] Extension installs without errors +- [ ] Extension appears in Extensions list +- [ ] Server starts successfully +- [ ] Tools appear in GitHub Copilot Chat +- [ ] Tool count matches expectations +- [ ] Configuration changes are applied +- [ ] Server restarts successfully +- [ ] Logs are visible in Output window + +#### Performance Checks +- [ ] Extension activation time: _____ ms +- [ ] Memory usage (extension): _____ MB +- [ ] Memory usage (server): _____ MB +- [ ] CPU usage during idle: _____ % + +--- + +### Visual Studio 2022 + +#### Versions to Test +- [ ] **Visual Studio 2022 Community** +- [ ] **Visual Studio 2022 Professional** +- [ ] **Visual Studio 2022 Enterprise** + +#### Installation Testing + +1. **Install Extension** + ``` + 1. Open Visual Studio 2022 + 2. Go to Extensions > Manage Extensions + 3. Search for "GitHub Copilot for Azure" + 4. Click Install + 5. Restart Visual Studio + ``` + +2. **Verify Installation** + ``` + 1. Open a solution or project + 2. Open GitHub Copilot Chat + 3. Switch to Agent mode + 4. Verify Azure MCP tools are available + ``` + +#### Things to Test +- [ ] Extension installs without errors +- [ ] Extension appears in Extensions list +- [ ] Extension integrates with Copilot +- [ ] Tools are accessible in chat +- [ ] Server responds to commands + +#### Performance Checks +- [ ] Extension load time: _____ seconds +- [ ] Memory usage: _____ MB +- [ ] CPU usage during idle: _____ % + +--- + +### IntelliJ IDEA + +#### Versions to Test +- [ ] **IntelliJ IDEA Ultimate** (2024.3+) +- [ ] **IntelliJ IDEA Community** (2024.3+) + +#### Installation Testing + +1. **Install Plugins** + ``` + 1. Open IntelliJ IDEA + 2. Go to Settings/Preferences > Plugins + 3. Search for "GitHub Copilot" + 4. Install GitHub Copilot plugin + 5. Search for "Azure Toolkit for IntelliJ" + 6. Install Azure Toolkit + 7. Restart IDE + ``` + +2. **Verify Installation** + ``` + 1. Open GitHub Copilot Chat + 2. Verify Azure MCP tools are available + 3. Test a simple command + ``` + +#### Things to Test +- [ ] Plugins install without conflicts +- [ ] Plugins appear in Plugins list +- [ ] Azure MCP integrates with Copilot +- [ ] Tools are accessible +- [ ] Server responds to commands + +#### Performance Checks +- [ ] Plugin load time: _____ seconds +- [ ] Memory usage: _____ MB +- [ ] CPU usage during idle: _____ % + +--- + +## Verification Steps + +After installation on any platform/IDE, verify: + +### 1. Server Status +```bash +# Check if server is running +# In VS Code: Check Output > MCP: Azure MCP Server +# Look for: "Server started successfully" +``` + +### 2. Tool Discovery +```bash +# In GitHub Copilot Chat (Agent mode): +# Ask: "What Azure MCP tools are available?" +# Verify: Tools are listed +``` + +### 3. Authentication +```bash +# Login to Azure +az login + +# Test authentication +# Ask Copilot: "List my Azure subscriptions" +``` + +### 4. Basic Functionality +```bash +# Test a simple command +# Ask Copilot: "List my Azure resource groups" +# Verify: Resource groups are returned +``` + +## Related Resources + +- [Main Bug Bash Guide](README.md) +- [Troubleshooting Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/TROUBLESHOOTING.md) +- [Installation Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/README.md#installation) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +--- + +**Next Steps**: After completing installation testing, proceed to [Scenario Testing](scenarios/README.md). diff --git a/docs/bug-bash/scenarios/agent-building.md b/docs/bug-bash/scenarios/agent-building.md new file mode 100644 index 0000000000..75a9d01ab6 --- /dev/null +++ b/docs/bug-bash/scenarios/agent-building.md @@ -0,0 +1,213 @@ +# Agent Building Testing Scenario + +Test Azure MCP Server's capabilities for building, deploying, and managing AI agents using Azure AI Foundry and related services. + +## Objectives + +- Create and configure Azure AI Foundry resources +- Build custom AI agents with Azure MCP integration +- Deploy agents to Azure +- Test agent interactions and workflows +- Validate agent monitoring and management +- Test multi-agent scenarios + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription with AI services enabled +- [ ] GitHub Copilot with Agent mode enabled +- [ ] Basic understanding of AI agent concepts + +## Test Scenarios + +### Scenario 1: Azure AI Foundry Setup + +**Objective**: Set up Azure AI Foundry resources for agent development + +#### Phase 1: Create AI Foundry Resources + +1. **Create AI Hub**: + ``` + Create an Azure AI Foundry hub: + - Name: 'bugbash-ai-hub-' + - Resource group: '' + - Location: East US + - SKU: Standard + ``` + +2. **Verify hub creation**: + ``` + List all Azure AI Foundry hubs in my subscription + ``` + +3. **Get hub details**: + ``` + Show me the details of AI Foundry hub 'bugbash-ai-hub-' + ``` + +4. **Create AI project**: + ``` + Create an Azure AI Foundry project: + - Name: 'bugbash-ai-project' + - Hub: 'bugbash-ai-hub-' + - Description: 'Bug bash agent testing project' + ``` + +**Verify**: +- [ ] AI Hub created successfully +- [ ] Hub is in running state +- [ ] AI Project created +- [ ] Resources are properly linked + +#### Phase 2: Configure AI Services + +5. **Check available AI models**: + ``` + What AI models are available in my Azure AI Foundry hub + 'bugbash-ai-hub-'? + ``` + +6. **Deploy OpenAI model**: + ``` + Deploy GPT-4o model to my AI Foundry project 'bugbash-ai-project' + ``` + +7. **List deployments**: + ``` + List all model deployments in AI Foundry project 'bugbash-ai-project' + ``` + +8. **Get endpoint information**: + ``` + Get the endpoint URL and keys for my deployed model + ``` + +**Verify**: +- [ ] Available models listed +- [ ] Model deployment successful +- [ ] Endpoint accessible +- [ ] Authentication configured + +**Expected Results**: +- AI Foundry hub created +- AI project configured +- AI model deployed +- Endpoints are accessible + +### Scenario 2: Deploy and Test Agent + +**Objective**: Deploy the agent and test its functionality + +#### Phase 1: Deploy Agent Code + +1. **Get deployment guidance**: + ``` + How do I deploy my Python agent code to Function App + 'bugbash-agent-func-'? + ``` + +2. **Generate sample agent code**: + ``` + Generate sample Python code for an Azure management agent that: + - Uses Azure OpenAI for natural language understanding + - Uses Azure MCP Server to manage Azure resources + - Stores conversation history in Cosmos DB + ``` + +3. **Deploy agent**: + ``` + Deploy my agent code to Function App 'bugbash-agent-func-' + ``` + +**Verify**: +- [ ] Deployment instructions clear +- [ ] Sample code provided +- [ ] Deployment successful +- [ ] Function App running + +#### Phase 2: Test Agent Interactions + +4. **Test basic agent query**: + ``` + Send a test request to my agent: "List all my storage accounts" + ``` + +5. **Test resource creation**: + ``` + Ask my agent to: "Create a new storage account named + 'agenttest' in East US" + ``` + +6. **Test monitoring capability**: + ``` + Ask my agent: "Show me the health status of all my resources in + resource group ''" + ``` + +7. **Test complex query**: + ``` + Ask my agent: "Which of my resources are costing the most this month?" + ``` + +**Verify**: +- [ ] Agent responds correctly +- [ ] Resource operations work +- [ ] Monitoring data retrieved +- [ ] Complex queries handled + +#### Phase 3: Monitor Agent Performance + +8. **Check agent logs**: + ``` + Show me the logs for Function App 'bugbash-agent-func-' + from the last hour + ``` + +9. **Monitor agent metrics**: + ``` + Show me performance metrics for my agent Function App + ``` + +10. **Check Cosmos DB usage**: + ``` + Show me the Cosmos DB usage statistics for my agent's state database + ``` + +**Verify**: +- [ ] Logs accessible +- [ ] Metrics available +- [ ] Performance acceptable +- [ ] State persistence working + +**Expected Results**: +- Agent deployed successfully +- Agent responds to queries +- Resource operations work +- Monitoring and logging functional + +## Common Issues to Watch For + +- **Authentication Failures**: Managed identity misconfiguration +- **Token Limits**: Exceeding model context windows +- **Rate Limiting**: API throttling from AI services +- **State Persistence**: Session state not saved correctly +- **Cold Starts**: Function App cold start latency +- **Network Timeouts**: Long-running operations timing out +- **Cost Overruns**: Unexpected charges from AI API calls +- **Model Hallucinations**: Incorrect or inconsistent agent responses +- **Integration Issues**: MCP Server connection problems +- **Event Ordering**: Messages processed out of sequence + +## Related Resources + +- [Azure AI Foundry Documentation](https://learn.microsoft.com/azure/ai-studio/) +- [Azure OpenAI Service](https://learn.microsoft.com/azure/ai-services/openai/) +- [Azure Functions for AI Agents](https://learn.microsoft.com/azure/azure-functions/) +- [Model Context Protocol](https://modelcontextprotocol.io/) +- [Azure MCP Server GitHub](https://github.com/microsoft/mcp) +- [Agent Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +**Next**: [Database Operations Testing](database-operations.md) \ No newline at end of file diff --git a/docs/bug-bash/scenarios/database-operations.md b/docs/bug-bash/scenarios/database-operations.md new file mode 100644 index 0000000000..8bd5f8a4fd --- /dev/null +++ b/docs/bug-bash/scenarios/database-operations.md @@ -0,0 +1,437 @@ +# Database Operations Testing Scenario + +Test Azure MCP Server's database management capabilities across Cosmos DB, PostgreSQL, and Azure SQL. + +## Objectives + +- Create and manage Cosmos DB accounts and containers +- Work with PostgreSQL flexible servers +- Manage Azure SQL databases +- Test query execution and data operations +- Verify connection string management +- Test performance and scaling + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] GitHub Copilot with Agent mode enabled + +## Test Scenarios + +### Scenario 1: Cosmos DB Operations + +**Objective**: Test Cosmos DB account, database, and container management + +#### Phase 1: Create Cosmos DB Resources + +1. **Create Cosmos DB account**: + ``` + Create a Cosmos DB account named 'bugbash-cosmos-' in + resource group '' in East US with SQL API + ``` + +2. **Verify account creation**: + ``` + List all Cosmos DB accounts in my subscription + ``` + +3. **Check account details**: + ``` + Show me the details of Cosmos DB account 'bugbash-cosmos-' + ``` + +**Verify**: +- [ ] Account created successfully +- [ ] API type is SQL (Core) +- [ ] Location is correct +- [ ] Account is accessible + +#### Phase 2: Database and Container Management + +4. **Create database**: + ``` + Create a database named 'ProductCatalog' in Cosmos DB account + 'bugbash-cosmos-' + ``` + +5. **List databases**: + ``` + List all databases in Cosmos DB account 'bugbash-cosmos-' + ``` + +6. **Create container**: + ``` + Create a container named 'Products' in database 'ProductCatalog' + with partition key '/category' and 400 RU/s throughput + ``` + +7. **List containers**: + ``` + List all containers in database 'ProductCatalog' + ``` + +**Verify**: +- [ ] Database created successfully +- [ ] Container created with correct partition key +- [ ] Throughput set correctly +- [ ] Resources are listed properly + +#### Phase 3: Data Operations + +8. **Insert sample data**: + ``` + Add the following items to the Products container in Cosmos DB + 'bugbash-cosmos-', database 'ProductCatalog': + + Item 1: + { + "id": "1", + "name": "Laptop", + "category": "Electronics", + "price": 999.99, + "stock": 50 + } + + Item 2: + { + "id": "2", + "name": "Desk Chair", + "category": "Furniture", + "price": 199.99, + "stock": 100 + } + ``` + +9. **Query data**: + ``` + Query all items from the Products container where category is 'Electronics' + ``` + +10. **Query with search**: + ``` + Show me all items that contain the word 'Laptop' in the Products + container in database 'ProductCatalog' + ``` + +**Verify**: +- [ ] Data inserted successfully +- [ ] Queries return correct results +- [ ] Partition key filtering works +- [ ] Search functionality works + +**Expected Results**: +- Cosmos DB account created +- Database and container created +- Data operations work correctly +- Queries return expected results + +--- + +### Scenario 2: PostgreSQL Operations + +**Objective**: Test Azure Database for PostgreSQL flexible server management + +#### Phase 1: Create PostgreSQL Server + +1. **Create PostgreSQL server**: + ``` + Create an Azure Database for PostgreSQL flexible server: + - Name: 'bugbash-postgres-' + - Resource group: '' + - Admin username: 'dbadmin' + - Password: '' + - Version: PostgreSQL 14 + - Tier: Burstable B1ms + - Storage: 32 GB + - Region: East US + - Allow Azure services and resources to access this server + ``` + +2. **List PostgreSQL servers**: + ``` + List all PostgreSQL servers in my subscription + ``` + +3. **Get server details**: + ``` + Show me the details of PostgreSQL server 'bugbash-postgres-' + ``` + +**Verify**: +- [ ] Server created successfully +- [ ] Version is PostgreSQL 14 +- [ ] Tier and storage correct +- [ ] Firewall allows Azure services + +#### Phase 2: Database and Table Operations + +4. **Create database**: + ``` + Create a database named 'inventory' on PostgreSQL server + 'bugbash-postgres-' + ``` + +5. **List databases**: + ``` + List all databases in PostgreSQL server 'bugbash-postgres-' + ``` + +6. **Create tables**: + ``` + In PostgreSQL database 'inventory', create a table named 'products' with: + - id: SERIAL PRIMARY KEY + - name: VARCHAR(100) NOT NULL + - description: TEXT + - price: DECIMAL(10,2) + - stock_quantity: INTEGER DEFAULT 0 + - created_at: TIMESTAMP DEFAULT CURRENT_TIMESTAMP + ``` + +7. **List tables**: + ``` + List all tables in PostgreSQL database 'inventory' on server + 'bugbash-postgres-' + ``` + +8. **Get table schema**: + ``` + Show me the schema of table 'products' in database 'inventory' + ``` + +**Verify**: +- [ ] Database created +- [ ] Tables created with correct schema +- [ ] Columns have correct data types +- [ ] Constraints are applied + +#### Phase 3: Data Operations and Queries + +9. **Insert data**: + ``` + Insert the following records into the products table: + - Laptop, High-performance laptop, 1299.99, 25 + - Mouse, Wireless mouse, 29.99, 100 + - Keyboard, Mechanical keyboard, 89.99, 50 + ``` + +10. **Query all data**: + ``` + Show me all products in the inventory database + ``` + +11. **Query with filter**: + ``` + Show me all products with price less than 100 in the inventory database + ``` + +12. **Search query**: + ``` + Show me all items that contain the word 'laptop' in the products + table in PostgreSQL database 'inventory' + ``` + +**Verify**: +- [ ] Data inserted successfully +- [ ] Queries return correct results +- [ ] Filters work properly +- [ ] Text search works + +#### Phase 4: Server Configuration + +13. **Get server configuration**: + ``` + Show me the configuration of PostgreSQL server 'bugbash-postgres-' + ``` + +14. **Check server parameters**: + ``` + Show me if the PostgreSQL server 'bugbash-postgres-' has + replication enabled + ``` + +15. **Get connection information**: + ``` + Get the connection string for PostgreSQL server 'bugbash-postgres-' + database 'inventory' + ``` + +**Verify**: +- [ ] Configuration retrieved +- [ ] Parameters are accessible +- [ ] Connection string provided + +**Expected Results**: +- PostgreSQL server created +- Database and tables created +- Data operations work +- Queries execute correctly +- Configuration accessible + +--- + +### Scenario 3: Azure SQL Database Operations + +**Objective**: Test Azure SQL Database and server management + +#### Phase 1: Create SQL Server and Database + +1. **Create SQL server**: + ``` + Create an Azure SQL server: + - Name: 'bugbash-sqlserver-' + - Resource group: '' + - Admin login: 'sqladmin' + - Password: '' + - Location: East US + - Allow Azure services and resources to access this server + ``` + +2. **List SQL servers**: + ``` + List all Azure SQL servers in resource group '' + ``` + +3. **Get server details**: + ``` + Show me the details of Azure SQL server 'bugbash-sqlserver-' + in resource group '' + ``` + +**Verify**: +- [ ] Server created successfully +- [ ] Admin login configured +- [ ] Location is correct +- [ ] Firewall configured + +#### Phase 2: Create and Manage Databases + +4. **Create database**: + ``` + Create a SQL database named 'OrdersDB' on server + 'bugbash-sqlserver-' with Basic tier + ``` + +5. **List databases**: + ``` + List all databases in Azure SQL server 'bugbash-sqlserver-' + ``` + +6. **Show database details**: + ``` + Get the configuration details for SQL database 'OrdersDB' on + server 'bugbash-sqlserver-' + ``` + +7. **Create another database**: + ``` + Create a SQL database named 'TestDB' on server + 'bugbash-sqlserver-' with Standard tier (S1) + ``` + +**Verify**: +- [ ] Databases created with correct tiers +- [ ] Database properties are correct +- [ ] Databases are accessible + +#### Phase 3: Firewall Rules + +8. **List firewall rules**: + ``` + List all firewall rules for SQL server 'bugbash-sqlserver-' + ``` + +9. **Create firewall rule**: + ``` + Create a firewall rule for Azure SQL server 'bugbash-sqlserver-' + to allow access from my current IP address + ``` + +10. **Verify firewall rule**: + ``` + Show me the firewall rules for SQL server 'bugbash-sqlserver-' + ``` + +**Verify**: +- [ ] Firewall rules listed +- [ ] New rule created +- [ ] Rule parameters correct + +#### Phase 4: Database Configuration + +11. **Update database tier**: + ``` + Update the performance tier of SQL database 'OrdersDB' on server + 'bugbash-sqlserver-' to Standard S2 + ``` + +12. **Rename database**: + ``` + Rename the SQL database 'TestDB' to 'StagingDB' on server + 'bugbash-sqlserver-' + ``` + +13. **Check elastic pools**: + ``` + List all elastic pools in SQL server 'bugbash-sqlserver-' + ``` + +14. **List Entra ID administrators**: + ``` + List Microsoft Entra ID administrators for SQL server + 'bugbash-sqlserver-' + ``` + +**Verify**: +- [ ] Tier update successful +- [ ] Database renamed correctly +- [ ] Elastic pools listed (if any) +- [ ] Admin info retrieved + +#### Phase 5: Database Deletion + +15. **Delete test database**: + ``` + Delete the SQL database 'StagingDB' from server + 'bugbash-sqlserver-' + ``` + +16. **Verify deletion**: + ``` + List databases in server 'bugbash-sqlserver-' to confirm + 'StagingDB' is deleted + ``` + +**Verify**: +- [ ] Database deleted successfully +- [ ] Deletion confirmed + +**Expected Results**: +- SQL server created +- Multiple databases created +- Firewall rules managed +- Database configuration updated +- Database operations work correctly + +## Common Issues to Watch For + +- **Connection String Format**: Different formats for different database types +- **Authentication Failures**: Credential or permission issues +- **Firewall Blocking**: Access blocked by firewall rules +- **Partition Key Issues**: Cosmos DB partition key mismatches +- **Query Syntax**: Different SQL dialects across databases +- **Resource Limits**: RU limits (Cosmos), connection limits (SQL/PostgreSQL) +- **Timeout Issues**: Long-running queries +- **Case Sensitivity**: PostgreSQL vs SQL Server differences + +## Related Resources + +- [Azure Cosmos DB Documentation](https://learn.microsoft.com/azure/cosmos-db/) +- [Azure Database for PostgreSQL](https://learn.microsoft.com/azure/postgresql/) +- [Azure SQL Database](https://learn.microsoft.com/azure/azure-sql/) +- [Database Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md#azure-database-for-mysql) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +**Next**: [Deployment Scenarios Testing](deployment.md) diff --git a/docs/bug-bash/scenarios/deployment.md b/docs/bug-bash/scenarios/deployment.md new file mode 100644 index 0000000000..5d38ac540e --- /dev/null +++ b/docs/bug-bash/scenarios/deployment.md @@ -0,0 +1,324 @@ +# Deployment Scenarios Testing + +Test Azure MCP Server's deployment capabilities, CI/CD integration, and deployment management features. + +## Objectives + +- Test application deployment workflows +- Verify deployment guidance and automation +- Test multi-environment deployments +- Validate deployment monitoring +- Test rollback and recovery procedures +- Verify CI/CD integration guidance + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] Sample application ready for deployment +- [ ] GitHub Copilot with Agent mode enabled + +## Test Scenarios + +### Scenario 1: Simple Application Deployment + +**Objective**: Test basic deployment workflow for a simple application + +#### Phase 1: Get Deployment Guidance + +1. **Ask for deployment plan**: + ``` + I have a Node.js Express application. Create a plan to deploy this + application to Azure. + ``` + +2. **Verify deployment plan includes**: + - [ ] Resource requirements (App Service, Storage, etc.) + - [ ] Configuration steps + - [ ] Deployment method recommendations + - [ ] Post-deployment verification steps + +3. **Get Azure CLI deployment commands**: + ``` + Generate Azure CLI commands to deploy my Node.js app to App Service + ``` + +4. **Get infrastructure as code**: + ``` + Generate a Bicep template to deploy my application infrastructure + ``` + +**Verify**: +- [ ] Deployment plan is comprehensive +- [ ] CLI commands are correct +- [ ] Infrastructure code is valid +- [ ] Steps are in logical order + +#### Phase 2: Execute Deployment + +5. **Deploy application**: + ``` + Deploy my Node.js application to Azure App Service 'bugbash-app-' + in resource group '' + ``` + +6. **Monitor deployment**: + ``` + Show me the deployment status for app service 'bugbash-app-' + ``` + +7. **Verify deployment**: + ``` + Check if my app is running on 'bugbash-app-' + ``` + +8. **Get application URL**: + ``` + What is the URL for my deployed application? + ``` + +**Verify**: +- [ ] Deployment completes successfully +- [ ] Application is accessible +- [ ] URL is correct +- [ ] App responds to requests + +**Expected Results**: +- Deployment plan provided +- Infrastructure created +- Application deployed +- App is accessible and functional + +--- + +### Scenario 2: Multi-Environment Deployment + +**Objective**: Test deployment across development, staging, and production environments + +#### Phase 1: Create Environment Infrastructure + +1. **Create development environment**: + ``` + Create Azure resources for a development environment: + - Resource group: 'bugbash-dev-rg' + - App Service: 'bugbash-app-dev' + - SQL Database: 'bugbash-db-dev' (Basic tier) + - Storage account: 'bugbashdevstore' + - Region: East US + ``` + +2. **Create staging environment**: + ``` + Create Azure resources for a staging environment: + - Resource group: 'bugbash-staging-rg' + - App Service: 'bugbash-app-staging' + - SQL Database: 'bugbash-db-staging' (Standard tier) + - Storage account: 'bugbashstagingstore' + - Region: East US + ``` + +3. **Create production environment**: + ``` + Create Azure resources for a production environment: + - Resource group: 'bugbash-prod-rg' + - App Service: 'bugbash-app-prod' (Premium tier) + - SQL Database: 'bugbash-db-prod' (Standard tier with geo-replication) + - Storage account: 'bugbashprodstore' with redundancy + - Region: East US + ``` + +**Verify**: +- [ ] All three environments created +- [ ] Appropriate tiers for each environment +- [ ] Naming conventions consistent +- [ ] Resources properly tagged + +#### Phase 2: Deploy to Environments + +4. **Deploy to development**: + ``` + Deploy my application to the development environment + (bugbash-app-dev) with debug settings enabled + ``` + +5. **Deploy to staging**: + ``` + Deploy my application to the staging environment + (bugbash-app-staging) with production-like settings + ``` + +6. **Verify staging deployment**: + ``` + Test my application in the staging environment and verify all + features work correctly + ``` + +7. **Deploy to production**: + ``` + After successful staging tests, deploy my application to production + environment (bugbash-app-prod) + ``` + +**Verify**: +- [ ] Each environment has correct configuration +- [ ] Deployments succeed in order +- [ ] Environment-specific settings applied +- [ ] Production deployment after verification + +**Expected Results**: +- All environments created with appropriate configurations +- Deployments succeed to each environment +- Progressive deployment strategy followed +- Each environment is accessible and functional + +--- + +### Scenario 3: CI/CD Pipeline Guidance + +**Objective**: Test guidance for setting up continuous integration and deployment + +#### Phase 1: Pipeline Recommendations + +1. **Get CI/CD guidance**: + ``` + How can I create a CI/CD pipeline to deploy this application to Azure? + ``` + +2. **Get GitHub Actions workflow**: + ``` + Generate a GitHub Actions workflow to: + - Build my Node.js application + - Run tests + - Deploy to Azure App Service + - Deploy to staging first, then production after approval + ``` + +3. **Get Azure DevOps pipeline**: + ``` + Generate an Azure DevOps pipeline YAML for deploying my application + ``` + +**Verify**: +- [ ] CI/CD recommendations provided +- [ ] GitHub Actions workflow is valid +- [ ] Azure DevOps pipeline is complete +- [ ] Best practices included + +#### Phase 2: Pipeline Configuration + +4. **Configure deployment credentials**: + ``` + How do I configure deployment credentials for GitHub Actions to + deploy to Azure? + ``` + +5. **Set up secrets**: + ``` + What secrets do I need to configure for automated deployment to Azure? + ``` + +6. **Configure service principal**: + ``` + Help me create a service principal for automated deployments with + minimal required permissions + ``` + +**Verify**: +- [ ] Credential setup instructions clear +- [ ] Secret requirements documented +- [ ] Service principal guidance provided +- [ ] Security best practices mentioned + +**Expected Results**: +- Complete CI/CD guidance provided +- Pipeline templates generated +- Security configuration documented +- Ready to implement automated deployments + +### Scenario 4: Deployment Monitoring and Logging + +**Objective**: Test deployment monitoring and log analysis capabilities + +#### Phase 1: Monitor Deployment + +1. **Check deployment history**: + ``` + Show me the deployment history for app service 'bugbash-app-prod' + ``` + +2. **Get deployment logs**: + ``` + Show me the deployment logs for the latest deployment to + 'bugbash-app-prod' + ``` + +3. **Check application logs**: + ``` + Show me the application logs for 'bugbash-app-prod' from the last + 30 minutes + ``` + +**Verify**: +- [ ] Deployment history retrieved +- [ ] Logs are accessible +- [ ] Log content is useful +- [ ] Timestamps are accurate + +#### Phase 2: Troubleshoot Failed Deployment + +4. **Simulate deployment failure**: + - Intentionally cause a deployment to fail + - Could be wrong runtime, missing dependency, etc. + +5. **Diagnose failure**: + ``` + My deployment to 'bugbash-app-staging' failed. Help me diagnose + what went wrong. + ``` + +6. **Get logs for failed deployment**: + ``` + Show me the error logs for the failed deployment + ``` + +7. **Get fix recommendations**: + ``` + Based on the error logs, what should I do to fix the deployment? + ``` + +**Verify**: +- [ ] Failure is detected +- [ ] Error logs retrieved +- [ ] Root cause identified +- [ ] Fix suggestions provided + +**Expected Results**: +- Deployment monitoring works +- Logs are accessible +- Failures are diagnosed +- Actionable recommendations provided + +## 🐛 Common Issues to Watch For + +- **Deployment Timeouts**: Long-running deployments +- **Configuration Errors**: Wrong settings causing failures +- **Resource Dependencies**: Resources deployed in wrong order +- **Authentication Issues**: Credential problems during deployment +- **Network Connectivity**: Firewall or network issues +- **Resource Quotas**: Hitting subscription limits +- **Naming Conflicts**: Duplicate resource names +- **Version Mismatches**: Incompatible runtime versions + +## Related Resources + +- [Azure Deployment Center](https://learn.microsoft.com/azure/app-service/deploy-continuous-deployment) +- [GitHub Actions for Azure](https://learn.microsoft.com/azure/developer/github/github-actions) +- [Azure DevOps Pipelines](https://learn.microsoft.com/azure/devops/pipelines/) +- [Deploy Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md#azure-deploy) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +--- + +**Next**: [Full Stack Applications Testing](full-stack-apps.md) diff --git a/docs/bug-bash/scenarios/full-stack-apps.md b/docs/bug-bash/scenarios/full-stack-apps.md new file mode 100644 index 0000000000..420714b015 --- /dev/null +++ b/docs/bug-bash/scenarios/full-stack-apps.md @@ -0,0 +1,445 @@ +# Full Stack Application Testing Scenario + +Test Azure MCP Server's ability to help create complete applications with database backends and deploy them to Azure. + +## Objectives + +- Create a complete full-stack application +- Set up database backends (Cosmos DB, PostgreSQL, or Azure SQL) +- Deploy application to Azure +- Test CRUD operations against the database +- Verify end-to-end functionality + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] Development tools installed (Node.js, Python, or .NET SDK) +- [ ] GitHub Copilot with Agent mode enabled +- [ ] VS Code or preferred IDE + +## Test Scenarios + +### Scenario 1: Node.js Web App with Cosmos DB + +**Objective**: Create a complete Node.js application with Cosmos DB backend + +**Steps**: + +#### Phase 1: Create the Database + +1. **Ask Copilot to create a Cosmos DB account**: + ``` + Create a Cosmos DB account named 'bugbash-cosmos-' in resource + group '' in East US with SQL API + ``` + +2. **Create a database**: + ``` + Create a database named 'TasksDB' in Cosmos DB account 'bugbash-cosmos-' + ``` + +3. **Create a container**: + ``` + Create a container named 'Tasks' in database 'TasksDB' with + partition key '/category' and 400 RU/s throughput + ``` + +4. **Add sample data**: + ``` + Add the following sample tasks to the Tasks container: + - { "id": "1", "title": "Buy groceries", "category": "personal", "completed": false } + - { "id": "2", "title": "Review PR", "category": "work", "completed": false } + - { "id": "3", "title": "Call dentist", "category": "personal", "completed": true } + ``` + +5. **Verify data**: + ``` + Query all tasks from the Tasks container in Cosmos DB + ``` + +#### Phase 2: Build the Application + +6. **Ask Copilot to create the application structure**: + ``` + Create a Node.js Express application for a task management system with: + - Express.js web framework + - Cosmos DB integration using @azure/cosmos + - RESTful API endpoints for: + * GET /api/tasks - list all tasks + * GET /api/tasks/:id - get a specific task + * POST /api/tasks - create a new task + * PUT /api/tasks/:id - update a task + * DELETE /api/tasks/:id - delete a task + - A simple HTML frontend with JavaScript + - Connection to Cosmos DB 'bugbash-cosmos-' + ``` + +7. **Review the generated code**: + - [ ] Check package.json dependencies + - [ ] Verify Cosmos DB connection configuration + - [ ] Review API endpoint implementations + - [ ] Check error handling + - [ ] Verify frontend HTML/JS + +8. **Ask for configuration guidance**: + ``` + How do I configure the Cosmos DB connection string in my Node.js + application securely? + ``` + +#### Phase 3: Test Locally + +9. **Install dependencies**: + ```bash + npm install + ``` + +10. **Set up environment variables**: + ``` + Get the Cosmos DB connection string for 'bugbash-cosmos-' + ``` + - Copy the connection string + - Create `.env` file with connection string + +11. **Run the application locally**: + ```bash + npm start + ``` + +12. **Test the API endpoints**: + - [ ] GET /api/tasks returns all tasks + - [ ] GET /api/tasks/1 returns specific task + - [ ] POST /api/tasks creates new task + - [ ] PUT /api/tasks/1 updates task + - [ ] DELETE /api/tasks/1 deletes task + +13. **Test the frontend**: + - [ ] Open http://localhost:3000 + - [ ] View task list + - [ ] Create a new task + - [ ] Edit a task + - [ ] Delete a task + - [ ] Verify changes in Cosmos DB + +#### Phase 4: Deploy to Azure + +14. **Create App Service resources**: + ``` + Create an Azure App Service for my Node.js application: + - Name: 'bugbash-taskapp-' + - Resource group: '' + - Runtime: Node.js 20 LTS + - Tier: B1 + - Region: East US + ``` + +15. **Configure application settings**: + ``` + Add the Cosmos DB connection string as an application setting + named 'COSMOS_CONNECTION_STRING' in app service 'bugbash-taskapp-' + ``` + +16. **Deploy the application**: + ``` + How do I deploy my Node.js application to Azure App Service + 'bugbash-taskapp-'? + ``` + - Follow the deployment instructions provided + +17. **Verify deployment**: + ``` + What is the URL for app service 'bugbash-taskapp-'? + ``` + - Open the URL in browser + - Test all CRUD operations + +**Expected Results**: +- Cosmos DB account and container created +- Application code generated correctly +- Local testing successful +- Deployment to Azure successful +- Production app works correctly +- Database operations work in production + +--- + +### Scenario 2: Python Web App with PostgreSQL + +**Objective**: Create a Flask application with PostgreSQL backend + +**Steps**: + +#### Phase 1: Create PostgreSQL Database + +1. **Create PostgreSQL server**: + ``` + Create an Azure Database for PostgreSQL flexible server: + - Name: 'bugbash-postgres-' + - Resource group: '' + - Admin user: 'dbadmin' + - Password: '' + - Version: PostgreSQL 14 + - Tier: Burstable B1ms + - Region: East US + - Allow Azure services access + ``` + +2. **Create database**: + ``` + Create a database named 'inventory' on PostgreSQL server + 'bugbash-postgres-' + ``` + +3. **Create table schema**: + ``` + In PostgreSQL database 'inventory', create a table named 'products' with: + - id (serial primary key) + - name (varchar 100) + - description (text) + - price (decimal 10,2) + - stock (integer) + - created_at (timestamp) + ``` + +4. **Insert sample data**: + ``` + Insert sample products into the 'products' table: + - Laptop, High-performance laptop, 999.99, 10 + - Mouse, Wireless mouse, 29.99, 50 + - Keyboard, Mechanical keyboard, 79.99, 30 + ``` + +#### Phase 2: Build Python Application + +5. **Create Flask application**: + ``` + Create a Python Flask application for an inventory management system: + - Flask web framework + - psycopg2 for PostgreSQL connection + - API endpoints: + * GET /api/products - list all products + * GET /api/products/ - get product details + * POST /api/products - add new product + * PUT /api/products/ - update product + * DELETE /api/products/ - delete product + - Simple HTML frontend + - Connection to PostgreSQL 'bugbash-postgres-' + ``` + +6. **Review generated code**: + - [ ] Check requirements.txt + - [ ] Verify PostgreSQL connection + - [ ] Review API routes + - [ ] Check SQL queries + - [ ] Verify error handling + +#### Phase 3: Test Locally + +7. **Set up virtual environment**: + ```bash + python -m venv venv + source venv/bin/activate # On Windows: venv\Scripts\activate + pip install -r requirements.txt + ``` + +8. **Configure database connection**: + ``` + Get the PostgreSQL connection string for 'bugbash-postgres-' + ``` + - Create `.env` file with credentials + +9. **Run locally**: + ```bash + python app.py + ``` + +10. **Test endpoints**: + - [ ] List products + - [ ] Get product details + - [ ] Add new product + - [ ] Update product + - [ ] Delete product + +#### Phase 4: Deploy to Azure + +11. **Create App Service for Python**: + ``` + Create an Azure App Service for Python: + - Name: 'bugbash-inventory-' + - Runtime: Python 3.11 + - Tier: B1 + - Region: East US + ``` + +12. **Configure connection**: + ``` + Configure the PostgreSQL connection in app service settings + ``` + +13. **Deploy application**: + ``` + Deploy my Python Flask app to 'bugbash-inventory-' + ``` + +14. **Test production app**: + - Open app URL + - Verify all CRUD operations + +**Expected Results**: +- PostgreSQL database created +- Python application generated +- Local testing successful +- Azure deployment successful +- Production app functional + +--- + +### Scenario 3: .NET Web App with Azure SQL + +**Objective**: Create an ASP.NET Core application with Azure SQL backend + +**Steps**: + +#### Phase 1: Create Azure SQL Database + +1. **Create SQL server**: + ``` + Create an Azure SQL server: + - Name: 'bugbash-sqlserver-' + - Resource group: '' + - Admin login: 'sqladmin' + - Password: '' + - Region: East US + - Allow Azure services + ``` + +2. **Create database**: + ``` + Create a database named 'CustomerDB' on SQL server + 'bugbash-sqlserver-' with Basic tier + ``` + +3. **Create tables**: + ``` + Create the following tables in CustomerDB: + + Customers table: + - CustomerId (int primary key identity) + - FirstName (nvarchar 50) + - LastName (nvarchar 50) + - Email (nvarchar 100) + - Phone (nvarchar 20) + - CreatedDate (datetime) + + Orders table: + - OrderId (int primary key identity) + - CustomerId (int foreign key) + - OrderDate (datetime) + - TotalAmount (decimal 18,2) + - Status (nvarchar 20) + ``` + +#### Phase 2: Build .NET Application + +4. **Create ASP.NET Core Web API**: + ``` + Create an ASP.NET Core 8.0 Web API application for customer management: + - Entity Framework Core for database access + - Controllers for Customers and Orders + - API endpoints following REST conventions + - Connection to Azure SQL 'bugbash-sqlserver-' + - Swagger/OpenAPI documentation + - Include a simple React frontend + ``` + +5. **Review code structure**: + - [ ] Check Models (Customer, Order) + - [ ] Review DbContext configuration + - [ ] Verify Controllers + - [ ] Check API routes + - [ ] Review frontend React components + +#### Phase 3: Test Locally + +6. **Configure connection string**: + ``` + Get the connection string for Azure SQL database 'CustomerDB' + ``` + - Add to appsettings.Development.json + +7. **Run migrations**: + ```bash + dotnet ef database update + ``` + +8. **Start application**: + ```bash + dotnet run + ``` + +9. **Test API**: + - Open Swagger UI: https://localhost:5001/swagger + - [ ] Test GET /api/customers + - [ ] Test POST /api/customers + - [ ] Test GET /api/orders + - [ ] Test POST /api/orders + +#### Phase 4: Deploy to Azure + +10. **Create App Service**: + ``` + Create an Azure App Service for .NET 8.0: + - Name: 'bugbash-customerapi-' + - Runtime: .NET 8 + - Tier: B1 + ``` + +11. **Configure SQL connection**: + ``` + Add SQL connection string to app service configuration + ``` + +12. **Deploy application**: + ``` + Deploy my .NET application to Azure App Service + ``` + +13. **Verify deployment**: + - Test API endpoints in production + - Verify database connectivity + - Test frontend functionality + +**Expected Results**: +- Azure SQL database created +- .NET application generated +- EF Core migrations work +- Local testing successful +- Azure deployment successful +- Production API operational + +## Common Issues to Watch For + +- **Database Connection Issues**: Connection string format, firewall rules +- **Authentication/Authorization**: Missing database credentials +- **CORS Errors**: Frontend can't communicate with backend +- **Port Conflicts**: Port already in use locally +- **Dependency Issues**: Missing or incompatible packages +- **Environment Variables**: Not properly configured in production +- **Database Migrations**: Schema changes not applied +- **Build Failures**: Platform-specific compilation issues +- **Timeout Issues**: Long-running database operations +- **Connection Pool Exhaustion**: Too many concurrent connections + +## Related Resources + +- [Azure App Service Documentation](https://learn.microsoft.com/azure/app-service/) +- [Azure Cosmos DB Documentation](https://learn.microsoft.com/azure/cosmos-db/) +- [Azure PostgreSQL Documentation](https://learn.microsoft.com/azure/postgresql/) +- [Azure SQL Documentation](https://learn.microsoft.com/azure/azure-sql/) +- [Deployment Testing](deployment.md) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +**Next**: [Infra As Code](infra-as-code.md) \ No newline at end of file diff --git a/docs/bug-bash/scenarios/infra-as-code.md b/docs/bug-bash/scenarios/infra-as-code.md new file mode 100644 index 0000000000..ff3b122cca --- /dev/null +++ b/docs/bug-bash/scenarios/infra-as-code.md @@ -0,0 +1,171 @@ +# Infrastructure as Code Testing Scenario + +Test Azure MCP Server's ability to generate and deploy infrastructure using Bicep templates and Azure resources. + +## Objectives + +- Generate Bicep templates for Azure resources +- Deploy infrastructure using generated templates +- Validate deployed resources +- Test different resource types +- Verify error handling during deployment + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] Resource group for testing (or permission to create one) +- [ ] GitHub Copilot with Agent mode enabled + +## Test Scenarios + +### Scenario 1: Generate Simple Storage Account Bicep Template + +**Objective**: Test basic Bicep template generation + +**Steps**: + +1. Open GitHub Copilot Chat in Agent mode + +2. Use this prompt: + ``` + Generate a Bicep template to create an Azure Storage account named + 'bugbashstorage' in the 'eastus' region with Standard_LRS redundancy + ``` + +3. **Verify**: + - [ ] Bicep template is generated + - [ ] Template includes storage account resource + - [ ] Properties match requested configuration + - [ ] Template syntax is valid + +4. **Save the template** to `storage-account.bicep` + +5. **Deploy the template**: + ```bash + az deployment group create \ + --resource-group \ + --template-file storage-account.bicep + ``` + +6. **Verify deployment**: + - Ask Copilot: `"List my storage accounts in resource group "` + - Confirm the storage account appears + +**Expected Results**: +- Valid Bicep template generated +- Template deploys successfully +- Resource appears in Azure + +**Things to Check**: +- [ ] Template follows Bicep best practices +- [ ] Resource names are valid +- [ ] Parameters are properly defined +- [ ] Outputs are included (if applicable) + + +### Scenario 2: Generate Multi-Resource Infrastructure + +**Objective**: Test complex infrastructure generation + +**Steps**: + +1. Use this prompt: + ``` + Generate a Bicep template that creates: + - A storage account + - An App Service plan (B1 tier) + - A web app connected to the storage account + - An Application Insights resource + All resources should be in the East US region + ``` + +2. **Verify the template includes**: + - [ ] All requested resources + - [ ] Proper resource dependencies + - [ ] Connection strings/references + - [ ] Valid parameter definitions + +3. **Review the template** for: + - [ ] Naming conventions + - [ ] Resource locations + - [ ] SKU/tier selections + - [ ] Resource links + +4. **Deploy the template**: + ```bash + az deployment group create \ + --resource-group \ + --template-file multi-resource.bicep + ``` + +5. **Verify all resources** are created: + - Ask Copilot: `"List all resources in resource group "` + - Confirm all expected resources appear + +**Expected Results**: +- Complete Bicep template with all resources +- Proper dependencies between resources +- All resources deploy successfully +- Resources are correctly connected + +### Scenario 3: Test Bicep Best Practices + +**Objective**: Verify generated templates follow Azure best practices + +**Steps**: + +1. Use this prompt: + ``` + Show me the best practices for creating a Bicep template for a + storage account with blob containers + ``` + +2. **Review the guidance** provided + +3. Generate a template with best practices: + ``` + Generate a Bicep template for a storage account following best + practices, including: + - Parameters for names and locations + - Variables for common values + - Outputs for important values + - Proper resource dependencies + ``` + +4. **Verify the template includes**: + - [ ] Parameterized values + - [ ] Variables for reusable values + - [ ] Descriptive resource descriptions + - [ ] Outputs for key information + - [ ] Tags for resource management + +**Expected Results**: +- Template follows Azure naming conventions +- Sensitive values are parameterized +- Template is reusable across environments +- Outputs provide useful information + +## Common Issues to Watch For + +- **Naming Conflicts**: Resources with duplicate names +- **Region Availability**: Services not available in requested regions +- **Quota Limits**: Exceeding subscription quotas +- **Dependency Ordering**: Resources deployed in wrong order +- **Authentication Issues**: Permission problems during deployment +- **Parameter Validation**: Invalid parameter values +- **Syntax Errors**: Invalid Bicep syntax +- **Version Compatibility**: API version mismatches + +## Related Resources + +- [Azure Bicep Documentation](https://learn.microsoft.com/azure/azure-resource-manager/bicep/) +- [Azure Best Practices Tool](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md#azure-mcp-best-practices) +- [Troubleshooting Deployments](troubleshooting.md) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +--- + +**Next**: [PaaS Services Testing](paas-services.md) diff --git a/docs/bug-bash/scenarios/paas-services.md b/docs/bug-bash/scenarios/paas-services.md new file mode 100644 index 0000000000..5abd437da7 --- /dev/null +++ b/docs/bug-bash/scenarios/paas-services.md @@ -0,0 +1,378 @@ +# PaaS Services Testing Scenario + +Test Azure MCP Server's capabilities with Platform-as-a-Service offerings including App Service, Container Apps, and Azure Functions. + +## Objectives + +- Deploy and manage Azure App Service web apps +- Work with Azure Container Apps +- Create and manage Azure Functions +- Test configuration and scaling +- Verify integration with other services +- Test deployment workflows + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] Sample application code (optional) +- [ ] GitHub Copilot with Agent mode enabled + +## Test Scenarios + +### Scenario 1: Azure App Service + +**Objective**: Test web app creation, configuration, and deployment + +#### Phase 1: Create App Service Resources + +1. **Create App Service Plan**: + ``` + Create an Azure App Service Plan: + - Name: 'bugbash-plan-' + - Resource group: '' + - SKU: B1 (Basic) + - Region: East US + - OS: Linux + ``` + +2. **Verify plan creation**: + ``` + List all App Service plans in resource group '' + ``` + +3. **Create web app**: + ``` + Create a web app: + - Name: 'bugbash-webapp-' + - Resource group: '' + - App Service Plan: 'bugbash-plan-' + - Runtime: Node.js 20 LTS + ``` + +4. **List web apps**: + ``` + List all web apps in my subscription + ``` + +5. **Get web app details**: + ``` + Show me the details for web app 'bugbash-webapp-' in + resource group '' + ``` + +**Verify**: +- [ ] App Service Plan created with correct SKU +- [ ] Web app created successfully +- [ ] Runtime configured correctly +- [ ] Web app URL is accessible + +#### Phase 2: Configure Web App + +6. **Get configuration**: + ``` + Get the configuration for web app 'bugbash-webapp-' + ``` + +7. **Add application settings**: + ``` + Add the following application settings to web app 'bugbash-webapp-': + - NODE_ENV: production + - API_URL: https://api.example.com + - LOG_LEVEL: info + ``` + +8. **Add connection strings**: + ``` + Add a connection string named 'Database' with value + '' of type SQLAzure to web app 'bugbash-webapp-' + ``` + +9. **Enable Application Insights**: + ``` + Enable Application Insights for web app 'bugbash-webapp-' + ``` + +**Verify**: +- [ ] Application settings added +- [ ] Connection strings configured +- [ ] Application Insights enabled +- [ ] Configuration changes applied + +#### Phase 3: Database Integration + +10. **Add database connection**: + ``` + Add a SQL Server database connection to app service 'bugbash-webapp-': + - Database name: '' + - Server: '' + ``` + +11. **Configure PostgreSQL**: + ``` + Add a PostgreSQL database to app service 'bugbash-webapp-' + ``` + +12. **Verify database connections**: + ``` + Show me all database connections for web app 'bugbash-webapp-' + ``` + +**Verify**: +- [ ] Database connections configured +- [ ] Connection strings created +- [ ] Databases accessible from app + +#### Phase 4: Deployment and Testing + +13. **Get deployment information**: + ``` + How do I deploy my Node.js application to web app 'bugbash-webapp-'? + ``` + +14. **Check web app status**: + ``` + What is the status of web app 'bugbash-webapp-'? + ``` + +15. **Get web app URL**: + ``` + What is the URL for web app 'bugbash-webapp-'? + ``` + +16. **Test accessibility**: + - Open the web app URL in browser + - Verify the app is running + - Check if default page loads + +**Verify**: +- [ ] Deployment instructions provided +- [ ] Web app is running +- [ ] URL is accessible +- [ ] App responds to requests + +**Expected Results**: +- App Service Plan created +- Web app created and configured +- Database connections working +- Application is deployable +- Web app is accessible + +--- + +### Scenario 2: Azure Container Apps + +**Objective**: Test container deployment and management + +#### Phase 1: Create Container Apps Environment + +1. **Create Container Apps Environment**: + ``` + Create an Azure Container Apps Environment: + - Name: 'bugbash-containerenv-' + - Resource group: '' + - Location: East US + ``` + +2. **List Container Apps Environments**: + ``` + List all Container Apps environments in my subscription + ``` + +3. **Get environment details**: + ``` + Show me details of Container Apps environment 'bugbash-containerenv-' + ``` + +**Verify**: +- [ ] Environment created successfully +- [ ] Environment is in running state +- [ ] Configuration is correct + +#### Phase 2: Deploy Container App + +4. **Create container app**: + ``` + Create a container app in environment 'bugbash-containerenv-': + - Name: 'bugbash-container-' + - Container image: nginx:latest + - Ingress: enabled, external + - Target port: 80 + ``` + +5. **List container apps**: + ``` + List all container apps in resource group '' + ``` + +6. **Get container app details**: + ``` + Show me details of container app 'bugbash-container-' + ``` + +**Verify**: +- [ ] Container app created +- [ ] Container is running +- [ ] Ingress configured correctly +- [ ] App is accessible externally + +#### Phase 3: Configuration and Scaling + +7. **Configure environment variables**: + ``` + Add environment variables to container app 'bugbash-container-': + - ENVIRONMENT: production + - DEBUG: false + ``` + +8. **Configure scaling**: + ``` + Configure scaling for container app 'bugbash-container-': + - Min replicas: 1 + - Max replicas: 5 + ``` + +9. **Get container app URL**: + ``` + What is the URL for container app 'bugbash-container-'? + ``` + +**Verify**: +- [ ] Environment variables set +- [ ] Scaling configured +- [ ] URL provided and accessible + +**Expected Results**: +- Container Apps environment created +- Container app deployed +- Configuration applied +- Scaling works +- App is accessible + +--- + +### Scenario 3: Azure Functions + +**Objective**: Test Function App creation and management + +#### Phase 1: Create Function App + +1. **Create storage account for functions**: + ``` + Create a storage account 'bugbashfunc' for Azure Functions + in resource group '' + ``` + +2. **Create Function App**: + ``` + Create an Azure Function App: + - Name: 'bugbash-funcapp-' + - Resource group: '' + - Runtime: Node.js 20 + - Plan: Consumption (serverless) + - Storage account: 'bugbashfunc' + - Region: East US + ``` + +3. **List Function Apps**: + ``` + List all function apps in my subscription + ``` + +4. **Get Function App details**: + ``` + Show me details for function app 'bugbash-funcapp-' in + resource group '' + ``` + +**Verify**: +- [ ] Storage account created +- [ ] Function App created +- [ ] Runtime configured correctly +- [ ] Consumption plan assigned + +#### Phase 2: Configure Function App + +5. **Get Function App configuration**: + ``` + Get configuration for function app 'bugbash-funcapp-' + ``` + +6. **Add application settings**: + ``` + Add application settings to function app 'bugbash-funcapp-': + - ENVIRONMENT: production + - API_KEY: + ``` + +7. **Enable Application Insights**: + ``` + Enable Application Insights for function app 'bugbash-funcapp-' + ``` + +8. **Get function app keys**: + ``` + Get the host keys for function app 'bugbash-funcapp-' + ``` + +**Verify**: +- [ ] Configuration retrieved +- [ ] Application settings added +- [ ] Application Insights enabled +- [ ] Host keys accessible + +#### Phase 3: Deployment Information + +9. **Get deployment instructions**: + ``` + How do I deploy my Node.js functions to function app + 'bugbash-funcapp-'? + ``` + +10. **Get function app URL**: + ``` + What is the URL for function app 'bugbash-funcapp-'? + ``` + +11. **Check function app status**: + ``` + What is the status of function app 'bugbash-funcapp-'? + ``` + +**Verify**: +- [ ] Deployment guidance provided +- [ ] URL retrieved +- [ ] Status is running + +**Expected Results**: +- Function App created +- Configuration applied +- Application Insights enabled +- Deployment guidance provided +- App is operational + + +## Common Issues to Watch For + +- **Naming Conflicts**: App names must be globally unique +- **Plan Compatibility**: Some features require specific plan tiers +- **Runtime Mismatches**: Ensure runtime version compatibility +- **Storage Dependencies**: Function Apps require storage accounts +- **Port Configuration**: Container Apps need correct port mappings +- **Memory Limits**: Consumption plan has memory constraints +- **Cold Starts**: Serverless functions have startup latency +- **CORS Issues**: Cross-origin requests may need configuration + +## Related Resources + +- [Azure App Service Documentation](https://learn.microsoft.com/azure/app-service/) +- [Azure Container Apps Documentation](https://learn.microsoft.com/azure/container-apps/) +- [Azure Functions Documentation](https://learn.microsoft.com/azure/azure-functions/) +- [PaaS Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md#azure-app-service) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +--- + +**Next**: [Storage Operations Testing](storage-operations.md) diff --git a/docs/bug-bash/scenarios/storage-operations.md b/docs/bug-bash/scenarios/storage-operations.md new file mode 100644 index 0000000000..777495ad72 --- /dev/null +++ b/docs/bug-bash/scenarios/storage-operations.md @@ -0,0 +1,299 @@ +# Storage Operations Testing Scenario + +Test Azure MCP Server's storage account and blob operations capabilities. + +## Objectives + +- Create and manage storage accounts +- Work with blob containers +- Upload and download files +- Test storage permissions +- Verify error handling + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] Test files ready for upload +- [ ] GitHub Copilot with Agent mode enabled + +## Test Scenarios + +### Scenario 1: Create a Storage Account + +**Objective**: Test basic storage account creation + +**Steps**: + +1. Open GitHub Copilot Chat in Agent mode + +2. Use this prompt: + ``` + Create a new storage account named 'bugbashstorage' in + resource group '' in East US with: + - Standard_LRS redundancy + - Hot access tier + - Enable hierarchical namespace for Data Lake + ``` + +3. **Monitor the creation**: + - Watch for progress updates + - Note any warnings or errors + - Record creation time: _____ seconds + +4. **Verify the storage account**: + ``` + Show me the details of storage account 'bugbashstorage' + ``` + +5. **Check the properties**: + - [ ] Account name is correct + - [ ] Location is East US + - [ ] Redundancy is Standard_LRS + - [ ] Access tier is Hot + - [ ] Hierarchical namespace is enabled + +**Expected Results**: +- Storage account created successfully +- All properties match specifications +- Account is accessible + +### Scenario 2: Manage Blob Containers + +**Objective**: Test container creation and management + +**Steps**: + +1. **Create a container**: + ``` + Create a blob container named 'test-data' in storage account + 'bugbashstorage' with private access + ``` + +2. **Verify container creation**: + ``` + List all containers in storage account 'bugbashstorage' + ``` + +3. **Check container properties**: + - [ ] Container name is correct + - [ ] Public access level is None (private) + - [ ] Container is empty + +4. **Create additional containers** with different access levels: + ``` + Create containers: + - 'public-blobs' with blob public access + - 'public-container' with container public access + - 'private-data' with no public access + ``` + +5. **List and verify all containers**: + ``` + Show me all containers in 'bugbashstorage' with their access levels + ``` + +**Expected Results**: +- All containers created +- Access levels set correctly +- Containers are accessible + +--- + +### Scenario 3: Upload Files to Blob Storage + +**Objective**: Test file upload capabilities + +**Steps**: + +1. **Prepare test files**: + - Create a small text file (< 1 MB): `test-small.txt` + - Create a medium file (10-50 MB): `test-medium.zip` + - Create a large file (> 100 MB): `test-large.iso` (optional) + +2. **Upload small file**: + ``` + Upload the file 'test-small.txt' to container 'test-data' in + storage account 'bugbashstorage' + ``` + +3. **Verify upload**: + ``` + List all blobs in container 'test-data' + ``` + +4. **Check blob properties**: + - [ ] File name is correct + - [ ] File size matches original + - [ ] Content type is detected + - [ ] Upload time is recent + +5. **Upload medium file** and measure performance: + ``` + Upload 'test-medium.zip' to 'test-data' container + ``` + - Upload time: _____ seconds + - Upload speed: _____ MB/s + +6. **Upload file with custom metadata**: + ``` + Upload 'test-small.txt' as 'document.txt' to 'test-data' with + content type 'text/plain' and metadata 'author=BugBash' + ``` + +7. **Verify metadata**: + ``` + Show me the properties of blob 'document.txt' in container 'test-data' + ``` + +**Expected Results**: +- All files upload successfully +- File integrity is maintained +- Metadata is stored correctly +- Performance is reasonable + +--- + +### Scenario 4: Download Files from Blob Storage + +**Objective**: Test file download capabilities + +**Steps**: + +1. **Download a file**: + ``` + Download the blob 'test-small.txt' from container 'test-data' + in storage account 'bugbashstorage' to 'downloaded-file.txt' + ``` + +2. **Verify download**: + - [ ] File is downloaded + - [ ] File size matches original + - [ ] Content is identical + - Compare checksums if possible + +3. **Download multiple files**: + ``` + Download all blobs from container 'test-data' to folder 'downloads' + ``` + +4. **Verify all files**: + - [ ] All blobs are downloaded + - [ ] Folder structure is correct + - [ ] No corruption + +**Expected Results**: +- Files download successfully +- Content integrity is preserved +- Reasonable download speeds + +--- + +### Scenario 5: Storage Account Configuration + +**Objective**: Test storage account settings and features + +**Steps**: + +1. **List all storage accounts**: + ``` + Show me all my storage accounts in subscription '' + ``` + +2. **Filter by resource group**: + ``` + List storage accounts in resource group '' + ``` + +3. **Get account properties**: + ``` + Show me detailed information for storage account 'bugbashstorage' + including: + - SKU and redundancy + - Access tier + - Network rules + - Encryption settings + - HTTPS enforcement + ``` + +4. **Check account keys** (if permissions allow): + ``` + Show me the access keys for storage account 'bugbashstorage' + ``` + +5. **Verify security settings**: + - [ ] HTTPS-only traffic is enforced + - [ ] Minimum TLS version is set + - [ ] Secure transfer is enabled + - [ ] Public access is configured correctly + +**Expected Results**: +- Account details are accurate +- Security settings are appropriate +- Configuration matches expectations + +--- + +### Scenario 6: Test Different Storage Types + +**Objective**: Test various Azure Storage services + +#### 6a. Table Storage + +**Steps**: +``` +Does storage account 'bugbashstorage' support Azure Table Storage? +If yes, show me how to work with it. +``` + +**Verify**: +- [ ] Table storage capability confirmed +- [ ] Guidance provided for usage + +#### 6b. Queue Storage + +**Steps**: +``` +Does storage account 'bugbashstorage' support Azure Queue Storage? +If yes, show me how to create and manage queues. +``` + +**Verify**: +- [ ] Queue storage capability confirmed +- [ ] Instructions for queue operations + +#### 6c. File Shares + +**Steps**: +``` +Does storage account 'bugbashstorage' support Azure File Shares? +If yes, show me how to create and mount a file share. +``` + +**Verify**: +- [ ] File share capability confirmed +- [ ] Mount instructions provided + +## Common Issues to Watch For + +- **Authentication Failures**: Credential or permission issues +- **Network Timeouts**: Slow or failed uploads/downloads +- **Name Conflicts**: Duplicate blob or container names +- **Path Issues**: Platform-specific path separator problems +- **File Encoding**: Text file encoding issues across platforms +- **Large File Handling**: Memory issues with big files +- **Concurrent Operations**: Race conditions with parallel uploads +- **Permission Errors**: RBAC or access key issues + + +## Related Resources + +- [Azure Storage Documentation](https://learn.microsoft.com/azure/storage/) +- [Azure Blob Storage Best Practices](https://learn.microsoft.com/azure/storage/blobs/storage-blobs-introduction) +- [Data Lake Storage Gen2](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-introduction) +- [Storage Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md#azure-storage) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +**Next**: [Agent Building](agent-building.md) From 1abcc8d8a06047ab9b56160eb421d71a75d748f4 Mon Sep 17 00:00:00 2001 From: Vinay Gera Date: Wed, 15 Oct 2025 00:23:19 -0700 Subject: [PATCH 02/10] update bug bash docs --- docs/bug-bash/README.md | 14 +- .../bug-bash/scenarios/database-operations.md | 155 +++++++++--------- docs/bug-bash/scenarios/deployment.md | 4 +- docs/bug-bash/scenarios/full-stack-apps.md | 4 +- docs/bug-bash/scenarios/paas-services.md | 4 +- 5 files changed, 93 insertions(+), 88 deletions(-) diff --git a/docs/bug-bash/README.md b/docs/bug-bash/README.md index 394e114075..f8dc2bdcb2 100644 --- a/docs/bug-bash/README.md +++ b/docs/bug-bash/README.md @@ -15,10 +15,15 @@ Welcome to the Azure MCP Server Bug Bash! We're excited to have you help us impr The Azure MCP Server enables AI agents to interact with Azure services through natural language commands. As we continue to enhance the server, we need your help to identify issues across different platforms, IDEs, and usage scenarios. +> **🎯 Important**: Please read [TESTING-SCOPE.md](TESTING-SCOPE.md) to understand what Azure MCP Server can do. **MCP tools primarily focus on reading, querying, and inspecting Azure resources**. Most resource creation requires Azure CLI - use it to set up test resources before testing MCP's discovery and inspection capabilities. + This bug bash focuses on: - **Multi-platform compatibility** (Windows, macOS, Linux) - **Installation and setup** across different IDEs -- **Performance and stability** under real-world usage +- **Resource discovery and inspection** - testing how well MCP finds and retrieves information +- **Querying capabilities** - testing database queries and data retrieval +- **Monitoring and diagnostics** - testing log/metric access and health checks +- **Deployment guidance** - testing how MCP helps plan and guide deployments - **Authentication** across different environments - **End-to-end scenarios** that developers commonly encounter @@ -124,7 +129,12 @@ If you're short on time, try these quick scenarios: ## Resources -### Documentation +### Bug Bash Documentation +- **[TESTING-SCOPE.md](TESTING-SCOPE.md)** - ⚠️ **READ THIS FIRST!** - Explains what MCP can and cannot do +- [Installation Testing Guide](installation-testing.md) - Test installation across platforms and IDEs +- [Testing Scenarios](scenarios/) - Detailed end-to-end testing scenarios + +### Azure MCP Server Documentation - [Azure MCP Server Documentation](https://learn.microsoft.com/azure/developer/azure-mcp-server/) - [Installation Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/README.md#installation) - [Troubleshooting Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/TROUBLESHOOTING.md) diff --git a/docs/bug-bash/scenarios/database-operations.md b/docs/bug-bash/scenarios/database-operations.md index 8bd5f8a4fd..3ad345e890 100644 --- a/docs/bug-bash/scenarios/database-operations.md +++ b/docs/bug-bash/scenarios/database-operations.md @@ -1,126 +1,117 @@ -# Database Operations Testing Scenario +# 🗄️ Database Operations Testing Scenario -Test Azure MCP Server's database management capabilities across Cosmos DB, PostgreSQL, and Azure SQL. +> **⚠️ READ FIRST**: [TESTING-SCOPE.md](../TESTING-SCOPE.md) explains what MCP tools can and cannot do. **Azure MCP Server focuses on querying and inspecting databases, NOT creating them**. Use Azure CLI to create test databases before running these scenarios. -## Objectives +Test Azure MCP Server's database querying and inspection capabilities across Cosmos DB, PostgreSQL, MySQL, and Azure SQL. -- Create and manage Cosmos DB accounts and containers -- Work with PostgreSQL flexible servers -- Manage Azure SQL databases -- Test query execution and data operations -- Verify connection string management -- Test performance and scaling +## 🎯 Objectives -## Prerequisites +- List and inspect Cosmos DB accounts, databases, and containers +- Query PostgreSQL and MySQL databases +- List and inspect Azure SQL servers and databases +- Test query execution and data retrieval +- Verify table schema inspection +- Test database configuration viewing + +## ✅ Prerequisites - [ ] Azure MCP Server installed and configured -- [ ] Azure CLI installed (`az --version`) +- [ ] **Existing databases already deployed** (use Azure CLI or Portal to create test databases before testing) +- [ ] Azure CLI installed (`az --version`) for database creation - [ ] Authenticated to Azure (`az login`) -- [ ] Active Azure subscription +- [ ] Active Azure subscription with existing database resources - [ ] GitHub Copilot with Agent mode enabled +> **Note**: Azure MCP Server tools focus on **reading and querying** existing databases. Use Azure CLI commands (`az cosmosdb create`, `az postgres flexible-server create`, `az sql server create`) to create database resources before testing these scenarios. + ## Test Scenarios ### Scenario 1: Cosmos DB Operations -**Objective**: Test Cosmos DB account, database, and container management +**Objective**: Test Cosmos DB account listing, database inspection, and querying -#### Phase 1: Create Cosmos DB Resources +> **Setup Required**: Use Azure CLI to create a test Cosmos DB account before this scenario: +> ```bash +> az cosmosdb create --name bugbash-cosmos-$RANDOM --resource-group --locations regionName=eastus +> az cosmosdb sql database create --account-name --name ProductCatalog --resource-group +> az cosmosdb sql container create --account-name --database-name ProductCatalog --name Products --partition-key-path "/category" --resource-group +> ``` -1. **Create Cosmos DB account**: - ``` - Create a Cosmos DB account named 'bugbash-cosmos-' in - resource group '' in East US with SQL API - ``` +#### Phase 1: List Cosmos DB Resources -2. **Verify account creation**: +1. **List all Cosmos DB accounts** (uses `azmcp_cosmos_account_list`): ``` List all Cosmos DB accounts in my subscription ``` -3. **Check account details**: +2. **Alternative phrasing**: ``` - Show me the details of Cosmos DB account 'bugbash-cosmos-' + Show me my Cosmos DB accounts + ``` + ``` + Show me the Cosmos DB accounts in my subscription ``` **Verify**: -- [ ] Account created successfully -- [ ] API type is SQL (Core) -- [ ] Location is correct -- [ ] Account is accessible +- [ ] Tool correctly invoked: `azmcp_cosmos_account_list` +- [ ] Account list displayed +- [ ] Account properties shown (name, location, capabilities) -#### Phase 2: Database and Container Management +#### Phase 2: Inspect Databases and Containers -4. **Create database**: +3. **List databases** (uses `azmcp_cosmos_database_list`): ``` - Create a database named 'ProductCatalog' in Cosmos DB account - 'bugbash-cosmos-' + List all databases in Cosmos DB account '' ``` -5. **List databases**: +4. **Alternative phrasing**: ``` - List all databases in Cosmos DB account 'bugbash-cosmos-' + Show me the databases in Cosmos DB account '' ``` -6. **Create container**: +**Verify**: +- [ ] Tool correctly invoked: `azmcp_cosmos_database_list` +- [ ] Database list displayed +- [ ] Database properties shown + +5. **List containers** (uses `azmcp_cosmos_database_container_list`): ``` - Create a container named 'Products' in database 'ProductCatalog' - with partition key '/category' and 400 RU/s throughput + List all containers in database 'ProductCatalog' for Cosmos DB account '' ``` -7. **List containers**: +6. **Alternative phrasing**: ``` - List all containers in database 'ProductCatalog' + Show me the containers in database 'ProductCatalog' for Cosmos DB account '' ``` **Verify**: -- [ ] Database created successfully -- [ ] Container created with correct partition key -- [ ] Throughput set correctly -- [ ] Resources are listed properly - -#### Phase 3: Data Operations - -8. **Insert sample data**: - ``` - Add the following items to the Products container in Cosmos DB - 'bugbash-cosmos-', database 'ProductCatalog': - - Item 1: - { - "id": "1", - "name": "Laptop", - "category": "Electronics", - "price": 999.99, - "stock": 50 - } - - Item 2: - { - "id": "2", - "name": "Desk Chair", - "category": "Furniture", - "price": 199.99, - "stock": 100 - } - ``` - -9. **Query data**: - ``` - Query all items from the Products container where category is 'Electronics' - ``` - -10. **Query with search**: - ``` - Show me all items that contain the word 'Laptop' in the Products - container in database 'ProductCatalog' - ``` +- [ ] Tool correctly invoked: `azmcp_cosmos_database_container_list` +- [ ] Container list displayed +- [ ] Container properties shown (partition key, throughput) + +#### Phase 3: Query Data + +> **Data Setup**: Use Azure Portal or Azure CLI to insert sample data into the Products container before querying. + +7. **Query items** (uses `azmcp_cosmos_database_container_item_query`): + ``` + Show me the items that contain the word 'Laptop' in container 'Products' + in database 'ProductCatalog' for Cosmos DB account '' + ``` + +8. **Alternative query patterns**: + ``` + Query all items from container 'Products' in database 'ProductCatalog' + ``` + ``` + Show me items in the Products container where category is 'Electronics' + ``` **Verify**: -- [ ] Data inserted successfully -- [ ] Queries return correct results -- [ ] Partition key filtering works -- [ ] Search functionality works +- [ ] Tool correctly invoked: `azmcp_cosmos_database_container_item_query` +- [ ] Query results displayed +- [ ] Query syntax accepted +- [ ] Results match expected data **Expected Results**: - Cosmos DB account created diff --git a/docs/bug-bash/scenarios/deployment.md b/docs/bug-bash/scenarios/deployment.md index 5d38ac540e..b2b2653472 100644 --- a/docs/bug-bash/scenarios/deployment.md +++ b/docs/bug-bash/scenarios/deployment.md @@ -1,6 +1,6 @@ -# Deployment Scenarios Testing +# 🚢 Deployment Scenarios Testing -Test Azure MCP Server's deployment capabilities, CI/CD integration, and deployment management features. +Test Azure MCP Server's deployment guidance capabilities, CI/CD integration recommendations, and deployment planning features. ## Objectives diff --git a/docs/bug-bash/scenarios/full-stack-apps.md b/docs/bug-bash/scenarios/full-stack-apps.md index 420714b015..245ba22df0 100644 --- a/docs/bug-bash/scenarios/full-stack-apps.md +++ b/docs/bug-bash/scenarios/full-stack-apps.md @@ -1,4 +1,6 @@ -# Full Stack Application Testing Scenario +# 🏗️ Full Stack Application Testing Scenario + +> **⚠️ READ FIRST**: [TESTING-SCOPE.md](../TESTING-SCOPE.md) explains what MCP tools can and cannot do. **This scenario requires BOTH Azure CLI and MCP tools**. Use Azure CLI to create databases and app services, then use MCP tools to query databases, get deployment guidance, add database connections, and monitor applications. Test Azure MCP Server's ability to help create complete applications with database backends and deploy them to Azure. diff --git a/docs/bug-bash/scenarios/paas-services.md b/docs/bug-bash/scenarios/paas-services.md index 5abd437da7..3579cdec12 100644 --- a/docs/bug-bash/scenarios/paas-services.md +++ b/docs/bug-bash/scenarios/paas-services.md @@ -1,4 +1,6 @@ -# PaaS Services Testing Scenario +# 🚀 PaaS Services Testing Scenario + +> **⚠️ READ FIRST**: [TESTING-SCOPE.md](../TESTING-SCOPE.md) explains what MCP tools can and cannot do. **Azure MCP Server has LIMITED support for PaaS services** - primarily Function App inspection and App Service database connections. Use Azure CLI to create App Services, Container Apps, and Function Apps before testing. Test Azure MCP Server's capabilities with Platform-as-a-Service offerings including App Service, Container Apps, and Azure Functions. From 2db716cc961e098e19401a10703fae234e4c75f5 Mon Sep 17 00:00:00 2001 From: Vinay Gera Date: Wed, 15 Oct 2025 10:38:46 -0700 Subject: [PATCH 03/10] refactor docs --- docs/bug-bash/README.md | 2 +- docs/bug-bash/installation-testing.md | 392 +++++++- docs/bug-bash/scenarios/agent-building.md | 492 ++++++---- .../bug-bash/scenarios/database-operations.md | 687 +++++++------- docs/bug-bash/scenarios/deployment.md | 580 +++++++----- docs/bug-bash/scenarios/full-stack-apps.md | 857 +++++++++--------- docs/bug-bash/scenarios/infra-as-code.md | 468 +++++++--- docs/bug-bash/scenarios/paas-services.md | 560 +++++------- docs/bug-bash/scenarios/storage-operations.md | 539 ++++++----- 9 files changed, 2733 insertions(+), 1844 deletions(-) diff --git a/docs/bug-bash/README.md b/docs/bug-bash/README.md index f8dc2bdcb2..00f0d2c073 100644 --- a/docs/bug-bash/README.md +++ b/docs/bug-bash/README.md @@ -15,7 +15,7 @@ Welcome to the Azure MCP Server Bug Bash! We're excited to have you help us impr The Azure MCP Server enables AI agents to interact with Azure services through natural language commands. As we continue to enhance the server, we need your help to identify issues across different platforms, IDEs, and usage scenarios. -> **🎯 Important**: Please read [TESTING-SCOPE.md](TESTING-SCOPE.md) to understand what Azure MCP Server can do. **MCP tools primarily focus on reading, querying, and inspecting Azure resources**. Most resource creation requires Azure CLI - use it to set up test resources before testing MCP's discovery and inspection capabilities. +> **🎯 Important**: **MCP tools primarily focus on reading, querying, and inspecting Azure resources**. Most resource creation requires Azure CLI - use it to set up test resources before testing MCP's discovery and inspection capabilities. This bug bash focuses on: - **Multi-platform compatibility** (Windows, macOS, Linux) diff --git a/docs/bug-bash/installation-testing.md b/docs/bug-bash/installation-testing.md index cc48088e05..c58a4e44e2 100644 --- a/docs/bug-bash/installation-testing.md +++ b/docs/bug-bash/installation-testing.md @@ -304,6 +304,396 @@ azmcp server start --help --- +--- + +--- + +## UI-Based Mode Toggling Testing (Distribution-Specific) + +Different distribution mechanisms provide different ways to configure server modes. Test the UI controls for each distribution method. + +### Distribution Method 1: VS Code Extension + +**Objective**: Test VS Code Settings UI for mode toggling + +#### Available Settings: + +1. **Azure Mcp: Enabled Services** + - Location: Settings → Extensions → Azure MCP + - Purpose: Filter which Azure service namespaces are exposed + - Type: Multi-select list + - Options: storage, keyvault, cosmos, sql, etc. + +2. **Azure Mcp: Read Only** + - Location: Settings → Extensions → Azure MCP + - Purpose: Enable read-only mode (blocks write operations) + - Type: Checkbox + - Default: Unchecked (read-write mode) + +3. **Azure Mcp: Server Mode** + - Location: Settings → Extensions → Azure MCP + - Purpose: Choose tool exposure strategy + - Type: Dropdown + - Options: `single`, `namespace` (default), `all` + +#### Test 1: Change Server Mode via UI + +**Steps:** +1. Open VS Code Settings (Ctrl+, or Cmd+,) +2. Search for "Azure MCP" +3. Locate "Azure Mcp: Server Mode" dropdown +4. Select "namespace" (default) +5. Note current tool count in Copilot Chat +6. Change to "all" +7. Restart server: Command Palette → "MCP: List Servers" → Azure MCP → Start/Restart +8. Check tool count again + +**Verify:** +- [ ] Setting UI is accessible and clear +- [ ] Dropdown shows all three options: single, namespace, all +- [ ] Default is "namespace" +- [ ] Changing mode requires server restart (prompt shown) +- [ ] Tool count changes after restart: + - namespace: ~40-50 tools + - all: 100+ tools + - single: 1 tool + +#### Test 2: Toggle Read-Only Mode via UI + +**Steps:** +1. Open VS Code Settings +2. Search for "Azure MCP" +3. Locate "Azure Mcp: Read Only" checkbox +4. Check the box to enable read-only mode +5. Restart server +6. Test read operation: "List my storage accounts" +7. Test write operation: "Create a storage account" +8. Uncheck the box to disable read-only mode +9. Restart server +10. Test write operation again + +**Verify:** +- [ ] Checkbox is visible and labeled clearly +- [ ] Default state is unchecked (read-write) +- [ ] Enabling requires server restart +- [ ] Read operations work in both modes +- [ ] Write operations blocked when enabled +- [ ] Write operations allowed when disabled +- [ ] Clear error messages when operations are blocked + +#### Test 3: Filter Enabled Services via UI + +**Steps:** +1. Open VS Code Settings +2. Search for "Azure MCP" +3. Locate "Azure Mcp: Enabled Services" setting +4. Click "Add Item" button +5. Add "storage" to the list +6. Add "keyvault" to the list +7. Restart server +8. Check available tools in Copilot Chat + +**Verify:** +- [ ] Setting shows "Add Item" button +- [ ] Can add multiple service namespaces +- [ ] Can remove items from list +- [ ] Empty list = all services enabled +- [ ] Non-empty list = only specified services enabled +- [ ] Tool count reflects filtered services +- [ ] Test prompt for allowed service: "List storage accounts" +- [ ] Test prompt for filtered service: "List Cosmos DB accounts" + +#### Test 4: Combine Multiple Settings + +**Steps:** +1. Set "Enabled Services" to: storage, keyvault +2. Set "Server Mode" to: namespace +3. Enable "Read Only" checkbox +4. Restart server +5. Test various operations + +**Verify:** +- [ ] All three settings applied correctly +- [ ] Only Storage and Key Vault namespaces exposed (~2 tools) +- [ ] Read operations work for allowed services +- [ ] Write operations blocked by read-only mode +- [ ] Filtered services not accessible + +#### Test 5: Settings Persistence + +**Steps:** +1. Configure settings: mode=namespace, readOnly=true, services=[storage] +2. Restart VS Code completely +3. Check if settings persisted +4. Open Settings UI and verify values +5. Start MCP server +6. Verify configuration is applied + +**Verify:** +- [ ] Settings persist across VS Code restarts +- [ ] Settings shown correctly in UI after restart +- [ ] Server starts with saved configuration +- [ ] No need to reconfigure + +--- + +### Distribution Method 2: NPM Package + +**Objective**: Test CLI flag-based mode configuration with NPM-installed package + +**Installation:** +```bash +npm install -g @azure/mcp@latest +``` + +#### Test 1: Server Mode via CLI Flags + +**Test namespace mode:** +```bash +azmcp server start --mode namespace +``` + +**Verify:** +- [ ] Server starts successfully +- [ ] ~40-50 namespace-level tools exposed +- [ ] Tools grouped by service + +**Test all mode:** +```bash +azmcp server start --mode all +``` + +**Verify:** +- [ ] Server starts successfully +- [ ] 100+ individual tools exposed +- [ ] Each operation has dedicated tool + +**Test single mode:** +```bash +azmcp server start --mode single +``` + +**Verify:** +- [ ] Server starts successfully +- [ ] Exactly 1 tool exposed +- [ ] Internal routing functional + +#### Test 2: Read-Only Mode via CLI Flag + +**Enable read-only:** +```bash +azmcp server start --read-only +``` + +**Verify:** +- [ ] Server starts in read-only mode +- [ ] Read operations allowed +- [ ] Write operations blocked + +**Combine with mode:** +```bash +azmcp server start --mode namespace --read-only +``` + +**Verify:** +- [ ] Both flags applied correctly +- [ ] Namespace mode + read-only restrictions + +#### Test 3: Namespace Filtering via CLI Flags + +**Filter to specific services:** +```bash +azmcp server start --namespace storage --namespace keyvault +``` + +**Verify:** +- [ ] Only specified namespaces exposed +- [ ] Other services filtered out +- [ ] Can specify multiple --namespace flags + +#### Test 4: Tool Filtering via CLI Flags + +**Filter to specific tools:** +```bash +azmcp server start --tool azmcp_subscription_list --tool azmcp_group_list +``` + +**Verify:** +- [ ] Only specified tools exposed +- [ ] Automatically switches to all mode +- [ ] Can specify multiple --tool flags + +--- + +### Distribution Method 3: .NET Tool (DNX) + +**Objective**: Test CLI flag-based configuration with .NET tool installation + +**Installation:** +```bash +dotnet tool install --global Azure.Mcp +``` + +#### Test 1: Server Mode via CLI Flags + +**Test namespace mode:** +```bash +azmcp server start --mode namespace +``` + +**Verify:** +- [ ] .NET tool version works identically to NPM version +- [ ] All mode flags functional +- [ ] Tool exposure matches expectations + +#### Test 2: Configuration File Support + +**Create config file** (`azmcp.config.json`): +```json +{ + "serverMode": "namespace", + "readOnly": true, + "enabledServices": ["storage", "keyvault"] +} +``` + +**Start with config:** +```bash +azmcp server start --config azmcp.config.json +``` + +**Verify:** +- [ ] Config file loaded successfully +- [ ] Settings from file applied +- [ ] CLI flags override config file if specified + +--- + +### Distribution Method 4: IntelliJ IDEA Plugin + +**Objective**: Test MCP Server configuration dialog for mode configuration + +**Installation:** +1. IntelliJ IDEA → Settings/Preferences → Plugins +2. Search "Model Context Protocol (MCP)" +3. Install plugin +4. Restart IDE + +#### Configuration Dialog Fields: + +Based on IntelliJ's MCP configuration dialog: +- **Name**: Display name for the MCP server (e.g., "Azure MCP Server") +- **Command**: Executable command (e.g., `npx` or `azmcp`) +- **Arguments**: Command-line arguments including mode flags +- **Environment variables**: Key-value pairs for configuration +- **Working directory**: Optional working directory path + +#### Test 1: Configure via MCP Server Dialog + +**Steps:** +1. Navigate to File → Settings +2. Go to Tools → AI Assistant → Model Context Protocol (MCP) +3. Click the `+` icon to open "New MCP Server" dialog +4. Enter the following values: + - **Name**: `Azure MCP Server` + - **Command**: `npx` + - **Arguments**: `-y @azure/mcp@latest server start` +5. Leave other fields blank for default configuration +6. Click OK to save +7. Restart AI Assistant or IDE if prompted + +**Verify:** +- [ ] Dialog opens correctly with all fields +- [ ] Can enter server name +- [ ] Command and arguments fields accept input +- [ ] Configuration saves successfully +- [ ] Server appears in MCP server list +- [ ] Server starts when AI Assistant is opened + +#### Test 2: Configure Namespace Mode via Arguments + +**Steps:** +1. Open MCP Server configuration dialog +2. Edit existing "Azure MCP Server" entry +3. Update **Arguments** field to: + ``` + -y @azure/mcp@latest server start --mode namespace + ``` +4. Click OK and restart +5. Test tool availability + +**Verify:** +- [ ] Arguments field accepts mode flag +- [ ] Server restarts with new configuration +- [ ] Namespace mode applied (~40-50 tools) +- [ ] Tools grouped by service namespace + +#### Test 3: Configure Read-Only Mode via Arguments + +**Steps:** +1. Edit "Azure MCP Server" configuration +2. Update **Arguments** field to: + ``` + -y @azure/mcp@latest server start --mode namespace --read-only + ``` +3. Click OK and restart +4. Test read operation: "List my storage accounts" +5. Test write operation: "Create a storage account" + +**Verify:** +- [ ] Read-only flag accepted in arguments +- [ ] Read operations work +- [ ] Write operations blocked +- [ ] Error messages displayed + +#### Test 4: Configure Service Filtering via Arguments + +**Steps:** +1. Edit "Azure MCP Server" configuration +2. Update **Arguments** field to: + ``` + -y @azure/mcp@latest server start --namespace storage --namespace keyvault + ``` +3. Click OK and restart +4. Test allowed service: "List storage accounts" +5. Test filtered service: "List Cosmos DB accounts" + +**Verify:** +- [ ] Multiple namespace flags work +- [ ] Only specified services exposed +- [ ] Allowed services functional +- [ ] Filtered services not accessible + +#### Test 5: Configuration Persistence + +**Steps:** +1. Configure server with specific mode and flags +2. Close IntelliJ completely +3. Reopen IntelliJ +4. Navigate to MCP settings +5. Verify configuration still present +6. Start AI Assistant and test + +**Verify:** +- [ ] Configuration persists across IDE restarts +- [ ] No need to reconfigure +- [ ] Server starts with saved settings + +//TODO: Add VS, CLaude, Docker instructions. + +## What to Report for UI Issues + +When logging UI configuration issues, include: +- [ ] Distribution method (VS Code ext, NPM, .NET tool, etc.) +- [ ] IDE/Platform version +- [ ] Screenshot of settings UI +- [ ] Expected vs actual behavior +- [ ] Steps to reproduce +- [ ] Settings file content (if applicable) +- [ ] Server startup logs +- [ ] Whether restart was performed + + ## Verification Steps After installation on any platform/IDE, verify: @@ -345,6 +735,4 @@ az login - [Installation Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/README.md#installation) - [Report Issues](https://github.com/microsoft/mcp/issues) ---- - **Next Steps**: After completing installation testing, proceed to [Scenario Testing](scenarios/README.md). diff --git a/docs/bug-bash/scenarios/agent-building.md b/docs/bug-bash/scenarios/agent-building.md index 75a9d01ab6..d4051ab472 100644 --- a/docs/bug-bash/scenarios/agent-building.md +++ b/docs/bug-bash/scenarios/agent-building.md @@ -1,213 +1,389 @@ -# Agent Building Testing Scenario +# 🤖 AI Agent Building Testing Scenario -Test Azure MCP Server's capabilities for building, deploying, and managing AI agents using Azure AI Foundry and related services. +> **MCP Tool Support Notice** +> Azure MCP Server provides **AI Foundry resource inspection, model listing, and agent interaction** capabilities. Resource creation and deployment require Azure CLI or Portal. This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. ## Objectives -- Create and configure Azure AI Foundry resources -- Build custom AI agents with Azure MCP integration -- Deploy agents to Azure -- Test agent interactions and workflows -- Validate agent monitoring and management -- Test multi-agent scenarios +- Test Azure AI Foundry resource discovery and inspection +- Test AI model listing and deployment management +- Test agent creation and interaction workflows +- Validate agent querying and evaluation capabilities ## Prerequisites - [ ] Azure MCP Server installed and configured - [ ] Azure CLI installed (`az --version`) - [ ] Authenticated to Azure (`az login`) -- [ ] Active Azure subscription with AI services enabled +- [ ] Active Azure subscription - [ ] GitHub Copilot with Agent mode enabled -- [ ] Basic understanding of AI agent concepts -## Test Scenarios +--- -### Scenario 1: Azure AI Foundry Setup +## Scenario 1: Azure AI Foundry Resource Discovery & Model Management -**Objective**: Set up Azure AI Foundry resources for agent development +**Objective**: Complete workflow for discovering AI Foundry resources and managing model deployments -#### Phase 1: Create AI Foundry Resources +### Step 1: Setup Resources (External - Not MCP) -1. **Create AI Hub**: - ``` - Create an Azure AI Foundry hub: - - Name: 'bugbash-ai-hub-' - - Resource group: '' - - Location: East US - - SKU: Standard - ``` +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. -2. **Verify hub creation**: - ``` - List all Azure AI Foundry hubs in my subscription - ``` +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-aifoundry-rg' in eastus, then create an Azure AI Services account with SKU S0, and deploy GPT-4o model with deployment name 'gpt-4o-deployment' +``` -3. **Get hub details**: - ``` - Show me the details of AI Foundry hub 'bugbash-ai-hub-' - ``` +**Option B: Run Azure CLI Commands Manually**: +```bash +# Create resource group +az group create --name bugbash-aifoundry-rg --location eastus -4. **Create AI project**: - ``` - Create an Azure AI Foundry project: - - Name: 'bugbash-ai-project' - - Hub: 'bugbash-ai-hub-' - - Description: 'Bug bash agent testing project' - ``` +# Create Azure AI Foundry resource (AI Services account) +az cognitiveservices account create \ + --name bugbash-ai-foundry-$RANDOM \ + --resource-group bugbash-aifoundry-rg \ + --location eastus \ + --kind AIServices \ + --sku S0 + +# Deploy a model (GPT-4o) +az cognitiveservices account deployment create \ + --name \ + --resource-group bugbash-aifoundry-rg \ + --deployment-name gpt-4o-deployment \ + --model-name gpt-4o \ + --model-version "2024-05-13" \ + --model-format OpenAI \ + --sku-capacity 10 \ + --sku-name Standard +``` + +### Step 2: Discover AI Foundry Resources with Azure MCP Server + +**2.1 Get AI Foundry resource details** (uses `azmcp_foundry_resource_get`): +``` +Show me details for Azure AI Foundry resources in my subscription +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_foundry_resource_get` +- [ ] Your newly created AI Foundry resource appears +- [ ] Resource properties shown (name, location, SKU) + +**2.2 Alternative phrasing**: +``` +List all Azure AI Foundry resources in resource group 'bugbash-aifoundry-rg' +``` + +### Step 3: List Available Models with Azure MCP Server + +**3.1 List all available models** (uses `azmcp_foundry_models_list`): +``` +List all available AI models in Azure AI Foundry +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_foundry_models_list` +- [ ] Model catalog displayed +- [ ] GPT-4o and other models listed +- [ ] Model details shown (publisher, license, capabilities) + +**3.2 Search for specific models**: +``` +Show me all GPT models available in Azure AI Foundry +``` **Verify**: -- [ ] AI Hub created successfully -- [ ] Hub is in running state -- [ ] AI Project created -- [ ] Resources are properly linked - -#### Phase 2: Configure AI Services - -5. **Check available AI models**: - ``` - What AI models are available in my Azure AI Foundry hub - 'bugbash-ai-hub-'? - ``` - -6. **Deploy OpenAI model**: - ``` - Deploy GPT-4o model to my AI Foundry project 'bugbash-ai-project' - ``` - -7. **List deployments**: - ``` - List all model deployments in AI Foundry project 'bugbash-ai-project' - ``` - -8. **Get endpoint information**: - ``` - Get the endpoint URL and keys for my deployed model - ``` +- [ ] Filtered list returned +- [ ] GPT models displayed +- [ ] Search functionality works + +**3.3 Check playground-compatible models**: +``` +Which models can I use in the free playground? +``` **Verify**: -- [ ] Available models listed -- [ ] Model deployment successful -- [ ] Endpoint accessible -- [ ] Authentication configured +- [ ] Playground-compatible models listed +- [ ] Filter applied correctly + +### Step 4: Inspect Model Deployments with Azure MCP Server + +**4.1 List model deployments** (uses `azmcp_foundry_models_deployments_list`): +``` +List all model deployments in my Azure AI Foundry resource +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_foundry_models_deployments_list` +- [ ] Your GPT-4o deployment appears +- [ ] Deployment details shown (name, model, SKU, capacity) + +**4.2 Alternative phrasing**: +``` +Show me all deployed models in my AI Foundry resource +``` + +### Step 5: List OpenAI Models with Azure MCP Server + +**5.1 List OpenAI models** (uses `azmcp_foundry_openai_models-list`): +``` +List all OpenAI models and deployments in my Azure AI resource '' in resource group 'bugbash-aifoundry-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_foundry_openai_models-list` +- [ ] OpenAI models listed +- [ ] Deployment information shown + +### Step 6: Cleanup (External - Not MCP) + +**Option A: Prompt GitHub Copilot Chat**: +``` +Delete the Azure resource group 'bugbash-aifoundry-rg' and all its resources +``` + +**Option B: Run Azure CLI Command Manually**: +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-aifoundry-rg --yes --no-wait +``` **Expected Results**: -- AI Foundry hub created -- AI project configured -- AI model deployed -- Endpoints are accessible +- AI Foundry resource discovery works +- Model catalog listing successful +- Deployment inspection accurate +- OpenAI model listing functional + +--- + +## Scenario 2: AI Agent Creation & Interaction End-to-End + +**Objective**: Complete workflow for creating, querying, and evaluating AI agents + +### Step 1: Setup Agent Resources (External - Not MCP) + +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. + +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-agents-rg' in eastus, create an Azure AI Services account with SKU S0, deploy GPT-4o model with deployment name 'agent-gpt4o', then show me the endpoint URL +``` + +**Option B: Run Azure CLI Commands Manually**: +```bash +# Create resource group +az group create --name bugbash-agents-rg --location eastus + +# Create Azure AI Foundry resource +az cognitiveservices account create \ + --name bugbash-agents-$RANDOM \ + --resource-group bugbash-agents-rg \ + --location eastus \ + --kind AIServices \ + --sku S0 + +# Get endpoint URL (you'll need this) +az cognitiveservices account show \ + --name \ + --resource-group bugbash-agents-rg \ + --query "properties.endpoint" -o tsv + +# Deploy GPT-4o for agent use +az cognitiveservices account deployment create \ + --name \ + --resource-group bugbash-agents-rg \ + --deployment-name agent-gpt4o \ + --model-name gpt-4o \ + --model-version "2024-05-13" \ + --model-format OpenAI \ + --sku-capacity 10 \ + --sku-name Standard +``` + +**Option C: Create Agent via Azure AI Foundry Portal**: +1. Go to [Azure AI Foundry Portal](https://ai.azure.com) +2. Create a new project +3. Create an agent with your deployed model +4. Note the agent ID for testing + +### Step 2: List Available Agents with Azure MCP Server + +**2.1 List all agents** (uses `azmcp_foundry_agents_list`): +``` +List all AI agents in my Azure AI Foundry project with endpoint '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_foundry_agents_list` +- [ ] Your created agent appears +- [ ] Agent properties shown (ID, name, model) + +**2.2 Alternative phrasing**: +``` +Show me all agents in my AI Foundry project +``` + +### Step 3: Query an Agent with Azure MCP Server + +**3.1 Connect and query agent** (uses `azmcp_foundry_agents_connect`): +``` +Query agent '' with: "What Azure services can you help me manage?" using endpoint '' +``` -### Scenario 2: Deploy and Test Agent +**Verify**: +- [ ] Tool invoked: `azmcp_foundry_agents_connect` +- [ ] Agent response received +- [ ] Response is coherent and relevant + +**3.2 Test agent capabilities**: +``` +Ask agent '' to: "List the most common Azure resources used in web applications" using endpoint '' +``` -**Objective**: Deploy the agent and test its functionality +**Verify**: +- [ ] Agent understands the query +- [ ] Response includes relevant Azure services +- [ ] Answer is accurate -#### Phase 1: Deploy Agent Code +**3.3 Test agent with specific query**: +``` +Query agent '': "Explain the difference between Azure SQL Database and Cosmos DB" using endpoint '' +``` -1. **Get deployment guidance**: - ``` - How do I deploy my Python agent code to Function App - 'bugbash-agent-func-'? - ``` +**Verify**: +- [ ] Detailed explanation provided +- [ ] Technical accuracy maintained +- [ ] Response format is clear -2. **Generate sample agent code**: - ``` - Generate sample Python code for an Azure management agent that: - - Uses Azure OpenAI for natural language understanding - - Uses Azure MCP Server to manage Azure resources - - Stores conversation history in Cosmos DB - ``` +### Step 4: Evaluate Agent Responses with Azure MCP Server -3. **Deploy agent**: - ``` - Deploy my agent code to Function App 'bugbash-agent-func-' - ``` +**4.1 Evaluate a response** (uses `azmcp_foundry_agents_evaluate`): +``` +Evaluate the response from agent '' for query "What is Azure Storage?" with response "Azure Storage is a cloud storage solution" using evaluator 'groundedness' with Azure OpenAI endpoint '' and deployment '' +``` **Verify**: -- [ ] Deployment instructions clear -- [ ] Sample code provided -- [ ] Deployment successful -- [ ] Function App running - -#### Phase 2: Test Agent Interactions - -4. **Test basic agent query**: - ``` - Send a test request to my agent: "List all my storage accounts" - ``` - -5. **Test resource creation**: - ``` - Ask my agent to: "Create a new storage account named - 'agenttest' in East US" - ``` - -6. **Test monitoring capability**: - ``` - Ask my agent: "Show me the health status of all my resources in - resource group ''" - ``` - -7. **Test complex query**: - ``` - Ask my agent: "Which of my resources are costing the most this month?" - ``` +- [ ] Tool invoked: `azmcp_foundry_agents_evaluate` +- [ ] Evaluation score returned +- [ ] Groundedness metrics shown + +**4.2 Query and evaluate in one step** (uses `azmcp_foundry_agents_query-and-evaluate`): +``` +Query and evaluate agent '' with: "What are the main features of Azure Functions?" using endpoint '' and evaluate with Azure OpenAI endpoint '' and deployment '' +``` **Verify**: -- [ ] Agent responds correctly -- [ ] Resource operations work -- [ ] Monitoring data retrieved -- [ ] Complex queries handled +- [ ] Tool invoked: `azmcp_foundry_agents_query-and-evaluate` +- [ ] Agent response received +- [ ] Evaluation metrics provided +- [ ] Combined workflow successful -#### Phase 3: Monitor Agent Performance +### Step 5: Explore Knowledge Indexes with Azure MCP Server -8. **Check agent logs**: - ``` - Show me the logs for Function App 'bugbash-agent-func-' - from the last hour - ``` +**5.1 List knowledge indexes** (uses `azmcp_foundry_knowledge_index_list`): +``` +List all knowledge indexes in my AI Foundry project with endpoint '' +``` -9. **Monitor agent metrics**: - ``` - Show me performance metrics for my agent Function App - ``` +**Verify**: +- [ ] Tool invoked: `azmcp_foundry_knowledge_index_list` +- [ ] Knowledge indexes listed (if any exist) +- [ ] Index properties shown -10. **Check Cosmos DB usage**: - ``` - Show me the Cosmos DB usage statistics for my agent's state database - ``` +**5.2 Get index schema** (uses `azmcp_foundry_knowledge_index_schema`): +``` +Show me the schema for knowledge index '' using endpoint '' +``` **Verify**: -- [ ] Logs accessible -- [ ] Metrics available -- [ ] Performance acceptable -- [ ] State persistence working +- [ ] Tool invoked: `azmcp_foundry_knowledge_index_schema` +- [ ] Schema information returned +- [ ] Field types and structure shown + +### Step 6: Cleanup (External - Not MCP) + +**Option A: Prompt GitHub Copilot Chat**: +``` +Delete the Azure resource group 'bugbash-agents-rg' and all its resources +``` + +**Option B: Run Azure CLI Command Manually**: +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-agents-rg --yes --no-wait +``` **Expected Results**: -- Agent deployed successfully -- Agent responds to queries -- Resource operations work -- Monitoring and logging functional +- Agent listing works correctly +- Agent querying returns responses +- Response evaluation provides metrics +- Combined query-and-evaluate workflow functional +- Knowledge index inspection successful + +--- ## Common Issues to Watch For -- **Authentication Failures**: Managed identity misconfiguration -- **Token Limits**: Exceeding model context windows -- **Rate Limiting**: API throttling from AI services -- **State Persistence**: Session state not saved correctly -- **Cold Starts**: Function App cold start latency -- **Network Timeouts**: Long-running operations timing out -- **Cost Overruns**: Unexpected charges from AI API calls -- **Model Hallucinations**: Incorrect or inconsistent agent responses -- **Integration Issues**: MCP Server connection problems -- **Event Ordering**: Messages processed out of sequence +| Issue | Description | Resolution | +|-------|-------------|------------| +| **Authentication Failures** | Can't connect to AI Foundry endpoint | Verify `az login` and endpoint URL is correct | +| **Agent Not Found** | Agent ID doesn't exist | List agents first to get valid agent IDs | +| **Token Limits** | Response truncated or incomplete | Model context window exceeded; use shorter prompts | +| **Rate Limiting** | API throttling errors | Reduce request frequency or upgrade SKU | +| **Endpoint Mismatch** | Wrong endpoint URL | Verify endpoint matches your AI Foundry resource | +| **Model Not Deployed** | Deployment not found | Check model deployments are active and provisioned | +| **Evaluation Failures** | Evaluator returns errors | Ensure Azure OpenAI deployment exists for evaluation | +| **Empty Agent List** | No agents returned | Create agents via AI Foundry Portal first | + +## What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output) +- [ ] Expected vs actual results +- [ ] Error messages (if any) +- [ ] Agent ID and endpoint URL (redact sensitive info) +- [ ] Model name and deployment name +- [ ] Screenshots of unexpected behavior ## Related Resources - [Azure AI Foundry Documentation](https://learn.microsoft.com/azure/ai-studio/) - [Azure OpenAI Service](https://learn.microsoft.com/azure/ai-services/openai/) -- [Azure Functions for AI Agents](https://learn.microsoft.com/azure/azure-functions/) +- [Azure AI Agents](https://learn.microsoft.com/azure/ai-services/agents/) - [Model Context Protocol](https://modelcontextprotocol.io/) -- [Azure MCP Server GitHub](https://github.com/microsoft/mcp) -- [Agent Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) +## 💡 Quick Reference: Supported MCP Tools + +### AI Foundry Resources +- `azmcp_foundry_resource_get` - Get AI Foundry resource details +- `azmcp_foundry_models_list` - List available AI models +- `azmcp_foundry_models_deployments_list` - List model deployments + +### OpenAI Integration +- `azmcp_foundry_openai_models-list` - List OpenAI models and deployments +- `azmcp_foundry_openai_chat-completions-create` - Create chat completions +- `azmcp_foundry_openai_create-completion` - Generate text completions +- `azmcp_foundry_openai_embeddings-create` - Generate embeddings + +### AI Agents +- `azmcp_foundry_agents_list` - List AI agents +- `azmcp_foundry_agents_connect` - Query an agent +- `azmcp_foundry_agents_evaluate` - Evaluate agent response +- `azmcp_foundry_agents_query-and-evaluate` - Query and evaluate in one step + +### Knowledge Management +- `azmcp_foundry_knowledge_index_list` - List knowledge indexes +- `azmcp_foundry_knowledge_index_schema` - Get index schema +- `azmcp_foundry_knowledge_source_get` - Get knowledge sources + +### Model Deployment +- `azmcp_foundry_models_deploy` - Deploy AI model (write operation) + +--- + **Next**: [Database Operations Testing](database-operations.md) \ No newline at end of file diff --git a/docs/bug-bash/scenarios/database-operations.md b/docs/bug-bash/scenarios/database-operations.md index 3ad345e890..954feab3db 100644 --- a/docs/bug-bash/scenarios/database-operations.md +++ b/docs/bug-bash/scenarios/database-operations.md @@ -1,428 +1,411 @@ -# 🗄️ Database Operations Testing Scenario -> **⚠️ READ FIRST**: [TESTING-SCOPE.md](../TESTING-SCOPE.md) explains what MCP tools can and cannot do. **Azure MCP Server focuses on querying and inspecting databases, NOT creating them**. Use Azure CLI to create test databases before running these scenarios. +# Database Operations Testing Scenario -Test Azure MCP Server's database querying and inspection capabilities across Cosmos DB, PostgreSQL, MySQL, and Azure SQL. +> **MCP Tool Support Notice** +> Azure MCP Server provides **read-only database inspection and querying** capabilities. Database creation, data insertion, and schema modifications require Azure CLI or Portal. This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. -## 🎯 Objectives +## Objectives -- List and inspect Cosmos DB accounts, databases, and containers -- Query PostgreSQL and MySQL databases -- List and inspect Azure SQL servers and databases -- Test query execution and data retrieval -- Verify table schema inspection -- Test database configuration viewing +- Test end-to-end Cosmos DB inspection and querying workflows +- Test end-to-end PostgreSQL/MySQL database operations and querying +- Verify MCP tool accuracy for listing, inspecting, and querying databases +- Validate query execution and schema inspection across database types -## ✅ Prerequisites +## Prerequisites - [ ] Azure MCP Server installed and configured -- [ ] **Existing databases already deployed** (use Azure CLI or Portal to create test databases before testing) -- [ ] Azure CLI installed (`az --version`) for database creation +- [ ] Azure CLI installed (`az --version`) - [ ] Authenticated to Azure (`az login`) -- [ ] Active Azure subscription with existing database resources +- [ ] Active Azure subscription - [ ] GitHub Copilot with Agent mode enabled -> **Note**: Azure MCP Server tools focus on **reading and querying** existing databases. Use Azure CLI commands (`az cosmosdb create`, `az postgres flexible-server create`, `az sql server create`) to create database resources before testing these scenarios. - -## Test Scenarios - -### Scenario 1: Cosmos DB Operations - -**Objective**: Test Cosmos DB account listing, database inspection, and querying - -> **Setup Required**: Use Azure CLI to create a test Cosmos DB account before this scenario: -> ```bash -> az cosmosdb create --name bugbash-cosmos-$RANDOM --resource-group --locations regionName=eastus -> az cosmosdb sql database create --account-name --name ProductCatalog --resource-group -> az cosmosdb sql container create --account-name --database-name ProductCatalog --name Products --partition-key-path "/category" --resource-group -> ``` - -#### Phase 1: List Cosmos DB Resources - -1. **List all Cosmos DB accounts** (uses `azmcp_cosmos_account_list`): - ``` - List all Cosmos DB accounts in my subscription - ``` +--- -2. **Alternative phrasing**: - ``` - Show me my Cosmos DB accounts - ``` - ``` - Show me the Cosmos DB accounts in my subscription - ``` +## Scenario 1: Cosmos DB End-to-End Workflow + +**Objective**: Complete workflow from resource setup through querying with Azure MCP Server + +### Step 1: Setup Resources (External - Not MCP) + +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. + +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-cosmosdb-rg' in eastus, create a Cosmos DB account, create a database named 'ProductCatalog', and create a container named 'Products' with partition key '/category' +``` + +Then use Azure Portal Data Explorer to insert sample data: +```json +{"id": "1", "name": "Laptop", "category": "Electronics", "price": 1299.99} +{"id": "2", "name": "Mouse", "category": "Electronics", "price": 29.99} +{"id": "3", "name": "Desk", "category": "Furniture", "price": 499.99} +``` + +**Option B: Run Azure CLI Commands Manually**: +```bash +# Create resource group +az group create --name bugbash-cosmosdb-rg --location eastus + +# Create Cosmos DB account (this takes 5-10 minutes) +az cosmosdb create \ + --name bugbash-cosmos-$RANDOM \ + --resource-group bugbash-cosmosdb-rg \ + --locations regionName=eastus + +# Create database +az cosmosdb sql database create \ + --account-name \ + --resource-group bugbash-cosmosdb-rg \ + --name ProductCatalog + +# Create container with partition key +az cosmosdb sql container create \ + --account-name \ + --resource-group bugbash-cosmosdb-rg \ + --database-name ProductCatalog \ + --name Products \ + --partition-key-path "/category" +``` + +**Insert sample data** using Azure Portal Data Explorer: +```json +{"id": "1", "name": "Laptop", "category": "Electronics", "price": 1299.99} +{"id": "2", "name": "Mouse", "category": "Electronics", "price": 29.99} +{"id": "3", "name": "Desk", "category": "Furniture", "price": 499.99} +``` + +### Step 2: Discovery with Azure MCP Server + +**2.1 List all Cosmos DB accounts** (uses `azmcp_cosmos_account_list`): +``` +List all Cosmos DB accounts in my subscription +``` **Verify**: -- [ ] Tool correctly invoked: `azmcp_cosmos_account_list` -- [ ] Account list displayed +- [ ] Tool invoked: `azmcp_cosmos_account_list` +- [ ] Your newly created account appears in the list - [ ] Account properties shown (name, location, capabilities) -#### Phase 2: Inspect Databases and Containers +**2.2 Alternative phrasing**: +``` +Show me my Cosmos DB accounts +``` -3. **List databases** (uses `azmcp_cosmos_database_list`): - ``` - List all databases in Cosmos DB account '' - ``` +### Step 3: Database Inspection with Azure MCP Server -4. **Alternative phrasing**: - ``` - Show me the databases in Cosmos DB account '' - ``` +**3.1 List databases** (uses `azmcp_cosmos_database_list`): +``` +List all databases in Cosmos DB account '' +``` **Verify**: -- [ ] Tool correctly invoked: `azmcp_cosmos_database_list` -- [ ] Database list displayed -- [ ] Database properties shown +- [ ] Tool invoked: `azmcp_cosmos_database_list` +- [ ] 'ProductCatalog' database is listed +- [ ] Database properties displayed -5. **List containers** (uses `azmcp_cosmos_database_container_list`): - ``` - List all containers in database 'ProductCatalog' for Cosmos DB account '' - ``` - -6. **Alternative phrasing**: - ``` - Show me the containers in database 'ProductCatalog' for Cosmos DB account '' - ``` +**3.2 List containers** (uses `azmcp_cosmos_database_container_list`): +``` +List all containers in database 'ProductCatalog' for Cosmos DB account '' +``` **Verify**: -- [ ] Tool correctly invoked: `azmcp_cosmos_database_container_list` -- [ ] Container list displayed -- [ ] Container properties shown (partition key, throughput) +- [ ] Tool invoked: `azmcp_cosmos_database_container_list` +- [ ] 'Products' container is listed +- [ ] Partition key path '/category' is shown -#### Phase 3: Query Data +### Step 4: Query Data with Azure MCP Server -> **Data Setup**: Use Azure Portal or Azure CLI to insert sample data into the Products container before querying. +**4.1 Query all items** (uses `azmcp_cosmos_database_container_item_query`): +``` +Query all items from container 'Products' in database 'ProductCatalog' for Cosmos DB account '' +``` -7. **Query items** (uses `azmcp_cosmos_database_container_item_query`): - ``` - Show me the items that contain the word 'Laptop' in container 'Products' - in database 'ProductCatalog' for Cosmos DB account '' - ``` +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_database_container_item_query` +- [ ] All 3 sample items returned +- [ ] Item properties correctly displayed -8. **Alternative query patterns**: - ``` - Query all items from container 'Products' in database 'ProductCatalog' - ``` - ``` - Show me items in the Products container where category is 'Electronics' - ``` +**4.2 Query with filter**: +``` +Show me items in the Products container where category is 'Electronics' in database 'ProductCatalog' for Cosmos DB account '' +``` **Verify**: -- [ ] Tool correctly invoked: `azmcp_cosmos_database_container_item_query` -- [ ] Query results displayed -- [ ] Query syntax accepted -- [ ] Results match expected data +- [ ] Query executed successfully +- [ ] Only 2 items returned (Laptop and Mouse) +- [ ] Filter correctly applied -**Expected Results**: -- Cosmos DB account created -- Database and container created -- Data operations work correctly -- Queries return expected results +**4.3 Search query**: +``` +Show me items that contain the word 'Laptop' in container 'Products' in database 'ProductCatalog' for Cosmos DB account '' +``` ---- +**Verify**: +- [ ] Text search works correctly +- [ ] Laptop item returned +- [ ] Query syntax accepted -### Scenario 2: PostgreSQL Operations +### Step 5: Cleanup (External - Not MCP) -**Objective**: Test Azure Database for PostgreSQL flexible server management +**Option A: Prompt GitHub Copilot Chat**: +``` +Delete the Azure resource group 'bugbash-cosmosdb-rg' and all its resources +``` -#### Phase 1: Create PostgreSQL Server +**Option B: Run Azure CLI Command Manually**: +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-cosmosdb-rg --yes --no-wait +``` -1. **Create PostgreSQL server**: - ``` - Create an Azure Database for PostgreSQL flexible server: - - Name: 'bugbash-postgres-' - - Resource group: '' - - Admin username: 'dbadmin' - - Password: '' - - Version: PostgreSQL 14 - - Tier: Burstable B1ms - - Storage: 32 GB - - Region: East US - - Allow Azure services and resources to access this server - ``` +**Expected Results**: +- All MCP listing tools work correctly +- Database and container inspection successful +- Query operations return accurate results +- Different query patterns supported -2. **List PostgreSQL servers**: - ``` - List all PostgreSQL servers in my subscription - ``` +--- -3. **Get server details**: - ``` - Show me the details of PostgreSQL server 'bugbash-postgres-' - ``` +## Scenario 2: PostgreSQL End-to-End Workflow + +**Objective**: Complete workflow for PostgreSQL server inspection, schema viewing, and querying + +### Step 1: Setup Resources (External - Not MCP) + +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. + +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-postgres-rg' in eastus, create a PostgreSQL flexible server with admin user 'dbadmin', create a database named 'inventory', then help me create a products table with columns: id, name, description, price, stock_quantity, created_at +``` + +Then use Azure Portal Query Editor or GitHub Copilot to insert sample data. + +**Option B: Run Azure CLI Commands Manually**: +```bash +# Create resource group +az group create --name bugbash-postgres-rg --location eastus + +# Create PostgreSQL flexible server (takes 5-10 minutes) +az postgres flexible-server create \ + --name bugbash-postgres-$RANDOM \ + --resource-group bugbash-postgres-rg \ + --location eastus \ + --admin-user dbadmin \ + --admin-password \ + --sku-name Standard_B1ms \ + --tier Burstable \ + --version 14 \ + --storage-size 32 \ + --public-access 0.0.0.0 + +# Create database +az postgres flexible-server db create \ + --resource-group bugbash-postgres-rg \ + --server-name \ + --database-name inventory + +# Connect and create table (use Azure Portal Query Editor or psql) +# CREATE TABLE products ( +# id SERIAL PRIMARY KEY, +# name VARCHAR(100) NOT NULL, +# description TEXT, +# price DECIMAL(10,2), +# stock_quantity INTEGER DEFAULT 0, +# created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP +# ); + +# Insert sample data +# INSERT INTO products (name, description, price, stock_quantity) VALUES +# ('Laptop', 'High-performance laptop', 1299.99, 25), +# ('Mouse', 'Wireless mouse', 29.99, 100), +# ('Keyboard', 'Mechanical keyboard', 89.99, 50); +``` + +### Step 2: Server Discovery with Azure MCP Server + +**2.1 List all PostgreSQL servers** (uses `azmcp_postgres_server_list`): +``` +List all PostgreSQL servers in my subscription +``` **Verify**: -- [ ] Server created successfully -- [ ] Version is PostgreSQL 14 -- [ ] Tier and storage correct -- [ ] Firewall allows Azure services - -#### Phase 2: Database and Table Operations - -4. **Create database**: - ``` - Create a database named 'inventory' on PostgreSQL server - 'bugbash-postgres-' - ``` - -5. **List databases**: - ``` - List all databases in PostgreSQL server 'bugbash-postgres-' - ``` - -6. **Create tables**: - ``` - In PostgreSQL database 'inventory', create a table named 'products' with: - - id: SERIAL PRIMARY KEY - - name: VARCHAR(100) NOT NULL - - description: TEXT - - price: DECIMAL(10,2) - - stock_quantity: INTEGER DEFAULT 0 - - created_at: TIMESTAMP DEFAULT CURRENT_TIMESTAMP - ``` - -7. **List tables**: - ``` - List all tables in PostgreSQL database 'inventory' on server - 'bugbash-postgres-' - ``` - -8. **Get table schema**: - ``` - Show me the schema of table 'products' in database 'inventory' - ``` +- [ ] Tool invoked: `azmcp_postgres_server_list` +- [ ] Your newly created server appears +- [ ] Server details shown (name, location, version, SKU) -**Verify**: -- [ ] Database created -- [ ] Tables created with correct schema -- [ ] Columns have correct data types -- [ ] Constraints are applied - -#### Phase 3: Data Operations and Queries - -9. **Insert data**: - ``` - Insert the following records into the products table: - - Laptop, High-performance laptop, 1299.99, 25 - - Mouse, Wireless mouse, 29.99, 100 - - Keyboard, Mechanical keyboard, 89.99, 50 - ``` - -10. **Query all data**: - ``` - Show me all products in the inventory database - ``` - -11. **Query with filter**: - ``` - Show me all products with price less than 100 in the inventory database - ``` - -12. **Search query**: - ``` - Show me all items that contain the word 'laptop' in the products - table in PostgreSQL database 'inventory' - ``` +**2.2 Get server configuration** (uses `azmcp_postgres_server_config_get`): +``` +Show me the configuration of PostgreSQL server '' in resource group 'bugbash-postgres-rg' +``` **Verify**: -- [ ] Data inserted successfully -- [ ] Queries return correct results -- [ ] Filters work properly -- [ ] Text search works - -#### Phase 4: Server Configuration - -13. **Get server configuration**: - ``` - Show me the configuration of PostgreSQL server 'bugbash-postgres-' - ``` - -14. **Check server parameters**: - ``` - Show me if the PostgreSQL server 'bugbash-postgres-' has - replication enabled - ``` - -15. **Get connection information**: - ``` - Get the connection string for PostgreSQL server 'bugbash-postgres-' - database 'inventory' - ``` +- [ ] Tool invoked: `azmcp_postgres_server_config_get` +- [ ] Configuration parameters displayed +- [ ] Server settings accessible -**Verify**: -- [ ] Configuration retrieved -- [ ] Parameters are accessible -- [ ] Connection string provided - -**Expected Results**: -- PostgreSQL server created -- Database and tables created -- Data operations work -- Queries execute correctly -- Configuration accessible +### Step 3: Database and Schema Inspection with Azure MCP Server ---- +**3.1 List databases** (uses `azmcp_postgres_database_list`): +``` +List all databases in PostgreSQL server '' in resource group 'bugbash-postgres-rg' +``` -### Scenario 3: Azure SQL Database Operations +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_database_list` +- [ ] 'inventory' database is listed +- [ ] System databases also shown (postgres, template0, template1) -**Objective**: Test Azure SQL Database and server management +**3.2 List tables** (uses `azmcp_postgres_table_list`): +``` +List all tables in PostgreSQL database 'inventory' on server '' in resource group 'bugbash-postgres-rg' +``` -#### Phase 1: Create SQL Server and Database +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_table_list` +- [ ] 'products' table is listed +- [ ] Table names correctly displayed -1. **Create SQL server**: - ``` - Create an Azure SQL server: - - Name: 'bugbash-sqlserver-' - - Resource group: '' - - Admin login: 'sqladmin' - - Password: '' - - Location: East US - - Allow Azure services and resources to access this server - ``` +**3.3 Get table schema** (uses `azmcp_postgres_table_schema_get`): +``` +Show me the schema of table 'products' in database 'inventory' on server '' in resource group 'bugbash-postgres-rg' +``` -2. **List SQL servers**: - ``` - List all Azure SQL servers in resource group '' - ``` +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_table_schema_get` +- [ ] All columns listed (id, name, description, price, stock_quantity, created_at) +- [ ] Data types correctly shown +- [ ] Primary key and constraints visible -3. **Get server details**: - ``` - Show me the details of Azure SQL server 'bugbash-sqlserver-' - in resource group '' - ``` +### Step 4: Query Data with Azure MCP Server -**Verify**: -- [ ] Server created successfully -- [ ] Admin login configured -- [ ] Location is correct -- [ ] Firewall configured - -#### Phase 2: Create and Manage Databases - -4. **Create database**: - ``` - Create a SQL database named 'OrdersDB' on server - 'bugbash-sqlserver-' with Basic tier - ``` - -5. **List databases**: - ``` - List all databases in Azure SQL server 'bugbash-sqlserver-' - ``` - -6. **Show database details**: - ``` - Get the configuration details for SQL database 'OrdersDB' on - server 'bugbash-sqlserver-' - ``` - -7. **Create another database**: - ``` - Create a SQL database named 'TestDB' on server - 'bugbash-sqlserver-' with Standard tier (S1) - ``` +**4.1 Query all data** (uses `azmcp_postgres_database_query`): +``` +Show me all products in the 'products' table in database 'inventory' on server '' in resource group 'bugbash-postgres-rg' +``` **Verify**: -- [ ] Databases created with correct tiers -- [ ] Database properties are correct -- [ ] Databases are accessible +- [ ] Tool invoked: `azmcp_postgres_database_query` +- [ ] All 3 sample records returned +- [ ] Column values correctly displayed -#### Phase 3: Firewall Rules +**4.2 Query with filter**: +``` +Show me all products with price less than 100 in the 'products' table in database 'inventory' on server '' in resource group 'bugbash-postgres-rg' +``` -8. **List firewall rules**: - ``` - List all firewall rules for SQL server 'bugbash-sqlserver-' - ``` - -9. **Create firewall rule**: - ``` - Create a firewall rule for Azure SQL server 'bugbash-sqlserver-' - to allow access from my current IP address - ``` +**Verify**: +- [ ] Query executed successfully +- [ ] Only 2 items returned (Mouse and Keyboard) +- [ ] WHERE clause correctly applied -10. **Verify firewall rule**: - ``` - Show me the firewall rules for SQL server 'bugbash-sqlserver-' - ``` +**4.3 Search query**: +``` +Show me all items that contain the word 'laptop' in the 'products' table in PostgreSQL database 'inventory' on server '' in resource group 'bugbash-postgres-rg' +``` **Verify**: -- [ ] Firewall rules listed -- [ ] New rule created -- [ ] Rule parameters correct - -#### Phase 4: Database Configuration - -11. **Update database tier**: - ``` - Update the performance tier of SQL database 'OrdersDB' on server - 'bugbash-sqlserver-' to Standard S2 - ``` - -12. **Rename database**: - ``` - Rename the SQL database 'TestDB' to 'StagingDB' on server - 'bugbash-sqlserver-' - ``` - -13. **Check elastic pools**: - ``` - List all elastic pools in SQL server 'bugbash-sqlserver-' - ``` - -14. **List Entra ID administrators**: - ``` - List Microsoft Entra ID administrators for SQL server - 'bugbash-sqlserver-' - ``` +- [ ] Text search works correctly +- [ ] Laptop item returned +- [ ] Case-insensitive search works -**Verify**: -- [ ] Tier update successful -- [ ] Database renamed correctly -- [ ] Elastic pools listed (if any) -- [ ] Admin info retrieved +**4.4 Check server parameters** (uses `azmcp_postgres_server_param_get`): +``` +Show me the max_connections parameter for PostgreSQL server '' in resource group 'bugbash-postgres-rg' +``` -#### Phase 5: Database Deletion +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_server_param_get` +- [ ] Parameter value displayed +- [ ] Parameter metadata shown -15. **Delete test database**: - ``` - Delete the SQL database 'StagingDB' from server - 'bugbash-sqlserver-' - ``` +### Step 5: Cleanup (External - Not MCP) -16. **Verify deletion**: - ``` - List databases in server 'bugbash-sqlserver-' to confirm - 'StagingDB' is deleted - ``` +**Option A: Prompt GitHub Copilot Chat**: +``` +Delete the Azure resource group 'bugbash-postgres-rg' and all its resources +``` -**Verify**: -- [ ] Database deleted successfully -- [ ] Deletion confirmed +**Option B: Run Azure CLI Command Manually**: +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-postgres-rg --yes --no-wait +``` **Expected Results**: -- SQL server created -- Multiple databases created -- Firewall rules managed -- Database configuration updated -- Database operations work correctly +- Server listing and discovery works +- Database and table inspection successful +- Schema information accurately retrieved +- Query operations return correct results +- Server configuration accessible ## Common Issues to Watch For -- **Connection String Format**: Different formats for different database types -- **Authentication Failures**: Credential or permission issues -- **Firewall Blocking**: Access blocked by firewall rules -- **Partition Key Issues**: Cosmos DB partition key mismatches -- **Query Syntax**: Different SQL dialects across databases -- **Resource Limits**: RU limits (Cosmos), connection limits (SQL/PostgreSQL) -- **Timeout Issues**: Long-running queries -- **Case Sensitivity**: PostgreSQL vs SQL Server differences +| Issue | Description | Resolution | +|-------|-------------|------------| +| **Authentication Failures** | MCP tool can't connect to database | Verify `az login` is active and you have RBAC permissions | +| **Firewall Blocking** | Access denied errors | Add your IP to firewall rules using Azure CLI/Portal | +| **Partition Key Issues** | Cosmos DB query fails | Ensure partition key is included in queries or specify cross-partition | +| **Query Syntax Differences** | SQL queries fail | Use PostgreSQL syntax for Postgres, T-SQL for Azure SQL | +| **Resource Limits** | Timeout or throttling | Check RU limits (Cosmos), connection limits (SQL/PostgreSQL) | +| **Case Sensitivity** | Query returns no results | PostgreSQL is case-sensitive; use proper casing or ILIKE | +| **Missing Resources** | MCP tool shows empty lists | Verify resources exist and are in the correct subscription | + +## What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output) +- [ ] Expected vs actual results +- [ ] Error messages (if any) +- [ ] Database type and version +- [ ] Screenshots of unexpected behavior ## Related Resources - [Azure Cosmos DB Documentation](https://learn.microsoft.com/azure/cosmos-db/) - [Azure Database for PostgreSQL](https://learn.microsoft.com/azure/postgresql/) +- [Azure Database for MySQL](https://learn.microsoft.com/azure/mysql/) - [Azure SQL Database](https://learn.microsoft.com/azure/azure-sql/) -- [Database Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md#azure-database-for-mysql) +- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) +## 💡 Quick Reference: Supported MCP Tools + +### Cosmos DB +- `azmcp_cosmos_account_list` - List accounts +- `azmcp_cosmos_database_list` - List databases +- `azmcp_cosmos_database_container_list` - List containers +- `azmcp_cosmos_database_container_item_query` - Query items + +### PostgreSQL +- `azmcp_postgres_server_list` - List servers +- `azmcp_postgres_server_config_get` - Get configuration +- `azmcp_postgres_server_param_get` - Get specific parameter +- `azmcp_postgres_database_list` - List databases +- `azmcp_postgres_table_list` - List tables +- `azmcp_postgres_table_schema_get` - Get table schema +- `azmcp_postgres_database_query` - Execute SELECT queries + +### MySQL +- `azmcp_mysql_server_list` - List servers +- `azmcp_mysql_server_config_get` - Get configuration +- `azmcp_mysql_server_param_get` - Get specific parameter +- `azmcp_mysql_database_list` - List databases +- `azmcp_mysql_table_list` - List tables +- `azmcp_mysql_table_schema_get` - Get table schema +- `azmcp_mysql_database_query` - Execute SELECT queries + +### Azure SQL +- `azmcp_sql_server_list` - List servers +- `azmcp_sql_server_show` - Get server details +- `azmcp_sql_db_list` - List databases +- `azmcp_sql_db_show` - Get database details +- `azmcp_sql_elastic-pool_list` - List elastic pools +- `azmcp_sql_server_firewall-rule_list` - List firewall rules +- `azmcp_sql_server_entra-admin_list` - List Entra admins + +--- + **Next**: [Deployment Scenarios Testing](deployment.md) diff --git a/docs/bug-bash/scenarios/deployment.md b/docs/bug-bash/scenarios/deployment.md index b2b2653472..f0fd4d6637 100644 --- a/docs/bug-bash/scenarios/deployment.md +++ b/docs/bug-bash/scenarios/deployment.md @@ -1,324 +1,400 @@ -# 🚢 Deployment Scenarios Testing +# 🚀 Deployment Scenarios Testing -Test Azure MCP Server's deployment guidance capabilities, CI/CD integration recommendations, and deployment planning features. +> **⚠️ MCP Tool Support Notice** +> Azure MCP Server provides **deployment planning guidance, CI/CD pipeline recommendations, and application log retrieval** capabilities. Actual resource creation and application deployment require Azure CLI, Portal, or CI/CD tools. This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. -## Objectives +## 🎯 Objectives -- Test application deployment workflows -- Verify deployment guidance and automation -- Test multi-environment deployments -- Validate deployment monitoring -- Test rollback and recovery procedures -- Verify CI/CD integration guidance +- Test deployment planning and guidance from Azure MCP Server +- Verify CI/CD pipeline recommendation accuracy +- Test application log retrieval and analysis +- Validate deployment plan generation for various project types -## Prerequisites +## ✅ Prerequisites - [ ] Azure MCP Server installed and configured - [ ] Azure CLI installed (`az --version`) +- [ ] Azure Developer CLI installed (`azd version`) - [ ] Authenticated to Azure (`az login`) - [ ] Active Azure subscription -- [ ] Sample application ready for deployment - [ ] GitHub Copilot with Agent mode enabled -## Test Scenarios +--- -### Scenario 1: Simple Application Deployment +## 📋 Scenario 1: Application Deployment Planning End-to-End -**Objective**: Test basic deployment workflow for a simple application +**Objective**: Complete workflow for getting deployment guidance and executing deployment -#### Phase 1: Get Deployment Guidance +### Step 1: Setup Sample Application (Local - Not MCP) -1. **Ask for deployment plan**: - ``` - I have a Node.js Express application. Create a plan to deploy this - application to Azure. - ``` +> **External Setup Required**: Create a sample Node.js application locally -2. **Verify deployment plan includes**: - - [ ] Resource requirements (App Service, Storage, etc.) - - [ ] Configuration steps - - [ ] Deployment method recommendations - - [ ] Post-deployment verification steps +```bash +# Create a sample Node.js application +mkdir bugbash-deploy-app && cd bugbash-deploy-app -3. **Get Azure CLI deployment commands**: - ``` - Generate Azure CLI commands to deploy my Node.js app to App Service - ``` +# Initialize Node.js project +npm init -y -4. **Get infrastructure as code**: - ``` - Generate a Bicep template to deploy my application infrastructure - ``` +# Install Express +npm install express -**Verify**: -- [ ] Deployment plan is comprehensive -- [ ] CLI commands are correct -- [ ] Infrastructure code is valid -- [ ] Steps are in logical order - -#### Phase 2: Execute Deployment - -5. **Deploy application**: - ``` - Deploy my Node.js application to Azure App Service 'bugbash-app-' - in resource group '' - ``` - -6. **Monitor deployment**: - ``` - Show me the deployment status for app service 'bugbash-app-' - ``` - -7. **Verify deployment**: - ``` - Check if my app is running on 'bugbash-app-' - ``` - -8. **Get application URL**: - ``` - What is the URL for my deployed application? - ``` +# Create simple server (index.js) +cat > index.js << 'EOF' +const express = require('express'); +const app = express(); +const port = process.env.PORT || 3000; -**Verify**: -- [ ] Deployment completes successfully -- [ ] Application is accessible -- [ ] URL is correct -- [ ] App responds to requests +app.get('/', (req, res) => { + res.json({ message: 'Hello from Azure!', timestamp: new Date() }); +}); -**Expected Results**: -- Deployment plan provided -- Infrastructure created -- Application deployed -- App is accessible and functional +app.listen(port, () => { + console.log(`Server running on port ${port}`); +}); +EOF ---- +# Create package.json start script +npm pkg set scripts.start="node index.js" +``` -### Scenario 2: Multi-Environment Deployment - -**Objective**: Test deployment across development, staging, and production environments - -#### Phase 1: Create Environment Infrastructure - -1. **Create development environment**: - ``` - Create Azure resources for a development environment: - - Resource group: 'bugbash-dev-rg' - - App Service: 'bugbash-app-dev' - - SQL Database: 'bugbash-db-dev' (Basic tier) - - Storage account: 'bugbashdevstore' - - Region: East US - ``` - -2. **Create staging environment**: - ``` - Create Azure resources for a staging environment: - - Resource group: 'bugbash-staging-rg' - - App Service: 'bugbash-app-staging' - - SQL Database: 'bugbash-db-staging' (Standard tier) - - Storage account: 'bugbashstagingstore' - - Region: East US - ``` - -3. **Create production environment**: - ``` - Create Azure resources for a production environment: - - Resource group: 'bugbash-prod-rg' - - App Service: 'bugbash-app-prod' (Premium tier) - - SQL Database: 'bugbash-db-prod' (Standard tier with geo-replication) - - Storage account: 'bugbashprodstore' with redundancy - - Region: East US - ``` +### Step 2: Get Deployment Plan with Azure MCP Server -**Verify**: -- [ ] All three environments created -- [ ] Appropriate tiers for each environment -- [ ] Naming conventions consistent -- [ ] Resources properly tagged - -#### Phase 2: Deploy to Environments - -4. **Deploy to development**: - ``` - Deploy my application to the development environment - (bugbash-app-dev) with debug settings enabled - ``` - -5. **Deploy to staging**: - ``` - Deploy my application to the staging environment - (bugbash-app-staging) with production-like settings - ``` - -6. **Verify staging deployment**: - ``` - Test my application in the staging environment and verify all - features work correctly - ``` - -7. **Deploy to production**: - ``` - After successful staging tests, deploy my application to production - environment (bugbash-app-prod) - ``` +**2.1 Request deployment plan** (uses `azmcp_deploy_plan_get`): +``` +I have a Node.js Express application in folder 'bugbash-deploy-app'. Create a deployment plan to deploy this to Azure App Service. +``` **Verify**: -- [ ] Each environment has correct configuration -- [ ] Deployments succeed in order -- [ ] Environment-specific settings applied -- [ ] Production deployment after verification +- [ ] Tool invoked: `azmcp_deploy_plan_get` +- [ ] Deployment plan includes App Service target +- [ ] Runtime configuration identified (Node.js) +- [ ] Provisioning tool recommendation provided (Bicep, Terraform, or azd) +- [ ] Project structure analysis included -**Expected Results**: -- All environments created with appropriate configurations -- Deployments succeed to each environment -- Progressive deployment strategy followed -- Each environment is accessible and functional +**2.2 Alternative phrasing**: +``` +Show me how to deploy my Node.js app in 'bugbash-deploy-app' to Azure +``` ---- +### Step 3: Get CI/CD Pipeline Guidance with Azure MCP Server -### Scenario 3: CI/CD Pipeline Guidance +**3.1 Get pipeline guidance** (uses `azmcp_deploy_pipeline_guidance_get`): +``` +How do I set up a CI/CD pipeline to automatically deploy my application to Azure App Service? +``` -**Objective**: Test guidance for setting up continuous integration and deployment +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_pipeline_guidance_get` +- [ ] GitHub Actions workflow guidance provided +- [ ] Azure DevOps pipeline guidance included +- [ ] Deployment credentials setup explained +- [ ] Best practices mentioned -#### Phase 1: Pipeline Recommendations +**3.2 Request specific pipeline configuration**: +``` +Generate GitHub Actions workflow guidance for deploying my Node.js app to Azure App Service with staging and production environments +``` -1. **Get CI/CD guidance**: - ``` - How can I create a CI/CD pipeline to deploy this application to Azure? - ``` +**Verify**: +- [ ] Multi-environment deployment strategy explained +- [ ] Approval workflows mentioned +- [ ] Environment-specific configuration guidance provided + +### Step 4: Deploy Application (Azure CLI/azd - Not MCP) + +> **External Deployment Required**: Use Azure CLI or azd to deploy + +```bash +# Create resource group +az group create --name bugbash-deploy-rg --location eastus + +# Create App Service plan +az appservice plan create \ + --name bugbash-plan \ + --resource-group bugbash-deploy-rg \ + --sku B1 \ + --is-linux + +# Create Web App +az webapp create \ + --name bugbash-app-$RANDOM \ + --resource-group bugbash-deploy-rg \ + --plan bugbash-plan \ + --runtime "NODE:18-lts" + +# Deploy application +cd bugbash-deploy-app +zip -r app.zip . +az webapp deployment source config-zip \ + --resource-group bugbash-deploy-rg \ + --name \ + --src app.zip +``` + +### Step 5: View Application Logs with Azure MCP Server + +**5.1 Get application logs** (uses `azmcp_deploy_app_logs_get`): +``` +Show me the logs for my application in Azure environment 'bugbash-deploy-rg' +``` -2. **Get GitHub Actions workflow**: - ``` - Generate a GitHub Actions workflow to: - - Build my Node.js application - - Run tests - - Deploy to Azure App Service - - Deploy to staging first, then production after approval - ``` +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_app_logs_get` +- [ ] Application logs retrieved +- [ ] Log timestamps shown +- [ ] Log content is readable -3. **Get Azure DevOps pipeline**: - ``` - Generate an Azure DevOps pipeline YAML for deploying my application - ``` +**5.2 Request recent logs**: +``` +Show me the last 50 log entries for my deployed application +``` **Verify**: -- [ ] CI/CD recommendations provided -- [ ] GitHub Actions workflow is valid -- [ ] Azure DevOps pipeline is complete -- [ ] Best practices included +- [ ] Limit parameter works +- [ ] Most recent logs displayed first +- [ ] Startup messages visible -#### Phase 2: Pipeline Configuration +### Step 6: Get Architecture Diagram with Azure MCP Server -4. **Configure deployment credentials**: - ``` - How do I configure deployment credentials for GitHub Actions to - deploy to Azure? - ``` +**6.1 Generate architecture diagram** (uses `azmcp_deploy_architecture_diagram_generate`): +``` +Generate a Mermaid architecture diagram for my deployment showing the App Service, application topology, and resource dependencies +``` -5. **Set up secrets**: - ``` - What secrets do I need to configure for automated deployment to Azure? - ``` +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_architecture_diagram_generate` +- [ ] Mermaid diagram generated +- [ ] Components clearly shown +- [ ] Relationships between resources displayed -6. **Configure service principal**: - ``` - Help me create a service principal for automated deployments with - minimal required permissions - ``` +### Step 7: Cleanup (External - Not MCP) -**Verify**: -- [ ] Credential setup instructions clear -- [ ] Secret requirements documented -- [ ] Service principal guidance provided -- [ ] Security best practices mentioned +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-deploy-rg --yes --no-wait +``` **Expected Results**: -- Complete CI/CD guidance provided -- Pipeline templates generated -- Security configuration documented -- Ready to implement automated deployments +- ✅ Deployment plan guidance accurate and actionable +- ✅ CI/CD pipeline recommendations comprehensive +- ✅ Application logs successfully retrieved +- ✅ Architecture diagram generated correctly -### Scenario 4: Deployment Monitoring and Logging +--- -**Objective**: Test deployment monitoring and log analysis capabilities +## 📋 Scenario 2: Infrastructure-as-Code Deployment Guidance End-to-End + +**Objective**: Complete workflow for getting IaC recommendations and deployment guidance + +### Step 1: Setup Project Structure (Local - Not MCP) + +> **External Setup Required**: Create a sample project with multiple services + +```bash +# Create multi-service project +mkdir bugbash-fullstack && cd bugbash-fullstack +mkdir frontend backend + +# Frontend (React app - simplified) +cat > frontend/package.json << 'EOF' +{ + "name": "frontend", + "version": "1.0.0", + "scripts": { + "build": "echo 'Building React app'" + } +} +EOF + +# Backend (Node.js API) +cat > backend/package.json << 'EOF' +{ + "name": "backend", + "version": "1.0.0", + "main": "server.js", + "scripts": { + "start": "node server.js" + }, + "dependencies": { + "express": "^4.18.0" + } +} +EOF + +cat > backend/server.js << 'EOF' +const express = require('express'); +const app = express(); +app.get('/api/health', (req, res) => res.json({ status: 'healthy' })); +app.listen(process.env.PORT || 8080); +EOF +``` + +### Step 2: Get IaC Generation Rules with Azure MCP Server + +**2.1 Request IaC rules for resources** (uses `azmcp_deploy_iac_rules_get`): +``` +Show me the infrastructure-as-code generation rules for deploying Azure App Service and Azure SQL Database using Bicep +``` -#### Phase 1: Monitor Deployment +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_iac_rules_get` +- [ ] Bicep rules provided for App Service +- [ ] Bicep rules provided for Azure SQL Database +- [ ] Best practices included +- [ ] Parameter recommendations shown + +**2.2 Get rules for different deployment tools**: +``` +Show me the IaC rules for Azure Storage Account and Cosmos DB using Terraform +``` -1. **Check deployment history**: - ``` - Show me the deployment history for app service 'bugbash-app-prod' - ``` +**Verify**: +- [ ] Terraform-specific rules returned +- [ ] Resource type rules accurate +- [ ] Configuration recommendations included -2. **Get deployment logs**: - ``` - Show me the deployment logs for the latest deployment to - 'bugbash-app-prod' - ``` +### Step 3: Get Deployment Plan for Multi-Service App with Azure MCP Server -3. **Check application logs**: - ``` - Show me the application logs for 'bugbash-app-prod' from the last - 30 minutes - ``` +**3.1 Request deployment plan** (uses `azmcp_deploy_plan_get`): +``` +I have a full-stack application with a React frontend in 'frontend' folder and Node.js API in 'backend' folder. Create a deployment plan for Azure with Static Web App for frontend and App Service for backend. +``` **Verify**: -- [ ] Deployment history retrieved -- [ ] Logs are accessible -- [ ] Log content is useful -- [ ] Timestamps are accurate +- [ ] Tool invoked: `azmcp_deploy_plan_get` +- [ ] Frontend deployment to Static Web App recommended +- [ ] Backend deployment to App Service recommended +- [ ] Service communication strategy included +- [ ] Infrastructure dependencies identified -#### Phase 2: Troubleshoot Failed Deployment +### Step 4: Get Pipeline Guidance for Full-Stack App with Azure MCP Server -4. **Simulate deployment failure**: - - Intentionally cause a deployment to fail - - Could be wrong runtime, missing dependency, etc. +**4.1 Request CI/CD guidance** (uses `azmcp_deploy_pipeline_guidance_get`): +``` +How do I set up a CI/CD pipeline for my full-stack application with separate build and deployment for frontend and backend? +``` -5. **Diagnose failure**: - ``` - My deployment to 'bugbash-app-staging' failed. Help me diagnose - what went wrong. - ``` +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_pipeline_guidance_get` +- [ ] Multi-stage pipeline guidance provided +- [ ] Frontend build process explained +- [ ] Backend build process explained +- [ ] Deployment orchestration guidance included -6. **Get logs for failed deployment**: - ``` - Show me the error logs for the failed deployment - ``` +**4.2 Request GitHub-specific guidance**: +``` +Generate GitHub Actions pipeline guidance for my full-stack app with automated testing and deployment to Azure +``` -7. **Get fix recommendations**: - ``` - Based on the error logs, what should I do to fix the deployment? - ``` +**Verify**: +- [ ] GitHub Actions workflow structure explained +- [ ] Testing stage recommendations included +- [ ] Deployment stages for both services covered +- [ ] Environment variables and secrets guidance provided + +### Step 5: Deploy Infrastructure (Azure CLI/Bicep - Not MCP) + +> **External Deployment Required**: Use Azure CLI with Bicep + +```bash +# Create resource group +az group create --name bugbash-fullstack-rg --location eastus + +# Create Static Web App for frontend +az staticwebapp create \ + --name bugbash-frontend-$RANDOM \ + --resource-group bugbash-fullstack-rg \ + --location eastus + +# Create App Service for backend +az appservice plan create \ + --name bugbash-backend-plan \ + --resource-group bugbash-fullstack-rg \ + --sku B1 \ + --is-linux + +az webapp create \ + --name bugbash-backend-$RANDOM \ + --resource-group bugbash-fullstack-rg \ + --plan bugbash-backend-plan \ + --runtime "NODE:18-lts" +``` + +### Step 6: Verify Deployment Guidance with Azure MCP Server + +**6.1 Get architecture diagram** (uses `azmcp_deploy_architecture_diagram_generate`): +``` +Generate a Mermaid architecture diagram showing my full-stack application with Static Web App frontend, App Service backend, and their connectivity +``` **Verify**: -- [ ] Failure is detected -- [ ] Error logs retrieved -- [ ] Root cause identified -- [ ] Fix suggestions provided +- [ ] Tool invoked: `azmcp_deploy_architecture_diagram_generate` +- [ ] Both services shown in diagram +- [ ] Communication flow displayed +- [ ] Azure resources clearly labeled + +### Step 7: Cleanup (External - Not MCP) + +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-fullstack-rg --yes --no-wait +``` **Expected Results**: -- Deployment monitoring works -- Logs are accessible -- Failures are diagnosed -- Actionable recommendations provided +- ✅ IaC rules provided for requested resource types +- ✅ Deployment plan includes multi-service architecture +- ✅ CI/CD guidance covers full-stack complexity +- ✅ Architecture diagrams accurately represent topology -## 🐛 Common Issues to Watch For +--- -- **Deployment Timeouts**: Long-running deployments -- **Configuration Errors**: Wrong settings causing failures -- **Resource Dependencies**: Resources deployed in wrong order -- **Authentication Issues**: Credential problems during deployment -- **Network Connectivity**: Firewall or network issues -- **Resource Quotas**: Hitting subscription limits -- **Naming Conflicts**: Duplicate resource names -- **Version Mismatches**: Incompatible runtime versions +## � Common Issues to Watch For -## Related Resources +| Issue | Description | Resolution | +|-------|-------------|------------| +| **Invalid Workspace Path** | Workspace folder not found | Verify the workspace folder path exists and is accessible | +| **Project Not Recognized** | MCP can't identify project type | Ensure package.json or project files exist in the folder | +| **Missing Dependencies** | Required project files missing | Check project structure has all necessary configuration files | +| **IaC Tool Mismatch** | Wrong provisioning tool recommended | Specify preferred tool (Bicep/Terraform) in the prompt | +| **Log Retrieval Fails** | Can't access application logs | Verify azd environment exists and is properly configured | +| **Architecture Diagram Errors** | Diagram generation fails | Ensure application topology is properly defined | +| **Pipeline Guidance Generic** | Recommendations too generic | Provide more context about your project structure and requirements | + +## 📝 What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output) +- [ ] Expected vs actual results +- [ ] Error messages (if any) +- [ ] Project type and structure +- [ ] Workspace folder path +- [ ] Screenshots of unexpected behavior + +## 📚 Related Resources - [Azure Deployment Center](https://learn.microsoft.com/azure/app-service/deploy-continuous-deployment) +- [Azure Developer CLI (azd)](https://learn.microsoft.com/azure/developer/azure-developer-cli/) - [GitHub Actions for Azure](https://learn.microsoft.com/azure/developer/github/github-actions) - [Azure DevOps Pipelines](https://learn.microsoft.com/azure/devops/pipelines/) -- [Deploy Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md#azure-deploy) +- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) +## 💡 Quick Reference: Supported MCP Tools + +### Deployment Planning +- `azmcp_deploy_plan_get` - Get deployment plan for a project +- `azmcp_deploy_iac_rules_get` - Get IaC generation rules for resource types +- `azmcp_deploy_architecture_diagram_generate` - Generate Mermaid architecture diagram + +### CI/CD Guidance +- `azmcp_deploy_pipeline_guidance_get` - Get CI/CD pipeline setup guidance + +### Application Logs +- `azmcp_deploy_app_logs_get` - Get application service logs for azd environment + --- **Next**: [Full Stack Applications Testing](full-stack-apps.md) diff --git a/docs/bug-bash/scenarios/full-stack-apps.md b/docs/bug-bash/scenarios/full-stack-apps.md index 245ba22df0..b0ca46c352 100644 --- a/docs/bug-bash/scenarios/full-stack-apps.md +++ b/docs/bug-bash/scenarios/full-stack-apps.md @@ -1,16 +1,14 @@ -# 🏗️ Full Stack Application Testing Scenario +# 🌐 Full Stack Application Testing Scenario -> **⚠️ READ FIRST**: [TESTING-SCOPE.md](../TESTING-SCOPE.md) explains what MCP tools can and cannot do. **This scenario requires BOTH Azure CLI and MCP tools**. Use Azure CLI to create databases and app services, then use MCP tools to query databases, get deployment guidance, add database connections, and monitor applications. - -Test Azure MCP Server's ability to help create complete applications with database backends and deploy them to Azure. +> **MCP Tool Support Notice** +> Azure MCP Server provides **database inspection, querying, and resource discovery** capabilities. Application code generation, resource creation, and deployment require external tools (Azure CLI, GitHub Copilot code generation, deployment tools). This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. ## Objectives -- Create a complete full-stack application -- Set up database backends (Cosmos DB, PostgreSQL, or Azure SQL) -- Deploy application to Azure -- Test CRUD operations against the database -- Verify end-to-end functionality +- Test MCP database inspection capabilities in full-stack context +- Verify resource discovery for deployed applications +- Test database querying for application backends +- Validate end-to-end workflow combining MCP and external tools ## Prerequisites @@ -18,422 +16,422 @@ Test Azure MCP Server's ability to help create complete applications with databa - [ ] Azure CLI installed (`az --version`) - [ ] Authenticated to Azure (`az login`) - [ ] Active Azure subscription -- [ ] Development tools installed (Node.js, Python, or .NET SDK) +- [ ] Development tools (Node.js or Python) - [ ] GitHub Copilot with Agent mode enabled -- [ ] VS Code or preferred IDE - -## Test Scenarios - -### Scenario 1: Node.js Web App with Cosmos DB - -**Objective**: Create a complete Node.js application with Cosmos DB backend - -**Steps**: - -#### Phase 1: Create the Database - -1. **Ask Copilot to create a Cosmos DB account**: - ``` - Create a Cosmos DB account named 'bugbash-cosmos-' in resource - group '' in East US with SQL API - ``` - -2. **Create a database**: - ``` - Create a database named 'TasksDB' in Cosmos DB account 'bugbash-cosmos-' - ``` - -3. **Create a container**: - ``` - Create a container named 'Tasks' in database 'TasksDB' with - partition key '/category' and 400 RU/s throughput - ``` - -4. **Add sample data**: - ``` - Add the following sample tasks to the Tasks container: - - { "id": "1", "title": "Buy groceries", "category": "personal", "completed": false } - - { "id": "2", "title": "Review PR", "category": "work", "completed": false } - - { "id": "3", "title": "Call dentist", "category": "personal", "completed": true } - ``` - -5. **Verify data**: - ``` - Query all tasks from the Tasks container in Cosmos DB - ``` - -#### Phase 2: Build the Application - -6. **Ask Copilot to create the application structure**: - ``` - Create a Node.js Express application for a task management system with: - - Express.js web framework - - Cosmos DB integration using @azure/cosmos - - RESTful API endpoints for: - * GET /api/tasks - list all tasks - * GET /api/tasks/:id - get a specific task - * POST /api/tasks - create a new task - * PUT /api/tasks/:id - update a task - * DELETE /api/tasks/:id - delete a task - - A simple HTML frontend with JavaScript - - Connection to Cosmos DB 'bugbash-cosmos-' - ``` - -7. **Review the generated code**: - - [ ] Check package.json dependencies - - [ ] Verify Cosmos DB connection configuration - - [ ] Review API endpoint implementations - - [ ] Check error handling - - [ ] Verify frontend HTML/JS - -8. **Ask for configuration guidance**: - ``` - How do I configure the Cosmos DB connection string in my Node.js - application securely? - ``` - -#### Phase 3: Test Locally - -9. **Install dependencies**: - ```bash - npm install - ``` - -10. **Set up environment variables**: - ``` - Get the Cosmos DB connection string for 'bugbash-cosmos-' - ``` - - Copy the connection string - - Create `.env` file with connection string - -11. **Run the application locally**: - ```bash - npm start - ``` - -12. **Test the API endpoints**: - - [ ] GET /api/tasks returns all tasks - - [ ] GET /api/tasks/1 returns specific task - - [ ] POST /api/tasks creates new task - - [ ] PUT /api/tasks/1 updates task - - [ ] DELETE /api/tasks/1 deletes task - -13. **Test the frontend**: - - [ ] Open http://localhost:3000 - - [ ] View task list - - [ ] Create a new task - - [ ] Edit a task - - [ ] Delete a task - - [ ] Verify changes in Cosmos DB - -#### Phase 4: Deploy to Azure - -14. **Create App Service resources**: - ``` - Create an Azure App Service for my Node.js application: - - Name: 'bugbash-taskapp-' - - Resource group: '' - - Runtime: Node.js 20 LTS - - Tier: B1 - - Region: East US - ``` - -15. **Configure application settings**: - ``` - Add the Cosmos DB connection string as an application setting - named 'COSMOS_CONNECTION_STRING' in app service 'bugbash-taskapp-' - ``` - -16. **Deploy the application**: - ``` - How do I deploy my Node.js application to Azure App Service - 'bugbash-taskapp-'? - ``` - - Follow the deployment instructions provided - -17. **Verify deployment**: - ``` - What is the URL for app service 'bugbash-taskapp-'? - ``` - - Open the URL in browser - - Test all CRUD operations - -**Expected Results**: -- Cosmos DB account and container created -- Application code generated correctly -- Local testing successful -- Deployment to Azure successful -- Production app works correctly -- Database operations work in production --- -### Scenario 2: Python Web App with PostgreSQL - -**Objective**: Create a Flask application with PostgreSQL backend - -**Steps**: - -#### Phase 1: Create PostgreSQL Database - -1. **Create PostgreSQL server**: - ``` - Create an Azure Database for PostgreSQL flexible server: - - Name: 'bugbash-postgres-' - - Resource group: '' - - Admin user: 'dbadmin' - - Password: '' - - Version: PostgreSQL 14 - - Tier: Burstable B1ms - - Region: East US - - Allow Azure services access - ``` - -2. **Create database**: - ``` - Create a database named 'inventory' on PostgreSQL server - 'bugbash-postgres-' - ``` - -3. **Create table schema**: - ``` - In PostgreSQL database 'inventory', create a table named 'products' with: - - id (serial primary key) - - name (varchar 100) - - description (text) - - price (decimal 10,2) - - stock (integer) - - created_at (timestamp) - ``` - -4. **Insert sample data**: - ``` - Insert sample products into the 'products' table: - - Laptop, High-performance laptop, 999.99, 10 - - Mouse, Wireless mouse, 29.99, 50 - - Keyboard, Mechanical keyboard, 79.99, 30 - ``` - -#### Phase 2: Build Python Application - -5. **Create Flask application**: - ``` - Create a Python Flask application for an inventory management system: - - Flask web framework - - psycopg2 for PostgreSQL connection - - API endpoints: - * GET /api/products - list all products - * GET /api/products/ - get product details - * POST /api/products - add new product - * PUT /api/products/ - update product - * DELETE /api/products/ - delete product - - Simple HTML frontend - - Connection to PostgreSQL 'bugbash-postgres-' - ``` - -6. **Review generated code**: - - [ ] Check requirements.txt - - [ ] Verify PostgreSQL connection - - [ ] Review API routes - - [ ] Check SQL queries - - [ ] Verify error handling - -#### Phase 3: Test Locally - -7. **Set up virtual environment**: - ```bash - python -m venv venv - source venv/bin/activate # On Windows: venv\Scripts\activate - pip install -r requirements.txt - ``` - -8. **Configure database connection**: - ``` - Get the PostgreSQL connection string for 'bugbash-postgres-' - ``` - - Create `.env` file with credentials - -9. **Run locally**: - ```bash - python app.py - ``` - -10. **Test endpoints**: - - [ ] List products - - [ ] Get product details - - [ ] Add new product - - [ ] Update product - - [ ] Delete product - -#### Phase 4: Deploy to Azure - -11. **Create App Service for Python**: - ``` - Create an Azure App Service for Python: - - Name: 'bugbash-inventory-' - - Runtime: Python 3.11 - - Tier: B1 - - Region: East US - ``` - -12. **Configure connection**: - ``` - Configure the PostgreSQL connection in app service settings - ``` - -13. **Deploy application**: - ``` - Deploy my Python Flask app to 'bugbash-inventory-' - ``` - -14. **Test production app**: - - Open app URL - - Verify all CRUD operations +## Scenario 1: Node.js App with Cosmos DB Backend End-to-End + +**Objective**: Complete workflow from database setup through querying with a deployed application + +### Step 1: Create Database Resources (External - Not MCP) + +> **External Setup Required**: Use Azure CLI to create Cosmos DB resources + +```bash +# Create resource group +az group create --name bugbash-fullstack-rg --location eastus + +# Create Cosmos DB account +az cosmosdb create \ + --name bugbash-cosmos-$RANDOM \ + --resource-group bugbash-fullstack-rg \ + --locations regionName=eastus + +# Create database +az cosmosdb sql database create \ + --account-name \ + --resource-group bugbash-fullstack-rg \ + --name TasksDB + +# Create container +az cosmosdb sql container create \ + --account-name \ + --resource-group bugbash-fullstack-rg \ + --database-name TasksDB \ + --name Tasks \ + --partition-key-path "/category" \ + --throughput 400 +``` + +**Insert sample data** using Azure Portal Data Explorer: +```json +{"id": "1", "title": "Buy groceries", "category": "personal", "completed": false} +{"id": "2", "title": "Review PR", "category": "work", "completed": false} +{"id": "3", "title": "Call dentist", "category": "personal", "completed": true} +``` + +### Step 2: Discover Database Resources with Azure MCP Server + +**2.1 List Cosmos DB accounts** (uses `azmcp_cosmos_account_list`): +``` +List all Cosmos DB accounts in my subscription +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_account_list` +- [ ] Your Cosmos DB account appears +- [ ] Account properties displayed + +**2.2 List databases** (uses `azmcp_cosmos_database_list`): +``` +List all databases in Cosmos DB account '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_database_list` +- [ ] 'TasksDB' database listed + +**2.3 List containers** (uses `azmcp_cosmos_database_container_list`): +``` +List all containers in database 'TasksDB' for Cosmos DB account '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_database_container_list` +- [ ] 'Tasks' container listed +- [ ] Partition key '/category' shown + +### Step 3: Query Application Data with Azure MCP Server + +**3.1 Query all tasks** (uses `azmcp_cosmos_database_container_item_query`): +``` +Query all items from container 'Tasks' in database 'TasksDB' for Cosmos DB account '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_database_container_item_query` +- [ ] All 3 sample tasks returned +- [ ] Task properties visible (id, title, category, completed) + +**3.2 Query completed tasks**: +``` +Show me all completed tasks from container 'Tasks' in database 'TasksDB' for Cosmos DB account '' +``` + +**Verify**: +- [ ] Only completed tasks returned +- [ ] Filter correctly applied + +**3.3 Query by category**: +``` +Show me all work tasks from container 'Tasks' in database 'TasksDB' for Cosmos DB account '' +``` + +**Verify**: +- [ ] Only work category tasks returned +- [ ] Category filter works + +### Step 4: Build and Deploy Application (External Tools - Not MCP) + +> **External Development Required**: Use GitHub Copilot for code generation + +**Generate application code** (use GitHub Copilot Chat in VS Code): +``` +Create a Node.js Express application for a task management system with: +- Express.js web framework +- Cosmos DB integration using @azure/cosmos +- RESTful API endpoints for CRUD operations +- Simple HTML frontend +- Connection to Cosmos DB '' +``` + +**Deploy to Azure** (use Azure CLI): +```bash +# Create App Service +az webapp create \ + --name bugbash-taskapp-$RANDOM \ + --resource-group bugbash-fullstack-rg \ + --plan bugbash-plan \ + --runtime "NODE:18-lts" + +# Configure connection string +az webapp config appsettings set \ + --name \ + --resource-group bugbash-fullstack-rg \ + --settings COSMOS_CONNECTION_STRING="" + +# Deploy code (zip deployment) +cd task-app +zip -r app.zip . +az webapp deployment source config-zip \ + --resource-group bugbash-fullstack-rg \ + --name \ + --src app.zip +``` + +### Step 5: Verify Deployed Application with Azure MCP Server + +**5.1 Get App Service details** (uses `azmcp_appservice_get`): +``` +Show me details for App Service '' in resource group 'bugbash-fullstack-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_appservice_get` +- [ ] App Service properties shown +- [ ] Runtime and configuration visible +- [ ] App URL displayed + +**5.2 Verify data through application**: +- Open the App Service URL in browser +- Test the task management interface +- Create/read/update/delete tasks +- Verify operations work end-to-end + +### Step 6: Query Updated Data with Azure MCP Server + +**6.1 Query database to verify application changes** (uses `azmcp_cosmos_database_container_item_query`): +``` +Query all items from container 'Tasks' in database 'TasksDB' to see changes made through my application +``` + +**Verify**: +- [ ] New tasks created via app are visible +- [ ] Updated tasks show modifications +- [ ] Deleted tasks are gone +- [ ] Database accurately reflects app operations + +### Step 7: Cleanup (External - Not MCP) + +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-fullstack-rg --yes --no-wait +``` **Expected Results**: -- PostgreSQL database created -- Python application generated -- Local testing successful -- Azure deployment successful -- Production app functional +- Database discovery works correctly +- Application data query succeeds +- App Service inspection functional +- End-to-end workflow complete +- Data changes visible through MCP queries --- -### Scenario 3: .NET Web App with Azure SQL - -**Objective**: Create an ASP.NET Core application with Azure SQL backend - -**Steps**: - -#### Phase 1: Create Azure SQL Database - -1. **Create SQL server**: - ``` - Create an Azure SQL server: - - Name: 'bugbash-sqlserver-' - - Resource group: '' - - Admin login: 'sqladmin' - - Password: '' - - Region: East US - - Allow Azure services - ``` - -2. **Create database**: - ``` - Create a database named 'CustomerDB' on SQL server - 'bugbash-sqlserver-' with Basic tier - ``` - -3. **Create tables**: - ``` - Create the following tables in CustomerDB: - - Customers table: - - CustomerId (int primary key identity) - - FirstName (nvarchar 50) - - LastName (nvarchar 50) - - Email (nvarchar 100) - - Phone (nvarchar 20) - - CreatedDate (datetime) - - Orders table: - - OrderId (int primary key identity) - - CustomerId (int foreign key) - - OrderDate (datetime) - - TotalAmount (decimal 18,2) - - Status (nvarchar 20) - ``` - -#### Phase 2: Build .NET Application - -4. **Create ASP.NET Core Web API**: - ``` - Create an ASP.NET Core 8.0 Web API application for customer management: - - Entity Framework Core for database access - - Controllers for Customers and Orders - - API endpoints following REST conventions - - Connection to Azure SQL 'bugbash-sqlserver-' - - Swagger/OpenAPI documentation - - Include a simple React frontend - ``` - -5. **Review code structure**: - - [ ] Check Models (Customer, Order) - - [ ] Review DbContext configuration - - [ ] Verify Controllers - - [ ] Check API routes - - [ ] Review frontend React components - -#### Phase 3: Test Locally - -6. **Configure connection string**: - ``` - Get the connection string for Azure SQL database 'CustomerDB' - ``` - - Add to appsettings.Development.json - -7. **Run migrations**: - ```bash - dotnet ef database update - ``` - -8. **Start application**: - ```bash - dotnet run - ``` - -9. **Test API**: - - Open Swagger UI: https://localhost:5001/swagger - - [ ] Test GET /api/customers - - [ ] Test POST /api/customers - - [ ] Test GET /api/orders - - [ ] Test POST /api/orders - -#### Phase 4: Deploy to Azure - -10. **Create App Service**: - ``` - Create an Azure App Service for .NET 8.0: - - Name: 'bugbash-customerapi-' - - Runtime: .NET 8 - - Tier: B1 - ``` - -11. **Configure SQL connection**: - ``` - Add SQL connection string to app service configuration - ``` - -12. **Deploy application**: - ``` - Deploy my .NET application to Azure App Service - ``` - -13. **Verify deployment**: - - Test API endpoints in production - - Verify database connectivity - - Test frontend functionality +## Scenario 2: Python Flask App with PostgreSQL Backend End-to-End + +**Objective**: Complete workflow for Python application with relational database backend + +### Step 1: Create PostgreSQL Resources (External - Not MCP) + +> **External Setup Required**: Use Azure CLI to create PostgreSQL server + +```bash +# Create resource group +az group create --name bugbash-flask-rg --location eastus + +# Create PostgreSQL flexible server +az postgres flexible-server create \ + --name bugbash-postgres-$RANDOM \ + --resource-group bugbash-flask-rg \ + --location eastus \ + --admin-user dbadmin \ + --admin-password \ + --sku-name Standard_B1ms \ + --tier Burstable \ + --version 14 \ + --storage-size 32 \ + --public-access 0.0.0.0 + +# Create database +az postgres flexible-server db create \ + --resource-group bugbash-flask-rg \ + --server-name \ + --database-name inventory + +# Create table and insert data (use Azure Portal Query Editor or psql) +# CREATE TABLE products ( +# id SERIAL PRIMARY KEY, +# name VARCHAR(100), +# description TEXT, +# price DECIMAL(10,2), +# stock INTEGER +# ); +# +# INSERT INTO products (name, description, price, stock) VALUES +# ('Laptop', 'High-performance laptop', 999.99, 10), +# ('Mouse', 'Wireless mouse', 29.99, 50), +# ('Keyboard', 'Mechanical keyboard', 79.99, 30); +``` + +### Step 2: Discover PostgreSQL Resources with Azure MCP Server + +**2.1 List PostgreSQL servers** (uses `azmcp_postgres_server_list`): +``` +List all PostgreSQL servers in my subscription +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_server_list` +- [ ] Your PostgreSQL server appears +- [ ] Server details shown + +**2.2 List databases** (uses `azmcp_postgres_database_list`): +``` +List all databases in PostgreSQL server '' in resource group 'bugbash-flask-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_database_list` +- [ ] 'inventory' database listed + +**2.3 List tables** (uses `azmcp_postgres_table_list`): +``` +List all tables in database 'inventory' on server '' in resource group 'bugbash-flask-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_table_list` +- [ ] 'products' table listed + +### Step 3: Inspect Table Schema with Azure MCP Server + +**3.1 Get table schema** (uses `azmcp_postgres_table_schema_get`): +``` +Show me the schema of table 'products' in database 'inventory' on server '' in resource group 'bugbash-flask-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_table_schema_get` +- [ ] All columns listed (id, name, description, price, stock) +- [ ] Data types shown +- [ ] Primary key visible + +### Step 4: Query Application Data with Azure MCP Server + +**4.1 Query all products** (uses `azmcp_postgres_database_query`): +``` +Show me all products in the 'products' table in database 'inventory' on server '' in resource group 'bugbash-flask-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_database_query` +- [ ] All 3 products returned +- [ ] Product details visible + +**4.2 Query with filter**: +``` +Show me all products with price less than 100 in database 'inventory' on server '' in resource group 'bugbash-flask-rg' +``` + +**Verify**: +- [ ] Only Mouse and Keyboard returned +- [ ] Price filter works correctly + +### Step 5: Build and Deploy Flask App (External Tools - Not MCP) + +> **External Development Required**: Use GitHub Copilot for code generation + +**Generate Flask application** (use GitHub Copilot Chat): +``` +Create a Python Flask application for inventory management with: +- Flask web framework +- psycopg2 for PostgreSQL +- API endpoints for products CRUD +- Simple HTML frontend +- Connection to PostgreSQL '' +``` + +**Deploy to Azure** (use Azure CLI): +```bash +# Create App Service Plan +az appservice plan create \ + --name bugbash-flask-plan \ + --resource-group bugbash-flask-rg \ + --sku B1 \ + --is-linux + +# Create Web App +az webapp create \ + --name bugbash-inventory-$RANDOM \ + --resource-group bugbash-flask-rg \ + --plan bugbash-flask-plan \ + --runtime "PYTHON:3.11" + +# Configure database connection +az webapp config appsettings set \ + --name \ + --resource-group bugbash-flask-rg \ + --settings DATABASE_URL="" + +# Deploy application +cd inventory-app +zip -r app.zip . +az webapp deployment source config-zip \ + --resource-group bugbash-flask-rg \ + --name \ + --src app.zip +``` + +### Step 6: Verify Deployed Application with Azure MCP Server + +**6.1 Get App Service details** (uses `azmcp_appservice_get`): +``` +Show me details for App Service '' in resource group 'bugbash-flask-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_appservice_get` +- [ ] Flask app details shown +- [ ] Python runtime visible + +**6.2 Test application**: +- Open the App Service URL +- Test product CRUD operations +- Verify database connectivity + +### Step 7: Query Updated Data with Azure MCP Server + +**7.1 Verify application changes in database** (uses `azmcp_postgres_database_query`): +``` +Show me all products in database 'inventory' to verify changes made through my Flask application +``` + +**Verify**: +- [ ] New products created via app visible +- [ ] Updated products show changes +- [ ] Deleted products are gone +- [ ] Database state matches app operations + +### Step 8: Cleanup (External - Not MCP) + +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-flask-rg --yes --no-wait +``` **Expected Results**: -- Azure SQL database created -- .NET application generated -- EF Core migrations work -- Local testing successful -- Azure deployment successful -- Production API operational +- PostgreSQL discovery works +- Table schema inspection accurate +- Database querying functional +- App Service inspection works +- End-to-end workflow complete + +--- ## Common Issues to Watch For -- **Database Connection Issues**: Connection string format, firewall rules -- **Authentication/Authorization**: Missing database credentials -- **CORS Errors**: Frontend can't communicate with backend -- **Port Conflicts**: Port already in use locally -- **Dependency Issues**: Missing or incompatible packages -- **Environment Variables**: Not properly configured in production -- **Database Migrations**: Schema changes not applied -- **Build Failures**: Platform-specific compilation issues -- **Timeout Issues**: Long-running database operations -- **Connection Pool Exhaustion**: Too many concurrent connections +| Issue | Description | Resolution | +|-------|-------------|------------| +| **Database Connection Failures** | Can't connect to database | Verify firewall rules allow your IP and Azure services | +| **Authentication Errors** | Credential issues | Check connection string format and credentials | +| **Query Syntax Errors** | SQL queries fail | Use correct syntax for database type (T-SQL vs PostgreSQL) | +| **App Service Configuration** | App can't connect to database | Verify connection string in App Service settings | +| **CORS Errors** | Frontend can't reach API | Configure CORS in App Service or API code | +| **Missing Resources** | MCP can't find resources | Ensure resources exist in specified subscription/resource group | +| **Partition Key Issues** | Cosmos DB queries fail | Include partition key or enable cross-partition queries | + +## What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output) +- [ ] Expected vs actual results +- [ ] Error messages (if any) +- [ ] Database type and resource names +- [ ] Query that was attempted +- [ ] Screenshots of unexpected behavior ## Related Resources @@ -441,7 +439,40 @@ Test Azure MCP Server's ability to help create complete applications with databa - [Azure Cosmos DB Documentation](https://learn.microsoft.com/azure/cosmos-db/) - [Azure PostgreSQL Documentation](https://learn.microsoft.com/azure/postgresql/) - [Azure SQL Documentation](https://learn.microsoft.com/azure/azure-sql/) -- [Deployment Testing](deployment.md) +- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) -**Next**: [Infra As Code](infra-as-code.md) \ No newline at end of file +## Quick Reference: Supported MCP Tools + +### Cosmos DB +- `azmcp_cosmos_account_list` - List Cosmos DB accounts +- `azmcp_cosmos_database_list` - List databases +- `azmcp_cosmos_database_container_list` - List containers +- `azmcp_cosmos_database_container_item_query` - Query container items + +### PostgreSQL +- `azmcp_postgres_server_list` - List PostgreSQL servers +- `azmcp_postgres_database_list` - List databases +- `azmcp_postgres_table_list` - List tables +- `azmcp_postgres_table_schema_get` - Get table schema +- `azmcp_postgres_database_query` - Execute SELECT queries + +### MySQL +- `azmcp_mysql_server_list` - List MySQL servers +- `azmcp_mysql_database_list` - List databases +- `azmcp_mysql_table_list` - List tables +- `azmcp_mysql_table_schema_get` - Get table schema +- `azmcp_mysql_database_query` - Execute SELECT queries + +### Azure SQL +- `azmcp_sql_server_list` - List SQL servers +- `azmcp_sql_db_list` - List databases +- `azmcp_sql_db_show` - Get database details + +### App Service +- `azmcp_appservice_get` - Get App Service details + +--- + +**Next**: [Infrastructure as Code Testing](infra-as-code.md) \ No newline at end of file diff --git a/docs/bug-bash/scenarios/infra-as-code.md b/docs/bug-bash/scenarios/infra-as-code.md index ff3b122cca..dc17b6451f 100644 --- a/docs/bug-bash/scenarios/infra-as-code.md +++ b/docs/bug-bash/scenarios/infra-as-code.md @@ -1,171 +1,377 @@ # Infrastructure as Code Testing Scenario -Test Azure MCP Server's ability to generate and deploy infrastructure using Bicep templates and Azure resources. +> **MCP Tool Support Notice** +> Azure MCP Server provides **Bicep schema retrieval and IaC generation rules** capabilities. Actual Bicep template generation and infrastructure deployment require external tools (GitHub Copilot for code generation, Azure CLI/Bicep for deployment). This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. ## Objectives -- Generate Bicep templates for Azure resources -- Deploy infrastructure using generated templates -- Validate deployed resources -- Test different resource types -- Verify error handling during deployment +- Test Bicep schema retrieval for Azure resources +- Verify IaC generation rules and recommendations +- Test Terraform best practices guidance +- Validate end-to-end IaC workflow with MCP assistance ## Prerequisites - [ ] Azure MCP Server installed and configured - [ ] Azure CLI installed (`az --version`) +- [ ] Bicep CLI installed (`az bicep version`) - [ ] Authenticated to Azure (`az login`) - [ ] Active Azure subscription -- [ ] Resource group for testing (or permission to create one) - [ ] GitHub Copilot with Agent mode enabled -## Test Scenarios +--- -### Scenario 1: Generate Simple Storage Account Bicep Template +## Scenario 1: Bicep Schema and Template Generation End-to-End -**Objective**: Test basic Bicep template generation +**Objective**: Complete workflow using Bicep schema assistance and deployment -**Steps**: +### Step 1: Get Bicep Schema with Azure MCP Server -1. Open GitHub Copilot Chat in Agent mode +**1.1 Get storage account schema** (uses `azmcp_bicepschema_get`): +``` +Show me the Bicep schema for Azure Storage Account resource type +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_bicepschema_get` +- [ ] Schema definition returned +- [ ] Resource properties documented +- [ ] Property types shown +- [ ] Required vs optional properties identified -2. Use this prompt: - ``` - Generate a Bicep template to create an Azure Storage account named - 'bugbashstorage' in the 'eastus' region with Standard_LRS redundancy - ``` +**1.2 Get App Service schema**: +``` +Show me the Bicep schema for Microsoft.Web/sites resource type +``` + +**Verify**: +- [ ] App Service schema retrieved +- [ ] Configuration properties listed +- [ ] Schema is accurate and complete -3. **Verify**: - - [ ] Bicep template is generated - - [ ] Template includes storage account resource - - [ ] Properties match requested configuration - - [ ] Template syntax is valid +**1.3 Get SQL Database schema**: +``` +Get the Bicep schema for Azure SQL Database +``` -4. **Save the template** to `storage-account.bicep` +**Verify**: +- [ ] SQL Database schema returned +- [ ] All properties documented -5. **Deploy the template**: - ```bash - az deployment group create \ - --resource-group \ - --template-file storage-account.bicep - ``` +### Step 2: Get IaC Generation Rules with Azure MCP Server -6. **Verify deployment**: - - Ask Copilot: `"List my storage accounts in resource group "` - - Confirm the storage account appears +**2.1 Get rules for Storage Account** (uses `azmcp_deploy_iac_rules_get`): +``` +Show me the IaC generation rules for Azure Storage Account using Bicep +``` -**Expected Results**: -- Valid Bicep template generated -- Template deploys successfully -- Resource appears in Azure - -**Things to Check**: -- [ ] Template follows Bicep best practices -- [ ] Resource names are valid -- [ ] Parameters are properly defined -- [ ] Outputs are included (if applicable) - - -### Scenario 2: Generate Multi-Resource Infrastructure - -**Objective**: Test complex infrastructure generation - -**Steps**: - -1. Use this prompt: - ``` - Generate a Bicep template that creates: - - A storage account - - An App Service plan (B1 tier) - - A web app connected to the storage account - - An Application Insights resource - All resources should be in the East US region - ``` - -2. **Verify the template includes**: - - [ ] All requested resources - - [ ] Proper resource dependencies - - [ ] Connection strings/references - - [ ] Valid parameter definitions - -3. **Review the template** for: - - [ ] Naming conventions - - [ ] Resource locations - - [ ] SKU/tier selections - - [ ] Resource links - -4. **Deploy the template**: - ```bash - az deployment group create \ - --resource-group \ - --template-file multi-resource.bicep - ``` - -5. **Verify all resources** are created: - - Ask Copilot: `"List all resources in resource group "` - - Confirm all expected resources appear +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_iac_rules_get` +- [ ] Best practices provided +- [ ] Recommended parameters shown +- [ ] Configuration guidance included +- [ ] Security recommendations mentioned + +**2.2 Get rules for App Service**: +``` +Show me the IaC generation rules for Azure App Service and App Service Plan using Bicep +``` + +**Verify**: +- [ ] Rules for both resources provided +- [ ] Dependency relationships explained +- [ ] SKU recommendations included + +### Step 3: Generate Bicep Template (GitHub Copilot - Not MCP) + +> **External Tool Required**: Use GitHub Copilot for template generation + +**Generate Bicep template** (use GitHub Copilot Chat in VS Code): +``` +Generate a Bicep template to create: +- Azure Storage Account with Standard_LRS redundancy +- Blob container for data storage +- App Service Plan (B1 tier) +- App Service web app with storage connection + +Include parameters for resource names and location. +``` + +**Save template** as `infrastructure.bicep` + +### Step 4: Deploy Infrastructure (Azure CLI/Bicep - Not MCP) + +> **External Deployment Required**: Use Azure CLI + +```bash +# Create resource group +az group create --name bugbash-iac-rg --location eastus + +# Deploy Bicep template +az deployment group create \ + --resource-group bugbash-iac-rg \ + --template-file infrastructure.bicep \ + --parameters storageAccountName=bugbashstorage$RANDOM \ + appServiceName=bugbash-app-$RANDOM \ + location=eastus +``` + +### Step 5: Verify Deployment with Azure MCP Server + +**5.1 List storage accounts** (uses `azmcp_storage_account_get`): +``` +List all storage accounts in my subscription +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_storage_account_get` +- [ ] Your deployed storage account appears +- [ ] Account properties match template + +**5.2 List blob containers** (uses `azmcp_storage_blob_container_get`): +``` +List all containers in storage account '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_storage_blob_container_get` +- [ ] Container from template is listed + +**5.3 Get App Service details** (uses `azmcp_appservice_get`): +``` +Show me details for App Service '' in resource group 'bugbash-iac-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_appservice_get` +- [ ] App Service configuration shown +- [ ] Properties match template specification + +### Step 6: Cleanup (External - Not MCP) + +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-iac-rg --yes --no-wait +``` **Expected Results**: -- Complete Bicep template with all resources -- Proper dependencies between resources -- All resources deploy successfully -- Resources are correctly connected - -### Scenario 3: Test Bicep Best Practices - -**Objective**: Verify generated templates follow Azure best practices - -**Steps**: - -1. Use this prompt: - ``` - Show me the best practices for creating a Bicep template for a - storage account with blob containers - ``` - -2. **Review the guidance** provided - -3. Generate a template with best practices: - ``` - Generate a Bicep template for a storage account following best - practices, including: - - Parameters for names and locations - - Variables for common values - - Outputs for important values - - Proper resource dependencies - ``` - -4. **Verify the template includes**: - - [ ] Parameterized values - - [ ] Variables for reusable values - - [ ] Descriptive resource descriptions - - [ ] Outputs for key information - - [ ] Tags for resource management +- Bicep schema retrieval successful +- IaC generation rules comprehensive +- Template generation guided by schema +- Deployment verification through MCP tools + +--- + +## Scenario 2: Terraform Best Practices End-to-End + +**Objective**: Complete workflow using Terraform guidance and deployment + +### Step 1: Get Terraform Best Practices with Azure MCP Server + +**1.1 Get general Terraform best practices** (uses `azmcp_azureterraformbestpractices_get`): +``` +Show me Azure Terraform best practices for resource deployment +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_azureterraformbestpractices_get` +- [ ] Best practices document returned +- [ ] Security recommendations included +- [ ] State management guidance provided +- [ ] Naming conventions explained + +**1.2 Alternative phrasing**: +``` +What are the best practices for writing Terraform code for Azure? +``` + +**Verify**: +- [ ] Same tool invoked +- [ ] Comprehensive guidance provided + +### Step 2: Get IaC Rules for Terraform with Azure MCP Server + +**2.1 Get Terraform rules for resources** (uses `azmcp_deploy_iac_rules_get`): +``` +Show me the IaC generation rules for Azure Storage Account and Key Vault using Terraform +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_iac_rules_get` +- [ ] Terraform-specific rules provided +- [ ] Resource configuration guidance included +- [ ] Security settings highlighted + +**2.2 Get rules for compute resources**: +``` +Show me the Terraform IaC rules for Azure Virtual Machine and Virtual Network +``` + +**Verify**: +- [ ] VM and VNet rules provided +- [ ] Network configuration guidance included +- [ ] Security group recommendations shown + +### Step 3: Create Terraform Configuration (External Tool - Not MCP) + +> **External Tool Required**: Create Terraform files manually or use GitHub Copilot + +**Create `main.tf`**: +```hcl +terraform { + required_providers { + azurerm = { + source = "hashicorp/azurerm" + version = "~> 3.0" + } + } +} + +provider "azurerm" { + features {} +} + +resource "azurerm_resource_group" "main" { + name = "bugbash-terraform-rg" + location = "eastus" +} + +resource "azurerm_storage_account" "main" { + name = "bugbashterraform${random_string.suffix.result}" + resource_group_name = azurerm_resource_group.main.name + location = azurerm_resource_group.main.location + account_tier = "Standard" + account_replication_type = "LRS" + + tags = { + environment = "testing" + project = "bug-bash" + } +} + +resource "random_string" "suffix" { + length = 8 + special = false + upper = false +} + +output "storage_account_name" { + value = azurerm_storage_account.main.name +} +``` + +### Step 4: Deploy with Terraform (External Tool - Not MCP) + +> **External Deployment Required**: Use Terraform CLI + +```bash +# Initialize Terraform +terraform init + +# Plan deployment +terraform plan -out=tfplan + +# Apply configuration +terraform apply tfplan +``` + +### Step 5: Verify Deployment with Azure MCP Server + +**5.1 List storage accounts** (uses `azmcp_storage_account_get`): +``` +List all storage accounts in my subscription +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_storage_account_get` +- [ ] Terraform-deployed storage account appears +- [ ] Tags match configuration + +**5.2 Get storage account details**: +``` +Show me details for storage account '' +``` + +**Verify**: +- [ ] Account properties match Terraform configuration +- [ ] Replication type is LRS +- [ ] Location is East US + +### Step 6: Get Azure Best Practices with Azure MCP Server + +**6.1 Get general Azure best practices** (uses `azmcp_bestpractices_get`): +``` +Show me Azure best practices for code generation +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_bestpractices_get` +- [ ] General Azure best practices provided +- [ ] Covers multiple resource types +- [ ] Security and compliance guidance included + +### Step 7: Cleanup (Terraform - Not MCP) + +```bash +# Destroy infrastructure +terraform destroy -auto-approve +``` **Expected Results**: -- Template follows Azure naming conventions -- Sensitive values are parameterized -- Template is reusable across environments -- Outputs provide useful information +- Terraform best practices retrieved successfully +- IaC rules for Terraform provided +- Azure general best practices available +- Deployment verification through MCP tools + +--- ## Common Issues to Watch For -- **Naming Conflicts**: Resources with duplicate names -- **Region Availability**: Services not available in requested regions -- **Quota Limits**: Exceeding subscription quotas -- **Dependency Ordering**: Resources deployed in wrong order -- **Authentication Issues**: Permission problems during deployment -- **Parameter Validation**: Invalid parameter values -- **Syntax Errors**: Invalid Bicep syntax -- **Version Compatibility**: API version mismatches +| Issue | Description | Resolution | +|-------|-------------|------------| +| **Invalid Resource Type** | Bicep schema not found | Verify resource type format (e.g., Microsoft.Storage/storageAccounts) | +| **Schema Version Mismatch** | API version differences | Use latest stable API version for resources | +| **Deployment Tool Confusion** | Wrong tool used | Specify "Bicep" or "Terraform" explicitly in prompts | +| **Missing Best Practices** | Generic guidance returned | Be specific about resource type and deployment tool | +| **Template Syntax Errors** | Generated template invalid | Use retrieved schema to validate template structure | + +## What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output) +- [ ] Expected vs actual results +- [ ] Resource type requested +- [ ] Deployment tool (Bicep/Terraform) +- [ ] Error messages (if any) +- [ ] Screenshots of unexpected behavior ## Related Resources - [Azure Bicep Documentation](https://learn.microsoft.com/azure/azure-resource-manager/bicep/) -- [Azure Best Practices Tool](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md#azure-mcp-best-practices) -- [Troubleshooting Deployments](troubleshooting.md) +- [Azure Terraform Documentation](https://learn.microsoft.com/azure/developer/terraform/) +- [Azure Resource Manager](https://learn.microsoft.com/azure/azure-resource-manager/) +- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) +## Quick Reference: Supported MCP Tools + +### Bicep Support +- `azmcp_bicepschema_get` - Get Bicep schema for resource types + +### IaC Guidance +- `azmcp_deploy_iac_rules_get` - Get IaC generation rules for resource types +- `azmcp_azureterraformbestpractices_get` - Get Terraform best practices +- `azmcp_bestpractices_get` - Get general Azure best practices + +### Resource Verification +- `azmcp_storage_account_get` - List storage accounts +- `azmcp_storage_blob_container_get` - List blob containers +- `azmcp_appservice_get` - Get App Service details +- `azmcp_sql_server_list` - List SQL servers +- `azmcp_cosmos_account_list` - List Cosmos DB accounts + --- **Next**: [PaaS Services Testing](paas-services.md) diff --git a/docs/bug-bash/scenarios/paas-services.md b/docs/bug-bash/scenarios/paas-services.md index 3579cdec12..4a96b39736 100644 --- a/docs/bug-bash/scenarios/paas-services.md +++ b/docs/bug-bash/scenarios/paas-services.md @@ -1,17 +1,14 @@ -# 🚀 PaaS Services Testing Scenario +# PaaS Services Testing Scenario -> **⚠️ READ FIRST**: [TESTING-SCOPE.md](../TESTING-SCOPE.md) explains what MCP tools can and cannot do. **Azure MCP Server has LIMITED support for PaaS services** - primarily Function App inspection and App Service database connections. Use Azure CLI to create App Services, Container Apps, and Function Apps before testing. - -Test Azure MCP Server's capabilities with Platform-as-a-Service offerings including App Service, Container Apps, and Azure Functions. +> **MCP Tool Support Notice** +> Azure MCP Server provides **App Service inspection, Function App discovery, and database connection management** capabilities. Resource creation and application deployment require Azure CLI or Portal. This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. ## Objectives -- Deploy and manage Azure App Service web apps -- Work with Azure Container Apps -- Create and manage Azure Functions -- Test configuration and scaling -- Verify integration with other services -- Test deployment workflows +- Test App Service discovery and inspection +- Verify Function App listing and details retrieval +- Test database connection addition to App Services +- Validate end-to-end PaaS resource workflows ## Prerequisites @@ -19,362 +16,301 @@ Test Azure MCP Server's capabilities with Platform-as-a-Service offerings includ - [ ] Azure CLI installed (`az --version`) - [ ] Authenticated to Azure (`az login`) - [ ] Active Azure subscription -- [ ] Sample application code (optional) - [ ] GitHub Copilot with Agent mode enabled -## Test Scenarios - -### Scenario 1: Azure App Service - -**Objective**: Test web app creation, configuration, and deployment - -#### Phase 1: Create App Service Resources - -1. **Create App Service Plan**: - ``` - Create an Azure App Service Plan: - - Name: 'bugbash-plan-' - - Resource group: '' - - SKU: B1 (Basic) - - Region: East US - - OS: Linux - ``` - -2. **Verify plan creation**: - ``` - List all App Service plans in resource group '' - ``` - -3. **Create web app**: - ``` - Create a web app: - - Name: 'bugbash-webapp-' - - Resource group: '' - - App Service Plan: 'bugbash-plan-' - - Runtime: Node.js 20 LTS - ``` - -4. **List web apps**: - ``` - List all web apps in my subscription - ``` - -5. **Get web app details**: - ``` - Show me the details for web app 'bugbash-webapp-' in - resource group '' - ``` - -**Verify**: -- [ ] App Service Plan created with correct SKU -- [ ] Web app created successfully -- [ ] Runtime configured correctly -- [ ] Web app URL is accessible - -#### Phase 2: Configure Web App - -6. **Get configuration**: - ``` - Get the configuration for web app 'bugbash-webapp-' - ``` - -7. **Add application settings**: - ``` - Add the following application settings to web app 'bugbash-webapp-': - - NODE_ENV: production - - API_URL: https://api.example.com - - LOG_LEVEL: info - ``` - -8. **Add connection strings**: - ``` - Add a connection string named 'Database' with value - '' of type SQLAzure to web app 'bugbash-webapp-' - ``` - -9. **Enable Application Insights**: - ``` - Enable Application Insights for web app 'bugbash-webapp-' - ``` +## Scenario 1: App Service Discovery and Database Integration End-to-End -**Verify**: -- [ ] Application settings added -- [ ] Connection strings configured -- [ ] Application Insights enabled -- [ ] Configuration changes applied - -#### Phase 3: Database Integration - -10. **Add database connection**: - ``` - Add a SQL Server database connection to app service 'bugbash-webapp-': - - Database name: '' - - Server: '' - ``` - -11. **Configure PostgreSQL**: - ``` - Add a PostgreSQL database to app service 'bugbash-webapp-' - ``` - -12. **Verify database connections**: - ``` - Show me all database connections for web app 'bugbash-webapp-' - ``` +**Objective**: Complete workflow for App Service inspection and database connection setup -**Verify**: -- [ ] Database connections configured -- [ ] Connection strings created -- [ ] Databases accessible from app +### Step 1: Create App Service Resources (External - Not MCP) -#### Phase 4: Deployment and Testing +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. -13. **Get deployment information**: - ``` - How do I deploy my Node.js application to web app 'bugbash-webapp-'? - ``` +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-paas-rg' in eastus, create an App Service Plan with B1 SKU (Linux), create a Web App with Node.js 18 runtime, create an Azure SQL server with a database named 'bugbash-db' using S0 service objective +``` -14. **Check web app status**: - ``` - What is the status of web app 'bugbash-webapp-'? - ``` +**Option B: Run Azure CLI Commands Manually**: +```bash +# Create resource group +az group create --name bugbash-paas-rg --location eastus -15. **Get web app URL**: - ``` - What is the URL for web app 'bugbash-webapp-'? - ``` +# Create App Service Plan +az appservice plan create \ + --name bugbash-plan \ + --resource-group bugbash-paas-rg \ + --sku B1 \ + --is-linux -16. **Test accessibility**: - - Open the web app URL in browser - - Verify the app is running - - Check if default page loads +# Create Web App +az webapp create \ + --name bugbash-webapp-$RANDOM \ + --resource-group bugbash-paas-rg \ + --plan bugbash-plan \ + --runtime "NODE:18-lts" -**Verify**: -- [ ] Deployment instructions provided -- [ ] Web app is running -- [ ] URL is accessible -- [ ] App responds to requests +# Create Azure SQL for database connection testing +az sql server create \ + --name bugbash-sqlserver-$RANDOM \ + --resource-group bugbash-paas-rg \ + --location eastus \ + --admin-user sqladmin \ + --admin-password -**Expected Results**: -- App Service Plan created -- Web app created and configured -- Database connections working -- Application is deployable -- Web app is accessible +az sql db create \ + --name bugbash-db \ + --resource-group bugbash-paas-rg \ + --server \ + --service-objective S0 +``` ---- +### Step 2: Discover App Services with Azure MCP Server -### Scenario 2: Azure Container Apps +**2.1 List all App Services** (uses `azmcp_appservice_get`): +``` +List all App Services in my subscription +``` -**Objective**: Test container deployment and management +**Verify**: +- [ ] Tool invoked: `azmcp_appservice_get` +- [ ] Your web app appears in the list +- [ ] App properties shown (name, location, state, URL) -#### Phase 1: Create Container Apps Environment +**2.2 Get specific App Service details**: +``` +Show me details for App Service '' in resource group 'bugbash-paas-rg' +``` -1. **Create Container Apps Environment**: - ``` - Create an Azure Container Apps Environment: - - Name: 'bugbash-containerenv-' - - Resource group: '' - - Location: East US - ``` +**Verify**: +- [ ] Detailed properties displayed +- [ ] Runtime stack shown (Node.js 18) +- [ ] App Service Plan details included +- [ ] Default hostname visible -2. **List Container Apps Environments**: - ``` - List all Container Apps environments in my subscription - ``` +### Step 3: Add Database Connection with Azure MCP Server -3. **Get environment details**: - ``` - Show me details of Container Apps environment 'bugbash-containerenv-' - ``` +**3.1 Add SQL Server database connection** (uses `azmcp_appservice_database_add`): +``` +Add a SQL Server database connection to App Service '' in resource group 'bugbash-paas-rg' for database '' on server '' +``` **Verify**: -- [ ] Environment created successfully -- [ ] Environment is in running state -- [ ] Configuration is correct - -#### Phase 2: Deploy Container App - -4. **Create container app**: - ``` - Create a container app in environment 'bugbash-containerenv-': - - Name: 'bugbash-container-' - - Container image: nginx:latest - - Ingress: enabled, external - - Target port: 80 - ``` - -5. **List container apps**: - ``` - List all container apps in resource group '' - ``` - -6. **Get container app details**: - ``` - Show me details of container app 'bugbash-container-' - ``` +- [ ] Tool invoked: `azmcp_appservice_database_add` +- [ ] Connection string created +- [ ] Database connection configured +- [ ] Success message displayed -**Verify**: -- [ ] Container app created -- [ ] Container is running -- [ ] Ingress configured correctly -- [ ] App is accessible externally - -#### Phase 3: Configuration and Scaling - -7. **Configure environment variables**: - ``` - Add environment variables to container app 'bugbash-container-': - - ENVIRONMENT: production - - DEBUG: false - ``` - -8. **Configure scaling**: - ``` - Configure scaling for container app 'bugbash-container-': - - Min replicas: 1 - - Max replicas: 5 - ``` - -9. **Get container app URL**: - ``` - What is the URL for container app 'bugbash-container-'? - ``` +**3.2 Alternative connection types**: +``` +Add a PostgreSQL database connection to App Service '' for database '' on server '' +``` **Verify**: -- [ ] Environment variables set -- [ ] Scaling configured -- [ ] URL provided and accessible +- [ ] PostgreSQL connection added +- [ ] Connection string format correct -**Expected Results**: -- Container Apps environment created -- Container app deployed -- Configuration applied -- Scaling works -- App is accessible +### Step 4: Verify Configuration (External - Not MCP) ---- +> **External Verification**: Use Azure CLI to check configuration + +```bash +# List app settings including database connection +az webapp config appsettings list \ + --name \ + --resource-group bugbash-paas-rg \ + --query "[?name=='SQLAZURECONNSTR_Database']" -### Scenario 3: Azure Functions +# Check connection strings +az webapp config connection-string list \ + --name \ + --resource-group bugbash-paas-rg +``` -**Objective**: Test Function App creation and management +### Step 5: Cleanup (External - Not MCP) -#### Phase 1: Create Function App +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-paas-rg --yes --no-wait +``` + +**Expected Results**: +- App Service discovery works correctly +- Detailed inspection successful +- Database connection addition functional +- Connection strings created properly -1. **Create storage account for functions**: - ``` - Create a storage account 'bugbashfunc' for Azure Functions - in resource group '' - ``` +--- -2. **Create Function App**: - ``` - Create an Azure Function App: - - Name: 'bugbash-funcapp-' - - Resource group: '' - - Runtime: Node.js 20 - - Plan: Consumption (serverless) - - Storage account: 'bugbashfunc' - - Region: East US - ``` +## Scenario 2: Azure Function App Discovery End-to-End + +**Objective**: Complete workflow for Function App inspection and discovery + +### Step 1: Create Function App Resources (External - Not MCP) + +> **External Setup Required**: Use Azure CLI to create Function App + +```bash +# Create resource group +az group create --name bugbash-functions-rg --location eastus + +# Create storage account (required for Functions) +az storage account create \ + --name bugbashfuncstorage$RANDOM \ + --resource-group bugbash-functions-rg \ + --location eastus \ + --sku Standard_LRS + +# Create Function App +az functionapp create \ + --name bugbash-funcapp-$RANDOM \ + --resource-group bugbash-functions-rg \ + --storage-account \ + --consumption-plan-location eastus \ + --runtime node \ + --runtime-version 18 \ + --functions-version 4 + +# Create another Function App for comparison +az functionapp create \ + --name bugbash-funcapp2-$RANDOM \ + --resource-group bugbash-functions-rg \ + --storage-account \ + --consumption-plan-location eastus \ + --runtime python \ + --runtime-version 3.11 \ + --functions-version 4 +``` + +### Step 2: Discover Function Apps with Azure MCP Server + +**2.1 List all Function Apps** (uses `azmcp_functionapp_get`): +``` +List all Function Apps in my subscription +``` -3. **List Function Apps**: - ``` - List all function apps in my subscription - ``` +**Verify**: +- [ ] Tool invoked: `azmcp_functionapp_get` +- [ ] Both Function Apps appear +- [ ] Function App properties shown (name, location, runtime, plan) -4. **Get Function App details**: - ``` - Show me details for function app 'bugbash-funcapp-' in - resource group '' - ``` +**2.2 Get specific Function App details**: +``` +Show me details for Function App '' in resource group 'bugbash-functions-rg' +``` **Verify**: -- [ ] Storage account created -- [ ] Function App created -- [ ] Runtime configured correctly -- [ ] Consumption plan assigned - -#### Phase 2: Configure Function App - -5. **Get Function App configuration**: - ``` - Get configuration for function app 'bugbash-funcapp-' - ``` - -6. **Add application settings**: - ``` - Add application settings to function app 'bugbash-funcapp-': - - ENVIRONMENT: production - - API_KEY: - ``` - -7. **Enable Application Insights**: - ``` - Enable Application Insights for function app 'bugbash-funcapp-' - ``` - -8. **Get function app keys**: - ``` - Get the host keys for function app 'bugbash-funcapp-' - ``` +- [ ] Detailed properties displayed +- [ ] Runtime stack shown (Node.js 18) +- [ ] Consumption plan indicated +- [ ] Storage account connection visible +- [ ] Default hostname shown + +**2.3 Filter by resource group**: +``` +List all Function Apps in resource group 'bugbash-functions-rg' +``` **Verify**: -- [ ] Configuration retrieved -- [ ] Application settings added -- [ ] Application Insights enabled -- [ ] Host keys accessible +- [ ] Only Function Apps in specified resource group listed +- [ ] Filter works correctly + +### Step 3: Compare Function App Configurations -#### Phase 3: Deployment Information +**3.1 Compare runtimes**: +- Note Node.js Function App properties +- Note Python Function App properties +- Verify runtime differences are visible -9. **Get deployment instructions**: - ``` - How do I deploy my Node.js functions to function app - 'bugbash-funcapp-'? - ``` +**3.2 Identify configuration differences**: +``` +Compare the configurations of Function Apps '' and '' +``` -10. **Get function app URL**: - ``` - What is the URL for function app 'bugbash-funcapp-'? - ``` +**Verify**: +- [ ] Runtime versions differ +- [ ] Both use Consumption plan +- [ ] Storage accounts may differ + +### Step 4: Storage Account Verification with Azure MCP Server -11. **Check function app status**: - ``` - What is the status of function app 'bugbash-funcapp-'? - ``` +**4.1 List storage accounts** (uses `azmcp_storage_account_get`): +``` +List all storage accounts in my subscription +``` **Verify**: -- [ ] Deployment guidance provided -- [ ] URL retrieved -- [ ] Status is running +- [ ] Tool invoked: `azmcp_storage_account_get` +- [ ] Function storage account appears +- [ ] Storage properties shown -**Expected Results**: -- Function App created -- Configuration applied -- Application Insights enabled -- Deployment guidance provided -- App is operational +**4.2 Verify Function App storage connection**: +- Check that Function App uses the created storage account +- Verify storage account location matches Function App + +### Step 5: Cleanup (External - Not MCP) +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-functions-rg --yes --no-wait +``` +**Expected Results**: +- Function App discovery works correctly +- Multiple Function Apps listed successfully +- Detailed inspection shows runtime and configuration +- Resource group filtering functional +- Storage account relationship visible + +--- ## Common Issues to Watch For -- **Naming Conflicts**: App names must be globally unique -- **Plan Compatibility**: Some features require specific plan tiers -- **Runtime Mismatches**: Ensure runtime version compatibility -- **Storage Dependencies**: Function Apps require storage accounts -- **Port Configuration**: Container Apps need correct port mappings -- **Memory Limits**: Consumption plan has memory constraints -- **Cold Starts**: Serverless functions have startup latency -- **CORS Issues**: Cross-origin requests may need configuration +| Issue | Description | Resolution | +|-------|-------------|------------| +| **App Service Not Found** | MCP can't locate App Service | Verify resource name and resource group are correct | +| **Function App Missing** | Function App not listed | Ensure Function App exists and you have access permissions | +| **Database Connection Fails** | Can't add database connection | Verify database server exists and firewall allows connections | +| **Runtime Mismatch** | Unexpected runtime shown | Check App Service configuration in Azure Portal | +| **Storage Account Missing** | Function App storage not found | Verify storage account exists in same subscription | +| **Resource Group Filter** | Filtering doesn't work | Ensure resource group name is exact match | + +## What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output) +- [ ] Expected vs actual results +- [ ] Error messages (if any) +- [ ] Resource names and resource group +- [ ] Runtime and plan tier +- [ ] Screenshots of unexpected behavior ## Related Resources - [Azure App Service Documentation](https://learn.microsoft.com/azure/app-service/) -- [Azure Container Apps Documentation](https://learn.microsoft.com/azure/container-apps/) - [Azure Functions Documentation](https://learn.microsoft.com/azure/azure-functions/) -- [PaaS Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md#azure-app-service) +- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) +## Quick Reference: Supported MCP Tools + +### App Service +- `azmcp_appservice_get` - Get App Service details (list all or specific app) +- `azmcp_appservice_database_add` - Add database connection to App Service + +### Function Apps +- `azmcp_functionapp_get` - Get Function App details (list all or specific function app) + +### Storage +- `azmcp_storage_account_get` - List storage accounts + +### Database Support +- SQL Server (`SqlServer`) +- PostgreSQL (`PostgreSQL`) +- MySQL (`MySQL`) +- Cosmos DB (`CosmosDB`) + --- **Next**: [Storage Operations Testing](storage-operations.md) diff --git a/docs/bug-bash/scenarios/storage-operations.md b/docs/bug-bash/scenarios/storage-operations.md index 777495ad72..bddae3d1da 100644 --- a/docs/bug-bash/scenarios/storage-operations.md +++ b/docs/bug-bash/scenarios/storage-operations.md @@ -1,14 +1,13 @@ # Storage Operations Testing Scenario -Test Azure MCP Server's storage account and blob operations capabilities. +> **IMPORTANT**: Azure MCP Server provides **read-only inspection and querying** of Azure Storage accounts. It **cannot create, modify, upload, or delete** storage resources. This scenario guides you through creating resources externally (Azure CLI or Portal) and then using Azure MCP Server to inspect them. ## Objectives -- Create and manage storage accounts -- Work with blob containers -- Upload and download files -- Test storage permissions -- Verify error handling +- Test storage account discovery and inspection capabilities +- Verify container listing functionality +- Test blob enumeration +- Validate storage account filtering and querying ## Prerequisites @@ -16,284 +15,378 @@ Test Azure MCP Server's storage account and blob operations capabilities. - [ ] Azure CLI installed (`az --version`) - [ ] Authenticated to Azure (`az login`) - [ ] Active Azure subscription -- [ ] Test files ready for upload +- [ ] Resource group for testing (or create one) - [ ] GitHub Copilot with Agent mode enabled -## Test Scenarios +--- -### Scenario 1: Create a Storage Account +## Scenario 1: Storage Account Discovery and Inspection End-to-End -**Objective**: Test basic storage account creation +**Objective**: Test complete workflow of creating a storage account externally, then discovering and inspecting it via Azure MCP Server. -**Steps**: +### External Setup Required -1. Open GitHub Copilot Chat in Agent mode +#### Step 1: Create Storage Account (External - Not MCP) -2. Use this prompt: - ``` - Create a new storage account named 'bugbashstorage' in - resource group '' in East US with: - - Standard_LRS redundancy - - Hot access tier - - Enable hierarchical namespace for Data Lake - ``` +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. -3. **Monitor the creation**: - - Watch for progress updates - - Note any warnings or errors - - Record creation time: _____ seconds +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-storage-rg' in eastus, create a storage account with Standard_LRS SKU and Hot access tier with hierarchical namespace enabled, create two blob containers 'test-data' (private) and 'public-data' (public blob access), then upload a test file to test-data container +``` -4. **Verify the storage account**: - ``` - Show me the details of storage account 'bugbashstorage' - ``` +**Option B: Run Azure CLI Commands Manually**: +```bash +# Set variables +RG_NAME="bugbash-storage-rg" +LOCATION="eastus" +STORAGE_ACCOUNT="bugbashstorage$(Get-Random -Maximum 9999)" + +# Create resource group +az group create --name $RG_NAME --location $LOCATION + +# Create storage account with hierarchical namespace +az storage account create \ + --name $STORAGE_ACCOUNT \ + --resource-group $RG_NAME \ + --location $LOCATION \ + --sku Standard_LRS \ + --access-tier Hot \ + --enable-hierarchical-namespace true + +# Create blob containers +az storage container create \ + --name test-data \ + --account-name $STORAGE_ACCOUNT \ + --public-access off + +az storage container create \ + --name public-data \ + --account-name $STORAGE_ACCOUNT \ + --public-access blob + +# Upload sample blob +echo "Test content for bug bash" > test-file.txt +az storage blob upload \ + --account-name $STORAGE_ACCOUNT \ + --container-name test-data \ + --name test-file.txt \ + --file test-file.txt +``` -5. **Check the properties**: - - [ ] Account name is correct - - [ ] Location is East US - - [ ] Redundancy is Standard_LRS - - [ ] Access tier is Hot - - [ ] Hierarchical namespace is enabled +**Verify External Setup**: +- [ ] Storage account created successfully +- [ ] Hierarchical namespace enabled +- [ ] Two containers created (test-data, public-data) +- [ ] Sample blob uploaded to test-data container -**Expected Results**: -- Storage account created successfully -- All properties match specifications -- Account is accessible +### Azure MCP Server Prompts -### Scenario 2: Manage Blob Containers +#### Step 2: Discover Storage Accounts -**Objective**: Test container creation and management +Open GitHub Copilot Chat in Agent mode and use these prompts: -**Steps**: +**Prompt 2a**: List all storage accounts in subscription +``` +Show me all storage accounts in my subscription +``` -1. **Create a container**: - ``` - Create a blob container named 'test-data' in storage account - 'bugbashstorage' with private access - ``` +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_account_get` +- [ ] Returns list of storage accounts +- [ ] Includes your new storage account -2. **Verify container creation**: - ``` - List all containers in storage account 'bugbashstorage' - ``` +**Prompt 2b**: Get specific storage account details +``` +Show me details for storage account '' in resource group 'bugbash-storage-rg' +``` -3. **Check container properties**: - - [ ] Container name is correct - - [ ] Public access level is None (private) - - [ ] Container is empty +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_account_get` with `--account` and `--resource-group` parameters +- [ ] Returns single storage account +- [ ] Shows SKU as Standard_LRS +- [ ] Shows access tier as Hot +- [ ] Shows hierarchical namespace enabled +- [ ] Shows location as eastus -4. **Create additional containers** with different access levels: - ``` - Create containers: - - 'public-blobs' with blob public access - - 'public-container' with container public access - - 'private-data' with no public access - ``` +#### Step 3: List Blob Containers -5. **List and verify all containers**: - ``` - Show me all containers in 'bugbashstorage' with their access levels - ``` +**Prompt 3a**: List all containers +``` +List all blob containers in storage account '' +``` -**Expected Results**: -- All containers created -- Access levels set correctly -- Containers are accessible +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_container_list` +- [ ] Returns both containers (test-data, public-data) +- [ ] Shows public access levels correctly ---- +#### Step 4: List Blobs in Container -### Scenario 3: Upload Files to Blob Storage - -**Objective**: Test file upload capabilities - -**Steps**: - -1. **Prepare test files**: - - Create a small text file (< 1 MB): `test-small.txt` - - Create a medium file (10-50 MB): `test-medium.zip` - - Create a large file (> 100 MB): `test-large.iso` (optional) - -2. **Upload small file**: - ``` - Upload the file 'test-small.txt' to container 'test-data' in - storage account 'bugbashstorage' - ``` - -3. **Verify upload**: - ``` - List all blobs in container 'test-data' - ``` - -4. **Check blob properties**: - - [ ] File name is correct - - [ ] File size matches original - - [ ] Content type is detected - - [ ] Upload time is recent - -5. **Upload medium file** and measure performance: - ``` - Upload 'test-medium.zip' to 'test-data' container - ``` - - Upload time: _____ seconds - - Upload speed: _____ MB/s - -6. **Upload file with custom metadata**: - ``` - Upload 'test-small.txt' as 'document.txt' to 'test-data' with - content type 'text/plain' and metadata 'author=BugBash' - ``` - -7. **Verify metadata**: - ``` - Show me the properties of blob 'document.txt' in container 'test-data' - ``` - -**Expected Results**: -- All files upload successfully -- File integrity is maintained -- Metadata is stored correctly -- Performance is reasonable +**Prompt 4a**: List blobs in test-data container +``` +Show me all blobs in container 'test-data' from storage account '' +``` ---- +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_blob_list` +- [ ] Returns test-file.txt blob +- [ ] Shows blob size +- [ ] Shows last modified date -### Scenario 4: Download Files from Blob Storage +**Prompt 4b**: Get specific blob details +``` +Get details for blob 'test-file.txt' in container 'test-data' from storage account '' +``` -**Objective**: Test file download capabilities +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_blob_get` +- [ ] Returns blob metadata +- [ ] Shows content type +- [ ] Shows blob properties -**Steps**: +#### Step 5: Filter Storage Accounts by Resource Group -1. **Download a file**: - ``` - Download the blob 'test-small.txt' from container 'test-data' - in storage account 'bugbashstorage' to 'downloaded-file.txt' - ``` +**Prompt 5a**: Filter by resource group +``` +Show me all storage accounts in resource group 'bugbash-storage-rg' +``` -2. **Verify download**: - - [ ] File is downloaded - - [ ] File size matches original - - [ ] Content is identical - - Compare checksums if possible +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_account_get` with `--resource-group` parameter +- [ ] Returns only storage accounts in specified resource group +- [ ] Excludes storage accounts from other resource groups -3. **Download multiple files**: - ``` - Download all blobs from container 'test-data' to folder 'downloads' - ``` +### Cleanup -4. **Verify all files**: - - [ ] All blobs are downloaded - - [ ] Folder structure is correct - - [ ] No corruption +**Option A: Prompt GitHub Copilot Chat**: +``` +Delete the Azure resource group 'bugbash-storage-rg' and all its resources +``` -**Expected Results**: -- Files download successfully -- Content integrity is preserved -- Reasonable download speeds +**Option B: Run Azure CLI Command Manually**: +```bash +# Delete the resource group and all resources +az group delete --name bugbash-storage-rg --yes --no-wait +``` --- -### Scenario 5: Storage Account Configuration - -**Objective**: Test storage account settings and features - -**Steps**: - -1. **List all storage accounts**: - ``` - Show me all my storage accounts in subscription '' - ``` - -2. **Filter by resource group**: - ``` - List storage accounts in resource group '' - ``` - -3. **Get account properties**: - ``` - Show me detailed information for storage account 'bugbashstorage' - including: - - SKU and redundancy - - Access tier - - Network rules - - Encryption settings - - HTTPS enforcement - ``` - -4. **Check account keys** (if permissions allow): - ``` - Show me the access keys for storage account 'bugbashstorage' - ``` - -5. **Verify security settings**: - - [ ] HTTPS-only traffic is enforced - - [ ] Minimum TLS version is set - - [ ] Secure transfer is enabled - - [ ] Public access is configured correctly - -**Expected Results**: -- Account details are accurate -- Security settings are appropriate -- Configuration matches expectations +## Scenario 2: Multi-Container Blob Discovery End-to-End ---- +**Objective**: Test Azure MCP Server's ability to discover and inspect multiple containers with different access levels and blob types. -### Scenario 6: Test Different Storage Types +### External Setup Required -**Objective**: Test various Azure Storage services +#### Step 1: Create Storage Account with Multiple Containers (External - Not MCP) -#### 6a. Table Storage +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. -**Steps**: +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-multicontainer-rg' in westus2, create a storage account with Standard_GRS SKU and Cool access tier, create three containers: 'private-documents' (private), 'public-images' (blob public access), 'shared-data' (container public access), then upload sample files with different content types ``` -Does storage account 'bugbashstorage' support Azure Table Storage? -If yes, show me how to work with it. + +**Option B: Run Azure CLI Commands Manually**: +```bash +# Set variables +RG_NAME="bugbash-multicontainer-rg" +LOCATION="westus2" +STORAGE_ACCOUNT="bugbashmulti$(Get-Random -Maximum 9999)" + +# Create resource group and storage account +az group create --name $RG_NAME --location $LOCATION + +az storage account create \ + --name $STORAGE_ACCOUNT \ + --resource-group $RG_NAME \ + --location $LOCATION \ + --sku Standard_GRS \ + --access-tier Cool + +# Create multiple containers with different access levels +az storage container create \ + --name private-documents \ + --account-name $STORAGE_ACCOUNT \ + --public-access off + +az storage container create \ + --name public-images \ + --account-name $STORAGE_ACCOUNT \ + --public-access blob + +az storage container create \ + --name shared-data \ + --account-name $STORAGE_ACCOUNT \ + --public-access container + +# Upload different file types +echo "Private document content" > document.txt +echo "{ \"data\": \"test\" }" > data.json +echo "Test" > index.html + +az storage blob upload --account-name $STORAGE_ACCOUNT --container-name private-documents --name document.txt --file document.txt +az storage blob upload --account-name $STORAGE_ACCOUNT --container-name public-images --name image.png --file document.txt --content-type "image/png" +az storage blob upload --account-name $STORAGE_ACCOUNT --container-name shared-data --name data.json --file data.json --content-type "application/json" +az storage blob upload --account-name $STORAGE_ACCOUNT --container-name shared-data --name index.html --file index.html --content-type "text/html" ``` -**Verify**: -- [ ] Table storage capability confirmed -- [ ] Guidance provided for usage +**Verify External Setup**: +- [ ] Storage account created with Standard_GRS SKU +- [ ] Three containers with different access levels +- [ ] Multiple blobs uploaded with different content types + +### Azure MCP Server Prompts -#### 6b. Queue Storage +#### Step 2: Discover Storage Account with Cool Tier -**Steps**: +**Prompt 2a**: Get storage account details ``` -Does storage account 'bugbashstorage' support Azure Queue Storage? -If yes, show me how to create and manage queues. +Show me the storage account '' in resource group 'bugbash-multicontainer-rg' ``` -**Verify**: -- [ ] Queue storage capability confirmed -- [ ] Instructions for queue operations +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_account_get` +- [ ] Shows SKU as Standard_GRS +- [ ] Shows access tier as Cool +- [ ] Shows location as westus2 -#### 6c. File Shares +#### Step 3: List All Containers and Their Access Levels -**Steps**: +**Prompt 3a**: List containers ``` -Does storage account 'bugbashstorage' support Azure File Shares? -If yes, show me how to create and mount a file share. +List all blob containers in storage account '' and show their public access settings ``` -**Verify**: -- [ ] File share capability confirmed -- [ ] Mount instructions provided +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_container_list` +- [ ] Returns three containers: private-documents, public-images, shared-data +- [ ] Shows correct public access level for each: + - [ ] private-documents: None/Off + - [ ] public-images: Blob + - [ ] shared-data: Container -## Common Issues to Watch For +#### Step 4: Inspect Blobs Across Multiple Containers -- **Authentication Failures**: Credential or permission issues -- **Network Timeouts**: Slow or failed uploads/downloads -- **Name Conflicts**: Duplicate blob or container names -- **Path Issues**: Platform-specific path separator problems -- **File Encoding**: Text file encoding issues across platforms -- **Large File Handling**: Memory issues with big files -- **Concurrent Operations**: Race conditions with parallel uploads -- **Permission Errors**: RBAC or access key issues +**Prompt 4a**: List blobs in private-documents +``` +Show me all blobs in the 'private-documents' container from storage account '' +``` +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_blob_list` with `--container` parameter +- [ ] Returns document.txt +- [ ] Shows blob size and last modified date + +**Prompt 4b**: List blobs in shared-data +``` +List all blobs in container 'shared-data' from storage account '' +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_blob_list` +- [ ] Returns both blobs: data.json, index.html +- [ ] Shows correct content types for each blob + +**Prompt 4c**: Get specific blob details +``` +Get details for blob 'data.json' in container 'shared-data' from storage account '' +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_blob_get` +- [ ] Shows content type as application/json +- [ ] Shows blob properties and metadata + +#### Step 5: Test Cross-Subscription Discovery (if available) + +**Prompt 5a**: List storage accounts across all accessible subscriptions +``` +Show me all storage accounts I have access to across all my subscriptions +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_account_get` without subscription filter +- [ ] Returns storage accounts from multiple subscriptions (if you have access) +- [ ] Groups results by subscription + +### Cleanup + +**Option A: Prompt GitHub Copilot Chat**: +``` +Delete the Azure resource group 'bugbash-multicontainer-rg' and all its resources +``` + +**Option B: Run Azure CLI Command Manually**: +```bash +# Delete the resource group and all resources +az group delete --name bugbash-multicontainer-rg --yes --no-wait +``` + +--- + +## Common Issues to Watch For + +| Issue | Description | Resolution | +|-------|-------------|------------| +| **Storage Account Not Found** | MCP can't locate storage account | Verify account name spelling and ensure you have access permissions | +| **Container List Empty** | No containers returned | Ensure containers exist and storage account name is correct | +| **Blob List Empty** | No blobs returned for container | Verify container name and that blobs have been uploaded | +| **Authentication Errors** | Can't access storage resources | Run `az login` and verify RBAC permissions (Storage Blob Data Reader/Contributor) | +| **SKU Mismatch** | Unexpected SKU displayed | Check actual storage account configuration in Azure Portal | +| **Access Tier Confusion** | Hot vs Cool tier not displayed correctly | Verify account tier in Azure Portal matches MCP output | +| **Hierarchical Namespace** | Data Lake Gen2 feature not shown | Confirm feature was enabled during storage account creation | +| **Cross-Subscription Issues** | Storage accounts missing from list | Verify you have appropriate permissions in all subscriptions | + +## What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output: `azmcp_storage_account_get`, `azmcp_storage_container_list`, `azmcp_storage_blob_list`, etc.) +- [ ] Expected vs actual results +- [ ] Error messages (if any) +- [ ] Storage account name and resource group +- [ ] Container names and public access levels +- [ ] Blob names and content types +- [ ] Screenshots of unexpected behavior +- [ ] Output from Azure CLI showing actual resource state ## Related Resources - [Azure Storage Documentation](https://learn.microsoft.com/azure/storage/) - [Azure Blob Storage Best Practices](https://learn.microsoft.com/azure/storage/blobs/storage-blobs-introduction) - [Data Lake Storage Gen2](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-introduction) -- [Storage Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md#azure-storage) +- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) +## Quick Reference: Supported MCP Tools + +### Storage Account Operations +- `azmcp_storage_account_get` - Get storage account details (list all or specific account) + - Parameters: `--subscription`, `--resource-group`, `--account` + - Returns: Storage account properties (SKU, access tier, location, hierarchical namespace) + +### Container Operations +- `azmcp_storage_container_list` - List blob containers in a storage account + - Parameters: `--account`, `--resource-group` + - Returns: Container names and public access levels + +### Blob Operations +- `azmcp_storage_blob_list` - List blobs in a container + - Parameters: `--account`, `--container`, `--resource-group` + - Returns: Blob names, sizes, last modified dates + +- `azmcp_storage_blob_get` - Get specific blob details + - Parameters: `--account`, `--container`, `--blob`, `--resource-group` + - Returns: Blob metadata, content type, properties + +### Important Notes +- **Read-Only**: All MCP tools are read-only inspection tools +- **No Write Operations**: Cannot create, upload, modify, or delete resources +- **Authentication**: Requires Azure RBAC permissions (Reader or Storage Blob Data Reader minimum) +- **Filtering**: Supports filtering by subscription, resource group, and specific resource names + +--- + **Next**: [Agent Building](agent-building.md) From 9a9f3f4c7b4c1bb61c581ce095e76091e2dfeb6c Mon Sep 17 00:00:00 2001 From: Vinay Gera Date: Wed, 15 Oct 2025 10:41:33 -0700 Subject: [PATCH 04/10] refactor docs --- docs/bug-bash/README.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/bug-bash/README.md b/docs/bug-bash/README.md index 00f0d2c073..b3167ec6be 100644 --- a/docs/bug-bash/README.md +++ b/docs/bug-bash/README.md @@ -15,7 +15,7 @@ Welcome to the Azure MCP Server Bug Bash! We're excited to have you help us impr The Azure MCP Server enables AI agents to interact with Azure services through natural language commands. As we continue to enhance the server, we need your help to identify issues across different platforms, IDEs, and usage scenarios. -> **🎯 Important**: **MCP tools primarily focus on reading, querying, and inspecting Azure resources**. Most resource creation requires Azure CLI - use it to set up test resources before testing MCP's discovery and inspection capabilities. +> **Important**: **MCP tools primarily focus on reading, querying, and inspecting Azure resources**. Most resource creation requires Azure CLI - use it to set up test resources before testing MCP's discovery and inspection capabilities. This bug bash focuses on: - **Multi-platform compatibility** (Windows, macOS, Linux) @@ -61,7 +61,7 @@ We encourage you to test the following areas: ### Authentication Testing - [ ] Test **Azure CLI authentication** (`az login`) -- [ ] Test **Azure PowerShell authentication** (`ConnectAzAccount`) +- [ ] Test **Azure PowerShell authentication** (`Connect-AzAccount`) - [ ] Test **Interactive browser authentication** - [ ] Test authentication across **multiple tenants** - [ ] Test authentication with **service principals** @@ -130,7 +130,6 @@ If you're short on time, try these quick scenarios: ## Resources ### Bug Bash Documentation -- **[TESTING-SCOPE.md](TESTING-SCOPE.md)** - ⚠️ **READ THIS FIRST!** - Explains what MCP can and cannot do - [Installation Testing Guide](installation-testing.md) - Test installation across platforms and IDEs - [Testing Scenarios](scenarios/) - Detailed end-to-end testing scenarios From fbf2a717201cd6be265375acdfab5ebf128e2902 Mon Sep 17 00:00:00 2001 From: Vinay Gera Date: Wed, 15 Oct 2025 11:04:56 -0700 Subject: [PATCH 05/10] update installation instructions --- docs/bug-bash/installation-testing.md | 106 ++++++++++++-------------- 1 file changed, 48 insertions(+), 58 deletions(-) diff --git a/docs/bug-bash/installation-testing.md b/docs/bug-bash/installation-testing.md index c58a4e44e2..40ebeed5db 100644 --- a/docs/bug-bash/installation-testing.md +++ b/docs/bug-bash/installation-testing.md @@ -29,18 +29,12 @@ Before testing installation, ensure you have: # If not installed, download from https://nodejs.org/ ``` -2. **Install via VS Code Extension** (Recommended) - - Open VS Code - - Go to Extensions (Ctrl+Shift+X) - - Search for "Azure MCP Server" - - Click Install - -3. **Install via NPM** (Alternative) +3. **Install via NPM** ```powershell npm install -g @azure/mcp@latest ``` -4. **Install via NuGet** (Alternative) +4. **Install via NuGet** ```powershell dotnet tool install Azure.Mcp ``` @@ -78,18 +72,12 @@ azmcp server start --help brew install node ``` -2. **Install via VS Code Extension** (Recommended) - - Open VS Code - - Go to Extensions (Cmd+Shift+X) - - Search for "Azure MCP Server" - - Click Install - -3. **Install via NPM** (Alternative) +2. **Install via NPM** ```bash npm install -g @azure/mcp@latest ``` -4. **Install via .NET Tool** (Alternative) +3. **Install via .NET Tool** ```bash dotnet tool install Azure.Mcp ``` @@ -138,18 +126,12 @@ azmcp server start --help sudo dnf install nodejs npm ``` -2. **Install via VS Code Extension** (Recommended) - - Open VS Code - - Go to Extensions (Ctrl+Shift+X) - - Search for "Azure MCP Server" - - Click Install - -3. **Install via NPM** (Alternative) +2. **Install via NPM** (Alternative) ```bash npm install -g @azure/mcp@latest ``` -4. **Install via .NET Tool** (Alternative) +3. **Install via .NET Tool** (Alternative) ```bash dotnet tool install Azure.Mcp ``` @@ -189,8 +171,7 @@ azmcp server start --help 1. **Install Extension** - Method 1: Via Extension Marketplace - - Method 2: Via Command Palette (`ext install ms-azuretools.vscode-azure-mcp-server`) - - Method 3: Via [Installation Link](https://vscode.dev/redirect?url=vscode:extension/ms-azuretools.vscode-azure-mcp-server) + - Method 2: Via [Installation Link](https://vscode.dev/redirect?url=vscode:extension/ms-azuretools.vscode-azure-mcp-server) 2. **Verify Installation** ``` @@ -201,11 +182,6 @@ azmcp server start --help 5. Check Output window for startup logs ``` -3. **Test Configuration** - - Open `.vscode/mcp.json` (if it exists) - - Verify server configuration - - Test with custom settings - #### Things to Test - [ ] Extension installs without errors - [ ] Extension appears in Extensions list @@ -233,29 +209,34 @@ azmcp server start --help #### Installation Testing -1. **Install Extension** +1. **Install GitHub Copilot for Azure Extension** ``` 1. Open Visual Studio 2022 - 2. Go to Extensions > Manage Extensions - 3. Search for "GitHub Copilot for Azure" - 4. Click Install - 5. Restart Visual Studio + 2. Go to Extensions → Manage Extensions... + 3. Switch to the Browse tab in Extension Manager + 4. Search for "GitHub Copilot for Azure" + 5. Click Install + 6. Restart Visual Studio when prompted ``` 2. **Verify Installation** ``` 1. Open a solution or project 2. Open GitHub Copilot Chat - 3. Switch to Agent mode - 4. Verify Azure MCP tools are available + 3. Select Agent Mode + 4. Click the tools icon to view available tools + 5. Search for "Azure MCP Server" to filter results + 6. Verify Azure MCP tools appear in the list ``` #### Things to Test -- [ ] Extension installs without errors -- [ ] Extension appears in Extensions list -- [ ] Extension integrates with Copilot -- [ ] Tools are accessible in chat -- [ ] Server responds to commands +- [ ] GitHub Copilot for Azure extension installs successfully +- [ ] Extension appears in Extensions list after restart +- [ ] GitHub Copilot Chat opens correctly +- [ ] Can switch to Agent Mode +- [ ] Tools icon shows available MCP tools +- [ ] Can search/filter for "Azure MCP Server" tools +- [ ] Server responds to commands in Agent mode #### Performance Checks - [ ] Extension load time: _____ seconds @@ -272,29 +253,43 @@ azmcp server start --help #### Installation Testing -1. **Install Plugins** +1. **Install GitHub Copilot Plugin** ``` 1. Open IntelliJ IDEA 2. Go to Settings/Preferences > Plugins 3. Search for "GitHub Copilot" 4. Install GitHub Copilot plugin - 5. Search for "Azure Toolkit for IntelliJ" - 6. Install Azure Toolkit - 7. Restart IDE + 5. Restart IDE ``` -2. **Verify Installation** +2. **Configure Azure MCP Server** + ``` + 1. Go to File → Settings (or IntelliJ IDEA → Preferences on macOS) + 2. Navigate to Tools → AI Assistant → Model Context Protocol (MCP) + 3. Click the + icon to add a new MCP server + 4. In the "New MCP Server" dialog, enter: + - Name: Azure MCP Server + - Command: npx (or azmcp if installed via dotnet tool) + - Arguments: -y @azure/mcp@latest server start + 5. Click OK to save + 6. Restart AI Assistant if prompted + ``` + +3. **Verify Installation** ``` 1. Open GitHub Copilot Chat 2. Verify Azure MCP tools are available - 3. Test a simple command + 3. Test a simple command (e.g., "List my Azure subscriptions") ``` #### Things to Test -- [ ] Plugins install without conflicts -- [ ] Plugins appear in Plugins list -- [ ] Azure MCP integrates with Copilot -- [ ] Tools are accessible +- [ ] GitHub Copilot plugin installs without errors +- [ ] Plugin appears in Plugins list +- [ ] MCP configuration dialog is accessible +- [ ] Can add Azure MCP Server configuration +- [ ] Server configuration saves successfully +- [ ] Azure MCP integrates with Copilot Chat +- [ ] Tools are accessible in chat - [ ] Server responds to commands #### Performance Checks @@ -302,11 +297,6 @@ azmcp server start --help - [ ] Memory usage: _____ MB - [ ] CPU usage during idle: _____ % ---- - ---- - ---- ## UI-Based Mode Toggling Testing (Distribution-Specific) From 4aa4cd005ea8c2353520961f4819d2501983edeb Mon Sep 17 00:00:00 2001 From: Vinay Gera Date: Wed, 15 Oct 2025 11:08:21 -0700 Subject: [PATCH 06/10] update README --- docs/bug-bash/README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/bug-bash/README.md b/docs/bug-bash/README.md index b3167ec6be..84c626e910 100644 --- a/docs/bug-bash/README.md +++ b/docs/bug-bash/README.md @@ -15,8 +15,6 @@ Welcome to the Azure MCP Server Bug Bash! We're excited to have you help us impr The Azure MCP Server enables AI agents to interact with Azure services through natural language commands. As we continue to enhance the server, we need your help to identify issues across different platforms, IDEs, and usage scenarios. -> **Important**: **MCP tools primarily focus on reading, querying, and inspecting Azure resources**. Most resource creation requires Azure CLI - use it to set up test resources before testing MCP's discovery and inspection capabilities. - This bug bash focuses on: - **Multi-platform compatibility** (Windows, macOS, Linux) - **Installation and setup** across different IDEs From 513d7239ee52f9e33b54a977cffc2c0927d41ef7 Mon Sep 17 00:00:00 2001 From: Vinay Gera Date: Wed, 15 Oct 2025 12:02:31 -0700 Subject: [PATCH 07/10] address README + refactor docs --- docs/bug-bash/README.md | 100 +++++++--- docs/bug-bash/scenarios/agent-building.md | 2 +- docs/bug-bash/scenarios/full-stack-apps.md | 205 +-------------------- docs/bug-bash/scenarios/infra-as-code.md | 164 ----------------- docs/bug-bash/scenarios/paas-services.md | 121 ------------ 5 files changed, 78 insertions(+), 514 deletions(-) diff --git a/docs/bug-bash/README.md b/docs/bug-bash/README.md index 84c626e910..fbce9d872c 100644 --- a/docs/bug-bash/README.md +++ b/docs/bug-bash/README.md @@ -17,7 +17,7 @@ The Azure MCP Server enables AI agents to interact with Azure services through n This bug bash focuses on: - **Multi-platform compatibility** (Windows, macOS, Linux) -- **Installation and setup** across different IDEs +- **Installation and setup** across different IDEs and package managers - **Resource discovery and inspection** - testing how well MCP finds and retrieves information - **Querying capabilities** - testing database queries and data retrieval - **Monitoring and diagnostics** - testing log/metric access and health checks @@ -29,46 +29,78 @@ This bug bash focuses on: The primary goals of this bug bash are to: -1. **Validate cross-platform compatibility** - Ensure the server works reliably on Windows, macOS, and Linux -2. **Verify installation experience** - Test installation across VS Code, Visual Studio, and IntelliJ IDEA -3. **Assess performance** - Monitor memory consumption and CPU usage under typical workloads -4. **Validate authentication** - Ensure auth works consistently across all platforms -5. **Test server modes** - Verify single, namespace, and all modes work as expected -6. **Validate feature flags** - Test enabling/disabling server features -7. **Exercise real-world scenarios** - Run through common developer workflows +1. **Exercise real-world scenarios** - Run through common developer workflows +2. **Validate cross-platform compatibility** - Ensure the server works reliably on Windows, macOS, and Linux +3. **Verify installation experience** - Test installation across VS Code, Visual Studio, and IntelliJ IDEA +4. **Assess performance** - Monitor memory consumption and CPU usage under typical workloads +5. **Validate authentication** - Ensure auth works consistently across all platforms +6. **Test server modes** - Verify single, namespace, and all modes work as expected +7. **Validate feature flags** - Test enabling/disabling server features ## What to Test We encourage you to test the following areas: ### Platform Testing -- [ ] **Windows** - Test on Windows 10/11 +- [ ] **Windows** - Test on Windows 11 - [ ] **macOS** - Test on macOS (Intel and Apple Silicon) - [ ] **Linux** - Test on Ubuntu, Fedora, or other distributions ### IDE Installation - [ ] **VS Code** - Stable and Insiders versions - [ ] **Visual Studio 2022** - Community, Professional, or Enterprise -- [ ] **IntelliJ IDEA** - Ultimate or Community editions +- [ ] **IntelliJ IDEA** - Ultimate or Community editions (2025.2+) +- [ ] **Claude Desktop** - macOS and Windows +- [ ] **Cursor** - AI-first code editor +- [ ] **Windsurf** - Codeium Cascade editor +- [ ] **Amazon Q Developer** - AWS IDE integration +- [ ] **Claude Code** - Web-based Claude interface ### Performance Monitoring + +**How to Monitor Performance:** + +**Windows:** +- Open Task Manager (Ctrl+Shift+Esc) +- Find `azmcp.exe` process +- Monitor Memory and CPU columns during operations + +**macOS:** +- Open Activity Monitor (Applications → Utilities → Activity Monitor) +- Search for `azmcp` process +- Monitor CPU % and Memory columns + +**Linux:** +- Use `htop` or `top` command +- Filter for `azmcp` process +- Monitor %CPU and RES (memory) columns + +**What to Test:** - [ ] Monitor **memory consumption** during typical operations - [ ] Monitor **CPU usage** during command execution - [ ] Test with **multiple concurrent operations** -- [ ] Observe behavior during **long-running sessions** +- [ ] Observe behavior during **long-running sessions** (2+ hours): + - Memory usage trends (stable, growing, or leaking) + - Server responsiveness (does it slow down over time?) + - Error frequency (do errors increase with time?) +- [ ] Record baseline memory usage at startup +- [ ] Check for memory leaks after extended use ### Authentication Testing - [ ] Test **Azure CLI authentication** (`az login`) - [ ] Test **Azure PowerShell authentication** (`Connect-AzAccount`) -- [ ] Test **Interactive browser authentication** -- [ ] Test authentication across **multiple tenants** -- [ ] Test authentication with **service principals** +- [ ] Test **Interactive browser authentication** - Set `AZURE_MCP_ONLY_USE_BROKER_CREDENTIAL=true` and sign in through the broker/browser when innvoking tools via Azure MCP Server. See [Authentication Guide](https://github.com/microsoft/mcp/blob/main/docs/Authentication.md#authentication-fundamentals) for details. +- [ ] Test authentication across **multiple tenants** - Switch between different Azure AD tenants ### Server Mode Testing -- [ ] **All mode** - All tools exposed individually -- [ ] **Namespace mode** - Tools grouped by Azure service -- [ ] **Single mode** - Single dynamic proxy tool -- [ ] **Filtered namespaces** - Specific services only + +See [Server Mode Testing](installation-testing.md#server-mode-testing) for detailed instructions. + +- [ ] **Namespace mode** (default) - Tools grouped by Azure service (~40-50 tools) +- [ ] **All mode** - All tools exposed individually (100+ tools) +- [ ] **Single mode** - Single unified tool with internal routing +- [ ] **Read-only mode** - Blocks all write/destructive operations +- [ ] **Namespace filtering** - Expose specific services only (e.g., storage, keyvault) ### Feature Flag Testing - [ ] Enable/disable server @@ -91,7 +123,7 @@ When you find a bug or issue, please report it on GitHub: **Required Information:** - **Platform**: Windows/macOS/Linux (include version) -- **IDE**: VS Code/Visual Studio/IntelliJ (include version) +- **IDE/Client**: VS Code/Visual Studio/IntelliJ/Claude Desktop/Cursor/Windsurf/Amazon Q/Claude Code (include version) - **Azure MCP Server Version**: Found in extension details or `azmcp --version` - **Node.js Version** (if using npm): Run `node --version` - **Description**: Clear description of the issue @@ -114,16 +146,36 @@ We've prepared detailed testing guides for common scenarios: 5. **[Database Operations](scenarios/database-operations.md)** - Work with Cosmos DB, PostgreSQL, and Azure SQL 6. **[Deployment Scenarios](scenarios/deployment.md)** - Deploy resources and applications 7. **[Full Stack Applications](scenarios/full-stack-apps.md)** - Build complete apps with database backends -9. **[Agent Building](scenarios/agent-building.md)** - Create and deploy Azure Foundry agents +8. **[Agent Building](scenarios/agent-building.md)** - Create and deploy Azure Foundry agents ### Quick Start Scenarios If you're short on time, try these quick scenarios: -- **5 minutes**: Install Azure MCP extension and verify tools are loaded -- **10 minutes**: List your Azure resources (subscriptions, resource groups, storage accounts) -- **15 minutes**: Create a simple storage account and upload a file -- **30 minutes**: Create a basic web app and deploy it to Azure App Service +**5 minutes: Install and Verify** +- Install Azure MCP extension/server in your IDE +- Verify tools are loaded +- Try these prompts: + - `"What Azure MCP tools are available?"` + - `"Show me my subscriptions"` + - `"List all resource groups in my subscription"` + +**10 minutes: Resource Discovery** +- List your Azure resources (subscriptions, resource groups, storage accounts) +- Try these prompts: + - `"List all storage accounts in my subscription"` + - `"Show me my Key Vaults"` + - `"List all App Services in resource group "` + +**15 minutes: Database Inspection** +- Inspect existing databases and query data +- Try these prompts: + - `"List all cosmosdb accounts in my subscription"` + - `"Show me databases in cosmosdb account "` + - `"List all containers in database in cosmosdb account "` + +**30 minutes: End-to-End Scenario** +- Follow one of the [detailed scenario guides](scenarios/) like Storage Operations or Database Operations ## Resources diff --git a/docs/bug-bash/scenarios/agent-building.md b/docs/bug-bash/scenarios/agent-building.md index d4051ab472..b6787b8f41 100644 --- a/docs/bug-bash/scenarios/agent-building.md +++ b/docs/bug-bash/scenarios/agent-building.md @@ -1,4 +1,4 @@ -# 🤖 AI Agent Building Testing Scenario +# AI Agent Building Testing Scenario > **MCP Tool Support Notice** > Azure MCP Server provides **AI Foundry resource inspection, model listing, and agent interaction** capabilities. Resource creation and deployment require Azure CLI or Portal. This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. diff --git a/docs/bug-bash/scenarios/full-stack-apps.md b/docs/bug-bash/scenarios/full-stack-apps.md index b0ca46c352..ded0010089 100644 --- a/docs/bug-bash/scenarios/full-stack-apps.md +++ b/docs/bug-bash/scenarios/full-stack-apps.md @@ -1,4 +1,4 @@ -# 🌐 Full Stack Application Testing Scenario +# Full Stack Application Testing Scenario > **MCP Tool Support Notice** > Azure MCP Server provides **database inspection, querying, and resource discovery** capabilities. Application code generation, resource creation, and deployment require external tools (Azure CLI, GitHub Copilot code generation, deployment tools). This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. @@ -207,209 +207,6 @@ az group delete --name bugbash-fullstack-rg --yes --no-wait - End-to-end workflow complete - Data changes visible through MCP queries ---- - -## Scenario 2: Python Flask App with PostgreSQL Backend End-to-End - -**Objective**: Complete workflow for Python application with relational database backend - -### Step 1: Create PostgreSQL Resources (External - Not MCP) - -> **External Setup Required**: Use Azure CLI to create PostgreSQL server - -```bash -# Create resource group -az group create --name bugbash-flask-rg --location eastus - -# Create PostgreSQL flexible server -az postgres flexible-server create \ - --name bugbash-postgres-$RANDOM \ - --resource-group bugbash-flask-rg \ - --location eastus \ - --admin-user dbadmin \ - --admin-password \ - --sku-name Standard_B1ms \ - --tier Burstable \ - --version 14 \ - --storage-size 32 \ - --public-access 0.0.0.0 - -# Create database -az postgres flexible-server db create \ - --resource-group bugbash-flask-rg \ - --server-name \ - --database-name inventory - -# Create table and insert data (use Azure Portal Query Editor or psql) -# CREATE TABLE products ( -# id SERIAL PRIMARY KEY, -# name VARCHAR(100), -# description TEXT, -# price DECIMAL(10,2), -# stock INTEGER -# ); -# -# INSERT INTO products (name, description, price, stock) VALUES -# ('Laptop', 'High-performance laptop', 999.99, 10), -# ('Mouse', 'Wireless mouse', 29.99, 50), -# ('Keyboard', 'Mechanical keyboard', 79.99, 30); -``` - -### Step 2: Discover PostgreSQL Resources with Azure MCP Server - -**2.1 List PostgreSQL servers** (uses `azmcp_postgres_server_list`): -``` -List all PostgreSQL servers in my subscription -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_postgres_server_list` -- [ ] Your PostgreSQL server appears -- [ ] Server details shown - -**2.2 List databases** (uses `azmcp_postgres_database_list`): -``` -List all databases in PostgreSQL server '' in resource group 'bugbash-flask-rg' -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_postgres_database_list` -- [ ] 'inventory' database listed - -**2.3 List tables** (uses `azmcp_postgres_table_list`): -``` -List all tables in database 'inventory' on server '' in resource group 'bugbash-flask-rg' -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_postgres_table_list` -- [ ] 'products' table listed - -### Step 3: Inspect Table Schema with Azure MCP Server - -**3.1 Get table schema** (uses `azmcp_postgres_table_schema_get`): -``` -Show me the schema of table 'products' in database 'inventory' on server '' in resource group 'bugbash-flask-rg' -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_postgres_table_schema_get` -- [ ] All columns listed (id, name, description, price, stock) -- [ ] Data types shown -- [ ] Primary key visible - -### Step 4: Query Application Data with Azure MCP Server - -**4.1 Query all products** (uses `azmcp_postgres_database_query`): -``` -Show me all products in the 'products' table in database 'inventory' on server '' in resource group 'bugbash-flask-rg' -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_postgres_database_query` -- [ ] All 3 products returned -- [ ] Product details visible - -**4.2 Query with filter**: -``` -Show me all products with price less than 100 in database 'inventory' on server '' in resource group 'bugbash-flask-rg' -``` - -**Verify**: -- [ ] Only Mouse and Keyboard returned -- [ ] Price filter works correctly - -### Step 5: Build and Deploy Flask App (External Tools - Not MCP) - -> **External Development Required**: Use GitHub Copilot for code generation - -**Generate Flask application** (use GitHub Copilot Chat): -``` -Create a Python Flask application for inventory management with: -- Flask web framework -- psycopg2 for PostgreSQL -- API endpoints for products CRUD -- Simple HTML frontend -- Connection to PostgreSQL '' -``` - -**Deploy to Azure** (use Azure CLI): -```bash -# Create App Service Plan -az appservice plan create \ - --name bugbash-flask-plan \ - --resource-group bugbash-flask-rg \ - --sku B1 \ - --is-linux - -# Create Web App -az webapp create \ - --name bugbash-inventory-$RANDOM \ - --resource-group bugbash-flask-rg \ - --plan bugbash-flask-plan \ - --runtime "PYTHON:3.11" - -# Configure database connection -az webapp config appsettings set \ - --name \ - --resource-group bugbash-flask-rg \ - --settings DATABASE_URL="" - -# Deploy application -cd inventory-app -zip -r app.zip . -az webapp deployment source config-zip \ - --resource-group bugbash-flask-rg \ - --name \ - --src app.zip -``` - -### Step 6: Verify Deployed Application with Azure MCP Server - -**6.1 Get App Service details** (uses `azmcp_appservice_get`): -``` -Show me details for App Service '' in resource group 'bugbash-flask-rg' -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_appservice_get` -- [ ] Flask app details shown -- [ ] Python runtime visible - -**6.2 Test application**: -- Open the App Service URL -- Test product CRUD operations -- Verify database connectivity - -### Step 7: Query Updated Data with Azure MCP Server - -**7.1 Verify application changes in database** (uses `azmcp_postgres_database_query`): -``` -Show me all products in database 'inventory' to verify changes made through my Flask application -``` - -**Verify**: -- [ ] New products created via app visible -- [ ] Updated products show changes -- [ ] Deleted products are gone -- [ ] Database state matches app operations - -### Step 8: Cleanup (External - Not MCP) - -```bash -# Delete resource group (removes all resources) -az group delete --name bugbash-flask-rg --yes --no-wait -``` - -**Expected Results**: -- PostgreSQL discovery works -- Table schema inspection accurate -- Database querying functional -- App Service inspection works -- End-to-end workflow complete - ---- - ## Common Issues to Watch For | Issue | Description | Resolution | diff --git a/docs/bug-bash/scenarios/infra-as-code.md b/docs/bug-bash/scenarios/infra-as-code.md index dc17b6451f..067207700c 100644 --- a/docs/bug-bash/scenarios/infra-as-code.md +++ b/docs/bug-bash/scenarios/infra-as-code.md @@ -160,170 +160,6 @@ az group delete --name bugbash-iac-rg --yes --no-wait - Template generation guided by schema - Deployment verification through MCP tools ---- - -## Scenario 2: Terraform Best Practices End-to-End - -**Objective**: Complete workflow using Terraform guidance and deployment - -### Step 1: Get Terraform Best Practices with Azure MCP Server - -**1.1 Get general Terraform best practices** (uses `azmcp_azureterraformbestpractices_get`): -``` -Show me Azure Terraform best practices for resource deployment -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_azureterraformbestpractices_get` -- [ ] Best practices document returned -- [ ] Security recommendations included -- [ ] State management guidance provided -- [ ] Naming conventions explained - -**1.2 Alternative phrasing**: -``` -What are the best practices for writing Terraform code for Azure? -``` - -**Verify**: -- [ ] Same tool invoked -- [ ] Comprehensive guidance provided - -### Step 2: Get IaC Rules for Terraform with Azure MCP Server - -**2.1 Get Terraform rules for resources** (uses `azmcp_deploy_iac_rules_get`): -``` -Show me the IaC generation rules for Azure Storage Account and Key Vault using Terraform -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_deploy_iac_rules_get` -- [ ] Terraform-specific rules provided -- [ ] Resource configuration guidance included -- [ ] Security settings highlighted - -**2.2 Get rules for compute resources**: -``` -Show me the Terraform IaC rules for Azure Virtual Machine and Virtual Network -``` - -**Verify**: -- [ ] VM and VNet rules provided -- [ ] Network configuration guidance included -- [ ] Security group recommendations shown - -### Step 3: Create Terraform Configuration (External Tool - Not MCP) - -> **External Tool Required**: Create Terraform files manually or use GitHub Copilot - -**Create `main.tf`**: -```hcl -terraform { - required_providers { - azurerm = { - source = "hashicorp/azurerm" - version = "~> 3.0" - } - } -} - -provider "azurerm" { - features {} -} - -resource "azurerm_resource_group" "main" { - name = "bugbash-terraform-rg" - location = "eastus" -} - -resource "azurerm_storage_account" "main" { - name = "bugbashterraform${random_string.suffix.result}" - resource_group_name = azurerm_resource_group.main.name - location = azurerm_resource_group.main.location - account_tier = "Standard" - account_replication_type = "LRS" - - tags = { - environment = "testing" - project = "bug-bash" - } -} - -resource "random_string" "suffix" { - length = 8 - special = false - upper = false -} - -output "storage_account_name" { - value = azurerm_storage_account.main.name -} -``` - -### Step 4: Deploy with Terraform (External Tool - Not MCP) - -> **External Deployment Required**: Use Terraform CLI - -```bash -# Initialize Terraform -terraform init - -# Plan deployment -terraform plan -out=tfplan - -# Apply configuration -terraform apply tfplan -``` - -### Step 5: Verify Deployment with Azure MCP Server - -**5.1 List storage accounts** (uses `azmcp_storage_account_get`): -``` -List all storage accounts in my subscription -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_storage_account_get` -- [ ] Terraform-deployed storage account appears -- [ ] Tags match configuration - -**5.2 Get storage account details**: -``` -Show me details for storage account '' -``` - -**Verify**: -- [ ] Account properties match Terraform configuration -- [ ] Replication type is LRS -- [ ] Location is East US - -### Step 6: Get Azure Best Practices with Azure MCP Server - -**6.1 Get general Azure best practices** (uses `azmcp_bestpractices_get`): -``` -Show me Azure best practices for code generation -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_bestpractices_get` -- [ ] General Azure best practices provided -- [ ] Covers multiple resource types -- [ ] Security and compliance guidance included - -### Step 7: Cleanup (Terraform - Not MCP) - -```bash -# Destroy infrastructure -terraform destroy -auto-approve -``` - -**Expected Results**: -- Terraform best practices retrieved successfully -- IaC rules for Terraform provided -- Azure general best practices available -- Deployment verification through MCP tools - ---- ## Common Issues to Watch For diff --git a/docs/bug-bash/scenarios/paas-services.md b/docs/bug-bash/scenarios/paas-services.md index 4a96b39736..ce38856c59 100644 --- a/docs/bug-bash/scenarios/paas-services.md +++ b/docs/bug-bash/scenarios/paas-services.md @@ -142,127 +142,6 @@ az group delete --name bugbash-paas-rg --yes --no-wait --- -## Scenario 2: Azure Function App Discovery End-to-End - -**Objective**: Complete workflow for Function App inspection and discovery - -### Step 1: Create Function App Resources (External - Not MCP) - -> **External Setup Required**: Use Azure CLI to create Function App - -```bash -# Create resource group -az group create --name bugbash-functions-rg --location eastus - -# Create storage account (required for Functions) -az storage account create \ - --name bugbashfuncstorage$RANDOM \ - --resource-group bugbash-functions-rg \ - --location eastus \ - --sku Standard_LRS - -# Create Function App -az functionapp create \ - --name bugbash-funcapp-$RANDOM \ - --resource-group bugbash-functions-rg \ - --storage-account \ - --consumption-plan-location eastus \ - --runtime node \ - --runtime-version 18 \ - --functions-version 4 - -# Create another Function App for comparison -az functionapp create \ - --name bugbash-funcapp2-$RANDOM \ - --resource-group bugbash-functions-rg \ - --storage-account \ - --consumption-plan-location eastus \ - --runtime python \ - --runtime-version 3.11 \ - --functions-version 4 -``` - -### Step 2: Discover Function Apps with Azure MCP Server - -**2.1 List all Function Apps** (uses `azmcp_functionapp_get`): -``` -List all Function Apps in my subscription -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_functionapp_get` -- [ ] Both Function Apps appear -- [ ] Function App properties shown (name, location, runtime, plan) - -**2.2 Get specific Function App details**: -``` -Show me details for Function App '' in resource group 'bugbash-functions-rg' -``` - -**Verify**: -- [ ] Detailed properties displayed -- [ ] Runtime stack shown (Node.js 18) -- [ ] Consumption plan indicated -- [ ] Storage account connection visible -- [ ] Default hostname shown - -**2.3 Filter by resource group**: -``` -List all Function Apps in resource group 'bugbash-functions-rg' -``` - -**Verify**: -- [ ] Only Function Apps in specified resource group listed -- [ ] Filter works correctly - -### Step 3: Compare Function App Configurations - -**3.1 Compare runtimes**: -- Note Node.js Function App properties -- Note Python Function App properties -- Verify runtime differences are visible - -**3.2 Identify configuration differences**: -``` -Compare the configurations of Function Apps '' and '' -``` - -**Verify**: -- [ ] Runtime versions differ -- [ ] Both use Consumption plan -- [ ] Storage accounts may differ - -### Step 4: Storage Account Verification with Azure MCP Server - -**4.1 List storage accounts** (uses `azmcp_storage_account_get`): -``` -List all storage accounts in my subscription -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_storage_account_get` -- [ ] Function storage account appears -- [ ] Storage properties shown - -**4.2 Verify Function App storage connection**: -- Check that Function App uses the created storage account -- Verify storage account location matches Function App - -### Step 5: Cleanup (External - Not MCP) - -```bash -# Delete resource group (removes all resources) -az group delete --name bugbash-functions-rg --yes --no-wait -``` - -**Expected Results**: -- Function App discovery works correctly -- Multiple Function Apps listed successfully -- Detailed inspection shows runtime and configuration -- Resource group filtering functional -- Storage account relationship visible - ---- ## Common Issues to Watch For | Issue | Description | Resolution | From e7fd71359ff4ea35cb6715782477da3e78625472 Mon Sep 17 00:00:00 2001 From: Vinay Gera Date: Wed, 15 Oct 2025 12:16:28 -0700 Subject: [PATCH 08/10] use absolute links --- docs/bug-bash/README.md | 24 +-- docs/bug-bash/installation-testing.md | 4 +- docs/bug-bash/scenarios/agent-building.md | 173 +----------------- .../bug-bash/scenarios/database-operations.md | 6 +- docs/bug-bash/scenarios/full-stack-apps.md | 6 +- docs/bug-bash/scenarios/infra-as-code.md | 4 +- 6 files changed, 25 insertions(+), 192 deletions(-) diff --git a/docs/bug-bash/README.md b/docs/bug-bash/README.md index fbce9d872c..da13702b58 100644 --- a/docs/bug-bash/README.md +++ b/docs/bug-bash/README.md @@ -94,7 +94,7 @@ We encourage you to test the following areas: ### Server Mode Testing -See [Server Mode Testing](installation-testing.md#server-mode-testing) for detailed instructions. +See [Server Mode Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/installation-testing.md#server-mode-testing) for detailed instructions. - [ ] **Namespace mode** (default) - Tools grouped by Azure service (~40-50 tools) - [ ] **All mode** - All tools exposed individually (100+ tools) @@ -139,14 +139,14 @@ We've prepared detailed testing guides for common scenarios: ### Scenario Guides -1. **[Installation Testing](installation-testing.md)** - Test installation across different platforms and IDEs -2. **[Infrastructure as Code](scenarios/infra-as-code.md)** - Generate and deploy Azure infrastructure -3. **[PaaS Services](scenarios/paas-services.md)** - Work with App Service, Container Apps, and Functions -4. **[Storage Operations](scenarios/storage-operations.md)** - Test blob storage and file operations -5. **[Database Operations](scenarios/database-operations.md)** - Work with Cosmos DB, PostgreSQL, and Azure SQL -6. **[Deployment Scenarios](scenarios/deployment.md)** - Deploy resources and applications -7. **[Full Stack Applications](scenarios/full-stack-apps.md)** - Build complete apps with database backends -8. **[Agent Building](scenarios/agent-building.md)** - Create and deploy Azure Foundry agents +1. **[Installation Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/installation-testing.md)** - Test installation across different platforms and IDEs +2. **[Infrastructure as Code](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/infra-as-code.md)** - Generate and deploy Azure infrastructure +3. **[PaaS Services](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/paas-services.md)** - Work with App Service, Container Apps, and Functions +4. **[Storage Operations](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/storage-operations.md)** - Test blob storage and file operations +5. **[Database Operations](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/database-operations.md)** - Work with Cosmos DB, PostgreSQL, and Azure SQL +6. **[Deployment Scenarios](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/deployment.md)** - Deploy resources and applications +7. **[Full Stack Applications](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/full-stack-apps.md)** - Build complete apps with database backends +8. **[Agent Building](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/agent-building.md)** - Create and deploy Azure Foundry agents ### Quick Start Scenarios @@ -175,13 +175,13 @@ If you're short on time, try these quick scenarios: - `"List all containers in database in cosmosdb account "` **30 minutes: End-to-End Scenario** -- Follow one of the [detailed scenario guides](scenarios/) like Storage Operations or Database Operations +- Follow one of the [detailed scenario guides](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios) like Storage Operations or Database Operations ## Resources ### Bug Bash Documentation -- [Installation Testing Guide](installation-testing.md) - Test installation across platforms and IDEs -- [Testing Scenarios](scenarios/) - Detailed end-to-end testing scenarios +- [Installation Testing Guide](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/installation-testing.md) - Test installation across platforms and IDEs +- [Testing Scenarios](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios) - Detailed end-to-end testing scenarios ### Azure MCP Server Documentation - [Azure MCP Server Documentation](https://learn.microsoft.com/azure/developer/azure-mcp-server/) diff --git a/docs/bug-bash/installation-testing.md b/docs/bug-bash/installation-testing.md index 40ebeed5db..d445e1662d 100644 --- a/docs/bug-bash/installation-testing.md +++ b/docs/bug-bash/installation-testing.md @@ -720,9 +720,9 @@ az login ## Related Resources -- [Main Bug Bash Guide](README.md) +- [Main Bug Bash Guide](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/README.md) - [Troubleshooting Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/TROUBLESHOOTING.md) - [Installation Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/README.md#installation) - [Report Issues](https://github.com/microsoft/mcp/issues) -**Next Steps**: After completing installation testing, proceed to [Scenario Testing](scenarios/README.md). +**Next Steps**: After completing installation testing, proceed to [Scenario Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios). diff --git a/docs/bug-bash/scenarios/agent-building.md b/docs/bug-bash/scenarios/agent-building.md index b6787b8f41..b426c2bcb8 100644 --- a/docs/bug-bash/scenarios/agent-building.md +++ b/docs/bug-bash/scenarios/agent-building.md @@ -155,173 +155,6 @@ az group delete --name bugbash-aifoundry-rg --yes --no-wait - Deployment inspection accurate - OpenAI model listing functional ---- - -## Scenario 2: AI Agent Creation & Interaction End-to-End - -**Objective**: Complete workflow for creating, querying, and evaluating AI agents - -### Step 1: Setup Agent Resources (External - Not MCP) - -> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. - -**Option A: Prompt GitHub Copilot Chat** (Recommended): -``` -Create an Azure resource group 'bugbash-agents-rg' in eastus, create an Azure AI Services account with SKU S0, deploy GPT-4o model with deployment name 'agent-gpt4o', then show me the endpoint URL -``` - -**Option B: Run Azure CLI Commands Manually**: -```bash -# Create resource group -az group create --name bugbash-agents-rg --location eastus - -# Create Azure AI Foundry resource -az cognitiveservices account create \ - --name bugbash-agents-$RANDOM \ - --resource-group bugbash-agents-rg \ - --location eastus \ - --kind AIServices \ - --sku S0 - -# Get endpoint URL (you'll need this) -az cognitiveservices account show \ - --name \ - --resource-group bugbash-agents-rg \ - --query "properties.endpoint" -o tsv - -# Deploy GPT-4o for agent use -az cognitiveservices account deployment create \ - --name \ - --resource-group bugbash-agents-rg \ - --deployment-name agent-gpt4o \ - --model-name gpt-4o \ - --model-version "2024-05-13" \ - --model-format OpenAI \ - --sku-capacity 10 \ - --sku-name Standard -``` - -**Option C: Create Agent via Azure AI Foundry Portal**: -1. Go to [Azure AI Foundry Portal](https://ai.azure.com) -2. Create a new project -3. Create an agent with your deployed model -4. Note the agent ID for testing - -### Step 2: List Available Agents with Azure MCP Server - -**2.1 List all agents** (uses `azmcp_foundry_agents_list`): -``` -List all AI agents in my Azure AI Foundry project with endpoint '' -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_foundry_agents_list` -- [ ] Your created agent appears -- [ ] Agent properties shown (ID, name, model) - -**2.2 Alternative phrasing**: -``` -Show me all agents in my AI Foundry project -``` - -### Step 3: Query an Agent with Azure MCP Server - -**3.1 Connect and query agent** (uses `azmcp_foundry_agents_connect`): -``` -Query agent '' with: "What Azure services can you help me manage?" using endpoint '' -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_foundry_agents_connect` -- [ ] Agent response received -- [ ] Response is coherent and relevant - -**3.2 Test agent capabilities**: -``` -Ask agent '' to: "List the most common Azure resources used in web applications" using endpoint '' -``` - -**Verify**: -- [ ] Agent understands the query -- [ ] Response includes relevant Azure services -- [ ] Answer is accurate - -**3.3 Test agent with specific query**: -``` -Query agent '': "Explain the difference between Azure SQL Database and Cosmos DB" using endpoint '' -``` - -**Verify**: -- [ ] Detailed explanation provided -- [ ] Technical accuracy maintained -- [ ] Response format is clear - -### Step 4: Evaluate Agent Responses with Azure MCP Server - -**4.1 Evaluate a response** (uses `azmcp_foundry_agents_evaluate`): -``` -Evaluate the response from agent '' for query "What is Azure Storage?" with response "Azure Storage is a cloud storage solution" using evaluator 'groundedness' with Azure OpenAI endpoint '' and deployment '' -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_foundry_agents_evaluate` -- [ ] Evaluation score returned -- [ ] Groundedness metrics shown - -**4.2 Query and evaluate in one step** (uses `azmcp_foundry_agents_query-and-evaluate`): -``` -Query and evaluate agent '' with: "What are the main features of Azure Functions?" using endpoint '' and evaluate with Azure OpenAI endpoint '' and deployment '' -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_foundry_agents_query-and-evaluate` -- [ ] Agent response received -- [ ] Evaluation metrics provided -- [ ] Combined workflow successful - -### Step 5: Explore Knowledge Indexes with Azure MCP Server - -**5.1 List knowledge indexes** (uses `azmcp_foundry_knowledge_index_list`): -``` -List all knowledge indexes in my AI Foundry project with endpoint '' -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_foundry_knowledge_index_list` -- [ ] Knowledge indexes listed (if any exist) -- [ ] Index properties shown - -**5.2 Get index schema** (uses `azmcp_foundry_knowledge_index_schema`): -``` -Show me the schema for knowledge index '' using endpoint '' -``` - -**Verify**: -- [ ] Tool invoked: `azmcp_foundry_knowledge_index_schema` -- [ ] Schema information returned -- [ ] Field types and structure shown - -### Step 6: Cleanup (External - Not MCP) - -**Option A: Prompt GitHub Copilot Chat**: -``` -Delete the Azure resource group 'bugbash-agents-rg' and all its resources -``` - -**Option B: Run Azure CLI Command Manually**: -```bash -# Delete resource group (removes all resources) -az group delete --name bugbash-agents-rg --yes --no-wait -``` - -**Expected Results**: -- Agent listing works correctly -- Agent querying returns responses -- Response evaluation provides metrics -- Combined query-and-evaluate workflow functional -- Knowledge index inspection successful - ---- ## Common Issues to Watch For @@ -353,8 +186,8 @@ When logging issues, include: - [Azure OpenAI Service](https://learn.microsoft.com/azure/ai-services/openai/) - [Azure AI Agents](https://learn.microsoft.com/azure/ai-services/agents/) - [Model Context Protocol](https://modelcontextprotocol.io/) -- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) -- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) ## 💡 Quick Reference: Supported MCP Tools @@ -386,4 +219,4 @@ When logging issues, include: --- -**Next**: [Database Operations Testing](database-operations.md) \ No newline at end of file +**Next**: [Database Operations Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/database-operations.md) \ No newline at end of file diff --git a/docs/bug-bash/scenarios/database-operations.md b/docs/bug-bash/scenarios/database-operations.md index 954feab3db..70148d9b03 100644 --- a/docs/bug-bash/scenarios/database-operations.md +++ b/docs/bug-bash/scenarios/database-operations.md @@ -367,8 +367,8 @@ When logging issues, include: - [Azure Database for PostgreSQL](https://learn.microsoft.com/azure/postgresql/) - [Azure Database for MySQL](https://learn.microsoft.com/azure/mysql/) - [Azure SQL Database](https://learn.microsoft.com/azure/azure-sql/) -- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) -- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) ## 💡 Quick Reference: Supported MCP Tools @@ -408,4 +408,4 @@ When logging issues, include: --- -**Next**: [Deployment Scenarios Testing](deployment.md) +**Next**: [Deployment Scenarios Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/deployment.md) diff --git a/docs/bug-bash/scenarios/full-stack-apps.md b/docs/bug-bash/scenarios/full-stack-apps.md index ded0010089..f6aaa59735 100644 --- a/docs/bug-bash/scenarios/full-stack-apps.md +++ b/docs/bug-bash/scenarios/full-stack-apps.md @@ -236,8 +236,8 @@ When logging issues, include: - [Azure Cosmos DB Documentation](https://learn.microsoft.com/azure/cosmos-db/) - [Azure PostgreSQL Documentation](https://learn.microsoft.com/azure/postgresql/) - [Azure SQL Documentation](https://learn.microsoft.com/azure/azure-sql/) -- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) -- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) ## Quick Reference: Supported MCP Tools @@ -272,4 +272,4 @@ When logging issues, include: --- -**Next**: [Infrastructure as Code Testing](infra-as-code.md) \ No newline at end of file +**Next**: [Infrastructure as Code Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/infra-as-code.md) \ No newline at end of file diff --git a/docs/bug-bash/scenarios/infra-as-code.md b/docs/bug-bash/scenarios/infra-as-code.md index 067207700c..afd473141b 100644 --- a/docs/bug-bash/scenarios/infra-as-code.md +++ b/docs/bug-bash/scenarios/infra-as-code.md @@ -187,8 +187,8 @@ When logging issues, include: - [Azure Bicep Documentation](https://learn.microsoft.com/azure/azure-resource-manager/bicep/) - [Azure Terraform Documentation](https://learn.microsoft.com/azure/developer/terraform/) - [Azure Resource Manager](https://learn.microsoft.com/azure/azure-resource-manager/) -- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) -- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) ## Quick Reference: Supported MCP Tools From 2e95928077c5f62fc3c905fd0b37ac1a565b5623 Mon Sep 17 00:00:00 2001 From: Vinay Gera Date: Wed, 15 Oct 2025 12:35:50 -0700 Subject: [PATCH 09/10] fix relative links --- docs/bug-bash/README.md | 3 +-- docs/bug-bash/scenarios/infra-as-code.md | 2 +- docs/bug-bash/scenarios/paas-services.md | 6 +++--- docs/bug-bash/scenarios/storage-operations.md | 6 +++--- 4 files changed, 8 insertions(+), 9 deletions(-) diff --git a/docs/bug-bash/README.md b/docs/bug-bash/README.md index da13702b58..abe30f8883 100644 --- a/docs/bug-bash/README.md +++ b/docs/bug-bash/README.md @@ -194,5 +194,4 @@ If you're short on time, try these quick scenarios: - [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - Sample prompts for testing ### Support -- [GitHub Issues](https://github.com/microsoft/mcp/issues) - Report bugs and issues -- [Discussions](https://github.com/microsoft/mcp/discussions) - Ask questions and share feedback \ No newline at end of file +- [GitHub Issues](https://github.com/microsoft/mcp/issues) - Report bugs and issues \ No newline at end of file diff --git a/docs/bug-bash/scenarios/infra-as-code.md b/docs/bug-bash/scenarios/infra-as-code.md index afd473141b..b5f5da936e 100644 --- a/docs/bug-bash/scenarios/infra-as-code.md +++ b/docs/bug-bash/scenarios/infra-as-code.md @@ -210,4 +210,4 @@ When logging issues, include: --- -**Next**: [PaaS Services Testing](paas-services.md) +**Next**: [PaaS Services Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/paas-services.md) diff --git a/docs/bug-bash/scenarios/paas-services.md b/docs/bug-bash/scenarios/paas-services.md index ce38856c59..960b1c5180 100644 --- a/docs/bug-bash/scenarios/paas-services.md +++ b/docs/bug-bash/scenarios/paas-services.md @@ -168,8 +168,8 @@ When logging issues, include: - [Azure App Service Documentation](https://learn.microsoft.com/azure/app-service/) - [Azure Functions Documentation](https://learn.microsoft.com/azure/azure-functions/) -- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) -- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) ## Quick Reference: Supported MCP Tools @@ -192,4 +192,4 @@ When logging issues, include: --- -**Next**: [Storage Operations Testing](storage-operations.md) +**Next**: [Storage Operations Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/storage-operations.md) diff --git a/docs/bug-bash/scenarios/storage-operations.md b/docs/bug-bash/scenarios/storage-operations.md index bddae3d1da..f34b2f5b1c 100644 --- a/docs/bug-bash/scenarios/storage-operations.md +++ b/docs/bug-bash/scenarios/storage-operations.md @@ -356,8 +356,8 @@ When logging issues, include: - [Azure Storage Documentation](https://learn.microsoft.com/azure/storage/) - [Azure Blob Storage Best Practices](https://learn.microsoft.com/azure/storage/blobs/storage-blobs-introduction) - [Data Lake Storage Gen2](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-introduction) -- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) -- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) ## Quick Reference: Supported MCP Tools @@ -389,4 +389,4 @@ When logging issues, include: --- -**Next**: [Agent Building](agent-building.md) +**Next**: [Agent Building](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/agent-building.md) From f9693cff70f689d0197e321dd45dd8b0e0aba276 Mon Sep 17 00:00:00 2001 From: Vinay Gera Date: Wed, 15 Oct 2025 12:56:17 -0700 Subject: [PATCH 10/10] fix relative links --- docs/bug-bash/scenarios/deployment.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/bug-bash/scenarios/deployment.md b/docs/bug-bash/scenarios/deployment.md index f0fd4d6637..5c525ef898 100644 --- a/docs/bug-bash/scenarios/deployment.md +++ b/docs/bug-bash/scenarios/deployment.md @@ -378,8 +378,8 @@ When logging issues, include: - [Azure Developer CLI (azd)](https://learn.microsoft.com/azure/developer/azure-developer-cli/) - [GitHub Actions for Azure](https://learn.microsoft.com/azure/developer/github/github-actions) - [Azure DevOps Pipelines](https://learn.microsoft.com/azure/devops/pipelines/) -- [MCP Command Reference](../../servers/Azure.Mcp.Server/docs/azmcp-commands.md) -- [E2E Test Prompts](../../servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - [Report Issues](https://github.com/microsoft/mcp/issues) ## 💡 Quick Reference: Supported MCP Tools @@ -397,4 +397,4 @@ When logging issues, include: --- -**Next**: [Full Stack Applications Testing](full-stack-apps.md) +**Next**: [Full Stack Applications Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/full-stack-apps.md)