diff --git a/docs/bug-bash/README.md b/docs/bug-bash/README.md new file mode 100644 index 0000000000..abe30f8883 --- /dev/null +++ b/docs/bug-bash/README.md @@ -0,0 +1,197 @@ +# Azure MCP Server - Bug Bash Welcome Guide + +Welcome to the Azure MCP Server Bug Bash! We're excited to have you help us improve the quality and reliability of Azure MCP Server across different platforms and scenarios. + +## Table of Contents + +- [Introduction](#introduction) +- [Bug Bash Goals](#bug-bash-goals) +- [What to Test](#what-to-test) +- [How to Report Issues](#how-to-report-issues) +- [Testing Scenarios](#testing-scenarios) +- [Resources](#resources) + +## Introduction + +The Azure MCP Server enables AI agents to interact with Azure services through natural language commands. As we continue to enhance the server, we need your help to identify issues across different platforms, IDEs, and usage scenarios. + +This bug bash focuses on: +- **Multi-platform compatibility** (Windows, macOS, Linux) +- **Installation and setup** across different IDEs and package managers +- **Resource discovery and inspection** - testing how well MCP finds and retrieves information +- **Querying capabilities** - testing database queries and data retrieval +- **Monitoring and diagnostics** - testing log/metric access and health checks +- **Deployment guidance** - testing how MCP helps plan and guide deployments +- **Authentication** across different environments +- **End-to-end scenarios** that developers commonly encounter + +## Bug Bash Goals + +The primary goals of this bug bash are to: + +1. **Exercise real-world scenarios** - Run through common developer workflows +2. **Validate cross-platform compatibility** - Ensure the server works reliably on Windows, macOS, and Linux +3. **Verify installation experience** - Test installation across VS Code, Visual Studio, and IntelliJ IDEA +4. **Assess performance** - Monitor memory consumption and CPU usage under typical workloads +5. **Validate authentication** - Ensure auth works consistently across all platforms +6. **Test server modes** - Verify single, namespace, and all modes work as expected +7. **Validate feature flags** - Test enabling/disabling server features + +## What to Test + +We encourage you to test the following areas: + +### Platform Testing +- [ ] **Windows** - Test on Windows 11 +- [ ] **macOS** - Test on macOS (Intel and Apple Silicon) +- [ ] **Linux** - Test on Ubuntu, Fedora, or other distributions + +### IDE Installation +- [ ] **VS Code** - Stable and Insiders versions +- [ ] **Visual Studio 2022** - Community, Professional, or Enterprise +- [ ] **IntelliJ IDEA** - Ultimate or Community editions (2025.2+) +- [ ] **Claude Desktop** - macOS and Windows +- [ ] **Cursor** - AI-first code editor +- [ ] **Windsurf** - Codeium Cascade editor +- [ ] **Amazon Q Developer** - AWS IDE integration +- [ ] **Claude Code** - Web-based Claude interface + +### Performance Monitoring + +**How to Monitor Performance:** + +**Windows:** +- Open Task Manager (Ctrl+Shift+Esc) +- Find `azmcp.exe` process +- Monitor Memory and CPU columns during operations + +**macOS:** +- Open Activity Monitor (Applications → Utilities → Activity Monitor) +- Search for `azmcp` process +- Monitor CPU % and Memory columns + +**Linux:** +- Use `htop` or `top` command +- Filter for `azmcp` process +- Monitor %CPU and RES (memory) columns + +**What to Test:** +- [ ] Monitor **memory consumption** during typical operations +- [ ] Monitor **CPU usage** during command execution +- [ ] Test with **multiple concurrent operations** +- [ ] Observe behavior during **long-running sessions** (2+ hours): + - Memory usage trends (stable, growing, or leaking) + - Server responsiveness (does it slow down over time?) + - Error frequency (do errors increase with time?) +- [ ] Record baseline memory usage at startup +- [ ] Check for memory leaks after extended use + +### Authentication Testing +- [ ] Test **Azure CLI authentication** (`az login`) +- [ ] Test **Azure PowerShell authentication** (`Connect-AzAccount`) +- [ ] Test **Interactive browser authentication** - Set `AZURE_MCP_ONLY_USE_BROKER_CREDENTIAL=true` and sign in through the broker/browser when innvoking tools via Azure MCP Server. See [Authentication Guide](https://github.com/microsoft/mcp/blob/main/docs/Authentication.md#authentication-fundamentals) for details. +- [ ] Test authentication across **multiple tenants** - Switch between different Azure AD tenants + +### Server Mode Testing + +See [Server Mode Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/installation-testing.md#server-mode-testing) for detailed instructions. + +- [ ] **Namespace mode** (default) - Tools grouped by Azure service (~40-50 tools) +- [ ] **All mode** - All tools exposed individually (100+ tools) +- [ ] **Single mode** - Single unified tool with internal routing +- [ ] **Read-only mode** - Blocks all write/destructive operations +- [ ] **Namespace filtering** - Expose specific services only (e.g., storage, keyvault) + +### Feature Flag Testing +- [ ] Enable/disable server +- [ ] Test read-only mode +- [ ] Test with different namespace configurations +- [ ] Test tool filtering + +## How to Report Issues + +When you find a bug or issue, please report it on GitHub: + +### Report Issues Here: [https://github.com/microsoft/mcp/issues](https://github.com/microsoft/mcp/issues) + +**Steps to Report:** + +1. Go to the [issues page](https://github.com/microsoft/mcp/issues) +2. Click **"New Issue"** +3. Select **"Azure MCP - Bug Bash Report"** template +4. Fill in the following information: + +**Required Information:** +- **Platform**: Windows/macOS/Linux (include version) +- **IDE/Client**: VS Code/Visual Studio/IntelliJ/Claude Desktop/Cursor/Windsurf/Amazon Q/Claude Code (include version) +- **Azure MCP Server Version**: Found in extension details or `azmcp --version` +- **Node.js Version** (if using npm): Run `node --version` +- **Description**: Clear description of the issue +- **Steps to Reproduce**: Detailed steps to reproduce the problem +- **Expected Behavior**: What you expected to happen +- **Actual Behavior**: What actually happened +- **Logs**: Include relevant error messages or logs (see [Troubleshooting Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/TROUBLESHOOTING.md#logging-and-diagnostics)) +- **Screenshots**: If applicable, include screenshots + +## Testing Scenarios + +We've prepared detailed testing guides for common scenarios: + +### Scenario Guides + +1. **[Installation Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/installation-testing.md)** - Test installation across different platforms and IDEs +2. **[Infrastructure as Code](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/infra-as-code.md)** - Generate and deploy Azure infrastructure +3. **[PaaS Services](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/paas-services.md)** - Work with App Service, Container Apps, and Functions +4. **[Storage Operations](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/storage-operations.md)** - Test blob storage and file operations +5. **[Database Operations](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/database-operations.md)** - Work with Cosmos DB, PostgreSQL, and Azure SQL +6. **[Deployment Scenarios](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/deployment.md)** - Deploy resources and applications +7. **[Full Stack Applications](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/full-stack-apps.md)** - Build complete apps with database backends +8. **[Agent Building](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/agent-building.md)** - Create and deploy Azure Foundry agents + +### Quick Start Scenarios + +If you're short on time, try these quick scenarios: + +**5 minutes: Install and Verify** +- Install Azure MCP extension/server in your IDE +- Verify tools are loaded +- Try these prompts: + - `"What Azure MCP tools are available?"` + - `"Show me my subscriptions"` + - `"List all resource groups in my subscription"` + +**10 minutes: Resource Discovery** +- List your Azure resources (subscriptions, resource groups, storage accounts) +- Try these prompts: + - `"List all storage accounts in my subscription"` + - `"Show me my Key Vaults"` + - `"List all App Services in resource group "` + +**15 minutes: Database Inspection** +- Inspect existing databases and query data +- Try these prompts: + - `"List all cosmosdb accounts in my subscription"` + - `"Show me databases in cosmosdb account "` + - `"List all containers in database in cosmosdb account "` + +**30 minutes: End-to-End Scenario** +- Follow one of the [detailed scenario guides](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios) like Storage Operations or Database Operations + +## Resources + +### Bug Bash Documentation +- [Installation Testing Guide](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/installation-testing.md) - Test installation across platforms and IDEs +- [Testing Scenarios](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios) - Detailed end-to-end testing scenarios + +### Azure MCP Server Documentation +- [Azure MCP Server Documentation](https://learn.microsoft.com/azure/developer/azure-mcp-server/) +- [Installation Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/README.md#installation) +- [Troubleshooting Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/TROUBLESHOOTING.md) +- [Authentication Guide](https://github.com/microsoft/mcp/blob/main/docs/Authentication.md) +- [Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) + +### Test Prompts +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) - Sample prompts for testing + +### Support +- [GitHub Issues](https://github.com/microsoft/mcp/issues) - Report bugs and issues \ No newline at end of file diff --git a/docs/bug-bash/installation-testing.md b/docs/bug-bash/installation-testing.md new file mode 100644 index 0000000000..d445e1662d --- /dev/null +++ b/docs/bug-bash/installation-testing.md @@ -0,0 +1,728 @@ +# Installation Testing Guide + +This guide covers testing Azure MCP Server installation across different platforms and IDEs. + +## Table of Contents + +- [Prerequisites](#prerequisites) +- [Platform-Specific Testing](#platform-specific-testing) +- [IDE Installation Testing](#ide-installation-testing) +- [Verification Steps](#verification-steps) + +## Prerequisites + +Before testing installation, ensure you have: + +- [ ] **Azure Subscription** - Access to an Azure subscription + +## Platform-Specific Testing + +### Windows Testing + +#### Installation Steps + +1. **Install Prerequisites** + ```powershell + # Check if Node.js is installed + node --version + + # If not installed, download from https://nodejs.org/ + ``` + +3. **Install via NPM** + ```powershell + npm install -g @azure/mcp@latest + ``` + +4. **Install via NuGet** + ```powershell + dotnet tool install Azure.Mcp + ``` + +#### Verification +```powershell +# Check Azure MCP version +azmcp --version + +# Verify Azure CLI +az --version + +# Test basic command +azmcp server start --help +``` + +#### Things to Test +- [ ] Installation completes without errors +- [ ] Binary is accessible from command line +- [ ] Extension loads in VS Code +- [ ] Server starts successfully +- [ ] Memory usage after installation: _____ MB +- [ ] Installation time: _____ minutes + +### macOS Testing + +#### Installation Steps + +1. **Install Prerequisites** + ```bash + # Check if Node.js is installed + node --version + + # Install via Homebrew if needed + brew install node + ``` + +2. **Install via NPM** + ```bash + npm install -g @azure/mcp@latest + ``` + +3. **Install via .NET Tool** + ```bash + dotnet tool install Azure.Mcp + ``` + +#### Verification +```bash +# Check Azure MCP version +azmcp --version + +# Verify Azure CLI +az --version + +# Test basic command +azmcp server start --help +``` + +#### Platform-Specific Checks +- [ ] **Intel Macs**: Installation works on x64 architecture +- [ ] **Apple Silicon**: Installation works on ARM64 architecture + +#### Things to Test +- [ ] Installation completes without errors +- [ ] Binary has correct permissions (executable) +- [ ] Extension loads in VS Code +- [ ] Server starts successfully +- [ ] No "unidentified developer" warnings +- [ ] Memory usage after installation: _____ MB +- [ ] Installation time: _____ minutes + +--- + +### Linux Testing + +#### Installation Steps + +1. **Install Prerequisites** + ```bash + # Check if Node.js is installed + node --version + + # Ubuntu/Debian + sudo apt update + sudo apt install nodejs npm + + # Fedora + sudo dnf install nodejs npm + ``` + +2. **Install via NPM** (Alternative) + ```bash + npm install -g @azure/mcp@latest + ``` + +3. **Install via .NET Tool** (Alternative) + ```bash + dotnet tool install Azure.Mcp + ``` + +#### Verification +```bash +# Check Azure MCP version +azmcp --version + +# Verify Azure CLI +az --version + +# Test basic command +azmcp server start --help +``` + +#### Things to Test +- [ ] Installation completes without errors +- [ ] Binary has correct permissions +- [ ] Extension loads in VS Code +- [ ] Server starts successfully +- [ ] No dependency conflicts +- [ ] Memory usage after installation: _____ MB +- [ ] Installation time: _____ minutes + +--- + +## IDE Installation Testing + +### VS Code + +#### Versions to Test +- [ ] **VS Code Stable** (latest) +- [ ] **VS Code Insiders** (latest) + +#### Installation Testing + +1. **Install Extension** + - Method 1: Via Extension Marketplace + - Method 2: Via [Installation Link](https://vscode.dev/redirect?url=vscode:extension/ms-azuretools.vscode-azure-mcp-server) + +2. **Verify Installation** + ``` + 1. Open Command Palette (Ctrl+Shift+P / Cmd+Shift+P) + 2. Run "MCP: List Servers" + 3. Verify "Azure MCP Server" appears in the list + 4. Click "Start Server" + 5. Check Output window for startup logs + ``` + +#### Things to Test +- [ ] Extension installs without errors +- [ ] Extension appears in Extensions list +- [ ] Server starts successfully +- [ ] Tools appear in GitHub Copilot Chat +- [ ] Tool count matches expectations +- [ ] Configuration changes are applied +- [ ] Server restarts successfully +- [ ] Logs are visible in Output window + +#### Performance Checks +- [ ] Extension activation time: _____ ms +- [ ] Memory usage (extension): _____ MB +- [ ] Memory usage (server): _____ MB +- [ ] CPU usage during idle: _____ % + +--- + +### Visual Studio 2022 + +#### Versions to Test +- [ ] **Visual Studio 2022 Community** +- [ ] **Visual Studio 2022 Professional** +- [ ] **Visual Studio 2022 Enterprise** + +#### Installation Testing + +1. **Install GitHub Copilot for Azure Extension** + ``` + 1. Open Visual Studio 2022 + 2. Go to Extensions → Manage Extensions... + 3. Switch to the Browse tab in Extension Manager + 4. Search for "GitHub Copilot for Azure" + 5. Click Install + 6. Restart Visual Studio when prompted + ``` + +2. **Verify Installation** + ``` + 1. Open a solution or project + 2. Open GitHub Copilot Chat + 3. Select Agent Mode + 4. Click the tools icon to view available tools + 5. Search for "Azure MCP Server" to filter results + 6. Verify Azure MCP tools appear in the list + ``` + +#### Things to Test +- [ ] GitHub Copilot for Azure extension installs successfully +- [ ] Extension appears in Extensions list after restart +- [ ] GitHub Copilot Chat opens correctly +- [ ] Can switch to Agent Mode +- [ ] Tools icon shows available MCP tools +- [ ] Can search/filter for "Azure MCP Server" tools +- [ ] Server responds to commands in Agent mode + +#### Performance Checks +- [ ] Extension load time: _____ seconds +- [ ] Memory usage: _____ MB +- [ ] CPU usage during idle: _____ % + +--- + +### IntelliJ IDEA + +#### Versions to Test +- [ ] **IntelliJ IDEA Ultimate** (2024.3+) +- [ ] **IntelliJ IDEA Community** (2024.3+) + +#### Installation Testing + +1. **Install GitHub Copilot Plugin** + ``` + 1. Open IntelliJ IDEA + 2. Go to Settings/Preferences > Plugins + 3. Search for "GitHub Copilot" + 4. Install GitHub Copilot plugin + 5. Restart IDE + ``` + +2. **Configure Azure MCP Server** + ``` + 1. Go to File → Settings (or IntelliJ IDEA → Preferences on macOS) + 2. Navigate to Tools → AI Assistant → Model Context Protocol (MCP) + 3. Click the + icon to add a new MCP server + 4. In the "New MCP Server" dialog, enter: + - Name: Azure MCP Server + - Command: npx (or azmcp if installed via dotnet tool) + - Arguments: -y @azure/mcp@latest server start + 5. Click OK to save + 6. Restart AI Assistant if prompted + ``` + +3. **Verify Installation** + ``` + 1. Open GitHub Copilot Chat + 2. Verify Azure MCP tools are available + 3. Test a simple command (e.g., "List my Azure subscriptions") + ``` + +#### Things to Test +- [ ] GitHub Copilot plugin installs without errors +- [ ] Plugin appears in Plugins list +- [ ] MCP configuration dialog is accessible +- [ ] Can add Azure MCP Server configuration +- [ ] Server configuration saves successfully +- [ ] Azure MCP integrates with Copilot Chat +- [ ] Tools are accessible in chat +- [ ] Server responds to commands + +#### Performance Checks +- [ ] Plugin load time: _____ seconds +- [ ] Memory usage: _____ MB +- [ ] CPU usage during idle: _____ % + + +## UI-Based Mode Toggling Testing (Distribution-Specific) + +Different distribution mechanisms provide different ways to configure server modes. Test the UI controls for each distribution method. + +### Distribution Method 1: VS Code Extension + +**Objective**: Test VS Code Settings UI for mode toggling + +#### Available Settings: + +1. **Azure Mcp: Enabled Services** + - Location: Settings → Extensions → Azure MCP + - Purpose: Filter which Azure service namespaces are exposed + - Type: Multi-select list + - Options: storage, keyvault, cosmos, sql, etc. + +2. **Azure Mcp: Read Only** + - Location: Settings → Extensions → Azure MCP + - Purpose: Enable read-only mode (blocks write operations) + - Type: Checkbox + - Default: Unchecked (read-write mode) + +3. **Azure Mcp: Server Mode** + - Location: Settings → Extensions → Azure MCP + - Purpose: Choose tool exposure strategy + - Type: Dropdown + - Options: `single`, `namespace` (default), `all` + +#### Test 1: Change Server Mode via UI + +**Steps:** +1. Open VS Code Settings (Ctrl+, or Cmd+,) +2. Search for "Azure MCP" +3. Locate "Azure Mcp: Server Mode" dropdown +4. Select "namespace" (default) +5. Note current tool count in Copilot Chat +6. Change to "all" +7. Restart server: Command Palette → "MCP: List Servers" → Azure MCP → Start/Restart +8. Check tool count again + +**Verify:** +- [ ] Setting UI is accessible and clear +- [ ] Dropdown shows all three options: single, namespace, all +- [ ] Default is "namespace" +- [ ] Changing mode requires server restart (prompt shown) +- [ ] Tool count changes after restart: + - namespace: ~40-50 tools + - all: 100+ tools + - single: 1 tool + +#### Test 2: Toggle Read-Only Mode via UI + +**Steps:** +1. Open VS Code Settings +2. Search for "Azure MCP" +3. Locate "Azure Mcp: Read Only" checkbox +4. Check the box to enable read-only mode +5. Restart server +6. Test read operation: "List my storage accounts" +7. Test write operation: "Create a storage account" +8. Uncheck the box to disable read-only mode +9. Restart server +10. Test write operation again + +**Verify:** +- [ ] Checkbox is visible and labeled clearly +- [ ] Default state is unchecked (read-write) +- [ ] Enabling requires server restart +- [ ] Read operations work in both modes +- [ ] Write operations blocked when enabled +- [ ] Write operations allowed when disabled +- [ ] Clear error messages when operations are blocked + +#### Test 3: Filter Enabled Services via UI + +**Steps:** +1. Open VS Code Settings +2. Search for "Azure MCP" +3. Locate "Azure Mcp: Enabled Services" setting +4. Click "Add Item" button +5. Add "storage" to the list +6. Add "keyvault" to the list +7. Restart server +8. Check available tools in Copilot Chat + +**Verify:** +- [ ] Setting shows "Add Item" button +- [ ] Can add multiple service namespaces +- [ ] Can remove items from list +- [ ] Empty list = all services enabled +- [ ] Non-empty list = only specified services enabled +- [ ] Tool count reflects filtered services +- [ ] Test prompt for allowed service: "List storage accounts" +- [ ] Test prompt for filtered service: "List Cosmos DB accounts" + +#### Test 4: Combine Multiple Settings + +**Steps:** +1. Set "Enabled Services" to: storage, keyvault +2. Set "Server Mode" to: namespace +3. Enable "Read Only" checkbox +4. Restart server +5. Test various operations + +**Verify:** +- [ ] All three settings applied correctly +- [ ] Only Storage and Key Vault namespaces exposed (~2 tools) +- [ ] Read operations work for allowed services +- [ ] Write operations blocked by read-only mode +- [ ] Filtered services not accessible + +#### Test 5: Settings Persistence + +**Steps:** +1. Configure settings: mode=namespace, readOnly=true, services=[storage] +2. Restart VS Code completely +3. Check if settings persisted +4. Open Settings UI and verify values +5. Start MCP server +6. Verify configuration is applied + +**Verify:** +- [ ] Settings persist across VS Code restarts +- [ ] Settings shown correctly in UI after restart +- [ ] Server starts with saved configuration +- [ ] No need to reconfigure + +--- + +### Distribution Method 2: NPM Package + +**Objective**: Test CLI flag-based mode configuration with NPM-installed package + +**Installation:** +```bash +npm install -g @azure/mcp@latest +``` + +#### Test 1: Server Mode via CLI Flags + +**Test namespace mode:** +```bash +azmcp server start --mode namespace +``` + +**Verify:** +- [ ] Server starts successfully +- [ ] ~40-50 namespace-level tools exposed +- [ ] Tools grouped by service + +**Test all mode:** +```bash +azmcp server start --mode all +``` + +**Verify:** +- [ ] Server starts successfully +- [ ] 100+ individual tools exposed +- [ ] Each operation has dedicated tool + +**Test single mode:** +```bash +azmcp server start --mode single +``` + +**Verify:** +- [ ] Server starts successfully +- [ ] Exactly 1 tool exposed +- [ ] Internal routing functional + +#### Test 2: Read-Only Mode via CLI Flag + +**Enable read-only:** +```bash +azmcp server start --read-only +``` + +**Verify:** +- [ ] Server starts in read-only mode +- [ ] Read operations allowed +- [ ] Write operations blocked + +**Combine with mode:** +```bash +azmcp server start --mode namespace --read-only +``` + +**Verify:** +- [ ] Both flags applied correctly +- [ ] Namespace mode + read-only restrictions + +#### Test 3: Namespace Filtering via CLI Flags + +**Filter to specific services:** +```bash +azmcp server start --namespace storage --namespace keyvault +``` + +**Verify:** +- [ ] Only specified namespaces exposed +- [ ] Other services filtered out +- [ ] Can specify multiple --namespace flags + +#### Test 4: Tool Filtering via CLI Flags + +**Filter to specific tools:** +```bash +azmcp server start --tool azmcp_subscription_list --tool azmcp_group_list +``` + +**Verify:** +- [ ] Only specified tools exposed +- [ ] Automatically switches to all mode +- [ ] Can specify multiple --tool flags + +--- + +### Distribution Method 3: .NET Tool (DNX) + +**Objective**: Test CLI flag-based configuration with .NET tool installation + +**Installation:** +```bash +dotnet tool install --global Azure.Mcp +``` + +#### Test 1: Server Mode via CLI Flags + +**Test namespace mode:** +```bash +azmcp server start --mode namespace +``` + +**Verify:** +- [ ] .NET tool version works identically to NPM version +- [ ] All mode flags functional +- [ ] Tool exposure matches expectations + +#### Test 2: Configuration File Support + +**Create config file** (`azmcp.config.json`): +```json +{ + "serverMode": "namespace", + "readOnly": true, + "enabledServices": ["storage", "keyvault"] +} +``` + +**Start with config:** +```bash +azmcp server start --config azmcp.config.json +``` + +**Verify:** +- [ ] Config file loaded successfully +- [ ] Settings from file applied +- [ ] CLI flags override config file if specified + +--- + +### Distribution Method 4: IntelliJ IDEA Plugin + +**Objective**: Test MCP Server configuration dialog for mode configuration + +**Installation:** +1. IntelliJ IDEA → Settings/Preferences → Plugins +2. Search "Model Context Protocol (MCP)" +3. Install plugin +4. Restart IDE + +#### Configuration Dialog Fields: + +Based on IntelliJ's MCP configuration dialog: +- **Name**: Display name for the MCP server (e.g., "Azure MCP Server") +- **Command**: Executable command (e.g., `npx` or `azmcp`) +- **Arguments**: Command-line arguments including mode flags +- **Environment variables**: Key-value pairs for configuration +- **Working directory**: Optional working directory path + +#### Test 1: Configure via MCP Server Dialog + +**Steps:** +1. Navigate to File → Settings +2. Go to Tools → AI Assistant → Model Context Protocol (MCP) +3. Click the `+` icon to open "New MCP Server" dialog +4. Enter the following values: + - **Name**: `Azure MCP Server` + - **Command**: `npx` + - **Arguments**: `-y @azure/mcp@latest server start` +5. Leave other fields blank for default configuration +6. Click OK to save +7. Restart AI Assistant or IDE if prompted + +**Verify:** +- [ ] Dialog opens correctly with all fields +- [ ] Can enter server name +- [ ] Command and arguments fields accept input +- [ ] Configuration saves successfully +- [ ] Server appears in MCP server list +- [ ] Server starts when AI Assistant is opened + +#### Test 2: Configure Namespace Mode via Arguments + +**Steps:** +1. Open MCP Server configuration dialog +2. Edit existing "Azure MCP Server" entry +3. Update **Arguments** field to: + ``` + -y @azure/mcp@latest server start --mode namespace + ``` +4. Click OK and restart +5. Test tool availability + +**Verify:** +- [ ] Arguments field accepts mode flag +- [ ] Server restarts with new configuration +- [ ] Namespace mode applied (~40-50 tools) +- [ ] Tools grouped by service namespace + +#### Test 3: Configure Read-Only Mode via Arguments + +**Steps:** +1. Edit "Azure MCP Server" configuration +2. Update **Arguments** field to: + ``` + -y @azure/mcp@latest server start --mode namespace --read-only + ``` +3. Click OK and restart +4. Test read operation: "List my storage accounts" +5. Test write operation: "Create a storage account" + +**Verify:** +- [ ] Read-only flag accepted in arguments +- [ ] Read operations work +- [ ] Write operations blocked +- [ ] Error messages displayed + +#### Test 4: Configure Service Filtering via Arguments + +**Steps:** +1. Edit "Azure MCP Server" configuration +2. Update **Arguments** field to: + ``` + -y @azure/mcp@latest server start --namespace storage --namespace keyvault + ``` +3. Click OK and restart +4. Test allowed service: "List storage accounts" +5. Test filtered service: "List Cosmos DB accounts" + +**Verify:** +- [ ] Multiple namespace flags work +- [ ] Only specified services exposed +- [ ] Allowed services functional +- [ ] Filtered services not accessible + +#### Test 5: Configuration Persistence + +**Steps:** +1. Configure server with specific mode and flags +2. Close IntelliJ completely +3. Reopen IntelliJ +4. Navigate to MCP settings +5. Verify configuration still present +6. Start AI Assistant and test + +**Verify:** +- [ ] Configuration persists across IDE restarts +- [ ] No need to reconfigure +- [ ] Server starts with saved settings + +//TODO: Add VS, CLaude, Docker instructions. + +## What to Report for UI Issues + +When logging UI configuration issues, include: +- [ ] Distribution method (VS Code ext, NPM, .NET tool, etc.) +- [ ] IDE/Platform version +- [ ] Screenshot of settings UI +- [ ] Expected vs actual behavior +- [ ] Steps to reproduce +- [ ] Settings file content (if applicable) +- [ ] Server startup logs +- [ ] Whether restart was performed + + +## Verification Steps + +After installation on any platform/IDE, verify: + +### 1. Server Status +```bash +# Check if server is running +# In VS Code: Check Output > MCP: Azure MCP Server +# Look for: "Server started successfully" +``` + +### 2. Tool Discovery +```bash +# In GitHub Copilot Chat (Agent mode): +# Ask: "What Azure MCP tools are available?" +# Verify: Tools are listed +``` + +### 3. Authentication +```bash +# Login to Azure +az login + +# Test authentication +# Ask Copilot: "List my Azure subscriptions" +``` + +### 4. Basic Functionality +```bash +# Test a simple command +# Ask Copilot: "List my Azure resource groups" +# Verify: Resource groups are returned +``` + +## Related Resources + +- [Main Bug Bash Guide](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/README.md) +- [Troubleshooting Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/TROUBLESHOOTING.md) +- [Installation Guide](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/README.md#installation) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +**Next Steps**: After completing installation testing, proceed to [Scenario Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios). diff --git a/docs/bug-bash/scenarios/agent-building.md b/docs/bug-bash/scenarios/agent-building.md new file mode 100644 index 0000000000..b426c2bcb8 --- /dev/null +++ b/docs/bug-bash/scenarios/agent-building.md @@ -0,0 +1,222 @@ +# AI Agent Building Testing Scenario + +> **MCP Tool Support Notice** +> Azure MCP Server provides **AI Foundry resource inspection, model listing, and agent interaction** capabilities. Resource creation and deployment require Azure CLI or Portal. This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. + +## Objectives + +- Test Azure AI Foundry resource discovery and inspection +- Test AI model listing and deployment management +- Test agent creation and interaction workflows +- Validate agent querying and evaluation capabilities + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] GitHub Copilot with Agent mode enabled + +--- + +## Scenario 1: Azure AI Foundry Resource Discovery & Model Management + +**Objective**: Complete workflow for discovering AI Foundry resources and managing model deployments + +### Step 1: Setup Resources (External - Not MCP) + +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. + +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-aifoundry-rg' in eastus, then create an Azure AI Services account with SKU S0, and deploy GPT-4o model with deployment name 'gpt-4o-deployment' +``` + +**Option B: Run Azure CLI Commands Manually**: +```bash +# Create resource group +az group create --name bugbash-aifoundry-rg --location eastus + +# Create Azure AI Foundry resource (AI Services account) +az cognitiveservices account create \ + --name bugbash-ai-foundry-$RANDOM \ + --resource-group bugbash-aifoundry-rg \ + --location eastus \ + --kind AIServices \ + --sku S0 + +# Deploy a model (GPT-4o) +az cognitiveservices account deployment create \ + --name \ + --resource-group bugbash-aifoundry-rg \ + --deployment-name gpt-4o-deployment \ + --model-name gpt-4o \ + --model-version "2024-05-13" \ + --model-format OpenAI \ + --sku-capacity 10 \ + --sku-name Standard +``` + +### Step 2: Discover AI Foundry Resources with Azure MCP Server + +**2.1 Get AI Foundry resource details** (uses `azmcp_foundry_resource_get`): +``` +Show me details for Azure AI Foundry resources in my subscription +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_foundry_resource_get` +- [ ] Your newly created AI Foundry resource appears +- [ ] Resource properties shown (name, location, SKU) + +**2.2 Alternative phrasing**: +``` +List all Azure AI Foundry resources in resource group 'bugbash-aifoundry-rg' +``` + +### Step 3: List Available Models with Azure MCP Server + +**3.1 List all available models** (uses `azmcp_foundry_models_list`): +``` +List all available AI models in Azure AI Foundry +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_foundry_models_list` +- [ ] Model catalog displayed +- [ ] GPT-4o and other models listed +- [ ] Model details shown (publisher, license, capabilities) + +**3.2 Search for specific models**: +``` +Show me all GPT models available in Azure AI Foundry +``` + +**Verify**: +- [ ] Filtered list returned +- [ ] GPT models displayed +- [ ] Search functionality works + +**3.3 Check playground-compatible models**: +``` +Which models can I use in the free playground? +``` + +**Verify**: +- [ ] Playground-compatible models listed +- [ ] Filter applied correctly + +### Step 4: Inspect Model Deployments with Azure MCP Server + +**4.1 List model deployments** (uses `azmcp_foundry_models_deployments_list`): +``` +List all model deployments in my Azure AI Foundry resource +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_foundry_models_deployments_list` +- [ ] Your GPT-4o deployment appears +- [ ] Deployment details shown (name, model, SKU, capacity) + +**4.2 Alternative phrasing**: +``` +Show me all deployed models in my AI Foundry resource +``` + +### Step 5: List OpenAI Models with Azure MCP Server + +**5.1 List OpenAI models** (uses `azmcp_foundry_openai_models-list`): +``` +List all OpenAI models and deployments in my Azure AI resource '' in resource group 'bugbash-aifoundry-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_foundry_openai_models-list` +- [ ] OpenAI models listed +- [ ] Deployment information shown + +### Step 6: Cleanup (External - Not MCP) + +**Option A: Prompt GitHub Copilot Chat**: +``` +Delete the Azure resource group 'bugbash-aifoundry-rg' and all its resources +``` + +**Option B: Run Azure CLI Command Manually**: +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-aifoundry-rg --yes --no-wait +``` + +**Expected Results**: +- AI Foundry resource discovery works +- Model catalog listing successful +- Deployment inspection accurate +- OpenAI model listing functional + + +## Common Issues to Watch For + +| Issue | Description | Resolution | +|-------|-------------|------------| +| **Authentication Failures** | Can't connect to AI Foundry endpoint | Verify `az login` and endpoint URL is correct | +| **Agent Not Found** | Agent ID doesn't exist | List agents first to get valid agent IDs | +| **Token Limits** | Response truncated or incomplete | Model context window exceeded; use shorter prompts | +| **Rate Limiting** | API throttling errors | Reduce request frequency or upgrade SKU | +| **Endpoint Mismatch** | Wrong endpoint URL | Verify endpoint matches your AI Foundry resource | +| **Model Not Deployed** | Deployment not found | Check model deployments are active and provisioned | +| **Evaluation Failures** | Evaluator returns errors | Ensure Azure OpenAI deployment exists for evaluation | +| **Empty Agent List** | No agents returned | Create agents via AI Foundry Portal first | + +## What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output) +- [ ] Expected vs actual results +- [ ] Error messages (if any) +- [ ] Agent ID and endpoint URL (redact sensitive info) +- [ ] Model name and deployment name +- [ ] Screenshots of unexpected behavior + +## Related Resources + +- [Azure AI Foundry Documentation](https://learn.microsoft.com/azure/ai-studio/) +- [Azure OpenAI Service](https://learn.microsoft.com/azure/ai-services/openai/) +- [Azure AI Agents](https://learn.microsoft.com/azure/ai-services/agents/) +- [Model Context Protocol](https://modelcontextprotocol.io/) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +## 💡 Quick Reference: Supported MCP Tools + +### AI Foundry Resources +- `azmcp_foundry_resource_get` - Get AI Foundry resource details +- `azmcp_foundry_models_list` - List available AI models +- `azmcp_foundry_models_deployments_list` - List model deployments + +### OpenAI Integration +- `azmcp_foundry_openai_models-list` - List OpenAI models and deployments +- `azmcp_foundry_openai_chat-completions-create` - Create chat completions +- `azmcp_foundry_openai_create-completion` - Generate text completions +- `azmcp_foundry_openai_embeddings-create` - Generate embeddings + +### AI Agents +- `azmcp_foundry_agents_list` - List AI agents +- `azmcp_foundry_agents_connect` - Query an agent +- `azmcp_foundry_agents_evaluate` - Evaluate agent response +- `azmcp_foundry_agents_query-and-evaluate` - Query and evaluate in one step + +### Knowledge Management +- `azmcp_foundry_knowledge_index_list` - List knowledge indexes +- `azmcp_foundry_knowledge_index_schema` - Get index schema +- `azmcp_foundry_knowledge_source_get` - Get knowledge sources + +### Model Deployment +- `azmcp_foundry_models_deploy` - Deploy AI model (write operation) + +--- + +**Next**: [Database Operations Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/database-operations.md) \ No newline at end of file diff --git a/docs/bug-bash/scenarios/database-operations.md b/docs/bug-bash/scenarios/database-operations.md new file mode 100644 index 0000000000..70148d9b03 --- /dev/null +++ b/docs/bug-bash/scenarios/database-operations.md @@ -0,0 +1,411 @@ + +# Database Operations Testing Scenario + +> **MCP Tool Support Notice** +> Azure MCP Server provides **read-only database inspection and querying** capabilities. Database creation, data insertion, and schema modifications require Azure CLI or Portal. This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. + +## Objectives + +- Test end-to-end Cosmos DB inspection and querying workflows +- Test end-to-end PostgreSQL/MySQL database operations and querying +- Verify MCP tool accuracy for listing, inspecting, and querying databases +- Validate query execution and schema inspection across database types + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] GitHub Copilot with Agent mode enabled + +--- + +## Scenario 1: Cosmos DB End-to-End Workflow + +**Objective**: Complete workflow from resource setup through querying with Azure MCP Server + +### Step 1: Setup Resources (External - Not MCP) + +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. + +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-cosmosdb-rg' in eastus, create a Cosmos DB account, create a database named 'ProductCatalog', and create a container named 'Products' with partition key '/category' +``` + +Then use Azure Portal Data Explorer to insert sample data: +```json +{"id": "1", "name": "Laptop", "category": "Electronics", "price": 1299.99} +{"id": "2", "name": "Mouse", "category": "Electronics", "price": 29.99} +{"id": "3", "name": "Desk", "category": "Furniture", "price": 499.99} +``` + +**Option B: Run Azure CLI Commands Manually**: +```bash +# Create resource group +az group create --name bugbash-cosmosdb-rg --location eastus + +# Create Cosmos DB account (this takes 5-10 minutes) +az cosmosdb create \ + --name bugbash-cosmos-$RANDOM \ + --resource-group bugbash-cosmosdb-rg \ + --locations regionName=eastus + +# Create database +az cosmosdb sql database create \ + --account-name \ + --resource-group bugbash-cosmosdb-rg \ + --name ProductCatalog + +# Create container with partition key +az cosmosdb sql container create \ + --account-name \ + --resource-group bugbash-cosmosdb-rg \ + --database-name ProductCatalog \ + --name Products \ + --partition-key-path "/category" +``` + +**Insert sample data** using Azure Portal Data Explorer: +```json +{"id": "1", "name": "Laptop", "category": "Electronics", "price": 1299.99} +{"id": "2", "name": "Mouse", "category": "Electronics", "price": 29.99} +{"id": "3", "name": "Desk", "category": "Furniture", "price": 499.99} +``` + +### Step 2: Discovery with Azure MCP Server + +**2.1 List all Cosmos DB accounts** (uses `azmcp_cosmos_account_list`): +``` +List all Cosmos DB accounts in my subscription +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_account_list` +- [ ] Your newly created account appears in the list +- [ ] Account properties shown (name, location, capabilities) + +**2.2 Alternative phrasing**: +``` +Show me my Cosmos DB accounts +``` + +### Step 3: Database Inspection with Azure MCP Server + +**3.1 List databases** (uses `azmcp_cosmos_database_list`): +``` +List all databases in Cosmos DB account '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_database_list` +- [ ] 'ProductCatalog' database is listed +- [ ] Database properties displayed + +**3.2 List containers** (uses `azmcp_cosmos_database_container_list`): +``` +List all containers in database 'ProductCatalog' for Cosmos DB account '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_database_container_list` +- [ ] 'Products' container is listed +- [ ] Partition key path '/category' is shown + +### Step 4: Query Data with Azure MCP Server + +**4.1 Query all items** (uses `azmcp_cosmos_database_container_item_query`): +``` +Query all items from container 'Products' in database 'ProductCatalog' for Cosmos DB account '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_database_container_item_query` +- [ ] All 3 sample items returned +- [ ] Item properties correctly displayed + +**4.2 Query with filter**: +``` +Show me items in the Products container where category is 'Electronics' in database 'ProductCatalog' for Cosmos DB account '' +``` + +**Verify**: +- [ ] Query executed successfully +- [ ] Only 2 items returned (Laptop and Mouse) +- [ ] Filter correctly applied + +**4.3 Search query**: +``` +Show me items that contain the word 'Laptop' in container 'Products' in database 'ProductCatalog' for Cosmos DB account '' +``` + +**Verify**: +- [ ] Text search works correctly +- [ ] Laptop item returned +- [ ] Query syntax accepted + +### Step 5: Cleanup (External - Not MCP) + +**Option A: Prompt GitHub Copilot Chat**: +``` +Delete the Azure resource group 'bugbash-cosmosdb-rg' and all its resources +``` + +**Option B: Run Azure CLI Command Manually**: +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-cosmosdb-rg --yes --no-wait +``` + +**Expected Results**: +- All MCP listing tools work correctly +- Database and container inspection successful +- Query operations return accurate results +- Different query patterns supported + +--- + +## Scenario 2: PostgreSQL End-to-End Workflow + +**Objective**: Complete workflow for PostgreSQL server inspection, schema viewing, and querying + +### Step 1: Setup Resources (External - Not MCP) + +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. + +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-postgres-rg' in eastus, create a PostgreSQL flexible server with admin user 'dbadmin', create a database named 'inventory', then help me create a products table with columns: id, name, description, price, stock_quantity, created_at +``` + +Then use Azure Portal Query Editor or GitHub Copilot to insert sample data. + +**Option B: Run Azure CLI Commands Manually**: +```bash +# Create resource group +az group create --name bugbash-postgres-rg --location eastus + +# Create PostgreSQL flexible server (takes 5-10 minutes) +az postgres flexible-server create \ + --name bugbash-postgres-$RANDOM \ + --resource-group bugbash-postgres-rg \ + --location eastus \ + --admin-user dbadmin \ + --admin-password \ + --sku-name Standard_B1ms \ + --tier Burstable \ + --version 14 \ + --storage-size 32 \ + --public-access 0.0.0.0 + +# Create database +az postgres flexible-server db create \ + --resource-group bugbash-postgres-rg \ + --server-name \ + --database-name inventory + +# Connect and create table (use Azure Portal Query Editor or psql) +# CREATE TABLE products ( +# id SERIAL PRIMARY KEY, +# name VARCHAR(100) NOT NULL, +# description TEXT, +# price DECIMAL(10,2), +# stock_quantity INTEGER DEFAULT 0, +# created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP +# ); + +# Insert sample data +# INSERT INTO products (name, description, price, stock_quantity) VALUES +# ('Laptop', 'High-performance laptop', 1299.99, 25), +# ('Mouse', 'Wireless mouse', 29.99, 100), +# ('Keyboard', 'Mechanical keyboard', 89.99, 50); +``` + +### Step 2: Server Discovery with Azure MCP Server + +**2.1 List all PostgreSQL servers** (uses `azmcp_postgres_server_list`): +``` +List all PostgreSQL servers in my subscription +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_server_list` +- [ ] Your newly created server appears +- [ ] Server details shown (name, location, version, SKU) + +**2.2 Get server configuration** (uses `azmcp_postgres_server_config_get`): +``` +Show me the configuration of PostgreSQL server '' in resource group 'bugbash-postgres-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_server_config_get` +- [ ] Configuration parameters displayed +- [ ] Server settings accessible + +### Step 3: Database and Schema Inspection with Azure MCP Server + +**3.1 List databases** (uses `azmcp_postgres_database_list`): +``` +List all databases in PostgreSQL server '' in resource group 'bugbash-postgres-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_database_list` +- [ ] 'inventory' database is listed +- [ ] System databases also shown (postgres, template0, template1) + +**3.2 List tables** (uses `azmcp_postgres_table_list`): +``` +List all tables in PostgreSQL database 'inventory' on server '' in resource group 'bugbash-postgres-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_table_list` +- [ ] 'products' table is listed +- [ ] Table names correctly displayed + +**3.3 Get table schema** (uses `azmcp_postgres_table_schema_get`): +``` +Show me the schema of table 'products' in database 'inventory' on server '' in resource group 'bugbash-postgres-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_table_schema_get` +- [ ] All columns listed (id, name, description, price, stock_quantity, created_at) +- [ ] Data types correctly shown +- [ ] Primary key and constraints visible + +### Step 4: Query Data with Azure MCP Server + +**4.1 Query all data** (uses `azmcp_postgres_database_query`): +``` +Show me all products in the 'products' table in database 'inventory' on server '' in resource group 'bugbash-postgres-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_database_query` +- [ ] All 3 sample records returned +- [ ] Column values correctly displayed + +**4.2 Query with filter**: +``` +Show me all products with price less than 100 in the 'products' table in database 'inventory' on server '' in resource group 'bugbash-postgres-rg' +``` + +**Verify**: +- [ ] Query executed successfully +- [ ] Only 2 items returned (Mouse and Keyboard) +- [ ] WHERE clause correctly applied + +**4.3 Search query**: +``` +Show me all items that contain the word 'laptop' in the 'products' table in PostgreSQL database 'inventory' on server '' in resource group 'bugbash-postgres-rg' +``` + +**Verify**: +- [ ] Text search works correctly +- [ ] Laptop item returned +- [ ] Case-insensitive search works + +**4.4 Check server parameters** (uses `azmcp_postgres_server_param_get`): +``` +Show me the max_connections parameter for PostgreSQL server '' in resource group 'bugbash-postgres-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_postgres_server_param_get` +- [ ] Parameter value displayed +- [ ] Parameter metadata shown + +### Step 5: Cleanup (External - Not MCP) + +**Option A: Prompt GitHub Copilot Chat**: +``` +Delete the Azure resource group 'bugbash-postgres-rg' and all its resources +``` + +**Option B: Run Azure CLI Command Manually**: +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-postgres-rg --yes --no-wait +``` + +**Expected Results**: +- Server listing and discovery works +- Database and table inspection successful +- Schema information accurately retrieved +- Query operations return correct results +- Server configuration accessible + +## Common Issues to Watch For + +| Issue | Description | Resolution | +|-------|-------------|------------| +| **Authentication Failures** | MCP tool can't connect to database | Verify `az login` is active and you have RBAC permissions | +| **Firewall Blocking** | Access denied errors | Add your IP to firewall rules using Azure CLI/Portal | +| **Partition Key Issues** | Cosmos DB query fails | Ensure partition key is included in queries or specify cross-partition | +| **Query Syntax Differences** | SQL queries fail | Use PostgreSQL syntax for Postgres, T-SQL for Azure SQL | +| **Resource Limits** | Timeout or throttling | Check RU limits (Cosmos), connection limits (SQL/PostgreSQL) | +| **Case Sensitivity** | Query returns no results | PostgreSQL is case-sensitive; use proper casing or ILIKE | +| **Missing Resources** | MCP tool shows empty lists | Verify resources exist and are in the correct subscription | + +## What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output) +- [ ] Expected vs actual results +- [ ] Error messages (if any) +- [ ] Database type and version +- [ ] Screenshots of unexpected behavior + +## Related Resources + +- [Azure Cosmos DB Documentation](https://learn.microsoft.com/azure/cosmos-db/) +- [Azure Database for PostgreSQL](https://learn.microsoft.com/azure/postgresql/) +- [Azure Database for MySQL](https://learn.microsoft.com/azure/mysql/) +- [Azure SQL Database](https://learn.microsoft.com/azure/azure-sql/) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +## 💡 Quick Reference: Supported MCP Tools + +### Cosmos DB +- `azmcp_cosmos_account_list` - List accounts +- `azmcp_cosmos_database_list` - List databases +- `azmcp_cosmos_database_container_list` - List containers +- `azmcp_cosmos_database_container_item_query` - Query items + +### PostgreSQL +- `azmcp_postgres_server_list` - List servers +- `azmcp_postgres_server_config_get` - Get configuration +- `azmcp_postgres_server_param_get` - Get specific parameter +- `azmcp_postgres_database_list` - List databases +- `azmcp_postgres_table_list` - List tables +- `azmcp_postgres_table_schema_get` - Get table schema +- `azmcp_postgres_database_query` - Execute SELECT queries + +### MySQL +- `azmcp_mysql_server_list` - List servers +- `azmcp_mysql_server_config_get` - Get configuration +- `azmcp_mysql_server_param_get` - Get specific parameter +- `azmcp_mysql_database_list` - List databases +- `azmcp_mysql_table_list` - List tables +- `azmcp_mysql_table_schema_get` - Get table schema +- `azmcp_mysql_database_query` - Execute SELECT queries + +### Azure SQL +- `azmcp_sql_server_list` - List servers +- `azmcp_sql_server_show` - Get server details +- `azmcp_sql_db_list` - List databases +- `azmcp_sql_db_show` - Get database details +- `azmcp_sql_elastic-pool_list` - List elastic pools +- `azmcp_sql_server_firewall-rule_list` - List firewall rules +- `azmcp_sql_server_entra-admin_list` - List Entra admins + +--- + +**Next**: [Deployment Scenarios Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/deployment.md) diff --git a/docs/bug-bash/scenarios/deployment.md b/docs/bug-bash/scenarios/deployment.md new file mode 100644 index 0000000000..5c525ef898 --- /dev/null +++ b/docs/bug-bash/scenarios/deployment.md @@ -0,0 +1,400 @@ +# 🚀 Deployment Scenarios Testing + +> **⚠️ MCP Tool Support Notice** +> Azure MCP Server provides **deployment planning guidance, CI/CD pipeline recommendations, and application log retrieval** capabilities. Actual resource creation and application deployment require Azure CLI, Portal, or CI/CD tools. This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. + +## 🎯 Objectives + +- Test deployment planning and guidance from Azure MCP Server +- Verify CI/CD pipeline recommendation accuracy +- Test application log retrieval and analysis +- Validate deployment plan generation for various project types + +## ✅ Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Azure Developer CLI installed (`azd version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] GitHub Copilot with Agent mode enabled + +--- + +## 📋 Scenario 1: Application Deployment Planning End-to-End + +**Objective**: Complete workflow for getting deployment guidance and executing deployment + +### Step 1: Setup Sample Application (Local - Not MCP) + +> **External Setup Required**: Create a sample Node.js application locally + +```bash +# Create a sample Node.js application +mkdir bugbash-deploy-app && cd bugbash-deploy-app + +# Initialize Node.js project +npm init -y + +# Install Express +npm install express + +# Create simple server (index.js) +cat > index.js << 'EOF' +const express = require('express'); +const app = express(); +const port = process.env.PORT || 3000; + +app.get('/', (req, res) => { + res.json({ message: 'Hello from Azure!', timestamp: new Date() }); +}); + +app.listen(port, () => { + console.log(`Server running on port ${port}`); +}); +EOF + +# Create package.json start script +npm pkg set scripts.start="node index.js" +``` + +### Step 2: Get Deployment Plan with Azure MCP Server + +**2.1 Request deployment plan** (uses `azmcp_deploy_plan_get`): +``` +I have a Node.js Express application in folder 'bugbash-deploy-app'. Create a deployment plan to deploy this to Azure App Service. +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_plan_get` +- [ ] Deployment plan includes App Service target +- [ ] Runtime configuration identified (Node.js) +- [ ] Provisioning tool recommendation provided (Bicep, Terraform, or azd) +- [ ] Project structure analysis included + +**2.2 Alternative phrasing**: +``` +Show me how to deploy my Node.js app in 'bugbash-deploy-app' to Azure +``` + +### Step 3: Get CI/CD Pipeline Guidance with Azure MCP Server + +**3.1 Get pipeline guidance** (uses `azmcp_deploy_pipeline_guidance_get`): +``` +How do I set up a CI/CD pipeline to automatically deploy my application to Azure App Service? +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_pipeline_guidance_get` +- [ ] GitHub Actions workflow guidance provided +- [ ] Azure DevOps pipeline guidance included +- [ ] Deployment credentials setup explained +- [ ] Best practices mentioned + +**3.2 Request specific pipeline configuration**: +``` +Generate GitHub Actions workflow guidance for deploying my Node.js app to Azure App Service with staging and production environments +``` + +**Verify**: +- [ ] Multi-environment deployment strategy explained +- [ ] Approval workflows mentioned +- [ ] Environment-specific configuration guidance provided + +### Step 4: Deploy Application (Azure CLI/azd - Not MCP) + +> **External Deployment Required**: Use Azure CLI or azd to deploy + +```bash +# Create resource group +az group create --name bugbash-deploy-rg --location eastus + +# Create App Service plan +az appservice plan create \ + --name bugbash-plan \ + --resource-group bugbash-deploy-rg \ + --sku B1 \ + --is-linux + +# Create Web App +az webapp create \ + --name bugbash-app-$RANDOM \ + --resource-group bugbash-deploy-rg \ + --plan bugbash-plan \ + --runtime "NODE:18-lts" + +# Deploy application +cd bugbash-deploy-app +zip -r app.zip . +az webapp deployment source config-zip \ + --resource-group bugbash-deploy-rg \ + --name \ + --src app.zip +``` + +### Step 5: View Application Logs with Azure MCP Server + +**5.1 Get application logs** (uses `azmcp_deploy_app_logs_get`): +``` +Show me the logs for my application in Azure environment 'bugbash-deploy-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_app_logs_get` +- [ ] Application logs retrieved +- [ ] Log timestamps shown +- [ ] Log content is readable + +**5.2 Request recent logs**: +``` +Show me the last 50 log entries for my deployed application +``` + +**Verify**: +- [ ] Limit parameter works +- [ ] Most recent logs displayed first +- [ ] Startup messages visible + +### Step 6: Get Architecture Diagram with Azure MCP Server + +**6.1 Generate architecture diagram** (uses `azmcp_deploy_architecture_diagram_generate`): +``` +Generate a Mermaid architecture diagram for my deployment showing the App Service, application topology, and resource dependencies +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_architecture_diagram_generate` +- [ ] Mermaid diagram generated +- [ ] Components clearly shown +- [ ] Relationships between resources displayed + +### Step 7: Cleanup (External - Not MCP) + +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-deploy-rg --yes --no-wait +``` + +**Expected Results**: +- ✅ Deployment plan guidance accurate and actionable +- ✅ CI/CD pipeline recommendations comprehensive +- ✅ Application logs successfully retrieved +- ✅ Architecture diagram generated correctly + +--- + +## 📋 Scenario 2: Infrastructure-as-Code Deployment Guidance End-to-End + +**Objective**: Complete workflow for getting IaC recommendations and deployment guidance + +### Step 1: Setup Project Structure (Local - Not MCP) + +> **External Setup Required**: Create a sample project with multiple services + +```bash +# Create multi-service project +mkdir bugbash-fullstack && cd bugbash-fullstack +mkdir frontend backend + +# Frontend (React app - simplified) +cat > frontend/package.json << 'EOF' +{ + "name": "frontend", + "version": "1.0.0", + "scripts": { + "build": "echo 'Building React app'" + } +} +EOF + +# Backend (Node.js API) +cat > backend/package.json << 'EOF' +{ + "name": "backend", + "version": "1.0.0", + "main": "server.js", + "scripts": { + "start": "node server.js" + }, + "dependencies": { + "express": "^4.18.0" + } +} +EOF + +cat > backend/server.js << 'EOF' +const express = require('express'); +const app = express(); +app.get('/api/health', (req, res) => res.json({ status: 'healthy' })); +app.listen(process.env.PORT || 8080); +EOF +``` + +### Step 2: Get IaC Generation Rules with Azure MCP Server + +**2.1 Request IaC rules for resources** (uses `azmcp_deploy_iac_rules_get`): +``` +Show me the infrastructure-as-code generation rules for deploying Azure App Service and Azure SQL Database using Bicep +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_iac_rules_get` +- [ ] Bicep rules provided for App Service +- [ ] Bicep rules provided for Azure SQL Database +- [ ] Best practices included +- [ ] Parameter recommendations shown + +**2.2 Get rules for different deployment tools**: +``` +Show me the IaC rules for Azure Storage Account and Cosmos DB using Terraform +``` + +**Verify**: +- [ ] Terraform-specific rules returned +- [ ] Resource type rules accurate +- [ ] Configuration recommendations included + +### Step 3: Get Deployment Plan for Multi-Service App with Azure MCP Server + +**3.1 Request deployment plan** (uses `azmcp_deploy_plan_get`): +``` +I have a full-stack application with a React frontend in 'frontend' folder and Node.js API in 'backend' folder. Create a deployment plan for Azure with Static Web App for frontend and App Service for backend. +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_plan_get` +- [ ] Frontend deployment to Static Web App recommended +- [ ] Backend deployment to App Service recommended +- [ ] Service communication strategy included +- [ ] Infrastructure dependencies identified + +### Step 4: Get Pipeline Guidance for Full-Stack App with Azure MCP Server + +**4.1 Request CI/CD guidance** (uses `azmcp_deploy_pipeline_guidance_get`): +``` +How do I set up a CI/CD pipeline for my full-stack application with separate build and deployment for frontend and backend? +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_pipeline_guidance_get` +- [ ] Multi-stage pipeline guidance provided +- [ ] Frontend build process explained +- [ ] Backend build process explained +- [ ] Deployment orchestration guidance included + +**4.2 Request GitHub-specific guidance**: +``` +Generate GitHub Actions pipeline guidance for my full-stack app with automated testing and deployment to Azure +``` + +**Verify**: +- [ ] GitHub Actions workflow structure explained +- [ ] Testing stage recommendations included +- [ ] Deployment stages for both services covered +- [ ] Environment variables and secrets guidance provided + +### Step 5: Deploy Infrastructure (Azure CLI/Bicep - Not MCP) + +> **External Deployment Required**: Use Azure CLI with Bicep + +```bash +# Create resource group +az group create --name bugbash-fullstack-rg --location eastus + +# Create Static Web App for frontend +az staticwebapp create \ + --name bugbash-frontend-$RANDOM \ + --resource-group bugbash-fullstack-rg \ + --location eastus + +# Create App Service for backend +az appservice plan create \ + --name bugbash-backend-plan \ + --resource-group bugbash-fullstack-rg \ + --sku B1 \ + --is-linux + +az webapp create \ + --name bugbash-backend-$RANDOM \ + --resource-group bugbash-fullstack-rg \ + --plan bugbash-backend-plan \ + --runtime "NODE:18-lts" +``` + +### Step 6: Verify Deployment Guidance with Azure MCP Server + +**6.1 Get architecture diagram** (uses `azmcp_deploy_architecture_diagram_generate`): +``` +Generate a Mermaid architecture diagram showing my full-stack application with Static Web App frontend, App Service backend, and their connectivity +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_architecture_diagram_generate` +- [ ] Both services shown in diagram +- [ ] Communication flow displayed +- [ ] Azure resources clearly labeled + +### Step 7: Cleanup (External - Not MCP) + +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-fullstack-rg --yes --no-wait +``` + +**Expected Results**: +- ✅ IaC rules provided for requested resource types +- ✅ Deployment plan includes multi-service architecture +- ✅ CI/CD guidance covers full-stack complexity +- ✅ Architecture diagrams accurately represent topology + +--- + +## � Common Issues to Watch For + +| Issue | Description | Resolution | +|-------|-------------|------------| +| **Invalid Workspace Path** | Workspace folder not found | Verify the workspace folder path exists and is accessible | +| **Project Not Recognized** | MCP can't identify project type | Ensure package.json or project files exist in the folder | +| **Missing Dependencies** | Required project files missing | Check project structure has all necessary configuration files | +| **IaC Tool Mismatch** | Wrong provisioning tool recommended | Specify preferred tool (Bicep/Terraform) in the prompt | +| **Log Retrieval Fails** | Can't access application logs | Verify azd environment exists and is properly configured | +| **Architecture Diagram Errors** | Diagram generation fails | Ensure application topology is properly defined | +| **Pipeline Guidance Generic** | Recommendations too generic | Provide more context about your project structure and requirements | + +## 📝 What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output) +- [ ] Expected vs actual results +- [ ] Error messages (if any) +- [ ] Project type and structure +- [ ] Workspace folder path +- [ ] Screenshots of unexpected behavior + +## 📚 Related Resources + +- [Azure Deployment Center](https://learn.microsoft.com/azure/app-service/deploy-continuous-deployment) +- [Azure Developer CLI (azd)](https://learn.microsoft.com/azure/developer/azure-developer-cli/) +- [GitHub Actions for Azure](https://learn.microsoft.com/azure/developer/github/github-actions) +- [Azure DevOps Pipelines](https://learn.microsoft.com/azure/devops/pipelines/) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +## 💡 Quick Reference: Supported MCP Tools + +### Deployment Planning +- `azmcp_deploy_plan_get` - Get deployment plan for a project +- `azmcp_deploy_iac_rules_get` - Get IaC generation rules for resource types +- `azmcp_deploy_architecture_diagram_generate` - Generate Mermaid architecture diagram + +### CI/CD Guidance +- `azmcp_deploy_pipeline_guidance_get` - Get CI/CD pipeline setup guidance + +### Application Logs +- `azmcp_deploy_app_logs_get` - Get application service logs for azd environment + +--- + +**Next**: [Full Stack Applications Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/full-stack-apps.md) diff --git a/docs/bug-bash/scenarios/full-stack-apps.md b/docs/bug-bash/scenarios/full-stack-apps.md new file mode 100644 index 0000000000..f6aaa59735 --- /dev/null +++ b/docs/bug-bash/scenarios/full-stack-apps.md @@ -0,0 +1,275 @@ +# Full Stack Application Testing Scenario + +> **MCP Tool Support Notice** +> Azure MCP Server provides **database inspection, querying, and resource discovery** capabilities. Application code generation, resource creation, and deployment require external tools (Azure CLI, GitHub Copilot code generation, deployment tools). This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. + +## Objectives + +- Test MCP database inspection capabilities in full-stack context +- Verify resource discovery for deployed applications +- Test database querying for application backends +- Validate end-to-end workflow combining MCP and external tools + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] Development tools (Node.js or Python) +- [ ] GitHub Copilot with Agent mode enabled + +--- + +## Scenario 1: Node.js App with Cosmos DB Backend End-to-End + +**Objective**: Complete workflow from database setup through querying with a deployed application + +### Step 1: Create Database Resources (External - Not MCP) + +> **External Setup Required**: Use Azure CLI to create Cosmos DB resources + +```bash +# Create resource group +az group create --name bugbash-fullstack-rg --location eastus + +# Create Cosmos DB account +az cosmosdb create \ + --name bugbash-cosmos-$RANDOM \ + --resource-group bugbash-fullstack-rg \ + --locations regionName=eastus + +# Create database +az cosmosdb sql database create \ + --account-name \ + --resource-group bugbash-fullstack-rg \ + --name TasksDB + +# Create container +az cosmosdb sql container create \ + --account-name \ + --resource-group bugbash-fullstack-rg \ + --database-name TasksDB \ + --name Tasks \ + --partition-key-path "/category" \ + --throughput 400 +``` + +**Insert sample data** using Azure Portal Data Explorer: +```json +{"id": "1", "title": "Buy groceries", "category": "personal", "completed": false} +{"id": "2", "title": "Review PR", "category": "work", "completed": false} +{"id": "3", "title": "Call dentist", "category": "personal", "completed": true} +``` + +### Step 2: Discover Database Resources with Azure MCP Server + +**2.1 List Cosmos DB accounts** (uses `azmcp_cosmos_account_list`): +``` +List all Cosmos DB accounts in my subscription +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_account_list` +- [ ] Your Cosmos DB account appears +- [ ] Account properties displayed + +**2.2 List databases** (uses `azmcp_cosmos_database_list`): +``` +List all databases in Cosmos DB account '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_database_list` +- [ ] 'TasksDB' database listed + +**2.3 List containers** (uses `azmcp_cosmos_database_container_list`): +``` +List all containers in database 'TasksDB' for Cosmos DB account '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_database_container_list` +- [ ] 'Tasks' container listed +- [ ] Partition key '/category' shown + +### Step 3: Query Application Data with Azure MCP Server + +**3.1 Query all tasks** (uses `azmcp_cosmos_database_container_item_query`): +``` +Query all items from container 'Tasks' in database 'TasksDB' for Cosmos DB account '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_cosmos_database_container_item_query` +- [ ] All 3 sample tasks returned +- [ ] Task properties visible (id, title, category, completed) + +**3.2 Query completed tasks**: +``` +Show me all completed tasks from container 'Tasks' in database 'TasksDB' for Cosmos DB account '' +``` + +**Verify**: +- [ ] Only completed tasks returned +- [ ] Filter correctly applied + +**3.3 Query by category**: +``` +Show me all work tasks from container 'Tasks' in database 'TasksDB' for Cosmos DB account '' +``` + +**Verify**: +- [ ] Only work category tasks returned +- [ ] Category filter works + +### Step 4: Build and Deploy Application (External Tools - Not MCP) + +> **External Development Required**: Use GitHub Copilot for code generation + +**Generate application code** (use GitHub Copilot Chat in VS Code): +``` +Create a Node.js Express application for a task management system with: +- Express.js web framework +- Cosmos DB integration using @azure/cosmos +- RESTful API endpoints for CRUD operations +- Simple HTML frontend +- Connection to Cosmos DB '' +``` + +**Deploy to Azure** (use Azure CLI): +```bash +# Create App Service +az webapp create \ + --name bugbash-taskapp-$RANDOM \ + --resource-group bugbash-fullstack-rg \ + --plan bugbash-plan \ + --runtime "NODE:18-lts" + +# Configure connection string +az webapp config appsettings set \ + --name \ + --resource-group bugbash-fullstack-rg \ + --settings COSMOS_CONNECTION_STRING="" + +# Deploy code (zip deployment) +cd task-app +zip -r app.zip . +az webapp deployment source config-zip \ + --resource-group bugbash-fullstack-rg \ + --name \ + --src app.zip +``` + +### Step 5: Verify Deployed Application with Azure MCP Server + +**5.1 Get App Service details** (uses `azmcp_appservice_get`): +``` +Show me details for App Service '' in resource group 'bugbash-fullstack-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_appservice_get` +- [ ] App Service properties shown +- [ ] Runtime and configuration visible +- [ ] App URL displayed + +**5.2 Verify data through application**: +- Open the App Service URL in browser +- Test the task management interface +- Create/read/update/delete tasks +- Verify operations work end-to-end + +### Step 6: Query Updated Data with Azure MCP Server + +**6.1 Query database to verify application changes** (uses `azmcp_cosmos_database_container_item_query`): +``` +Query all items from container 'Tasks' in database 'TasksDB' to see changes made through my application +``` + +**Verify**: +- [ ] New tasks created via app are visible +- [ ] Updated tasks show modifications +- [ ] Deleted tasks are gone +- [ ] Database accurately reflects app operations + +### Step 7: Cleanup (External - Not MCP) + +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-fullstack-rg --yes --no-wait +``` + +**Expected Results**: +- Database discovery works correctly +- Application data query succeeds +- App Service inspection functional +- End-to-end workflow complete +- Data changes visible through MCP queries + +## Common Issues to Watch For + +| Issue | Description | Resolution | +|-------|-------------|------------| +| **Database Connection Failures** | Can't connect to database | Verify firewall rules allow your IP and Azure services | +| **Authentication Errors** | Credential issues | Check connection string format and credentials | +| **Query Syntax Errors** | SQL queries fail | Use correct syntax for database type (T-SQL vs PostgreSQL) | +| **App Service Configuration** | App can't connect to database | Verify connection string in App Service settings | +| **CORS Errors** | Frontend can't reach API | Configure CORS in App Service or API code | +| **Missing Resources** | MCP can't find resources | Ensure resources exist in specified subscription/resource group | +| **Partition Key Issues** | Cosmos DB queries fail | Include partition key or enable cross-partition queries | + +## What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output) +- [ ] Expected vs actual results +- [ ] Error messages (if any) +- [ ] Database type and resource names +- [ ] Query that was attempted +- [ ] Screenshots of unexpected behavior + +## Related Resources + +- [Azure App Service Documentation](https://learn.microsoft.com/azure/app-service/) +- [Azure Cosmos DB Documentation](https://learn.microsoft.com/azure/cosmos-db/) +- [Azure PostgreSQL Documentation](https://learn.microsoft.com/azure/postgresql/) +- [Azure SQL Documentation](https://learn.microsoft.com/azure/azure-sql/) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +## Quick Reference: Supported MCP Tools + +### Cosmos DB +- `azmcp_cosmos_account_list` - List Cosmos DB accounts +- `azmcp_cosmos_database_list` - List databases +- `azmcp_cosmos_database_container_list` - List containers +- `azmcp_cosmos_database_container_item_query` - Query container items + +### PostgreSQL +- `azmcp_postgres_server_list` - List PostgreSQL servers +- `azmcp_postgres_database_list` - List databases +- `azmcp_postgres_table_list` - List tables +- `azmcp_postgres_table_schema_get` - Get table schema +- `azmcp_postgres_database_query` - Execute SELECT queries + +### MySQL +- `azmcp_mysql_server_list` - List MySQL servers +- `azmcp_mysql_database_list` - List databases +- `azmcp_mysql_table_list` - List tables +- `azmcp_mysql_table_schema_get` - Get table schema +- `azmcp_mysql_database_query` - Execute SELECT queries + +### Azure SQL +- `azmcp_sql_server_list` - List SQL servers +- `azmcp_sql_db_list` - List databases +- `azmcp_sql_db_show` - Get database details + +### App Service +- `azmcp_appservice_get` - Get App Service details + +--- + +**Next**: [Infrastructure as Code Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/infra-as-code.md) \ No newline at end of file diff --git a/docs/bug-bash/scenarios/infra-as-code.md b/docs/bug-bash/scenarios/infra-as-code.md new file mode 100644 index 0000000000..b5f5da936e --- /dev/null +++ b/docs/bug-bash/scenarios/infra-as-code.md @@ -0,0 +1,213 @@ +# Infrastructure as Code Testing Scenario + +> **MCP Tool Support Notice** +> Azure MCP Server provides **Bicep schema retrieval and IaC generation rules** capabilities. Actual Bicep template generation and infrastructure deployment require external tools (GitHub Copilot for code generation, Azure CLI/Bicep for deployment). This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. + +## Objectives + +- Test Bicep schema retrieval for Azure resources +- Verify IaC generation rules and recommendations +- Test Terraform best practices guidance +- Validate end-to-end IaC workflow with MCP assistance + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Bicep CLI installed (`az bicep version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] GitHub Copilot with Agent mode enabled + +--- + +## Scenario 1: Bicep Schema and Template Generation End-to-End + +**Objective**: Complete workflow using Bicep schema assistance and deployment + +### Step 1: Get Bicep Schema with Azure MCP Server + +**1.1 Get storage account schema** (uses `azmcp_bicepschema_get`): +``` +Show me the Bicep schema for Azure Storage Account resource type +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_bicepschema_get` +- [ ] Schema definition returned +- [ ] Resource properties documented +- [ ] Property types shown +- [ ] Required vs optional properties identified + +**1.2 Get App Service schema**: +``` +Show me the Bicep schema for Microsoft.Web/sites resource type +``` + +**Verify**: +- [ ] App Service schema retrieved +- [ ] Configuration properties listed +- [ ] Schema is accurate and complete + +**1.3 Get SQL Database schema**: +``` +Get the Bicep schema for Azure SQL Database +``` + +**Verify**: +- [ ] SQL Database schema returned +- [ ] All properties documented + +### Step 2: Get IaC Generation Rules with Azure MCP Server + +**2.1 Get rules for Storage Account** (uses `azmcp_deploy_iac_rules_get`): +``` +Show me the IaC generation rules for Azure Storage Account using Bicep +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_deploy_iac_rules_get` +- [ ] Best practices provided +- [ ] Recommended parameters shown +- [ ] Configuration guidance included +- [ ] Security recommendations mentioned + +**2.2 Get rules for App Service**: +``` +Show me the IaC generation rules for Azure App Service and App Service Plan using Bicep +``` + +**Verify**: +- [ ] Rules for both resources provided +- [ ] Dependency relationships explained +- [ ] SKU recommendations included + +### Step 3: Generate Bicep Template (GitHub Copilot - Not MCP) + +> **External Tool Required**: Use GitHub Copilot for template generation + +**Generate Bicep template** (use GitHub Copilot Chat in VS Code): +``` +Generate a Bicep template to create: +- Azure Storage Account with Standard_LRS redundancy +- Blob container for data storage +- App Service Plan (B1 tier) +- App Service web app with storage connection + +Include parameters for resource names and location. +``` + +**Save template** as `infrastructure.bicep` + +### Step 4: Deploy Infrastructure (Azure CLI/Bicep - Not MCP) + +> **External Deployment Required**: Use Azure CLI + +```bash +# Create resource group +az group create --name bugbash-iac-rg --location eastus + +# Deploy Bicep template +az deployment group create \ + --resource-group bugbash-iac-rg \ + --template-file infrastructure.bicep \ + --parameters storageAccountName=bugbashstorage$RANDOM \ + appServiceName=bugbash-app-$RANDOM \ + location=eastus +``` + +### Step 5: Verify Deployment with Azure MCP Server + +**5.1 List storage accounts** (uses `azmcp_storage_account_get`): +``` +List all storage accounts in my subscription +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_storage_account_get` +- [ ] Your deployed storage account appears +- [ ] Account properties match template + +**5.2 List blob containers** (uses `azmcp_storage_blob_container_get`): +``` +List all containers in storage account '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_storage_blob_container_get` +- [ ] Container from template is listed + +**5.3 Get App Service details** (uses `azmcp_appservice_get`): +``` +Show me details for App Service '' in resource group 'bugbash-iac-rg' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_appservice_get` +- [ ] App Service configuration shown +- [ ] Properties match template specification + +### Step 6: Cleanup (External - Not MCP) + +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-iac-rg --yes --no-wait +``` + +**Expected Results**: +- Bicep schema retrieval successful +- IaC generation rules comprehensive +- Template generation guided by schema +- Deployment verification through MCP tools + + +## Common Issues to Watch For + +| Issue | Description | Resolution | +|-------|-------------|------------| +| **Invalid Resource Type** | Bicep schema not found | Verify resource type format (e.g., Microsoft.Storage/storageAccounts) | +| **Schema Version Mismatch** | API version differences | Use latest stable API version for resources | +| **Deployment Tool Confusion** | Wrong tool used | Specify "Bicep" or "Terraform" explicitly in prompts | +| **Missing Best Practices** | Generic guidance returned | Be specific about resource type and deployment tool | +| **Template Syntax Errors** | Generated template invalid | Use retrieved schema to validate template structure | + +## What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output) +- [ ] Expected vs actual results +- [ ] Resource type requested +- [ ] Deployment tool (Bicep/Terraform) +- [ ] Error messages (if any) +- [ ] Screenshots of unexpected behavior + +## Related Resources + +- [Azure Bicep Documentation](https://learn.microsoft.com/azure/azure-resource-manager/bicep/) +- [Azure Terraform Documentation](https://learn.microsoft.com/azure/developer/terraform/) +- [Azure Resource Manager](https://learn.microsoft.com/azure/azure-resource-manager/) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +## Quick Reference: Supported MCP Tools + +### Bicep Support +- `azmcp_bicepschema_get` - Get Bicep schema for resource types + +### IaC Guidance +- `azmcp_deploy_iac_rules_get` - Get IaC generation rules for resource types +- `azmcp_azureterraformbestpractices_get` - Get Terraform best practices +- `azmcp_bestpractices_get` - Get general Azure best practices + +### Resource Verification +- `azmcp_storage_account_get` - List storage accounts +- `azmcp_storage_blob_container_get` - List blob containers +- `azmcp_appservice_get` - Get App Service details +- `azmcp_sql_server_list` - List SQL servers +- `azmcp_cosmos_account_list` - List Cosmos DB accounts + +--- + +**Next**: [PaaS Services Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/paas-services.md) diff --git a/docs/bug-bash/scenarios/paas-services.md b/docs/bug-bash/scenarios/paas-services.md new file mode 100644 index 0000000000..960b1c5180 --- /dev/null +++ b/docs/bug-bash/scenarios/paas-services.md @@ -0,0 +1,195 @@ +# PaaS Services Testing Scenario + +> **MCP Tool Support Notice** +> Azure MCP Server provides **App Service inspection, Function App discovery, and database connection management** capabilities. Resource creation and application deployment require Azure CLI or Portal. This scenario guides you through complete end-to-end workflows, clearly marking when to use MCP tools vs external tools. + +## Objectives + +- Test App Service discovery and inspection +- Verify Function App listing and details retrieval +- Test database connection addition to App Services +- Validate end-to-end PaaS resource workflows + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] GitHub Copilot with Agent mode enabled + +## Scenario 1: App Service Discovery and Database Integration End-to-End + +**Objective**: Complete workflow for App Service inspection and database connection setup + +### Step 1: Create App Service Resources (External - Not MCP) + +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. + +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-paas-rg' in eastus, create an App Service Plan with B1 SKU (Linux), create a Web App with Node.js 18 runtime, create an Azure SQL server with a database named 'bugbash-db' using S0 service objective +``` + +**Option B: Run Azure CLI Commands Manually**: +```bash +# Create resource group +az group create --name bugbash-paas-rg --location eastus + +# Create App Service Plan +az appservice plan create \ + --name bugbash-plan \ + --resource-group bugbash-paas-rg \ + --sku B1 \ + --is-linux + +# Create Web App +az webapp create \ + --name bugbash-webapp-$RANDOM \ + --resource-group bugbash-paas-rg \ + --plan bugbash-plan \ + --runtime "NODE:18-lts" + +# Create Azure SQL for database connection testing +az sql server create \ + --name bugbash-sqlserver-$RANDOM \ + --resource-group bugbash-paas-rg \ + --location eastus \ + --admin-user sqladmin \ + --admin-password + +az sql db create \ + --name bugbash-db \ + --resource-group bugbash-paas-rg \ + --server \ + --service-objective S0 +``` + +### Step 2: Discover App Services with Azure MCP Server + +**2.1 List all App Services** (uses `azmcp_appservice_get`): +``` +List all App Services in my subscription +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_appservice_get` +- [ ] Your web app appears in the list +- [ ] App properties shown (name, location, state, URL) + +**2.2 Get specific App Service details**: +``` +Show me details for App Service '' in resource group 'bugbash-paas-rg' +``` + +**Verify**: +- [ ] Detailed properties displayed +- [ ] Runtime stack shown (Node.js 18) +- [ ] App Service Plan details included +- [ ] Default hostname visible + +### Step 3: Add Database Connection with Azure MCP Server + +**3.1 Add SQL Server database connection** (uses `azmcp_appservice_database_add`): +``` +Add a SQL Server database connection to App Service '' in resource group 'bugbash-paas-rg' for database '' on server '' +``` + +**Verify**: +- [ ] Tool invoked: `azmcp_appservice_database_add` +- [ ] Connection string created +- [ ] Database connection configured +- [ ] Success message displayed + +**3.2 Alternative connection types**: +``` +Add a PostgreSQL database connection to App Service '' for database '' on server '' +``` + +**Verify**: +- [ ] PostgreSQL connection added +- [ ] Connection string format correct + +### Step 4: Verify Configuration (External - Not MCP) + +> **External Verification**: Use Azure CLI to check configuration + +```bash +# List app settings including database connection +az webapp config appsettings list \ + --name \ + --resource-group bugbash-paas-rg \ + --query "[?name=='SQLAZURECONNSTR_Database']" + +# Check connection strings +az webapp config connection-string list \ + --name \ + --resource-group bugbash-paas-rg +``` + +### Step 5: Cleanup (External - Not MCP) + +```bash +# Delete resource group (removes all resources) +az group delete --name bugbash-paas-rg --yes --no-wait +``` + +**Expected Results**: +- App Service discovery works correctly +- Detailed inspection successful +- Database connection addition functional +- Connection strings created properly + +--- + +## Common Issues to Watch For + +| Issue | Description | Resolution | +|-------|-------------|------------| +| **App Service Not Found** | MCP can't locate App Service | Verify resource name and resource group are correct | +| **Function App Missing** | Function App not listed | Ensure Function App exists and you have access permissions | +| **Database Connection Fails** | Can't add database connection | Verify database server exists and firewall allows connections | +| **Runtime Mismatch** | Unexpected runtime shown | Check App Service configuration in Azure Portal | +| **Storage Account Missing** | Function App storage not found | Verify storage account exists in same subscription | +| **Resource Group Filter** | Filtering doesn't work | Ensure resource group name is exact match | + +## What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output) +- [ ] Expected vs actual results +- [ ] Error messages (if any) +- [ ] Resource names and resource group +- [ ] Runtime and plan tier +- [ ] Screenshots of unexpected behavior + +## Related Resources + +- [Azure App Service Documentation](https://learn.microsoft.com/azure/app-service/) +- [Azure Functions Documentation](https://learn.microsoft.com/azure/azure-functions/) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +## Quick Reference: Supported MCP Tools + +### App Service +- `azmcp_appservice_get` - Get App Service details (list all or specific app) +- `azmcp_appservice_database_add` - Add database connection to App Service + +### Function Apps +- `azmcp_functionapp_get` - Get Function App details (list all or specific function app) + +### Storage +- `azmcp_storage_account_get` - List storage accounts + +### Database Support +- SQL Server (`SqlServer`) +- PostgreSQL (`PostgreSQL`) +- MySQL (`MySQL`) +- Cosmos DB (`CosmosDB`) + +--- + +**Next**: [Storage Operations Testing](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/storage-operations.md) diff --git a/docs/bug-bash/scenarios/storage-operations.md b/docs/bug-bash/scenarios/storage-operations.md new file mode 100644 index 0000000000..f34b2f5b1c --- /dev/null +++ b/docs/bug-bash/scenarios/storage-operations.md @@ -0,0 +1,392 @@ +# Storage Operations Testing Scenario + +> **IMPORTANT**: Azure MCP Server provides **read-only inspection and querying** of Azure Storage accounts. It **cannot create, modify, upload, or delete** storage resources. This scenario guides you through creating resources externally (Azure CLI or Portal) and then using Azure MCP Server to inspect them. + +## Objectives + +- Test storage account discovery and inspection capabilities +- Verify container listing functionality +- Test blob enumeration +- Validate storage account filtering and querying + +## Prerequisites + +- [ ] Azure MCP Server installed and configured +- [ ] Azure CLI installed (`az --version`) +- [ ] Authenticated to Azure (`az login`) +- [ ] Active Azure subscription +- [ ] Resource group for testing (or create one) +- [ ] GitHub Copilot with Agent mode enabled + +--- + +## Scenario 1: Storage Account Discovery and Inspection End-to-End + +**Objective**: Test complete workflow of creating a storage account externally, then discovering and inspecting it via Azure MCP Server. + +### External Setup Required + +#### Step 1: Create Storage Account (External - Not MCP) + +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. + +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-storage-rg' in eastus, create a storage account with Standard_LRS SKU and Hot access tier with hierarchical namespace enabled, create two blob containers 'test-data' (private) and 'public-data' (public blob access), then upload a test file to test-data container +``` + +**Option B: Run Azure CLI Commands Manually**: +```bash +# Set variables +RG_NAME="bugbash-storage-rg" +LOCATION="eastus" +STORAGE_ACCOUNT="bugbashstorage$(Get-Random -Maximum 9999)" + +# Create resource group +az group create --name $RG_NAME --location $LOCATION + +# Create storage account with hierarchical namespace +az storage account create \ + --name $STORAGE_ACCOUNT \ + --resource-group $RG_NAME \ + --location $LOCATION \ + --sku Standard_LRS \ + --access-tier Hot \ + --enable-hierarchical-namespace true + +# Create blob containers +az storage container create \ + --name test-data \ + --account-name $STORAGE_ACCOUNT \ + --public-access off + +az storage container create \ + --name public-data \ + --account-name $STORAGE_ACCOUNT \ + --public-access blob + +# Upload sample blob +echo "Test content for bug bash" > test-file.txt +az storage blob upload \ + --account-name $STORAGE_ACCOUNT \ + --container-name test-data \ + --name test-file.txt \ + --file test-file.txt +``` + +**Verify External Setup**: +- [ ] Storage account created successfully +- [ ] Hierarchical namespace enabled +- [ ] Two containers created (test-data, public-data) +- [ ] Sample blob uploaded to test-data container + +### Azure MCP Server Prompts + +#### Step 2: Discover Storage Accounts + +Open GitHub Copilot Chat in Agent mode and use these prompts: + +**Prompt 2a**: List all storage accounts in subscription +``` +Show me all storage accounts in my subscription +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_account_get` +- [ ] Returns list of storage accounts +- [ ] Includes your new storage account + +**Prompt 2b**: Get specific storage account details +``` +Show me details for storage account '' in resource group 'bugbash-storage-rg' +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_account_get` with `--account` and `--resource-group` parameters +- [ ] Returns single storage account +- [ ] Shows SKU as Standard_LRS +- [ ] Shows access tier as Hot +- [ ] Shows hierarchical namespace enabled +- [ ] Shows location as eastus + +#### Step 3: List Blob Containers + +**Prompt 3a**: List all containers +``` +List all blob containers in storage account '' +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_container_list` +- [ ] Returns both containers (test-data, public-data) +- [ ] Shows public access levels correctly + +#### Step 4: List Blobs in Container + +**Prompt 4a**: List blobs in test-data container +``` +Show me all blobs in container 'test-data' from storage account '' +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_blob_list` +- [ ] Returns test-file.txt blob +- [ ] Shows blob size +- [ ] Shows last modified date + +**Prompt 4b**: Get specific blob details +``` +Get details for blob 'test-file.txt' in container 'test-data' from storage account '' +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_blob_get` +- [ ] Returns blob metadata +- [ ] Shows content type +- [ ] Shows blob properties + +#### Step 5: Filter Storage Accounts by Resource Group + +**Prompt 5a**: Filter by resource group +``` +Show me all storage accounts in resource group 'bugbash-storage-rg' +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_account_get` with `--resource-group` parameter +- [ ] Returns only storage accounts in specified resource group +- [ ] Excludes storage accounts from other resource groups + +### Cleanup + +**Option A: Prompt GitHub Copilot Chat**: +``` +Delete the Azure resource group 'bugbash-storage-rg' and all its resources +``` + +**Option B: Run Azure CLI Command Manually**: +```bash +# Delete the resource group and all resources +az group delete --name bugbash-storage-rg --yes --no-wait +``` + +--- + +## Scenario 2: Multi-Container Blob Discovery End-to-End + +**Objective**: Test Azure MCP Server's ability to discover and inspect multiple containers with different access levels and blob types. + +### External Setup Required + +#### Step 1: Create Storage Account with Multiple Containers (External - Not MCP) + +> **External Setup Required**: Azure MCP Server cannot create resources. Use GitHub Copilot Chat to run Azure CLI commands or use Azure Portal. + +**Option A: Prompt GitHub Copilot Chat** (Recommended): +``` +Create an Azure resource group 'bugbash-multicontainer-rg' in westus2, create a storage account with Standard_GRS SKU and Cool access tier, create three containers: 'private-documents' (private), 'public-images' (blob public access), 'shared-data' (container public access), then upload sample files with different content types +``` + +**Option B: Run Azure CLI Commands Manually**: +```bash +# Set variables +RG_NAME="bugbash-multicontainer-rg" +LOCATION="westus2" +STORAGE_ACCOUNT="bugbashmulti$(Get-Random -Maximum 9999)" + +# Create resource group and storage account +az group create --name $RG_NAME --location $LOCATION + +az storage account create \ + --name $STORAGE_ACCOUNT \ + --resource-group $RG_NAME \ + --location $LOCATION \ + --sku Standard_GRS \ + --access-tier Cool + +# Create multiple containers with different access levels +az storage container create \ + --name private-documents \ + --account-name $STORAGE_ACCOUNT \ + --public-access off + +az storage container create \ + --name public-images \ + --account-name $STORAGE_ACCOUNT \ + --public-access blob + +az storage container create \ + --name shared-data \ + --account-name $STORAGE_ACCOUNT \ + --public-access container + +# Upload different file types +echo "Private document content" > document.txt +echo "{ \"data\": \"test\" }" > data.json +echo "Test" > index.html + +az storage blob upload --account-name $STORAGE_ACCOUNT --container-name private-documents --name document.txt --file document.txt +az storage blob upload --account-name $STORAGE_ACCOUNT --container-name public-images --name image.png --file document.txt --content-type "image/png" +az storage blob upload --account-name $STORAGE_ACCOUNT --container-name shared-data --name data.json --file data.json --content-type "application/json" +az storage blob upload --account-name $STORAGE_ACCOUNT --container-name shared-data --name index.html --file index.html --content-type "text/html" +``` + +**Verify External Setup**: +- [ ] Storage account created with Standard_GRS SKU +- [ ] Three containers with different access levels +- [ ] Multiple blobs uploaded with different content types + +### Azure MCP Server Prompts + +#### Step 2: Discover Storage Account with Cool Tier + +**Prompt 2a**: Get storage account details +``` +Show me the storage account '' in resource group 'bugbash-multicontainer-rg' +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_account_get` +- [ ] Shows SKU as Standard_GRS +- [ ] Shows access tier as Cool +- [ ] Shows location as westus2 + +#### Step 3: List All Containers and Their Access Levels + +**Prompt 3a**: List containers +``` +List all blob containers in storage account '' and show their public access settings +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_container_list` +- [ ] Returns three containers: private-documents, public-images, shared-data +- [ ] Shows correct public access level for each: + - [ ] private-documents: None/Off + - [ ] public-images: Blob + - [ ] shared-data: Container + +#### Step 4: Inspect Blobs Across Multiple Containers + +**Prompt 4a**: List blobs in private-documents +``` +Show me all blobs in the 'private-documents' container from storage account '' +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_blob_list` with `--container` parameter +- [ ] Returns document.txt +- [ ] Shows blob size and last modified date + +**Prompt 4b**: List blobs in shared-data +``` +List all blobs in container 'shared-data' from storage account '' +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_blob_list` +- [ ] Returns both blobs: data.json, index.html +- [ ] Shows correct content types for each blob + +**Prompt 4c**: Get specific blob details +``` +Get details for blob 'data.json' in container 'shared-data' from storage account '' +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_blob_get` +- [ ] Shows content type as application/json +- [ ] Shows blob properties and metadata + +#### Step 5: Test Cross-Subscription Discovery (if available) + +**Prompt 5a**: List storage accounts across all accessible subscriptions +``` +Show me all storage accounts I have access to across all my subscriptions +``` + +**Tool Verification**: +- [ ] Tool invoked: `azmcp_storage_account_get` without subscription filter +- [ ] Returns storage accounts from multiple subscriptions (if you have access) +- [ ] Groups results by subscription + +### Cleanup + +**Option A: Prompt GitHub Copilot Chat**: +``` +Delete the Azure resource group 'bugbash-multicontainer-rg' and all its resources +``` + +**Option B: Run Azure CLI Command Manually**: +```bash +# Delete the resource group and all resources +az group delete --name bugbash-multicontainer-rg --yes --no-wait +``` + +--- + +## Common Issues to Watch For + +| Issue | Description | Resolution | +|-------|-------------|------------| +| **Storage Account Not Found** | MCP can't locate storage account | Verify account name spelling and ensure you have access permissions | +| **Container List Empty** | No containers returned | Ensure containers exist and storage account name is correct | +| **Blob List Empty** | No blobs returned for container | Verify container name and that blobs have been uploaded | +| **Authentication Errors** | Can't access storage resources | Run `az login` and verify RBAC permissions (Storage Blob Data Reader/Contributor) | +| **SKU Mismatch** | Unexpected SKU displayed | Check actual storage account configuration in Azure Portal | +| **Access Tier Confusion** | Hot vs Cool tier not displayed correctly | Verify account tier in Azure Portal matches MCP output | +| **Hierarchical Namespace** | Data Lake Gen2 feature not shown | Confirm feature was enabled during storage account creation | +| **Cross-Subscription Issues** | Storage accounts missing from list | Verify you have appropriate permissions in all subscriptions | + +## What to Report + +When logging issues, include: +- [ ] Exact prompt used +- [ ] Tool invoked (from MCP tool output: `azmcp_storage_account_get`, `azmcp_storage_container_list`, `azmcp_storage_blob_list`, etc.) +- [ ] Expected vs actual results +- [ ] Error messages (if any) +- [ ] Storage account name and resource group +- [ ] Container names and public access levels +- [ ] Blob names and content types +- [ ] Screenshots of unexpected behavior +- [ ] Output from Azure CLI showing actual resource state + +## Related Resources + +- [Azure Storage Documentation](https://learn.microsoft.com/azure/storage/) +- [Azure Blob Storage Best Practices](https://learn.microsoft.com/azure/storage/blobs/storage-blobs-introduction) +- [Data Lake Storage Gen2](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-introduction) +- [MCP Command Reference](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/azmcp-commands.md) +- [E2E Test Prompts](https://github.com/microsoft/mcp/blob/main/servers/Azure.Mcp.Server/docs/e2eTestPrompts.md) +- [Report Issues](https://github.com/microsoft/mcp/issues) + +## Quick Reference: Supported MCP Tools + +### Storage Account Operations +- `azmcp_storage_account_get` - Get storage account details (list all or specific account) + - Parameters: `--subscription`, `--resource-group`, `--account` + - Returns: Storage account properties (SKU, access tier, location, hierarchical namespace) + +### Container Operations +- `azmcp_storage_container_list` - List blob containers in a storage account + - Parameters: `--account`, `--resource-group` + - Returns: Container names and public access levels + +### Blob Operations +- `azmcp_storage_blob_list` - List blobs in a container + - Parameters: `--account`, `--container`, `--resource-group` + - Returns: Blob names, sizes, last modified dates + +- `azmcp_storage_blob_get` - Get specific blob details + - Parameters: `--account`, `--container`, `--blob`, `--resource-group` + - Returns: Blob metadata, content type, properties + +### Important Notes +- **Read-Only**: All MCP tools are read-only inspection tools +- **No Write Operations**: Cannot create, upload, modify, or delete resources +- **Authentication**: Requires Azure RBAC permissions (Reader or Storage Blob Data Reader minimum) +- **Filtering**: Supports filtering by subscription, resource group, and specific resource names + +--- + +**Next**: [Agent Building](https://github.com/g2vinay/mcp/tree/add-bug-bash-docs/docs/bug-bash/scenarios/agent-building.md)