[DRAFT] Vibe-code POC: add custom code agent template #4246
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Sid Murching <sid.murching@databricks.com>
smurching
changed the title
Signed-off-by: Sid Murching <sid.murching@databricks.com>
smurching
changed the title
…ent resources - Add comprehensive OAuth authentication setup instructions - Document MLflow experiment resource configuration in databricks.yml - Add app.yaml environment variable configuration examples - Include troubleshooting section for deployed apps - Add detailed deployment workflow with bundle and app deploy commands - Document testing with OAuth tokens and debugging deployed apps
Add comprehensive documentation for agent template: AGENT_TEMPLATE_ARCHITECTURE.md: - Analysis of keeping agent template separate vs merging with app template - Trade-off comparison between approaches - Recommendation to keep templates separate with clear rationale - Future considerations for template categories and new app types - Reference implementation details from default-python template - Decision matrix for when to use conditionals vs separate templates CLAUDE_AGENT_SETUP.md: - Complete guide for configuring Claude Code with CLI from source - Step-by-step installation and verification instructions - How to create agent projects with local and remote templates - Troubleshooting common issues - MCP server architecture explanation - Template development workflow - Environment variable override patterns - Comparison of experimental vs stable templates These docs provide clear guidance for developers working with the agent template and help future decision-making around template architecture.
Update flow.tmpl to include agent template scaffolding example: - Add example command for creating agent projects - Clarify that agent creates conversational AI with OpenAI Agents SDK - Update naming convention notes to include agents This fixes issue where Claude Code would incorrectly use 'app' template instead of 'agent' template because the agent option wasn't shown in databricks_discover output.
Add comprehensive documentation of the issue where Claude Code would incorrectly use the app template instead of agent template. Covers: - Root cause: agent template missing from databricks_discover output - Why SQL warehouse is required for apps but not agents - Template validation mechanics - Solution: added agent example to flow.tmpl - Before/after comparison - Testing verification steps - Future improvement suggestions This helps future developers understand the relationship between databricks_discover output and Claude Code's template selection.
Add comprehensive findings from deploying agent template to Dogfood: Issues categorized by priority: - TODO: Fix MLflow experiment creation in databricks.yml - TODO: Fix port flag being ignored in start-app script - TODO: Add git init guidance to AGENTS.md - TODO: Fix quickstart script requiring interactivity - Backlog: Validation tool assumes Node.js (needs app devex discussion) - Backlog: Agent vs app template naming confusion (needs discussion) Includes detailed logs, error messages, and Claude's workarounds. This provides a roadmap for improving the agent template developer experience based on real deployment attempts.
This commit addresses three critical issues found during agent deployment testing:
1. **Automatic MLflow experiment creation**: Changed databricks.yml to create
the MLflow experiment as a bundle resource instead of requiring manual
variable passing. The experiment is now automatically created at
/Users/${workspace.current_user.userName}/${bundle.name}-${bundle.target}
and granted to the app with CAN_MANAGE permission.
2. **Port flag support in start-app**: Added argparse support to start_app.py
to accept --port, --host, --workers, and --reload flags. These arguments
are now properly passed to the backend server command.
3. **Git repository initialization**: Updated AGENTS.md to document that users
should run 'git init' if they want version control, as the template does not
automatically initialize a git repository.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Added --profile and --host flags to support non-interactive execution: - --profile NAME: Use specified Databricks profile without prompting - --host URL: Specify workspace URL for initial setup - -h, --help: Show usage information This allows Claude Code and other automation tools to run the quickstart script without requiring interactive input for profile selection. Updated AGENTS.md to document the new command-line options. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Added discover-tools CLI command that discovers available resources: - Unity Catalog functions (can be used as agent tools via MCP) - Unity Catalog tables (structured data sources) - Vector Search indexes (for RAG applications) - Genie spaces (conversational data access) - Custom MCP servers (mcp-* Python packages) Usage: uv run discover-tools uv run discover-tools --catalog my_catalog uv run discover-tools --format json --output tools.json Updated AGENTS.md with comprehensive documentation on tool discovery, including examples of how to use discovered UC functions in agents. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Enhanced discovery script to find three types of MCP servers: 1. MCP Server Packages - Local Python packages (mcp-* or containing "mcp") 2. Custom MCP Servers - Databricks Apps starting with "mcp-" 3. External MCP Servers - UC connections with is_mcp_connection=true Fixed JSON serialization issues: - Convert ColumnTypeName enum to string for UC tables - Convert ConnectionType enum to string for connections Updated AGENTS.md with detailed explanations of all MCP server types and how to use them in agents. Tested successfully, discovering: - 126 UC Functions - 1035 UC Tables - 20 Genie Spaces - 2 MCP Server Packages - 6 Custom MCP Servers - 8 External MCP Servers Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Changes to discover-tools script: - Removed discovery of local MCP server packages (pip/uv packages) - Added TOOL_LIST_PAGE_SIZE=100 constant to limit results - Changed Genie spaces discovery to use WorkspaceClient().genie.list_spaces() instead of databricks CLI - Fixed Genie space attribute mapping (title instead of display_name) Updated AGENTS.md to remove MCP server packages section and renumbered remaining resource types (5 instead of 7 total types). Tested successfully: - UC Functions: 100 (limited) - UC Tables: 100 (limited) - Vector Search Indexes: 0 - Genie Spaces: 20 (working with SDK) - Custom MCP Servers: 6 - External MCP Servers: 8 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The agent template was being detected as non-app-only because it has both apps and experiments resources, causing the generic CLAUDE.md and AGENTS.md to overwrite the template-specific versions. Updated detector logic to treat apps + experiments as app-only projects, since experiments are typically used for agent tracing and should be considered part of the app scaffolding. Detection logic now considers a project app-only if: - It has apps and no other resources, OR - It has apps + experiments only (agent pattern) Verified fix: - Created new agent template with full expanded path - CLAUDE.md is 3 lines (reference to AGENTS.md) - AGENTS.md is 591 lines (full agent documentation) - Both files contain template-specific content Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
An authorized user can trigger integration tests manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Hacks/items worth discussing (did these to make the CUJ work):
databricks bundleCLI commands as a fallback in the meantimeOther limitations
Why
Tests