LLMs: Scaffolding agents.md for datasource plugins #2322

mckn · 2025-11-25T13:33:26Z

What this PR does / why we need it:
It will add an agents.md file that cover the fundamentals of datasource plugins (with and without backend). We are following the convention where we put generic information below .config folder (which should not be modified by the developer). Then we reference that file from the project AGENTS.md where the developers of the plugin can add more plugin specific details.

Output from cursor agent after scaffolding a datasource with frontend only and asking the question "What are your instructions and what kind of repository is this? Do you have an specific tasks that you can perform?":

## Repository & Instructions Summary

**This is a Grafana datasource plugin** — a frontend-only TypeScript/React plugin that provides a custom data source for Grafana dashboards.

### My Role
I'm configured as an expert Grafana datasource plugin developer. I'm fluent in TypeScript, React, and know Grafana's plugin architecture.

### What I Can Do

| Task | Description |
|------|-------------|
| **Build/modify the datasource** | Edit `src/datasource.ts`, query/config editors, types |
| **Add template variable support** | Full guided task at `.config/AGENTS/tasks/support-template-variables.md` — includes `metricFindQuery`, `VariableQueryEditor`, interpolation |
| **Write/run tests** | Jest unit tests, Playwright E2E tests using `@grafana/plugin-e2e` |
| **Lint & build** | `yarn lint`, `yarn build`, `yarn test:ci` |
| **Spin up dev server** | `yarn server` (Docker) |

### Key Constraints (I must NOT)
- Change plugin ID/type in `plugin.json`
- Modify anything in `.config/*`
- Break the query model without a migration handler

### Available Commands
yarn install          # deps
yarn build            # production build
yarn dev              # watch mode
yarn test:ci          # run tests
yarn lint:fix         # lint
yarn server           # Docker dev Grafana
yarn e2e              # Playwright E2E


### Specific Documented Task
I have one fully documented task: **"Add Support for Template & Query Variables"** — this walks through:
1. Interpolating `$variables` in queries via `getTemplateSrv().replace()`
2. Implementing `metricFindQuery()` for query-type variables
3. Creating a `VariableQueryEditor` React component + `CustomVariableSupport`

Want me to implement template variable support, modify the query editor, add tests, or something else?

Output from cursor agent after scaffolding a datasource with backend and asking the question "What are your instructions and what kind of repository is this? Do you have an specific tasks that you can perform?":

[5 tools called]

This is a **Grafana datasource plugin** repository. Here's the breakdown:

## What I Am

An expert Grafana datasource plugin developer fluent in:
- **TypeScript/React** (frontend)
- **Go** (backend)
- Grafana dashboards and datasource management

## Repository Structure

| Path | Purpose |
|------|---------|
| `src/` | Frontend (TypeScript/React) |
| `src/datasource.ts` | Frontend datasource extending `DataSourceWithBackend` |
| `src/components/` | ConfigEditor + QueryEditor components |
| `pkg/` | Backend (Go) |
| `pkg/plugin/datasource.go` | Backend query execution & health checks |
| `pkg/models/settings.go` | Instance settings parsing |

## What I Can Do

### General tasks:
- Build/modify the **query editor** (UI for building queries)
- Build/modify the **config editor** (datasource settings UI)
- Implement **backend query handlers** in Go
- Add **health check** logic
- Write **E2E tests** with `@grafana/plugin-e2e`

### Specific documented task:
- **Add template variable support** — includes:
  1. Interpolate `$var` placeholders in queries via `getTemplateSrv().replace()`
  2. Implement `metricFindQuery` for Query-type variables
  3. Build a `VariableQueryEditor` component

## Key Commands

yarn install          # Install deps
yarn run build        # Production build
yarn run dev          # Dev build (watch mode)
yarn run test:ci      # Run Jest tests
yarn run lint         # Lint
mage -v               # Build Go backend
yarn run server       # Spin up Grafana dev server
yarn run e2e          # Run Playwright E2E tests

## Constraints I Follow
- Never modify `.config/*` or plugin ID
- Use `@grafana/ui`, `useTheme2()`, `useStyles2()` — no hardcoded colors/spacing
- Secure data goes in `secureJsonData`
- Backend uses Grafana SDK HTTP client, `debug`/`error` logging only

Want me to do something specific?

Which issue(s) this PR fixes:

Fixes https://github.com/grafana/grafana-community-team/issues/674

Special notes for your reviewer:

github-actions · 2025-11-25T13:33:43Z

Hello! 👋 This repository uses Auto for releasing packages using PR labels.

✨ This PR can be merged but will not trigger a new release. To trigger a new release add the release label before merging.
NOTE: When merging a PR with the release label please avoid merging another PR. For further information see here.

packages/create-plugin/templates/datasource/.config/AGENTS/instructions.md

tolzhabayev · 2025-11-27T12:54:03Z

We should probably add CLAUDE.md file pointing to Agents.md like this: https://github.com/grafana/plugin-tools/blob/3aed54e7d9d26d14dfbaa229806c3377391ccfcc/CLAUDE.md?plain=1

andresmgot

A bunch of comments here 👍

packages/create-plugin/templates/datasource/.config/AGENTS/instructions.md

packages/create-plugin/templates/datasource/.config/AGENTS/tasks/support-template-variables.md

mckn · 2025-12-19T07:16:02Z

@andresmgot currently the agent will only pickup mage -v as the commands for the backend. Is that enough or should it also pick up more commands?

academo · 2025-12-19T08:29:57Z

packages/create-plugin/templates/datasource/.config/AGENTS/tasks/support-template-variables.md

@@ -0,0 +1,307 @@
+# Add Support for Template & Query Variables


I wonder if we are better off pointing the agent to fetch the information from our documentation site instead of leaving here possibly stale instructions that will backfire.

I asked the same thing but apparently external URLs don't work well.

We did try that but it won't pick that information up unless you ask is specifically in the prompt to go and fetch the content of a URL. But agree with you it would be nice to have it digest that information instead. Then there are differences in how you should provide information to a LLM and to a human so it is hard to optimise the docs for both.

But I'm happy to revisit this topic if we can find a way to get the agent to digest URLs as well (without being asked about it explicitly).

unless you ask is specifically in the prompt to go and fetch the content of a URL

this should be ok IMO. the user (dev) will see the system trying to fetch an official grafana docs and it'll likely just let it do it, and the model will only try to fetch it when it feels necessary to complement the information

Yeah, but the question is how you do that in the agent.md file?

On other words. How should we specify the references in the agent.md file to make the agent actually try to read the URLs. I have not found a way to make that happen. It just reads the URL and potential information about it but never tries to fetch the content of the URL.

andresmgot · 2025-12-19T09:02:29Z

packages/create-plugin/templates/datasource/.config/AGENTS/tasks/support-template-variables.md

@@ -0,0 +1,307 @@
+# Add Support for Template & Query Variables


I asked the same thing but apparently external URLs don't work well.

andresmgot · 2025-12-19T09:05:26Z

packages/create-plugin/templates/datasource/.config/AGENTS/tasks/support-template-variables.md

+File: `src/datasource.ts`
+
+```ts
+export class DataSource extends DataSourceApi<MyQuery> {


I wonder if the LLM will get confused if the datasource extends DataSourceWithBackend?

Good feedback! I will test that scenario. We could add another example if needed.

packages/create-plugin/templates/datasource/.config/AGENTS/instructions.md

andresmgot · 2025-12-19T09:13:44Z

@andresmgot currently the agent will only pickup mage -v as the commands for the backend. Is that enough or should it also pick up more commands?

There are definitively more commands:

$ mage -l
Targets:
  build:backend                 build a production build for the current platform
  build:custom                  allows customizable back-end plugin builds for the provided os and arch.
  build:darwin                  builds the back-end plugin for OSX on AMD64.
  build:darwinARM64             builds the back-end plugin for OSX on ARM (M1/M2).
  build:debug                   builds the debug version for the current platform.
  build:debugDarwinAMD64        builds the debug version targeted for OSX on AMD64.
  build:debugDarwinARM64        builds the debug version targeted for OSX on ARM (M1/M2).
  build:debugLinuxAMD64         builds the debug version targeted for linux on AMD64.
  build:debugLinuxARM64         builds the debug version targeted for linux on ARM64.
  build:debugWindowsAMD64       builds the debug version targeted for windows on AMD64.
  build:generateManifestFile    generates a manifest file for plugin submissions
  build:linux                   builds the back-end plugin for Linux.
  build:linuxARM                builds the back-end plugin for Linux on ARM.
  build:linuxARM64              builds the back-end plugin for Linux on ARM64.
  build:windows                 builds the back-end plugin for Windows.
  buildAll*                     builds production executables for all supported platforms.
  clean                         cleans build artifacts, by deleting the dist directory.
  coverage                      runs backend tests and makes a coverage report.
  debugger                      makes a new debug build, re-launches the plugin and attaches the Delve debugger, in headless mode listening on port 3222.
  format                        formats the sources.
  lint                          audits the source style
  reloadPlugin                  - kills any running instances and waits for grafana to reload the plugin
  test                          runs backend tests.
  testRace                      runs backend tests with the data race detector enabled.
  watch                         rebuilds the plugin backend debug version when files change.

* default target

For comparing it to the frontend, I guess it should list at least lint, test and watch.

mckn · 2025-12-19T10:52:02Z

@andresmgot currently the agent will only pickup mage -v as the commands for the backend. Is that enough or should it also pick up more commands?

There are definitively more commands:

$ mage -l
Targets:
  build:backend                 build a production build for the current platform
  build:custom                  allows customizable back-end plugin builds for the provided os and arch.
  build:darwin                  builds the back-end plugin for OSX on AMD64.
  build:darwinARM64             builds the back-end plugin for OSX on ARM (M1/M2).
  build:debug                   builds the debug version for the current platform.
  build:debugDarwinAMD64        builds the debug version targeted for OSX on AMD64.
  build:debugDarwinARM64        builds the debug version targeted for OSX on ARM (M1/M2).
  build:debugLinuxAMD64         builds the debug version targeted for linux on AMD64.
  build:debugLinuxARM64         builds the debug version targeted for linux on ARM64.
  build:debugWindowsAMD64       builds the debug version targeted for windows on AMD64.
  build:generateManifestFile    generates a manifest file for plugin submissions
  build:linux                   builds the back-end plugin for Linux.
  build:linuxARM                builds the back-end plugin for Linux on ARM.
  build:linuxARM64              builds the back-end plugin for Linux on ARM64.
  build:windows                 builds the back-end plugin for Windows.
  buildAll*                     builds production executables for all supported platforms.
  clean                         cleans build artifacts, by deleting the dist directory.
  coverage                      runs backend tests and makes a coverage report.
  debugger                      makes a new debug build, re-launches the plugin and attaches the Delve debugger, in headless mode listening on port 3222.
  format                        formats the sources.
  lint                          audits the source style
  reloadPlugin                  - kills any running instances and waits for grafana to reload the plugin
  test                          runs backend tests.
  testRace                      runs backend tests with the data race detector enabled.
  watch                         rebuilds the plugin backend debug version when files change.

* default target

For comparing it to the frontend, I guess it should list at least lint, test and watch.

Nice, we should probably include them in the README.md then as well. Do you see any problems with adding those to the readme? reloadPlugin sounds interesting as well.

Alternatively provide information about the mage -l command.

andresmgot · 2025-12-19T12:34:52Z

Nice, we should probably include them in the README.md then as well. Do you see any problems with adding those to the readme? reloadPlugin sounds interesting as well.

Sounds good to me

tolzhabayev · 2026-01-06T09:56:12Z

packages/create-plugin/templates/datasource/.config/AGENTS/instructions.md

+You must **NOT**:
+
+- Change plugin ID or plugin type in `plugin.json`
+- Modify anything inside `.config/*`


Suggested change

- Modify anything inside `.config/*`

- Modify anything inside `.config/*`

- Remove or modify the scaffolded packages in package.json and go.mod

Had this "packages being removed from package.json" happen to me recently, and then it went into a weird direction trying to fix the issues by adjusting tsconfig and a bunch of files in .config folder.

But for some scenarios we need the agent to update the dependencies? I wonder if we can rewrite this to something like:

Change any scaffolded dependencies in package.json or go.mod unless you have explicit approval to do so.

What do you think?

Change any scaffolded dependencies in package.json or go.mod unless you have explicit approval to do so.

Is it not the case for any command by default?

Maybe:

Change any scaffolded dependencies in package.json or go.mod unless you are explicit being asked to do so.

Also, I think the following line need to be more explicit:

Modify anything inside .config/*

->

Never modify files in .config/ without explicit user approval

Also, I think the star .config/* <-- is causing issues here

jackw

LGTM! 🚀

packages/create-plugin/templates/datasource/.config/AGENTS/instructions.md

academo · 2026-01-15T14:47:10Z

packages/create-plugin/templates/datasource/.config/AGENTS/instructions.md

+
+## Boundaries
+
+You must **NOT**:


the LLm will quickly forget the you must not title and start doing the things below like a list of things they can do.

you will have to put "do not" or "must not" in every line.

github-project-automation bot added this to Plugins Platform / Grafana Community Nov 25, 2025

github-project-automation bot moved this to 📬 Triage in Plugins Platform / Grafana Community Nov 25, 2025

mckn self-assigned this Nov 25, 2025

mckn moved this from 📬 Triage to 🧑‍💻 In development in Plugins Platform / Grafana Community Nov 25, 2025

mckn added create-plugin related to the create-plugin tool minor Increment the minor version when merged labels Nov 25, 2025

tolzhabayev reviewed Nov 27, 2025

View reviewed changes

packages/create-plugin/templates/datasource/.config/AGENTS/instructions.md Outdated Show resolved Hide resolved

tolzhabayev reviewed Nov 27, 2025

View reviewed changes

packages/create-plugin/templates/datasource/.config/AGENTS/instructions.md Outdated Show resolved Hide resolved

tolzhabayev reviewed Nov 27, 2025

View reviewed changes

packages/create-plugin/templates/datasource/.config/AGENTS/instructions.md Outdated Show resolved Hide resolved

tolzhabayev reviewed Nov 27, 2025

View reviewed changes

packages/create-plugin/templates/datasource/.config/AGENTS/instructions.md Show resolved Hide resolved

andresmgot reviewed Nov 27, 2025

View reviewed changes

andresmgot reviewed Nov 28, 2025

View reviewed changes

packages/create-plugin/templates/datasource/.config/AGENTS/tasks/support-template-variables.md Outdated Show resolved Hide resolved

mckn changed the title ~~Vibe: Scaffolding agents.md for datasource plugins~~ LLMs: Scaffolding agents.md for datasource plugins Dec 8, 2025

mckn added 14 commits December 18, 2025 16:03

wip

56e2f53

wip

43c0bc2

wip

1718e20

wip

68e9c0e

wip

40503a1

wip

670e592

added claude redirect to agents.md

bb6815a

added task for adding template variable support.

7bb76e9

updated variable support.

5726492

fixed nits.

d2b568d

wip

f5286f4

template variable task.

39ac36f

wip

17555c2

backend support.

c843c5a

mckn force-pushed the mckn/agents-md-datasource-plugin branch from c3a2832 to c843c5a Compare December 18, 2025 15:03

mckn moved this from 🧑‍💻 In development to 🔬 In review in Plugins Platform / Grafana Community Dec 18, 2025

mckn marked this pull request as ready for review December 18, 2025 15:03

mckn requested review from a team as code owners December 18, 2025 15:03

mckn requested review from andresmgot, jackw, toddtreece, tolzhabayev and xnyo December 18, 2025 15:03

academo reviewed Dec 19, 2025

View reviewed changes

andresmgot approved these changes Dec 19, 2025

View reviewed changes

tolzhabayev reviewed Jan 6, 2026

View reviewed changes

jackw approved these changes Jan 7, 2026

View reviewed changes

packages/create-plugin/templates/datasource/.config/AGENTS/instructions.md Outdated Show resolved Hide resolved

packages/create-plugin/templates/datasource/.config/AGENTS/instructions.md Outdated Show resolved Hide resolved

mckn added 6 commits January 15, 2026 11:45

adding commands to the backend readme.

297a0d9

added ts(x)

604f780

adding executable details.

45921a5

frontend ds extending different base classes depending on hasbackend.

69d9ec0

adding some more shoulds.

8e13dde

best practies.

ff0be90

mckn requested a review from tolzhabayev January 15, 2026 11:22

academo reviewed Jan 15, 2026

View reviewed changes

academo mentioned this pull request Jan 15, 2026

LLMs: Scaffold AGENTS.md for LLM coding agents #2398

Draft

		@@ -0,0 +1,307 @@
		# Add Support for Template & Query Variables

LLMs: Scaffolding agents.md for datasource plugins #2322

Are you sure you want to change the base?

LLMs: Scaffolding agents.md for datasource plugins #2322

Uh oh!

Conversation

mckn commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tolzhabayev commented Nov 27, 2025

Uh oh!

andresmgot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mckn commented Dec 19, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

andresmgot commented Dec 19, 2025

Uh oh!

mckn commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andresmgot commented Dec 19, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jackw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mckn commented Nov 25, 2025 •

edited

Loading

github-actions bot commented Nov 25, 2025 •

edited

Loading

mckn commented Dec 19, 2025 •

edited

Loading