[FEATURE] Add SQL Datasource Plugin with Explorer Support

[ZPascal]

Description

This PR introduces a comprehensive SQL datasource plugin for Perses, enabling users to query and visualize data from PostgreSQL, MySQL, and MariaDB databases. The plugin includes a full-featured Explorer mode similar to the Prometheus plugin, providing both table and graph views for SQL query results.

Key Features

SQL Datasource Plugin

• Support for PostgreSQL, MySQL, and MariaDB
• TLS/SSL connection support with certificate validation
• Secret management integration for secure credential storage
• Connection string validation and error handling
• Database-specific driver selection

SQL Time Series Query Plugin

• Rich query editor with SQL syntax highlighting
• SQL macro support for dynamic time-based queries:
  • $__timeFilter(column) - Generates time range filter
  • $__timeFrom - Query start time
  • $__timeTo - Query end time
  • $__interval - Auto-calculated interval
  • $__interval_ms - Interval in milliseconds
• Backend proxy integration for secure query execution
• Full support for Perses variables in queries

SQL Explorer Plugin (New)

• Prometheus-style explorer with dual-tab interface
• Table View: Raw query results with column sorting
• Graph View: Time series visualization using existing Perses panels
• Multi-query editor with add/remove capabilities
• Panel preview functionality
• Seamless integration with dashboard workflow

Architecture

The plugin follows Perses plugin architecture best practices:

Frontend (TypeScript/React)

• Plugin module using Module Federation for dynamic loading
• CUE schema validation for datasource and query configurations
• React components for the datasource editor, query editor, and explorer UI
• Utility functions for SQL macro replacement and time handling

Backend Integration

• Backend proxy handles actual database connections
• Secure credential management via secrets API
• Support for sslmode=disable for development environments
• Detailed error messages for connection and query failures

Testing

Unit Tests (12 tests, all passing)

• SQL macro replacement logic (replace-sql-builtin-variables.test.ts)
• SQL client configuration (sql-client.test.ts)
• Datasource component rendering (SQLDatasource.test.tsx)

Test Data

• Docker Compose setup with PostgreSQL, MySQL, and MariaDB
• Sample databases with realistic time-series data
• Adminer UI for database inspection
• Makefile commands for easy database management

Documentation

• README.md: Comprehensive usage guide with examples
• CUE Schemas: Schema definitions for validation
• Code Comments: Apache License headers and JSDoc documentation

Screenshots

SQL Datasource Configuration

SQL Explorer - Table View

SQL Plugin View

Dashboard with SQL Panel

Checklist

• Pull request has a descriptive title and context useful to a reviewer.
• Pull request title follows the [FEATURE] <commit message> naming convention.
• All commits have DCO signoffs.
• Test the SQL plugin for MariaDB
• Test the SQL plugin for MySQL

UI Changes

• Changes that impact the UI include screenshots and/or screencasts of the relevant changes.
• Code follows the UI guidelines.
• E2E tests are stable and unlikely to be flaky.
  • Note: E2E tests not yet implemented. Unit tests cover business logic, and manual testing has been performed. E2E tests can be added in a follow-up PR if required.

Additional Notes

What's Included

✅ Production-ready features:

• 3 database drivers (PostgreSQL, MySQL, MariaDB)
• Full TLS/SSL support
• SQL macro system
• Explorer mode with table and graph views
• Secret management integration
• Comprehensive documentation
• Unit test coverage for core logic

Future Enhancements (Not Blocking)

• E2E tests for UI workflows
• Additional UI component tests
• ESLint configuration (TypeScript already catches most issues)
• Performance optimizations for large datasets

Testing Instructions

1. Start test databases:

   cd plugins/sql
   make db-up

2. Run unit tests:

   npm test

3. Build the plugin:

   npm run build

4. Manual testing:

   • Configure a SQL datasource in Perses
   • Use the query editor to write SQL queries with macros
   • Open the Explorer to build queries interactively
   • Create a dashboard panel using the SQL datasource

Database Compatibility

Database	Version	Status
PostgreSQL	15+	✅ Fully supported
MySQL	8.0+	✅ Fully supported
MariaDB	10.6+	✅ Fully supported

Migration Notes

This is a new plugin and does not require migration from existing configurations.

Related issues

Add PostgreSQL plugin support

Related PRs

Adapt the proxy output
Handle the MariaDB support for the backend proxy

[jgbernalp]

Related to: perses/perses#2930

[rickardsjp]

Thanks for the contribution. Please add the sql plugin to the top level (monorepo root) package.json workspaces list. Otherwise, it will not be included in e.g. the build and lint commands. Once you include it, please take a look at the linter issues, too.

[AntoineThebaud]

Please check how to improve this query schema to ensure it's compatible with the DatasourceVariable plugin https://github.com/perses/plugins/pull/540/changes#diff-6f1afe7036188e8a850133c2bbb0e495ec791fc97629478bb7581e291a7e916a

You'll have to edit the datasource schema to provide a #selector, see https://github.com/perses/plugins/pull/540/changes#diff-8761f27760297b04f98f79b9ff23665a6649ab00556da9fe03091cf7d5f25915R28

[ZPascal]

Update: I'll further update the source code after this PR is merged.