Skip to content

Refactor/modernize project structure#4

Open
Luminger wants to merge 44 commits intomainfrom
refactor/modernize-project-structure
Open

Refactor/modernize project structure#4
Luminger wants to merge 44 commits intomainfrom
refactor/modernize-project-structure

Conversation

@Luminger
Copy link
Collaborator

Not ready for merge yet...

@Luminger Luminger force-pushed the refactor/modernize-project-structure branch 2 times, most recently from d6bb979 to 966c31b Compare September 21, 2025 13:59
- Move all source code from knoepfe/ to src/knoepfe/ following Python src layout
- Implement plugin system with entry points for widget registration
- Remove OBS and audio widgets from core, moved to separate plugins
- Update project structure to support workspace with plugin development
- Migrate from docopt to click for CLI interface
- Update dependencies and remove version upper bounds
- Add plugin manager for dynamic widget loading
- Restructure configuration with separate streaming config
- Update GitHub Actions and pre-commit config for new structure
- Bump version to 0.2.0 for breaking changes
- Switch from black/mypy to ruff for linting and formatting
- Update test suite for new architecture

BREAKING CHANGE: Project structure changed to src layout, OBS and audio widgets moved to plugins
- Implement cython-hidapi based transport for StreamDeck devices
- Add hidapi>=0.14.0.post4 dependency to pyproject.toml
- Provides compiled performance and safer resource management
- Addresses shutdown race conditions with proper cleanup ordering
- Maintains full compatibility with LibUSBHIDAPI interface
- Includes platform-specific workarounds (macOS HIDAPI 0.9.0 bug)
- Thread-safe operations with consistent error handling
- Drop-in replacement that can be used via monkey-patching

The transport offers improved performance through compiled Cython code,
better resource management with weakref finalizers, and enhanced API
coverage while maintaining backward compatibility.
…onal

- Extract Knoepfe application class to dedicated app.py module
- Move CLI commands and main entry point to cli.py module
- Rename log.py to logging.py for clarity
- Remove redundant __main__.py file
- Update entry point in pyproject.toml to point directly to cli.main
- Add --no-cython-hid flag to optionally disable CythonHIDAPI transport
- Apply monkey patch conditionally instead of at import time
- Replace hardcoded versions in pyproject.toml with dynamic = ["version"]
- Configure Hatchling to read versions from respective __init__.py files
- Applied to main project and all plugins (audio, example, obs)
- Fix uv sync command to use dependency groups (--group dev) instead of deprecated --extra dev
- Replace pre-commit hooks with astral-sh/ruff-action@v3 for better integration
- Add comprehensive plugin coverage: check and format all plugins (obs, audio, example)
- Add missing example plugin tests to ensure all plugins are tested
- Remove .pre-commit-config.yaml and .flake8 files (no longer needed)
- Remove ruff from apt dependencies (now handled by ruff-action)

Fixes the failing GitHub Actions workflow that was unable to sync dependencies
and modernizes the linting approach to use the official ruff-action instead of
pre-commit. All plugins are now properly tested and linted with comprehensive
coverage across the entire workspace.
Apply ruff format to OBS plugin source files and core test files to resolve
formatting inconsistencies and ensure compliance with project code style standards.

- Fixed formatting in OBS plugin: base.py, config.py, current_scene.py
- Fixed formatting in OBS plugin: recording.py, streaming.py, switch_scene.py
- Fixed formatting in core tests: test_config.py, test_main.py
Configure ruff to focus on project-specific directories for linting and formatting.

- Added include patterns for src/**/*.py, tests/**/*.py, plugins/**/*.py
- Removed redundant exclude configuration
- Replace knoepfe.__main__ imports with knoepfe.app and knoepfe.cli
- Update all patch references to use correct module paths
- Handle SystemExit exception from Click CLI framework
- Fix test assertions to use run_sync instead of run

Resolves test failures caused by recent project structure modernization.
@Luminger Luminger force-pushed the refactor/modernize-project-structure branch from 08e5a0c to 5fbf910 Compare September 21, 2025 14:21
- Replace exclude list with explicit include list in hatch sdist config
- Ensures only necessary files are included in source distribution
- Fixes build failure caused by unwanted files in packaging
- Resolves 'uv build --all-packages' packaging errors
- Remove SCM version automation from GitHub workflows
- Standardize pyproject.toml formatting across all packages
- Add comprehensive build configuration (sdist includes, ruff settings)
- Update workspace dependencies and optional dependencies
- Extend type checking and testing to include tests/ directories
- Add proper plugin dependencies to main package optional-dependencies
- Update uv.lock with new dependency structure
@Luminger Luminger force-pushed the refactor/modernize-project-structure branch 3 times, most recently from e30c408 to 9668f8b Compare September 21, 2025 20:34
Replace bundled fonts with system font access through fontconfig integration.

Added:
- FontManager class with fontconfig pattern support and caching
- text_at() method for precise text positioning with anchors
- Support for fontconfig patterns (e.g., "Ubuntu:style=Bold", "monospace")
- python-fontconfig dependency
- Tests for fontconfig functionality with shared mock helper

Changed:
- Renderer.text() now accepts font parameter and anchor positioning
- _render_text() updated to support fontconfig patterns and Pillow anchors
- Default font set to "Roboto" for backward compatibility
- Refactored test mocking to use shared mock_fontconfig_system() helper

Removed:
- Bundled Roboto-Regular.ttf (now uses system version)
@Luminger Luminger force-pushed the refactor/modernize-project-structure branch from 9668f8b to 24f7a14 Compare September 21, 2025 20:43
@Luminger Luminger marked this pull request as draft September 21, 2025 20:47
- Add WidgetAction system with SwitchDeckAction for extensible widget communication
- Replace SwitchDeckException with return-based actions in Widget.released()
- Update Deck.handle_key() to propagate widget actions to DeckManager
- Modify DeckManager to handle actions instead of catching exceptions
- Remove bolted-on switch_deck logic in favor of integrated action handling
- Update all tests to work with new action-based system

This change eliminates exceptions for control flow and provides a cleaner,
more extensible foundation for widget-to-manager communication.
@Luminger Luminger force-pushed the refactor/modernize-project-structure branch from 4dc2b32 to 9a09041 Compare September 22, 2025 19:19
…pendency

BREAKING CHANGE: Remove icon-specific methods in favor of unified text rendering

- Remove icon(), icon_and_text(), and _get_font() methods from Renderer
- Eliminate MaterialIcons-Regular.codepoints file dependency
- Simplify _render_text() to only handle fontconfig patterns
- Update all plugins to use Unicode characters with Material Icons font
- Maintain backward compatibility for existing text rendering
- All tests passing (52/52)

Users must now specify Unicode characters directly:
- Before: renderer.icon("videocam")
- After: renderer.text("\ue04b", font="Material Icons")

This enables flexible font usage via fontconfig patterns while
simplifying the codebase and removing hardcoded icon mappings.
@Luminger Luminger force-pushed the refactor/modernize-project-structure branch from e572b07 to ce5d933 Compare September 23, 2025 21:23
- Fix Material Icons text alignment by adding anchor="mm" to center icons
  properly in mic mute, timer, and all OBS widgets
- Fix OBS connector to use ws_open property instead of deprecated
  ws.ws.open access pattern
- Add proper module exports to OBS plugin __init__.py with config
  import and __all__ declaration
- Update OBS recording tests to match new text-based Material Icons
  rendering instead of deprecated icon() calls
- Update audio mic mute tests to match new text-based Material Icons
  rendering instead of deprecated icon() calls
@Luminger Luminger force-pushed the refactor/modernize-project-structure branch from ce5d933 to 6fa9680 Compare September 23, 2025 21:27
- python-fontconfig >=0.6.2.post1 is required as it fixes a segfault
- Update uv.lock with new dependency version
- Add ConfigPluginNotFoundError for missing config plugins
- Add WidgetNotFoundError for missing widget plugins
- Replace generic "Failed to parse configuration" with specific error messages
- Config errors now suggest plugin installation is needed
- Widget errors now direct users to 'knoepfe list-widgets' command
- Update tests to expect new WidgetNotFoundError instead of ValueError
- Replace widget-based entry points with plugin-based architecture
- Add Plugin base class with standardized interface for all plugins
- Introduce WidgetManager for centralized widget registration and lookup
- Enhance PluginManager with plugin lifecycle management and configuration
- Add name attribute to all widgets and make Widget abstract
- Change configuration syntax from type-based to name-based widget creation
- Update entry points from "knoepfe.widgets" to "knoepfe.plugins"
- Restructure plugin configuration from type-based to plugin-name-based
- Update all existing plugins (audio, example, obs) to new architecture
- Remove obsolete config.py files and consolidate plugin initialization
- Update documentation with new plugin installation and usage instructions
- Fix OBS Widget long press crash when OBS is not connected
- Add comprehensive renderer API supporting 4 use cases:
  * Icon/picture only rendering
  * Icon/picture with text combinations
  * Text-only rendering with wrapping
  * Direct drawing access for custom visualizations

- Implement new convenience methods:
  * icon() for Material Icons rendering
  * image_centered() for centered image placement
  * icon_and_text() for icon+label layouts
  * image_and_text() for image+label layouts
  * text_wrapped() for automatic text wrapping
  * draw_image() for flexible image rendering

- Update Key class to pass global config to renderer for default fonts
- Migrate all core widgets (Text, Clock, Timer) to new API
- Migrate all plugin widgets (OBS, Audio, Example) to new API
- Fix OBS CurrentScene widget to not show text when disconnected
- Update Deck class to pass global config through to renderers
@Luminger Luminger force-pushed the refactor/modernize-project-structure branch from 3346dcd to f477e7e Compare September 26, 2025 13:13
- Update widget() calls to use correct syntax: widget('Name', {config}) instead of widget({'type': 'Name', ...})
- Fix OBS plugin config() syntax to use config('obs', {...}) instead of incorrect nested structure
- Update OBS WebSocket default port from 4444 to 4455 to match actual defaults
…s widgets

Add PluginState container pattern to allow plugins to share state across
their widget instances while avoiding circular imports between Plugin and
Widget classes.

Core changes:
- Add PluginState base class for plugin state containers
- Plugin.create_state() creates state instance passed to widgets
- Widget.__init__ now accepts state parameter (3rd argument)
- Widget is Generic[TPluginState] for type-safe state access
- Plugin.config_schema returns Schema (not Schema | None)
- Remove Plugin.name class attribute (use entry point name)
- Add TYPE_CHECKING guard in plugin.py to prevent circular imports
- Add Widget.description optional class attribute

PluginManager refactoring:
- Merge WidgetManager functionality into PluginManager
- Rename PluginMetadata → PluginInfo, add WidgetInfo dataclass
- Validate Plugin subclass on load (not just any callable)
- Validate plugin configs against schemas on instantiation
- Discover widgets via Plugin.widgets property
- Add WidgetNotFoundError exception
- Store plugin name from entry point, not class attribute

Plugin implementations:
- AudioPlugin: Add AudioPluginState with default_source
- ExamplePlugin: Add ExamplePluginState with click counter
- OBSPlugin: Add OBSPluginState with shared OBS connector
- BuiltinPlugin: New plugin for Clock, Text, Timer widgets

All widgets updated to accept state parameter in __init__.
All tests updated with state fixtures and mocking.
- Replace schema library with pydantic for type-safe configuration validation
- Introduce generic typing with TypeVar for Plugin[TConfig, TContext] and Widget[TConfig, TContext]
- Rename PluginState to PluginContext and move shutdown logic from Plugin to PluginContext
- Restructure project with new package organization (core/, config/, plugins/, utils/, rendering/)
- Implement Python DSL for configuration files with builder pattern
- Add pydantic models for DeviceConfig, GlobalConfig, DeckConfig, WidgetSpec, PluginConfig, and WidgetConfig
- Move builtin widgets (Clock, Text, Timer) to widgets/builtin/ with typed configs
- Update CLI with new command structure (widgets/plugins subcommands with list/info)
- Add type extraction utilities for runtime generic parameter inspection
- Update all tests to use pydantic ValidationError instead of SchemaError
- Add test script for running all plugin tests
- Move default config files to src/knoepfe/data/ directory
- Update dependency: remove schema>=0.7.7, add pydantic>=2.11.9
…cycle

- Add TaskManager utility class for unified task lifecycle management across widgets and plugins
- Replace manual Task tracking with TaskManager in Widget base class (periodic updates, long press detection)
- Migrate AudioWidget to use TaskManager for event listener tasks
- Migrate OBSWidget to use TaskManager for event listener tasks
- Update PulseAudioConnector to use TaskManager for event watcher task
- Update OBS connector to use TaskManager for connection and status watcher tasks
- Implement automatic task cleanup in Deck.deactivate() before widget deactivation
- Add TaskManager to PluginContext for plugin-wide task management
- Preserve Timer widget state across deck switches while managing periodic updates via TaskManager
- Remove manual task cleanup from Clock widget (now handled automatically)
- Update all tests to mock TaskManager and verify new task management behavior
@Luminger Luminger force-pushed the refactor/modernize-project-structure branch 2 times, most recently from 345f3dc to b5e7e1c Compare October 12, 2025 18:31
Replace simple format string with flexible segment-based rendering.
Segments support individual positioning, sizing, fonts, and colors.

BREAKING CHANGE: Clock config changed from `format` to `segments` list.
Old: widget.Clock(format='%H:%M')
New: widget.Clock(segments=[{'format': '%H:%M', 'x': 0, 'y': 0, 'width': 96, 'height': 96}])

- Add ClockSegment config with position/size/font/color
- Auto-calculate font size to fit segment bounds
- Add configurable update interval
- Update all configs and tests
- Add clock_examples.cfg with 6 layout demos
Both CythonHIDAPI and Dummy transports were returning non-None values
(b"" and bytearray respectively) when no data was available. This caused
the StreamDeck library's polling loop to never sleep between reads,
resulting in 100% CPU usage on a single core.

The StreamDeck._read() loop checks if _read_control_states() returns None
and only then sleeps for 1.0/read_poll_hz seconds. By returning non-None
values, this sleep was never triggered, causing a tight busy loop.

Changes:
- CythonHIDAPI.Device.read(): Return None instead of b""
- Created transport/patches.py with apply_transport_patches()
- Dummy.Device.read(): Patched to return None instead of bytearray(length)
- Moved CythonHIDAPI enablement into patches module
- Updated cli.py to use centralized patching function
- Updated transport README with usage examples

Added type: ignore comments since the base class Transport.Device.read()
has an incorrect signature that doesn't allow None returns, despite the
actual implementations returning None.

Fixes: 100% CPU usage when using cython-hidapi or dummy transports
…ering

- Replace Roboto + Material Icons with single Roboto Mono Nerd Font
- Remove default_icon_font config (icons now bundled in text font)
- Update all widget icon codepoints to Nerd Font Material Design equivalents
- Add inline comments with icon names for all codepoints
- Update tests and documentation to reflect new font system

BREAKING CHANGE: default_icon_font config option removed, use default_text_font for both text and icons
Add optional `index` parameter to widgets for specifying display position.
Widgets without an index are auto-assigned sequentially, filling gaps left
by explicitly indexed widgets.

- Add index field to WidgetConfig (default: None)
- Implement index assignment in Deck with validation
- Log warning when widgets exceed device capacity
- Add comprehensive tests for all index scenarios
- Update example configs with documentation
Fixes horizontal misalignment of icons in monospace fonts by calculating
actual glyph bounds instead of relying on font advance width.

Changes:
- Add _text_centered_visual() helper to calculate true glyph center
- Update icon() method to use visual centering for accurate positioning
- Switch default font to RobotoMono Nerd Font:bold for consistency
- Convert all icon literals from escape sequences to actual Unicode chars
  for better readability and consistency across codebase

The issue occurred because monospace fonts have fixed-width character
cells where glyphs start at the same left edge regardless of their
actual width. Nerd Font icons (51-59px) extended further right than
regular text (~38px), causing 6-11px horizontal shift. PIL's anchor="mm"
centers based on cell width, not visual bounds.

The fix dynamically calculates glyph bounding boxes and adjusts position
to align the visual center with the target center point, working
correctly with any font type and size.
Allow users to specify a device serial number in the configuration
to connect to a specific Stream Deck when multiple devices are present.
Falls back to first available device when serial_number is None.

- Add serial_number field to DeviceConfig model
- Update connect_device() to filter by serial number
- Add comprehensive tests for serial filtering behavior
- Document new config option in default.cfg
Move PulseAudio and OBS connection management from individual widget
activation to plugin context lifecycle hooks. Connections are now
lazily initialized when the first widget activates and remain active
for the plugin lifetime.

- Add on_widget_activate/on_widget_deactivate hooks to PluginContext
- Implement lazy connection in AudioPluginContext and OBSPluginContext
- Update Deck to call context lifecycle hooks before/after widget methods
- Remove connection calls from individual widget activate methods
- Add comprehensive tests for lifecycle hook behavior
- Update existing tests to reflect new connection management
BREAKING CHANGE: The plugin system architecture has been refactored with new naming conventions:

- `Plugin` class renamed to `PluginDescriptor` - represents plugin metadata and widget declarations
- `PluginContext` class renamed to `Plugin` - represents plugin runtime instances with shared state
- All plugin implementation files renamed from `context.py` to `plugin.py`
- Widget base class now uses `plugin` attribute instead of `context` to access plugin instances
- Type parameters updated: `TContext` → `TPlugin` throughout the codebase

This change clarifies the distinction between plugin descriptors (type containers) and plugin instances (runtime state containers).
Replace the `description` class attribute with automatic extraction from
class docstrings in both PluginDescriptor and Widget base classes. This
streamlines the API by eliminating redundancy between docstrings and
description fields.

Changes:
- Update PluginManager to extract descriptions using inspect.getdoc()
- Remove description attribute from PluginDescriptor and Widget base classes
- Migrate all plugin descriptors and widget implementations to use docstrings
- Add comprehensive tests for attribute extraction and validation
- Update example plugin documentation to reflect current API

BREAKING CHANGE: Plugin descriptors and widgets must use class docstrings
instead of the `description` attribute. The description attribute is no
longer supported.
BREAKING CHANGE: Widget.update() now receives Renderer directly instead of Key and must return UpdateResult

- Move Renderer class from core/key.py to rendering/renderer.py for better module organization
- Remove Key class and its context manager pattern - no longer needed
- Update Widget.update() signature to accept Renderer and return UpdateResult enum
- Add UpdateResult enum (UPDATED/UNCHANGED) to control device updates
- Update Deck.update() to create Renderer, call widgets, and conditionally push based on UpdateResult
- Update all builtin widgets (Text, Timer, Clock) to new interface
- Update all plugin widgets (Example, OBS, Audio) to return UpdateResult
- Update all tests and documentation

This refactoring provides cleaner separation of concerns, with widgets only receiving
rendering capabilities they need, while Deck manages device communication.
Update the default size parameter in Renderer.icon() from 64 to 86 pixels
to match the most commonly used size throughout the codebase. Remove all
explicit size=86 and size=64 parameters from renderer.icon() calls across
widgets and tests, as they now use the new default.
Update test mock to use new update() signature that takes Renderer
parameter and returns UpdateResult, matching the refactoring in
e46e020.
@Luminger Luminger force-pushed the refactor/modernize-project-structure branch from 671a39d to 9a9e4c1 Compare October 13, 2025 08:03
Replace unused image_centered() with simpler image() method matching
icon() behavior. Includes comprehensive test coverage.
Reintroduce split font configuration with default_text_font (Roboto)
and default_icons_font (RobotoMono Nerd Font) for better text
readability on StreamDeck keys while preserving icon support.
…ttings

BREAKING CHANGE: Configuration format changed from custom Python DSL (.cfg) to standard TOML (.toml)

- Replace custom config DSL with pydantic-settings TOML loader
- Add pydantic-settings dependency for TOML configuration support
- Implement TomlConfigSettingsSource with explicit file path handling
- Support environment variable overrides with KNOEPFE_ prefix
- Remove all .cfg files and DSL parsing code (src/knoepfe/config/dsl.py)

Configuration improvements:
- Convert all example configs to TOML format (default.toml, clocks.toml, streaming.toml)
- Add visual separators and section headers for better readability
- Include inline comments explaining configuration options
- Document widget positioning with index parameter

Documentation updates:
- Update README.md with TOML configuration examples
- Add "Widget Positioning" section explaining index parameter usage
- Enhance plugin README files (OBS, Audio) with formatted TOML examples
- Document environment variable support and usage patterns
Replace character-based word wrapping with explicit newline handling for
more predictable and user-controlled text layout. Use font.getmetrics()
for accurate line height calculation.

Changes:
- Rename text_wrapped() to text_multiline()
- Remove textwrap dependency and automatic word wrapping
- Remove max_width parameter (no longer needed)
- Add automatic line spacing (defaults to 20% of font size)
- Use font.getmetrics() for accurate line height (ascent + descent)
- Add early return optimization for empty text
- Split text only on explicit newlines (\n)
- Preserve empty lines from consecutive newlines

Tests:
- Update test to use text_multiline()
- Mock getmetrics() instead of character-based estimation
- Add test for newline preservation
- Add test for multiple consecutive newlines
- Add test for custom line spacing
- Move all widget files to widgets/ subdirectory
- Rename base widget classes for clarity (base.py → {plugin}_widget.py)
- Update imports and tests
- Standardize structure across audio, obs, and example plugins
…et.py

Split actions module for better separation of concerns:
- Move WidgetAction, SwitchDeckAction, WidgetActionType to core/actions.py
  (system-level actions used by deck manager)
- Move UpdateResult into widgets/widget.py alongside Widget class
  (tightly coupled to Widget.update() method)
- Delete widgets/actions.py (now empty)

Rename for consistency with plugins/plugin.py:
- Rename widgets/base.py → widgets/widget.py
- Rename tests/widgets/test_base.py → test_widget.py
- Update all imports across codebase (core, plugins, tests)
- Update module exports in widgets/__init__.py

Benefits:
- Consistent naming: plugins.plugin.Plugin and widgets.widget.Widget
- Clear architectural boundaries between widget API and system actions
- Better semantic organization with UpdateResult alongside Widget

All tests pass (176 tests).
@Luminger Luminger marked this pull request as ready for review October 15, 2025 08:29
…mpatibility

The context manager implementation (__enter__/__exit__) was removed from
CythonHIDAPI.Device as it was:
- Not reentrant-safe (would close device prematurely in nested contexts)
- Not present in upstream LibUSBHIDAPI (causing crashes with --no-cython-hid)
- Unnecessary (devices should remain open for application lifetime)

This fixes a critical bug where using `with device:` in deck.py would fail
when CythonHIDAPI was disabled, as the upstream LibUSBHIDAPI has no context
manager support.

Changes:
- Remove __enter__ and __exit__ from CythonHIDAPI.Device
- Remove `with device:` usage from deck.activate() and deck.update()
- Keep _handle_hid_errors context manager (used for error handling only)
- Rewrite transport/README.md to be concise and focused
- Remove unnecessary upstream patch recommendations (bus_type, missing APIs, RLock)
- Keep only critical shutdown race condition fix for upstream
- Document both patches: Dummy transport fix and CythonHIDAPI replacement

The code now works correctly with both CythonHIDAPI (default) and the
original LibUSBHIDAPI transport implementations.

Fixes: Device context manager incompatibility with upstream transport
@Luminger Luminger force-pushed the refactor/modernize-project-structure branch from 1100fb9 to 7fa2dec Compare October 24, 2025 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants