feat: add Flutter native player client with remote access infrastructure #59
+106,801
−38,897
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Pull request overview
This PR adds a comprehensive Flutter-based player client with native support for iOS, macOS, and web, along with remote access infrastructure for secure device pairing and media streaming. The changes introduce GraphQL API endpoints, collections system, offline downloads with transcoding, and HLS streaming capabilities.
Key changes:
- GraphQL schema and API with authentication, media browsing, search, and playback progress tracking
- Remote access infrastructure with relay tunneling, Noise protocol encryption, and QR-based device pairing
- Collections system (favorites, smart collections) with CRUD operations
Reviewed changes
Copilot reviewed 117 out of 537 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| lib/mydia_web/schema/enum_types.ex | GraphQL enum definitions for media types, library types, sort options, and media categories |
| lib/mydia_web/schema/common_types.ex | GraphQL object types for artwork, media files, progress, search results, authentication, and devices |
| lib/mydia_web/schema.ex | Main GraphQL schema definition importing all type modules and defining query/mutation/subscription fields |
| lib/mydia_web/router.ex | Router updates adding GraphQL endpoints, player routes, collections routes, and API v2 endpoints |
| lib/mydia_web/plugs/media_auth.ex | Authentication plug for media requests using JWT tokens with permission validation |
| lib/mydia_web/plugs/api_auth.ex | API authentication with rate limiting for failed API key attempts |
| lib/mydia_web/plugs/absinthe_context.ex | Context builder for GraphQL requests extracting current user from Guardian |
| lib/mydia_web/live/media_live/show/search_helpers.ex | Enhanced search result sorting with title relevance scoring based on query matching |
| lib/mydia_web/live/media_live/show/components.ex | UI components for favorites and collections with dropdown menu |
| lib/mydia_web/live/media_live/show.html.heex | Template updates passing collection data to components |
| lib/mydia_web/live/media_live/show.ex | Event handlers for favorites and collections operations |
| lib/mydia_web/live/media_live/index.html.heex | Batch operations modal for adding selected items to collections |
| lib/mydia_web/live/media_live/index.ex | Event handlers for batch collection operations |
| lib/mydia_web/live/collection_live/show.ex | LiveView for viewing and managing individual collections |
| lib/mydia_web/live/collection_live/index.ex | LiveView for browsing and managing all collections |
| lib/mydia_web/live/admin_import_lists_live/index.html.heex | UI for selecting target collection for import lists |
| lib/mydia_web/live/admin_import_lists_live/index.ex | Loading manual collections for import list target selection |
| lib/mydia_web/live/admin_devices_live/index.html.heex | Device management UI for viewing and revoking paired devices |
| lib/mydia_web/live/admin_devices_live/index.ex | Admin device management LiveView implementation |
| lib/mydia_web/live/admin_config_live/remote_access_component.ex | Remote access configuration component with device pairing and management |
| lib/mydia_web/live/admin_config_live/index.html.heex | Remote access tab in admin configuration |
| lib/mydia_web/live/admin_config_live/index.ex | Remote access component message handlers |
| lib/mydia_web/live/admin_config_live/components.ex | Simplified media servers tab UI |
| lib/mydia_web/live/activity_live/index.html.heex | Search event details in activity log |
| lib/mydia_web/live/activity_live/index.ex | Event description formatting for search activities |
| lib/mydia_web/endpoint.ex | WebSocket endpoints for GraphQL subscriptions and device connections |
| lib/mydia_web/controllers/session_controller.ex | Guardian token storage for session authentication |
| lib/mydia_web/controllers/player_controller.ex | Flutter web player serving with auth configuration injection |
| lib/mydia_web/controllers/auth_controller.ex | Guardian token storage for authentication |
| lib/mydia_web/controllers/api/v2/subtitle_controller.ex | REST API for listing and streaming subtitle tracks |
| lib/mydia_web/controllers/api/v2/search_controller.ex | REST API for full-text media search |
| lib/mydia_web/controllers/api/v2/discover_controller.ex | REST API for content discovery (continue watching, recent, up next) |
| lib/mydia_web/controllers/api/v2/browse_controller.ex | REST API for browsing movies and TV shows with pagination |
| lib/mydia_web/controllers/api/thumbnail_controller.ex | Serving video thumbnail sprite sheets and VTT files |
| lib/mydia_web/controllers/api/hls_controller.ex | HLS streaming with readiness check for playlist availability |
| lib/mydia_web/controllers/api/download_controller.ex | Download API with quality selection and progressive download support |
| lib/mydia_web/components/layouts.ex | Player navigation button in sidebar and mobile nav |
| lib/mydia_web/components/collection_components.ex | Reusable UI components for collection views |
| lib/mydia_web/channels/user_socket.ex | WebSocket setup for device connections |
| lib/mydia_web/channels/device_channel.ex | Phoenix Channel for device pairing and reconnection using Noise protocol |
| lib/mydia_web.ex | Collection components import and player static path |
| lib/mydia/subtitles/extractor.ex | Subtitle extraction from media files using FFprobe and FFmpeg |
| lib/mydia/streaming/hls_session.ex | HLS session with readiness notification for playlist availability |
| lib/mydia/streaming/hls_pipeline.ex | Removed Membrane Framework backend (FFmpeg-only approach) |
| lib/mydia/streaming/ffmpeg_hls_transcoder.ex | FFmpeg transcoder with ready notification when playlist is available |
| lib/mydia/streaming/HLS_BACKENDS.md | Removed backend comparison documentation |
| lib/mydia/settings.ex | Library paths query excluding disabled paths |
| lib/mydia/remote_access/remote_device.ex | Schema for paired client devices with revocation support |
| infra/kubernetes/apps/metadata-relay/service.yaml | Kubernetes service definition for metadata relay |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
arsfeld
force-pushed
the
feature/native-player-client
branch
from
f194b30 to
3ea7df0
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 120 out of 512 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
lib/mydia_web/live/admin_config_live/index.ex:1
- Debug event handlers should be removed before merging to production. These test and catch-all handlers appear to be temporary debugging code.
defmodule MydiaWeb.AdminConfigLive.Index do
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
1167
to
1188
| defp check_port_accessible(ip, port) do | ||
| url = "https://portchecker.io/api/v1/query" |
There was a problem hiding this comment.
The port checker makes an external HTTP request to a third-party service (portchecker.io) which could fail or timeout. Consider adding retry logic or caching the result to avoid repeated external calls, and document the dependency on this external service.
lib/mydia/streaming/hls_session.ex
Outdated
|
|
||
| # Reply to all waiters | ||
| Enum.each(state.ready_waiters, fn from -> | ||
| GenServer.reply(from, :ok) |
There was a problem hiding this comment.
If a waiter process has already timed out or terminated, GenServer.reply could fail. Wrap the reply in a try-catch or use a safer mechanism to avoid potential crashes when replying to dead processes.
Suggested change
| GenServer.reply(from, :ok) | |
| try do | |
| GenServer.reply(from, :ok) | |
| rescue | |
| _ -> :ok | |
| end |
Complete implementation of cross-platform native player: Backend (Phoenix/Elixir): - GraphQL API with Absinthe for content browsing - Media authentication and token system - GraphQL subscriptions for real-time progress sync - Device channel for WebSocket communication - Remote access relay with Noise protocol encryption - Device pairing flow with claim codes and rate limiting - Subtitle extraction support - Flutter dev proxy for hot reload Flutter Client: - Login, home, library, and detail screens - Video player with HLS and native codec support - Audio/subtitle track selection - Playback progress sync via GraphQL - Seek preview thumbnails - Offline downloads support - Chromecast (Android) and AirPlay (iOS) casting - iOS background audio and Bluetooth support - Settings screen with theme/quality options - Platform-specific conditional compilation for web/native - Web build integrated at /player route Remote Access: - Secure remote connections via relay service - Device pairing with claim codes - Remote Access tab in Configuration page - Connected devices management CI/CD: - GitHub Actions workflow for multi-platform builds - Web build integrated into Docker image - Native platform workflows ready (iOS/Android/desktop)
- Move Flutter player from clients/player/ to player/ - Add direct URL access for remote streaming with certificate pinning - Implement API key rate limiting and enhanced authentication - Add GraphQL resolvers for API keys, auth, and device management - Update metadata-relay with relay connection infrastructure - Add migrations for API key fields and remote access config changes
- Add streaming decision service with codec detection - Implement progressive download with quality selection - Add Noise_NK handshake and connection management - Add server-side MP4 transcoding system for downloads - Add HLS session ready notification - Improve metadata-relay security and rate limiting - Add Terraform/Kubernetes infrastructure configs - Improve player UI screens and download integration - Update Flutter web player build
Add macOS platform configuration for running the Flutter player as a native desktop app with a modern, title-bar-less window appearance.
Add create_claim function to register claim codes with the relay service, allowing remote clients to discover and connect to mydia instances. Improve relay socket connection handling with separate registration message flow and better heartbeat management. Add claim code normalization to handle codes with or without hyphens. Also includes updated Flutter player static assets.
Instead of Mydia generating claim codes locally and registering them with the relay, the relay now generates codes and returns them to Mydia. This fixes the code format mismatch issue where Mydia generated codes with dashes (e.g., "ABCD-1234") but the relay expected 6-8 alphanumeric characters without dashes. Changes: - Add request_claim/3 to Relay module that requests a code from relay - Add changeset_with_code/2 to PairingClaim for relay-generated codes - Update RemoteAccess.generate_claim_code to request from relay first - Add start_relay/0 and stop_relay/0 for dynamic relay management - Track actual WebSocket connection state (connected field in state) - Update UI to show proper connection states and error messages - Update metadata-relay to generate codes instead of accepting them
Attempts to fix the relay process registration issue where Process.whereis returns nil even though the relay is running. Changes: - Add Process.whereis checks before starting relay to prevent duplicates - Add explicit child_spec to Relay module for DynamicSupervisor - Add whereis check inside start_link to handle race conditions - Clean up debug logging Note: Issue still not fully resolved - see task-19 for investigation
Separates the relay connection into a stable GenServer (Relay) and a managed WebSockex process (Relay.WebSocket). This prevents name registration conflicts during hot-reload and provides more reliable connection state management. Key changes: - GenServer handles all public API calls and state management - WebSocket process runs without name registration, managed by GenServer - Pending claim requests are tracked in state for proper reply handling - Automatic reconnection with exponential backoff handled by GenServer
Enable cross-origin requests for standalone player and native apps: - Add corsica for CORS handling with permissive settings - Move GraphQL authentication to resolver level - Allow unauthenticated access to GraphQL endpoint (resolvers enforce auth)
Remote Access: - Consolidate settings into admin config (remove separate LiveView) - Add public_port configuration for external access - Improve direct URL generation with public IP/port support - Add GraphQL mutation for claim code generation Relay Service: - Add public IP tracking for instances - Add Redis support for Kubernetes deployment - Improve connection handling and claim code flow - Add database migration for public IP storage Player Client: - Improve reconnection service with better error handling - Enhance relay tunnel service connection management - Add connection status tracking in GraphQL provider - Update pairing service for new claim flow Infrastructure: - Add E2E testing setup (Dockerfiles, compose, GitHub workflow) - Add Redis configuration for metadata-relay k8s deployment - Update ingress configuration for relay service Cleanup: - Remove deprecated HLS pipeline implementation - Remove unused remote access settings LiveView and tests
- Set iOS deployment target to 14.0 (required by flutter_chrome_cast) - Add AirPlayChannelHandler.swift to Xcode project build sources - Add Podfile.lock for reproducible builds - Configure CocoaPods workspace integration
Collections feature: - Add new collections context with manual and smart collections - Smart collections support rule-based filtering (genre, year, etc.) - Migrate favorites to collection system - Add target_collection_id to import lists for auto-organization - Add collection management UI (create, edit, add items) - Integrate collections into media index and show pages Player pairing improvements: - Add QR code parsing for self-hosted relay pairing - Enhance NoiseService with proper handshake flow - Add QrPairingData class for structured QR content - Improve login screen with scan functionality - Update pairing flow integration tests Other changes: - Consolidate remote access component settings - Update compose.player-e2e.yml for testing improvements - Bump dependencies
- Add title relevance bonus to search result ranking that boosts results matching the exact search query terms (up to 20 points added to quality score) - Restructure Dockerfile to use separate Flutter build stage for faster builds and better caching - Add release ranker tests for parsing and scoring torrent titles - Add search helpers tests for title relevance scoring - Apply code formatting fixes across multiple LiveView files
Add BuildKit cache mounts to all Dockerfiles for faster incremental builds: - Enable Docker buildx bake in CI workflow with GHA caching - Add cache mounts for Hex, Mix, pub-cache, deps, and _build - Reduce healthcheck intervals for faster service startup detection - Add `./dev player e2e-build` command for pre-building E2E images - Remove unused membrane and related dependencies from mix.lock
Add comprehensive search event tracking to improve observability: - New event types: search.started, search.completed, search.no_results, search.filtered_out, search.error - Helper functions in Events module for logging search outcomes - Integrate search events into MovieSearch and TVShowSearch jobs - Add "Search" category filter to activity view - Show expandable search details (query, results count, score breakdown, filter stats, selected release) in activity feed - Add tests for all search event helper functions
The compiled Flutter web output (main.dart.js, canvaskit, etc.) was accidentally committed, adding ~340k lines to the PR. These build artifacts should be generated by CI/CD, not tracked in git. Removes 40MB of build output and adds priv/static/player/ to .gitignore.
Migrate from metadata-relay.arsfeld.dev to the new relay.mydia.dev domain.
- Remove unused compose.e2e.yml and docker-compose.postgres.yml - Simplify compose.player-e2e.yml configuration - Add debug logging to RelayService for troubleshooting - Update player Dockerfile.test and integration tests - Fix e2e docs to include --profile e2e flag
Allow using HTTP instead of HTTPS for sslip.io URLs during development/testing.
Move initialization code inside runZonedGuarded to catch async errors during startup. Fix QR scanner overlay z-order in login screen. Add camera permission and mobile_scanner pod for QR code scanning. Configure automatic code signing for iOS builds.
Adds ref.mounted checks before state updates in login controller async methods to prevent errors when widget is disposed during async operations.
The tests were failing with DBConnection.OwnershipError because they accessed the database without checking out a connection from the sandbox.
The LibraryPath schema doesn't have a :name field, causing a compilation error in CI.
- Fix WebSocket path from /socket to /ws to match Phoenix endpoint - Add web platform guard for CastService to prevent Platform._operatingSystem error - Increase E2E test timeout to 120s for Flutter web startup - Improve error detection in tests to log but continue waiting - Add debug logging in auth flow for troubleshooting
Improve the paired devices UI with a compact card grid layout, dropdown menus for actions, and a "clear inactive" bulk action. Add collapsible device list for installations with many devices. Also removes unused variables from flutter_watcher and cleans up player flake.nix.
- Update Video widget API: use SizedBox.expand with fill color - Add mimalloc linker flags to Linux CMake config for memory leak prevention - Remove deprecated volume_controller plugin (no longer a dependency) - Update pubspec.lock with new transitive dependencies
- Fix release_test.exs to preserve Repo config when overriding database path - Fix stringify_keys to handle structs (ScoreBreakdown) in search jobs - Update loader_test.exs expectation for optional legacy paths (tv_path=nil) - Tag relay-dependent tests with @moduletag :requires_relay - Exclude :requires_ffmpeg and :requires_relay tests by default - Increase token expiration test sleep for reliability
Add checks to prevent showing resume dialog and calculating progress when video duration is not yet loaded (zero). Clamps progress values to valid ranges to avoid invalid UI states.
Generated Flutter tooling files that should not be tracked.
Add multi-layer protocol version negotiation to ensure client-server compatibility. Each component (Mydia server, metadata-relay, player) now advertises supported versions during handshake. If versions are incompatible, clients receive an update_required error with details about which layers failed negotiation and an optional update URL. Protocol layers: - relay_protocol: Relay WebSocket message format - encryption_protocol: E2E encryption scheme - pairing_protocol: Device pairing handshake - api_protocol: Tunneled API request/response format This enables graceful handling of breaking protocol changes and guides users to update their app when needed.
Custom video controls with glass morphism design, replacing the default MaterialVideoControls. Includes modular components for center play button, progress bar, bottom controls bar, and glass container styling.
- Add explicit ShowDetail type annotations to show detail screen methods - Use Wrap instead of Row for episode metadata to handle overflow - Add ProtocolVersion.all getter for protocol negotiation - Add url_launcher_ios to iOS dependencies
Replace inline pairing UI with compact trigger card that opens a modal. The modal provides a cleaner experience with QR code, code entry field, and an animated radial countdown timer.
Publishes claim consumption events via PubSub when a device successfully pairs, allowing the admin UI to automatically close the pairing modal and refresh the device list.
…ment - Adds offline mode when server is unreachable, allowing access to downloaded content without a connection - Implements download queue with configurable concurrent download limits (downloads are queued and auto-start when slots become available) - Adds storage quota management with warning thresholds and automatic cleanup policies (oldest-first or least-recently-used) - Integrates background_downloader package for iOS/Android to continue downloads when app is backgrounded - Shows offline banner with retry connection option - Disables non-download navigation items when offline
Adds Android platform configuration for the Flutter app and updates project metadata and dependency lockfile.
Move async storage reads before the state check to handle the case where setRelayTunnel runs during the async gap and would be overwritten.
Add lifecycle listener to detect when the app resumes from background and automatically reconnect the relay tunnel if it died. Only active on native platforms (not web).
The setup_runtime_config function was setting a plain map instead of the proper Mydia.Config.Schema struct. This caused the runtime config to lack the :auth key, which caused player_controller_test.exs to fail with a KeyError when accessing config.auth.local_enabled. This fixes the CI failure in the end-to-end login test.
Move do_register/5 helper function after all handle_message/2 clauses to fix Elixir compiler warning about clauses with same name and arity not being grouped together.
Cache ~/.pub-cache and player/.dart_tool directories to speed up subsequent builds by avoiding repeated package downloads and leveraging build_runner incremental compilation cache.
Add unique scope=metadata-relay to GitHub Actions cache configuration to prevent cache contamination between metadata-relay and other Docker builds. This fixes the build failure where MetadataRelay.MixProject was picking up stale cached hex/mix data from other builds.
The relay container runs as non-root user 'relay' but the /data directory (where SQLite database is stored) wasn't being created in the Dockerfile. When Docker mounts the volume, it creates the directory owned by root, causing database_open_failed errors. Creates /data with proper relay:relay ownership before switching to the non-root user.
- flutter-player-release.yml → player-release.yml - metadata-relay-ci.yml → relay-ci.yml - metadata-relay-release.yml → relay-release.yml Update workflow names and job names to match shorter naming convention. Update path references in relay-ci.yml to point to renamed file.
Refactor direct URL configuration to support both HTTP (primary) and HTTPS (secondary) URLs for client connections. This provides flexibility for different deployment scenarios: - HTTP URLs for reverse proxy setups - HTTPS URLs for direct secure access Replace single external_port with separate http_port and https_port config. Add public_https_port for NAT scenarios with different public HTTPS port. Enable HTTPS endpoint in development for consistent behavior with production. Remove auto-generated Dart mock file that shouldn't be tracked.
The Linux CMake build was failing when native_assets/linux/ directory doesn't exist. This happens when no packages provide native assets via build.dart. The fix wraps the install in a conditional check.
Rebrand app from generic "player" to "Mydia" across all platforms: - Update display names and window titles - Change bundle IDs from com.example.player to dev.mydia.player - Update CI artifacts to use Mydia.app naming Add app store deployment infrastructure: - Android: signing config, Play Store Fastlane, AAB builds - iOS: certificate/profile handling, TestFlight Fastlane - Update release workflow with store upload steps
New features: - Add global search screen with movie/show results - Improve episode detail screen with full metadata and playback - Add connection diagnostics panel in settings - Add relay mode indicator (glowing dot in sidebar/header) - Add library search filtering UX improvements: - Use push navigation for detail screens (preserves back stack) - Remove custom page transitions (use platform defaults) - Switch GraphQL to cacheAndNetwork for better responsiveness - Fix download options provider dependency ordering
Relay tunnel: - Reorganize handle_tunnel_message/3 clauses for better grouping - Move catch-all clause before helper functions as required by Elixir Discovery resolver: - Fix episode type to return :episode instead of :tv_show - Always include show artwork (poster/backdrop) in episode results
- Update DirectUrls tests to use 3-arity build_sslip_url with explicit scheme - Update detect_local_urls test to accept HTTP or HTTPS URLs (HTTP is now default) - Update detect_public_url tests to match new behavior (:detection_failed error) - Fix DirectUrlRefresher test to accept HTTP URLs from auto-detection - Fix MediaImport test to expect :client_error when client is unreachable The direct URLs module now generates HTTP URLs by default (for reverse proxy setups) with optional HTTPS support when explicitly configured.
Replace Phoenix framework logo with custom Mydia logo featuring: - Blue primary color (#3b82f6) matching brand theme - Apple-style rounded corners (squircle) - Dark slate center with stylized M letterform - Maskable version with safe zone padding for PWA icons Regenerate all PNG icons at 192px and 512px sizes.
Replace generic hero-film icon with the custom Mydia logo SVG.
- Ignore Phoenix digest artifacts (fingerprinted/gzipped files) - Ignore .DS_Store files - Remove accidentally committed digest files - Add fastlane Gemfile.lock files for reproducible builds
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Key Components
Player Client (
player/)Server Infrastructure
Metadata Relay Service
Test plan
./dev mix test