Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,9 @@ src/ui/src/core/pkg/

# Research and documentation files (generated, not part of source)
*.md
!README.md
!README.md

# E2E Test Artifacts
src/ui/playwright-report/
src/ui/test-results/
temp/
261 changes: 261 additions & 0 deletions FRONTEND_TEST_CONCEPT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,261 @@
# Testing Concept: Omnect-UI (Strategy: Core-First)

## Strategy

Leverage the Crux architecture's testability by design. The Core contains all business logic as pure, deterministic functions - making it the highest-ROI test target. The Shell is intentionally thin (renders ViewModel, executes effects) and needs minimal testing.

**Approach:** Test the Core exhaustively (cheap, fast, deterministic), keep E2E minimal for regression safety.

**Phase 1 Status:** ✅ **Complete** (92 tests across 4 PRs)
- PR #77: Authentication Tests (17 tests)
- PR #78: Device Tests (27 tests)
- PR #79: Network Tests (21 tests)
- PR #80: Reconnection Tests (27 tests)

## Implementation Plan

### Phase 1: Core State Transitions (Unit Tests)

*Goal: Secure business logic and state machines with fast, deterministic tests.*

#### PR 1.1: Authentication Tests ✅
- [x] Test login flow (loading state, success, failure)
- [x] Test logout and session cleanup
- [x] Test token management state
- [x] Test password change flow

#### PR 1.2: Device Tests ✅
- [x] Test system info updates (WebSocket events)
- [x] Test online status transitions
- [x] Test factory reset state machine
- [x] Test reboot flows
- [x] Test firmware upload state transitions

#### PR 1.3: Network Tests ✅
- [x] Test network configuration updates
- [x] Test IP change detection and rollback state
- [x] Test DHCP/static switching logic
- [x] Test network form state management

#### PR 1.4: Update/Reconnection Tests ✅
- [x] Test reconnection state machine for all operations (reboot, factory reset, update)
- [x] Test reconnection timeout handling with operation-specific durations
- [x] Test update completion detection based on validation status
- [x] Test healthcheck response handling during reconnection
- [x] Test network IP reachability detection

**Note:** Original PR 1.5 (WebSocket Tests) was merged into PR 1.2 as WebSocket event handling tests are naturally colocated with device state updates.

### Phase 2: Core Effect Emissions

**Status:** 🚫 **Skipped - Not Recommended**

After implementing Phase 1, we've determined that effect emission testing provides minimal value:

**Why Skip Effect Testing:**

1. **Implementation Detail Testing**: Effects are how the Core communicates with the Shell, not what it does. Testing effect structure couples tests to implementation details.

2. **Macros Handle Correctness**: The codebase uses well-tested macros (`auth_post!`, `http_get!`, `http_get_silent!`) that generate effects consistently. These macros are the single source of truth for effect creation.

3. **Auto-Generated Types**: The `Effect` enum is auto-generated via `#[derive(crux_core::macros::Effect)]`. Testing against generated types is brittle and adds maintenance burden.

4. **Response Testing is Sufficient**: Phase 1 already tests response handling (e.g., `LoginResponse`, `SetNetworkConfigResponse`), which validates the complete request/response cycle behavior from the user's perspective.

5. **Integration Coverage**: E2E tests (Phase 3) will validate actual HTTP requests reach the backend correctly.

**What We Test Instead:**
- ✅ State transitions (Phase 1) - validates business logic
- ✅ Response handling (Phase 1) - validates correct reactions to success/error
- ✅ Critical paths (Phase 3) - validates actual network communication

**Original Phase 2 Tasks** (archived for reference):
- ~~Test login emits correct POST request~~
- ~~Test authenticated requests include bearer token~~
- ~~Test network config changes emit correct payloads~~
- ~~Test Centrifugo connection/subscription effects~~

### Phase 3: E2E Regression Tests (Selective)

*Goal: Guard critical user journeys against regression. Keep minimal. Run in a standardized Docker environment.*

**Environment:** `omnectshareddevacr.azurecr.io/rust:bookworm` (includes Rust, Node, Playwright, Centrifugo).

#### PR 3.1: E2E Infrastructure & Docker Integration ✅
- [x] **Dependencies:** Add `@playwright/test` to `src/ui/package.json`
- [x] **Config:** Add `src/ui/playwright.config.ts` (Base URL, artifacts, projects)
- [x] **Script 1 (`scripts/run-e2e-tests.sh`):** Internal script to run tests inside the container
- [x] **Script 2 (`scripts/test-e2e-in-container.sh`):** Host script to:
- Launch the Docker container
- Build frontend (`scripts/build-frontend.sh`) if needed
- Execute `scripts/run-e2e-tests.sh`
- [x] **Centrifugo Setup:** Ensure tests interact with the real Centrifugo instance (e.g., via `curl` to publish) rather than mocking

#### PR 3.2: Critical Path Tests ✅
- [x] **Auth Flow:**
- Mock HTTP responses for Login/Logout
- Verify Login form → Dashboard transition
- Verify Logout action → Login form
- [x] **Device Read-Only:**
- Inject System Info via real Centrifugo instance (publish via `curl`)
- Assert values appear on the Dashboard
- [x] **Network Safety UI:**
- Simulate network change action
- Assert "Rollback Timer" overlay appears

#### PR 3.3: CI Pipeline Integration
- [ ] Create GitHub Actions workflow
- [ ] Execute `scripts/test-e2e-in-container.sh` in CI

## Test Patterns

### State Transition Test
```rust
#[test]
fn test_login_sets_loading() {
let app = AppTester::<App>::default();
let mut model = Model::default();

app.update(Event::Login { password: "test".into() }, &mut model);

assert!(model.is_loading);
assert!(model.error_message.is_none());
}
```

### Effect Emission Test (Not Recommended - See Phase 2)
```rust
// ❌ NOT RECOMMENDED: Testing implementation details
// Effects are auto-generated and handled by macros
// This test is brittle and provides minimal value

#[test]
fn test_login_emits_http_request() {
let app = AppTester::<App>::default();
let mut model = Model::default();

let effects = app.update(Event::Login { password: "test".into() }, &mut model);

// This tests HOW the Core communicates, not WHAT it does
// Better to test state transitions and response handling instead
}
```

### Response Handling Test (✅ Recommended Pattern)
```rust
// ✅ RECOMMENDED: Test response handling and state changes
// This validates WHAT the Core does from the user's perspective

#[test]
fn test_login_success_sets_authenticated() {
let app = AppTester::<App>::default();
let mut model = Model {
is_loading: true,
..Default::default()
};

let _ = app.update(
Event::Auth(AuthEvent::LoginResponse(Ok(AuthToken {
token: "test_token_123".into(),
}))),
&mut model,
);

assert!(model.is_authenticated);
assert!(!model.is_loading);
assert_eq!(model.auth_token, Some("test_token_123".into()));
}

#[test]
fn test_login_failure_sets_error() {
let app = AppTester::<App>::default();
let mut model = Model {
is_loading: true,
..Default::default()
};

let _ = app.update(
Event::Auth(AuthEvent::LoginResponse(Err("Invalid password".into()))),
&mut model,
);

assert!(!model.is_authenticated);
assert!(!model.is_loading);
assert!(model.error_message.is_some());
}
```

### Colocated Test Pattern (✅ Used in Phase 1)
```rust
// Tests are colocated with the code they test using #[cfg(test)] mod tests
// Example: src/app/src/update/auth.rs

#[cfg(test)]
mod tests {
use super::*;
use crate::events::{AuthEvent, Event};
use crate::model::Model;
use crate::types::AuthToken;
use crate::App;
use crux_core::testing::AppTester;

mod login {
use super::*;

#[test]
fn success_sets_authenticated_and_stores_token() {
let app = AppTester::<App>::default();
let mut model = Model {
is_loading: true,
..Default::default()
};

let _ = app.update(
Event::Auth(AuthEvent::LoginResponse(Ok(AuthToken {
token: "test_token_123".into(),
}))),
&mut model,
);

assert!(model.is_authenticated);
assert!(!model.is_loading);
assert_eq!(model.auth_token, Some("test_token_123".into()));
}
}
}
```

## Tools

| Scope | Tool | Purpose |
|:------|:-----|:--------|
| **Core Logic** | `cargo test` + `crux_core::testing` | State transitions, effect emissions |
| **E2E** | Playwright | Critical user journey regression |

## ROI Summary

| Phase | Speed | Stability | Coverage | Priority | Status |
|:------|:------|:----------|:---------|:---------|:-------|
| Core State Tests | Fast (ms) | Deterministic | High | **High** | ✅ **Complete (92 tests)** |
| ~~Core Effect Tests~~ | ~~Fast (ms)~~ | ~~Deterministic~~ | ~~High~~ | **Skipped** | 🚫 **Not recommended** |
| E2E Tests | Slow (s) | Flaky-prone | Low | Low | ⏳ **Planned (Phase 3)** |

## Lessons Learned

### What Worked Well
1. **Colocated Tests**: Keeping tests next to the code they test (`#[cfg(test)] mod tests`) improves maintainability
2. **Domain Organization**: Organizing tests by domain (auth, device, network) mirrors code structure
3. **Response-Focused Testing**: Testing response handling validates behavior without coupling to implementation
4. **State Machine Validation**: Comprehensive state transition testing catches edge cases early

### What to Avoid
1. **Effect Emission Testing**: Testing auto-generated effect structures is brittle and low-value
2. **Testing Macros**: Well-tested macros (`auth_post!`) don't need per-use validation
3. **Testing Request Events**: Events that trigger HTTP requests don't have immediate state changes to test

### Key Patterns
- Use `let _ = app.update(...)` to ignore unused `Update<Effect, Event>` results
- Test response events (e.g., `LoginResponse`) not request events (e.g., `Login`)
- Organize tests in nested modules matching code structure
- Use helper functions to create test data (e.g., `create_healthcheck()`)
- Test state transitions, not implementation details
102 changes: 102 additions & 0 deletions scripts/run-e2e-tests.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
#!/bin/bash
set -e

# Internal script to run E2E tests inside the container

echo "🔧 Setting up test environment..."

# 0. Ensure bun is installed (needed for UI)
if ! command -v bun &> /dev/null; then
echo "⚠️ Bun not found, installing..."
curl -fsSL https://bun.sh/install | bash
export BUN_INSTALL="$HOME/.bun"
export PATH="$BUN_INSTALL/bin:$PATH"
fi

# 1. Ensure Centrifugo is available (using the tool script if needed)
if ! command -v centrifugo &> /dev/null; then
echo "⚠️ Centrifugo not found in PATH, checking tools directory..."
if [ ! -f "tools/centrifugo" ]; then
./tools/setup-centrifugo.sh
fi
export PATH=$PATH:$(pwd)/tools
fi

# 2. Start Centrifugo directly (Backend is mocked, but we need real WS)
echo "🚀 Starting Centrifugo..."
# Using the config from backend/config/centrifugo_config.json
CENTRIFUGO_CONFIG="src/backend/config/centrifugo_config.json"

# Generate self-signed certs for testing if missing
mkdir -p temp/certs
if [ ! -f "temp/certs/server.cert.pem" ]; then
openssl req -newkey rsa:2048 -nodes -keyout temp/certs/server.key.pem -x509 -days 365 -out temp/certs/server.cert.pem -subj "/CN=localhost" 2>/dev/null
fi

# Env vars for Centrifugo
export CENTRIFUGO_HTTP_SERVER_TLS_CERT_PEM="temp/certs/server.cert.pem"
export CENTRIFUGO_HTTP_SERVER_TLS_KEY_PEM="temp/certs/server.key.pem"
export CENTRIFUGO_HTTP_SERVER_PORT="8000"
export CENTRIFUGO_CLIENT_TOKEN_HMAC_SECRET_KEY="secret"
export CENTRIFUGO_HTTP_API_KEY="api_key"
export CENTRIFUGO_LOG_LEVEL="info"

centrifugo -c "$CENTRIFUGO_CONFIG" > /tmp/centrifugo.log 2>&1 &
CENTRIFUGO_PID=$!

echo "⏳ Waiting for Centrifugo..."
for i in {1..30}; do
if curl -k -s https://localhost:8000/health > /dev/null; then
echo "✅ Centrifugo is ready!"
break
fi
if [ $i -eq 30 ]; then
echo "❌ Centrifugo failed to start."
cat /tmp/centrifugo.log
kill $CENTRIFUGO_PID || true
exit 1
fi
sleep 1
done

# 3. Serve the Frontend
echo "🌐 Starting Frontend Dev Server..."
cd src/ui
# Install dependencies if needed (container might not have node_modules)
if [ ! -d "node_modules" ]; then
echo "📦 Installing UI dependencies..."
bun install
fi

# Start vite dev server in background
bun run dev --port 5173 > /tmp/vite.log 2>&1 &
FRONTEND_PID=$!

# Wait for Frontend
echo "⏳ Waiting for Frontend..."
for i in {1..30}; do
if curl -s http://localhost:5173 > /dev/null; then
echo "✅ Frontend is ready!"
break
fi
if [ $i -eq 30 ]; then
echo "❌ Frontend failed to start."
cat /tmp/vite.log
kill $FRONTEND_PID || true
kill $CENTRIFUGO_PID || true
exit 1
fi
sleep 1
done

# 4. Run Playwright Tests
echo "🧪 Running Playwright Tests..."
# BASE_URL is set for playwright.config.ts
export BASE_URL="http://localhost:5173"

# Run tests
npx playwright test "$@"

TEST_EXIT_CODE=$?

exit $TEST_EXIT_CODE
Loading
Loading