jeena/jtube-tv
1
0
Fork 0
Code Issues 5 Pull requests Projects Releases Packages Wiki Activity Actions
jtube-tv/TESTS.md
Jeena 0cebe3b710 docs: Remove the test account name from the public repo 
The account name identifies a real, seeded account on the instance;
the public source doesn't need it to make sense. Credentials were
never in the repo and still live only in sops.
2026-10-12 00:33:42 +00:00

7.5 KiB
Raw Permalink Blame History

E2E tests — release gate

How to validate a release. Two layers:

  1. Scripted API smoke test — scripts/e2e-api-check.sh, runs anywhere sops+jtube.env are available (container or any agent machine). Fast feedback on the backend contract.
  2. Manual device test — on an Android TV with a remote (D-pad!). Compose TV focus behavior can only be validated by hand; use the checklist below.

The manual checklist is a hard gate for releases; the script alone is not enough even when everything passes (playback, focus and UI states are untestable by script).

Test environment / prerequisites

  • Instance: any PeerTube (baseline: https://tube.jeena.net, v8.3.1, media on R2 object storage).
  • Account: a dedicated test account (credentials via sops from ~/.agents/secrets/jtube.env). Credentials resolved at login; do not store the password in the device during tests — it's typed once on the setup screen (which is exactly feature 0).
  • Seeded content (tagged jtube-tv-test, in the test account's channel):
    • JTube TV Test 1 - Sintel Teaser Private (~52 s)
    • JTube TV Test 2 - Caminandes Llamigos Private (~2:30)
    • JTube TV Test 3 - Cosmos Laundromat Private (~50 s)
    • the test account has watched entries in server history (public + test videos).
  • Subscription: mediterranean_homestead@lone.earth (remote channel — exercises federation in the subscription feed).
  • Public content: instance's own local videos (≥ 3000).
  • Device: Android TV with D-pad; also sanity-check a phone profile for the touch path.
  • Network conditions if available: Wi-Fi + (later) metered/lossy playback test.

A. Scripted API checks (run before every release)

./scripts/e2e-api-check.sh     # exit 0 = backend contract intact

Covers: login/refresh, local videos, my-videos incl. private rows, own channels, local channel search, search, subscription feed (incl. remote channel), history shape, watching POST → 204 → stored position, stream URL resolution, file-token endpoint, playlist reachability.

B. Manual device checklist

Legend: [T] must pass for TV (D-pad). Pass criteria named per item; log failures with APK version + device.

0 — Credentials onboarding

  • T1 Fresh install: no URL/account stored → setup screen appears with URL prefilled to https://tube.jeena.net.
  • T2 Login with wrong password → app shows "Invalid credentials", user stays on setup, nothing persisted (clear app data → setup shows again). Repeat once to catch double-submit lockout via spinner + disabled button.
  • T3 Login with bad host (x.invalid) → reachable-error message, no crash.
  • T4 Successful login → home screen; force-stop app, relaunch → home screen directly (session persisted, encrypted). Tokens survive (no re-login).
  • T5 Long-run: 24 h later (token expiry ~86 400 s) relaunch → any authorized screen still works without re-login (refresh-token flow ran silently). If this can't be waited out, simulate by adb shell clearing only the access token? — skip; instead validate refresh via the API script (F0) and mark T5 after one real 24 h exposure per release cycle.

1 — Local videos / My videos

  • T6 "Videos" entry: list loads, newest public local videos first, thumbnails visible, D-pad scroll down past 25 rows loads page 2 (spinner at bottom disappears, list grows).
  • T7 "My videos": the 3 seeded Private test videos appear with a privacy badge; non-test public videos of the account also appear if any.
  • T8 Scroll from "Videos" to "My videos" and back → each screen keeps its own list (no cross-contamination of cached pages).
  • T9 Long video title truncates (2 lines max) without breaking the row layout; long channel names truncate.

2 — Channels

  • T10 "Channels" lists the account's own channels, avatar when present, video count.
  • T11 Click channel → that channel's videos list; open and play one; back returns to the channel's list (not the channels screen lost state).
  • T12 Remote channel via direct handle is out of MVP scope, but local-channel search (when implemented) must be re-tested: searchTarget=local ignores federated results.

3 — Search

  • T13 Search screen: field focusable via D-pad; on-screen/IME input works. Search "Sintel" → Test 1 private video appears even though it's private (search endpoint filters locally: verify it does — if private videos are NOT in search results, this is expected PeerTube behavior; adjust expectation: my-videos screen is the place private content is discoverable).
  • T14 Empty search string → no crash, stays on empty state. Gibberish → "Nothing here yet".

4 — Subscriptions

  • T15 Subscriptions feed shows the remote (lone.earth) channel's videos — mixed-feed behavior works; newest first.
  • T16 Feed pagination works beyond 25 items when subscribed to enough channels (skip if < 50 items total; note skip reason).
  • T17 Click remote video → plays or fails gracefully (a remote video's media is on the remote host; MVP may fail to resolve — must show error toast/message + back, never crash/freeze).

5 — History

  • T18 History lists server-side history newest-first. NOTE (verified 2026-10-10): PeerTube's history endpoint OMITS private videos — private test entries will not appear; adjust expectations accordingly (app is fine).
  • T19 Play an entry to ≥ 20 s, leave (back) → entry is present/updated in history when re-opening the screen (server round-trip within ~1 s; app must not show stale cache as fresh — reload on screen entry).

6 — Playback

  • T20 Play a public local video → video+audio start < 5 s, D-pad pauses/plays, seek bar appears via center press, back closes player and lands on the list.
  • T21 Play a private test video → plays. (On tube.jeena.net media is publicly readable today; the app must still send the file token — verify in logs that ?videoFileToken= was appended, since that is what private-bucket instances need.)
  • T22 Long watch: stay in a video > 60 s (watching reports every ~20 s) → server userHistory.currentTime updated; re-entering the video later resumes or at least proves the reports landed (curl check).
  • T23 Switch between two videos (from list, play, back, play another) → no audio ghosting/duplicate players (only one active ExoPlayer; release resources on exit).
  • T24 Airplane-mode / dead WiFi: opening player → error state shown, no crash; back works.
  • T25 Remote-hosted video (subscription item, media on lone.earth) → if it plays it must not break when network switches; at minimum no crash.
  • T26 Video without HLS (progressive-only instance): force by playing an item whose detail has no streamingPlaylists — if none exists on the test instance, note skip.

7 — Robustness / release hygiene

  • T27 App data cleared while logged out → setup screen, no crash loops.
  • T28 Rotate/screen-size changes are irrelevant on TV (landscape only), but app must not crash on phones when sideloaded.
  • T29 Kill server (or wrong URL after instance rename): any authorized screen → consistent error UI + retry; stale-token case must eventually surface "re-login" (authenticator gives up).
  • T30 APK is signed, minified build installs and boots (release build, minifyEnabled once enabled — re-run the whole checklist for it, R8 has bitten before).

Release gate

All [T] items pass on the release build; script exits 0. Skips must be documented with reason. New features added in a release must append their checklist items to section B.