test: Add e2e release gate (script + manual checklist)

Releases need more than eyeballs: scripts/e2e-api-check.sh now runs
18 assertions over the live instance (login/refresh, listings, private
rows, subscriptions incl. remote channels, watching + stored position,
file token, playlist reachability) and TESTS.md defines the manual
D-pad checkout that a script cannot do.

Also switch the channels screen to the user's own channels (the
search-based local listing rejects empty queries) and resolve the real
account name at login since logins accept email addresses.
This commit is contained in:
Jeena 2026-10-10 11:02:22 +00:00
parent 764fa8b34c
commit 83dde8ba74
10 changed files with 284 additions and 8 deletions

View file

@ -65,7 +65,13 @@ setup/login screen, `Ready` renders `HomeScreen`, which switches content on a
- `GET /api/v1/videos/{shortUuid}` for playback URLs: prefer
`streamingPlaylists[].playlistUrl` (HLS) else `files[].fileUrl`.
- Private/internal media: attach `Authorization: Bearer` to playlist, segment and file
requests (configured on the Media3 `HttpDataSource.Factory`).
requests (configured on the Media3 `HttpDataSource.Factory`) AND — for instances
whose media lives in a private object-storage bucket — pass the short-lived
`POST /api/v1/videos/{shortUuid}/token` token as `?videoFileToken=` on those URLs
(response shape: `{ files: { token, expires } }`; verified live).
- tube.jeena.net itself streams non-public media from a **publicly readable R2
bucket** (`pub-….r2.dev`): no auth is enforced there today, but do not rely on
that — always request the file token for non-public videos.
## Conventions

View file

@ -23,8 +23,12 @@ Status vocabulary: `todo` · `doing` · `done`.
`GET videos/{shortUuid}?userHistory=true` (verified working, returns
`userHistory.currentTime`), next/previous video from list, focus current item in
history screen.
- Thumbnails of private/internal videos may 404 — probe server behavior and add a
fallback (generic poster) when required.
- Thumbnails of private/internal videos may 404 — verified on tube.jeena.net:
instance thumbnail paths are publicly readable (HTTP 200 without auth); add a
fallback poster only if other instances behave differently.
- Security note for tube.jeena.net: object-storage (R2) media for private test
videos is publicly readable without token ("private" only via URL obscurity).
Flag to the instance admin; the app must not rely on this laxness.
- Verify `searchTarget=local` channel search behaves well for instances without
federation; consider `search/video-channels` endpoint as an alternative.
- Release build: enable `minifyEnabled` + R8 rules, decide on signing, versioning scheme.

95
TESTS.md Normal file
View file

@ -0,0 +1,95 @@
# E2E tests — release gate
How to validate a release. Two layers:
1. **Scripted API smoke test** — `scripts/e2e-api-check.sh`, runs anywhere sops+jtube.env
are available (container or any agent machine). Fast feedback on the backend contract.
2. **Manual device test** — on an Android TV with a remote (D-pad!). Compose TV focus
behavior can only be validated by hand; use the checklist below.
The manual checklist is a hard gate for releases; the script alone is not enough even
when everything passes (playback, focus and UI states are untestable by script).
## Test environment / prerequisites
- Instance: any PeerTube (baseline: <https://tube.jeena.net>, v8.3.1, media on R2 object storage).
- Account: `the test account` (sops: `~/.agents/secrets/jtube.env`). Credentials resolved
at login; **do not store the password in the device during tests** — it's typed once
on the setup screen (which is exactly feature 0).
- Seeded content (tagged `jtube-tv-test`, channel `the test channel`):
- `JTube TV Test 1 - Sintel Teaser` **Private** (~52 s)
- `JTube TV Test 2 - Caminandes Llamigos` **Private** (~2:30)
- `JTube TV Test 3 - Cosmos Laundromat` **Private** (~50 s)
- the test account has watched entries in server history (public + test videos).
- Subscription: `mediterranean_homestead@lone.earth` (remote channel — exercises
federation in the subscription feed).
- Public content: instance's own local videos (≥ 3000).
- Device: Android TV with D-pad; also sanity-check a phone profile for the touch path.
- Network conditions if available: Wi-Fi + (later) metered/lossy playback test.
## A. Scripted API checks (run before every release)
```bash
./scripts/e2e-api-check.sh # exit 0 = backend contract intact
```
Covers: login/refresh, local videos, my-videos incl. private rows, own channels,
local channel search, search, subscription feed (incl. remote channel), history shape,
watching POST → 204 → stored position, stream URL resolution, file-token endpoint,
playlist reachability.
## B. Manual device checklist
Legend: [T] must pass for TV (D-pad). Pass criteria named per item; log failures with
APK version + device.
### 0 — Credentials onboarding
- [ ] T1 Fresh install: no URL/account stored → setup screen appears with URL prefilled to `https://tube.jeena.net`.
- [ ] T2 Login with wrong password → app shows "Invalid credentials", user stays on setup, nothing persisted (clear app data → setup shows again). Repeat once to catch double-submit lockout via spinner + disabled button.
- [ ] T3 Login with bad host (`x.invalid`) → reachable-error message, no crash.
- [ ] T4 Successful login → home screen; force-stop app, relaunch → home screen directly (session persisted, encrypted). Tokens survive (no re-login).
- [ ] T5 Long-run: 24 h later (token expiry ~86 400 s) relaunch → any authorized screen still works without re-login (refresh-token flow ran silently). If this can't be waited out, simulate by `adb shell` clearing only the access token? — skip; instead validate refresh via the API script (F0) and mark T5 after one real 24 h exposure per release cycle.
### 1 — Local videos / My videos
- [ ] T6 "Videos" entry: list loads, newest public local videos first, thumbnails visible, D-pad scroll down past 25 rows loads page 2 (spinner at bottom disappears, list grows).
- [ ] T7 "My videos": the 3 seeded **Private** test videos appear with a privacy badge; non-test public videos of the account also appear if any.
- [ ] T8 Scroll from "Videos" to "My videos" and back → each screen keeps its own list (no cross-contamination of cached pages).
- [ ] T9 Long video title truncates (2 lines max) without breaking the row layout; long channel names truncate.
### 2 — Channels
- [ ] T10 "Channels" lists account's own channels (`the test channel`), avatar when present, video count.
- [ ] T11 Click channel → that channel's videos list; open and play one; back returns to the channel's list (not the channels screen lost state).
- [ ] T12 Remote channel via direct handle is out of MVP scope, but local-channel search (when implemented) must be re-tested: `searchTarget=local` ignores federated results.
### 3 — Search
- [ ] T13 Search screen: field focusable via D-pad; on-screen/IME input works. Search "Sintel" → Test 1 private video appears even though it's private (search endpoint filters locally: verify it does — if private videos are NOT in search results, this is expected PeerTube behavior; adjust expectation: my-videos screen is the place private content is discoverable).
- [ ] T14 Empty search string → no crash, stays on empty state. Gibberish → "Nothing here yet".
### 4 — Subscriptions
- [ ] T15 Subscriptions feed shows the remote (`lone.earth`) channel's videos — mixed-feed behavior works; newest first.
- [ ] T16 Feed pagination works beyond 25 items when subscribed to enough channels (skip if < 50 items total; note skip reason).
- [ ] T17 Click remote video → plays or fails *gracefully* (a remote video's media is on the remote host; MVP may fail to resolve — must show error toast/message + back, never crash/freeze).
### 5 — History
- [ ] T18 History lists server-side history newest-first, incl. both public and private watched entries.
- [ ] T19 Play an entry to ≥ 20 s, leave (back) → entry is present/updated in history when re-opening the screen (server round-trip within ~1 s; app must not show stale cache as fresh — reload on screen entry).
### 6 — Playback
- [ ] T20 Play a **public local** video → video+audio start < 5 s, D-pad pauses/plays, seek bar appears via center press, back closes player and lands on the list.
- [ ] T21 Play a **private** test video → plays. (On tube.jeena.net media is publicly readable today; the app must still send the file token — verify in logs that `?videoFileToken=` was appended, since that is what private-bucket instances need.)
- [ ] T22 Long watch: stay in a video > 60 s (watching reports every ~20 s) → server `userHistory.currentTime` updated; re-entering the video later resumes or at least proves the reports landed (curl check).
- [ ] T23 Switch between two videos (from list, play, back, play another) → no audio ghosting/duplicate players (only one active ExoPlayer; release resources on exit).
- [ ] T24 Airplane-mode / dead WiFi: opening player → error state shown, no crash; back works.
- [ ] T25 Remote-hosted video (subscription item, media on lone.earth) → if it plays it must not break when network switches; at minimum no crash.
- [ ] T26 Video without HLS (progressive-only instance): force by playing an item whose detail has no `streamingPlaylists` — if none exists on the test instance, note skip.
### 7 — Robustness / release hygiene
- [ ] T27 App data cleared while logged out → setup screen, no crash loops.
- [ ] T28 Rotate/screen-size changes are irrelevant on TV (landscape only), but app must not crash on phones when sideloaded.
- [ ] T29 Kill server (or wrong URL after instance rename): any authorized screen → consistent error UI + retry; stale-token case must eventually surface "re-login" (authenticator gives up).
- [ ] T30 APK is signed, minified build installs and boots (release build, `minifyEnabled` once enabled — re-run the whole checklist for it, R8 has bitten before).
## Release gate
All [T] items pass on the release build; script exits 0. Skips must be documented with
reason. New features added in a release must append their checklist items to section B.

View file

@ -67,6 +67,12 @@ class PeerTubeRepository(
expiresAtMillis = System.currentTimeMillis() +
(token.expiresIn ?: 86400L) * 1000L,
)
// Logins accept emails; resolve the real account name for /accounts/{handle}.
try {
store.accountName = api.me().username
} catch (_: Exception) {
store.accountName = username // best effort fallback
}
}
suspend fun me(): Result<UserDto> = runCatching { network.api.me() }
@ -110,6 +116,14 @@ class PeerTubeRepository(
suspend fun reportWatching(shortUuid: String, currentTimeSeconds: Int): Result<Unit> =
runCatching { network.api.watching(shortUuid, WatchingBody(currentTimeSeconds)) }
/**
* Short-lived URL token for non-public media. On instances storing media in a
* private object-storage bucket, playlist/segment URLs only accept this token as
* `?videoFileToken=`; the Bearer header only works for instance-served files.
*/
suspend fun videoToken(shortUuid: String): Result<String> =
runCatching { network.api.videoToken(shortUuid).files.token }
// ---- media URLs --------------------------------------------------------
/**

View file

@ -115,6 +115,17 @@ data class HistoryElement(
@SerialName("lastViewedAt") val lastViewedAt: String? = null,
)
@Serializable
data class VideoTokenResponse(
val files: VideoTokenFiles,
)
@Serializable
data class VideoTokenFiles(
val token: String,
val expires: String? = null,
)
@Serializable
data class UserDto(
val username: String,

View file

@ -103,6 +103,12 @@ interface PeerTubeApi {
@Body body: WatchingBody,
)
/** Short-lived token for private/internal media URLs (object storage safe). */
@POST("videos/{shortUuid}/token")
suspend fun videoToken(
@Path("shortUuid") shortUuid: String,
): VideoTokenResponse
@GET("users/me")
suspend fun me(): UserDto
}

View file

@ -48,6 +48,11 @@ class TokenStore(context: Context) {
get() = prefs.getString("refreshToken", null)
private set(value) = prefs.edit().putString("refreshToken", value).apply()
/** Real account name from users/me — may differ from the login input (email). */
var accountName: String?
get() = prefs.getString("accountName", null)
set(value) = prefs.edit().putString("accountName", value).apply()
var tokenExpiresAtMillis: Long
get() = prefs.getLong("tokenExpiresAtMillis", 0L)
private set(value) = prefs.edit().putLong("tokenExpiresAtMillis", value).apply()

View file

@ -32,20 +32,32 @@ import androidx.tv.material3.Card
import androidx.tv.material3.CardDefaults
import coil.compose.AsyncImage
import net.jeena.jtubetv.AppGraph
import net.jeena.jtubetv.data.PeerTubeRepository
import net.jeena.jtubetv.data.auth.TokenStore
import net.jeena.jtubetv.data.api.VideoChannelDto
import net.jeena.jtubetv.ui.list.PagedListViewModel
class ChannelsViewModel(private val repository: PeerTubeRepository) : PagedListViewModel<VideoChannelDto>() {
class ChannelsViewModel(private val store: TokenStore) : PagedListViewModel<VideoChannelDto>() {
private var cached: List<VideoChannelDto>? = null
override suspend fun fetch(start: Long, count: Int): Result<List<VideoChannelDto>> =
repository.localChannels(start, count).map { it.data }
runCatching {
val existing = cached
if (existing == null) {
// MVP: the user's own channels (fully local). A paged, searchable
// local-channels listing needs a non-empty search term server-side.
val response = AppGraph.repository.accountChannels(
store.accountName ?: store.username ?: "",
)
response.fold(onSuccess = { c -> cached = c; c }, onFailure = { throw it })
} else existing
}
companion object {
val factory: ViewModelProvider.Factory = object : ViewModelProvider.Factory {
@Suppress("UNCHECKED_CAST")
override fun <T : ViewModel> create(modelClass: Class<T>): T =
ChannelsViewModel(AppGraph.repository) as T
ChannelsViewModel(AppGraph.tokenStore) as T
}
}
}

View file

@ -60,7 +60,18 @@ class PlayerViewModel(
if (play == null) {
_state.value = PlayerUiState.Error("No playable stream found")
} else {
_state.value = PlayerUiState.Ready(detail, play.url, play.isHls)
// Non-public media may need a short-lived file token instead of
// (or in addition to) the Bearer header — matters on instances
// whose object-storage bucket rejects anonymous requests.
var url = play.url
if (detail.privacy?.id != 1) {
repository.videoToken(detail.shortUUID).onSuccess { token ->
url += (if (url.contains('?')) "&" else "?") +
"videoFileToken=" +
java.net.URLEncoder.encode(token, "UTF-8")
}
}
_state.value = PlayerUiState.Ready(detail, url, play.isHls)
}
},
onFailure = { e ->

112
scripts/e2e-api-check.sh Executable file
View file

@ -0,0 +1,112 @@
#!/usr/bin/env bash
#
# e2e-api-check.sh — scripted end-to-end smoke test against a live PeerTube instance.
#
# Usage: ./e2e-api-check.sh (uses ~/.agents/secrets/jtube.env via sops)
# sops -d <file>.env | bash <(cat e2e-api-check.sh) — alternative pipe form
#
# Requires: curl, python3. The account may be read-only-ish; only /watching POSTs and
# history reads are needed besides the listings. Prints PASS/FAIL per group; exits
# non-zero if any FAIL.
set -u
pass=0; fail=0
ok() { pass=$((pass+1)); echo "PASS $1"; }
bad() { fail=$((fail+1)); echo "FAIL $1"; }
assert() { # assert <name> <true/false>
if [ "${2:-x}" = "true" ]; then ok "$1"; else bad "$1"; fi
}
# ---- credentials -----------------------------------------------------------
if [ "${JTUBE_URL:-}" = "" ]; then
if command -v sops >/dev/null && [ -f "$HOME/.agents/secrets/jtube.env" ]; then
eval "$(sops -d --output-type dotenv "$HOME/.agents/secrets/jtube.env" | sed 's/^/export /')"
else
echo "No credentials: set JTUBE_URL/JTUBE_USER/JTUBE_PASS or provide the store"; exit 2
fi
fi
# ---- login -----------------------------------------------------------------
CLIENT=$(curl -s "$JTUBE_URL/api/v1/oauth-clients/local")
CID=$(echo "$CLIENT" | python3 -c "import json,sys;print(json.load(sys.stdin)['client_id'])")
CSEC=$(echo "$CLIENT" | python3 -c "import json,sys;print(json.load(sys.stdin)['client_secret'])")
TOKJSON=$(curl -s -X POST "$JTUBE_URL/api/v1/users/token" \
-d "grant_type=password" -d "client_id=$CID" -d "client_secret=$CSEC" \
-d "username=$JTUBE_USER" -d "password=$JTUBE_PASS")
TOK=$(echo "$TOKJSON" | python3 -c "import json,sys;print(json.load(sys.stdin).get('access_token',''))")
RTOK=$(echo "$TOKJSON" | python3 -c "import json,sys;print(json.load(sys.stdin).get('refresh_token',''))")
assert "F0 login (password grant)" $([ -n "$TOK" ] && [ -n "$RTOK" ] && echo true || echo false)
AUTH="Authorization: Bearer $TOK"
jqp() { python3 -c "import json,sys;d=json.load(sys.stdin);import functools;print(eval('d'+sys.argv[1]))" "$1" 2>/dev/null; }
# ---- F0b: refresh flow -----------------------------------------------------
NEW=$(curl -s -X POST "$JTUBE_URL/api/v1/users/token" \
-d "grant_type=refresh_token" -d "client_id=$CID" -d "client_secret=$CSEC" -d "refresh_token=$RTOK" \
| jqp "['access_token']")
assert "F0 token refresh" $([ -n "$NEW" ] && echo true || echo false)
TOK="$NEW"; AUTH="Authorization: Bearer $TOK"
# ---- F1 public local videos ------------------------------------------------
L=$(curl -s "$JTUBE_URL/api/v1/videos?isLocal=true&count=10&sort=-publishedAt")
N=$(echo "$L" | jqp "['total']")
assert "F1 local videos total>0" $([ "${N:-0}" -gt 0 ] && echo true || echo false)
FIRST_UUID=$(echo "$L" | jqp "['data'][0]['shortUUID']")
# ---- F1 me/videos incl. private -------------------------------------------
M=$(curl -s -H "$AUTH" "$JTUBE_URL/api/v1/users/me/videos?count=25&sort=-publishedAt")
assert "F1 my-videos wrapped {total,data}" $(echo "$M" | python3 -c "import json,sys;d=json.load(sys.stdin);print('true' if 'total' in d and isinstance(d.get('data'),list) else 'false')")
assert "F1 my-videos has entries" $(echo "$M" | jqp "['total'] > 0 and 'true' or 'false'")
PRIV=$(echo "$M" | python3 -c "
import json,sys
d=json.load(sys.stdin)
print('true' if any(v['privacy']['id']!=1 for v in d['data']) else 'false')")
assert "F1 non-public entries present (seeded test videos)" "$PRIV"
# ---- F2 own channels -------------------------------------------------------
USERNAME=$(curl -s -H "$AUTH" "$JTUBE_URL/api/v1/users/me" | jqp "['username']")
AC=$(curl -s -H "$AUTH" "$JTUBE_URL/api/v1/accounts/$USERNAME/video-channels")
assert "F2 own channels listed" $(echo "$AC" | python3 -c "import json,sys;print('true' if 'total' in json.load(sys.stdin) else 'false')")
CN=$(echo "$AC" | python3 -c "import json,sys; d=json.load(sys.stdin)['data'][0]; print(d['name']+'@'+d['host'])")
CV=$(curl -s -H "$AUTH" "$JTUBE_URL/api/v1/video-channels/$CN/videos?count=3")
assert "F2 channel videos endpoint" $(echo "$CV" | python3 -c "import json,sys;print('true' if 'total' in json.load(sys.stdin) else 'false')")
LC=$(curl -s "$JTUBE_URL/api/v1/video-channels?search=tube&searchTarget=local&count=5")
assert "F2 local-channel search (non-empty q)" $(echo "$LC" | python3 -c "import json,sys;print('true' if 'total' in json.load(sys.stdin) else 'false')")
# ---- F3 search --------------------------------------------------------------
S=$(curl -s "$JTUBE_URL/api/v1/videos?search=$FIRST_UUID&isLocal=true&count=5")
S2=$(curl -s "$JTUBE_URL/api/v1/videos?search=test&isLocal=true&count=5")
assert "F3 search returns list shape" $(echo "$S2" | python3 -c "import json,sys;d=json.load(sys.stdin);print('true' if 'total' in d else 'false')")
# ---- F4 subscriptions -------------------------------------------------------
SUBS=$(curl -s -H "$AUTH" "$JTUBE_URL/api/v1/users/me/subscriptions/videos?count=10&sort=-publishedAt")
assert "F4 subscription feed wrapped" $(echo "$SUBS" | python3 -c "import json,sys;d=json.load(sys.stdin);print('true' if 'total' in d else 'false')")
SUBS_REMOTE=$(echo "$SUBS" | python3 -c "
import json,sys,os
d=json.load(sys.stdin)
host=os.environ['JTUBE_URL'].replace('https://','').rstrip('/')
print('true' if any(v['channel']['host']!=host for v in d['data']) else 'false')")
assert "F4 remote-channel entries present" "$SUBS_REMOTE"
# ---- F5+F6 history & watching ----------------------------------------------
H=$(curl -s -H "$AUTH" "$JTUBE_URL/api/v1/users/me/history/videos?count=10")
assert "F5 history wrapped" $(echo "$H" | python3 -c "import json,sys;d=json.load(sys.stdin);print('true' if 'total' in d else 'false')")
CODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST -H "$AUTH" -H "Content-Type: application/json" \
-d '{"currentTime":3}' "$JTUBE_URL/api/v1/videos/$FIRST_UUID/watching")
assert "F6 watching -> 204" $([ "$CODE" = "204" ] && echo true || echo false)
UH=$(curl -s -H "$AUTH" "$JTUBE_URL/api/v1/videos/$FIRST_UUID?userHistory=true" | jqp "['userHistory']['currentTime']")
assert "F6 watching stored (userHistory)" $( [ -n "$UH" ] && echo true || echo false )
# ---- playback URLs ----------------------------------------------------------
D=$(curl -s -H "$AUTH" "$JTUBE_URL/api/v1/videos/$FIRST_UUID")
PL=$(echo "$D" | jqp "['streamingPlaylists'][0]['playlistUrl']")
assert "F6 detail exposes stream URL" $([ -n "$PL" ] && echo true || echo false)
VFTCODE=$(curl -s -o /dev/null -w "%{http_code}" -X POST -H "$AUTH" "$JTUBE_URL/api/v1/videos/$FIRST_UUID/token")
assert "F6 file token endpoint exists" $([ "$VFTCODE" = "200" ] && echo true || echo false)
VFT=$(curl -s -X POST -H "$AUTH" "$JTUBE_URL/api/v1/videos/$FIRST_UUID/token" | jqp "['files']['token']")
HC=$(curl -s -o /dev/null -w "%{http_code}" "$PL?videoFileToken=$VFT")
assert "F6 playlist reachable" $([ "$HC" = "200" ] && echo true || echo false)
echo "-----------------------------"
echo "PASS=$pass FAIL=$fail"
[ "$fail" = 0 ]