docs: design via AI
This commit is contained in:
145
DESIGN.md
145
DESIGN.md
@@ -1,26 +1,139 @@
|
||||
# Lockstep - Android music player
|
||||
# Lockstep Android MVP — design
|
||||
|
||||
Imagine as a runner that instead of you adapting the pace to the music, the music adapts its pace to you. When running with music, Lockstep feels your rhythm and adapts your own playlist to your running pace.
|
||||
Product goals, DSP scope, and repo layout are summarized in [SPECS.md](SPECS.md). This document captures **Android shell architecture**, **decisions made so far**, **libpasada JNI + state machine**, and **open UI questions**.
|
||||
|
||||
We are designing the prototype of an Android app with the minimum viable feature set to demonstrate the concept. The playing song will be adapted to the user's running pace as detected by the accelerometer.
|
||||
## Already captured from SPECS.md
|
||||
|
||||
For MVP 1, there will be a Playlist with multiple songs that plays back in an order such that the next song chosen by Lockstep closely matches its original rhythm (BPM) to the running pace.
|
||||
- **Product**: Pace-aware playlist ordering + real-time playback adaptation via accelerometer, **libpasada** (C++/JNI) + **Oboe**, user-supplied **MP3** via file descriptors, feedback loop for sensor-driven playback.
|
||||
- **Metadata**: **[jukebox](jukebox/)** submodule — Spotify-backed metadata in **SQLite**, separate from DSP.
|
||||
- **Native**: libpasada independently buildable/testable; Android side may **stub** JNI until native is ready.
|
||||
|
||||
Frameworks: from Java, we will use oboe and a newly designed JNI library, called libpasada, as dependencies.
|
||||
## Your decisions (this session)
|
||||
|
||||
libpasada will implement:
|
||||
* signal processing of the accelerometer time signal, turning it into a running pace time signal (an indication of frequency over time),
|
||||
* song analysis to a rhythm sequence time signal (an indication of frequency over time),
|
||||
* synthesis of the playback frequency spectrum over time, from the song's rhythm sequence time signal and the running pace time signal,
|
||||
* synthesis of the audio playback time signal from the playback frequency spectrum,
|
||||
* sampling rate conversion (running pace is in accelerometer fps, everything else must be done in audio fps),
|
||||
* buffering as necessary to feed oboe in real-time with audio frames.
|
||||
| Topic | Choice |
|
||||
|-------|--------|
|
||||
| Session context | **Background / screen off** — needs foreground service, ongoing notification, and media-style controls expectation |
|
||||
| Playlist assembly | **Manual** local MP3 pick + align/display with jukebox metadata where possible |
|
||||
| Playback ownership | **Run mode owned by a foreground service** — Oboe, libpasada JNI, and accelerometer feeding live in the service while a run is active; UI binds for display/control only |
|
||||
| Accel → native path | **`SensorManager` callbacks on a dedicated `HandlerThread` (not main)** — **`onSensorChanged` may call JNI directly**; **no intermediate Java queue** for MVP. libpasada must be safe relative to concurrent Oboe/audio-thread work (locking or lock-free handoff inside native). |
|
||||
| Jukebox scope (MVP) | **Playlist identity + BPM/metadata in SQLite**; **album/track artwork deferred** — artwork is **intended to live in jukebox later**, not in the app long-term. |
|
||||
| Spotify credentials | **Refresh is triggered and handled by the host app** (lifecycle, WorkManager, error handling, user re-login). Jukebox remains a **library**: expose/sync APIs that **accept valid credentials or sessions** the app supplies after refresh; avoid owning background refresh loops unless you later consolidate that into jukebox by explicit choice. |
|
||||
| MP3 ↔ jukebox matching | **1)** Match primarily using **embedded MP3 tags** (ID3 / common frames—title, artist, album; plus **ISRC** when present on both file and jukebox row). **2)** If no confident unique match, **prompt the user to link manually** (pick jukebox track ↔ local file). Persist manual links in app or jukebox-backed **mapping store**. **Ambiguous multi-match** should fall through to manual/disambiguation (exact field weights deferred at implementation). |
|
||||
| libpasada JNI + states | **State machine** (below); JNI entry points **`init` (per run start)**, **`feedAccel`**, **`play`** (MP3 FD + begin adaptive playback), **`pause`**, **`resume`**, **`stop`**, **`getDiagnostics`**. |
|
||||
|
||||
libpasada is implemented in C++ and may use additional libraries or frameworks as necessary to succinctly express the signal processing goals in C++.
|
||||
## libpasada state machine (native)
|
||||
|
||||
libpasada is independently testable from Android NDK tools, meaning it includes a C++ testing framework and CMake files which allow standalone building with a toolchain separate from Android NDK.
|
||||
Oboe behavior is summarized per state:
|
||||
|
||||
For libpasada, assume that there will be an implementation. Only implement stubs necessary to control the music. Assume that mp3 files are provided by the user for their playlist, and transfer open file descriptors to C++ via JNI. Assume there is a feedback loop in C++ that receives raw accelerometer data and will adjust playback accordingly in real-time.
|
||||
| State | Meaning |
|
||||
|-------|---------|
|
||||
| **LOADED** | JNI `.so` loaded (`System.loadLibrary`); no Oboe callbacks registered yet. |
|
||||
| **INITIALIZED** | Oboe stream active, outputting **silence**; callbacks registered; ready for **`play()`**. |
|
||||
| **PLAYING** | Oboe outputting **decoded/adapted music** (pace-driven processing active). |
|
||||
| **PAUSED** | User paused: Oboe still runs but outputs **silence** (distinct from STOPPED). |
|
||||
| **FINISHED** | Current track ended: Oboe outputs **silence** until the app starts another **`play()`** (or you later add explicit advance rules). |
|
||||
| **STOPPED** | Oboe **callbacks unregistered** / stream torn down for this run segment; lowest-power idle after **`stop()`**. |
|
||||
|
||||
For managing song metadata, there is a library called jukebox. It is a separate Android library, resides in a separate git repo, and is checked as a submodule into the project. It fetches metadata from the user's Spotify account and stores it in local SQLite.
|
||||
**JNI calls (frozen names for stubs)**
|
||||
|
||||
- **`init()`** — Called **each time a run is started** (not once per app launch): transition into **INITIALIZED** (register Oboe, run silent audio, allocate buffers, reset pace pipeline as needed). Preconditions: typically **LOADED** or returning from **STOPPED** after prior **`stop()`**.
|
||||
- **`feedAccel()`** — Submit accelerometer samples to the pace estimator (unchanged intent).
|
||||
- **`play(...)`** — Accept **open MP3 file descriptor** (and any offset/length/index the native decoder needs), begin decoding and **adaptive playback** → **PLAYING**. Used when starting or switching tracks from **INITIALIZED** or **FINISHED** (new FD); **not** used to leave **PAUSED** (use **`resume()`**).
|
||||
- **`pause()`** — **PLAYING → PAUSED** (silent output; keep graph alive).
|
||||
- **`resume()`** — **PAUSED → PLAYING**: continue the **same** track/decode position and adaptive processing without re-opening the FD.
|
||||
- **`stop()`** — Tear down to **STOPPED** (unregister Oboe callbacks / release stream for this run).
|
||||
- **`getDiagnostics()`** — Runtime metrics / last error for logging and UI (unchanged intent).
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
direction LR
|
||||
[*] --> Loaded
|
||||
Loaded --> Initialized: init on run start
|
||||
Initialized --> Playing: play fd
|
||||
Playing --> Paused: pause
|
||||
Paused --> Playing: resume
|
||||
Playing --> Finished: track ends native
|
||||
Finished --> Playing: play next fd
|
||||
Paused --> Stopped: stop
|
||||
Playing --> Stopped: stop
|
||||
Finished --> Stopped: stop
|
||||
Initialized --> Stopped: stop
|
||||
Stopped --> Initialized: init next run
|
||||
```
|
||||
|
||||
## Architecture implications (non-negotiable follow-through)
|
||||
|
||||
- **Foreground service** + **notification channel** (Android 8+) with **media controls** (Compact/MediaStyle or MediaSession) so runs survive screen-off and fit user expectations.
|
||||
- **Lifecycle split**: **ForegroundService is the single owner** of the real-time pipeline during a run (Oboe + JNI + `SensorManager` registration with a **`Handler` on a background `HandlerThread`** for listener delivery). Activity/Compose is for setup, library mapping, and reflecting service state—not for owning playback threads. Accelerometer-driven JNI runs on that **sensor HandlerThread**, concurrent with Oboe’s audio callback path unless libpasada merges them internally. (Separate process for the service remains optional if OEM kills warrant it—defer unless needed.)
|
||||
- **Permissions & battery**: `POST_NOTIFICATIONS` (13+), possibly **exact alarm** if you schedule anything (likely N/A for MVP); document **ignore battery optimizations** only if you discover OEM kills sensor/audio — question below.
|
||||
- **Jukebox vs app**: Treat jukebox as **metadata persistence + Spotify fetch helpers**; the **app owns OAuth/token refresh scheduling** and passes renewed tokens (or session handles) into jukebox when syncing. **Reads** should stay useful **offline** from SQLite; **network sync** requires valid credentials (exact failure UX still covered under onboarding/errors below).
|
||||
- **Library ingestion**: Reading tags runs **off main thread** (e.g. during folder scan or when user adds files); store resolved **local URI/path ↔ jukebox track id** so the foreground service can open FDs without redoing fuzzy matching.
|
||||
- **ForegroundService ↔ JNI**: Starting a run calls **`init()`**; handing off each local track calls **`play(fd, …)`**; **`pause()`** / notification pause map to **`pause()`**; **`resume()`** when the user resumes from pause; **`stop()`** when the run ends. Sensor **`HandlerThread`** keeps calling **`feedAccel()`** whenever registered.
|
||||
|
||||
---
|
||||
|
||||
## UI / UX questions (MVP 1)
|
||||
|
||||
### Navigation and screens
|
||||
|
||||
1. **Minimum screens**: e.g. Run (now playing + pace indicator), Library (local files + jukebox side-by-side?), Settings — which are **must-have** vs deferred?
|
||||
2. **Run screen layout**: Dominant **Now Playing** + subtle pace vs dominant **pace/BPM match** + compact transport?
|
||||
3. **Start/stop run**: Explicit **Start run** that arms sensors + service vs auto-start on play?
|
||||
|
||||
### System UI integration
|
||||
|
||||
4. **Notification**: Full **MediaSession** (lockscreen, BT headsets, Wear) vs minimal ongoing notification only?
|
||||
5. **Widgets / Wear**: Explicitly **out of scope** for MVP?
|
||||
|
||||
### Onboarding and permissions
|
||||
|
||||
6. **First-run flow**: Grant notifications, pick folder, sign-in to Spotify (if required by jukebox), battery optimization warning — **single wizard** or inline prompts?
|
||||
7. **Empty states**: No MP3s mapped, no Spotify login, no BPM in DB — copy and primary actions?
|
||||
|
||||
### Visual design
|
||||
|
||||
8. **Design system**: Material 3 / dynamic color vs fixed brand palette for sports contexts (high contrast sunlight)?
|
||||
9. **Accessibility**: Minimum targets for sweaty fingers, **TalkBack** labeling for pace and controls?
|
||||
|
||||
### Feedback and failure modes
|
||||
|
||||
10. **When pace detection is noisy**: Show confidence, freeze adaptation, or fallback to **nominal BPM**?
|
||||
11. **Errors**: Missing file, codec failure, JNI assertion — **blocking dialog** vs toast + safe stop?
|
||||
|
||||
---
|
||||
|
||||
## Suggested order to resolve
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph core [Core commitments]
|
||||
FS[ForegroundService]
|
||||
MS[MediaSession optional depth]
|
||||
JNI[libpasada JNI states]
|
||||
end
|
||||
subgraph data [Data layer]
|
||||
Match[Tags then manual map]
|
||||
JB[Jukebox SQLite plus app auth]
|
||||
end
|
||||
subgraph ui [UI shell]
|
||||
Nav[Screen map and navigation]
|
||||
RunUX[Run lifecycle UX]
|
||||
end
|
||||
FS --> MS
|
||||
FS --> JNI
|
||||
JNI --> Match
|
||||
Match --> JB
|
||||
FS --> RunUX
|
||||
Nav --> RunUX
|
||||
```
|
||||
|
||||
1. **JNI + libpasada states** — frozen (see **libpasada state machine** section).
|
||||
2. **MP3↔metadata matching** — **embedded tags first**, then **manual link** — decided.
|
||||
3. Lock **screen map** + **notification depth**.
|
||||
4. Fill **edge-case UX** (permissions, empty states, failures).
|
||||
|
||||
---
|
||||
|
||||
## Optional codebase alignment
|
||||
|
||||
Quick pass over existing Gradle modules and any placeholder Activity will turn this questionnaire into file-specific tasks (app module layout, jukebox submodule APIs).
|
||||
|
||||
12
README.md
Normal file
12
README.md
Normal file
@@ -0,0 +1,12 @@
|
||||
# Lockstep Player
|
||||
|
||||
Android prototype: music playback adapts to running pace (accelerometer + native DSP).
|
||||
|
||||
**Docs (read these first):**
|
||||
|
||||
| Doc | Contents |
|
||||
|-----|----------|
|
||||
| [SPECS.md](SPECS.md) | Product intent, MVP scope, **libpasada** DSP responsibilities, Oboe/JNI/MP3 FD assumptions, **jukebox** submodule role |
|
||||
| [DESIGN.md](DESIGN.md) | Android architecture decisions, foreground run service, JNI/state machine (`init` / `play` / `pause` / `resume` / `stop` / …), open UI questions |
|
||||
|
||||
Submodule: [`jukebox/`](jukebox/).
|
||||
26
SPECS.md
Normal file
26
SPECS.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# Lockstep - Android music player
|
||||
|
||||
Imagine as a runner that instead of you adapting the pace to the music, the music adapts its pace to you. When running with music, Lockstep feels your rhythm and adapts your own playlist to your running pace.
|
||||
|
||||
We are designing the prototype of an Android app with the minimum viable feature set to demonstrate the concept. The playing song will be adapted to the user's running pace as detected by the accelerometer.
|
||||
|
||||
For MVP 1, there will be a Playlist with multiple songs that plays back in an order such that the next song chosen by Lockstep closely matches its original rhythm (BPM) to the running pace.
|
||||
|
||||
Frameworks: from Java, we will use oboe and a newly designed JNI library, called libpasada, as dependencies.
|
||||
|
||||
libpasada will implement:
|
||||
* signal processing of the accelerometer time signal, turning it into a running pace time signal (an indication of frequency over time),
|
||||
* song analysis to a rhythm sequence time signal (an indication of frequency over time),
|
||||
* synthesis of the playback frequency spectrum over time, from the song's rhythm sequence time signal and the running pace time signal,
|
||||
* synthesis of the audio playback time signal from the playback frequency spectrum,
|
||||
* sampling rate conversion (running pace is in accelerometer fps, everything else must be done in audio fps),
|
||||
* buffering as necessary to feed oboe in real-time with audio frames.
|
||||
|
||||
libpasada is implemented in C++ and may use additional libraries or frameworks as necessary to succinctly express the signal processing goals in C++.
|
||||
|
||||
libpasada is independently testable from Android NDK tools, meaning it includes a C++ testing framework and CMake files which allow standalone building with a toolchain separate from Android NDK.
|
||||
|
||||
For libpasada, assume that there will be an implementation. Only implement stubs necessary to control the music. Assume that mp3 files are provided by the user for their playlist, and transfer open file descriptors to C++ via JNI. Assume there is a feedback loop in C++ that receives raw accelerometer data and will adjust playback accordingly in real-time.
|
||||
|
||||
For managing song metadata, there is a library called jukebox. It is a separate Android library, resides in a separate git repo, and is checked as a submodule into the project. It fetches metadata from the user's Spotify account and stores it in local SQLite.
|
||||
|
||||
Reference in New Issue
Block a user