Compare commits

...
21 Commits
Author SHA1 Message Date
Nathan Rashleigh 9fbd222b23 throw 2026-06-22 22:45:04 +10:00
Nathan Rashleigh d5ef56c3ed dataclass refactor anlz 2026-06-22 22:18:46 +10:00
Nathan Rashleigh 5810a536bd so bad 2026-06-22 19:17:24 +10:00
Nathan Rashleigh 096bce803a change bpm detect 2026-06-22 18:06:45 +10:00
Nathan Rashleigh b85cea496f zzz 2026-06-22 00:15:27 +10:00
Nathan Rashleigh 9ee5a70266 fix artwork 2026-06-21 22:27:11 +10:00
Nathan Rashleigh ea1669babf claude idiot 2026-06-21 21:46:39 +10:00
Nathan Rashleigh 8b4ac2cdaf spotify 2026-06-21 18:33:00 +10:00
Nathan Rashleigh f06d3aa0f9 py 2026-06-21 10:50:46 +10:00
Nathan Rashleigh 96d0331ff3 spot init 2026-06-21 10:16:36 +10:00
Nathan Rashleigh 9ccc3cea95 the downfall has begun 2026-06-09 18:46:25 +10:00
Nathan Rashleigh f71a2a3512 ghidra 2026-06-08 23:22:16 +10:00
Nathan Rashleigh 6a6553077b idk 2026-05-26 18:14:46 +10:00
Nathan Rashleigh 1e49010aa9 ts 2026-05-26 16:27:50 +10:00
Nathan Rashleigh 32adb5ddac kaitai py 2026-05-26 16:27:47 +10:00
Nathan Rashleigh 6a32066069 kaitai 2026-05-23 10:17:35 +10:00
Nathan Rashleigh f804c327de mut rgs 2025-09-27 09:35:26 +10:00
Nathan Rashleigh 690f92f565 errores 2025-09-25 08:58:09 +10:00
Nathan Rashleigh 076c60658d fix alignment of tables 2025-09-24 20:49:05 +10:00
Nathan Rashleigh 13ee0dcf01 errors 2025-09-24 16:33:01 +10:00
Nathan Rashleigh d35b51d4c3 errors 2025-09-23 22:42:18 +10:00
82 changed files with 24113 additions and 314 deletions

No files matched your search

+50
View File
@@ -0,0 +1,50 @@
---
name: pdb-maple
description: Updates maple's pdb serializer (maple-py/src/maple/pdb/) to match decoded format rules from the kaitai schema + FORMAT_SPEC, then regenerates and diffs the AceMo pdb against rekordbox's. Use after pdb-re records a finding.
tools: Bash, Read, Edit, Write, Grep, Glob
model: inherit
---
You make maple's writer emit `export.pdb` byte-for-byte like rekordbox, using the
format rules the `pdb-re` agent records. You implement the writer; you do NOT
reverse-engineer the binary.
## Standing context
- maple writer: `maple-py/src/maple/pdb/{writer,pager,rows,strings}.py`, model in
`maple-py/src/maple/model.py`.
- Format truth (your inputs): `maple-py/src/kaitai/rekordbox_pdb.py` (layout/types)
and `maple-py/re/FORMAT_SPEC.md` (value-construction rules). Read both each run.
- Reference real export: `maple-py/data/acemo/PIONEER/rekordbox/export.pdb`.
- Regenerate maple's AceMo pdb (identical-content reconstruction):
```bash
cd maple-py && uv run python tools/build_acemo.py # -> captures/maple_acemo.pdb
```
- Already matched (don't regress): 20 tables; page_flags 0x24/0x34 per table
(`PAGE_FLAG_34` in writer.py); 2-pages-per-table layout (strange odd / data even,
overflow 41+, `CORE_TABLES`); keys row; empty history_playlists/entries.
## Your loop, per finding
1. Read the latest `FORMAT_SPEC.md` rule + any changed kaitai fields.
2. Implement it in the writer (`rows.py` for row fields, `pager.py`/`writer.py`
for page/layout). Match the rule exactly; if a field is a "stored, not
reproducible" value, set the closest faithful default and note it in a code
comment.
3. `uv run python tools/build_acemo.py`, then diff against the reference at three
levels and report what still differs:
- file header + per-table row counts + page-header flags/free/used
- raw row bytes (e.g. track id=1) byte-by-byte vs the real row
Reuse/extend the diff approach in `/tmp/diff_pdb.py` + `/tmp/rowdiff.py` (or
write equivalents). `uv run pytest -q` must still pass.
4. Return: what you changed, the new diff result (which bytes/fields still
differ), and what pdb-re needs to decode next (the first remaining mismatch).
Goal: ZERO diff between `captures/maple_acemo.pdb` and the reference. Do not
excuse ANY difference as harmless — every byte must be explained or matched:
- `sequence`: increment a real counter, don't hardcode 1. If you can't match the
reference's exact value, flag it for pdb-re to check what the firmware parser
does with it (it's rekordbox's internal transaction counter).
- page layout / `next_unused` / overflow-page placement: match it. If the
reference looks incrementally-grown (path-dependent gaps), flag that the
reference should be a FRESH from-scratch rekordbox export, not an incremental one.
- density / free / used: investigate, never assume harmless.
Report any remaining nonzero diff with a concrete explanation, not a dismissal.
+59
View File
@@ -0,0 +1,59 @@
---
name: pdb-re
description: Reverse-engineers rekordbox export.pdb serialization by driving Ghidra (headless) and records each decoded field/formula in the kaitai schema + the shared FORMAT_SPEC. Use to decode an unknown/wrong pdb field.
tools: Bash, Read, Edit, Write, Grep, Glob
model: inherit
---
You reverse-engineer how rekordbox writes `export.pdb` and record what you find
so the `pdb-maple` agent can reproduce it byte-for-byte. You decode the FORMAT
and the VALUE-CONSTRUCTION RULES; you do not touch maple's writer.
## Standing context (already established — do not rediscover)
- Target binary: `rekordbox.exe` (89MB, JUCE C++). `export.pdb` is written by its
bundled **DeviceSQL "Page Manager"** (`dsql*`/`edb_dyn_*`).
- Analyzed Ghidra project: `~/ghidra_proj` (project `rbx`, program `rekordbox.exe`),
already imported + analyzed. JDK at `~/jdk/jdk-21.0.11+10`. Ghidra at
`/mnt/c/Users/ntr/code/ghidra_12.1.2_PUBLIC_20260605/ghidra_12.1.2_PUBLIC`.
- Run a Ghidra script (no re-analysis):
```bash
export JAVA_HOME=~/jdk/jdk-21.0.11+10; export GHIDRA_HEADLESS_MAXMEM=10G
GH=/mnt/c/Users/ntr/code/ghidra_12.1.2_PUBLIC_20260605/ghidra_12.1.2_PUBLIC
"$GH/support/analyzeHeadless" ~/ghidra_proj rbx -process rekordbox.exe -noanalysis \
-scriptPath ~/ghidra_scripts -postScript DecPdb.java
```
Edit `~/ghidra_scripts/DecPdb.java`'s ADDRS to decompile any addresses; output
→ `~/ghidra_proj/decpdb.txt`. Other scripts: FindPdb, CallTree, FindHash. Copies
also live in `maple-py/re/scripts/`.
- Address rebasing: Ghidra static = `0x140000000 + module_offset`. Frida traces
print `rekordbox.exe+0xOFFSET` (the agent `maple-py/tools/rbx_pdb_capture.js`);
never hardcode the ASLR runtime base.
- The serializer call stack is in `~/ghidra_proj/decpdb.txt` / `re/findings/`.
- Reference real export: `maple-py/data/acemo/PIONEER/rekordbox/export.pdb` (6
tracks). Parse with the kaitai schema (`maple-py/src/kaitai/rekordbox_pdb.py`)
via `uv run` from `maple-py/`.
## Open unknowns (current priorities)
- track row `0x14` (u4, per-track, 28-bit) — varies per track, not plain crc32;
likely a stored `djmdContent` value or a seeded hash. `0x18`/`0x1a` = per-export
constant.
- track row `0x02` `index_shift` — positional (~0x20/row, resets per page); exact
formula unknown. maple writes 0.
## Your loop, per field
1. Pick the unknown/wrong field you were asked about.
2. Decompile the relevant serializer function(s) in Ghidra (DecPdb on the call
stack; use FindHash/CallTree to locate the builder). Read the C; if it's a
stored DB value with no runtime formula, SAY SO explicitly.
3. Cross-check against real values dumped from the reference pdb (raw row bytes).
4. Record the finding in BOTH:
- `maple-py/src/kaitai/rekordbox_pdb.py` — rename the field / fix its type if
the layout was wrong.
- `maple-py/re/FORMAT_SPEC.md` — the value-construction RULE in prose +
pseudocode (this is what pdb-maple implements). Create the file if missing.
5. Return a concise summary: field, offset, decoded meaning, the exact rule (or
"stored value, not reproducible"), and the Ghidra evidence (function @addr).
Be rigorous: prefer reading the decompiled code over guessing. If you can only
partially decode, state confidence and what's still unknown. Keep edits to the
kaitai schema minimal and correct — pdb-maple depends on them.
+6
View File
@@ -20,3 +20,9 @@ maple.sublime-workspace
*.secret.yaml
etc/sldl-macos.yaml
__pycache__/
*.py[cod]
.venv/
maple-py/captures
maple-py/node_modules
+14
View File
@@ -0,0 +1,14 @@
{
"mcpServers": {
"ghidra": {
"command": "uv",
"args": [
"run",
"--script",
"/mnt/c/Users/ntr/Downloads/GhidraMCP-release-1-4/GhidraMCP-release-1-4/bridge_mcp_ghidra.py",
"--ghidra-server",
"http://127.0.0.1:8080/"
]
}
}
}
+31 -9
View File
@@ -1,9 +1,31 @@
[x] download playlist
[x] store everything by artist - album
[ ] convert flac to mp3
[ ] sync to and from s3 api storage
[ ] create entries in rekordbox db for playlists
[ ] have a rekordbox vm
[ ] analyse files downloaded by all users with rekordbox vm
[ ] store analysis metadata in pg
[ ]
## end to end
### collection management
just spot should:
download playlist from spotify into data/collection
determine files missing from collection
download each of those files with sockseek
- should be flac
- get artwork if missing
- correct tags
- quick validate
update m3u8 matching files in spotify to files in collection
### prepare for decks
store ANLZ in metadata directory by isrc
create data/usb directory
convert to mp3 v0 jstereo
- output into the data/usb directory
write pdb
- match playlist tree structure
generate anlz
- detect bpm is important
- waveforms are nice but not essential
### sync (later)
sync collection and usb to cloud
File diff suppressed because it is too large. Load diff
+4 -4
View File
@@ -1,7 +1,7 @@
path = "/data"
index-path = "/data/_metadata/index.sldl.csv"
username = "${SLDL_USER}"
password = "${SLDL_PASS}"
path = "${SOCKSEEK_DATA}"
index-path = "${SOCKSEEK_DATA}/_metadata/index.sldl.csv"
username = "${SOCKSEEK_USER}"
password = "${SOCKSEEK_PASS}"
name-format = "{artist(/)album(/)track( )title|(_missing_tags/)slsk-foldername(/)slsk-filename}"
+67
View File
@@ -28,9 +28,76 @@ dl trg:
link plist dir:
python3 maple-py/link.py {{plist}} {{dir}}
# Dump tracks from Spotify playlists (ids, urls, or spotify: uris)
spotify ids="7f5tPIfj21UJI4zfDmH4rx 6sAl8LhU6ediFNJRuQQIQX":
cd maple-py && uv run python -m maple.spotify {{ids}} --csv
# Download via sockseek (csv path, url, search string, ...) using sldl.tpl.conf settings
sockseek input:
cd maple-py && uv run src/sockseek.py {{input}}
# Sync data/collection with one or more Spotify playlists (download + m3u8s).
collection ids="7f5tPIfj21UJI4zfDmH4rx 6sAl8LhU6ediFNJRuQQIQX":
cd maple-py && uv run python -m maple.collection {{ids}}
# Build a rekordbox USB layout at <dst> from the collection at <src>.
# Converts FLAC→MP3 V0, extracts artwork, writes ANLZ (with PWAV/PWV3
# blue/white waveforms) and PIONEER/rekordbox/export.pdb.
export src="data/collection" dst="data/usb":
cd maple-py && uv run python -m maple "{{justfile_directory() / src}}" "{{justfile_directory() / dst}}"
sync args:
s5cmd {{args}}
# Sync data/usb/ → mounted USB stick (default /media/ntr/MAPLE), then
# safely unmount so the stick can be pulled without risk to the FAT
# write-back cache.
usb dst="/media/ntr/MAPLE":
@test -d {{dst}} || (echo "USB not mounted at {{dst}}"; exit 1)
@# -c (checksum), NOT --size-only: maple re-exports keep the SAME file size
@# (ANLZ section lengths are deterministic) but change content (waveforms,
@# pdb), so a size-only check silently skips them and leaves stale data.
rsync -rtvh --delete -c --no-perms --no-owner --no-group --modify-window=2 "{{justfile_directory()}}/data/usb/" "{{dst}}/"
sync
@# Try udisksctl first (no sudo, handles the udev event); fall back to
@# plain `umount` if udisks isn't available. Either way the recipe
@# exits 0 even if the unmount fails (e.g. someone has a file open).
@udisksctl unmount -b "$(findmnt -no SOURCE {{dst}})" 2>/dev/null && echo "unmounted {{dst}}" \
|| (umount {{dst}} 2>/dev/null && echo "unmounted {{dst}}") \
|| echo "warning: could not unmount {{dst}} — close any file browsers + retry"
# Build a FAT32 image of data/usb/ — ready to dd onto a stick.
# `size` must exceed the data/usb/ tree (rough rule: data size + ~50 MB FAT overhead).
usb-image size="2G" img="data/maple.img":
@command -v mcopy >/dev/null || (echo "mtools not installed (apt install mtools)"; exit 1)
@test -d "{{justfile_directory()}}/data/usb" || (echo "no data/usb (run \`just export\` first)"; exit 1)
rm -f "{{justfile_directory()}}/{{img}}"
truncate -s {{size}} "{{justfile_directory()}}/{{img}}"
mkfs.vfat -F 32 -n MAPLE "{{justfile_directory()}}/{{img}}"
cd "{{justfile_directory()}}/data/usb" && mcopy -i "{{justfile_directory()}}/{{img}}" -s -Q -p -m * ::/
@echo "image ready:" && ls -lh "{{justfile_directory()}}/{{img}}"
# DESTRUCTIVE: dd data/maple.img onto a block device (e.g. /dev/sdb).
# Double-check `lsblk` first — wrong device wipes that drive.
[confirm("usb-flash will OVERWRITE the entire device. Proceed?")]
usb-flash dev img="data/maple.img":
@test -b {{dev}} || (echo "{{dev}} is not a block device"; exit 1)
@test -f "{{justfile_directory()}}/{{img}}" || (echo "no image at {{img}}; run \`just usb-image\` first"; exit 1)
@lsblk -no NAME,SIZE,LABEL,MODEL {{dev}}
sudo dd if="{{justfile_directory()}}/{{img}}" of={{dev}} bs=4M status=progress oflag=direct conv=fsync
sudo sync
@echo "done — safe to remove (try: udisksctl power-off -b {{dev}})"
usb-pdb dst="/Volumes/MAPLE":
@test -d {{dst}}/PIONEER/rekordbox || (echo "no rekordbox dir at {{dst}}/PIONEER"; exit 1)
COPYFILE_DISABLE=1 cp maple-py/data/MAPLE/PIONEER/rekordbox/export.pdb {{dst}}/PIONEER/rekordbox/export.pdb
usb-reset dst="/Volumes/MAPLE":
@test -d {{dst}} || (echo "USB not mounted at {{dst}}"; exit 1)
rm -rf {{dst}}/Contents {{dst}}/PIONEER
COPYFILE_DISABLE=1 rsync -rtvh --no-perms --no-owner --no-group --modify-window=2 maple-py/data/MAPLE/Contents/ {{dst}}/Contents/
COPYFILE_DISABLE=1 rsync -rtvh --no-perms --no-owner --no-group --modify-window=2 maple-py/data/MAPLE/PIONEER/ {{dst}}/PIONEER/
maple-setup:
bash ./etc/maple_setup.sh
+1
View File
@@ -0,0 +1 @@
3.14
+57
View File
@@ -0,0 +1,57 @@
# maple-py conventions
Python conventions for the `maple-py/` package. These OVERRIDE default behavior;
follow them exactly. (General repo guidance is in the root `CLAUDE.md`; 4-space
indentation applies here too.)
The package is pyright-checked (basic mode, Python 3.14). `Track`
(`src/maple/track.py`) and the `Analysis` family (`src/maple/analyze.py`) are the
reference examples for all of the below.
## Imports at the top of the file
All imports go at module top. No lazy/local imports inside functions. Never wrap
an import in `try/except ImportError` to degrade gracefully — if an integral
dependency (e.g. `mixxx_analyzer`, `scipy`) is missing, the process must **die at
startup** via the top-level import failing. The only acceptable indented import
is an `if TYPE_CHECKING:` block to break a circular type-only reference.
## Fail loud — never swallow into None
"Don't swallow into None" is a general principle at every level, including call
sites. An operation that fails must **let the exception propagate** — never catch
it and return `None` / an empty / degraded result. BPM/beat analysis and waveform
extraction are integral: `_mixxx_analyze` raises rather than returning None,
`_decode_pcm` raises (`RuntimeError`) if ffmpeg is missing or fails to decode
rather than returning an empty array, and every `Analysis.analyze` call site (in
`anlz.py`, `cli.py`, `collection/core.py`) lets failures abort the run (including
not catching around `ThreadPoolExecutor` `fut.result()`).
Reserve `None` returns for *legitimate absence* (e.g. a cache miss in
`Analysis.load`), never for *failure*. The one tolerated catch is a genuinely
*optional* side-effect failing (e.g. writing the optimization cache via
`analysis.save()`), which logs and continues because it doesn't affect
correctness.
## Specific types over generic ones
- **Dataclasses, not bare dicts**, for structured values passed around.
- **`pathlib.Path`, not `str`**, for local filesystem paths (strings are fine for
serialization / rekordbox-USB paths that are inherently string data).
- **`typing.NamedTuple` (or a small dataclass), not bare tuples**, for fixed
records (e.g. `Beat(number, tempo_x100, time_ms)`, `Bands(low, mid, high)`).
- **No `None`-valued fields.** Absent values resolve at instantiation to a
concrete sentinel — `0.0` (0.0 = unknown), `""`, `[]`, `b""` — per `Track`.
`None` is acceptable only as a *factory return* signalling absence, never as a
stored field.
Idiomatic dataclass shape: plain `@dataclass` (no frozen/slots), `@property` for
computed views, `@classmethod from_*()` factories, concrete field defaults.
## Type every function parameter
Always annotate every parameter, including nested/inner functions and
`@classmethod` factory args (`self`/`cls` exempt). No bare `def f(res):`. Use the
real type even from a third-party lib — e.g. `mixxx_analyzer.AnalysisResult`
(referenced directly; no extra import needed under
`from __future__ import annotations`).
+112
View File
@@ -0,0 +1,112 @@
# maple-py — rekordbox `export.pdb` writer + reverse-engineering notes
Python tooling to generate a rekordbox USB export (`PIONEER/rekordbox/export.pdb`
+ `exportExt.pdb` + `USBANLZ` analysis files) so a CDJ/XDJ can read a
self-assembled music library. The writer lives in `src/maple/pdb/`.
This README also documents what we learned reverse-engineering rekordbox's own
exporter, because the on-disk format has real unknowns / magic numbers that the
public kaitai schema (`src/kaitai/rekordbox_pdb.py`) only partially decodes.
## The format, briefly
`export.pdb` is a **DeviceSQL** database: a flat file of 4096-byte **pages**,
grouped into 20 **tables** (tracks, artists, albums, playlists, artwork,
colors, columns, history, and several still-unknown ones). Page 0 is the file
header + table directory; each table is a linked list of pages. See
`src/maple/pdb/{writer,pager,rows,strings}.py`.
## Reverse-engineering findings (rekordbox 7.0.5, Windows)
### Where the export code lives
- The exporter is in **`rekordbox.exe`** (the 89 MB main binary), **not**
`rekordboxAgent.exe`. Confirmed by string scan: only `rekordbox.exe` contains
`USBANLZ`, `PIONEER`, `djmd…` table names, `.pdb`. It's a JUCE C++ app.
- `export.pdb` is written by rekordbox's **bundled DeviceSQL "Page Manager"**
engine — functions prefixed `dsql*` / `edb_dyn_*` (error strings:
`dsqlOpenDeviceDB`, `edb_dyn_CriticalSectionOn ERROR`,
`Page Manager (Close/)Open returns ERROR`).
- **Key consequence:** the page-level "magic numbers" maple guesses
(page flags `0x64`/`0x44`, `0x1fff` markers, `0x3ec`, free/used sizes) are
**DeviceSQL internals**, not rekordbox-specific. They are NOT immediate
constants in the code — they're produced by the page-manager logic, so they
must be read out of the `dsql*`/`edb_dyn_*` functions, not grep'd.
### A rekordbox USB export writes three databases
| file | format | writer |
|------|--------|--------|
| `export.pdb` | DeviceSQL (CDJ reads this) | DeviceSQL Page Manager |
| `exportExt.pdb` | DeviceSQL extended | same |
| `exportLibrary.db` | **SQLite** (rekordbox's own) | bundled `sqlite3_*` |
| `USBANLZ/**/ANLZ*.DAT/.EXT/.2EX` | per-track waveform/beatgrid | main app |
### `export.pdb` serializer call stack (live `WriteFile` backtrace → Ghidra)
Ghidra static address = `0x140000000 + module_offset` (Ghidra image base is
`0x140000000`; the ASLR runtime base varies per launch — never hardcode it,
read `module+offset` from Frida).
```
0x1420e8804 calls WriteFile ─┐
0x1420d7358 │ DeviceSQL write plumbing
0x1420d6e74 │ (transactions, page I/O)
0x1420d62a4 │
0x1420eac94 │
0x1420e07d8 │
0x1420de140 ─┘
FUN_141d75c20 ← dsqlOpenDeviceDB / Page Manager (export.pdb-specific)
0x141cad100
0x141cac390
0x141b8b4c0
0x141aca850
0x1423c9ab0 ← outer export driver
```
Other anchors found by string xref: `FUN_14119d2d0` (export orchestrator,
builds `/PIONEER/rekordbox/export.pdb`, walks "Contents"), `FUN_140f37e50`
(builds `PIONEER`/`USBANLZ` paths).
### Gotchas
- **Incremental writes:** rekordbox rewrites only the *changed* pages (seeked
`WriteFile`s), so a Frida capture is partial. The complete file is on the USB
(`F:\PIONEER\rekordbox\export.pdb`) — copy it from there for a full diff.
- **CIG / injection:** `rekordboxAgent.exe` refuses live Frida injection
(Code Integrity Guard). The pdb writer (main `rekordbox.exe`) is hookable, but
only via **child-gating at spawn** — Windows has **no spawn-gating**.
## Tooling
### Static analysis (Ghidra, headless, in WSL)
- JDK 21 (`~/jdk/jdk-21.0.11+10`), Ghidra 12.1.2. Ghidra rejects JDK 25 — use 21.
- Analyzed project: `~/ghidra_proj` (project `rbx`, program `rekordbox.exe`).
Full analysis ≈ 1 h with `GHIDRA_HEADLESS_MAXMEM=10G -max-cpu 8`.
- Re-run a script without re-analyzing:
```bash
"$GHIDRA/support/analyzeHeadless" ~/ghidra_proj rbx -process rekordbox.exe \
-noanalysis -scriptPath ~/ghidra_scripts -postScript FindPdb.java
```
Scripts: `~/ghidra_scripts/FindPdb.java` (string+constant xref search),
`DecPdb.java` (decompile an address list), `CallTree.java` (call-tree drill).
### Dynamic capture (Frida, on Windows)
- `tools/rbx_pdb_capture.js` — Frida agent. Hooks `NtCreateFile`/`NtWriteFile`/
`NtClose` + `NtCreateSection`/`NtMapViewOfSection` (memory-mapped writes).
Reconstructs files under `PIONEER`/`*.pdb` to `captures/`, prints each write's
backtrace as `rekordbox.exe+0xOFFSET` (→ Ghidra `0x140000000 + offset`).
- `tools/capture.mjs` — Node host (`npm i frida`). Attaches to all rekordbox
processes, streams `[pid]`-tagged output live, stays alive, **child-gates** so
the spawn writer is hooked at birth. Clears `captures/` on start.
```powershell
npm i frida # once
node tools\capture.mjs --spawn # launch rekordbox under the gate, then Export to USB
```
## Diffing maple's output vs rekordbox
`compare_pdb.py` parses two pdbs with the kaitai schema and summarises tables/
rows. Reference real exports: `data/PIONEER/rekordbox/export.pdb`,
`data/acemo/PIONEER/rekordbox/export.pdb`.
## Status / next steps
- Writer fully located; capture + diff pipeline works.
- To decode a specific page magic number: search the `dsql*`/`edb_dyn_*`
functions (image region `0x142000000+`) for the page-header builder.
- To fix a rejected export: diff a complete USB `export.pdb` against maple's
output field-by-field; resolve any wrong value via the matching `dsql` function.
+57
View File
@@ -0,0 +1,57 @@
"""Parse two .pdb files with the kaitai schema and summarise differences."""
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).parent / "src"))
from io import BytesIO
from kaitaistruct import KaitaiStream
from kaitai.rekordbox_pdb import RekordboxPdb
PAGE_TYPE_NAMES = {t.value: t.name for t in RekordboxPdb.PageType}
def summarise(path: Path):
data = path.read_bytes()
pdb = RekordboxPdb(is_ext=False, _io=KaitaiStream(BytesIO(data)))
print(f"\n=== {path} ({path.stat().st_size} bytes) ===")
print(f"page_size={pdb.len_page} num_tables={pdb.num_tables} next_unused_page={pdb.next_unused_page} sequence={pdb.sequence}")
for tbl in pdb.tables:
ptype = PAGE_TYPE_NAMES.get(tbl.type.value, str(tbl.type))
page_count = 0
row_count = 0
err = None
try:
ref = tbl.first_page
while True:
page = ref.body
page_count += 1
if page.is_data_page:
for rg in page.row_groups:
for rpw in rg.rows:
if rpw.present:
row_count += 1
if ref.index == tbl.last_page.index:
break
ref = page.next_page
if page_count > 5000:
break
except Exception as e:
err = f"{type(e).__name__}: {e}"
line = f" {ptype:24s} first={tbl.first_page.index:>4} last={tbl.last_page.index:>4} pages={page_count:>3} rows={row_count}"
if err:
line += f" ERR={err}"
print(line)
return pdb
def main():
base = Path(__file__).parent
ref = base / "data/MAPLE/PIONEER/rekordbox/export.pdb"
ours = base / "data/usb/PIONEER/rekordbox/export.pdb"
summarise(ref)
summarise(ours)
if __name__ == "__main__":
main()
Generated Executable
+501
View File
@@ -0,0 +1,501 @@
{
"name": "maple-py",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"dependencies": {
"frida": "^17.11.0"
}
},
"node_modules/balanced-match": {
"version": "4.0.4",
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-4.0.4.tgz",
"integrity": "sha512-BLrgEcRTwX2o6gGxGOCNyMvGSp35YofuYzw9h1IMTRmKqttAZZVU67bdb9Pr2vUHA8+j3i2tJfjO6C6+4myGTA==",
"license": "MIT",
"engines": {
"node": "18 || 20 || >=22"
}
},
"node_modules/base64-js": {
"version": "1.5.1",
"resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",
"integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT"
},
"node_modules/bindings": {
"version": "1.5.0",
"resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",
"integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==",
"license": "MIT",
"dependencies": {
"file-uri-to-path": "1.0.0"
}
},
"node_modules/bl": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz",
"integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",
"license": "MIT",
"dependencies": {
"buffer": "^5.5.0",
"inherits": "^2.0.4",
"readable-stream": "^3.4.0"
}
},
"node_modules/brace-expansion": {
"version": "5.0.6",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz",
"integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==",
"license": "MIT",
"dependencies": {
"balanced-match": "^4.0.2"
},
"engines": {
"node": "18 || 20 || >=22"
}
},
"node_modules/buffer": {
"version": "5.7.1",
"resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
"integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT",
"dependencies": {
"base64-js": "^1.3.1",
"ieee754": "^1.1.13"
}
},
"node_modules/chownr": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",
"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
"license": "ISC"
},
"node_modules/decompress-response": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",
"integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",
"license": "MIT",
"dependencies": {
"mimic-response": "^3.1.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/deep-extend": {
"version": "0.6.0",
"resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",
"integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",
"license": "MIT",
"engines": {
"node": ">=4.0.0"
}
},
"node_modules/detect-libc": {
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
"integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",
"license": "Apache-2.0",
"engines": {
"node": ">=8"
}
},
"node_modules/end-of-stream": {
"version": "1.4.5",
"resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz",
"integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",
"license": "MIT",
"dependencies": {
"once": "^1.4.0"
}
},
"node_modules/expand-template": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz",
"integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==",
"license": "(MIT OR WTFPL)",
"engines": {
"node": ">=6"
}
},
"node_modules/file-uri-to-path": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz",
"integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==",
"license": "MIT"
},
"node_modules/frida": {
"version": "17.11.0",
"resolved": "https://registry.npmjs.org/frida/-/frida-17.11.0.tgz",
"integrity": "sha512-ZymNV1mr8lWMP2vo1Ed4D7w1/8VQRVeoZ439sCZN/vKjj/E0UhfQP4BYsRAUrCQZ4nBzfRNN4BouA+WvOZ1z5w==",
"hasInstallScript": true,
"license": "LGPL-2.0 WITH WxWindows-exception-3.1",
"dependencies": {
"bindings": "^1.5.0",
"minimatch": "^10.0.1",
"prebuild-install": "^7.1.3"
},
"engines": {
"node": ">=16"
}
},
"node_modules/fs-constants": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",
"integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",
"license": "MIT"
},
"node_modules/github-from-package": {
"version": "0.0.0",
"resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",
"integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==",
"license": "MIT"
},
"node_modules/ieee754": {
"version": "1.2.1",
"resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
"integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "BSD-3-Clause"
},
"node_modules/inherits": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"license": "ISC"
},
"node_modules/ini": {
"version": "1.3.8",
"resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
"license": "ISC"
},
"node_modules/mimic-response": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",
"integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",
"license": "MIT",
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/minimatch": {
"version": "10.2.5",
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-10.2.5.tgz",
"integrity": "sha512-MULkVLfKGYDFYejP07QOurDLLQpcjk7Fw+7jXS2R2czRQzR56yHRveU5NDJEOviH+hETZKSkIk5c+T23GjFUMg==",
"license": "BlueOak-1.0.0",
"dependencies": {
"brace-expansion": "^5.0.5"
},
"engines": {
"node": "18 || 20 || >=22"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/minimist": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
"integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/mkdirp-classic": {
"version": "0.5.3",
"resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
"integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",
"license": "MIT"
},
"node_modules/napi-build-utils": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",
"integrity": "sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==",
"license": "MIT"
},
"node_modules/node-abi": {
"version": "3.92.0",
"resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.92.0.tgz",
"integrity": "sha512-KdHvFWZjEKDf0cakgFjebl371GPsISX2oZHcuyKqM7DtogIsHrqKeLTo8wBHxaXRAQlY2PsPlZmfo+9ZCxEREQ==",
"license": "MIT",
"dependencies": {
"semver": "^7.3.5"
},
"engines": {
"node": ">=10"
}
},
"node_modules/once": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
"integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
"license": "ISC",
"dependencies": {
"wrappy": "1"
}
},
"node_modules/prebuild-install": {
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
"integrity": "sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==",
"deprecated": "No longer maintained. Please contact the author of the relevant native addon; alternatives are available.",
"license": "MIT",
"dependencies": {
"detect-libc": "^2.0.0",
"expand-template": "^2.0.3",
"github-from-package": "0.0.0",
"minimist": "^1.2.3",
"mkdirp-classic": "^0.5.3",
"napi-build-utils": "^2.0.0",
"node-abi": "^3.3.0",
"pump": "^3.0.0",
"rc": "^1.2.7",
"simple-get": "^4.0.0",
"tar-fs": "^2.0.0",
"tunnel-agent": "^0.6.0"
},
"bin": {
"prebuild-install": "bin.js"
},
"engines": {
"node": ">=10"
}
},
"node_modules/pump": {
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",
"integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",
"license": "MIT",
"dependencies": {
"end-of-stream": "^1.1.0",
"once": "^1.3.1"
}
},
"node_modules/rc": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",
"integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",
"license": "(BSD-2-Clause OR MIT OR Apache-2.0)",
"dependencies": {
"deep-extend": "^0.6.0",
"ini": "~1.3.0",
"minimist": "^1.2.0",
"strip-json-comments": "~2.0.1"
},
"bin": {
"rc": "cli.js"
}
},
"node_modules/readable-stream": {
"version": "3.6.2",
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",
"integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
"license": "MIT",
"dependencies": {
"inherits": "^2.0.3",
"string_decoder": "^1.1.1",
"util-deprecate": "^1.0.1"
},
"engines": {
"node": ">= 6"
}
},
"node_modules/safe-buffer": {
"version": "5.2.1",
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
"integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT"
},
"node_modules/semver": {
"version": "7.8.2",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.8.2.tgz",
"integrity": "sha512-c8jsqUZm3omBOI66G90z1Dyw5z622G8oLG+omfsHBJf3CWQTlOcwOjvOG6wtiNfW6anKm/eA39LMwMtMez2TiQ==",
"license": "ISC",
"bin": {
"semver": "bin/semver.js"
},
"engines": {
"node": ">=10"
}
},
"node_modules/simple-concat": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz",
"integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT"
},
"node_modules/simple-get": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz",
"integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT",
"dependencies": {
"decompress-response": "^6.0.0",
"once": "^1.3.1",
"simple-concat": "^1.0.0"
}
},
"node_modules/string_decoder": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
"integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
"license": "MIT",
"dependencies": {
"safe-buffer": "~5.2.0"
}
},
"node_modules/strip-json-comments": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
"integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/tar-fs": {
"version": "2.1.4",
"resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.4.tgz",
"integrity": "sha512-mDAjwmZdh7LTT6pNleZ05Yt65HC3E+NiQzl672vQG38jIrehtJk/J3mNwIg+vShQPcLF/LV7CMnDW6vjj6sfYQ==",
"license": "MIT",
"dependencies": {
"chownr": "^1.1.1",
"mkdirp-classic": "^0.5.2",
"pump": "^3.0.0",
"tar-stream": "^2.1.4"
}
},
"node_modules/tar-stream": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz",
"integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",
"license": "MIT",
"dependencies": {
"bl": "^4.0.3",
"end-of-stream": "^1.4.1",
"fs-constants": "^1.0.0",
"inherits": "^2.0.3",
"readable-stream": "^3.1.1"
},
"engines": {
"node": ">=6"
}
},
"node_modules/tunnel-agent": {
"version": "0.6.0",
"resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",
"integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==",
"license": "Apache-2.0",
"dependencies": {
"safe-buffer": "^5.0.1"
},
"engines": {
"node": "*"
}
},
"node_modules/util-deprecate": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
"integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
"license": "MIT"
},
"node_modules/wrappy": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"license": "ISC"
}
}
}
+5
View File
@@ -0,0 +1,5 @@
{
"dependencies": {
"frida": "^17.11.0"
}
}
+31
View File
@@ -0,0 +1,31 @@
[project]
name = "maple"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
authors = [
{ name = "Nathan Rashleigh", email = "nathan.rashleigh@switchdin.com" }
]
requires-python = ">=3.14"
dependencies = [
"aiohttp>=3.10",
"kaitaistruct>=0.11",
"loguru>=0.7",
"mixxx-analyzer>=0.1.2",
"mutagen>=1.47.0",
"pillow>=12.2.0",
"pyrekordbox>=0.4.4",
"scipy>=1.10",
]
[project.scripts]
maple = "maple.cli:main"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[dependency-groups]
dev = [
"pytest>=9.0.3",
]
+23
View File
@@ -0,0 +1,23 @@
{
"include": [
"src",
"tests"
],
"extraPaths": [
"src"
],
"exclude": [
"**/__pycache__",
"**/.venv",
"data",
"re"
],
"venvPath": ".",
"venv": ".venv",
"pythonVersion": "3.14",
"pythonPlatform": "Linux",
"typeCheckingMode": "basic",
"reportMissingImports": "error",
"reportMissingModuleSource": "warning",
"useLibraryCodeForTypes": true
}
+609
View File
@@ -0,0 +1,609 @@
# export.pdb value-construction rules (RE findings)
This file records the exact rules rekordbox uses to populate `export.pdb`
fields that maple currently writes wrong or as zero. Each section is a
self-contained spec for the `maple` writer to implement. Evidence is from
reverse-engineering `rekordbox.exe` (DeviceSQL "Page Manager", `dsql*` /
`edb_dyn_*`) in Ghidra plus byte-level analysis of the reference export
`maple-py/data/acemo/PIONEER/rekordbox/export.pdb` (6 tracks, 1 album,
1 artist, 1 key, 6 artwork).
Reference-pdb facts used throughout:
- `len_page` = 4096.
- Row index grows backward from end of page; each group spans 36 (0x24) bytes:
`group_base = len_page - group_index*36`. Within a group, `present_flags`
(u2) live at `group_base-4`; the row offset for slot `i` (0..15) is the u2 at
`group_base-(6+2*i)`; offsets are relative to the heap start (0x28, i.e. end
of the 0x28-byte page header).
---
## 1. Track row `index_shift` (offset 0x02, u2) — SOLVED
**Field:** `TrackRow.index_shift` (also present on artist/album/genre/label/
key rows — every row type whose layout begins `subtype:u2, index_shift:u2`).
**Rule:**
```
index_shift = slot_index * 0x20 # i.e. slot_index << 5
```
where `slot_index` is the row's **0-based position in the page row index**
(`group_index*16 + slot_within_group`), NOT its ordinal among present rows and
NOT its position in the heap. It resets to 0 at the start of every data page.
Key consequences for the writer:
- The first row index slot (slot 0) gets `index_shift = 0`.
- Deleted/absent slots still consume a slot number, so a gap in
`present_flags` produces a gap in `index_shift`. (See evidence: the AceMo
page has slot 5 deleted, so the row in slot 6 has `index_shift = 0xc0`,
skipping `0xa0`.)
- `index_shift` is purely a function of the row's slot, so the value is fully
reproducible from the page layout maple already controls — there is no
external/runtime input. If maple writes each track to slot `i`, set
`index_shift = i*0x20`.
**Evidence:**
- Reference AceMo `tracks` page (page 2): present slots are 1,2,3,4,6 (slots 0
and 5 are deleted), with `index_shift` = 0x20,0x40,0x60,0x80,0xc0
respectively. Overflow page (page 48) has one row in slot 1 →
`index_shift = 0x20`. The 0x80→0xc0 jump across the deleted slot 5 proves the
value tracks `slot_index`, not present-count.
- Cross-checked against `artists` (page 5) and `albums` (page 7) rows: every
row satisfies `index_shift == slot_index*0x20` with **0 mismatches** over all
present rows in the reference file
(`maple-py/re/scripts`-style raw parse, 8 rows checked, see RE notes).
- Ghidra: the value is assigned inside the DeviceSQL Page Manager record-insert
path, which is reached only through vtable dispatch
(`(**(code**)(*obj + 0x4d0))(obj,...)` etc. in `FUN_141b8b4c0` /
`FUN_141cac390`), so there is no single statically-anchored
`index_shift = slot<<5` instruction to cite; a constant-scan for `shl ,5`
paired with the row-index masks (0x1fff/0x7ff) and stride 0x24 produced only
CRT/`printf("%a")` false positives (`FUN_14232a86c`). The empirical rule
above is exact and zero-mismatch, so confidence is **high** on the formula;
confidence on "no hidden runtime input" is high because the value is fully
explained by slot position alone.
**Confidence:** high.
---
## 2. Track row 0x14 / 0x18 / 0x1a (opaque id blob) — SOLVED (stored value, not a formula)
**Fields:** `TrackRow.opaque_track_id_lo` (u4 @0x14), `TrackRow.opaque_seed_lo`
(u2 @0x18), `TrackRow.opaque_seed_hi` (u2 @0x1a). Together these are one
8-byte little-endian value:
```
blob_u64 = opaque_track_id_lo | (opaque_seed << 32)
opaque_seed = opaque_seed_lo | (opaque_seed_hi << 16) # u4 at offset 0x18
```
(@flesniak's schema labels these "Some ID? unknown", "always 19048?",
"always 30967?". The "19048"/"30967" = 0x4A68/0x78F7 are only correct for some
older exports; they are NOT universal — see below.)
**What they are (proven empirically across 5 GENUINE rekordbox exports):**
> NOTE (2026-06-09 correction): a prior draft included a 5th row
> "usb (176), seq=1, seed 0x78f74a68, lo all 0" and concluded rekordbox writes
> lo=0 / seed 0x78f74a68 for unanalyzed tracks. **That file
> (`data/usb/.../export.pdb`) is maple's OWN output, not a rekordbox export**
> (it is the compare_pdb.py "ours" path). Reading it back was circular; that
> claim is WRONG and has been dropped. Provenance rule: maple output = seed
> `0x78f74a68` + ALL per-track lo == 0; genuine rekordbox = per-install nonzero
> seed + nonzero per-track lo.
| export (tracks) | header.seq | opaque_seed | opaque_track_id_lo |
|----------------------------|-----------:|---------------|-------------------------------|
| acemo (6) | 353 | `0x21b4a7ab` | 6/6 nonzero |
| acemo_fresh (7, same lib) | 35 | `0x21b4a7ab` | 7/7 nonzero |
| lib_multi (10, same lib) | 56 | `0x21b4a7ab` | 10/10 nonzero |
| MAPLE (176, install B) | 546 | `0xd6d3a567` | 176/176 nonzero |
| data/PIONEER (847, inst. B)| 4211 | `0xd6d3a567` | 847/847 nonzero |
Across **all 1046 real tracks in 5 genuine exports: 100% nonzero per-track lo,
exactly one nonzero per-install seed per file. There is NO evidence rekordbox
ever writes lo=0 or seed `0x78f74a68`.** The lo values span nearly the full
28-bit space (`0x0003edeb`..`0x0ffe7f4e`).
Findings:
1. **`opaque_seed` is constant within a single export** (exactly one value per
file) and is **per-rekordbox-library/install**: acemo/acemo_fresh/lib_multi
are three exports of the same library and all share `0x21b4a7ab`; the separate
install behind MAPLE + data/PIONEER uses `0xd6d3a567`. (The old maple hardcode
`0xa567`/`0xd6d3` = `0xd6d3a567` was correct only for install B.) The seed is
NOT a binary constant (proven: no `0xa7ab`/`0x21b4`/`0xd6d3`/`0xa567`
immediate or data-byte sequence in rekordbox.exe), so it is derived at runtime
from the per-library DB (most likely a hash/truncation of the master.db's
per-library UUID or creation marker). maple cannot reproduce a specific
install's seed (see recommendation).
2. **`opaque_track_id_lo` is a stable per-track value within a library.** Every
track that appears in both the acemo and acemo_fresh exports has an
**identical** lo (matched by title): "All My Life"→`0x07ff5a56`,
"Smooth Move"→`0x0a20de34`, "Jelly Fish Jam"→`0x0f8fd598`, etc. It is NOT
per-export-random.
3. **It is NOT derivable from the track's file metadata.** Tested and ruled out
(no match, 28-bit and full, utf-8/utf-16, several hash funcs): crc32 / adler32
/ md5 / sha1 / fnv1 / fnv1a / djb2 of the title, filename, full
`/Contents/...` path, and crc32 of the ANLZ `.DAT`/`.EXT` and the `.mp3`
bytes. Also not the USBANLZ folder id, not sequential, not evenly spaced.
4. **The values are runtime-derived, not binary constants.** A full
instruction-immediate scan of `rekordbox.exe` found NO occurrence of either
real seed (`0x21b4a7ab`, `0xd6d3a567`), the per-track lo values, or the
track-row `bitmask` constant `0xc0700`. So these come from rekordbox's
internal database (master.db / `djmdContent` + a per-library seed), copied
into the row by the DeviceSQL record builder (vtable-dispatched, not
statically anchorable). (`0x78f74a68` does not appear either — and per the
correction above it was never a real rekordbox value here, just maple's
placeholder.)
**Round-2 evidence (2026-06-09, after team-lead supplied acemo_fresh and asked
to chase an audio-content hash):**
5. **Audio-content-hash hypothesis: DISPROVEN.** Hashed the actual `.mp3`
bytes in 8 framings — full file, ID3v2-stripped (audio frames only),
first 1k/2k/4k/64k, last 64k, ID3-stripped-first-64k — each with crc32,
adler32, md5 (lo/hi, LE+BE), sha1 (LE+BE), sha256, fnv1a, both full-32 and
masked-to-28-bit. NONE matched any track's `opaque_track_id_lo`. So it is
not a hash of the audio file content (at least not of the file bytes; a hash
of decoded PCM cannot be ruled out but there is no evidence for it and it is
not reproducible anyway).
6. **Source DB confirmed, and it is ENCRYPTED.** Both `exportLibrary.db` (next
to the pdb) and the captured `master.db`
(`maple-py/captures/p17328_master.db`) fail to open as plain SQLite
("file is not a database") — rekordbox 6/7 encrypts them with SQLCipher.
The `opaque_track_id_lo` originates there; it cannot be read without the key.
7. **The value is a pure stored field, used nowhere else.** It appears in the
export.pdb ONLY at each track row's own offset 0x14 (present and stale/deleted
rows alike); it does NOT appear in exportExt.pdb (no tag_tracks/ContentID
link), nor in the ANLZ `.DAT`/`.EXT` (searched LE+BE), nor does it equal the
USBANLZ folder id. So it is not a foreign key the player resolves.
8. **Range note (suggestive, not conclusive):** across the 5 real exports the
lo values span nearly the full 28-bit space (`0x0003edeb`..`0x0ffe7f4e`),
the band of rekordbox-6/7 9-digit `djmdContent.ID` integers, so the most
likely identity is "the rekordbox internal Content ID" — a stored
random/sequential DB id, NOT a content-derived hash. Consistent with every
other finding and with it being unreadable (encrypted DB).
**Conclusion:** this 8-byte blob is a **stored rekordbox-internal id (most
likely the encrypted-DB Content ID), not a function of the track metadata or
audio bytes maple has. It cannot be reproduced byte-for-byte** for an arbitrary
track without rekordbox's own (encrypted) DB. The audio-content-hash idea is
ruled out. There is **no "unanalyzed/zero" form** in any real export — every
real track carries a nonzero seed and a nonzero lo.
**Recommendation for maple's writer:**
- Players/crate-digger treat this blob as opaque (never decoded) and play fine
regardless — so any value works for playback.
- For maximum faithfulness to genuine rekordbox output (which never has
lo=0/seed `0x78f74a68`), maple should write a **nonzero per-install seed and a
nonzero, stable per-track lo**:
- `opaque_seed`: use a real per-install seed. If maple has no install of its
own, `0x21b4a7ab` (this RE machine's install) is a known-real value;
keep it constant across all tracks in one export. Do NOT use `0x78f74a68`
(that was maple's own placeholder, not a rekordbox value).
- `opaque_track_id_lo`: a nonzero 28-bit value, stable per track. Since the
real value is an opaque DB id maple can't recover, a deterministic stand-in
(e.g. a 28-bit hash of a stable per-track key, or the track's own pdb id
offset into the ContentID band) is fine — the goal is "looks like a real
nonzero ContentID", not byte-parity.
- Either way these 8 bytes will NOT match a specific reference export
byte-for-byte (the real ids live in that install's encrypted DB) — record the
divergence in re/KNOWN_DIVERGENCE.md.
**Confidence:** high (5-export empirical proof: 1046/1046 real tracks nonzero;
exhaustive negative hash search incl. audio bytes; binary const-absence). The
earlier "write 0 / 0x78f74a68" advice was based on a maple-output file and is
retracted.
**Round-3 (task #10): can pyrekordbox read the real ids out of master.db?**
User insight: pyrekordbox holds the SQLCipher key, so the DBs are not truly
unreadable. Validated — partly true, but blocked on the data we have:
- pyrekordbox 0.4.4 + `sqlcipher3-wheels` installs and wires up correctly. Its
master.db key is the single **global** rekordbox-6 key
`402fd482c38817c35ffa8ffb8c7d93143b749e7d315df7a81732a1ff43608497`
(`deobfuscate(BLOB)`, must start with `402fd`) — same for every install, NOT
per-install. So a VALID master.db *would* open and expose `DjmdContent`.
- BUT the captured `maple-py/captures/p17328_master.db` is **corrupt/truncated**:
1,024,000 bytes but only 4.4% non-zero, and the first 32 KB (the mandatory
SQLCipher salt + page 1) is **all zeros**. No key can decrypt a DB with a
zeroed salt. `p17328_master.backup.db` is 0 bytes. The Frida NtWriteFile
capture only caught scattered later pages, never the header. → unusable.
- The genuine `exportLibrary.db` shipped next to export.pdb (valid 16-byte salt
`ed0d3a3c…`) does NOT open with the global master.db key (different key
scheme) and pyrekordbox has **no exportLibrary.db support**. → can't read it
here.
- Strong inference (untested for lack of a readable DB): the per-track
`opaque_track_id_lo` is almost certainly `DjmdContent.ID` (the 9-digit
rekordbox Content ID; our lo range 0x0003edeb..0x0ffe7f4e fits). If maple can
open the user's LIVE `%APPDATA%/Pioneer/rekordbox6/master.db` with pyrekordbox
(global key), `db.get_content()` would give `(ID, …)` per track, and the blob
would become EXACTLY reproducible: `opaque_track_id_lo = DjmdContent.ID & 0xFFFFFFFF`.
- The per-install `opaque_seed` (0x21b4a7ab here) is NOT a DjmdContent column;
it is one value for the whole export and most likely derived from a
master.db-level per-library marker (a USB/library UUID or an `agentRegistry`/
`local_usn`-style value). Identifying it needs a readable master.db to inspect
candidate columns; unconfirmed.
**pyrekordbox write / export capability (asked in #10):**
- It CAN write the desktop master.db: `add_content`/`add_album`/`add_artist`/
`create_playlist`/`commit`, plus `set_local_usn`, `update_content_path`. But
`add_content` lets rekordbox ASSIGN the Content ID; there is no API to force a
specific ID. So it is a source of real ids, not a way to dictate them.
- It does NOT drive a USB export — only `get_device` (read). The actual
`export.pdb`/`exportLibrary.db` writing is rekordbox.exe's DeviceSQL engine
(the thing maple reimplements). pyrekordbox manipulates only the desktop
library DB, not the USB export. So pyrekordbox cannot replace maple's writer;
its only use to maple is as a READ source of the real `DjmdContent.ID` (and
potentially the seed) from a valid master.db.
**Net for #10:** the approach is sound (pyrekordbox + global key reads a real
master.db → Content IDs), so `opaque_track_id_lo` is likely reproducible *if
maple reads the live master.db*. It is NOT reproducible from the corrupt
captured DB we have, and the seed source is still unidentified. pyrekordbox
cannot produce the USB pdb itself. Recommended next step for maple/RE: point
pyrekordbox at the user's LIVE rekordbox6 master.db
(`%APPDATA%/Pioneer/rekordbox6/master.db`), confirm
`DjmdContent.ID == opaque_track_id_lo` for the AceMo tracks, then hunt the seed
in that DB. Until then, the nonzero stand-in (above) stays.
---
## 3. File-header `sequence` (and per-page `sequence`)
**Field:** `RekordboxPdb.sequence` (u4 @0x14 of the file header) and
`Page.sequence` (u4 in each page header).
**Rule / meaning (confirmed from schema docs + the 5 genuine exports above):**
- `sequence` is a monotonically increasing **edit counter** for the database.
The file-header value is the **next** page-sequence number; each page records
the counter value as of its last edit (so `header.sequence >=
max(page.sequence)`). Observed header values in genuine rekordbox exports:
35, 56, 353, 546, 4211 — scales with edit history, not track count. (No real
export with sequence==1 is available; `data/usb`'s seq==1 is maple's own
output, not a rekordbox sample — see §2 correction.)
- The firmware/players (and crate-digger) use page `sequence` only as
informational ordering metadata; **they do not validate it** for playback.
No special value is required beyond keeping `header.sequence` greater than any
page's `sequence`.
- For maple it is enough to (a) pick a header `sequence` larger than every page
`sequence` it writes, and (b) give each page a `sequence` <= header. A simple
scheme (all pages = N, header = N+1) is accepted. This is what task #3
(incrementing sequence) implements.
**Confidence:** high for semantics and "not validated by players"; the exact
increment rekordbox uses per edit is not needed for a valid export.
---
## 4. File-header offset 0x10 (u4, schema `_unnamed4`) — HISTORY-DEPENDENT; not byte-reproducible
> CORRECTION (2026-06-09): a prior draft cited a "usb fresh, sequence==1,
> _unnamed4==0" row as proof that 0 is the fresh value. That file is maple's own
> output, not a rekordbox export, so it is removed below. The remaining proof
> (same-library divergence) does not depend on it.
**Field:** the unnamed u4 between `next_unused_page` (0x0c) and `sequence`
(0x14). maple writes 0; the AceMo ref has 4.
**Finding (5 genuine rekordbox exports):** an **edit/allocator accumulator**,
not a structural value derivable from the tables. It does NOT equal track count,
page count, or `next_unused_page - npages`:
| export | sequence | _unnamed4@0x10 | next_unused | npages |
|---------------|---------:|---------------:|------------:|-------:|
| acemo_fresh | 35 | 5 | 52 | 49 |
| lib_multi | 56 | 4 | 57 | 52 |
| acemo | 353 | 4 | 53 | 49 |
| MAPLE | 546 | 5 | — | — |
| data/PIONEER | 4211 | 1 | 174 | 174 |
**Decisive (same-library proof):** acemo and acemo_fresh are the SAME library
(identical track set + identical per-track opaque ids) yet have `_unnamed4` =
4 vs 5. A structural field would be identical; differing proves it is edit
state. (We have NO genuine rekordbox export with `sequence==1`, so the exact
never-edited value is unconfirmed — but it is clearly not derivable from the
file's structure.)
**Rule for maple:** there is no reproducible rule. Write **0** (maple already
does) as a **reasonable minimal default — NOT a confirmed match** (every genuine
export we have is nonzero here, and we have no never-edited rekordbox export to
check what fresh rekordbox writes). Record the divergence in
re/KNOWN_DIVERGENCE.md. It cannot be matched to the AceMo ref's `4` without that
DB's edit history.
**Confidence:** high that it is edit-history/allocator state and not
structurally reproducible; the specific fresh-write value is unconfirmed (no
seq==1 rekordbox sample available).
---
## 5. Page-header offset 0x26 (u2, "unknown7"/strange-page) — HISTORY-DEPENDENT; maple's 0 is a reasonable default
> CORRECTION (2026-06-09): do NOT cite `data/usb` (sequence==1) here — it is
> maple's OWN output, so "maple's 0 matches usb" would be circular. The
> observations below are from the five GENUINE rekordbox exports only.
**Field:** the second of the two trailing u2s in the page header (after
`transaction_row_index` @0x22 and the "1004 for strange blocks" u2 @0x24).
@flesniak: "always 0 except 1 for history pages, num entries for strange
pages?". maple writes 0; the AceMo ref has 0/1/2 on various tables' first
("strange") pages.
**Finding (5 genuine rekordbox exports — all incrementally edited):**
- On DATA pages it is **always 0** (all five exports). This is a solid
observation about genuine rekordbox output.
- It is nonzero only on a table's first/"strange" page (flags 0x64), and the
value does NOT correlate with that table's data-page count or row count
(acemo tracks: 2 data pages → 2; data/PIONEER tracks: 105 data pages → 23).
It is transaction/edit-tracking state.
- **Decisive (same-library proof):** acemo and acemo_fresh hold the identical
tracks table, yet its strange-page unknown7 = 2 (acemo) vs 1 (acemo_fresh).
Same data → different value ⇒ not structural, it is edit state. (lib_multi:
tracks strange unk7 = 2 with 3 data pages / 10 rows — again no count relation.)
**Rule for maple:** write **0** on every page (maple already does). The data
pages being 0 across all five genuine exports makes 0 a well-supported value
*for data pages*. For strange pages the real value is per-export edit state
(0/1/2…) that cannot be reproduced — maple's 0 there is a **reasonable minimal
default, NOT a confirmed match** (we have no never-edited rekordbox export to
check what fresh rekordbox stamps on a strange page). Record the strange-page
divergence in re/KNOWN_DIVERGENCE.md.
**Confidence:** high that it is history-dependent and that data-page 0 matches
real rekordbox; the fresh-write strange-page value is unconfirmed (no seq==1
rekordbox sample).
---
## 6. Row-group trailing u2 (`transaction_row_flags`) — write `present_flags` (NOT 16); residual mismatch is edit history
> CORRECTION (2026-06-09): an earlier draft of this section claimed "a fresh
> export writes 16 everywhere, so maple's 16 is correct." That was based on
> `data/usb/.../export.pdb`, which on re-check is **maple's OWN output**, not a
> rekordbox export (opaque seed `0x78f74a68` + all-zero per-track lo +
> `sequence`==low). Reading it back was circular. The analysis below uses ONLY
> the five genuine rekordbox exports (provenance verified by nonzero per-track
> opaque ids + per-install seed): acemo, acemo_fresh, lib_multi, MAPLE,
> data/PIONEER.
**Field:** the last u2 of each 36-byte row-group block in the page row index
(crate-digger `transaction_row_flags`, at `base`; in maple's `pager.py` it is
`pack("<H", 16) # unknown (rekordcrate default)`).
**Finding (5 genuine rekordbox exports):** this u2 is the
**"rows touched by the last transaction" bitmask** (same bit layout as
`row_present_flags`), and is therefore **edit-history state, not reproducible**:
- **No genuine rekordbox export writes 16 as the field's value** on settled
pages (count of groups == 16 across the five exports: 0, 0, 0, 1, 8 — and
those few are coincidental transaction bitmasks, not a default). So maple's
constant **16 does NOT match rekordbox** — it is a rekordcrate invention.
- The dominant value in heavily-edited tables is **0x0** (last transaction
touched no rows on that page), e.g. data/PIONEER 170/294 groups = 0x0.
- Three DIFFERENT patterns coexist in ONE file, proving it is not structural:
(a) some pages `trailing == present_flags` (columns/unk17/unk18); (b) artwork
pages consistently use `last group = 1<<(rows_in_last_group-1)`, earlier groups
= 0 (data/PIONEER p28 g6 10rows→0x200, p147 g2 13rows→0x1000; acemo_fresh
artwork 7rows→0x40); (c) churned tracks pages use an arbitrary single bit
(0x1/0x2/0x4/0x8/0x10/0x40…) = whatever row that page's last edit touched.
A candidate clean rule `[last grp=1<<(rows-1), others=0]` matches only
**181/257** fully-present pages in data/PIONEER — confirming NO single formula.
- **Same-library proof:** acemo and acemo_fresh hold identical data, yet the
tracks page-2 group differs (acemo present=0x5e trailing=0x20; acemo_fresh
present=0xbb trailing=0x4) — same rows, different value ⇒ not structural.
The previously-flagged artwork `0x40` is likewise transaction state
(acemo_fresh artwork=0x40 vs lib_multi artwork p49=0x4), not an open
reproducible byte.
**Rule for maple (DECISION): write `present_flags`, NOT the constant 16.**
maple's old constant 16 (rekordcrate's placeholder) matches **0/14** real groups;
`trailing = row_present_flags` matches **12/14** clean groups in the real
`data/acemo_fresh` export (8 rows→0xff, 16→0xffff, 11→0x7ff, 6→0x3f, 1→0x1). The
2 misses (page2→0x4, page28→0x40, page40→0xc0) are the last-transaction bitmask
on pages that were edited after creation — i.e. residual edit history, not a
different rule. So for a from-scratch single-transaction write the correct value
IS `present_flags` (all rows touched in the one creating transaction). maple now
ships `present_flags` (pager.py) — this is the best achievable; reverting to 16
would regress 12 groups. Remaining mismatch vs an EDITED reference (the per-page
last-tx bitmask) is unavoidable edit history → KNOWN_DIVERGENCE.
> NOTE: the earlier "keep 16 / fresh usb writes 0x10 everywhere" framing was
> WRONG — it read `data/usb`, which is MAPLE'S OWN OUTPUT (an old build that
> emitted the rekordcrate-default 16), not a real export. `data/usb` must NOT be
> used as a 'fresh rekordbox' baseline for ANY field (same contamination that
> produced the wrong opaque-blob default in §2). The fresh-write value is
> established from the REAL `data/acemo_fresh` as `present_flags`.
**Confidence:** high that it is history-dependent and that real rekordbox never
uses 16; the correct fresh-write value is an open question (no seq==1 sample).
---
## Note on the three "reproducible header fields" (task #9)
After multi-export analysis across the FIVE genuine rekordbox exports (acemo,
acemo_fresh, lib_multi, MAPLE, data/PIONEER — provenance verified by nonzero
per-track opaque ids + per-install seed), all three fields are
**edit-history / transaction state, NOT structurally reproducible**:
- §4 file-header 0x10: allocator/edit accumulator. → write 0, DIVERGENCE.
- §5 page 0x26 unknown7: 0 on all data pages (exact); per-table 0/1/2 on strange
pages is edit state. → write 0, strange-page DIVERGENCE.
- §6 row-group trailing u2: "rows touched by last transaction" bitmask. Fresh
single-transaction value = `present_flags` (12/14 in real data/acemo_fresh;
constant 16 = 0/14). → maple writes `present_flags`; residual mismatch vs an
EDITED ref is the per-page last-tx bitmask = edit history, DIVERGENCE.
The decisive evidence is the **same-library pair** (acemo vs acemo_fresh:
identical track data, yet header 0x10 = 4 vs 5, tracks strange unk7 = 2 vs 1,
tracks-page trailing differs) — a structural field would be identical.
maple's values: §4 header0x10 = 0 and §5 unknown7 = 0 are reasonable minimal
defaults (confirmed = real exports on data pages; fresh strange-page value
unconfirmed). §6 trailing u2 = `present_flags` is the established fresh-write
value (NOT the old constant 16). No further writer change needed; remaining
mismatches vs edited references are unavoidable edit history.
IMPORTANT corrections to the earlier draft: (1) `data/usb/.../export.pdb` is
maple's OWN output (an old build that emitted the rekordcrate-default 16), not a
rekordbox export — so any "fresh usb" claim was circular; `data/usb` is NOT an
authority for ANY field (same contamination that produced the wrong opaque-blob
default in §2). (2) We have NO genuine rekordbox export with sequence==1, so the
never-edited header0x10 / strange-page-unknown7 values are unconfirmed. (3) The
row-group trailing u2 fresh value is `present_flags`, established from the REAL
data/acemo_fresh (12/14); the rekordcrate "Default=16" is a placeholder that
matches 0/14 real groups.
All three go to re/KNOWN_DIVERGENCE.md, not the writer.
---
## 7. Artwork + USBANLZ path schemes (task #11)
Decoded from genuine `data/MAPLE` (176 tracks/176 artwork) + acemo. The guiding
principle (per the task): for **player acceptance only PATH SELF-CONSISTENCY
matters** — every path stored in the pdb must resolve to a real on-disk file.
Matching rekordbox's exact id-derivation is NOT required; maple's own consistent
scheme is acceptable.
### 7a. Artwork — SOLVED, exactly reproducible (maple already matches)
- Path: `/PIONEER/Artwork/{folder:05d}/a{artwork_id}.jpg` (80×80) plus a
`a{artwork_id}_m.jpg` (240×240) variant in the same folder (rekordbox writes
both; verified on disk in data/MAPLE).
- **Folder bucket rule: `folder = artwork_id // 20 + 1`** (20 ids per bucket;
bucket 00001 = ids 1–19 because ids are 1-based, 00002 = 20–39, 00003 = 40–59,
…). Verified **176/176** in data/MAPLE.
- maple's `artwork._bucket()` is literally `f"{artwork_id // 20 + 1:05d}"` and
`extract_artwork` writes both jpg variants at that path → **byte-identical to
rekordbox AND self-consistent**. No change needed.
### 7b. USBANLZ — opaque id; maple's own scheme is self-consistent (DIVERGENCE from rekordbox)
- Path: `/PIONEER/USBANLZ/P{bucket}/{8-hex}/ANLZ0000.DAT` (+`.EXT`).
- rekordbox's `{8-hex}` is a per-track **opaque analysis id** (range seen
0x117..0x309fd over 176 tracks; NOT == track id, NOT == the row-0x14
`opaque_track_id_lo`, NOT a crc of path/title — tested) and the `P{bucket}`
(0–127, 94 distinct over 176) is a spreading bucket that is **NOT a derivable
function** of the 8-hex or any track field (tested: `&0x7f`, `>>k`, `%127/128`,
low-byte — all fail). Like the row-0x14 blob, these come from rekordbox's
internal DB and are **not reproducible** without it.
- maple (cli.py `build()`): generates its OWN deterministic scheme —
`bucket = P{(crc32(file_path) >> 20) & 0xFFF:03X}`,
`analyze_id = {crc32(file_path):08X}`, then `write_anlz()` writes the .DAT at
exactly that path. This is **self-consistent by construction** (the path it
records is the path it writes) and is the correct approach per the
self-consistency principle. It does NOT match rekordbox's id/bucket (expected
DIVERGENCE).
- Minor cosmetic note (not a correctness issue): maple's bucket is 3 hex
digits `P000..PFFF` whereas rekordbox uses `P00..P7F` (0–127). Harmless;
optionally narrow to `& 0x7F` (`P{..:02X}`) to look more rekordbox-like.
### 7c. Self-consistency verification
- **Genuine rekordbox (data/MAPLE): 176/176 ANLZ paths and 176/176 artwork
paths resolve to on-disk files.** (rekordbox is fully self-consistent — sanity
check that confirms the parser and the principle.)
- **maple production `build()` (cli.py): self-consistent BY CONSTRUCTION** —
`extract_artwork` (artwork.py:62-65) writes `a{id}.jpg`/`a{id}_m.jpg` at the
recorded path; `write_anlz` (cli.py:93) writes `ANLZ0000.DAT` at the recorded
`analyze_path`. Each pdb path is computed and the file is written there in the
same step, so every path resolves.
- CAVEAT — the byte-diff harness `tools/build_acemo.py` is NOT the production
path: it MIRRORS a reference rekordbox export (copies its `analyze_path`/
artwork `path` strings verbatim, writes no ANLZ/artwork files), so the
`data/usb` export it produces references the reference's USBANLZ files which
are not co-located → those paths don't resolve. This is expected for a
structural-diff fixture and says nothing about production self-consistency.
**Confidence:** high. Artwork rule is exact (176/176). USBANLZ rekordbox ids are
opaque/non-reproducible (consistent with the row-0x14 finding); maple's own
USBANLZ scheme is self-consistent, which is all the player requires.
---
## 8. Page-manager free-space: `empty_candidate` + free pool (task #13 — THE FREEZE)
DECK TEST: maple's pdb FROZE the player (parser hang, not reject). Decoded the
page-manager/free-space divergence against genuine rekordbox exports
(`data/working` 9trk seq44, `data/MAPLE` 176trk, `data/PIONEER` 847trk — all
load) vs maple output (`captures/maple_acemo.pdb`).
### Data-page `next_page` SELF-LOOP — ROOT CAUSE (confirmed divergence; high confidence)
- The player reads each table by following `first_page → next_page → …`. maple's
**last data page of each multi-page chain set `next_page == its own index`**
(page 2→2, 6→6, 8→8, 12, 14, 28, 34, 36, 38, 40 — 10 self-loops in the 9-track
build). A page whose `next_page` is itself is an **infinite loop → FREEZE**.
- **Genuine rekordbox has ZERO self-loops.** A terminal data page uses
`next_page = 0` (the "no next" sentinel; seen on pages 10/16/18/20/…/41-47 in
`data/working`) OR points FORWARD to a distinct free page (never to itself).
- FIX (maple, done in writer.py): point every chain's last page at a single
dedicated zero-filled free-pool page (`free_sentinel`, just past the overflow
region) instead of itself; materialise that page and set `next_unused` beyond
it. Verified on the rebuilt pdb: 10 self-loops → 0; all 20 chains terminate at
the blank sentinel; player tolerates last-page→free-page (genuine exports do
the same). This is the primary freeze fix.
### Table-dir `empty_candidate` (u4 after page_type) — SECONDARY divergence (also wrong; lower-confidence freeze contributor)
- Layout: each 16-byte table-dir entry is `type:u4, empty_candidate:u4,
first_page:u4, last_page:u4`.
- `empty_candidate` is the head of that table's **empty/free-page list** (the
page the Page Manager grabs when the table needs to grow).
- **In EVERY genuine rekordbox export all 20 `empty_candidate`s are NONZERO and
distinct**, and each points to an **all-zero "free" page** — either an
in-file allocated-but-empty page (e.g. `last_page+1`, or a pooled page like
41–47/49) or a **beyond-EOF sentinel** (index ≥ npages, e.g. 50/51/52 when
npages=49 — the firmware tolerates beyond-EOF here). It NEVER points to a
used data page and NEVER to 0.
- **maple writes `empty_candidate = 0` for all 20 tables.** Page 0 is the FILE
HEADER. The firmware Page Manager, walking a table's empty-page list, follows
`0` → reads the 4 KB file header AS a page → its "next_page" field (header
offset 0x0c = `next_unused_page`) and bogus type/flags send the walk into a
loop or out-of-bounds read → **hang/FREEZE**. This is the highest-confidence
cause: it is the only systematic, data-independent divergence, and `0` is
never valid (collides with the header).
### Free pool / page layout
- rekordbox keeps the data pages in chains `1..40,48,…` and pre-allocates a
pool of all-zero "free" pages past them; `next_unused_page` = one past the
highest allocated page (so `next_unused > npages` is normal, e.g. 53 > 49).
- maple **compacts**: 41 pages, every page used, `next_unused == npages == 41`,
no free pool. The compaction itself is structurally legal (chains are clean),
but it left `empty_candidate = 0` with no free page to point at.
- Beyond-EOF links (a page's `next_page` ≥ npages, or `empty_candidate` ≥
npages) are TOLERATED — the genuine `working` file has many (page 4→53,
46→52, empty_candidate 51/52/50/49 all ≥ 49). So beyond-EOF is a valid
"no-more-pages / no-free-page" sentinel; it is NOT the freeze.
### Strange-page free-map — NOT the cause (decoded, maple matches)
- Each table's first/"strange" page (flags 0x64) carries, after a small prefix
`[this_page, next_or_0x03ffffff, 0x03ffffff, 0, 0x1fff0000]`, a heap filled
with **1004 × `0x1ffffff8`** (`f8 ff ff 1f` LE) — the empty-heap free-map
(count == the `m1004`=1004 marker at header 0x24). `0x03ffffff` in the 2nd u4
is the legitimate **"this table has no data page"** sentinel (genuine
rekordbox writes it for every empty table; e.g. labels/pltree/plentries).
- maple reproduces this free-map correctly; its strange-page prefixes match
rekordbox's pattern byte-for-byte (data table → u1=first_data_page; empty
table → u1=0x03ffffff). **So the free-map is NOT the freeze.**
### FIX (for maple → deck test)
PRIMARY (high confidence, DONE): **eliminate the data-page `next_page` self-loop**
— last page of each chain must point to a blank free page (or use `next_page=0`),
never to itself. maple now points all last pages at one zero-filled `free_sentinel`
page. Verified: 0 self-loops in the rebuilt pdb.
SECONDARY (cheap, also a real divergence — fold into the same build): **stop
writing `empty_candidate = 0`.** Point it at the same `free_sentinel`
(`pack("<I", free_sentinel)` instead of 0). Genuine rekordbox NEVER writes 0
(0 = the header page); it always points each table's empty_candidate at a blank
free page. Lower-confidence as a freeze cause for a read-only player than the
self-loop, but wrong and trivially fixable with the page maple already
materialised.
If the deck STILL hangs after both, next suspects (lower confidence):
`next_unused_page == npages` (give a little slack), and whether the single shared
sentinel needs to be per-table-distinct (rekordbox uses distinct free pages).
**Confidence:** HIGH that the self-loop was the freeze (self-referential
next_page is an unconditional infinite loop; 0/all genuine exports self-loop).
empty_candidate=0 is independently wrong (0/all genuine exports use it; collides
with header). Exact firmware behaviour is inferred (no firmware to trace); the
deck test confirms.
is the confirmation.
+167
View File
@@ -0,0 +1,167 @@
# KNOWN_DIVERGENCE.md — fields maple's writer intentionally does NOT match
This is the canonical list of bytes that differ between maple's
`captures/maple_acemo.pdb` and the AceMo reference
`data/acemo/PIONEER/rekordbox/export.pdb` **on purpose**, because they are
*history-dependent* artifacts of the particular reference file rather than rules
of the `.pdb` format.
## TL;DR
The AceMo reference `export.pdb` is **not a fresh from-scratch export — it is an
incrementally-edited rekordbox database.** rekordbox's DeviceSQL page manager
edits rows *in place*: when a row changes it appends a new copy, clears the old
slot's presence bit, and advances `used_size` past the dead bytes **without
compacting**. The reference therefore carries the residue of its entire edit
history.
maple writes a **fresh, fully-compacted** database (tightly 4-byte-aligned rows,
no dead rows, no leftover slots). That output is byte-correct for a *from-scratch*
rekordbox export, but it cannot — and should not — reproduce another file's edit
history. The fields below are the consequences of that, and the team has decided
**not to chase them**. They only become matchable against a fresh from-scratch
reference export (export to an empty USB, never edited afterward).
Everything NOT listed here is fair game and must match. The task-#7 diff should
filter out (and reference) this list, then drive the remaining bytes to zero.
## Evidence (tracks table, page 2 of the reference)
5 present rows, each with ~377–417 bytes of real content, yet:
| row id | ofs | real content len | next ofs | gap | gap bytes are… |
|--------|-------|------------------|----------|-------|---------------------------|
| 1 | 428 | 377 | 856 | 51 | zero |
| 5 | 856 | 377 | 1324 | 91 | partial stale string |
| 2 | 1324 | 417 | 1792 | 51 | zero |
| 3 | 1792 | 386 | 2656 | 478 | a whole dead track row |
| 4 | 2656 | 404 | 3996* | 936 | two whole dead track rows |
`used_size=3996`, `free_size=38`. The gap after id=3 holds a dead copy of a row
ending `"...Jelly Fish Jam.mp3"`; the gap after id=4 holds dead copies of
`"Myrtle Ave Party Track"` and `"Smooth Move"` rows. These are previous versions
of edited rows, left uncompacted. (Verified by ASCII-dumping the gap bytes.)
## The fields we do NOT match (and why)
1. **File / page count and `next_unused_page`.**
Reference: 49 pages, `next_unused=53`. maple: 41 pages, `next_unused=41`.
The reference is larger purely because dead-row bloat pushed live rows onto
extra pages. A compacted export needs fewer pages.
2. **Overflow / data-page placement (tracks `last_page`).**
Reference tracks table spans pages **1 → 48** (live track id=7 spilled onto
overflow page 48 because dead rows "filled" page 2). maple spans **1 → 2**
(all 6 tracks fit one compacted page). This is the *only* table-directory
entry that differs — the other 19 tables match first/last page exactly. The
divergence is entirely a function of the dead-row bloat above.
3. **File-header `sequence` value.**
Reference: `353`. maple: an incrementing counter (reads the destination
export's sequence + 1, default 1; see `writer._next_sequence`). rekordbox's
value comes from its internal `master.db` transaction counter and is not
reproducible from content. We match the *semantics* (monotonic increment),
not the absolute value.
4. **Per-page `sequence` (page-header offset 16).**
Reference pages carry assorted values (e.g. 26, 45, 320, 342, 347) — each a
snapshot of the global counter at the moment that page was last rewritten.
maple writes a clean per-build value. Same root cause as (3); not reproducible.
5. **`transaction_row_count` / `transaction_row_index` (page-header u2 pair).**
Reference: nonzero per edited page (e.g. `1/5`, `1/8`, `2/0`). maple: `0/0`.
These track the last in-place transaction on the page; meaningless for a
freshly written, never-edited page.
6. **Dead-row offset arrays + `num_row_offsets` > `num_rows`.**
Reference row groups list offsets for dead slots too: e.g. tracks page 2 has
`nro=9, nrows=5` with offsets `[0,428,856,1324,1792,2224,2656,3112,3568]`
(slots 0,5,7,8 dead); artists page 6 has `nro=9, nrows=1`. maple emits exactly
`nro == nrows` with no dead slots. Reproducing dead offsets would mean
reproducing the edit history.
7. **Live row in a non-zero slot.**
Reference artists page 6 keeps its single live row in **slot 8** (slots 0–7
are dead). maple places the single live row in **slot 0**. A from-scratch
export also writes slot 0.
8. **`free_size` / `used_size` totals on data pages.**
Reference `used+free` totals vary (4034, 4050, …) because `used_size` covers
the dead-row region and the strange-block markers differ. maple's pages total
a clean `4020` (= heap capacity for a single 36-byte row group). Direct
consequence of (6) — dead bytes are counted as "used".
9. **Track row opaque id blob — `0x14`/`0x18`/`0x1a` (8 bytes/track).**
One 8-byte LE value `opaque_track_id_lo | (opaque_seed << 32)`, both halves
stored in rekordbox's internal master.db (a per-install `opaque_seed` + a
per-track `opaque_track_id_lo`). Proven non-derivable from track metadata
across every real export + an exhaustive negative hash search
(FORMAT_SPEC.md s2). **Every real rekordbox export has BOTH halves non-zero**
(this install's seed = `0x21b4a7ab`; `data/MAPLE` = `0xd6d3a567`); the
"unanalyzed default" lo=0/seed=0x78f74a68 was a false lead — it came from
maple's own earlier output (`data/usb`), not a real export. maple emits a
plausible non-zero stand-in of the right shape: `opaque_seed = 0x21b4a7ab`
(module const `TRACK_OPAQUE_SEED`, overridable per install) and a stable
per-track `opaque_track_id_lo` mapped into the real Content-ID band
(`OPAQUE_LO_MIN..MAX` = 0x08000000..0x0F900000, where every observed AceMo lo
lands) via `crc32(id|file_path)`. This will NOT match rekordbox's master.db
id byte-for-byte — unreproducible without its DB — but it is non-zero,
in-range and deterministic, the safest shape for deck acceptance. Players
treat the blob as opaque.
10. **Strange-page free-map extension + per-page seq on tracks/history.**
On the tracks (type 0) and history (type 19) strange pages only, the
free-space map carries an extra accounting word (body offset 16: `01 00`,
offset 20: a u4 = the data page's used_size for history=320; tracks differs
because of its dead-row/overflow state). maple writes the plain free-map.
Tied to the same content/overflow state as items (1)/(2); not reproducible
for a compacted writer. (The rest of every strange page — header, free-map
head, 1004× `f8 ff ff 1f`, the data-page pointer word, and unknown7 — now
matches the fresh reference byte-for-byte.)
11. **Table-directory `empty_candidate` (u32, table_dir entry +4) + header
`next_unused_page`.** Reference: a nonzero per-table `empty_candidate`
(e.g. 0x33, 0x04, 0x2e…); maple: 0. rekordcrate documents this field as
"maybe links to a chain of empty pages if the database is ever garbage
collected", and its fixtures show non-sequential free-pool page indices
(47, 4, 49, 8, 50, 46, 42, 16…). It is free-list/GC allocator state, not
derivable from content. `next_unused_page` (rekordcrate: "points past end of
file", unused) is the same class — see also item (1).
> Canonical reference note: the diff target is now the FRESH export
> `data/acemo_fresh/PIONEER/rekordbox/export.pdb` (7 tracks). It is cleaner than
> the edited `data/acemo` (6 tracks) but still incrementally built per-file, so
> the divergences above still apply — just smaller (e.g. track 7's dead-row span
> is 464B not 3568B, but it still overflows to page 48).
## What IS reproducible (NOW matched against the fresh reference)
- The 20-table directory layout and page placement: an empty table is
strange-page-only; a table gets a data page IFF it has rows (`CORE_TABLES` is
now empty — the old genres data page was an edit-history artifact of the
edited ref).
- `page_flags`: data pages are `0x34` for **tracks and history only**, `0x24`
for every other table (the wider `{0,2,3,13,17,19}` was edited-ref noise);
empty data pages `0x44`; strange pages `0x64`.
- Strange-page body: the full DeviceSQL free-space map — `u4(page_index)`,
`u4(first_data_page or 0x03ffffff)`, the 12-byte const tail, 1004×
`f8 ff ff 1f`, 20 zero bytes — plus header `free=used=0`, `txc=txi=0x1fff`,
`next_page = page_index+1`, and `unknown7 = 1` for tracks/history else 0.
- Track row `index_shift` (`slot_index<<5`) and every other live-row content
field except the opaque blob in divergence item (9).
- Three header fields `re` decoded (FORMAT_SPEC §4/§5/§6) as edit-history state
whose FRESH-export value maple already writes — so they're matched on a fresh
export and only diverge against edited refs:
- file-header offset 16 (`_unnamed4`): fresh=`0` (maple writes 0); edited DBs
accumulate a small nonzero with no structural meaning.
- strange/data-page `unknown7` (0x26): fresh=`0` everywhere (maple writes 0);
nonzero only on edited DBs' strange pages.
- row-group trailing u2: it is the "rows-touched-in-last-transaction" bitmask.
On a fresh export it equals the row-present mask, so maple writes
`presence` (matches 12/14 groups in `data/acemo_fresh`; the 2 misses are
edited pages — divergence). NOTE: `re`'s "keep constant 16" advice came from
reading `data/usb`, which is maple's OWN output, not a real export — a real
fresh export uses the presence value, not 16.
See `re/FORMAT_SPEC.md` for the value-construction rules and `re/README.md` for
the open RE questions.
+54
View File
@@ -0,0 +1,54 @@
# re/ — rekordbox `export.pdb` reverse-engineering
Shared record of the Ghidra/Frida work behind maple's pdb writer. See the
top-level `README.md` for the high-level findings; this dir holds the raw
artifacts so others can reproduce and extend the analysis.
## What's here
`scripts/` — Ghidra headless scripts (run against an analyzed `rekordbox.exe`):
- `FindPdb.java` — locate the writer via string + magic-constant xrefs
- `DecPdb.java` — decompile an explicit list of addresses
- `CallTree.java` — walk a call tree, flag file-I/O / 4096-using functions
- `FindHash.java` — hunt the per-track hash: decompile `formatForSearch` + callers + content-string xrefs
`findings/` — script output (decompiled C + xref maps):
- `findpdb.txt` — string xrefs → the export functions; magic-constant hit counts
- `decpdb.txt` — decompiled `export.pdb` serializer call stack (DeviceSQL Page Manager)
- `calltree.txt` — call tree under the export orchestrator `FUN_14119d2d0`
- `hash.txt` — `formatForSearch` + callers + `djmdContent`/`ContentID` xrefs
## Reproduce (not committed: the binary + Ghidra project)
`rekordbox.exe` and the analyzed Ghidra project are **not** in the repo
(copyright + size). To rebuild from your own rekordbox install (Windows path
visible from WSL):
```bash
export JAVA_HOME=~/jdk/jdk-21.0.11+10 # Ghidra needs JDK 21
GH=/path/to/ghidra_12.1.2_PUBLIC
# one-time: import + analyze (~1h, 10G heap, 8 cores)
GHIDRA_HEADLESS_MAXMEM=10G "$GH/support/analyzeHeadless" ~/ghidra_proj rbx \
-import "/mnt/c/Program Files/rekordbox/rekordbox 7.0.5/rekordbox.exe" \
-overwrite -max-cpu 8 -analysisTimeoutPerFile 7200
# then run any script without re-analyzing:
"$GH/support/analyzeHeadless" ~/ghidra_proj rbx -process rekordbox.exe -noanalysis \
-scriptPath re/scripts -postScript FindPdb.java
```
Runtime→Ghidra address map: `static = 0x140000000 + module_offset` (Ghidra image
base `0x140000000`; ASLR runtime base varies — read `rekordbox.exe+0xOFFSET` from
the Frida agent, never hardcode it).
## Live capture (Frida, Windows)
`../tools/capture.mjs` (Node, `npm i frida`) + `../tools/rbx_pdb_capture.js`
(agent). Child-gates so the CIG-protected writer is hooked at spawn:
`node tools/capture.mjs --spawn`.
## Open questions
- per-track `0x14` (28-bit) — varies per track, not a plain crc32; likely a
stored `djmdContent` value. `0x18`/`0x1a` are a per-export constant.
- `index_shift` (row `0x02`) — positional (`~0x20`/row, resets per page) but the
exact formula isn't pinned. maple writes `0`; rekordbox doesn't.
+470
View File
@@ -0,0 +1,470 @@
# REFERENCE_COMPARISON.md — pyrekordbox & rekordcrate vs maple
What the two mature open-source rekordbox libraries can decode/encode, how
that lines up with maple's writer and our RE findings, and the concrete GAPS
maple must close for a full from-scratch USB export.
- **pyrekordbox** (Python, dylanljones) — https://pyrekordbox.readthedocs.io
- **rekordcrate** (Rust, Holzhaus) — https://holzhaus.github.io/rekordcrate/
local source at `/mnt/c/Users/ntr/code/rekordcrate/src`
- **maple** — `maple-py/src/maple/*`, this `re/` dir.
Scope note (coordination): the per-format **library-capability** comparison and
the from-scratch-export gaps are mine (task #12). The master.db SQLCipher key
mechanics and opaque Content ID identity are owned by `re` (#10); the exact
USBANLZ/artwork on-disk path schemes by `re` (#11). Where those are cited below
they are tagged `[see re #10]` / `[see re #11]` and should be treated as
`re`'s authority, not re-derived here.
---
## 0. One-screen capability matrix
| Format | pyrekordbox | rekordcrate | maple |
|---|---|---|---|
| **export.pdb** | parse only (via the `pdb` parser, read) | **parse + WRITE** (`BinWrite` on rows/pages, full roundtrip) | **WRITE only** (from-scratch, the whole point) |
| **exportExt.pdb** | not supported | not supported | not written |
| **ANLZ (.DAT/.EXT/.2EX)** | **parse only** ("creating … is planned") | parse + *partial* write (binrw `BinWrite` derived on waveform/cue structs; not a whole-file builder) | **WRITE** (minimal: PMAI + PPTH only) |
| **artwork (.jpg thumbs)** | reads `ImagePath` from db only | n/a (just the pdb `Artwork` row path string) | **WRITE** (extracts embedded art) |
| **master.db (db6/7)** | **READ + WRITE** (SQLCipher unlock) | not supported | not used |
| **mysetting (\*SETTING.DAT)** | **READ + WRITE** (build from scratch) | parse only (`setting` module) | not written |
| **rekordbox XML** | **READ + WRITE** | parse only (`xml` module) | not used |
The headline: **no existing library writes a complete from-scratch USB export.**
rekordcrate writes `export.pdb` rows but not ANLZ/exportExt and has no
metadata source; pyrekordbox writes master.db + mysetting + XML but not the USB
`export.pdb`/ANLZ. maple is filling a real gap. The reference libs are most
valuable to maple as (a) a **read-back oracle** (parse maple's output and diff)
and (b) for pyrekordbox, a **source of the real opaque Content IDs** from
master.db.
---
## 1. export.pdb
### What rekordcrate does (the closest reference writer)
`pdb/mod.rs` decodes **and writes** every row type maple emits: Track, Artist,
Album, Artwork, Color, Genre, Key, Label, PlaylistTreeNode, PlaylistEntry,
ColumnEntry, HistoryPlaylist, HistoryEntry (`pub enum Row`, lines ~1064-1103).
It has a genuine page serializer: `write_page_contents` (mod.rs:212),
`write_options_and_get_row_offset` (mod.rs:463), per-row-type alignment
(`align_by`, mod.rs:1121-1133), and a `Track` `BinWrite` impl (mod.rs:934).
### Where rekordcrate CONFIRMS maple's layout (high-value confirmations)
- **Track fixed prefix = 0x5e bytes, then a 21×u2 (0x2a) offset block.**
rekordcrate asserts `start_of_string_section - base_position == 0x5e` and
seeks `0x2a` for 21 string offsets (mod.rs ~986-996). maple's
`TRACK_FIXED_PREFIX = 0x5E`, `TRACK_HEADER_LEN = 0x5E + 21*2`
(`rows.py:8-10`) and its 21-string list match field-for-field, including the
exact string order (isrc, …, analyze_path, analyze_date, comment, title,
unknown_string8, filename, file_path). **Independent confirmation of maple's
track row.**
- **Field order of the 0x5e prefix** is identical between rekordcrate's
`Track::write_options` and maple's `serialize_track`: unknown1/index_shift/
bitmask/sample_rate/composer_id/file_size/`unknown2`(=opaque lo @0x14)/
`unknown3`+`unknown4`(=seed @0x18/0x1a)/artwork_id/key_id/orig_artist_id/
label_id/remixer_id/bitrate/track_number/tempo/genre_id/album_id/artist_id/
id/disc/play_count/year/sample_depth/duration/unknown5/color/rating/unknown6/
unknown7.
- **DeviceSQL string encoding.** rekordcrate `pdb/string.rs` roundtrips the
same three forms maple's `strings.py` writes: short-ASCII
(`(len<<1)|1`), long-ASCII (`0x40`), long-UTF16LE (`0x90`). Its tests encode
`"I ❤ Rust"` as 0x90 and a long string as 0x40 — exactly maple's selection
rule. The empty string is `0x03` in both. **Confirms strings.py.**
- **Row-group / page-index mechanics.** rekordcrate's `RowGroup` (mod.rs:451,
2102) uses the same 16-rows-per-group, backward-growing offset index, and
presence bitmask maple builds in `pager.py`. `Header` (mod.rs:162) carries
the same 13-bit num_row_offsets / 11-bit num_rows packing maple writes
(`pager.py:131`).
### Where rekordcrate caught a maple BUG — `next_page` chain termination (the FREEZE)
rekordcrate `pdb/mod.rs` documents `Page::next_page` (lines 262-265) precisely:
> *"Index of the next page with the same page type. If this page is the last one
> of that type, the page index stored in the field will point **past the end of
> the file**."*
maple's writer originally set the **last** page of each table chain to
`next_page = its own page_index` (a self-pointer). That is exactly what
rekordcrate says must NOT happen — the last page must point *forward, past the
allocated pages*. A self-referencing `next_page` makes a firmware page-walk that
terminates on "next_page past EOF" **loop forever on the last page → the deck
freezes.** This is the root cause being fixed in task #13: `writer.py` now points
every last page at a dedicated free-pool sentinel page just past the overflow
region (`free_sentinel`, materialised zero-filled) and sets `next_unused_page`
beyond it — which removes the self-loop.
(rekordcrate's own `read_pages`, mod.rs:185-196, terminates the walk on
`page_index == table.last_page` from the directory, so its *parser* tolerates a
self-pointer — which is why maple's read-back oracle never flagged it. The
*firmware* uses the chain-walk, so only a real deck exposed the freeze.
Lesson: a clean rekordcrate parse is NOT proof the chain is firmware-valid.)
> **Verified against a real export — the EXACT rule (not just "past EOF").**
> I decoded `data/PIONEER/rekordbox/export.pdb` (847 tracks, 174 pages,
> `next_unused=177`). rekordcrate's "points past the end of file" is a
> simplification; the real, consistent rule across **all 20 tables** is:
> **each table's last/only page sets `next_page == that table's
> `empty_candidate`**, where `empty_candidate` (table-dir entry +4) is a
> *distinct, nonzero* page index `< next_unused` — the head of that table's
> free-page chain. Evidence: every last page's `next_page` equalled its own
> dir `empty_candidate` (type0 173→174=ec; type1 4→53=ec; type19 40→41=ec; …),
> and those targets (174,53,158,163,54,49,42,46,170,20,22,175,176,165,30,32,
> 43,44,45,41) are all `< next_unused`. Strange-only tables already match
> (next_page = page_idx+1 = their ec; free-map `data_ptr=0x03ffffff`) — maple's
> `build_strange_page` does this correctly.
>
> So maple's current #13 fix (a **single shared sentinel** for every data
> table + `empty_candidate=0` in the directory) removes the freeze but still
> **diverges** from real rekordbox, which uses a **distinct per-table
> `empty_candidate`** that the directory entry also records (nonzero). If
> firmware ever cross-checks `next_page` against the directory's
> `empty_candidate`, the shared sentinel + zero ec could still misbehave.
> Recommended (reported to `maple`): give each data table its own free page,
> set `next_page = that page` AND write that index into the table-dir
> `empty_candidate`, keep them all `< next_unused`. This is straightforwardly
> reproducible (unlike the *value* of `empty_candidate` on an edited ref, which
> is GC state — here we only need a valid distinct free page, which a
> from-scratch writer fully controls).
### Where rekordcrate is WRONG and maple is RIGHT — page row-count header (>255 rows)
Audited because rekordcrate's `Page` header (mod.rs:277-377) reads the row count
as **`num_rows_small: u8` @0x18** plus **`num_rows_large: u16` @0x22** with a
`calculate_num_rows` rule (`large if large>small && large!=0x1fff else small`).
maple instead writes the **canonical crate-digger .ksy bitfield** at 0x18:
13-bit `num_row_offsets` + 11-bit `num_rows`, then `page_flags` @0x1b
(`pager.py:131`). These are two different readings of the same bytes — so I
**verified against a real export** (`data/PIONEER`, which has 284-row
playlist-entry pages):
- Real page 18 bytes @0x18 = `1c 81 23 24`. The 24-bit LE value `0x23811c`
decodes (kaitai/maple way) to `num_row_offsets=284, num_rows=284` — **correct**.
- maple's packer reproduces `1c 81 23` **exactly** for n=284 (and `a6 c0 14` for
166, `7e c0 0f` for 126 — all matching real pages byte-for-byte).
- rekordcrate's reading of the *same* bytes: `num_rows_small = 0x1c = 28` (just
the low byte) and `num_rows_large @0x22 = 283` (which is really the
`transaction_row_index` edit field), giving `calculate_num_rows = 283` — i.e.
rekordcrate would report **283 rows on a 284-row page, and 28 if the
transaction field were 0.** Its small/large model only coincidentally agrees
when `n <= 255`.
- **Conclusion:** maple's header encoding is correct and firmware-faithful for
pages up to 2047 rows (the 11-bit max); rekordcrate's `num_rows_small`/
`num_rows_large` is an older/divergent decode that breaks for `n > 255`. This
matters because maple *can* emit >255-row pages (playlist_entries fits ~284/
page, colors ~394). No maple change needed — this is a maple-is-right result.
### Where rekordcrate's row alignment diverges (maple matches real rekordbox)
rekordcrate aligns each row by its Rust struct's natural `align_of_val`
(mod.rs:1114-1136), which would 2-align u16-only row types. maple aligns **every
row to 4** (`pager._align4`). Checked real exports: **every row in
`data/PIONEER` (3567/3567) and `rb_ref` (101/101) is 4-aligned.** So maple's
universal 4-alignment matches rekordbox; rekordcrate's per-type scheme is the
divergent one (its own code comments admit "we should survey all the different
kind of rows and ensure they're aligned properly", mod.rs:1110). No maple change
needed.
### Where maple's RE goes BEYOND rekordcrate (we know more than the reference)
These are fields rekordcrate leaves as opaque caller-supplied "unknown"s but
maple's RE solved/characterised — important because it means **rekordcrate
cannot be used to validate these; maple's FORMAT_SPEC is the authority**:
- **`index_shift` (row +0x02).** rekordcrate calls it `index_shift` but writes
whatever the caller stored — it does **not** know the value (mod.rs:104-105
"Unknown field, called `index_shift` by flesniak"). maple **derives** it as
`slot_index << 5` and patches it at page-build time (`pager.py:142-145`,
FORMAT_SPEC.md §1). maple is correct where rekordcrate is agnostic.
- **The 8-byte opaque blob (0x14 lo / 0x18+0x1a seed).** rekordcrate:
`unknown2: u32`, `unknown3: u16` ("always 19048?"), `unknown4: u16`
("always 30967?") — three blind unknowns it round-trips verbatim
(mod.rs:115-119). maple's RE (FORMAT_SPEC.md §2) proved this is one LE u64 =
`opaque_track_id_lo | (opaque_seed<<32)`, a per-track Content ID + per-install
seed stored in master.db, **non-derivable** from metadata, and that the
"19048/30967" constants are NOT universal (they only hold for some old
exports). maple emits a non-zero stand-in; the real values live in master.db
`[see re #10]`. maple's understanding is strictly deeper than rekordcrate's.
- **`bitmask` (0x06).** rekordcrate: opaque "unknown" round-tripped. maple
writes a fixed `788224` (0xC0700) — itself not yet RE'd to a formula, but
maple at least pins a known-good value.
- **Strange/"first" pages.** rekordcrate leaves the strange-page heap **zero**
(it round-trips parsed input); maple **constructs** the full DeviceSQL
free-space map from scratch byte-for-byte (`pager.build_strange_page`,
verified vs `data/acemo_fresh` and the real strange pages in data/PIONEER),
including the `f8 ff ff 1f`×1004 map, the `0x03ffffff` no-data sentinel, and
`unknown7=1` for tracks/history. `re` confirms maple is **strictly more
faithful than rekordcrate on strange pages.** This is from-scratch generation
rekordcrate never had to do.
- **RowGroup trailing u2.** rekordcrate writes a constant `16` placeholder
(`Default::default`, mod.rs:454) and does **not** decode it; `re` confirms
genuine rekordbox **never** uses 16 there — it is the "rows touched by the
last transaction" bitmask (FORMAT_SPEC §6). maple writes `present_flags` (the
single-transaction value), closer to real output; either way it is
edit-history state (KNOWN_DIVERGENCE §6), not a rekordcrate-validatable field.
### Where rekordcrate (and our RE) AGREE the field is unreproducible — with a caveat
rekordcrate documents `empty_candidate` ("maybe links to a chain of empty
pages if the database is ever garbage collected") and `next_unused_page`
("points past end of file") as allocator/GC state — matching
KNOWN_DIVERGENCE.md items 11 and 1. The **historical VALUE** on an edited
reference is indeed unreproducible. **BUT** my data-page audit (see the FREEZE
section above) found `empty_candidate` is **load-bearing structure**, not just
GC trivia: in data/PIONEER every table's last page has
`next_page == that table's empty_candidate`, a distinct free page `< next_unused`.
So a from-scratch writer must still emit a *valid* distinct `empty_candidate`
per table (which it fully controls) even though it can't match an edited ref's
specific value. **Confirms** these are correctly on
maple's divergence list, not bugs.
### GAP — none structural for export.pdb
maple writes a complete, players-accepted export.pdb. Remaining export.pdb
divergences from a reference file are all **edit-history artifacts**
(KNOWN_DIVERGENCE.md) that neither library can reproduce from content either.
The one content-level gap is the opaque blob (see §4).
---
## 2. exportExt.pdb — GAP (nobody writes it; maybe nobody needs to)
- **pyrekordbox:** no support.
- **rekordcrate:** no support (its `pdb` module is export.pdb only).
- **maple:** does not emit it.
Format facts (Deep Symmetry export analysis; reverse-engineered by Dominik
Stolz, crate-digger issue #11): `exportExt.pdb` is a *second, smaller*
DeviceSQL file beside `export.pdb` holding only two table types — **Tags
(type 03)** and **Tag Tracks (type 04)** — i.e. the "My Tag" categorisation
feature and its tag→track associations. It uses the same page/row container
format as export.pdb.
**Assessment / recommendation:** exportExt.pdb is **optional**. It only carries
the My-Tag feature; players that don't find it simply show no custom tags, and
tracks/playlists from export.pdb play normally. maple has no tag data to
populate anyway. **Do not block a v1 export on this.** If maple ever supports
tags, the existing `pager.py`/`PageBuilder` already produces the container
format — only two new row serialisers (tag row, tag-track row) and a second
`write_pdb`-style call writing the 2-table directory would be needed.
Confidence the file is skippable: high (it post-dates the core format and is
absent from older exports that still play).
---
## 3. ANLZ (.DAT / .EXT / .2EX) — the biggest real GAP
### What the references decode
- **pyrekordbox** (`pyrekordbox.anlz`): full **parser** for PMAI + all section
tags — PPTH, PQTZ/PQT2 (beat grids), PCOB/PCO2 (cues/loops), PVBR, PSSI
(phrase/song-structure, XOR-masked), PWAV/PWV2 (previews), PWV3/4/5
(detailed + colour waveforms), PWV6/7/PWVC (.2EX). Writing is **"planned …
but the full structure has to be understood first"** — i.e. **read-only
today.**
- **rekordcrate** (`anlz.rs`): parses the same tag set; magic constants present
in source: PMAI, PPTH, PQTZ, PVBR, PWAV, PWV2, PCOB, PCO2, PCPT, PCP2, PSSI,
PWV3/4/5. It derives `BinWrite` on the waveform-column structs
(`WaveformPreviewColumn` etc., anlz.rs:368-428) and has a `write_encrypted`
for the XOR-masked PSSI (anlz.rs:902) — so it has *piece-level* write code,
but **no whole-file ANLZ builder / CLI to emit a valid .DAT/.EXT from track
audio.**
### What maple does
`anlz.py` writes a **minimal** `ANLZ0000.DAT`: a 28-byte PMAI header + a single
**PPTH** section (u4 len + UTF-16BE path). No beat grid, no waveform, no cues,
no `.EXT`, no `.2EX`. Rationale in the module docstring: firmware reads PPTH to
confirm which audio file the analysis belongs to, and a track whose
`analyze_path` points at *nothing* can be hidden from the library, so PPTH
alone makes the track "appear analysed enough to surface."
### CONFIRMATIONS
- maple's PMAI/PPTH layout (4-cc + u4 len_header + u4 len_tag; PPTH = u4
len_path + UTF-16**BE**, big-endian throughout) matches the format both
references parse. The big-endian-for-ANLZ-vs-little-endian-for-pdb split that
`anlz.py` calls out is corroborated by both libraries.
### GAPS maple must close for full faithfulness
1. **No `.EXT` file.** Modern CDJs/XDJs (NXS2+) read `ANLZ0000.EXT` for colour
waveforms (PWV4/5), extended cues (PCO2), and phrase data (PSSI). Without it
maple tracks show **no colour waveform and no phrase/structure** on newer
players. The `analyze_path` in the pdb points at the `.DAT`; the player
derives the `.EXT` sibling. maple writes neither beyond the `.DAT` PPTH stub.
2. **No beat grid (PQTZ).** Without a beat grid, beat-sync / quantize / loop
features have nothing to lock to; the track plays but degrades the CDJ
experience. This is the highest-value ANLZ section to add after PPTH.
3. **No waveform (PWAV/PWV3-5).** Cosmetic but conspicuous — tracks show a flat
or absent waveform. pyrekordbox can *parse* these to learn the exact byte
layout, and rekordcrate's `BinWrite` waveform-column structs are a concrete
reference for the encoding, but **maple must generate the waveform data
itself** (decode audio → downsample → colour-band), which neither library
provides.
4. **Generation, not just layout, is the hard part.** The byte format is fully
documented (both libs + Deep Symmetry); what's missing everywhere is the
DSP that *produces* beat positions and waveform columns from PCM. That DSP
is maple's to build — the references only validate the container.
**Recommendation:** keep the PPTH-only `.DAT` as the working baseline (it is
what gets tracks visible), and treat PQTZ beat grid + a `.EXT` with PWV5 colour
waveform as the next ANLZ milestones. Use pyrekordbox's parser as the
round-trip oracle (write a section, parse it back, compare). Confidence the
PPTH-only stub is sufficient for *visibility/playback*: medium-high (matches
maple's own deck testing rationale); confidence it's sufficient for a *complete*
experience: low — beat grid/waveform are expected by users.
---
## 4. master.db (db6/7) + the opaque Content IDs — GAP maple can close via pyrekordbox
### What pyrekordbox does (this is its core strength) — VALIDATED by re #10
- Reads **and writes** the encrypted `master.db`. The db is **SQLCipher4**
encrypted with a **single GLOBAL key, identical on every install** (not
machine/license-derived). `re` validated the concrete key (FORMAT_SPEC §2
Round-3): `402fd482c38817c35ffa8ffb8c7d93143b749e7d315df7a81732a1ff43608497`.
pyrekordbox 0.4.4 + `sqlcipher3-wheels` installs and wires up correctly, so a
**valid** master.db opens and exposes `DjmdContent` `[see re #10]`.
- Exposes `DjmdContent` (one row per track) with: **`ID`** (the rekordbox
internal Content ID), `FolderPath`, `FileNameL`, **`AnalysisDataPath`** (the
`/PIONEER/USBANLZ/...` analysis path), **`ImagePath`** (artwork), and FK ids
to Artist/Album/Genre/Label/Key/Composer; plus `djmdCue`, `djmdPlaylist`/
`djmdSongPlaylist`, `djmdHistory`, `djmdMixerParam`, `djmdDevice`, etc.
> **Reality check from re #10 — currently BLOCKED on data, not on method:**
> - The captured `maple-py/captures/p17328_master.db` is **corrupt/truncated**
> (zeroed SQLCipher salt + page 1) — no key can open it. The genuine
> `exportLibrary.db` shipped beside export.pdb uses a *different* key scheme
> that the global master.db key does not open, and pyrekordbox has no
> exportLibrary.db support. So the IDs have **not yet been read** on this RE
> machine — the path works only against a **live**
> `%APPDATA%/Pioneer/rekordbox6/master.db`.
> - `opaque_track_id_lo == DjmdContent.ID & 0xFFFFFFFF` is a **strong inference,
> still UNTESTED** (the lo range fits the 9-digit Content-ID band), not a
> confirmed equality.
> - The per-install **seed** (`0x21b4a7ab`) is **NOT** a DjmdContent column and
> its source is **still unidentified** (likely a library/USB UUID or
> agentRegistry/local_usn-style marker; needs a readable master.db to find).
> - pyrekordbox `add_content` lets *rekordbox* assign the Content ID — there is
> **no API to force a specific ID** — and it does **not** drive a USB export
> (`get_device` is read-only). So pyrekordbox is a **read source of real IDs**,
> not a writer/exporter and not a way to dictate IDs.
### Why this matters for maple (ties to FORMAT_SPEC §2 / KNOWN_DIVERGENCE #9)
maple's RE proved the export.pdb track blob at 0x14/0x18 is the rekordbox
**Content ID** (per-track, 28-bit band) plus a **per-install seed**, both stored
in master.db and **not derivable** from the audio/metadata maple has. maple
currently emits a non-zero *stand-in* (`crc32(id|file_path)` lo +
`TRACK_OPAQUE_SEED = 0x21b4a7ab`). Therefore:
- **GAP (closable in principle, not yet proven): real Content IDs.** If maple
runs on a machine with rekordbox installed, it can use **pyrekordbox + the
global key to open the LIVE `%APPDATA%/Pioneer/rekordbox6/master.db` and read
`DjmdContent.ID`** per track, then write those into the pdb blob instead of
the stand-in. `re` validated the *mechanism* (global key, DjmdContent schema)
but could NOT confirm `ID == opaque_track_id_lo` here because the only
captured master.db is corrupt — so this is a **strong-but-untested** lead, not
a closed item. Next step (re #10): point pyrekordbox at a live master.db and
confirm the equality for the AceMo tracks. The per-install **seed**
`0x21b4a7ab` source is still unidentified, so maple keeps the configurable
`TRACK_OPAQUE_SEED` either way.
- **Caveats:** (a) works only *with a rekordbox install present and a readable
master.db* (not the corrupt capture), and only for tracks rekordbox already
knows; (b) pyrekordbox `add_content` cannot **force** a chosen ID (rekordbox
assigns it) and does **not** drive a USB export, so it can supply IDs but
cannot replace maple's writer. Pure from-scratch (no rekordbox install) stays
a documented divergence (KNOWN_DIVERGENCE #9), with maple's non-zero stand-in.
### rekordcrate
No master.db support at all — it is a *device-export* (USB) library, not a
*library-database* library. So it cannot help with Content IDs.
---
## 5. mysettings (\*SETTING.DAT) — GAP (small, easy) maple doesn't write
- **pyrekordbox:** reads **and writes/builds** `DEVSETTING.DAT`,
`MYSETTING.DAT` (40B, CDJ play settings), `MYSETTING2.DAT` (40B), and
`DJMMYSETTING.DAT` (52B, DJM mixer) — dict-style get/set + save. (`DEVSETTING`
payload still partly ununderstood.)
- **rekordcrate:** `setting` module **parses** these (read-only).
- **maple:** does **not** write any of them.
These live in `PIONEER/` on the USB alongside `rekordbox/`. They are **optional
preference files** — absent, the player just uses its own defaults; tracks and
playlists are unaffected. **Recommendation:** low priority. If desired,
pyrekordbox can *build* default ones in a few lines (it's read/write), so maple
could shell out to / import pyrekordbox to drop standard `MYSETTING.DAT` +
`MYSETTING2.DAT` rather than re-implementing the 40-byte structs. Confidence
they're optional for playback: high.
---
## 6. Overall directory layout + path schemes (cross-check) `[see re #11 / FORMAT_SPEC §7]`
Both references and Deep Symmetry agree on the USB tree maple targets:
`/PIONEER/rekordbox/export.pdb` (+ optional `exportExt.pdb`),
`/PIONEER/USBANLZ/<...>/ANLZ0000.{DAT,EXT,2EX}`,
`/PIONEER/Artwork/<...>/a<id>.jpg`, plus optional `/PIONEER/*SETTING.DAT`.
maple writes `PIONEER/rekordbox/export.pdb` (`cli.py:156`) and
`PIONEER/USBANLZ/<bucket>/<id>/ANLZ0000.DAT` (`cli.py:92`).
`re` resolved both path schemes (#11, FORMAT_SPEC §7):
- **Artwork — EXACTLY reproducible, and maple already matches.** Real rekordbox
uses `/PIONEER/Artwork/{id//20+1:05d}/a{id}.jpg` (+ a `a{id}_m.jpg` 240px
thumb), 20 ids per bucket folder (bucket 1 = ids 1-19). `re` verified
176/176 in data/MAPLE, and maple's `artwork._bucket()` already implements this
rule. **Reproducible, confirmed — no gap.**
- **USBANLZ — NOT reproducible; maple's own scheme is correct-by-construction.**
Real rekordbox uses `/PIONEER/USBANLZ/P{bucket}/{8hex}/ANLZ0000.DAT` where the
8-hex is an **opaque per-track analysis id** (NOT the track id, NOT the row
0x14 lo, NOT any crc of the path — `re` ruled these out) and the P-bucket
(0-127) is **not a derivable function** of it. Like the 0x14 blob, it is
DB-sourced and non-reproducible. maple synthesises its own
`P{(crc32(path)>>20)&0xFFF:03X}/{crc32(path):08X}` and writes the `.DAT`
there, so the pdb's `analyze_path` and the on-disk file always agree. **This
is fine:** the player only requires every pdb path to resolve to a real
on-disk file, not to match rekordbox's id scheme.
> **Gotcha for the gaps below (from `re`):** `tools/build_acemo.py` MIRRORS a
> reference export's paths verbatim and writes **no** ANLZ/artwork files (it's a
> byte-diff fixture builder). The **production** `cli.py build()` is the one that
> actually writes each ANLZ/artwork file at the recorded path. Don't conflate
> the two when reasoning about what maple emits on disk.
---
## 7. Bottom line — prioritised GAP list for a full from-scratch export
1. **(high, lead — untested) Real opaque Content IDs via pyrekordbox/master.db**
— mechanism validated by `re` (global SQLCipher key `402fd482…` +
`Rekordbox6Database` reads `DjmdContent`), but `DjmdContent.ID ==
opaque_track_id_lo` is a **strong INFERENCE, NOT validated** (the only
captured master.db is corrupt — zeroed salt — so `re` never confirmed the
equality on real data). Would close KNOWN_DIVERGENCE #9 *if* confirmed against
a readable **live** `%APPDATA%/Pioneer/rekordbox6/master.db`.
`[see re #10 / FORMAT_SPEC §2 Round-3]`
2. **(OPEN — Ghidra trace pending) Per-install `opaque_seed` source.** The seed
(`0x21b4a7ab` / `0xd6d3a567`) is **NOT** a `DjmdContent` column and `re` has
**not located it** — likely a per-library UUID/agent/usn marker or a computed
value. It needs a readable master.db to inspect candidate rows, OR a Ghidra
trace of rekordbox.exe (which `re` can now attempt — the Ghidra MCP just came
online), so it is **not permanently blocked.** Until resolved, maple keeps the
configurable `TRACK_OPAQUE_SEED` stand-in. This is the one genuinely open
question touching the opaque blob. `[see re #10 — OPEN; FORMAT_SPEC §2 Round-3]`
3. **(high) ANLZ beat grid (PQTZ) + an `.EXT` colour waveform (PWV5)** — needs
DSP maple must build; references validate the byte format only. Biggest
functional gap on modern players.
4. **(structural — in progress) Per-table `empty_candidate` / last-page
`next_page`** — the FREEZE fix (#13). maple must point each table's last page
at a distinct free page recorded in the dir's `empty_candidate`, not a shared
sentinel with `empty_candidate=0`. See the FREEZE section in §1.
5. **(low) mysettings** — optional; pyrekordbox can build them.
6. **(low/skip) exportExt.pdb** — optional My-Tag data; only if maple adds tags.
7. **(reference, not a gap) Use rekordcrate as a read-back oracle** — parse
maple's `export.pdb` with rekordcrate's CLI to catch container regressions;
use pyrekordbox's anlz parser as the oracle for any new ANLZ sections.
**Caveat (proven this session):** rekordcrate's parser tolerates things the
firmware rejects (the `next_page` self-loop) and is itself wrong on some
fields (>255-row count, per-type alignment) — so it is a *regression* oracle,
NOT a correctness authority. Validate load-critical fields against REAL
export bytes (e.g. data/PIONEER), not against rekordcrate.
**Headline confirmation for the team:** rekordcrate's *writer* independently
validates maple's export.pdb track-row layout, DeviceSQL string encoding, and
page/row-group container down to the 0x5e/0x2a offsets — but it is *agnostic*
exactly where maple's RE added knowledge (`index_shift` formula, the opaque-id
semantics). So maple's FORMAT_SPEC is the authority for those fields, and the
remaining real work is **ANLZ generation** and **pulling the real Content IDs
from master.db** — neither of which any existing library does end-to-end.
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+595
View File
@@ -0,0 +1,595 @@
// HIT 0x4a68 @141f0a08b in FUN_141f09e70@141f09e70 : LEA RCX,[RDI + 0x4a68]
// HIT 0x4a68 @141f0a08b in FUN_141f09e70@141f09e70 : LEA RCX,[RDI + 0x4a68]
// HIT 0x4a68 @141f15d17 in FUN_141f15bf0@141f15bf0 : LEA RCX,[RBX + 0x4a68]
// HIT 0x4a68 @141f15d17 in FUN_141f15bf0@141f15bf0 : LEA RCX,[RBX + 0x4a68]
// HIT 0x4a68 @141f637b7 in FUN_141f63210@141f63210 : MOV RAX,qword ptr [RBX + 0x4a68]
// HIT 0x4a68 @141f637b7 in FUN_141f63210@141f63210 : MOV RAX,qword ptr [RBX + 0x4a68]
// HIT 0x4a68 @141f637be in FUN_141f63210@141f63210 : LEA RCX,[RBX + 0x4a68]
// HIT 0x4a68 @141f637be in FUN_141f63210@141f63210 : LEA RCX,[RBX + 0x4a68]
// HIT 0x4a68 @141f637da in FUN_141f63210@141f63210 : LEA RCX,[RBX + 0x4a68]
// HIT 0x4a68 @141f637da in FUN_141f63210@141f63210 : LEA RCX,[RBX + 0x4a68]
// HIT 0x4a68 @141f637e1 in FUN_141f63210@141f63210 : MOV RAX,qword ptr [RBX + 0x4a68]
// HIT 0x4a68 @141f637e1 in FUN_141f63210@141f63210 : MOV RAX,qword ptr [RBX + 0x4a68]
// --- decompiling containing functions ---
//===== FUN_141f09e70 @ 141f09e70 =====
undefined8 *
FUN_141f09e70(undefined8 *param_1,undefined4 param_2,undefined8 param_3,longlong param_4)
{
undefined8 *puVar1;
char cVar2;
undefined8 uVar3;
undefined8 *puVar4;
undefined8 *puVar5;
longlong lVar6;
undefined8 *puVar7;
undefined8 *puVar8;
longlong local_res20;
undefined1 local_f8 [4];
undefined8 local_f4;
undefined1 local_ec;
undefined4 local_e8;
undefined1 local_e0 [8];
undefined1 local_d8 [8];
undefined1 local_d0 [8];
undefined1 local_c8 [8];
undefined2 local_c0;
undefined1 local_b8 [8];
undefined1 local_b0;
undefined1 local_a8 [8];
undefined4 local_a0;
undefined1 local_98 [8];
undefined1 local_90 [8];
undefined1 local_88 [8];
undefined1 local_80 [8];
char *local_78;
undefined1 local_70 [16];
undefined1 local_60 [16];
undefined1 local_50 [24];
puVar8 = (undefined8 *)0x0;
uVar3 = FUN_1423d5980(&local_res20,param_3);
FUN_141f09b00(param_1,0xffffffff,uVar3);
FUN_14246e8b0(param_1 + 0x134);
*param_1 = djplay::HidMapCDJ2000NXS2::vftable;
param_1[0x14] = djplay::HidMapCDJ2000NXS2::vftable;
puVar1 = param_1 + 0x15;
*puVar1 = djplay::HidMapCDJ2000NXS2::vftable;
param_1[0x134] = djplay::HidMapCDJ2000NXS2::vftable;
param_1[0x138] = djplay::HidMapCDJ2000NXS2::vftable;
param_1[0x139] = djplay::HidMapCDJ2000NXS2::vftable;
param_1[0x13a] = djplay::HidMapCDJ2000NXS2::vftable;
*(undefined4 *)(param_1 + 0x13b) = 0x25f8;
param_1[0x1a3] = 0;
param_1[0x1a4] = 0;
param_1[0x1bb] = 0;
param_1[0x1bd] = 0;
param_1[0x1be] = 0;
param_1[0x1bf] = 0;
param_1[0x1c0] = 0;
*(undefined4 *)(param_1 + 0x1c1) = 0;
*(undefined2 *)((longlong)param_1 + 0xe0c) = 0;
*(undefined1 *)((longlong)param_1 + 0xe0e) = 0;
*(undefined4 *)(param_1 + 0x1c2) = 3;
puVar7 = param_1 + 0x1c4;
*puVar7 = 0;
param_1[0x1c5] = 0;
param_1[0x1c6] = 0;
param_1[0x1c8] = 0;
FUN_141ee27c0(param_1 + 0x1c9);
_eh_vector_constructor_iterator_(param_1 + 0x2d5,0x470,4,FUN_141ee3f50,FUN_141ee3fe0);
FUN_141ee3f50(param_1 + 0x50d);
FUN_141ee4d90(param_1 + 0x59c);
FUN_141ee5080(param_1 + 0x62b);
FUN_141ee5400(param_1 + 0x6ba);
FUN_141ee5970(param_1 + 0x74a);
FUN_141ee5db0(param_1 + 0x7da);
FUN_141ee6210(param_1 + 0x86a);
FUN_141ee97f0(param_1 + 0x8fb);
FUN_141ee9a10(param_1 + 0x907);
FUN_141ee9c10(param_1 + 0x915);
FUN_141ee9c10(param_1 + 0x920);
FUN_141ee9c10(param_1 + 0x92b);
FUN_141ee9c10(param_1 + 0x936);
FUN_141ee9e30(param_1 + 0x941);
FUN_141eea060(param_1 + 0x94d);
FUN_141eea150(param_1 + 0x959);
FUN_141eea330(param_1 + 0x964);
*(undefined2 *)(param_1 + 0x96f) = 0;
*(undefined1 *)((longlong)param_1 + 0x4b7a) = 0;
*(undefined4 *)((longlong)param_1 + 0x4b7c) = 0;
*(undefined1 *)(param_1 + 0x970) = 1;
param_1[0x197] = 0;
puVar4 = (undefined8 *)FUN_1423ce840(local_98,"CDJ-TOUR1");
cVar2 = FUN_1423d2890(param_3,*puVar4);
if (cVar2 != '\0') {
*(undefined1 *)((longlong)param_1 + 0xe0b) = 1;
}
*(undefined4 *)((longlong)param_1 + 0xe1a) = 0;
*(undefined1 *)((longlong)param_1 + 0xe1e) = 0;
LOCK();
*(undefined4 *)(param_1 + 0x1c8) = 0;
UNLOCK();
LOCK();
*(undefined4 *)((longlong)param_1 + 0xe44) = 0;
UNLOCK();
puVar5 = (undefined8 *)FUN_14217c644(0x40);
puVar4 = puVar8;
if (puVar5 != (undefined8 *)0x0) {
FUN_1423aa4f0(puVar5);
puVar5[6] = 0;
*(undefined4 *)(puVar5 + 7) = 8;
puVar4 = puVar5;
}
param_1[0x1c7] = puVar4;
lVar6 = FUN_14217c644(0x478);
puVar4 = puVar8;
if (lVar6 != 0) {
puVar4 = (undefined8 *)FUN_141ee41c0(lVar6,8);
}
param_1[0x59b] = puVar4;
local_f8[0] = 0;
local_f4 = 0;
local_ec = 0;
local_e8 = 0;
FUN_1423d59e0(local_e0);
FUN_1423d59e0(local_d8);
FUN_1423d59e0(local_d0);
FUN_1423d59e0(local_c8);
local_c0 = 0;
FUN_1423d59e0(local_b8);
local_b0 = 0;
FUN_1423d59e0(local_a8);
local_a0 = 0;
lVar6 = param_1[0x1c5];
if (lVar6 == param_1[0x1c6]) {
FUN_141edc340(puVar7,lVar6,local_f8);
}
else {
FUN_141068df0(lVar6,local_f8);
param_1[0x1c5] = param_1[0x1c5] + 0x60;
}
lVar6 = param_1[0x1c5];
if (lVar6 == param_1[0x1c6]) {
FUN_141edc340(puVar7,lVar6,local_f8);
}
else {
FUN_141068df0(lVar6,local_f8);
param_1[0x1c5] = param_1[0x1c5] + 0x60;
}
lVar6 = param_1[0x1c5];
if (lVar6 == param_1[0x1c6]) {
FUN_141edc340(puVar7,lVar6,local_f8);
}
else {
FUN_141068df0(lVar6,local_f8);
param_1[0x1c5] = param_1[0x1c5] + 0x60;
}
lVar6 = param_1[0x1c5];
if (lVar6 == param_1[0x1c6]) {
FUN_141edc340(puVar7,lVar6,local_f8);
}
else {
FUN_141068df0(lVar6,local_f8);
param_1[0x1c5] = param_1[0x1c5] + 0x60;
}
lVar6 = param_1[0x1c5];
if (lVar6 == param_1[0x1c6]) {
FUN_141edc340(puVar7,lVar6,local_f8);
}
else {
FUN_141068df0(lVar6,local_f8);
param_1[0x1c5] = param_1[0x1c5] + 0x60;
}
lVar6 = param_1[0x1c5];
if (lVar6 == param_1[0x1c6]) {
FUN_141edc340(puVar7,lVar6,local_f8);
}
else {
FUN_141068df0(lVar6,local_f8);
param_1[0x1c5] = param_1[0x1c5] + 0x60;
}
lVar6 = param_1[0x1c5];
if (lVar6 == param_1[0x1c6]) {
FUN_141edc340(puVar7,lVar6,local_f8);
}
else {
FUN_141068df0(lVar6,local_f8);
param_1[0x1c5] = param_1[0x1c5] + 0x60;
}
lVar6 = param_1[0x1c5];
if (lVar6 == param_1[0x1c6]) {
FUN_141edc340(puVar7,lVar6,local_f8);
}
else {
FUN_141068df0(lVar6,local_f8);
param_1[0x1c5] = param_1[0x1c5] + 0x60;
}
if (param_4 == 0) {
uVar3 = FUN_1423e00f0(local_60,0x41,1);
local_78 = "@Activate";
FUN_1423dfec0(local_70,uVar3);
FUN_141f07e80(param_1 + 0x132,local_50,&local_78);
FUN_1423dffc0(local_70);
FUN_1423dffc0(local_60);
puVar4 = (undefined8 *)FUN_14217c644(0x30);
puVar7 = puVar8;
if (puVar4 != (undefined8 *)0x0) {
*puVar4 = djplay::HidExtentFormat::vftable;
*(undefined2 *)(puVar4 + 1) = 0xf0;
puVar4[2] = 0;
puVar4[3] = 0;
puVar4[4] = 0;
puVar4[5] = 0;
puVar7 = puVar4;
}
puVar4 = (undefined8 *)param_1[0x1be];
param_1[0x1be] = puVar7;
if (puVar4 != (undefined8 *)0x0) {
(**(code **)*puVar4)(puVar4,1);
}
uVar3 = FUN_1423d5720(&local_res20,"@AuthReq");
FUN_141f1be20(param_1,0,0x11,uVar3,puVar1);
uVar3 = FUN_1423d5720(local_90,"@AuthChallengeA");
FUN_141f1be20(param_1,0,0x12,uVar3,puVar1);
uVar3 = FUN_1423d5720(local_88,"@AuthResponseA");
FUN_141f1be20(param_1,0,0x13,uVar3,puVar1);
uVar3 = FUN_1423d5720(local_80,"@AuthResponseE");
FUN_141f1be20(param_1,0,0x14,uVar3,puVar1);
uVar3 = FUN_1423d5720(local_98,"@AuthEnd");
FUN_141f1be20(param_1,0,0x15,uVar3,puVar1);
local_res20 = FUN_14217c644(0x1d0);
if (local_res20 != 0) {
puVar8 = (undefined8 *)FUN_141f093a0(local_res20,puVar1,0);
}
param_1[0x1bd] = puVar8;
*(undefined4 *)(puVar8 + 0x2f) = 4;
if (param_1[0x1bd] != 0) {
FUN_141f69980(param_1[0x1bd],param_1 + 0x13a);
}
}
else {
param_1[0x1bd] = param_4;
*(undefined1 *)((longlong)param_1 + 0xe09) = 1;
*(undefined4 *)((longlong)param_1 + 0x4b7c) = param_2;
FUN_141f40750(param_1);
}
FUN_1423d59b0(local_a8);
FUN_1423d59b0(local_b8);
FUN_1423d59b0(local_c8);
FUN_1423d59b0(local_d0);
FUN_1423d59b0(local_d8);
FUN_1423d59b0(local_e0);
FUN_1423d59b0(param_3);
return param_1;
}
//===== FUN_141f15bf0 @ 141f15bf0 =====
void FUN_141f15bf0(undefined8 *param_1)
{
undefined8 *puVar1;
LPCRITICAL_SECTION lpCriticalSection;
longlong *plVar2;
undefined8 uVar3;
int iVar4;
*param_1 = djplay::HidMapCDJ2000NXS2::vftable;
param_1[0x14] = djplay::HidMapCDJ2000NXS2::vftable;
param_1[0x15] = djplay::HidMapCDJ2000NXS2::vftable;
param_1[0x134] = djplay::HidMapCDJ2000NXS2::vftable;
param_1[0x138] = djplay::HidMapCDJ2000NXS2::vftable;
param_1[0x139] = djplay::HidMapCDJ2000NXS2::vftable;
param_1[0x13a] = djplay::HidMapCDJ2000NXS2::vftable;
if (*(char *)((longlong)param_1 + 0xe09) == '\0') {
if (param_1[0x1bd] != 0) {
FUN_141f72e10(param_1[0x1bd],param_1 + 0x13a);
}
puVar1 = (undefined8 *)param_1[0x1bd];
if (puVar1 != (undefined8 *)0x0) {
(**(code **)*puVar1)(puVar1,1);
}
}
uVar3 = FUN_14106db40();
FUN_141ed5ce0(uVar3,param_1 + 0x138);
uVar3 = FUN_1414518c0();
FUN_141ede760(uVar3,param_1 + 0x139);
lpCriticalSection = (LPCRITICAL_SECTION)param_1[0x1c7];
if (lpCriticalSection != (LPCRITICAL_SECTION)0x0) {
DeleteCriticalSection(lpCriticalSection);
thunk_FUN_14217cc5c(lpCriticalSection,0x40);
}
puVar1 = (undefined8 *)param_1[0x59b];
if (puVar1 != (undefined8 *)0x0) {
(**(code **)*puVar1)(puVar1,1);
}
FUN_141eea370(param_1 + 0x964);
FUN_141eea1a0(param_1 + 0x959);
FUN_141eea0a0(param_1 + 0x94d);
FUN_141ee9e80(param_1 + 0x941);
FUN_141ee9c60(param_1 + 0x936);
FUN_141ee9c60(param_1 + 0x92b);
FUN_141ee9c60(param_1 + 0x920);
FUN_141ee9c60(param_1 + 0x915);
FUN_141ee9a70(param_1 + 0x907);
FUN_141ee9860(param_1 + 0x8fb);
FUN_141ee6340(param_1 + 0x86a);
FUN_141ee5ee0(param_1 + 0x7da);
FUN_141ee5aa0(param_1 + 0x74a);
FUN_141ee54a0(param_1 + 0x6ba);
FUN_141ee5120(param_1 + 0x62b);
FUN_141ee4ea0(param_1 + 0x59c);
FUN_141ee3fe0(param_1 + 0x50d);
_eh_vector_destructor_iterator_(param_1 + 0x2d5,0x470,4,FUN_141ee3fe0);
FUN_141ee2880(param_1 + 0x1c9);
FUN_141edd2a0(param_1 + 0x1c4);
iVar4 = 0;
if (0 < *(int *)((longlong)param_1 + 0xe04)) {
do {
FUN_1423dffc0((longlong)iVar4 * 0x10 + param_1[0x1bf]);
iVar4 = iVar4 + 1;
} while (iVar4 < *(int *)((longlong)param_1 + 0xe04));
}
*(undefined4 *)((longlong)param_1 + 0xe04) = 0;
FUN_14230889c(param_1[0x1bf]);
puVar1 = (undefined8 *)param_1[0x1be];
if (puVar1 != (undefined8 *)0x0) {
(**(code **)*puVar1)(puVar1,1);
}
plVar2 = (longlong *)param_1[0x1bb];
if (plVar2 != (longlong *)0x0) {
(**(code **)(*plVar2 + 0x38))(plVar2,1);
}
puVar1 = (undefined8 *)param_1[0x1a4];
if (puVar1 != (undefined8 *)0x0) {
(**(code **)*puVar1)(puVar1,1);
}
puVar1 = (undefined8 *)param_1[0x1a3];
if (puVar1 != (undefined8 *)0x0) {
(**(code **)*puVar1)(puVar1,1);
}
param_1[0x13a] = djplay::DeviceAuth::Listener::vftable;
param_1[0x139] = djplay::TrackInformationManager::Listener::vftable;
param_1[0x138] = djplay::DisplayDataManager::Listener::vftable;
FUN_14246e800(param_1 + 0x134);
FUN_141f15b10(param_1);
return;
}
//===== FUN_141f63210 @ 141f63210 =====
void FUN_141f63210(longlong param_1,undefined8 *param_2,undefined4 *param_3,undefined8 param_4,
undefined1 param_5)
{
char cVar1;
undefined8 uVar2;
undefined8 local_18 [2];
switch(param_5) {
case 0x21:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0xe48) + 0x10))(param_1 + 0xe48);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0xe48) + 8))(param_1 + 0xe48,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x22:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x16a8) + 0x10))(param_1 + 0x16a8);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x16a8) + 8))(param_1 + 0x16a8,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x23:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x1b18) + 0x10))(param_1 + 0x1b18);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x1b18) + 8))(param_1 + 0x1b18,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x24:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x1f88) + 0x10))(param_1 + 0x1f88);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x1f88) + 8))(param_1 + 0x1f88,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x25:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x23f8) + 0x10))(param_1 + 0x23f8);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x23f8) + 8))(param_1 + 0x23f8,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x26:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x2868) + 0x10))(param_1 + 0x2868);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x2868) + 8))(param_1 + 0x2868,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x2a:
if ((*(longlong **)(param_1 + 0x2cd8) != (longlong *)0x0) &&
(cVar1 = (**(code **)(**(longlong **)(param_1 + 0x2cd8) + 0x10))(), cVar1 != '\0')) {
local_18[0] = 0;
uVar2 = (**(code **)(**(longlong **)(param_1 + 0x2cd8) + 8))
(*(longlong **)(param_1 + 0x2cd8),local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x2b:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x2ce0) + 0x10))(param_1 + 0x2ce0);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x2ce0) + 8))(param_1 + 0x2ce0,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x2c:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x3158) + 0x10))(param_1 + 0x3158);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x3158) + 8))(param_1 + 0x3158,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x2d:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x35d0) + 0x10))(param_1 + 0x35d0);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x35d0) + 8))(param_1 + 0x35d0,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x2e:
if ((*(char *)(param_1 + 0xe0c) == '\0') &&
(cVar1 = (**(code **)(*(longlong *)(param_1 + 0x3a50) + 0x10))(param_1 + 0x3a50),
cVar1 != '\0')) {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x3a50) + 8))(param_1 + 0x3a50,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x2f:
if ((*(char *)(param_1 + 0xe0c) == '\0') &&
(cVar1 = (**(code **)(*(longlong *)(param_1 + 0x3ed0) + 0x10))(param_1 + 0x3ed0),
cVar1 != '\0')) {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x3ed0) + 8))(param_1 + 0x3ed0,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x30:
if ((*(char *)(param_1 + 0xe0c) == '\0') &&
(cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4350) + 0x10))(param_1 + 0x4350),
cVar1 != '\0')) {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4350) + 8))(param_1 + 0x4350,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x40:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x47d8) + 0x10))(param_1 + 0x47d8);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x47d8) + 8))(param_1 + 0x47d8,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x41:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4838) + 0x10))(param_1 + 0x4838);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4838) + 8))(param_1 + 0x4838,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x42:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x48a8) + 0x10))(param_1 + 0x48a8);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x48a8) + 8))(param_1 + 0x48a8,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x43:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4900) + 0x10))(param_1 + 0x4900);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4900) + 8))(param_1 + 0x4900,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x44:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4958) + 0x10))(param_1 + 0x4958);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4958) + 8))(param_1 + 0x4958,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x45:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x49b0) + 0x10))(param_1 + 0x49b0);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x49b0) + 8))(param_1 + 0x49b0,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x46:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4a08) + 0x10))(param_1 + 0x4a08);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4a08) + 8))(param_1 + 0x4a08,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x47:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4a68) + 0x10))(param_1 + 0x4a68);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4a68) + 8))(param_1 + 0x4a68,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x48:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4ac8) + 0x10))(param_1 + 0x4ac8);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4ac8) + 8))(param_1 + 0x4ac8,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
break;
case 0x4f:
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4b20) + 0x10))(param_1 + 0x4b20);
if (cVar1 != '\0') {
local_18[0] = 0;
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4b20) + 8))(param_1 + 0x4b20,local_18,param_4);
*param_2 = uVar2;
goto LAB_141f63871;
}
}
local_18[0]._0_4_ = 0;
*param_2 = 0;
LAB_141f63871:
*param_3 = (undefined4)local_18[0];
return;
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+85
View File
@@ -0,0 +1,85 @@
// Walk the call tree down from the export orchestrator and surface the page
// serializer: flag functions that call file-I/O APIs or use the 4096 page size,
// then decompile those candidates. Output -> ~/ghidra_proj/calltree.txt
import java.io.PrintWriter;
import java.util.*;
import ghidra.app.script.GhidraScript;
import ghidra.app.decompiler.*;
import ghidra.program.model.listing.*;
import ghidra.program.model.scalar.Scalar;
public class CallTree extends GhidraScript {
static final String ROOT = "0x14119d2d0";
static final int MAX_DEPTH = 4;
boolean hasPageConst(Function f) {
InstructionIterator it = currentProgram.getListing().getInstructions(f.getBody(), true);
while (it.hasNext()) {
Instruction insn = it.next();
for (int op = 0; op < insn.getNumOperands(); op++)
for (Object o : insn.getOpObjects(op))
if (o instanceof Scalar) {
long v = ((Scalar) o).getUnsignedValue();
if (v == 0x1000L || v == 4096L) return true;
}
}
return false;
}
String ioCalls(Function f) {
StringBuilder sb = new StringBuilder();
for (Function c : f.getCalledFunctions(monitor)) {
String n = c.getName().toLowerCase();
if (n.contains("write") || n.contains("createfile") || n.contains("fopen")
|| n.contains("fwrite") || n.contains("mapviewof") || n.contains("setfilepointer")
|| n.contains("flushview") || n.contains("fclose"))
sb.append(c.getName()).append(" ");
}
return sb.toString();
}
@Override
public void run() throws Exception {
PrintWriter out = new PrintWriter("/home/ntr/ghidra_proj/calltree.txt");
FunctionManager fm = currentProgram.getFunctionManager();
Function root = fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(ROOT));
Map<Function, Integer> depth = new LinkedHashMap<>();
Deque<Function> q = new ArrayDeque<>();
depth.put(root, 0); q.add(root);
while (!q.isEmpty() && !monitor.isCancelled()) {
Function f = q.poll();
int d = depth.get(f);
if (d >= MAX_DEPTH) continue;
for (Function c : f.getCalledFunctions(monitor))
if (!depth.containsKey(c)) { depth.put(c, d + 1); q.add(c); }
}
out.println("call tree from " + root.getName() + " depth<=" + MAX_DEPTH + ": " + depth.size() + " funcs\n");
List<Function> candidates = new ArrayList<>();
for (Map.Entry<Function, Integer> e : depth.entrySet()) {
Function f = e.getKey();
boolean page = hasPageConst(f);
String io = ioCalls(f);
String flags = (page ? "[4096]" : "") + (io.isEmpty() ? "" : "[io:" + io.trim() + "]");
out.println(String.format("d%d %-12s size=%-6d %s %s",
e.getValue(), f.getEntryPoint(), f.getBody().getNumAddresses(), f.getName(), flags));
if (page || !io.isEmpty()) candidates.add(f);
}
DecompInterface di = new DecompInterface();
di.openProgram(currentProgram);
out.println("\n===== CANDIDATE DECOMPILES (" + candidates.size() + ") =====");
for (Function f : candidates) {
if (monitor.isCancelled()) break;
out.println("\n//===== " + f.getName() + " @ " + f.getEntryPoint() + " =====");
DecompileResults r = di.decompileFunction(f, 60, monitor);
if (r != null && r.decompileCompleted())
out.println(r.getDecompiledFunction().getC());
else
out.println("// decompile failed");
}
out.flush(); out.close();
println("wrote calltree.txt: " + depth.size() + " funcs, " + candidates.size() + " candidates");
}
}
+44
View File
@@ -0,0 +1,44 @@
// Decompile the pdb export functions found via string xrefs.
import java.io.PrintWriter;
import ghidra.app.script.GhidraScript;
import ghidra.app.decompiler.*;
import ghidra.program.model.address.Address;
import ghidra.program.model.listing.Function;
public class DecPdb extends GhidraScript {
// export.pdb serializer call stack from a live Frida WriteFile backtrace.
// module+offset -> Ghidra static = 0x140000000 + offset (base confirmed).
static final String[] ADDRS = {
"0x1420e883e", // calls WriteFile
"0x1420d73a6",
"0x1420d6e92",
"0x1420d6368",
"0x1420eae07",
"0x1420e0852",
"0x1420de1c4",
"0x141d75d21", // export.pdb-specific frame
"0x141cad188",
"0x141cac585",
"0x141b8ba75",
"0x141acaf8c",
"0x1423c9b3b",
};
@Override public void run() throws Exception {
PrintWriter out = new PrintWriter("/home/ntr/ghidra_proj/decpdb.txt");
DecompInterface di = new DecompInterface();
di.openProgram(currentProgram);
for (String a : ADDRS) {
Address addr = currentProgram.getAddressFactory().getAddress(a);
Function f = currentProgram.getFunctionManager().getFunctionContaining(addr);
if (f == null) { out.println("// no function at " + a); continue; }
out.println("\n//================ " + f.getName() + " @ " + f.getEntryPoint() + " ================");
DecompileResults r = di.decompileFunction(f, 90, monitor);
if (r != null && r.decompileCompleted())
out.println(r.getDecompiledFunction().getC());
else
out.println("// decompile failed: " + (r != null ? r.getErrorMessage() : "null"));
}
out.flush(); out.close();
println("wrote decpdb.txt");
}
}
+31
View File
@@ -0,0 +1,31 @@
// Decompile the export root FUN_14119d2d0 and selected callees, to find the
// per-track row-insert that sets index_shift (0x02) and the 0x14/0x18/0x1a blob.
// Also dump callees of FUN_14117be20 (rekordbox/export.pdb string xref).
// Output -> ~/ghidra_proj/decroot.txt
import java.io.PrintWriter;
import java.util.*;
import ghidra.app.script.GhidraScript;
import ghidra.app.decompiler.*;
import ghidra.program.model.listing.*;
public class DecRoot extends GhidraScript {
PrintWriter out; DecompInterface di; FunctionManager fm;
Set<String> done = new HashSet<>();
void dec(String a, String why) {
Function f = fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(a));
if (f == null) { out.println("// no fn at " + a + " (" + why + ")"); return; }
if (!done.add(f.getEntryPoint().toString())) return;
out.println("\n//===== " + f.getName() + " @ " + f.getEntryPoint() + " [" + why + "] =====");
DecompileResults r = di.decompileFunction(f, 120, monitor);
out.println(r != null && r.decompileCompleted() ? r.getDecompiledFunction().getC() : "// decompile failed: "+(r!=null?r.getErrorMessage():"null"));
}
@Override public void run() throws Exception {
out = new PrintWriter("/home/ntr/ghidra_proj/decroot.txt");
di = new DecompInterface(); di.openProgram(currentProgram);
fm = currentProgram.getFunctionManager();
dec("0x14119d2d0", "export root (/.PIONEER/rekordbox/export.pdb)");
dec("0x14117be20", "rekordbox/export.pdb xref");
out.flush(); out.close();
println("wrote decroot.txt");
}
}
+79
View File
@@ -0,0 +1,79 @@
// Decode the track-row builder: the function that maps a djmdContent record into
// the on-disk DeviceSQL row (offset 0x02 index_shift, 0x14 u4 hash, 0x18/0x1a u2).
// Decompiles FUN_141c04be0 (AnalysisDataPath/SearchStr/MasterContentID mapper) and
// FUN_141ae4860 (djmdContent), plus scans for code referencing the constant
// 0x21b4a7ab / 0xa7ab and for `<<5` row-index assignment.
// Output -> ~/ghidra_proj/decrow.txt
import java.io.PrintWriter;
import java.util.*;
import ghidra.app.script.GhidraScript;
import ghidra.app.decompiler.*;
import ghidra.program.model.address.*;
import ghidra.program.model.listing.*;
import ghidra.program.model.scalar.Scalar;
import ghidra.program.model.lang.OperandType;
public class DecRow extends GhidraScript {
PrintWriter out;
DecompInterface di;
Set<String> done = new HashSet<>();
FunctionManager fm;
void dec(String a, String why) {
Function f = fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(a));
if (f == null) { out.println("// no function at " + a + " (" + why + ")"); return; }
if (!done.add(f.getEntryPoint().toString())) return;
out.println("\n//===== " + f.getName() + " @ " + f.getEntryPoint() + " [" + why + "] =====");
DecompileResults r = di.decompileFunction(f, 90, monitor);
out.println(r != null && r.decompileCompleted() ? r.getDecompiledFunction().getC() : "// decompile failed");
}
@Override public void run() throws Exception {
out = new PrintWriter("/home/ntr/ghidra_proj/decrow.txt");
di = new DecompInterface(); di.openProgram(currentProgram);
fm = currentProgram.getFunctionManager();
dec("0x141c04be0", "AnalysisDataPath/SearchStr/MasterContentID mapper");
dec("0x141c07230", "SearchStr 2");
dec("0x141c076d0", "SearchStr 3");
dec("0x141ae4860", "djmdContent xrefs");
// Scan all instructions for the per-export constants used in 0x18/0x1a
// (0xa7ab=42923, 0x21b4=8628, combined u4 0x21b4a7ab=565353387) and for
// an immediate 0x20 / shift-by-5 pattern that would build index_shift.
long[] consts = {0xa7abL, 0x21b4L, 0x21b4a7abL, 0xa7ab21b4L};
Set<String> hitFns = new LinkedHashSet<>();
Listing lst = currentProgram.getListing();
InstructionIterator it = lst.getInstructions(true);
int scanned = 0;
while (it.hasNext() && !monitor.isCancelled()) {
Instruction ins = it.next();
scanned++;
for (int op = 0; op < ins.getNumOperands(); op++) {
Object[] objs = ins.getOpObjects(op);
for (Object o : objs) {
if (o instanceof Scalar) {
long v = ((Scalar) o).getUnsignedValue();
for (long c : consts) {
if (v == c) {
Function f = fm.getFunctionContaining(ins.getAddress());
String tag = "const 0x" + Long.toHexString(c) + " @" + ins.getAddress()
+ " in " + (f != null ? f.getName() + "@" + f.getEntryPoint() : "?")
+ " : " + ins.toString();
out.println("// SCAN " + tag);
if (f != null) hitFns.add(f.getEntryPoint().toString() + "|const0x" + Long.toHexString(c));
}
}
}
}
}
}
out.println("// scanned " + scanned + " instructions");
for (String h : hitFns) {
String[] parts = h.split("\\|");
dec("0x" + parts[0], parts[1]);
}
out.flush(); out.close();
println("wrote decrow.txt");
}
}
+30
View File
@@ -0,0 +1,30 @@
// Decompile the prime index_shift candidate FUN_14232a86c (shl5 + mask 0x7ff) and
// a few neighboring DeviceSQL page-manager functions. Output -> ~/ghidra_proj/decshift.txt
import java.io.PrintWriter;
import java.util.*;
import ghidra.app.script.GhidraScript;
import ghidra.app.decompiler.*;
import ghidra.program.model.listing.*;
public class DecShift extends GhidraScript {
PrintWriter out; DecompInterface di; FunctionManager fm; Set<String> done=new HashSet<>();
void dec(String a, String why) {
Function f = fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(a));
if (f == null) { out.println("// no fn at " + a + " (" + why + ")"); return; }
if (!done.add(f.getEntryPoint().toString())) return;
out.println("\n//===== " + f.getName() + " @ " + f.getEntryPoint() + " [" + why + "] =====");
DecompileResults r = di.decompileFunction(f, 120, monitor);
out.println(r != null && r.decompileCompleted() ? r.getDecompiledFunction().getC() : "// decompile failed");
}
@Override public void run() throws Exception {
out = new PrintWriter("/home/ntr/ghidra_proj/decshift.txt");
di = new DecompInterface(); di.openProgram(currentProgram);
fm = currentProgram.getFunctionManager();
dec("0x14232a86c", "shl5+mask7ff PRIME index_shift candidate");
// neighbors in DeviceSQL page-manager range
dec("0x1423e1550", "page-mgr neighbor");
dec("0x1423e12a0", "page-mgr neighbor");
out.flush(); out.close();
println("wrote decshift.txt");
}
}
+47
View File
@@ -0,0 +1,47 @@
// @flesniak documented track-row 0x18=u2 "always 19048" (0x4A68) and 0x1a=u2
// "always 30967" (0x78F7). Our AceMo export has 0xa7ab / 0x21b4 instead, and a
// const-scan found no 0xa7ab/0x21b4 immediates. So locate every reference to the
// documented defaults 0x4A68 and 0x78F7 (and combined 0x78F74A68 / 0x4A6878F7),
// decompile the containing functions, and report. This should reveal how 0x18/0x1a
// are produced (literal default vs computed). Output -> ~/ghidra_proj/find4a68.txt
import java.io.PrintWriter;
import java.util.*;
import ghidra.app.script.GhidraScript;
import ghidra.app.decompiler.*;
import ghidra.program.model.listing.*;
import ghidra.program.model.scalar.Scalar;
public class Find4A68 extends GhidraScript {
@Override public void run() throws Exception {
PrintWriter out = new PrintWriter("/home/ntr/ghidra_proj/find4a68.txt");
FunctionManager fm = currentProgram.getFunctionManager();
DecompInterface di = new DecompInterface(); di.openProgram(currentProgram);
long[] consts = {0x4A68L, 0x78F7L, 0x78F74A68L, 0x4A6878F7L, 19048L, 30967L};
Set<String> fns = new LinkedHashSet<>();
InstructionIterator it = currentProgram.getListing().getInstructions(true);
while (it.hasNext() && !monitor.isCancelled()) {
Instruction ins = it.next();
for (int op=0; op<ins.getNumOperands(); op++)
for (Object o : ins.getOpObjects(op))
if (o instanceof Scalar) {
long v=((Scalar)o).getUnsignedValue();
for (long c: consts) if (v==c) {
Function f=fm.getFunctionContaining(ins.getAddress());
out.println("// HIT 0x"+Long.toHexString(c)+" @"+ins.getAddress()
+" in "+(f!=null?f.getName()+"@"+f.getEntryPoint():"?")+" : "+ins.toString());
if (f!=null) fns.add(f.getEntryPoint().toString());
}
}
}
// also search defined data for the byte patterns
out.println("// --- decompiling containing functions ---");
for (String a: fns) {
Function f=fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(a));
out.println("\n//===== "+f.getName()+" @ "+f.getEntryPoint()+" =====");
DecompileResults r=di.decompileFunction(f,90,monitor);
out.println(r!=null&&r.decompileCompleted()?r.getDecompiledFunction().getC():"// decompile failed");
}
out.flush(); out.close();
println("wrote find4a68.txt");
}
}
+44
View File
@@ -0,0 +1,44 @@
// Anchor the track-row builder via the track-row bitmask 0xc0700 (constant in
// every AceMo track row at offset 0x04) and the subtype/0x24 nearby. Find any
// instruction with immediate 0xc0700 and decompile the function. Output ->
// ~/ghidra_proj/findbitmask.txt
import java.io.PrintWriter;
import java.util.*;
import ghidra.app.script.GhidraScript;
import ghidra.app.decompiler.*;
import ghidra.program.model.listing.*;
import ghidra.program.model.scalar.Scalar;
public class FindBitmask extends GhidraScript {
@Override public void run() throws Exception {
PrintWriter out = new PrintWriter("/home/ntr/ghidra_proj/findbitmask.txt");
FunctionManager fm = currentProgram.getFunctionManager();
DecompInterface di = new DecompInterface(); di.openProgram(currentProgram);
long[] consts = {0xc0700L, 0x700L, 0xc07L};
Set<String> fns = new LinkedHashSet<>();
InstructionIterator it = currentProgram.getListing().getInstructions(true);
while (it.hasNext() && !monitor.isCancelled()) {
Instruction ins = it.next();
for (int op=0; op<ins.getNumOperands(); op++)
for (Object o : ins.getOpObjects(op))
if (o instanceof Scalar) {
long v=((Scalar)o).getUnsignedValue();
if (v==0xc0700L) {
Function f=fm.getFunctionContaining(ins.getAddress());
out.println("// HIT 0xc0700 @"+ins.getAddress()+" in "
+(f!=null?f.getName()+"@"+f.getEntryPoint():"?")+" : "+ins.toString());
if (f!=null) fns.add(f.getEntryPoint().toString());
}
}
}
out.println("// --- decompiling ---");
for (String a: fns) {
Function f=fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(a));
out.println("\n//===== "+f.getName()+" @ "+f.getEntryPoint()+" =====");
DecompileResults r=di.decompileFunction(f,120,monitor);
out.println(r!=null&&r.decompileCompleted()?r.getDecompiledFunction().getC():"// fail");
}
out.flush(); out.close();
println("wrote findbitmask.txt");
}
}
+28
View File
@@ -0,0 +1,28 @@
// Confirm 0x78f74a68 / 0x4a68 / 0x78f7 is rekordbox's default opaque_seed by
// searching DEFINED DATA (not just instruction immediates) for the byte pattern
// 68 4a f7 78 (LE u4) and the two u2s, and report containing context.
// Output -> ~/ghidra_proj/finddefault.txt
import java.io.PrintWriter;
import ghidra.app.script.GhidraScript;
import ghidra.program.model.address.*;
import ghidra.program.model.mem.*;
public class FindDefault extends GhidraScript {
@Override public void run() throws Exception {
PrintWriter out = new PrintWriter("/home/ntr/ghidra_proj/finddefault.txt");
Memory mem = currentProgram.getMemory();
byte[] pat = { (byte)0x68,(byte)0x4a,(byte)0xf7,(byte)0x78 }; // 0x78f74a68 LE
Address a = mem.getMinAddress();
int found=0;
while (a != null && found < 50) {
Address hit = mem.findBytes(a, pat, null, true, monitor);
if (hit == null) break;
out.println("// 0x78f74a68 bytes @ " + hit);
found++;
a = hit.add(1);
}
out.println("// total occurrences of 0x78f74a68 LE in memory image: " + found);
out.flush(); out.close();
println("wrote finddefault.txt found="+found);
}
}
+78
View File
@@ -0,0 +1,78 @@
// Hunt for the track-row builder / the per-track 8-byte hash (row offsets
// 0x14 u4, 0x18 u2, 0x1a u2) and index_shift (0x02). Decompiles the string-hash
// function formatForSearch + its callers, and dumps xrefs of track/content/
// search-related strings. Output -> ~/ghidra_proj/hash.txt
import java.io.PrintWriter;
import java.util.*;
import ghidra.app.script.GhidraScript;
import ghidra.app.decompiler.*;
import ghidra.program.model.listing.*;
import ghidra.program.model.symbol.*;
public class FindHash extends GhidraScript {
PrintWriter out;
DecompInterface di;
Set<String> done = new HashSet<>();
FunctionManager fm;
ReferenceManager rm;
void dec(Function f, String why) {
if (f == null) return;
if (!done.add(f.getEntryPoint().toString())) return;
out.println("\n//===== " + f.getName() + " @ " + f.getEntryPoint() + " [" + why + "] =====");
DecompileResults r = di.decompileFunction(f, 60, monitor);
out.println(r != null && r.decompileCompleted() ? r.getDecompiledFunction().getC() : "// decompile failed");
}
Function fn(String a) {
return fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(a));
}
@Override
public void run() throws Exception {
out = new PrintWriter("/home/ntr/ghidra_proj/hash.txt");
di = new DecompInterface(); di.openProgram(currentProgram);
fm = currentProgram.getFunctionManager();
rm = currentProgram.getReferenceManager();
// 1. formatForSearch (string hash) + its callers
Function fmt = fn("0x141a1afc0");
dec(fmt, "formatForSearch");
if (fmt != null) {
int n = 0;
for (Reference ref : rm.getReferencesTo(fmt.getEntryPoint())) {
if (n++ >= 10) break;
dec(fm.getFunctionContaining(ref.getFromAddress()), "caller of formatForSearch");
}
}
// 2. strings that hint at the track/content row builder -> xref functions
String[] kw = {"djmdContent", "SearchStr", "formatForSearch", "AnalysisDataPath",
"ContentID", "rb_data_status", "rb_local", "dsqlInsert", "AddTrack"};
DataIterator dit = currentProgram.getListing().getDefinedData(true);
int shown = 0;
while (dit.hasNext() && !monitor.isCancelled()) {
Data d = dit.next();
if (d == null || !d.hasStringValue()) continue;
Object v = d.getValue();
if (!(v instanceof String)) continue;
String s = (String) v;
for (String k : kw) {
if (s.contains(k)) {
out.println("\n// STRING @" + d.getAddress() + " : " + s);
int n = 0;
for (Reference ref : rm.getReferencesTo(d.getAddress())) {
if (n++ >= 5) break;
Function c = fm.getFunctionContaining(ref.getFromAddress());
out.println("// xref in " + (c != null ? c.getName() + " @" + c.getEntryPoint() : ref.getFromAddress()));
}
shown++;
break;
}
}
if (shown >= 30) break;
}
out.flush(); out.close();
println("wrote hash.txt");
}
}
+96
View File
@@ -0,0 +1,96 @@
// Locate the rekordbox .pdb serializer via known strings + magic-number constants.
// Run headless: analyzeHeadless <proj> rbx -process rekordbox.exe -noanalysis \
// -scriptPath ~/ghidra_scripts -postScript FindPdb.java
// Output goes to ~/ghidra_proj/findpdb.txt (and the log).
import java.io.PrintWriter;
import java.util.*;
import ghidra.app.script.GhidraScript;
import ghidra.program.model.address.Address;
import ghidra.program.model.listing.*;
import ghidra.program.model.scalar.Scalar;
import ghidra.program.model.symbol.*;
public class FindPdb extends GhidraScript {
// Strings that mark the export code paths.
static final String[] KEYWORDS = {
"export.pdb", "exportExt.pdb", "USBANLZ", "PIONEER",
"ANLZ", "HOT CUE BANK", "ORIGINAL ARTIST", "rekordbox/export"
};
// Magic numbers maple currently guesses (various widths/combinations).
static final long[] MAGICS = {
42343L, 54995L, // 0xA567 / 0xD6D3 per-track markers
0xD6D3A567L, 0xA567D6D3L, // combined 32-bit forms
788224L, // 0x000C0700 track bitmask
1004L, // 0x03EC "strange page" marker
0x64L, 0x44L, 0x24L, // page-flag bytes
19048L, 30967L // the schema author's original guesses
};
PrintWriter out;
void emit(String s) { println(s); if (out != null) out.println(s); }
@Override
public void run() throws Exception {
out = new PrintWriter("/home/ntr/ghidra_proj/findpdb.txt");
Listing listing = currentProgram.getListing();
ReferenceManager rm = currentProgram.getReferenceManager();
FunctionManager fm = currentProgram.getFunctionManager();
emit("===== STRING XREFS =====");
DataIterator di = listing.getDefinedData(true);
while (di.hasNext() && !monitor.isCancelled()) {
Data d = di.next();
if (d == null || !d.hasStringValue()) continue;
Object val = d.getValue();
if (!(val instanceof String)) continue;
String s = (String) val;
for (String kw : KEYWORDS) {
if (s.contains(kw)) {
emit("\nSTRING @ " + d.getAddress() + " : " + s.replace("\n", "\\n"));
ReferenceIterator ri = rm.getReferencesTo(d.getAddress());
int n = 0;
while (ri.hasNext() && n++ < 25) {
Reference r = ri.next();
Function f = fm.getFunctionContaining(r.getFromAddress());
emit(" xref from " + r.getFromAddress()
+ (f != null ? " in " + f.getName() + " @ " + f.getEntryPoint() : " (no func)"));
}
break;
}
}
}
emit("\n===== CONSTANT XREFS =====");
Set<Long> targets = new HashSet<>();
for (long m : MAGICS) targets.add(m);
Map<Long, Integer> counts = new TreeMap<>();
InstructionIterator ii = listing.getInstructions(true);
while (ii.hasNext() && !monitor.isCancelled()) {
Instruction insn = ii.next();
for (int op = 0; op < insn.getNumOperands(); op++) {
for (Object o : insn.getOpObjects(op)) {
if (o instanceof Scalar) {
long v = ((Scalar) o).getUnsignedValue();
if (targets.contains(v)) {
Function f = fm.getFunctionContaining(insn.getAddress());
emit(String.format("CONST 0x%X (%d) @ %s %s in %s", v, v,
insn.getAddress(), insn.toString(),
f != null ? f.getName() + " @ " + f.getEntryPoint() : "(no func)"));
counts.merge(v, 1, Integer::sum);
}
}
}
}
}
emit("\n===== CONSTANT HIT COUNTS =====");
for (Map.Entry<Long, Integer> e : counts.entrySet())
emit(String.format("0x%X (%d): %d hits", e.getKey(), e.getKey(), e.getValue()));
out.flush();
out.close();
emit("\nwrote /home/ntr/ghidra_proj/findpdb.txt");
}
}
+51
View File
@@ -0,0 +1,51 @@
// Find the DeviceSQL row-index code that computes index_shift = slot << 5.
// Heuristic: functions that contain BOTH a left-shift-by-5 (SHL r,5 / LEA r,[r*0x20])
// AND reference the row-index masks (0x1fff / 0x7ff) or stride 0x24, AND a u2 store.
// Output -> ~/ghidra_proj/findshift.txt
import java.io.PrintWriter;
import java.util.*;
import ghidra.app.script.GhidraScript;
import ghidra.program.model.listing.*;
import ghidra.program.model.scalar.Scalar;
public class FindShift extends GhidraScript {
@Override public void run() throws Exception {
PrintWriter out = new PrintWriter("/home/ntr/ghidra_proj/findshift.txt");
FunctionManager fm = currentProgram.getFunctionManager();
Listing lst = currentProgram.getListing();
// accumulate per-function evidence
Map<String,Set<String>> ev = new HashMap<>();
InstructionIterator it = lst.getInstructions(true);
while (it.hasNext() && !monitor.isCancelled()) {
Instruction ins = it.next();
Function f = fm.getFunctionContaining(ins.getAddress());
if (f == null) continue;
String key = f.getEntryPoint().toString();
String mn = ins.getMnemonicString();
String txt = ins.toString();
boolean shl5 = (mn.equalsIgnoreCase("SHL") && txt.endsWith(",0x5"))
|| (mn.equalsIgnoreCase("LEA") && txt.contains("*0x20"));
if (shl5) ev.computeIfAbsent(key,k->new HashSet<>()).add("shl5@"+ins.getAddress());
for (int op=0; op<ins.getNumOperands(); op++)
for (Object o : ins.getOpObjects(op))
if (o instanceof Scalar) {
long v = ((Scalar)o).getUnsignedValue();
if (v==0x1fff) ev.computeIfAbsent(key,k->new HashSet<>()).add("mask1fff@"+ins.getAddress());
if (v==0x7ff) ev.computeIfAbsent(key,k->new HashSet<>()).add("mask7ff@"+ins.getAddress());
if (v==0x24) ev.computeIfAbsent(key,k->new HashSet<>()).add("stride24@"+ins.getAddress());
}
}
// report functions that have shl5 AND (mask or stride)
for (Map.Entry<String,Set<String>> e : ev.entrySet()) {
Set<String> tags = e.getValue();
boolean hasShl = tags.stream().anyMatch(t->t.startsWith("shl5"));
boolean hasCtx = tags.stream().anyMatch(t->t.startsWith("mask1fff")||t.startsWith("mask7ff")||t.startsWith("stride24"));
if (hasShl && hasCtx) {
Function f = fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(e.getKey()));
out.println("CANDIDATE " + (f!=null?f.getName():"?") + " @" + e.getKey() + " : " + tags);
}
}
out.flush(); out.close();
println("wrote findshift.txt");
}
}
View File
Whitespace-only changes.
+673
View File
@@ -0,0 +1,673 @@
meta:
id: rekordbox_anlz
title: rekordbox track analysis file
application: rekordbox
file-extension:
- dat
- ext
license: EPL-1.0
endian: be
doc: |
These files are created by rekordbox when analyzing audio tracks
to facilitate DJ performance. They include waveforms, beat grids
(information about the precise time at which each beat occurs),
time indices to allow efficient seeking to specific positions
inside variable bit-rate audio streams, and lists of memory cues
and loop points. They are used by Pioneer professional DJ
equipment.
The format has been reverse-engineered to facilitate sophisticated
integrations with light and laser shows, videos, and other musical
instruments, by supporting deep knowledge of what is playing and
what is coming next through monitoring the network communications
of the players.
doc-ref: https://reverseengineering.stackexchange.com/questions/4311/help-reversing-a-edb-database-file-for-pioneers-rekordbox-software
seq:
- id: magic
contents: "PMAI"
doc: Identifies this as an analysis file.
- id: len_header
type: u4
doc: |
The number of bytes of this header section.
- id: len_file
type: u4
doc: |
The number of bytes in the entire file.
- size: len_header - _io.pos
- id: sections
type: tagged_section
repeat: eos
doc: |
The remainder of the file is a sequence of type-tagged sections,
identified by a four-byte magic sequence.
types:
tagged_section:
doc: |
A type-tagged file section, identified by a four-byte magic
sequence, with a header specifying its length, and whose payload
is determined by the type tag.
seq:
- id: fourcc
type: s4
enum: section_tags
doc: |
A tag value indicating what kind of section this is.
- id: len_header
type: u4
doc: |
The size, in bytes, of the header portion of the tag.
- id: len_tag
type: u4
doc: |
The size, in bytes, of this entire tag, counting the header.
- id: body
size: len_tag - 12
type:
switch-on: fourcc
cases:
'section_tags::cues_2': cue_extended_tag # PCO2
'section_tags::cues': cue_tag # PCOB
'section_tags::path': path_tag # PPTH
'section_tags::beat_grid': beat_grid_tag # PQTZ
'section_tags::vbr': vbr_tag # PVBR
'section_tags::wave_preview': wave_preview_tag # PWAV
'section_tags::wave_tiny': wave_preview_tag # PWV2
'section_tags::wave_scroll': wave_scroll_tag # PWV3, seen in .EXT
'section_tags::wave_color_preview': wave_color_preview_tag # PWV4, in .EXT
'section_tags::wave_color_scroll': wave_color_scroll_tag # PWV5, in .EXT
'section_tags::wave_3band_preview': wave_3band_preview_tag # P@V6, in .2EX
'section_tags::wave_3band_scroll': wave_3band_scroll_tag # PWV7, in .2EX
'section_tags::song_structure': song_structure_tag # PSSI, in .EXT
_: unknown_tag
-webide-representation: '{fourcc}'
beat_grid_tag:
doc: |
Holds a list of all the beats found within the track, recording
their bar position, the time at which they occur, and the tempo
at that point.
seq:
- type: u4
- type: u4 # @flesniak says this is always 0x80000
- id: num_beats
type: u4
doc: |
The number of beat entries which follow.
- id: beats
type: beat_grid_beat
repeat: expr
repeat-expr: num_beats
doc: The entries of the beat grid.
beat_grid_beat:
doc: |
Describes an individual beat in a beat grid.
seq:
- id: beat_number
type: u2
doc: |
The position of the beat within its musical bar, where beat 1
is the down beat.
- id: tempo
type: u2
doc: |
The tempo at the time of this beat, in beats per minute,
multiplied by 100.
- id: time
type: u4
doc: |
The time, in milliseconds, at which this beat occurs when
the track is played at normal (100%) pitch.
cue_tag:
doc: |
Stores either a list of ordinary memory cues and loop points, or
a list of hot cues and loop points.
seq:
- id: type
type: u4
enum: cue_list_type
doc: |
Identifies whether this tag stores ordinary or hot cues.
- size: 2
- id: num_cues
type: u2
doc: |
The length of the cue list.
- id: memory_count
type: u4
doc: |
Unsure what this means.
- id: cues
type: cue_entry
repeat: expr
repeat-expr: num_cues
cue_entry:
doc: |
A cue list entry. Can either represent a memory cue or a loop.
seq:
- id: magic
contents: "PCPT"
doc: Identifies this as a cue list entry (cue point).
- id: len_header
type: u4
- id: len_entry
type: u4
- id: hot_cue
type: u4
doc: |
If zero, this is an ordinary memory cue, otherwise this a
hot cue with the specified number.
- id: status
type: u4
enum: cue_entry_status
doc: |
Indicates if this is an active loop.
- type: u4 # Seems to always be 0x10000
- id: order_first
type: u2
doc: |
@flesniak says: "0xffff for first cue, 0,1,3 for next"
- id: order_last
type: u2
doc: |
@flesniak says: "1,2,3 for first, second, third cue, 0xffff for last"
- id: type
type: u1
enum: cue_entry_type
doc: |
Indicates whether this is a memory cue or a loop.
- size: 3 # seems to always be 1000
- id: time
type: u4
doc: |
The position, in milliseconds, at which the cue point lies
in the track.
- id: loop_time
type: u4
doc: |
The position, in milliseconds, at which the player loops
back to the cue time if this is a loop.
- size: 16
cue_extended_tag:
doc: |
A variation of cue_tag which was introduced with the nxs2 line,
and adds descriptive names. (Still comes in two forms, either
holding memory cues and loop points, or holding hot cues and
loop points.) Also includes hot cues D through H and color assignment.
seq:
- id: type
type: u4
enum: cue_list_type
doc: |
Identifies whether this tag stores ordinary or hot cues.
- id: num_cues
type: u2
doc: |
The length of the cue comment list.
- size: 2
- id: cues
type: cue_extended_entry
repeat: expr
repeat-expr: num_cues
cue_extended_entry:
doc: |
A cue extended list entry. Can either describe a memory cue or a
loop.
seq:
- id: magic
contents: "PCP2"
doc: Identifies this as an extended cue list entry (cue point).
- id: len_header
type: u4
- id: len_entry
type: u4
- id: hot_cue
type: u4
doc: |
If zero, this is an ordinary memory cue, otherwise this a
hot cue with the specified number.
- id: type
type: u1
enum: cue_entry_type
doc: |
Indicates whether this is a regular cue point or a loop.
- size: 3 # seems to always be 1000
- id: time
type: u4
doc: |
The position, in milliseconds, at which the cue point lies
in the track.
- id: loop_time
type: u4
doc: |
The position, in milliseconds, at which the player loops
back to the cue time if this is a loop.
- id: color_id
type: u1
doc: |
References a row in the colors table if this is a memory cue or loop
and has been assigned a color.
- size: 7
- id: loop_numerator
type: u2
doc: |
The numerator of the loop length in beats.
Zero if the loop is not quantized.
- id: loop_denominator
type: u2
doc: |
The denominator of the loop length in beats.
Zero if the loop is not quantized.
- id: len_comment
type: u4
if: len_entry > 43
- id: comment
type: str
size: len_comment
encoding: utf-16be
doc: |
The comment assigned to this cue by the DJ, if any, with a trailing NUL.
if: len_entry > 43
- id: color_code
type: u1
doc: |
A lookup value for a color table? We use this to index to the hot cue colors shown in rekordbox.
if: (len_entry - len_comment) > 44
- id: color_red
type: u1
doc: |
The red component of the hot cue color to be displayed.
if: (len_entry - len_comment) > 45
- id: color_green
type: u1
doc: |
The green component of the hot cue color to be displayed.
if: (len_entry - len_comment) > 46
- id: color_blue
type: u1
doc: |
The blue component of the hot cue color to be displayed.
if: (len_entry - len_comment) > 47
- size: len_entry - 48 - len_comment # The remainder after the color
if: (len_entry - len_comment) > 48
path_tag:
doc: |
Stores the file path of the audio file to which this analysis
applies.
seq:
- id: len_path
type: u4
- id: path
type: str
size: len_path - 2
encoding: utf-16be
if: len_path > 1
-webide-representation: '{path}'
vbr_tag:
doc: |
Stores an index allowing rapid seeking to particular times
within a variable-bitrate audio file.
seq:
- type: u4
- id: index
type: u4
repeat: expr
repeat-expr: 400
wave_preview_tag:
doc: |
Stores a waveform preview image suitable for display above
the touch strip for jumping to a track position.
seq:
- id: len_data
type: u4
doc: |
The length, in bytes, of the preview data itself. This is
slightly redundant because it can be computed from the
length of the tag.
- type: u4 # This seems to always have the value 0x10000
- id: data
size: len_data
doc: |
The actual bytes of the waveform preview.
if: _parent.len_tag > _parent.len_header
wave_scroll_tag:
doc: |
A larger waveform image suitable for scrolling along as a track
plays.
seq:
- id: len_entry_bytes
type: u4
doc: |
The size of each entry, in bytes. Seems to always be 1.
- id: len_entries
type: u4
doc: |
The number of waveform data points, each of which takes one
byte.
- type: u4 # Always 0x960000?
- id: entries
size: len_entries * len_entry_bytes
wave_color_preview_tag:
doc: |
A larger, colorful waveform preview image suitable for display
above the touch strip for jumping to a track position on newer
high-resolution players.
seq:
- id: len_entry_bytes
type: u4
doc: |
The size of each entry, in bytes. Seems to always be 6.
- id: len_entries
type: u4
doc: |
The number of waveform data points, each of which takes one
byte for each of six channels of information.
- type: u4
- id: entries
size: len_entries * len_entry_bytes
wave_color_scroll_tag:
doc: |
A larger, colorful waveform image suitable for scrolling along
as a track plays on newer high-resolution hardware. Also
contains a higher-resolution blue/white waveform.
seq:
- id: len_entry_bytes
type: u4
doc: |
The size of each entry, in bytes. Seems to always be 2.
- id: len_entries
type: u4
doc: |
The number of columns of waveform data (this matches the
non-color waveform length.
- type: u4
- id: entries
size: len_entries * len_entry_bytes
wave_3band_preview_tag:
doc: |
The minimalist CDJ-3000 waveform preview image suitable for display
above the touch strip for jumping to a track position.
seq:
- id: len_entry_bytes
type: u4
doc: |
The size of each entry, in bytes. Seems to always be 3.
- id: len_entries
type: u4
doc: |
The number of waveform data points, each of which takes one
byte for each of six channels of information.
- id: entries
size: len_entries * len_entry_bytes
wave_3band_scroll_tag:
doc: |
The minimalist CDJ-3000 waveform image suitable for scrolling along
as a track plays on newer high-resolution hardware.
seq:
- id: len_entry_bytes
type: u4
doc: |
The size of each entry, in bytes. Seems to always be 3.
- id: len_entries
type: u4
doc: |
The number of columns of waveform data (this matches the
non-color waveform length.
- type: u4
- id: entries
size: len_entries * len_entry_bytes
song_structure_tag:
doc: |
Stores the song structure, also known as phrases (intro, verse,
bridge, chorus, up, down, outro).
seq:
- id: len_entry_bytes
type: u4
doc: |
The size of each entry, in bytes. Seems to always be 24.
- id: len_entries
type: u2
doc: |
The number of phrases.
- id: body
type: song_structure_body
doc: |
The rest of the tag, which needs to be unmasked before it
can be parsed.
size-eos: true
process: 'xor(is_masked ? mask : [0])'
instances:
c:
value: len_entries
mask:
value: |
[
(0xCB+c).as<s1>, (0xE1+c).as<s1>, (0xEE+c).as<s1>, (0xFA+c).as<s1>, (0xE5+c).as<s1>, (0xEE+c).as<s1>, (0xAD+c).as<s1>, (0xEE+c).as<s1>,
(0xE9+c).as<s1>, (0xD2+c).as<s1>, (0xE9+c).as<s1>, (0xEB+c).as<s1>, (0xE1+c).as<s1>, (0xE9+c).as<s1>, (0xF3+c).as<s1>, (0xE8+c).as<s1>,
(0xE9+c).as<s1>, (0xF4+c).as<s1>, (0xE1+c).as<s1>
].as<bytes>
raw_mood:
type: u2
pos: 6
doc: |
This is a way to tell whether the rest of the tag has been masked. The value is supposed
to range from 1 to 3, but in masked files it will be much larger.
is_masked:
value: 'raw_mood > 20' # This is almost certainly not true for an unmasked file.
-webide-representation: '{body.mood}'
song_structure_body:
doc: |
Stores the rest of the song structure tag, which can only be
parsed after unmasking.
seq:
- id: mood
type: u2
enum: track_mood
doc: |
The mood which rekordbox assigns the track as a whole during phrase analysis.
- size: 6
- id: end_beat
type: u2
doc: |
The beat number at which the last phrase ends. The track may
continue after the last phrase ends. If this is the case, it will
mostly be silence.
- size: 2
- id: raw_bank
type: u1
doc: |
Number identifying a stylistic bank which can be assigned to the track in rekordbox Lighting mode.
- size: 1
- id: entries
type: song_structure_entry
repeat: expr
repeat-expr: _parent.len_entries
instances:
bank:
value: raw_bank
enum: track_bank
doc: |
The stylistic bank which can be assigned to the track in rekordbox Lighting mode, if raw_bank has a legal value.
if: 'raw_bank < 9'
song_structure_entry:
doc: |
A song structure entry, represents a single phrase.
seq:
- id: index
type: u2
doc: |
The absolute number of the phrase, starting at one.
- id: beat
type: u2
doc: |
The beat number at which the phrase starts.
- id: kind
type:
switch-on: _parent.mood
cases:
'track_mood::high': phrase_high
'track_mood::mid': phrase_mid
'track_mood::low': phrase_low
_: phrase_mid # We don't recognize this mood, so pick a generic interpretation.
doc: |
The kind of phrase as displayed in rekordbox.
- size: 1
- id: k1
type: u1
doc: One of three flags that identify phrase kind variants in high-mood tracks.
- size: 1
- id: k2
type: u1
doc: One of three flags that identify phrase kind variants in high-mood tracks.
- size: 1
- id: b
type: u1
doc: |
Flags how many more beat numbers are in a high-mood "Up 3" phrase.
- id: beat2
type: u2
doc: |
Extra beat number (falling within phrase) always present in high-mood "Up 3" phrases.
- id: beat3
type: u2
doc: |
Extra beat number (falling within phrase, larger than beat2)
present in high-mood "Up 3" phrases when b has value 1.
- id: beat4
type: u2
doc: |
Extra beat number (falling within phrase, larger than beat3)
present in high-mood "Up 3" phrases when b has value 1.
- size: 1
- id: k3
type: u1
doc: One of three flags that identify phrase kind variants in high-mood tracks.
- size: 1
- id: fill
type: u1
doc: |
If nonzero, fill-in is present at end of phrase.
- id: beat_fill
type: u2
doc: |
The beat number at which fill-in starts.
-webide-representation: '{kind.id}'
phrase_high:
seq:
- id: id
type: u2
enum: mood_high_phrase
phrase_mid:
seq:
- id: id
type: u2
enum: mood_mid_phrase
phrase_low:
seq:
- id: id
type: u2
enum: mood_low_phrase
unknown_tag: {}
enums:
section_tags:
0x50434f42: cues # PCOB
0x50434f32: cues_2 # PCO2 (seen in .EXT)
0x50505448: path # PPTH
0x50564252: vbr # PVBR
0x5051545a: beat_grid # PQTZ
0x50574156: wave_preview # PWAV
0x50575632: wave_tiny # PWV2
0x50575633: wave_scroll # PWV3 (seen in .EXT)
0x50575634: wave_color_preview # PWV4 (seen in .EXT)
0x50575635: wave_color_scroll # PWV5 (seen in .EXT)
0x50535349: song_structure # PSSI (seen in .EXT)
0x50575636: wave_3band_preview # PWV6 (seen in .2EX)
0x50575637: wave_3band_scroll # PWV7 (seen in .2EX)
cue_list_type:
0: memory_cues
1: hot_cues
cue_entry_type:
1: memory_cue
2: loop
cue_entry_status:
0: disabled
1: enabled
4: active_loop
track_mood:
1: high
2: mid
3: low
mood_low_phrase:
1: intro
2: verse_1
3: verse_1b # Just displayed as "Verse 1" in rekordbox.
4: verse_1c # Just displayed as "Verse 1" in rekordbox.
5: verse_2
6: verse_2b # Just displayed as "Verse 2" in rekordbox.
7: verse_2c # Just displayed as "Verse 2" in rekordbox.
8: bridge
9: chorus
10: outro
mood_mid_phrase:
1: intro
2: verse_1
3: verse_2
4: verse_3
5: verse_4
6: verse_5
7: verse_6
8: bridge
9: chorus
10: outro
mood_high_phrase:
1: intro
2: up
3: down
5: chorus
6: outro
track_bank:
0: default
1: cool
2: natural
3: hot
4: subtle
5: warm
6: vivid
7: club_1
8: club_2
+897
View File
@@ -0,0 +1,897 @@
# This is a generated file! Please edit source .ksy file and use kaitai-struct-compiler to rebuild
# type: ignore
import kaitaistruct
from kaitaistruct import KaitaiStruct, KaitaiStream, BytesIO
from enum import IntEnum
import struct
if getattr(kaitaistruct, 'API_VERSION', (0, 9)) < (0, 11):
raise Exception("Incompatible Kaitai Struct Python API: 0.11 or later is required, but you have %s" % (kaitaistruct.__version__))
class RekordboxAnlz(KaitaiStruct):
"""These files are created by rekordbox when analyzing audio tracks
to facilitate DJ performance. They include waveforms, beat grids
(information about the precise time at which each beat occurs),
time indices to allow efficient seeking to specific positions
inside variable bit-rate audio streams, and lists of memory cues
and loop points. They are used by Pioneer professional DJ
equipment.
The format has been reverse-engineered to facilitate sophisticated
integrations with light and laser shows, videos, and other musical
instruments, by supporting deep knowledge of what is playing and
what is coming next through monitoring the network communications
of the players.
.. seealso::
Source - https://reverseengineering.stackexchange.com/questions/4311/help-reversing-a-edb-database-file-for-pioneers-rekordbox-software
"""
class CueEntryStatus(IntEnum):
disabled = 0
enabled = 1
active_loop = 4
class CueEntryType(IntEnum):
memory_cue = 1
loop = 2
class CueListType(IntEnum):
memory_cues = 0
hot_cues = 1
class MoodHighPhrase(IntEnum):
intro = 1
up = 2
down = 3
chorus = 5
outro = 6
class MoodLowPhrase(IntEnum):
intro = 1
verse_1 = 2
verse_1b = 3
verse_1c = 4
verse_2 = 5
verse_2b = 6
verse_2c = 7
bridge = 8
chorus = 9
outro = 10
class MoodMidPhrase(IntEnum):
intro = 1
verse_1 = 2
verse_2 = 3
verse_3 = 4
verse_4 = 5
verse_5 = 6
verse_6 = 7
bridge = 8
chorus = 9
outro = 10
class SectionTags(IntEnum):
cues_2 = 1346588466
cues = 1346588482
path = 1347441736
beat_grid = 1347507290
song_structure = 1347638089
vbr = 1347830354
wave_preview = 1347895638
wave_tiny = 1347900978
wave_scroll = 1347900979
wave_color_preview = 1347900980
wave_color_scroll = 1347900981
wave_3band_preview = 1347900982
wave_3band_scroll = 1347900983
class TrackBank(IntEnum):
default = 0
cool = 1
natural = 2
hot = 3
subtle = 4
warm = 5
vivid = 6
club_1 = 7
club_2 = 8
class TrackMood(IntEnum):
high = 1
mid = 2
low = 3
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz, self).__init__(_io)
self._parent = _parent
self._root = _root or self
self._read()
def _read(self):
self.magic = self._io.read_bytes(4)
if not self.magic == b"\x50\x4D\x41\x49":
raise kaitaistruct.ValidationNotEqualError(b"\x50\x4D\x41\x49", self.magic, self._io, u"/seq/0")
self.len_header = self._io.read_u4be()
self.len_file = self._io.read_u4be()
self._unnamed3 = self._io.read_bytes(self.len_header - self._io.pos())
self.sections = []
i = 0
while not self._io.is_eof():
self.sections.append(RekordboxAnlz.TaggedSection(self._io, self, self._root))
i += 1
def _fetch_instances(self):
pass
for i in range(len(self.sections)):
pass
self.sections[i]._fetch_instances()
class BeatGridBeat(KaitaiStruct):
"""Describes an individual beat in a beat grid.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.BeatGridBeat, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.beat_number = self._io.read_u2be()
self.tempo = self._io.read_u2be()
self.time = self._io.read_u4be()
def _fetch_instances(self):
pass
class BeatGridTag(KaitaiStruct):
"""Holds a list of all the beats found within the track, recording
their bar position, the time at which they occur, and the tempo
at that point.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.BeatGridTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self._unnamed0 = self._io.read_u4be()
self._unnamed1 = self._io.read_u4be()
self.num_beats = self._io.read_u4be()
self.beats = []
for i in range(self.num_beats):
self.beats.append(RekordboxAnlz.BeatGridBeat(self._io, self, self._root))
def _fetch_instances(self):
pass
for i in range(len(self.beats)):
pass
self.beats[i]._fetch_instances()
class CueEntry(KaitaiStruct):
"""A cue list entry. Can either represent a memory cue or a loop.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.CueEntry, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.magic = self._io.read_bytes(4)
if not self.magic == b"\x50\x43\x50\x54":
raise kaitaistruct.ValidationNotEqualError(b"\x50\x43\x50\x54", self.magic, self._io, u"/types/cue_entry/seq/0")
self.len_header = self._io.read_u4be()
self.len_entry = self._io.read_u4be()
self.hot_cue = self._io.read_u4be()
self.status = KaitaiStream.resolve_enum(RekordboxAnlz.CueEntryStatus, self._io.read_u4be())
self._unnamed5 = self._io.read_u4be()
self.order_first = self._io.read_u2be()
self.order_last = self._io.read_u2be()
self.type = KaitaiStream.resolve_enum(RekordboxAnlz.CueEntryType, self._io.read_u1())
self._unnamed9 = self._io.read_bytes(3)
self.time = self._io.read_u4be()
self.loop_time = self._io.read_u4be()
self._unnamed12 = self._io.read_bytes(16)
def _fetch_instances(self):
pass
class CueExtendedEntry(KaitaiStruct):
"""A cue extended list entry. Can either describe a memory cue or a
loop.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.CueExtendedEntry, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.magic = self._io.read_bytes(4)
if not self.magic == b"\x50\x43\x50\x32":
raise kaitaistruct.ValidationNotEqualError(b"\x50\x43\x50\x32", self.magic, self._io, u"/types/cue_extended_entry/seq/0")
self.len_header = self._io.read_u4be()
self.len_entry = self._io.read_u4be()
self.hot_cue = self._io.read_u4be()
self.type = KaitaiStream.resolve_enum(RekordboxAnlz.CueEntryType, self._io.read_u1())
self._unnamed5 = self._io.read_bytes(3)
self.time = self._io.read_u4be()
self.loop_time = self._io.read_u4be()
self.color_id = self._io.read_u1()
self._unnamed9 = self._io.read_bytes(7)
self.loop_numerator = self._io.read_u2be()
self.loop_denominator = self._io.read_u2be()
if self.len_entry > 43:
pass
self.len_comment = self._io.read_u4be()
if self.len_entry > 43:
pass
self.comment = (self._io.read_bytes(self.len_comment)).decode(u"UTF-16BE")
if self.len_entry - self.len_comment > 44:
pass
self.color_code = self._io.read_u1()
if self.len_entry - self.len_comment > 45:
pass
self.color_red = self._io.read_u1()
if self.len_entry - self.len_comment > 46:
pass
self.color_green = self._io.read_u1()
if self.len_entry - self.len_comment > 47:
pass
self.color_blue = self._io.read_u1()
if self.len_entry - self.len_comment > 48:
pass
self._unnamed18 = self._io.read_bytes((self.len_entry - 48) - self.len_comment)
def _fetch_instances(self):
pass
if self.len_entry > 43:
pass
if self.len_entry > 43:
pass
if self.len_entry - self.len_comment > 44:
pass
if self.len_entry - self.len_comment > 45:
pass
if self.len_entry - self.len_comment > 46:
pass
if self.len_entry - self.len_comment > 47:
pass
if self.len_entry - self.len_comment > 48:
pass
class CueExtendedTag(KaitaiStruct):
"""A variation of cue_tag which was introduced with the nxs2 line,
and adds descriptive names. (Still comes in two forms, either
holding memory cues and loop points, or holding hot cues and
loop points.) Also includes hot cues D through H and color assignment.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.CueExtendedTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.type = KaitaiStream.resolve_enum(RekordboxAnlz.CueListType, self._io.read_u4be())
self.num_cues = self._io.read_u2be()
self._unnamed2 = self._io.read_bytes(2)
self.cues = []
for i in range(self.num_cues):
self.cues.append(RekordboxAnlz.CueExtendedEntry(self._io, self, self._root))
def _fetch_instances(self):
pass
for i in range(len(self.cues)):
pass
self.cues[i]._fetch_instances()
class CueTag(KaitaiStruct):
"""Stores either a list of ordinary memory cues and loop points, or
a list of hot cues and loop points.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.CueTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.type = KaitaiStream.resolve_enum(RekordboxAnlz.CueListType, self._io.read_u4be())
self._unnamed1 = self._io.read_bytes(2)
self.num_cues = self._io.read_u2be()
self.memory_count = self._io.read_u4be()
self.cues = []
for i in range(self.num_cues):
self.cues.append(RekordboxAnlz.CueEntry(self._io, self, self._root))
def _fetch_instances(self):
pass
for i in range(len(self.cues)):
pass
self.cues[i]._fetch_instances()
class PathTag(KaitaiStruct):
"""Stores the file path of the audio file to which this analysis
applies.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.PathTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.len_path = self._io.read_u4be()
if self.len_path > 1:
pass
self.path = (self._io.read_bytes(self.len_path - 2)).decode(u"UTF-16BE")
def _fetch_instances(self):
pass
if self.len_path > 1:
pass
class PhraseHigh(KaitaiStruct):
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.PhraseHigh, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.id = KaitaiStream.resolve_enum(RekordboxAnlz.MoodHighPhrase, self._io.read_u2be())
def _fetch_instances(self):
pass
class PhraseLow(KaitaiStruct):
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.PhraseLow, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.id = KaitaiStream.resolve_enum(RekordboxAnlz.MoodLowPhrase, self._io.read_u2be())
def _fetch_instances(self):
pass
class PhraseMid(KaitaiStruct):
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.PhraseMid, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.id = KaitaiStream.resolve_enum(RekordboxAnlz.MoodMidPhrase, self._io.read_u2be())
def _fetch_instances(self):
pass
class SongStructureBody(KaitaiStruct):
"""Stores the rest of the song structure tag, which can only be
parsed after unmasking.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.SongStructureBody, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.mood = KaitaiStream.resolve_enum(RekordboxAnlz.TrackMood, self._io.read_u2be())
self._unnamed1 = self._io.read_bytes(6)
self.end_beat = self._io.read_u2be()
self._unnamed3 = self._io.read_bytes(2)
self.raw_bank = self._io.read_u1()
self._unnamed5 = self._io.read_bytes(1)
self.entries = []
for i in range(self._parent.len_entries):
self.entries.append(RekordboxAnlz.SongStructureEntry(self._io, self, self._root))
def _fetch_instances(self):
pass
for i in range(len(self.entries)):
pass
self.entries[i]._fetch_instances()
@property
def bank(self):
"""The stylistic bank which can be assigned to the track in rekordbox Lighting mode, if raw_bank has a legal value.
"""
if hasattr(self, '_m_bank'):
return self._m_bank
if self.raw_bank < 9:
pass
self._m_bank = KaitaiStream.resolve_enum(RekordboxAnlz.TrackBank, self.raw_bank)
return getattr(self, '_m_bank', None)
class SongStructureEntry(KaitaiStruct):
"""A song structure entry, represents a single phrase.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.SongStructureEntry, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.index = self._io.read_u2be()
self.beat = self._io.read_u2be()
_on = self._parent.mood
if _on == RekordboxAnlz.TrackMood.high:
pass
self.kind = RekordboxAnlz.PhraseHigh(self._io, self, self._root)
elif _on == RekordboxAnlz.TrackMood.low:
pass
self.kind = RekordboxAnlz.PhraseLow(self._io, self, self._root)
elif _on == RekordboxAnlz.TrackMood.mid:
pass
self.kind = RekordboxAnlz.PhraseMid(self._io, self, self._root)
else:
pass
self.kind = RekordboxAnlz.PhraseMid(self._io, self, self._root)
self._unnamed3 = self._io.read_bytes(1)
self.k1 = self._io.read_u1()
self._unnamed5 = self._io.read_bytes(1)
self.k2 = self._io.read_u1()
self._unnamed7 = self._io.read_bytes(1)
self.b = self._io.read_u1()
self.beat2 = self._io.read_u2be()
self.beat3 = self._io.read_u2be()
self.beat4 = self._io.read_u2be()
self._unnamed12 = self._io.read_bytes(1)
self.k3 = self._io.read_u1()
self._unnamed14 = self._io.read_bytes(1)
self.fill = self._io.read_u1()
self.beat_fill = self._io.read_u2be()
def _fetch_instances(self):
pass
_on = self._parent.mood
if _on == RekordboxAnlz.TrackMood.high:
pass
self.kind._fetch_instances()
elif _on == RekordboxAnlz.TrackMood.low:
pass
self.kind._fetch_instances()
elif _on == RekordboxAnlz.TrackMood.mid:
pass
self.kind._fetch_instances()
else:
pass
self.kind._fetch_instances()
class SongStructureTag(KaitaiStruct):
"""Stores the song structure, also known as phrases (intro, verse,
bridge, chorus, up, down, outro).
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.SongStructureTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.len_entry_bytes = self._io.read_u4be()
self.len_entries = self._io.read_u2be()
self._raw__raw_body = self._io.read_bytes_full()
self._raw_body = KaitaiStream.process_xor_many(self._raw__raw_body, (self.mask if self.is_masked else b"\x00"))
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.SongStructureBody(_io__raw_body, self, self._root)
def _fetch_instances(self):
pass
self.body._fetch_instances()
_ = self.raw_mood
if hasattr(self, '_m_raw_mood'):
pass
@property
def c(self):
if hasattr(self, '_m_c'):
return self._m_c
self._m_c = self.len_entries
return getattr(self, '_m_c', None)
@property
def is_masked(self):
if hasattr(self, '_m_is_masked'):
return self._m_is_masked
self._m_is_masked = self.raw_mood > 20
return getattr(self, '_m_is_masked', None)
@property
def mask(self):
if hasattr(self, '_m_mask'):
return self._m_mask
self._m_mask = struct.pack('19B', (203 + self.c), (225 + self.c), (238 + self.c), (250 + self.c), (229 + self.c), (238 + self.c), (173 + self.c), (238 + self.c), (233 + self.c), (210 + self.c), (233 + self.c), (235 + self.c), (225 + self.c), (233 + self.c), (243 + self.c), (232 + self.c), (233 + self.c), (244 + self.c), (225 + self.c))
return getattr(self, '_m_mask', None)
@property
def raw_mood(self):
"""This is a way to tell whether the rest of the tag has been masked. The value is supposed
to range from 1 to 3, but in masked files it will be much larger.
"""
if hasattr(self, '_m_raw_mood'):
return self._m_raw_mood
_pos = self._io.pos()
self._io.seek(6)
self._m_raw_mood = self._io.read_u2be()
self._io.seek(_pos)
return getattr(self, '_m_raw_mood', None)
class TaggedSection(KaitaiStruct):
"""A type-tagged file section, identified by a four-byte magic
sequence, with a header specifying its length, and whose payload
is determined by the type tag.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.TaggedSection, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.fourcc = KaitaiStream.resolve_enum(RekordboxAnlz.SectionTags, self._io.read_s4be())
self.len_header = self._io.read_u4be()
self.len_tag = self._io.read_u4be()
_on = self.fourcc
if _on == RekordboxAnlz.SectionTags.beat_grid:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.BeatGridTag(_io__raw_body, self, self._root)
elif _on == RekordboxAnlz.SectionTags.cues:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.CueTag(_io__raw_body, self, self._root)
elif _on == RekordboxAnlz.SectionTags.cues_2:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.CueExtendedTag(_io__raw_body, self, self._root)
elif _on == RekordboxAnlz.SectionTags.path:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.PathTag(_io__raw_body, self, self._root)
elif _on == RekordboxAnlz.SectionTags.song_structure:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.SongStructureTag(_io__raw_body, self, self._root)
elif _on == RekordboxAnlz.SectionTags.vbr:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.VbrTag(_io__raw_body, self, self._root)
elif _on == RekordboxAnlz.SectionTags.wave_3band_preview:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.Wave3bandPreviewTag(_io__raw_body, self, self._root)
elif _on == RekordboxAnlz.SectionTags.wave_3band_scroll:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.Wave3bandScrollTag(_io__raw_body, self, self._root)
elif _on == RekordboxAnlz.SectionTags.wave_color_preview:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.WaveColorPreviewTag(_io__raw_body, self, self._root)
elif _on == RekordboxAnlz.SectionTags.wave_color_scroll:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.WaveColorScrollTag(_io__raw_body, self, self._root)
elif _on == RekordboxAnlz.SectionTags.wave_preview:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.WavePreviewTag(_io__raw_body, self, self._root)
elif _on == RekordboxAnlz.SectionTags.wave_scroll:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.WaveScrollTag(_io__raw_body, self, self._root)
elif _on == RekordboxAnlz.SectionTags.wave_tiny:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.WavePreviewTag(_io__raw_body, self, self._root)
else:
pass
self._raw_body = self._io.read_bytes(self.len_tag - 12)
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
self.body = RekordboxAnlz.UnknownTag(_io__raw_body, self, self._root)
def _fetch_instances(self):
pass
_on = self.fourcc
if _on == RekordboxAnlz.SectionTags.beat_grid:
pass
self.body._fetch_instances()
elif _on == RekordboxAnlz.SectionTags.cues:
pass
self.body._fetch_instances()
elif _on == RekordboxAnlz.SectionTags.cues_2:
pass
self.body._fetch_instances()
elif _on == RekordboxAnlz.SectionTags.path:
pass
self.body._fetch_instances()
elif _on == RekordboxAnlz.SectionTags.song_structure:
pass
self.body._fetch_instances()
elif _on == RekordboxAnlz.SectionTags.vbr:
pass
self.body._fetch_instances()
elif _on == RekordboxAnlz.SectionTags.wave_3band_preview:
pass
self.body._fetch_instances()
elif _on == RekordboxAnlz.SectionTags.wave_3band_scroll:
pass
self.body._fetch_instances()
elif _on == RekordboxAnlz.SectionTags.wave_color_preview:
pass
self.body._fetch_instances()
elif _on == RekordboxAnlz.SectionTags.wave_color_scroll:
pass
self.body._fetch_instances()
elif _on == RekordboxAnlz.SectionTags.wave_preview:
pass
self.body._fetch_instances()
elif _on == RekordboxAnlz.SectionTags.wave_scroll:
pass
self.body._fetch_instances()
elif _on == RekordboxAnlz.SectionTags.wave_tiny:
pass
self.body._fetch_instances()
else:
pass
self.body._fetch_instances()
class UnknownTag(KaitaiStruct):
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.UnknownTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
pass
def _fetch_instances(self):
pass
class VbrTag(KaitaiStruct):
"""Stores an index allowing rapid seeking to particular times
within a variable-bitrate audio file.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.VbrTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self._unnamed0 = self._io.read_u4be()
self.index = []
for i in range(400):
self.index.append(self._io.read_u4be())
def _fetch_instances(self):
pass
for i in range(len(self.index)):
pass
class Wave3bandPreviewTag(KaitaiStruct):
"""The minimalist CDJ-3000 waveform preview image suitable for display
above the touch strip for jumping to a track position.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.Wave3bandPreviewTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.len_entry_bytes = self._io.read_u4be()
self.len_entries = self._io.read_u4be()
self.entries = self._io.read_bytes(self.len_entries * self.len_entry_bytes)
def _fetch_instances(self):
pass
class Wave3bandScrollTag(KaitaiStruct):
"""The minimalist CDJ-3000 waveform image suitable for scrolling along
as a track plays on newer high-resolution hardware.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.Wave3bandScrollTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.len_entry_bytes = self._io.read_u4be()
self.len_entries = self._io.read_u4be()
self._unnamed2 = self._io.read_u4be()
self.entries = self._io.read_bytes(self.len_entries * self.len_entry_bytes)
def _fetch_instances(self):
pass
class WaveColorPreviewTag(KaitaiStruct):
"""A larger, colorful waveform preview image suitable for display
above the touch strip for jumping to a track position on newer
high-resolution players.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.WaveColorPreviewTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.len_entry_bytes = self._io.read_u4be()
self.len_entries = self._io.read_u4be()
self._unnamed2 = self._io.read_u4be()
self.entries = self._io.read_bytes(self.len_entries * self.len_entry_bytes)
def _fetch_instances(self):
pass
class WaveColorScrollTag(KaitaiStruct):
"""A larger, colorful waveform image suitable for scrolling along
as a track plays on newer high-resolution hardware. Also
contains a higher-resolution blue/white waveform.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.WaveColorScrollTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.len_entry_bytes = self._io.read_u4be()
self.len_entries = self._io.read_u4be()
self._unnamed2 = self._io.read_u4be()
self.entries = self._io.read_bytes(self.len_entries * self.len_entry_bytes)
def _fetch_instances(self):
pass
class WavePreviewTag(KaitaiStruct):
"""Stores a waveform preview image suitable for display above
the touch strip for jumping to a track position.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.WavePreviewTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.len_data = self._io.read_u4be()
self._unnamed1 = self._io.read_u4be()
if self._parent.len_tag > self._parent.len_header:
pass
self.data = self._io.read_bytes(self.len_data)
def _fetch_instances(self):
pass
if self._parent.len_tag > self._parent.len_header:
pass
class WaveScrollTag(KaitaiStruct):
"""A larger waveform image suitable for scrolling along as a track
plays.
"""
def __init__(self, _io, _parent=None, _root=None):
super(RekordboxAnlz.WaveScrollTag, self).__init__(_io)
self._parent = _parent
self._root = _root
self._read()
def _read(self):
self.len_entry_bytes = self._io.read_u4be()
self.len_entries = self._io.read_u4be()
self._unnamed2 = self._io.read_u4be()
self.entries = self._io.read_bytes(self.len_entries * self.len_entry_bytes)
def _fetch_instances(self):
pass
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+2
View File
@@ -0,0 +1,2 @@
def hello() -> str:
return "Hello from maple!"
+3
View File
@@ -0,0 +1,3 @@
from maple.cli import main
raise SystemExit(main())
+500
View File
@@ -0,0 +1,500 @@
"""Audio analysis: BPM + beat grid via mixxx-analyzer, plus waveform envelopes.
BPM and the beat grid come from `mixxx-analyzer` — Python bindings around
Mixxx's own analysis (the Queen Mary tempo tracker) — which produces a
CDJ-grade beat grid. Its wheel bundles a native binary, so there's no extra
system dependency. ffmpeg still decodes the source to 44.1 kHz mono float32 PCM
for the per-column waveform envelopes (PWAV / PWV3 / PWV4 / PWV5).
The decoded audio never touches disk — ffmpeg's `-f f32le` output is piped
straight to numpy.
Everything an analysis produces is held by the `Analysis` dataclass, composed of
`BeatGrid` (BPM + PQTZ beats), `MonoWaveforms` (PWAV/PWV2/PWV3) and
`BandWaveforms` (the 3-band colour streams). Build one with
`Analysis.analyze(path)` (or `Analysis.from_track(track)`); persist it with
`.save(cache_dir)` and read it back with `Analysis.load(cache_dir, isrc)`.
"""
from __future__ import annotations
import json
import shutil
import subprocess
from base64 import b64decode, b64encode
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING, NamedTuple
import mixxx_analyzer
import numpy as np
from loguru import logger
from scipy.signal import butter, sosfilt
if TYPE_CHECKING:
from maple.track import Track
SAMPLE_RATE = 44100
# Waveform envelope dimensions.
# PWAV preview is **200 segments × 2 bytes** = 400 bytes — per Beat Link's
# reader, height = data[seg*2] & 0x1f, brightness = data[seg*2+1] & 0x07. (An
# earlier single-packed-byte reading was wrong; the player decoded it as garbled
# height/intensity pairs → "always full, never bright".) PWV2 = 100 columns × 1
# byte, 4-bit height. PWV3 (detail/scroll) genuinely IS 1 byte/col.
PWAV_SEGMENTS = 200 # PWAV: 200 segments × 2 bytes (height, brightness)
PWV2_COLS = 100 # PWV2: 100 single bytes (4-bit height)
SCROLL_FPS = 150 # PWV3 / PWV5 / PWV7 = ~150 cols/sec
PWAV_HEIGHT_LEVELS = 31 # 5-bit (0-31), bits 0-4
PWV2_HEIGHT_LEVELS = 15 # 4-bit (0-15), bits 0-3
PWV6_COLS = 1200 # 3-band / colour preview width (fixed by spec)
CACHE_VERSION = 9 # PWAV paired (2B/seg); treble-based brightness (not crest)
class Band(NamedTuple):
"""A band-pass frequency range, in Hz."""
lo_hz: int
hi_hz: int
# 3-band waveform split: bass (kick + sub), mid (snare, vocals, chords),
# high (hats, cymbals).
BAND_LOW = Band(20, 200)
BAND_MID = Band(200, 2000)
BAND_HIGH = Band(2000, 15000)
class Beat(NamedTuple):
"""One PQTZ beat-grid row. Unpacks straight into `struct.pack(">HHI", *beat)`."""
number: int # 1..4 (cycles from the anchor; no downbeat info)
tempo_x100: int # BPM × 100
time_ms: int # position in milliseconds
class Bands(NamedTuple):
"""A low/mid/high triple of per-column waveform byte-streams."""
low: bytes = b""
mid: bytes = b""
high: bytes = b""
# --------------------------------------------------------------------------- #
# mixxx-analyzer + ffmpeg decode #
# --------------------------------------------------------------------------- #
def _mixxx_analyze(path: Path) -> mixxx_analyzer.AnalysisResult:
"""Run mixxx-analyzer (Mixxx's BPM / beat-grid / key algorithms) on `path`.
Returns the `AnalysisResult` (fields incl. `bpm` and `beatgrid` — beat
positions in seconds). BPM/beat detection is integral to an export, so a
failure here propagates rather than degrading to an analysis with no grid."""
return mixxx_analyzer.analyze(str(path))
def _project_pqtz(beatgrid: list[float], bpm: float) -> list[Beat]:
"""Project a continuous PQTZ grid from mixxx's tempo + phase.
Returns `[Beat(number 1-4, tempo×100, time_ms), ...]`. mixxx gives an
accurate BPM and beat phase, but its raw beat list can have GAPS in quiet
sections (e.g. 907 detected beats where a continuous 160 BPM grid has
~1010). rekordbox/CDJs expect a gap-free grid — beat-sync and quantise lock
to it — so we anchor on mixxx's first beat, walk back to t≥0 to cover the
intro, and emit one beat every 60/bpm seconds through to the last detected
beat. beat_number cycles 1→4 from the anchor (no downbeat info)."""
if not beatgrid or bpm <= 0:
return []
interval_ms = 60000.0 / bpm
first_ms = beatgrid[0] * 1000.0
last_ms = beatgrid[-1] * 1000.0
n_before = int(first_ms / interval_ms) # preserve phase back to t≥0
anchor = first_ms - n_before * interval_ms
tempo_x100 = round(bpm * 100)
grid: list[Beat] = []
i = 0
t = anchor
while t <= last_ms + interval_ms:
grid.append(Beat((i % 4) + 1, tempo_x100, int(round(t))))
i += 1
t = anchor + i * interval_ms
return grid
def _decode_pcm(path: Path) -> np.ndarray:
"""Decode `path` to a 1D float32 numpy array of mono PCM at SAMPLE_RATE.
Raises `RuntimeError` if ffmpeg is missing or fails to decode — waveform
extraction is integral, so a failure here is fatal rather than swallowed
into an empty array (which would yield a waveform-less ANLZ).
"""
if not shutil.which("ffmpeg"):
raise RuntimeError("ffmpeg not on PATH; cannot extract waveform")
proc = subprocess.run(
["ffmpeg", "-nostdin", "-v", "error",
"-i", str(path),
"-f", "f32le", "-ac", "1", "-ar", str(SAMPLE_RATE), "-"],
capture_output=True,
)
if proc.returncode != 0:
raise RuntimeError(
f"ffmpeg failed for {path.name}: "
f"{proc.stderr.decode('utf-8', 'replace').strip()}"
)
return np.frombuffer(proc.stdout, dtype=np.float32)
# --------------------------------------------------------------------------- #
# Quantisation + per-column binning (the waveform DSP) #
# --------------------------------------------------------------------------- #
# Height: gate near-silence to 0, then a linear map to the level range
# (QUANT_GAMMA=1.0). Heights use per-track-normalised RMS loudness, which on the
# reference export spreads low (scroll ~5.7/31, overview ~13/31) with no
# saturation. Colour/whiteness use a sub-linear lift (COLOR_GAMMA) so quiet
# bands still read vivid mid-range instead of near-zero.
SILENCE_GATE_PCT = 0.02 # < 2% of peak → 0
QUANT_GAMMA = 1.0 # HEIGHT: linear; do NOT lift mids toward max
COLOR_GAMMA = 0.4 # COLOUR/whiteness: sub-linear lift
# PWAV (preview) height uses an ABSOLUTE loudness scale instead of per-track
# normalisation, so a quiet track/section reads short and a loud one tall
# (per-track norm always pinned the loudest column to full → "always too high").
# height = round(rms * PWAV_HEIGHT_SCALE), clamped to 31. The scale was DERIVED
# by least-squares fitting the genuine reference export's PWAV heights against
# per-column audio RMS across 3 tracks: 42.2 / 43.0 / 39.9 → 42 (mean err ~2/31;
# reference peaks at ~24, i.e. full ≈ RMS 0.74). (PWV3 keeps the per-track scale
# — it's good as-is.)
PWAV_HEIGHT_SCALE = 42
def _quantise(peaks: np.ndarray, levels: int, peak_max: float | None = None,
gamma: float | None = None) -> np.ndarray:
"""Gate + (optional gamma) + quantise to 0..levels uint8.
`peak_max` lets callers pass a SHARED normaliser (e.g. across bands) so the
relative energy between channels is preserved; defaults to this array's own
max. `gamma` overrides QUANT_GAMMA (use COLOR_GAMMA to lift colour)."""
g = QUANT_GAMMA if gamma is None else gamma
pm = float(peaks.max()) if peak_max is None else peak_max
if pm <= 0:
return np.zeros(peaks.size, dtype=np.uint8)
gated = np.where(peaks < pm * SILENCE_GATE_PCT, 0.0, peaks)
normalised = np.clip(gated / pm, 0.0, 1.0)
curved = normalised if g == 1.0 else np.power(normalised, g)
return np.clip(np.round(curved * levels), 0, levels).astype(np.uint8)
def _col_rms(values: np.ndarray, n_cols: int) -> np.ndarray:
"""Per-column RMS over n_cols evenly-split chunks (loudness, not peak)."""
chunks = np.array_split(values.astype(np.float64), n_cols)
return np.array([float(np.sqrt(np.mean(c * c))) if len(c) else 0.0 for c in chunks])
def _env_peaks(env: np.ndarray, n_cols: int) -> np.ndarray:
chunks = np.array_split(env, n_cols)
return np.array([float(c.max()) if len(c) else 0.0 for c in chunks])
def _packed_mono(samples: np.ndarray, hi_env: np.ndarray, n_cols: int,
height_levels: int = PWAV_HEIGHT_LEVELS,
abs_scale: float | None = None) -> bytes:
"""One byte per column = (whiteness << 5) | height — the real PWAV/PWV3
on-disk format. whiteness = lifted treble (high-band) peak.
height: when `abs_scale` is given, an ABSOLUTE loudness map
`round(rms * abs_scale)` (quiet → short, loud → tall — used by PWAV); when
None, per-track-normalised RMS via `_quantise` (loudest column → full —
used by PWV3)."""
if n_cols <= 0 or samples.size == 0:
return bytes(n_cols)
rms = _col_rms(np.abs(samples), n_cols)
if abs_scale is not None:
height = np.clip(np.round(rms * abs_scale), 0, height_levels).astype(np.uint8)
else:
height = _quantise(rms, height_levels)
white = _quantise(_env_peaks(hi_env, n_cols), 7, gamma=COLOR_GAMMA) # lifted
white = np.where(height > 0, white, 0).astype(np.uint8) # silence → no white
return ((white << 5) | height).astype(np.uint8).tobytes()
def _packed_preview(samples: np.ndarray, hi_env: np.ndarray,
segments: int = PWAV_SEGMENTS) -> bytes:
"""PWAV preview — **2 bytes per segment** (NOT one packed byte). This is the
format Beat Link / the CDJ actually read:
even byte: bits 0-4 = height (0-31) ← data[seg*2] & 0x1f
odd byte: bits 0-2 = brightness (0-7) ← data[seg*2+1] & 0x07 (>=5 → cyan)
height = absolute loudness `round(rms*PWAV_HEIGHT_SCALE)` (fit to the
reference: full ≈ RMS 0.74). brightness = the same treble (high-band)
whiteness PWV3 uses — it matches the reference brightness distribution
(~14-38% ≥5) far better than a crest factor (which sat ≥5 on ~all columns).
Whiteness is mirrored into the high 3 bits of both bytes, matching the
reference layout (even≈odd).
"""
if segments <= 0 or samples.size == 0:
return bytes(segments * 2)
rms = _col_rms(np.abs(samples), segments)
height = np.clip(np.round(rms * PWAV_HEIGHT_SCALE), 0, 31).astype(np.uint8)
# brightness: modest treble level (linear, no lift). The reference's exact
# brightness channel doesn't track any computable feature (uncorrelated with
# treble/height/crest across reference tracks), so this just gives some
# transient/treble sparkle without washing the whole preview cyan.
bright = _quantise(_env_peaks(hi_env, segments), 7)
bright = np.where(height > 0, bright, 0).astype(np.uint8) # silence → no bright
even = ((bright << 5) | height).astype(np.uint8) # &0x1f → height
odd = ((bright << 5) | bright).astype(np.uint8) # &0x07 → brightness
out = np.empty(segments * 2, dtype=np.uint8)
out[0::2] = even
out[1::2] = odd
return out.tobytes()
def _packed_tiny(samples: np.ndarray, n_cols: int) -> bytes:
"""PWV2: one byte per column, 4-bit height (0-15) in the low nibble."""
if n_cols <= 0 or samples.size == 0:
return bytes(n_cols)
return _quantise(_col_rms(np.abs(samples), n_cols), PWV2_HEIGHT_LEVELS).astype(np.uint8).tobytes()
def _bin_bands(lo_env: np.ndarray, mid_env: np.ndarray, hi_env: np.ndarray,
n_cols: int, shared: bool, levels: int = 255) -> Bands:
"""Bin three band envelopes (per-column PEAK) into n_cols, lifted by
COLOR_GAMMA. `shared=True` normalises all three by one SHARED peak → natural
bass→treble rolloff (PWV4 overview). `shared=False` normalises each band to
its OWN peak → vivid balanced colour where sustained bands read strong
(PWV5 colour scroll)."""
lp, mp, hp = _env_peaks(lo_env, n_cols), _env_peaks(mid_env, n_cols), _env_peaks(hi_env, n_cols)
if shared:
s = max(float(lp.max()), float(mp.max()), float(hp.max()), 1e-9)
nlo = nmd = nhi = s
else:
nlo, nmd, nhi = None, None, None
return Bands(
_quantise(lp, levels, nlo, gamma=COLOR_GAMMA).tobytes(),
_quantise(mp, levels, nmd, gamma=COLOR_GAMMA).tobytes(),
_quantise(hp, levels, nhi, gamma=COLOR_GAMMA).tobytes(),
)
def _band_envelopes(samples: np.ndarray, sr: int) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
"""Per-sample low/mid/high envelopes (4-pole Butterworth band-pass + abs)."""
def env(band: Band) -> np.ndarray:
sos = butter(4, [band.lo_hz, band.hi_hz], btype="band", fs=sr, output="sos")
return np.abs(sosfilt(sos, samples)).astype(np.float32)
return env(BAND_LOW), env(BAND_MID), env(BAND_HIGH)
# --------------------------------------------------------------------------- #
# ISRC-keyed cache paths #
# --------------------------------------------------------------------------- #
def _sanitise_isrc(isrc: str) -> str:
"""Reduce an ISRC to a filename-safe form (uppercase, alnum only)."""
return "".join(c for c in (isrc or "").upper() if c.isalnum())
def cache_path_for(cache_dir: Path, isrc: str) -> Path | None:
"""Return the path the cache file for `isrc` lives at, or None if no isrc."""
s = _sanitise_isrc(isrc)
if not s:
return None
return cache_dir / f"{s}.json"
# --------------------------------------------------------------------------- #
# Composed analysis dataclasses #
# --------------------------------------------------------------------------- #
@dataclass
class BeatGrid:
"""BPM + the projected PQTZ beat grid. `bpm == 0.0` means unknown — there is
no `None` here, an absent grid is simply empty."""
bpm: float = 0.0
beats: list[Beat] = field(default_factory=list)
@property
def tempo_x100(self) -> int:
"""BPM × 100, the integer tempo every PQTZ/PQT2 row carries."""
return round(self.bpm * 100)
@classmethod
def from_mixxx(cls, res: mixxx_analyzer.AnalysisResult) -> "BeatGrid":
"""Build from a mixxx `AnalysisResult`. `_project_pqtz` tolerates an
empty beat list, so a result with no usable tempo/phase yields an empty
grid (bpm 0.0) rather than raising."""
bpm = float(res.bpm) if res.bpm else 0.0
beats = _project_pqtz(list(res.beatgrid), bpm) if res.beatgrid else []
return cls(bpm=bpm, beats=beats)
@dataclass
class MonoWaveforms:
"""The single-channel waveform streams.
`preview` = PWAV (200 segments × 2 bytes), `tiny` = PWV2 (100 × 1 byte,
4-bit height), `scroll` = PWV3 (~150 cols/sec, packed whiteness|height)."""
preview: bytes = b""
tiny: bytes = b""
scroll: bytes = b""
@classmethod
def from_audio(cls, samples: np.ndarray, hi_env: np.ndarray, scroll_cols: int) -> "MonoWaveforms":
return cls(
preview=_packed_preview(samples, hi_env, PWAV_SEGMENTS),
tiny=_packed_tiny(samples, PWV2_COLS),
scroll=_packed_mono(samples, hi_env, scroll_cols, PWAV_HEIGHT_LEVELS),
)
@dataclass
class BandWaveforms:
"""The 3-band colour streams. `preview` is SHARED-normalised (energy rolloff,
feeds PWV4/PWV6); `scroll` is per-band-normalised (vivid, feeds PWV5/PWV7)."""
preview: Bands = Bands()
scroll: Bands = Bands()
@classmethod
def from_envelopes(cls, lo_env: np.ndarray, mid_env: np.ndarray,
hi_env: np.ndarray, scroll_cols: int) -> "BandWaveforms":
return cls(
preview=_bin_bands(lo_env, mid_env, hi_env, PWV6_COLS, shared=True),
scroll=_bin_bands(lo_env, mid_env, hi_env, scroll_cols, shared=False),
)
@dataclass
class Analysis:
"""A track's full audio analysis: BPM + beat grid and every waveform stream
the rekordbox ANLZ writer needs. Build with `Analysis.analyze(path)` or
`Analysis.from_track(track)`; cache with `.save()` / `Analysis.load()`."""
isrc: str = ""
grid: BeatGrid = field(default_factory=BeatGrid)
mono: MonoWaveforms = field(default_factory=MonoWaveforms)
bands: BandWaveforms = field(default_factory=BandWaveforms)
# ---- views ----------------------------------------------------------- #
@property
def bpm(self) -> float:
return self.grid.bpm
@property
def beats(self) -> list[Beat]:
return self.grid.beats
# ---- construction ---------------------------------------------------- #
@classmethod
def analyze(cls, path: Path, *, isrc: str = "") -> "Analysis":
"""One-pass decode + analysis of an audio file.
BPM + beat grid come from mixxx-analyzer (independent of the waveform
decode, so a track still gets its grid even if ffmpeg fails). Always
returns a populated `Analysis` — empty waveforms when ffmpeg can't
decode the source."""
grid = BeatGrid.from_mixxx(_mixxx_analyze(path))
samples = _decode_pcm(path)
if samples.size == 0:
return cls(isrc=isrc, grid=grid)
scroll_cols = max(1, int(round(samples.size / SAMPLE_RATE * SCROLL_FPS)))
# Band envelopes first — the high band feeds the mono whiteness channel,
# and all three feed the colour sections.
lo_env, mid_env, hi_env = _band_envelopes(samples, SAMPLE_RATE)
return cls(
isrc=isrc,
grid=grid,
mono=MonoWaveforms.from_audio(samples, hi_env, scroll_cols),
bands=BandWaveforms.from_envelopes(lo_env, mid_env, hi_env, scroll_cols),
)
@classmethod
def from_track(cls, track: "Track") -> "Analysis":
"""Seed `isrc` from the track and compute the rest from `track.path`.
Returns a bare (isrc-only) `Analysis` when the track has no audio
downloaded yet (`track.path is None`)."""
if track.path is None:
return cls(isrc=track.isrc)
return cls.analyze(track.path, isrc=track.isrc)
# ---- cache I/O ------------------------------------------------------- #
def to_cache(self) -> dict:
"""Serialise to the on-disk JSON payload (bytes → base64). Mirrors the
`CACHE_VERSION` v9 layout so existing caches stay readable."""
return {
"version": CACHE_VERSION,
"isrc": _sanitise_isrc(self.isrc),
"bpm": self.grid.bpm or None,
"beats": [list(b) for b in self.grid.beats],
"preview_b64": b64encode(self.mono.preview).decode("ascii"),
"tiny_b64": b64encode(self.mono.tiny).decode("ascii"),
"scroll_b64": b64encode(self.mono.scroll).decode("ascii"),
# 3-band envelopes — kept separate from the mono streams so callers
# can ask for whichever they need without re-decoding.
"preview_low_b64": b64encode(self.bands.preview.low).decode("ascii"),
"preview_mid_b64": b64encode(self.bands.preview.mid).decode("ascii"),
"preview_high_b64": b64encode(self.bands.preview.high).decode("ascii"),
"scroll_low_b64": b64encode(self.bands.scroll.low).decode("ascii"),
"scroll_mid_b64": b64encode(self.bands.scroll.mid).decode("ascii"),
"scroll_high_b64": b64encode(self.bands.scroll.high).decode("ascii"),
}
@classmethod
def from_cache(cls, d: dict) -> "Analysis | None":
"""Rebuild from a `to_cache()` payload, or None if the version is stale."""
if d.get("version") != CACHE_VERSION:
return None
return cls(
isrc=d.get("isrc") or "",
grid=BeatGrid(
bpm=float(d.get("bpm") or 0.0),
beats=[Beat(*b) for b in d.get("beats", [])],
),
mono=MonoWaveforms(
preview=b64decode(d.get("preview_b64") or ""),
tiny=b64decode(d.get("tiny_b64") or ""),
scroll=b64decode(d.get("scroll_b64") or ""),
),
bands=BandWaveforms(
preview=Bands(
b64decode(d.get("preview_low_b64") or ""),
b64decode(d.get("preview_mid_b64") or ""),
b64decode(d.get("preview_high_b64") or ""),
),
scroll=Bands(
b64decode(d.get("scroll_low_b64") or ""),
b64decode(d.get("scroll_mid_b64") or ""),
b64decode(d.get("scroll_high_b64") or ""),
),
),
)
def save(self, cache_dir: Path) -> Path | None:
"""Persist this analysis under `cache_dir`, keyed by ISRC. No-op
(returns None) when the ISRC is empty."""
target = cache_path_for(cache_dir, self.isrc)
if target is None:
return None
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(json.dumps(self.to_cache(), separators=(",", ":")))
return target
@classmethod
def load(cls, cache_dir: Path, isrc: str) -> "Analysis | None":
"""Return the cached analysis for `isrc`, or None if
absent / stale / unreadable."""
target = cache_path_for(cache_dir, isrc)
if target is None or not target.exists():
return None
try:
return cls.from_cache(json.loads(target.read_text()))
except (json.JSONDecodeError, OSError, ValueError) as e:
logger.warning("analysis cache read failed for {}: {}", isrc, e)
return None
+499
View File
@@ -0,0 +1,499 @@
"""Generate a rekordbox ANLZ0000.{DAT,EXT} pair.
ANLZ0000.DAT — what CDJ-900-era through CDJ-3000 firmware reads to display
the loaded-deck BPM, the touch-strip preview waveform, and the cue list:
PPTH player-relative audio path
PVBR variable-bitrate seek index (zero-filled placeholder; matches the
shape rekordbox writes for files where the seek index isn't useful)
PQTZ beat grid (per-beat: beat_number, tempo*100, time_ms)
PWAV 400-byte blue/white touch-strip preview
PWV2 100-byte "tiny" overview waveform
PCOB memory-cue list (empty placeholder)
PCOB hot-cue list (empty placeholder)
ANLZ0000.EXT — extended data; we write a minimal PPTH-only file so newer
firmware that opens .EXT as part of its load flow finds a valid container.
Color / 3-band waveforms (PWV4/PWV5/PWV6/PWV7) belong in .EXT/.2EX and
aren't generated yet.
Field discoveries documented by byte-diffing maple's output against a real
rekordbox export at data/q1-rekordbox/.
All multi-byte fields in ANLZ files are BIG-endian, unlike the pdb.
"""
from __future__ import annotations
from pathlib import Path
from struct import pack
from loguru import logger
from maple.analyze import Analysis, Beat
# Real rekordbox PMAI header carries three non-zero u4 markers in the bytes
# after the standard PMAI/len_header/len_file triple. Verified by hex-dump
# of data/q1-rekordbox/PIONEER/USBANLZ/P020/0002E10A/ANLZ0000.{DAT,EXT,2EX}
# — all three files share the same 16-byte trailer.
PMAI_HEADER_TRAILER = pack(">IIII", 0x00000001, 0x00010000, 0x00010000, 0x00000000)
ANLZ_HEADER_LEN = 28 # PMAI + 2 size u4s + the 16-byte trailer above
def _section_pth(path: str) -> bytes:
"""PPTH section: u4 len_path, then UTF-16BE bytes (null-terminated)."""
utf16 = path.encode("utf-16-be") + b"\x00\x00"
len_path = len(utf16)
body = pack(">I", len_path) + utf16
fourcc = b"PPTH"
len_header = 16 # fourcc + 2 u4 size fields + len_path u4
len_tag = 12 + len(body)
return fourcc + pack(">II", len_header, len_tag) + body
def _section_pvbr() -> bytes:
"""PVBR section: 400-entry VBR seek index. We write a zero-filled
placeholder — rekordbox accepts this for fixed-bitrate sources and it
matches the section's presence in real exports."""
fourcc = b"PVBR"
len_header = 16 # prelude(12) + u4 unnamed0(4)
# 400 u4 seek index + 4 trailer bytes = 1604 bytes of variable data
# (verified by len_tag=1620 in the real Q1.1 export).
body_var = b"\x00" * 1604
unnamed0 = b"\x00\x00\x00\x00"
len_tag = len_header + len(body_var) # 16 + 1604 = 1620
return fourcc + pack(">II", len_header, len_tag) + unnamed0 + body_var
def _section_pqtz(beats: list[Beat]) -> bytes:
"""PQTZ: BeatGridTag — per-beat (beat_number, tempo×100, time_ms).
The CDJ's loaded-deck BPM readout, tempo slider and sync engine all
read from this section. Real rekordbox sets `_unnamed1=0x00080000`
(was 0 in maple's earlier writer — likely the cause of the deck-side
BPM display staying blank even when the grid was otherwise valid).
"""
fourcc = b"PQTZ"
len_header = 24 # prelude(12) + 3 u4 body-header fields
num_beats = len(beats)
body = pack(">III", 0x00000000, 0x00080000, num_beats)
for bn, tempo_x100, t_ms in beats:
body += pack(">HHI", bn, tempo_x100, t_ms)
len_tag = 12 + len(body)
return fourcc + pack(">II", len_header, len_tag) + body
def _section_pwav(preview: bytes) -> bytes:
"""PWAV: WavePreviewTag — 400-byte touch-strip preview, **400 columns ×
1 byte**, each byte = (whiteness << 5) | height (3-bit whiteness, 5-bit
height). `preview` is produced already in that form by
`maple.analyze._packed_mono`. (The earlier "200 height/intensity col-pairs"
reading was wrong — the player decoded those as garbled half-columns with
no whiteness.)
"""
fourcc = b"PWAV"
len_header = 20
len_tag = len_header + len(preview)
return fourcc + pack(">IIII", len_header, len_tag, len(preview), 0x00010000) + preview
def _section_pwv2(tiny: bytes) -> bytes:
"""PWV2: 100-byte tiny preview — 100 columns × 1 byte, 4-bit height
(bits 0-3, 0-15) in the low nibble. `tiny` is produced by
`maple.analyze._packed_tiny`."""
fourcc = b"PWV2"
len_header = 20
len_tag = len_header + len(tiny)
return fourcc + pack(">IIII", len_header, len_tag, len(tiny), 0x00010000) + tiny
def _section_pcob(cue_list_type: int) -> bytes:
"""PCOB: CueTag — cue/loop list. Real exports always include both
sections in a fixed order — hot-cues (type=1) FIRST, then memory-cues
(type=0) — and use `0xffffffff` as the memory_count sentinel for
empty lists. Body: u4 type, 2 bytes unnamed, u2 num_cues, u4
memory_count. Verified against data/q1-rekordbox/.../ANLZ0000.DAT
byte-for-byte."""
fourcc = b"PCOB"
len_header = 24 # prelude(12) + body fixed prefix(12)
body = pack(">I", cue_list_type) + b"\x00\x00" + pack(">HI", 0, 0xFFFFFFFF)
len_tag = 12 + len(body)
return fourcc + pack(">II", len_header, len_tag) + body
def _section_pwv3(scroll: bytes) -> bytes:
"""PWV3: WaveScrollTag — full-track mono scroll waveform. Written
into ANLZ0000.EXT (real rekordbox puts PWV3 there, not in .DAT).
The loaded-deck scroll display reads from here."""
fourcc = b"PWV3"
len_header = 24
len_entry_bytes = 1
len_entries = len(scroll)
len_tag = len_header + len(scroll)
return fourcc + pack(">IIIII", len_header, len_tag, len_entry_bytes, len_entries, 0x00960000) + scroll
def _section_pco2(cue_list_type: int) -> bytes:
"""PCO2: ExtendedCueTag — empty extended cue list. Real exports ship
both type=1 (hot) and type=0 (memory) variants in .EXT alongside the
PCOB sections. Body: u4 type, u4 reserved."""
fourcc = b"PCO2"
len_header = 20 # prelude(12) + body fixed prefix(8)
body = pack(">II", cue_list_type, 0)
len_tag = 12 + len(body)
return fourcc + pack(">II", len_header, len_tag) + body
# --------------------------------------------------------------------------- #
# Colour + 3-band waveform sections (live in .EXT / .2EX) #
# --------------------------------------------------------------------------- #
def _section_pwv4(low: bytes, mid: bytes, high: bytes) -> bytes:
"""PWV4: WaveColorPreviewTag — colour overview, 1200 cols × 6 bytes/col:
byte 0/1 : whiteness/brightness (= overall)
byte 2 : overall (full-spectrum) energy
byte 3 : bass (low band)
byte 4 : mid (mid band)
byte 5 : treble (high band)
`low/mid/high` arrive SHARED-normalised (0-255); `overall = max(bands)`."""
fourcc = b"PWV4"
len_header = 24
n = min(len(low), len(mid), len(high))
lo, md, hi = bytearray(low[:n]), bytearray(mid[:n]), bytearray(high[:n])
body = bytearray(n * 6)
for i in range(n):
overall = max(lo[i], md[i], hi[i])
body[i * 6 + 0] = overall
body[i * 6 + 1] = overall
body[i * 6 + 2] = overall
body[i * 6 + 3] = lo[i] # bass
body[i * 6 + 4] = md[i] # mid
body[i * 6 + 5] = hi[i] # treble
len_tag = len_header + len(body)
return fourcc + pack(">IIIII", len_header, len_tag, 6, n, 0x00000000) + bytes(body)
def _section_pwv5(scroll: bytes, low: bytes, mid: bytes, high: bytes) -> bytes:
"""PWV5: WaveColorScrollTag — colour scroll, u16 BE per col:
fedcba9876543210
RRRGGGBBBHHHHH--
Colour **R=treble, G=mid, B=bass** (per spec). Bands arrive per-band
normalised 0-255 → 3-bit via >>5. Height is the broadband value from the
mono `scroll` byte's low 5 bits, so colour and mono scrolls share a height.
"""
fourcc = b"PWV5"
len_header = 24
n = min(len(scroll), len(low), len(mid), len(high))
body = bytearray(n * 2)
for i in range(n):
r = high[i] >> 5 # red = treble
g = mid[i] >> 5 # green = mid
b = low[i] >> 5 # blue = bass
h = scroll[i] & 0x1F # broadband height (0-31), shared with PWV3
word = (r & 0x7) << 13 | (g & 0x7) << 10 | (b & 0x7) << 7 | (h & 0x1F) << 2
body[i * 2] = (word >> 8) & 0xFF
body[i * 2 + 1] = word & 0xFF
len_tag = len_header + len(body)
return fourcc + pack(">IIIII", len_header, len_tag, 2, n, 0x00960305) + bytes(body)
def _section_pwv6(low: bytes, mid: bytes, high: bytes) -> bytes:
"""PWV6: Wave3bandPreviewTag (CDJ-3000). 3 bytes/col, order **mid,
high, low** per the docs (NOT low-mid-high). Fixed 1200 cols."""
fourcc = b"PWV6"
len_header = 20
n = min(len(low), len(mid), len(high))
body = bytearray(n * 3)
for i in range(n):
body[i * 3 + 0] = mid[i]
body[i * 3 + 1] = high[i]
body[i * 3 + 2] = low[i]
len_entry_bytes = 3
len_entries = n
len_tag = len_header + len(body)
return fourcc + pack(">IIII", len_header, len_tag, len_entry_bytes, len_entries) + bytes(body)
def _section_pwv7(low: bytes, mid: bytes, high: bytes) -> bytes:
"""PWV7: Wave3bandScrollTag (CDJ-3000). Same 3-byte mid/high/low
encoding as PWV6 but variable length at ~150 cols/sec."""
fourcc = b"PWV7"
len_header = 24
n = min(len(low), len(mid), len(high))
body = bytearray(n * 3)
for i in range(n):
body[i * 3 + 0] = mid[i]
body[i * 3 + 1] = high[i]
body[i * 3 + 2] = low[i]
len_entry_bytes = 3
len_entries = n
len_tag = len_header + len(body)
return fourcc + pack(">IIIII", len_header, len_tag, len_entry_bytes, len_entries, 0x00960000) + bytes(body)
def _section_pqt2(beats: list[Beat]) -> bytes:
"""PQT2: extended beat grid (CDJ-3000+). Layout is only partially
documented; the format isn't in the kaitai schema or the deepsymmetry
spec page. The hex dump of a real Q1 PQT2 shows a 56-byte header
carrying num_beats, first_beat_ms, mid_ms, end_ms (matching the
corresponding PQTZ values), then per-beat 2-byte entries.
We write a placeholder: header fields mirror PQTZ where we know
them, the rest zero-filled. Per-beat entries are 2 bytes of zero
each (encoding unknown). Empirically the CDJ-2000NXS2+ uses the
presence of PQT2 — independently of its content — as a gate for the
deck-side waveform / BPM display.
"""
fourcc = b"PQT2"
len_header = 56
num_beats = len(beats)
first_beat_ms = beats[0][2] if beats else 0
if num_beats >= 2:
interval = beats[-1][2] - beats[-2][2]
end_ms = beats[-1][2] + interval
else:
end_ms = first_beat_ms
mid_ms = end_ms // 2
# 44 bytes of body header (11 × u4). First few fields known from the
# real Q1 PQT2 dump; later u4s look like an analyser-timestamp that
# we can't reproduce, so we zero them.
body_header = pack(
">11I",
0, # unknown1
0x01000002, # unknown2 — observed constant in real exports
0, # unknown3
0x00013200, # unknown4 — observed constant
first_beat_ms,
mid_ms,
end_ms,
num_beats,
0, # unknown9 (date/timestamp; not reproduced)
0, # unknown10
0, # unknown11
)
entries = b"\x00\x00" * num_beats
body = body_header + entries
len_tag = 12 + len(body)
return fourcc + pack(">II", len_header, len_tag) + body
def _section_pwvc() -> bytes:
"""PWVC: 20-byte tag at the tail of `.2EX`. Undocumented. Real exports
always include it; we emit a zero-filled placeholder of the expected
length so the file ends with the same section list."""
fourcc = b"PWVC"
len_header = 14
body = b"\x00" * 8
len_tag = 12 + len(body)
return fourcc + pack(">II", len_header, len_tag) + body
# XOR mask used by rekordbox to obfuscate the PSSI body. Mask is offset
# per byte by the (unmasked) `len_entries` value before XOR'ing in.
_PSSI_MASK = bytes.fromhex("CBE1EEFAE5EEADEEE9D2E9EBE1E9F3E8E9F4E1")
def _section_pssi(num_beats: int) -> bytes:
"""PSSI: SongStructureTag — phrase analysis. We emit an empty (0
phrases) placeholder so the CDJ-3000 sees a valid PSSI and doesn't
blank its phrase indicators. Mood=2 (mid), bank=0 (default).
Per spec, bytes 0x12 onwards are XOR-masked with `_PSSI_MASK[i] +
len_entries` (mod 256). Since `len_entries == 0`, the mask itself is
the XOR key.
"""
fourcc = b"PSSI"
len_header = 32
# Unmasked body fields:
len_entry_bytes = 0x18
len_entries = 0 # no phrases
mood = 2 # mid
unknown_u4 = 0
end_beat = num_beats # final phrase end beat = last beat in PQTZ
unknown2_u4 = 0
bank = 0 # default
unknown3 = 0
unmasked = pack(">I", len_entry_bytes) # offset 0x0c-0x0f (NOT masked per spec)
masked_block = (
pack(">H", len_entries) # 0x10-0x11
+ pack(">H", mood) # 0x12-0x13
+ pack(">I", unknown_u4) # 0x14-0x17
+ pack(">H", end_beat) # 0x18-0x19
+ pack(">I", unknown2_u4) # 0x1a-0x1d
+ pack(">B", bank) # 0x1e
+ pack(">B", unknown3) # 0x1f
)
# Apply mask: spec says "all bytes from offset 0x12 onwards" — that's
# the 2-byte mood field onward. Skip the first 2 bytes of masked_block
# (which are len_entries at 0x10).
masked_out = bytearray(masked_block[:2]) # len_entries unmasked
for i, b in enumerate(masked_block[2:]):
masked_out.append(b ^ ((_PSSI_MASK[i % len(_PSSI_MASK)] + len_entries) & 0xFF))
body = unmasked + bytes(masked_out)
# No entries to append (len_entries=0).
len_tag = 12 + len(body)
return fourcc + pack(">II", len_header, len_tag) + body
def _pmai_header(file_len: int) -> bytes:
"""28-byte PMAI header — fourcc + sizes + the canonical trailer that
rekordbox writes (was zero-padded in maple's earlier output)."""
return b"PMAI" + pack(">II", ANLZ_HEADER_LEN, file_len) + PMAI_HEADER_TRAILER
def build_dat(rb_path: str, analysis: Analysis | None) -> bytes:
"""Build ANLZ0000.DAT — the file every CDJ reads on track load.
Section order matches rekordbox's own output: PPTH, PVBR, PQTZ, PWAV,
PWV2, PCOB(memory), PCOB(hot).
"""
sections = [_section_pth(rb_path), _section_pvbr()]
if analysis is not None:
if analysis.beats:
sections.append(_section_pqtz(analysis.beats))
if analysis.mono.preview:
sections.append(_section_pwav(analysis.mono.preview))
if analysis.mono.tiny:
sections.append(_section_pwv2(analysis.mono.tiny))
sections.append(_section_pcob(1)) # hot cues (empty) — real writes hot-cues first
sections.append(_section_pcob(0)) # memory cues (empty)
body = b"".join(sections)
return _pmai_header(ANLZ_HEADER_LEN + len(body)) + body
def build_ext(rb_path: str, analysis: Analysis | None = None) -> bytes:
"""Build ANLZ0000.EXT — extended analysis data the loaded-deck reads.
Section order matches the real Q1 reference:
PPTH, PWV3 (mono scroll), PCOB×2, PCO2×2,
[PQT2 — skipped, format undocumented],
PWV5 (colour scroll), PWV4 (colour preview), PSSI (empty).
"""
sections = [_section_pth(rb_path)]
if analysis is not None and analysis.mono.scroll:
sections.append(_section_pwv3(analysis.mono.scroll))
sections.append(_section_pcob(1))
sections.append(_section_pcob(0))
sections.append(_section_pco2(1))
sections.append(_section_pco2(0))
if analysis is not None and analysis.beats:
# PQT2 between PCO2 and PWV5 — matches real-export section order
# and (empirically) gates the CDJ's deck-side waveform/BPM display.
sections.append(_section_pqt2(analysis.beats))
if analysis is not None:
scroll = analysis.mono.scroll
s_lo, s_md, s_hi = analysis.bands.scroll
if scroll and s_lo and s_md and s_hi:
sections.append(_section_pwv5(scroll, s_lo, s_md, s_hi))
p_lo, p_md, p_hi = analysis.bands.preview
if p_lo and p_md and p_hi:
sections.append(_section_pwv4(p_lo, p_md, p_hi))
if analysis.beats:
sections.append(_section_pssi(len(analysis.beats)))
body = b"".join(sections)
return _pmai_header(ANLZ_HEADER_LEN + len(body)) + body
def build_2ex(rb_path: str, analysis: Analysis | None = None) -> bytes:
"""Build ANLZ0000.2EX — CDJ-3000 3-band waveforms.
Section order matches the real Q1 reference:
PPTH, PWV7 (3-band scroll), PWV6 (3-band preview), PWVC (skipped).
"""
sections = [_section_pth(rb_path)]
if analysis is not None:
s_lo, s_md, s_hi = analysis.bands.scroll
if s_lo and s_md and s_hi:
sections.append(_section_pwv7(s_lo, s_md, s_hi))
p_lo, p_md, p_hi = analysis.bands.preview
if p_lo and p_md and p_hi:
sections.append(_section_pwv6(p_lo, p_md, p_hi))
sections.append(_section_pwvc()) # 20-byte trailer real exports always include
body = b"".join(sections)
return _pmai_header(ANLZ_HEADER_LEN + len(body)) + body
def build_anlz(rb_path: str, source_audio: Path | None = None) -> bytes:
"""Compatibility wrapper for callers that want the .DAT bytes only.
Analysis failures propagate — a broken analysis is fatal, not swallowed."""
analysis = Analysis.analyze(source_audio) if source_audio is not None else None
return build_dat(rb_path, analysis)
def build_minimal_anlz(rb_path: str) -> bytes:
"""PPTH-only-plus-empties-DAT. Used when no source audio is available."""
return build_dat(rb_path, analysis=None)
def _isrc_for(path: Path) -> str | None:
"""Best-effort ISRC read from a file tag (mp3/flac/m4a)."""
try:
from mutagen import File as MutagenFile
audio = MutagenFile(path, easy=True)
if audio is None or audio.tags is None:
return None
v = audio.tags.get("isrc")
if not v:
return None
return str(v[0]).strip() or None
except Exception:
return None
def write_anlz(
rb_path: str,
dest: Path,
source_audio: Path | None = None,
analysis_cache_dir: Path | None = None,
) -> Analysis | None:
"""Write ANLZ0000.DAT (and a minimal ANLZ0000.EXT alongside).
`dest` is the .DAT path. The .EXT companion is written next to it
automatically — every real CDJ export ships both files.
Returns the `Analysis` (cache hit or fresh mixxx run) or None when
no source audio was given.
"""
dest.parent.mkdir(parents=True, exist_ok=True)
analysis: Analysis | None = None
cache_hit = False
if source_audio is not None:
isrc = _isrc_for(source_audio) if analysis_cache_dir else None
if isrc and analysis_cache_dir:
cached = Analysis.load(analysis_cache_dir, isrc)
if cached is not None:
analysis = cached
cache_hit = True
logger.info("anlz: cache hit for {} (isrc {})", source_audio.name, isrc)
if analysis is None:
# Not a cache hit — analyse fresh. Failures propagate (a broken
# analysis is fatal, not swallowed into a grid-less ANLZ).
analysis = Analysis.analyze(source_audio, isrc=isrc or "")
# Populate the cache on every successful fresh analysis so subsequent
# exports (and collection runs) can skip the ffmpeg+mixxx pass. The
# cache is the single source of truth — populated by whichever
# pipeline (`just collection` or `just export`) sees a track first.
if analysis is not None and not cache_hit and analysis_cache_dir is not None and isrc:
try:
cp = analysis.save(analysis_cache_dir)
if cp is not None:
logger.info("anlz: cached analysis for {} → {}", source_audio.name, cp)
except Exception as e:
logger.warning("could not write analysis cache for {}: {}", source_audio.name, e)
dest.write_bytes(build_dat(rb_path, analysis))
ext_path = dest.with_suffix(".EXT")
ext_path.write_bytes(build_ext(rb_path, analysis))
# .2EX carries the CDJ-3000's 3-band waveforms (PWV6/PWV7). Older
# firmware ignores the file; CDJ-3000 reads it for the three-band
# strip display.
two_ex_path = dest.with_suffix(".2EX")
two_ex_path.write_bytes(build_2ex(rb_path, analysis))
return analysis
+70
View File
@@ -0,0 +1,70 @@
"""Extract embedded album art and write Pioneer-format JPEGs.
Pioneer's CDJ/XDJ players expect two square JPEGs per artwork:
/PIONEER/Artwork/{bucket:05d}/a{id}.jpg -> 80x80
/PIONEER/Artwork/{bucket:05d}/a{id}_m.jpg -> 240x240
where bucket = id // 20 + 1. The pdb's artwork_row.path points at the
small (80x80) variant; the player auto-resolves the `_m` companion.
"""
import hashlib
import io
from pathlib import Path
from mutagen.id3 import ID3, ID3NoHeaderError
from PIL import Image
SMALL = 80
LARGE = 240
def _read_apic(mp3: Path) -> bytes | None:
try:
tags = ID3(mp3)
except ID3NoHeaderError:
return None
frames = tags.getall("APIC")
if not frames:
return None
return frames[0].data
def _bucket(artwork_id: int) -> str:
return f"{artwork_id // 20 + 1:05d}"
def _resize_jpeg(data: bytes, size: int) -> bytes:
img = Image.open(io.BytesIO(data))
if img.mode != "RGB":
img = img.convert("RGB")
img = img.resize((size, size), Image.LANCZOS)
out = io.BytesIO()
img.save(out, format="JPEG", quality=85)
return out.getvalue()
def extract_artwork(mp3: Path, usb_root: Path, next_id: int, cache: dict[str, int]) -> tuple[int, str | None]:
"""Return (artwork_id, file_path_for_pdb). artwork_id is 0 if no art."""
data = _read_apic(mp3)
if data is None:
return 0, None
sha = hashlib.sha256(data).hexdigest()
cached = cache.get(sha)
if cached is not None:
return cached, None
aid = next_id
bucket = _bucket(aid)
rel_small = f"/PIONEER/Artwork/{bucket}/a{aid}.jpg"
rel_large = f"/PIONEER/Artwork/{bucket}/a{aid}_m.jpg"
abs_small = usb_root / rel_small.lstrip("/")
abs_large = usb_root / rel_large.lstrip("/")
abs_small.parent.mkdir(parents=True, exist_ok=True)
# Always overwrite — the artwork_id↔file mapping is recomputed every
# export, and a stale a<id>.jpg left over from a previous run would
# silently render the wrong cover on the CDJ. The intra-run dedup
# (cache hit at the top of this function) already prevents wasteful
# rewrites within one export.
abs_small.write_bytes(_resize_jpeg(data, SMALL))
abs_large.write_bytes(_resize_jpeg(data, LARGE))
cache[sha] = aid
return aid, rel_small
+528
View File
@@ -0,0 +1,528 @@
"""maple CLI: scan a directory of audio + m3u8 playlists and build a rekordbox USB."""
import argparse
import os
import re
import sys
import time
from collections import defaultdict
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import date
from pathlib import Path
from loguru import logger
from maple.analyze import Analysis, cache_path_for
from maple.anlz import _isrc_for, write_anlz
from maple.artwork import _bucket as _artwork_bucket
from maple.artwork import extract_artwork
from maple.convert import convert_flac_to_mp3, verify_duration
from maple.copy import stage_audio, staged_path
from maple.log import configure as configure_logging
from maple.meta import read_tags
from maple.model import Album, Artist, Artwork, Database, Genre, PlaylistEntry, PlaylistNode, Track
from maple.enrich import enrich_from_manifest, load_spotify_manifest, write_bpm_tag
from maple.pdb.writer import write_pdb
from maple.scan import scan_audio, scan_playlists
from maple.validate import TrackIssue, log_issues, validate_track
# Genre tag delimiters seen across our sources: Spotify-derived uses
# ';', European labels often use ',', some store '/'-separated lists.
_GENRE_SPLIT_RE = re.compile(r"\s*[;,/]\s*")
def usbanlz_location(usb_path: str) -> tuple[int, int]:
"""rekordbox's ANLZ folder derivation from a USB-relative track path.
Returns ``(bucket, anlz_id)`` for ``/PIONEER/USBANLZ/P{bucket:03X}/{anlz_id:08X}/``.
`usb_path` is the forward-slashed, leading-slash USB-relative path that the
pdb stores as `file_path` (e.g. ``/Contents/Artist/Album/01 Title.mp3``).
CRITICAL: the CDJ/XDJ firmware recomputes this from `file_path` and reads a
track's analysis from the resulting folder, *ignoring* the pdb's
`analyze_path` string. So maple must write its ANLZ files here and nowhere
else, or the player shows no waveform/beatgrid. Verified 1002/1002 against a
genuine rekordbox export and confirmed on XDJ-RX2 hardware.
`anlz_id` is a polynomial hash of the path's UTF-16 code units mod 0x30D43
(200003 — hence ids never exceed 0x30D42); `bucket` is a fixed bit-extraction
from that id. (Algorithm: github.com/Radexito/DjManager protocol.md §2.)
"""
h = 0
for ch in usb_path:
h = (h * 0x34F5501D + ord(ch) * 0x93B6) & 0xFFFFFFFF
anlz_id = h % 0x30D43
bucket = (
((((((((((anlz_id >> 2) & 0x4000) | (anlz_id & 0x2000)) >> 3)
| (anlz_id & 0x200)) >> 1) | (anlz_id & 0xC0)) >> 3)
| (anlz_id & 0x4)) >> 1) | (anlz_id & 0x1)
)
return bucket, anlz_id
def split_genre_tags(raw: str) -> list[str]:
"""Tokenise a possibly multi-value genre string into normalised parts.
Each token's casing is preserved as-given. Empty parts are dropped.
Duplicates within the same source string are collapsed (first wins).
"""
if not raw:
return []
seen: set[str] = set()
out: list[str] = []
for token in _GENRE_SPLIT_RE.split(raw):
t = token.strip()
if not t:
continue
key = t.casefold()
if key in seen:
continue
seen.add(key)
out.append(t)
return out
def build(input_dir: Path, usb_root: Path, with_playlists: bool = True) -> tuple[Database, set[Path]]:
"""Build the rekordbox USB layout under `usb_root`.
Returns the populated `Database` plus the set of absolute paths that
were written (audio files, ANLZ files, artwork JPEGs). The caller adds
the pdb path and calls `prune_stale` to remove anything else under
`usb_root` left over from a previous export.
"""
db = Database()
today = date.today().isoformat()
artist_index: dict[str, int] = {}
album_index: dict[tuple[int, str], int] = {}
# Key by casefolded name so "Dub Techno" and "dub techno" collapse to one row.
genre_index: dict[str, int] = {}
# The user-facing genre→tracks map. Populated as tracks are processed.
genre_to_tracks: dict[str, list[int]] = defaultdict(list)
track_by_src: dict[Path, int] = {}
artwork_cache: dict[str, int] = {}
# USBANLZ folder occupancy. Two distinct paths can hash to the same
# P{bucket}/{id} folder (a hash collision; also happens for genuine
# duplicate paths). rekordbox keeps both by bumping the filename to
# ANLZ0001.DAT, ANLZ0002.DAT, … inside the shared folder (verified: a
# real export had `Celestica` + `Soak It Up` co-located at P049/00030991).
# Mirror that so colliding tracks don't clobber each other's analysis.
anlz_folder_seen: dict[str, int] = defaultdict(int)
written: set[Path] = set()
audio_files = scan_audio(input_dir)
logger.info("scanned {}: {} audio file(s)", input_dir, len(audio_files))
# ---------- Spotify enrichment (symmetric with `just collection`) -------- #
# Per the validation convention: both pipelines must be capable of
# bringing a track to "valid" state. Validate first; only files that
# actually need it pay the enrichment cost (mutagen tag rewrite +
# possible artwork fetch). Repeated exports of an already-valid
# collection skip the enrichment loop entirely.
by_key, by_isrc = load_spotify_manifest(input_dir / "_metadata")
cache_dir = input_dir / "_analysis"
if by_key or by_isrc:
logger.info(
"loaded spotify manifest: {} (artist,title) keys, {} ISRCs",
len(by_key), len(by_isrc),
)
enriched = 0
skipped = 0
for p in audio_files:
if not validate_track(p, analysis_cache_dir=cache_dir):
skipped += 1
continue
if enrich_from_manifest(p, by_key, by_isrc, analysis_cache_dir=cache_dir) is not None:
enriched += 1
logger.info(
"enrichment: {} enriched, {} already valid (skipped)",
enriched, skipped,
)
else:
logger.warning(
"no spotify manifest at {} — tracks won't be Spotify-enriched. "
"Run `just collection` first to fetch + persist the manifest.",
input_dir / "_metadata" / "spotify-tracks.json",
)
t = time.perf_counter()
initial_meta = {p: read_tags(p) for p in audio_files}
logger.debug("read tags in {:.2f}s", time.perf_counter() - t)
flac_jobs: list[tuple[Path, Path]] = []
for original in audio_files:
if original.suffix.lower() == ".flac":
m = initial_meta[original]
dest_abs, _ = staged_path(usb_root, m.artist, m.album, original.with_suffix(".mp3").name)
flac_jobs.append((original, dest_abs))
def _job(pair: tuple[Path, Path]) -> tuple[Path, Path, bool]:
src, dst = pair
t0 = time.perf_counter()
converted = convert_flac_to_mp3(src, dst)
return src, dst, converted, time.perf_counter() - t0
workers = max(1, os.cpu_count() or 1)
if flac_jobs:
logger.info("converting {} flac(s) → mp3 v0 (workers={})", len(flac_jobs), workers)
t = time.perf_counter()
n_converted = 0
with ThreadPoolExecutor(max_workers=workers) as ex:
for i, (src, dst, converted, dt) in enumerate(ex.map(_job, flac_jobs), 1):
tag = "convert" if converted else "cached "
logger.info("[{}/{}] {} {} ({:.1f}s)", i, len(flac_jobs), tag, src.relative_to(input_dir), dt)
if converted:
n_converted += 1
verify_duration(src, dst)
logger.info(
"flac→mp3 pass: {} converted, {} cached in {:.1f}s total",
n_converted, len(flac_jobs) - n_converted, time.perf_counter() - t,
)
# Parallel pre-analysis. The per-track build loop below is sequential
# (the pdb / artwork-id assignments depend on iteration order), but
# `Analysis.analyze` is the slow step and trivially parallel: ffmpeg +
# mixxx + scipy each release the GIL, so threads scale linearly with
# cores. After this pass every track has its ISRC-keyed analysis
# cached, so write_anlz in the build loop is a cache hit.
def _dest_for(p: Path) -> Path:
m = initial_meta[p]
if p.suffix.lower() == ".flac":
dest_abs, _ = staged_path(usb_root, m.artist, m.album, p.with_suffix(".mp3").name)
else:
dest_abs, _ = staged_path(usb_root, m.artist, m.album, p.name)
return dest_abs
# Make sure non-FLAC sources are staged to data/usb first so the
# analysis pass operates on the file the CDJ will actually read.
for p in audio_files:
if p.suffix.lower() != ".flac":
m = initial_meta[p]
stage_audio(p, usb_root, m.artist, m.album)
def _ensure_analysis_cache(p: Path) -> bool:
"""Populate `<cache>/<isrc>.json` if missing. Returns True iff a
fresh analysis actually ran (i.e. cache was cold). Analysis failures
propagate — a broken analysis is fatal, not skipped."""
dest = _dest_for(p)
if not dest.exists():
return False
isrc = _isrc_for(dest)
if not isrc:
return False
cp = cache_path_for(cache_dir, isrc)
if cp is None or cp.exists():
return False
a = Analysis.analyze(dest, isrc=isrc)
a.save(cache_dir)
return True
pre_t0 = time.perf_counter()
logger.info("pre-analysing {} tracks (workers={})...", len(audio_files), workers)
new_count = 0
done = 0
with ThreadPoolExecutor(max_workers=workers) as ex:
futures = {ex.submit(_ensure_analysis_cache, p): p for p in audio_files}
for fut in as_completed(futures):
done += 1
if fut.result(): # re-raises any worker analysis failure
new_count += 1
if done % 8 == 0 or done == len(audio_files):
logger.info(" [{}/{}] cache misses analysed: {}", done, len(audio_files), new_count)
logger.info(
"pre-analysis: {} new in {:.1f}s ({} already cached)",
new_count, time.perf_counter() - pre_t0, len(audio_files) - new_count,
)
logger.info("building track table ({} files)", len(audio_files))
build_t0 = time.perf_counter()
validation_issues: list[TrackIssue] = []
for track_idx, original in enumerate(audio_files, 1):
track_t0 = time.perf_counter()
meta = initial_meta[original]
artist_id = artist_index.get(meta.artist)
if artist_id is None:
artist_id = len(db.artists) + 1
artist_index[meta.artist] = artist_id
db.artists.append(Artist(id=artist_id, name=meta.artist))
album_key = (artist_id, meta.album)
album_id = album_index.get(album_key)
if album_id is None:
album_id = len(db.albums) + 1
album_index[album_key] = album_id
db.albums.append(Album(id=album_id, name=meta.album, artist_id=artist_id))
if original.suffix.lower() == ".flac":
dest_abs, file_path = staged_path(usb_root, meta.artist, meta.album, original.with_suffix(".mp3").name)
meta = read_tags(dest_abs)
else:
dest_abs, file_path = stage_audio(original, usb_root, meta.artist, meta.album)
written.add(dest_abs.resolve())
# Re-enrich the STAGED file too — `stage_audio` and the FLAC→MP3
# converter cache by hash, so a cached dest may still carry stale
# tags from a previous export. Skip when the staged file already
# passes validation so re-exports of an unchanged collection don't
# re-mutate every track.
if (by_key or by_isrc) and validate_track(dest_abs, analysis_cache_dir=cache_dir):
if enrich_from_manifest(dest_abs, by_key, by_isrc, analysis_cache_dir=cache_dir) is not None:
meta = read_tags(dest_abs)
next_artwork_id = len(db.artwork) + 1
artwork_id, art_file_path = extract_artwork(dest_abs, usb_root, next_artwork_id, artwork_cache)
if art_file_path is not None:
db.artwork.append(Artwork(id=artwork_id, path=art_file_path))
# Track artwork paths for the prune sweep — both this run's writes
# and any artwork_cache hits (where extract_artwork returned None
# for art_file_path because the jpeg already exists from a prior
# iteration). _bucket() is the same id-to-folder mapping artwork.py
# uses, so this rebuilds the absolute on-disk path exactly.
if artwork_id:
art_bucket = _artwork_bucket(artwork_id)
for fname in (f"a{artwork_id}.jpg", f"a{artwork_id}_m.jpg"):
written.add((usb_root / "PIONEER" / "Artwork" / art_bucket / fname).resolve())
# Genre: split multi-value tag strings ("dub techno;minimal techno",
# "Drum & Bass, Électronique", etc.) into individual genre rows so the
# CDJ's Genre filter is usable. The track FK points at the FIRST
# (most-specific) token; every other token still gets its own row in
# the pdb's genre table so it appears in the browse list, and we
# record the {genre→track_ids} map so we can log + reuse it later.
primary_genre_id = 0
tokens = split_genre_tags(meta.genre)
for n, token in enumerate(tokens):
key = token.casefold()
gid = genre_index.get(key)
if gid is None:
gid = len(db.genres) + 1
genre_index[key] = gid
db.genres.append(Genre(id=gid, name=token))
if n == 0:
primary_genre_id = gid
genre_to_tracks[token].append(len(db.tracks) + 1) # this track's id-to-be
genre_id = primary_genre_id
track_id = len(db.tracks) + 1
# The USBANLZ folder is derived from the USB-relative `file_path` by
# rekordbox's bespoke path hash (see `usbanlz_location`). The CDJ/XDJ
# firmware RE-DERIVES this same path from `file_path` and reads the
# analysis from there — it IGNORES the pdb's `analyze_path` string —
# so the folder MUST match rekordbox exactly or the player silently
# shows no waveform/beatgrid. Verified 1002/1002 vs a real export and
# on XDJ-RX2 hardware. (Earlier crc32 scheme matched 0/984 → grids
# missing.)
bucket_n, anlz_id = usbanlz_location(file_path)
anlz_folder = f"/PIONEER/USBANLZ/P{bucket_n:03X}/{anlz_id:08X}"
anlz_index = anlz_folder_seen[anlz_folder] # 0, then 1, 2, … on collision
anlz_folder_seen[anlz_folder] += 1
analyze_path = f"{anlz_folder}/ANLZ{anlz_index:04d}.DAT"
anlz_abs = (usb_root / analyze_path.lstrip("/")).resolve()
# `_analysis/` sits at the input collection root; ISRC-keyed cache
# populated during `just collection`. write_anlz reads it before
# falling back to a fresh ffmpeg+mixxx pass.
analysis = write_anlz(
file_path, anlz_abs, source_audio=dest_abs,
analysis_cache_dir=input_dir / "_analysis",
)
written.add(anlz_abs)
# .EXT and .2EX sister files are written automatically next to .DAT.
written.add(anlz_abs.with_suffix(".EXT"))
written.add(anlz_abs.with_suffix(".2EX"))
# Prefer the analytically detected BPM over the file tag when we have one.
# CDJ browse panel reads pdb.tempo; loaded-deck readout reads the PQTZ
# we just wrote — keeping the two in sync avoids the "120.0 in browse,
# blank on deck" UX bug.
if analysis is not None and analysis.bpm:
tempo_x100 = analysis.grid.tempo_x100
# Also write the bpm TAG to the file (symmetric with `just
# collection`'s _write_bpm_tag). overwrite=False so any
# pre-existing label-set BPM is preserved.
write_bpm_tag(dest_abs, analysis.bpm, overwrite=False)
else:
tempo_x100 = meta.tempo
db.tracks.append(Track(
id=track_id,
title=meta.title,
artist_id=artist_id,
album_id=album_id,
filename=Path(file_path).name,
file_path=file_path,
file_size=meta.file_size,
sample_rate=meta.sample_rate,
sample_depth=meta.sample_depth,
bitrate=meta.bitrate,
duration=meta.duration,
track_number=meta.track_number,
year=meta.year,
tempo=tempo_x100,
date_added=today,
analyze_path=analyze_path,
analyze_date=today,
artwork_id=artwork_id,
genre_id=genre_id,
source_path=original,
))
track_by_src[original.resolve()] = track_id
# Cross-cutting validation: same checks as `just collection`. The
# input audio_path is the staged file under data/usb/ so its tags
# are what end up on the player. analysis cache lives at the source
# collection root (input_dir/_analysis/<isrc>.json).
track_issues = validate_track(
dest_abs,
analysis_cache_dir=input_dir / "_analysis",
)
validation_issues.extend(track_issues)
track_dt = time.perf_counter() - track_t0
bpm_str = f"{analysis.bpm:.1f} bpm" if (analysis is not None and analysis.bpm) else f"tag {tempo_x100/100:.1f} bpm"
issue_tag = f" [{len(track_issues)} issue(s)]" if track_issues else ""
logger.info(
"[{}/{}] {} - {} ({}, {:.1f}s){}",
track_idx, len(audio_files),
meta.artist, meta.title,
bpm_str, track_dt, issue_tag,
)
logger.info(
"track table built in {:.1f}s ({} tracks, {} artists, {} albums, "
"{} artwork, {} genres)",
time.perf_counter() - build_t0,
len(db.tracks), len(db.artists), len(db.albums),
len(db.artwork), len(db.genres),
)
# Genre map summary: largest buckets first; helps spot mistagged tracks.
if genre_to_tracks:
logger.info("genre→tracks map ({} genres):", len(genre_to_tracks))
for genre, track_ids in sorted(
genre_to_tracks.items(), key=lambda kv: (-len(kv[1]), kv[0])
):
logger.info(" {:<32s} {:>3} track(s)", genre[:32], len(track_ids))
no_genre = sum(1 for t in db.tracks if t.genre_id == 0)
if no_genre:
logger.warning("{} of {} tracks have no genre tag", no_genre, len(db.tracks))
# Emit the validation report. Same TrackIssue shape as `just collection`,
# surfaced through loguru so the user gets visibility plus a roll-up at
# the end.
if validation_issues:
logger.warning(
"{} validation issue(s) across {} track(s)",
len(validation_issues),
len({i.track.display_name for i in validation_issues}),
)
log_issues(validation_issues, report=False)
else:
logger.info("validation: all {} tracks passed", len(audio_files))
if not with_playlists:
return db, written, validation_issues
playlists = scan_playlists(input_dir)
if playlists:
logger.info("scanning {} playlist(s)", len(playlists))
for sort_order, (name, paths) in enumerate(playlists.items(), start=1):
playlist_id = len(db.playlist_nodes) + 1
db.playlist_nodes.append(PlaylistNode(
id=playlist_id,
parent_id=0,
sort_order=sort_order,
name=name,
is_folder=False,
))
resolved = 0
for entry_index, p in enumerate(paths, start=1):
tid = track_by_src.get(p.resolve())
if tid is None:
logger.warning("playlist {!r} references missing file {}", name, p)
continue
db.playlist_entries.append(PlaylistEntry(
playlist_id=playlist_id,
track_id=tid,
entry_index=entry_index,
))
resolved += 1
logger.info(" {}: {} entries ({} resolved, {} missing)",
name, len(paths), resolved, len(paths) - resolved)
return db, written, validation_issues
def prune_stale(usb_root: Path, kept: set[Path]) -> tuple[int, int]:
"""Delete every file under `usb_root` not in `kept`, then drop empty dirs.
Returns (files_removed, dirs_removed). Operates only under `usb_root`,
so anything outside is untouched. The `kept` set is expected to hold
fully-resolved absolute paths.
"""
files_removed = 0
for p in usb_root.rglob("*"):
if not p.is_file():
continue
if p.resolve() in kept:
continue
try:
p.unlink()
files_removed += 1
except OSError as e:
logger.warning("failed to remove {}: {}", p, e)
# Bottom-up empty-dir removal: deepest first so each parent gets a
# chance to become empty once its children are gone.
dirs = sorted(
(d for d in usb_root.rglob("*") if d.is_dir()),
key=lambda d: len(d.parts),
reverse=True,
)
dirs_removed = 0
for d in dirs:
try:
d.rmdir()
dirs_removed += 1
except OSError:
pass # not empty; expected for the directories we still want
return files_removed, dirs_removed
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(prog="maple", description="Build a rekordbox USB from a folder of audio + m3u8 playlists.")
parser.add_argument("input_dir", nargs="?", type=Path, default=Path("./data/input"))
parser.add_argument("output_root", nargs="?", type=Path, default=Path("./data/usb"))
parser.add_argument("--no-playlists", action="store_true", help="Skip m3u8 playlist scanning")
parser.add_argument("--no-prune", action="store_true", help="Skip the post-build sweep of stale files")
parser.add_argument("--strict", action="store_true", help="Fail with non-zero exit if any track fails validation")
args = parser.parse_args(argv)
configure_logging()
if not args.input_dir.is_dir():
parser.error(f"input_dir {args.input_dir} is not a directory")
overall_t0 = time.perf_counter()
logger.info("export: {} → {}", args.input_dir, args.output_root)
db, written, validation_issues = build(args.input_dir, args.output_root, with_playlists=not args.no_playlists)
pdb_path = args.output_root / "PIONEER" / "rekordbox" / "export.pdb"
write_pdb(db, pdb_path)
written.add(pdb_path.resolve())
if not args.no_prune:
t = time.perf_counter()
n_files, n_dirs = prune_stale(args.output_root, written)
if n_files or n_dirs:
logger.info(
"pruned {} stale file(s) + {} empty dir(s) in {:.1f}s",
n_files, n_dirs, time.perf_counter() - t,
)
else:
logger.debug("nothing to prune ({:.1f}s)", time.perf_counter() - t)
logger.info(
"done in {:.1f}s: {} tracks, {} artists, {} albums, "
"{} playlists ({} entries) → {}",
time.perf_counter() - overall_t0,
len(db.tracks), len(db.artists), len(db.albums),
len(db.playlist_nodes), len(db.playlist_entries), pdb_path,
)
if args.strict and validation_issues:
logger.error("--strict: refusing to succeed with {} validation issue(s)", len(validation_issues))
return 1
return 0
+38
View File
@@ -0,0 +1,38 @@
"""Sync data/collection with one or more Spotify playlists.
Pipeline (one shot, in `maple.collection.cli.main_async`):
1. Async-fetch each Spotify playlist via aiohttp (concurrently).
2. Diff against the Collection's index (sldl `index.sldl.csv` + tag scan).
3. Write the missing rows as a CSV and shell out to `sockseek` to download
them (streaming `--progress-json` events back so each completed track
updates its playlist's m3u8 immediately).
4. Validate every track that now resolves to a file: ffprobe-decodable,
duration within 5s of Spotify, fuzzy tag match, embedded cover art.
If artwork is missing, fetch the largest image from
`track.album.images` and embed it.
5. Write one m3u8 per playlist into `<root>/_playlists/<name>.m3u8` with
paths relative to the m3u8 file (portable across synced devices).
Usage:
uv run python -m maple.collection <id-or-url ...> [--collection PATH]
[--no-download] [--no-validate]
Env (from .env, loaded by just):
MAPLE_SPOTIFY_CLIENT_ID, MAPLE_SPOTIFY_CLIENT_SECRET (spotify.py reads)
SOCKSEEK_USER, SOCKSEEK_PASS (forwarded to sockseek)
"""
# Re-exports are lazy via __getattr__ so importing `maple.collection.index`
# (or any other submodule) doesn't drag the whole package — `maple.validate`
# depends on `maple.collection.index` and would otherwise circular-import
# through `maple.collection.core` → `maple.validate` → here.
__all__ = ["Collection", "CollectionIndex", "TrackIssue"]
def __getattr__(name):
if name == "Collection" or name == "TrackIssue":
from maple.collection.core import Collection, TrackIssue
return {"Collection": Collection, "TrackIssue": TrackIssue}[name]
if name == "CollectionIndex":
from maple.collection.index import CollectionIndex
return CollectionIndex
raise AttributeError(name)
@@ -0,0 +1,4 @@
"""Entrypoint for `python -m maple.collection`."""
from maple.collection.cli import main
raise SystemExit(main())
+196
View File
@@ -0,0 +1,196 @@
"""`just collection` CLI entrypoint.
End-to-end pipeline: fetch Spotify playlists, diff against the on-disk
collection, run sockseek for the missing tracks, validate the result,
write per-playlist m3u8s, and emit a report.
"""
from __future__ import annotations
import argparse
import asyncio
import sys
from pathlib import Path
import aiohttp
from loguru import logger
from maple.collection.core import Collection, TrackIssue
from maple.collection.index import first_artist, norm
from maple.enrich import save_spotify_manifest
from maple.log import configure as configure_logging
from maple.spotify import fetch_playlists
def parse_args(argv: list[str]) -> argparse.Namespace:
p = argparse.ArgumentParser(prog="maple.collection", description="Sync data/collection with Spotify playlists.")
p.add_argument("ids", nargs="+", help="Spotify playlist ids, URLs, or spotify: URIs")
p.add_argument("--collection", help="Override collection root (default: <repo>/data/collection)")
p.add_argument("--no-download", action="store_true", help="Skip the sockseek download step")
p.add_argument("--no-validate", action="store_true", help="Skip ffprobe/duration/tag/art validation")
return p.parse_args(argv)
async def main_async(argv: list[str]) -> int:
args = parse_args(argv)
configure_logging()
collection = Collection(Path(args.collection).resolve()) if args.collection else Collection.from_repo()
collection.ensure_dirs()
collection.attach_report_sink()
logger.info("collection root: {}", collection.root)
# Startup hygiene: drop unfinished downloads, prune the sldl index, and
# rewrite any stale entries out of existing m3u8s.
removed_incomplete = collection.remove_unfinished_downloads()
if removed_incomplete:
logger.warning("removed {} unfinished download(s)", removed_incomplete)
kept, removed = collection.validate_index_file()
if removed:
logger.warning("index: pruned {} row(s) ({} kept)", removed, kept)
pruned_pl, rewritten_pl = collection.validate_playlists()
if pruned_pl or rewritten_pl:
logger.warning(
"playlists: pruned {} stale entr(ies), rewrote {} absolute path(s)",
pruned_pl, rewritten_pl,
)
playlists = await fetch_playlists(args.ids)
total = sum(len(p["tracks"]) for p in playlists)
logger.info("fetched {} playlist(s), {} tracks", len(playlists), total)
# Persist the full set of Spotify tracks so `just export` (and any
# later `just collection` run) can enrich tags without re-fetching.
save_spotify_manifest(collection.metadata_dir, playlists)
collection.refresh_index()
logger.info(
"existing in collection: {} strict / {} loose keys",
len(collection.index.by_strict),
len(collection.index.by_loose),
)
missing = collection.find_missing(playlists)
logger.info("missing tracks to download: {}", len(missing))
# Issues accumulate from both the streaming callback (per-track on
# completion) and the end-of-run validate sweep. process_track dedups
# paths so neither path double-reports.
issues: list[TrackIssue] = []
if missing and not args.no_download:
# (artist_norm, title_norm) -> [(playlist_dict, spotify_track)] so the
# callback can both rewrite the right m3u8 and reconcile tags against
# the canonical Spotify entry (sockseek echoes back only artist/title).
track_to_playlists: dict[tuple[str, str], list[tuple[dict, dict]]] = {}
for pl in playlists:
for t in pl["tracks"]:
key = (
norm(first_artist(", ".join(t["artists"]))),
norm(t["title"]),
)
track_to_playlists.setdefault(key, []).append((pl, t))
def _human_size(n: int) -> str:
for unit in ("B", "KiB", "MiB", "GiB"):
if n < 1024:
return f"{n:.0f}{unit}" if unit == "B" else f"{n:.1f}{unit}"
n /= 1024
return f"{n:.1f}TiB"
async def on_event(evt: dict) -> None:
etype = evt.get("type")
data = evt.get("data") or {}
if etype == "search_start":
logger.info(
"search: {} - {}",
data.get("artist") or "?", data.get("title") or "?",
)
return
if etype == "download_start":
size = data.get("size") or 0
logger.info(
"downloading: {} - {} ({} {}, from {})",
data.get("artist") or "?",
data.get("title") or "?",
_human_size(size) if size else "?",
(data.get("extension") or "").lstrip(".") or "?",
data.get("username") or "?",
)
return
if etype != "track_state":
return
outcome = (data.get("terminalOutcome") or "").lower()
skip_reason = (data.get("skipReason") or "").lower()
failure_reason = (data.get("failureReason") or "").strip()
artist = data.get("artist") or ""
title = data.get("title") or ""
success = outcome == "succeeded" or (
outcome == "skipped" and skip_reason == "alreadyexists"
)
if not success:
# Surface failures/skips so the user sees why a track didn't land.
if outcome == "skipped":
logger.info("skipped: {} - {} ({})", artist, title, skip_reason or "skipped")
else:
logger.warning(
"failed: {} - {} ({})",
artist, title, failure_reason or outcome or "unknown",
)
return
download_path = data.get("downloadPath")
if not download_path:
return
path = Path(download_path)
if not path.exists():
logger.debug("track_state success but path missing: {}", download_path)
return
collection.index.add(artist, title, path)
key = (norm(first_artist(artist)), norm(title))
affected = track_to_playlists.get(key, [])
if not affected:
logger.info("downloaded {} - {} ({})", artist, title, path.name)
return
# Reconcile against the Spotify track (writes Spotify tags
# + records any mismatches). One spotify track might appear
# in several playlists; process_track dedups via collection._reconciled.
spotify_track = affected[0][1]
new_issues = collection.process_track(
affected[0][0]["name"] or affected[0][0]["id"],
spotify_track,
path,
)
for issue in new_issues:
logger.warning(
"{}: {}: {}", issue.track.display_name, issue.field, issue.reason,
)
issues.extend(new_issues)
for pl, _ in affected:
out = collection.write_m3u8_for(pl)
logger.info(
"downloaded {} - {} → updated {}",
artist, title, out.name,
)
rc = await collection.download(missing, on_event=on_event)
if rc != 0:
logger.warning("sockseek exited with {}", rc)
collection.refresh_index()
if not args.no_validate:
timeout = aiohttp.ClientTimeout(total=30)
async with aiohttp.ClientSession(timeout=timeout) as session:
issues.extend(await collection.validate(session, playlists))
for out in collection.write_m3u8s(playlists, issues):
logger.info("wrote {}", out)
collection.report_issues(issues)
logger.info("{} issue(s) recorded in {}", len(issues), collection.report_path)
return 0
def main() -> int:
return asyncio.run(main_async(sys.argv[1:]))
+608
View File
@@ -0,0 +1,608 @@
"""`Collection` dataclass: the on-disk music collection rooted at `root`.
Owns dir layout, the in-memory `CollectionIndex` of (artist, title) →
file paths, startup validation (drop unfinished downloads, prune the
sldl index, prune stale m3u8 entries), the missing-track diff, validation
of downloaded tracks, the per-playlist m3u8 writer, and the loguru file
sink used for `collection-report.txt`.
Sockseek subprocess plumbing lives in `maple.collection.sockseek`; this
module just calls into it from `Collection.download()`.
"""
from __future__ import annotations
import asyncio
import csv
import os
import re
from collections.abc import Awaitable, Callable
from dataclasses import dataclass, field
from pathlib import Path
import aiohttp
from loguru import logger
from mutagen import File as MutagenFile
from maple.analyze import Analysis
from maple.enrich import write_spotify_tags as _shared_write_spotify_tags
from maple.validate import TrackIssue as _ValidateTrackIssue
from maple.validate import validate_track as _validate_track
from maple.collection.index import (
CollectionIndex,
first_artist,
fuzzy_ratio,
norm,
norm_loose,
)
from maple.collection.sockseek import download as run_sockseek
from maple.collection.validate import (
ARTWORK_CONCURRENCY,
DURATION_TOLERANCE_S,
embed_art,
fetch_jpeg,
ffprobe_ok,
file_duration,
has_embedded_art,
)
from maple.scan import AUDIO_EXTS
BPM_DIVERGENCE_WARN = 0.05 # warn when aubio vs existing tag differs >5%
# Re-export the shared validator's TrackIssue so callers that imported it
# from maple.collection.core continue to work. There is one TrackIssue
# definition in the codebase — `maple.validate.TrackIssue` — and both the
# `just collection` and `just export` flows produce/consume the same shape.
TrackIssue = _ValidateTrackIssue
_M3U_BAD = re.compile(r"[\\/\x00]+")
def _sanitise_name(name: str) -> str:
s = _M3U_BAD.sub("_", name or "playlist").strip().lstrip(".")
return (s or "playlist")[:200]
@dataclass
class Collection:
"""A music collection rooted at `root` on disk."""
root: Path
index: CollectionIndex = field(default_factory=CollectionIndex)
# Paths whose tags have already been reconciled against Spotify this run.
# `process_track` consults this so the on_event hot path and the
# end-of-run `validate` sweep don't double-handle the same file.
_reconciled: set[Path] = field(default_factory=set, repr=False)
# --- paths ------------------------------------------------------------- #
@classmethod
def from_repo(cls) -> "Collection":
"""Default location: `<repo>/data/collection`."""
here = Path(__file__).resolve()
# src/maple/collection/core.py → src/maple/collection → src/maple → src → maple-py → repo
return cls((here.parents[4] / "data" / "collection").resolve())
@property
def metadata_dir(self) -> Path:
return self.root / "_metadata"
@property
def playlists_dir(self) -> Path:
return self.root / "_playlists"
@property
def index_csv(self) -> Path:
return self.metadata_dir / "index.sldl.csv"
@property
def report_path(self) -> Path:
return self.metadata_dir / "collection-report.txt"
@property
def analysis_dir(self) -> Path:
"""`<root>/_analysis/<isrc>.json` — cached aubio output keyed on ISRC,
consumed by `just export` to skip the decode+detect pass."""
return self.root / "_analysis"
def ensure_dirs(self) -> None:
self.root.mkdir(parents=True, exist_ok=True)
self.metadata_dir.mkdir(parents=True, exist_ok=True)
self.playlists_dir.mkdir(parents=True, exist_ok=True)
self.analysis_dir.mkdir(parents=True, exist_ok=True)
# --- startup validation ----------------------------------------------- #
def remove_unfinished_downloads(self) -> int:
"""Delete sldl's `*.incomplete` leftover files. Returns count removed."""
if not self.root.exists():
return 0
count = 0
for p in self.root.rglob("*.incomplete"):
if "_metadata" in p.parts or "_playlists" in p.parts:
continue
try:
p.unlink()
count += 1
except OSError as e:
logger.warning("failed to remove {}: {}", p, e)
return count
def validate_index_file(self) -> tuple[int, int]:
"""Rewrite `index.sldl.csv` dropping duplicate rows and orphaned-path rows.
Returns (kept, removed). Failure rows (no filepath) are kept so
sldl can still honour `--skip-not-found` on the next run.
"""
if not self.index_csv.exists():
return (0, 0)
with self.index_csv.open(newline="", encoding="utf-8", errors="replace") as fh:
rows = list(csv.reader(fh))
if len(rows) <= 1:
return (max(0, len(rows) - 1), 0)
header = rows[0]
cols = {h.strip().lower(): i for i, h in enumerate(header)}
path_i = cols.get("filepath", 0)
kept: list[list[str]] = []
seen: set[tuple[str, ...]] = set()
removed = 0
for row in rows[1:]:
key = tuple(row)
if key in seen:
removed += 1
continue
seen.add(key)
rel = row[path_i] if path_i < len(row) else ""
if rel:
p = self.root / rel if not Path(rel).is_absolute() else Path(rel)
if not p.exists():
removed += 1
continue
kept.append(row)
if removed > 0:
with self.index_csv.open("w", newline="", encoding="utf-8") as fh:
w = csv.writer(fh)
w.writerow(header)
w.writerows(kept)
return (len(kept), removed)
def validate_playlists(self) -> tuple[int, int]:
"""Sweep `_playlists/*.m3u8`. Two passes per entry:
1. Lines whose file doesn't resolve are replaced with a
`# NOT FOUND: <title>` comment so the playlist still tells you
what's missing.
2. Lines that resolve but were written with an absolute path get
rewritten relative to the m3u8's parent dir — playlists need to
survive being synced to another machine where the collection
sits at a different root.
Returns (pruned, rewritten)."""
if not self.playlists_dir.exists():
return (0, 0)
total_pruned = 0
total_rewritten = 0
for m3u in self.playlists_dir.glob("*.m3u8"):
lines = m3u.read_text(encoding="utf-8").splitlines()
out: list[str] = []
pending_extinf: str | None = None
pruned_here = 0
rewritten_here = 0
for line in lines:
s = line.strip()
if not s:
out.append(line)
continue
if s.startswith("#EXTINF"):
pending_extinf = line
continue
if s.startswith("#"):
if pending_extinf is not None:
out.append(pending_extinf)
pending_extinf = None
out.append(line)
continue
# Bare file path (relative to m3u or absolute).
was_absolute = Path(s).is_absolute()
target = Path(s) if was_absolute else (m3u.parent / s)
if target.exists():
if pending_extinf is not None:
out.append(pending_extinf)
pending_extinf = None
if was_absolute:
rel = os.path.relpath(target.resolve(), start=m3u.parent.resolve())
out.append(rel)
rewritten_here += 1
else:
out.append(line)
else:
desc = s
if pending_extinf and "," in pending_extinf:
desc = pending_extinf.split(",", 1)[1].strip() or s
out.append(f"# NOT FOUND: {desc}")
pending_extinf = None
pruned_here += 1
if pruned_here or rewritten_here:
m3u.write_text("\n".join(out) + "\n", encoding="utf-8")
if pruned_here:
logger.warning("pruned {} missing entries from {}", pruned_here, m3u.name)
if rewritten_here:
logger.warning("rewrote {} absolute path(s) → relative in {}", rewritten_here, m3u.name)
total_pruned += pruned_here
total_rewritten += rewritten_here
return (total_pruned, total_rewritten)
# --- report sink ------------------------------------------------------- #
def attach_report_sink(self) -> int:
"""Install a loguru sink that mirrors every reported issue to disk.
Filters on `extra["report"] is True` so unrelated log lines don't
leak into the report. Returns the loguru handle so callers can
detach it later.
"""
self.metadata_dir.mkdir(parents=True, exist_ok=True)
if self.report_path.exists():
self.report_path.unlink()
return logger.add(
self.report_path,
format="{message}",
filter=lambda r: r["extra"].get("report") is True,
level="INFO",
)
def report_issues(self, issues: list[TrackIssue]) -> None:
"""Emit each issue through the report sink; emit `ok` if none."""
bound = logger.bind(report=True)
if not issues:
bound.info("ok")
return
for i in issues:
bound.warning(f"{i.track.display_name}: {i.field}: {i.reason}")
# --- indexing ---------------------------------------------------------- #
def refresh_index(self) -> CollectionIndex:
"""Rebuild `self.index` from sldl's csv + an on-disk tag scan."""
idx = CollectionIndex()
self._load_sldl_index(idx)
self._scan_collection_tags(idx)
self.index = idx
return idx
def _load_sldl_index(self, idx: CollectionIndex) -> None:
if not self.index_csv.exists():
return
with self.index_csv.open(newline="", encoding="utf-8", errors="replace") as fh:
reader = csv.reader(fh)
try:
header = next(reader)
except StopIteration:
return
cols = {h.strip().lower(): i for i, h in enumerate(header)}
path_i = cols.get("filepath") or cols.get("path") or 0
artist_i = cols.get("artist")
title_i = cols.get("title")
state_i = cols.get("state")
for row in reader:
if not row:
continue
if state_i is not None and state_i < len(row):
state = row[state_i].strip().lower()
if state in {"2", "failed", "notfounderror", "error"}:
continue
try:
rel = row[path_i]
except IndexError:
continue
if not rel:
continue
path = self.root / rel if not Path(rel).is_absolute() else Path(rel)
if not path.exists():
continue
artist = row[artist_i] if artist_i is not None and artist_i < len(row) else ""
title = row[title_i] if title_i is not None and title_i < len(row) else ""
idx.add(artist, title, path)
def _scan_collection_tags(self, idx: CollectionIndex) -> None:
if not self.root.exists():
return
for p in self.root.rglob("*"):
if not p.is_file() or p.suffix.lower() not in AUDIO_EXTS:
continue
if "_metadata" in p.parts or "_playlists" in p.parts:
continue
try:
audio = MutagenFile(p, easy=True)
except Exception:
continue
if audio is None:
continue
tags = audio.tags or {}
artist = (tags.get("artist") or [""])[0] if hasattr(tags, "get") else ""
title = (tags.get("title") or [""])[0] if hasattr(tags, "get") else ""
if not title:
title = p.stem
idx.add(str(artist), str(title), p)
# --- planning + downloads --------------------------------------------- #
def find_missing(self, playlists: list[dict]) -> list[dict]:
"""Return the Spotify tracks not currently resolvable in this collection."""
seen: set[tuple[str, str]] = set()
missing: list[dict] = []
for pl in playlists:
for t in pl["tracks"]:
artist = ", ".join(t["artists"])
if self.index.lookup(artist, t["title"]) is not None:
continue
key = (norm(first_artist(artist)), norm(t["title"]))
if key in seen:
continue
seen.add(key)
missing.append(t)
return missing
async def download(
self,
missing: list[dict],
on_event: Callable[[dict], Awaitable[None]] | None = None,
) -> int:
"""Thin wrapper around `maple.collection.sockseek.download`."""
return await run_sockseek(self.root, self.index_csv, missing, on_event)
# --- per-track Spotify reconciliation --------------------------------- #
def compare_against_spotify(
self,
playlist_name: str,
track: dict,
path: Path,
) -> list[TrackIssue]:
"""Compare a downloaded file's tags + duration against Spotify.
Spotify is the source of truth. Each mismatched field yields a
`TrackIssue` whose `reason` reads as
`<field> mismatch: spotify=<sv> file=<fv>`. Empty/missing file
tags count as mismatches against a non-empty Spotify value.
"""
sp_artist = ", ".join(track.get("artists") or [])
sp_title = (track.get("title") or "").strip()
sp_album = (track.get("album") or "").strip()
sp_isrc = (track.get("isrc") or "").strip()
sp_duration = (track.get("duration_ms") or 0) / 1000.0
sp_date = (track.get("release_date") or "").strip()
try:
audio = MutagenFile(path, easy=True)
tags = audio.tags if audio is not None else None
except Exception:
tags = None
f_artist = (tags.get("artist") or [""])[0] if tags and hasattr(tags, "get") else ""
f_title = (tags.get("title") or [""])[0] if tags and hasattr(tags, "get") else ""
f_album = (tags.get("album") or [""])[0] if tags and hasattr(tags, "get") else ""
f_isrc = (tags.get("isrc") or [""])[0] if tags and hasattr(tags, "get") else ""
f_date = (tags.get("date") or [""])[0] if tags and hasattr(tags, "get") else ""
f_duration = file_duration(path)
issues: list[TrackIssue] = []
def mismatch(field: str, sv: object, fv: object) -> None:
issues.append(TrackIssue(
playlist_name, sp_artist, sp_title,
f"{field} mismatch: spotify={sv!r} file={fv!r} ({path.name})",
))
# Artist + title use fuzzy compare because tags & spotify quote
# remixers / features differently. Album + ISRC + date are exact-ish.
if sp_artist and fuzzy_ratio(norm(first_artist(sp_artist)), norm(first_artist(f_artist))) < 0.5:
mismatch("artist", sp_artist, f_artist)
if sp_title and fuzzy_ratio(norm_loose(sp_title), norm_loose(f_title)) < 0.5:
mismatch("title", sp_title, f_title)
if sp_album and f_album and fuzzy_ratio(norm_loose(sp_album), norm_loose(f_album)) < 0.5:
mismatch("album", sp_album, f_album)
elif sp_album and not f_album:
mismatch("album", sp_album, f_album)
if sp_isrc and f_isrc and sp_isrc.upper() != f_isrc.upper():
mismatch("isrc", sp_isrc, f_isrc)
# Compare only the year portion of `date` — file tags are usually "YYYY"
# while Spotify is "YYYY[-MM[-DD]]".
if sp_date and f_date and sp_date[:4] != f_date[:4]:
mismatch("date", sp_date, f_date)
if sp_duration > 0 and abs(f_duration - sp_duration) > DURATION_TOLERANCE_S:
mismatch("duration", f"{sp_duration:.1f}s", f"{f_duration:.1f}s")
return issues
def write_spotify_tags(self, track: dict, path: Path) -> None:
"""Delegate to the shared `maple.enrich.write_spotify_tags` so the
collection-side and export-side writes are byte-for-byte identical
and can't drift."""
_shared_write_spotify_tags(track, path)
def _existing_bpm(self, path: Path) -> float | None:
"""Return the parsed `bpm` tag value, or None if absent/invalid."""
try:
audio = MutagenFile(path, easy=True)
if audio is None or audio.tags is None:
return None
v = audio.tags.get("bpm")
if not v:
return None
return float(v[0])
except (ValueError, TypeError, Exception):
return None
def _write_bpm_tag(self, path: Path, bpm: float) -> None:
try:
audio = MutagenFile(path, easy=True)
if audio is None:
return
if audio.tags is None:
try:
audio.add_tags()
except Exception:
return
audio.tags["bpm"] = f"{bpm:.2f}"
audio.save()
except Exception as e:
logger.warning("failed to write bpm to {}: {}", path.name, e)
def process_track(
self,
playlist_name: str,
track: dict,
path: Path,
) -> list[TrackIssue]:
"""Reconcile a single downloaded file against Spotify + analyse it.
Steps:
1. ffprobe sanity check (one issue + early return on failure).
2. Compare every relevant field; collect mismatch issues.
3. Write Spotify's values back into the file's tags.
4. If the file has no `bpm` tag yet, run aubio BPM detection
and write the analytical value. Existing tags are left alone.
Idempotent within a run — paths already processed are no-ops.
"""
if path in self._reconciled:
return []
self._reconciled.add(path)
# Source-of-truth tag writes first: ensures the file mirrors Spotify
# before validation reads its tags back. Mismatches caught here are
# written to the report regardless of the validator's subsequent
# findings (the validator may dedup them if Spotify still disagrees).
self.write_spotify_tags(track, path)
# Single audio analysis pass: BPM + beat grid + waveform envelopes.
# Cached under collection/_analysis/<isrc>.json so `just export`
# reads the bytes back instead of redoing ffmpeg+mixxx. Analysis
# failures propagate — a broken analysis is fatal, not skipped.
isrc = (track.get("isrc") or "").strip()
existing_bpm = self._existing_bpm(path)
analysis = Analysis.analyze(path, isrc=isrc)
if analysis.bpm:
if existing_bpm is None:
self._write_bpm_tag(path, analysis.bpm)
logger.info("{}: bpm {:.2f} (mixxx)", path.name, analysis.bpm)
else:
logger.debug(
"{}: bpm {:.2f} (existing tag, kept; mixxx measured {:.2f})",
path.name, existing_bpm, analysis.bpm,
)
if isrc:
cp = analysis.save(self.analysis_dir)
logger.info("cached analysis for {}: {} → {}", isrc, path.name, cp)
else:
logger.warning(
"skipping analysis cache for {}: no ISRC in spotify track ({} - {})",
path.name, ", ".join(track.get("artists") or []), track.get("title") or "?",
)
# Full cross-cutting validation pass. Runs the SAME checks as
# `just export`, so a clean run here means the track will export
# without issues — and vice versa.
return _validate_track(
path,
spotify_track=track,
analysis_cache_dir=self.analysis_dir,
playlist=playlist_name,
)
# --- validation -------------------------------------------------------- #
async def validate(
self,
session: aiohttp.ClientSession,
playlists: list[dict],
) -> list[TrackIssue]:
"""Per-track Spotify reconciliation + artwork embed.
Spotify is the source of truth: any field that differs from the
downloaded file's tags is recorded as a mismatch issue, then the
file's tags are overwritten with the Spotify values. Missing
embedded artwork is queued for an async batch fetch.
"""
resolved: list[tuple[str, dict, Path]] = []
for pl in playlists:
for t in pl["tracks"]:
path = self.index.lookup(", ".join(t["artists"]), t["title"])
if path is not None:
resolved.append((pl["name"] or pl["id"], t, path))
issues: list[TrackIssue] = []
sem = asyncio.Semaphore(ARTWORK_CONCURRENCY)
art_jobs: list[tuple[str, dict, Path]] = []
for playlist_name, track, path in resolved:
issues.extend(self.process_track(playlist_name, track, path))
if not has_embedded_art(path) and track.get("album_image_url"):
art_jobs.append((playlist_name, track, path))
async def _embed(job: tuple[str, dict, Path]) -> None:
playlist_name, track, path = job
async with sem:
data = await fetch_jpeg(session, track["album_image_url"])
if not data:
issues.append(TrackIssue(
playlist_name, ", ".join(track["artists"]), track["title"],
"artwork download failed",
))
return
try:
embed_art(path, data)
except Exception as e:
issues.append(TrackIssue(
playlist_name, ", ".join(track["artists"]), track["title"],
f"artwork embed failed: {e}",
))
if art_jobs:
await asyncio.gather(*(_embed(j) for j in art_jobs))
return issues
# --- output ------------------------------------------------------------ #
def write_m3u8_for(
self,
playlist: dict,
issues: list[TrackIssue] | None = None,
) -> Path:
"""Write a single playlist's m3u8 against the current index.
Track paths are emitted relative to the m3u8's parent directory so
the playlist is portable across devices that sync the collection.
Pass `issues=None` for incremental rewrites during download — the
list of unresolved tracks gets rebuilt on the final pass instead.
"""
self.playlists_dir.mkdir(parents=True, exist_ok=True)
playlist_name = playlist["name"] or playlist["id"]
out_path = self.playlists_dir / f"{_sanitise_name(playlist_name)}.m3u8"
lines = ["#EXTM3U"]
for t in playlist["tracks"]:
artist = ", ".join(t["artists"])
title = t["title"]
secs = round(t["duration_ms"] / 1000)
path = self.index.lookup(artist, title)
if path is None:
lines.append(f"# NOT FOUND: {artist} - {title}")
if issues is not None:
issues.append(TrackIssue(playlist_name, artist, title, "unresolved in collection"))
continue
rel = os.path.relpath(path.resolve(), start=out_path.parent.resolve())
lines.append(f"#EXTINF:{secs},{artist} - {title}")
lines.append(rel)
out_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
return out_path
def write_m3u8s(
self,
playlists: list[dict],
issues: list[TrackIssue],
) -> list[Path]:
"""One m3u8 per playlist in `<root>/_playlists/`. Returns the paths written."""
return [self.write_m3u8_for(pl, issues) for pl in playlists]
+81
View File
@@ -0,0 +1,81 @@
"""Artist/title normalisation + `CollectionIndex` lookup table.
No I/O. Pure functions plus a small dataclass keyed on normalised
(artist, title) tuples so we can answer "is this track already in the
collection?" against an in-memory map.
"""
from __future__ import annotations
import re
import unicodedata
from dataclasses import dataclass, field
from pathlib import Path
FUZZY_THRESHOLD = 0.85
_PARENS = re.compile(r"\s*[\(\[][^)\]]*[\)\]]\s*")
_NONWORD = re.compile(r"[^\w\s]+", flags=re.UNICODE)
_WS = re.compile(r"\s+")
def norm(s: str) -> str:
"""Lowercase + NFKC + strip non-word, preserves remix/version suffixes."""
s = unicodedata.normalize("NFKC", s or "").lower()
s = _NONWORD.sub(" ", s)
s = _WS.sub(" ", s).strip()
return s
def norm_loose(s: str) -> str:
"""Same as norm() but with parenthesised suffixes removed."""
s = unicodedata.normalize("NFKC", s or "").lower()
s = _PARENS.sub(" ", s)
s = _NONWORD.sub(" ", s)
s = _WS.sub(" ", s).strip()
return s
def first_artist(artists: str) -> str:
"""Split sldl/spotify multi-artist strings on common delimiters."""
if not artists:
return ""
for sep in (",", " feat.", " ft.", " & ", "&", " and ", ";"):
if sep in artists:
return artists.split(sep, 1)[0].strip()
return artists.strip()
def fuzzy_ratio(a: str, b: str) -> float:
"""Cheap token-overlap ratio in [0, 1]. No external dep."""
ta, tb = set(a.split()), set(b.split())
if not ta or not tb:
return 0.0
return len(ta & tb) / max(len(ta), len(tb))
@dataclass
class CollectionIndex:
"""Maps normalised (artist, title) pairs to file paths in the collection."""
by_strict: dict[tuple[str, str], Path] = field(default_factory=dict)
by_loose: dict[tuple[str, str], Path] = field(default_factory=dict)
def add(self, artist: str, title: str, path: Path) -> None:
a, t = norm(first_artist(artist)), norm(title)
if a and t:
self.by_strict.setdefault((a, t), path)
la, lt = norm_loose(first_artist(artist)), norm_loose(title)
if la and lt:
self.by_loose.setdefault((la, lt), path)
def lookup(self, artist: str, title: str) -> Path | None:
a, t = norm(first_artist(artist)), norm(title)
if (a, t) in self.by_strict:
return self.by_strict[(a, t)]
la, lt = norm_loose(first_artist(artist)), norm_loose(title)
if (la, lt) in self.by_loose:
return self.by_loose[(la, lt)]
for (ka, kt), p in self.by_loose.items():
if ka == la and fuzzy_ratio(kt, lt) >= FUZZY_THRESHOLD:
return p
return None
+149
View File
@@ -0,0 +1,149 @@
"""Sockseek subprocess plumbing.
Owns:
* the CLI argument shape (`args_for`) used to run sockseek/sldl,
* the temp-CSV writer that feeds tracks in,
* the async `download()` entrypoint that streams `--progress-json`
events out and forwards sockseek stderr through loguru.
`Collection.download()` is a thin wrapper around `download()` here so the
subprocess concerns stay in one file.
"""
from __future__ import annotations
import asyncio
import csv
import json
import os
import re
import shutil
import sys
import tempfile
from collections.abc import Awaitable, Callable
from pathlib import Path
from loguru import logger
NAME_FORMAT = "{artist(/)album(/)track( )title|(_missing_tags/)slsk-foldername(/)slsk-filename}"
_ANSI_RE = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
_LEVEL_RE = re.compile(
r"^\s*\[?(?P<lvl>TRACE|TRC|DEBUG|DBG|INFO|INF|WARN|WRN|ERROR|ERR|FATAL|CRITICAL|CRT)\]?\b",
re.IGNORECASE,
)
_LEVEL_MAP = {
"TRC": "TRACE", "TRACE": "TRACE",
"DBG": "DEBUG", "DEBUG": "DEBUG",
"INF": "INFO", "INFO": "INFO",
"WRN": "WARNING", "WARN": "WARNING",
"ERR": "ERROR", "ERROR": "ERROR",
"FATAL": "CRITICAL", "CRT": "CRITICAL", "CRITICAL": "CRITICAL",
}
def log_level_for(line: str) -> str:
"""Map a sockseek/.NET-style log prefix to a loguru level. Default WARNING."""
m = _LEVEL_RE.match(line)
if not m:
return "WARNING"
return _LEVEL_MAP.get(m.group("lvl").upper(), "WARNING")
def args_for(root: Path, index_csv: Path, user: str, password: str) -> list[str]:
"""Mirror of src/sockseek.py:config_args(), pinned for FLAC-strict."""
return [
"--path", str(root),
"--index-path", str(index_csv),
"--user", user,
"--pass", password,
"--name-format", NAME_FORMAT,
"--format", "flac",
"--pref-format", "flac",
"--pref-min-bitrate", "200",
"--pref-max-bitrate", "2500",
"--strict-conditions",
"--fast-search",
"--concurrent-jobs", "4",
]
def write_download_csv(missing: list[dict], fh) -> None:
"""sldl-compatible CSV (Artist,Title,Album,Length)."""
w = csv.writer(fh)
w.writerow(["Artist", "Title", "Album", "Length"])
for t in missing:
w.writerow([
", ".join(t["artists"]),
t["title"],
t["album"],
round(t["duration_ms"] / 1000),
])
async def download(
root: Path,
index_csv: Path,
missing: list[dict],
on_event: Callable[[dict], Awaitable[None]] | None = None,
) -> int:
"""Shell out to sockseek for every track in `missing`.
When `on_event` is provided, sockseek is run with `--progress-json` and
every NDJSON envelope on stdout is parsed and awaited through the
callback. stderr is always piped and forwarded line-by-line through
loguru with a best-effort log-level guess.
"""
if not missing:
return 0
binary = shutil.which("sockseek") or shutil.which("sldl")
if binary is None:
logger.error("sockseek/sldl not on PATH; install slsk-batchdl first")
sys.exit(1)
user = os.environ.get("SOCKSEEK_USER")
password = os.environ.get("SOCKSEEK_PASS")
if not (user and password):
logger.error("set SOCKSEEK_USER and SOCKSEEK_PASS for downloads")
sys.exit(1)
index_csv.parent.mkdir(parents=True, exist_ok=True)
with tempfile.NamedTemporaryFile("w", suffix=".csv", newline="", delete=False) as tf:
write_download_csv(missing, tf)
tf_path = Path(tf.name)
try:
args = [binary, str(tf_path), *args_for(root, index_csv, user, password)]
if on_event is not None:
args.append("--progress-json")
masked = ["***" if a == password else a for a in args]
logger.info("$ {}", " ".join(masked))
proc = await asyncio.create_subprocess_exec(
*args,
stdout=asyncio.subprocess.PIPE if on_event is not None else None,
stderr=asyncio.subprocess.PIPE,
)
async def _forward_stderr() -> None:
assert proc.stderr is not None
async for raw in proc.stderr:
line = _ANSI_RE.sub("", raw.decode("utf-8", errors="replace")).rstrip()
if not line:
continue
logger.log(log_level_for(line), "sockseek: {}", line)
stderr_task = asyncio.create_task(_forward_stderr())
if on_event is not None:
assert proc.stdout is not None
async for raw in proc.stdout:
line = raw.decode("utf-8", errors="replace").strip()
if not line or not line.startswith("{"):
continue
try:
evt = json.loads(line)
except ValueError:
continue
await on_event(evt)
rc = await proc.wait()
await stderr_task
return rc
finally:
tf_path.unlink(missing_ok=True)
+103
View File
@@ -0,0 +1,103 @@
"""Per-track validation helpers: ffprobe sanity, duration probe, embedded
artwork inspection + embed, async JPEG fetch for missing cover art.
Pure helpers — no Collection coupling.
"""
from __future__ import annotations
import io
import shutil
import subprocess
from pathlib import Path
import aiohttp
from mutagen import File as MutagenFile
from mutagen.flac import FLAC, Picture
from mutagen.id3 import APIC, ID3, ID3NoHeaderError
from mutagen.mp4 import MP4, MP4Cover
DURATION_TOLERANCE_S = 5.0
ARTWORK_CONCURRENCY = 8
def ffprobe_ok(path: Path) -> bool:
if not shutil.which("ffprobe"):
return True # ffprobe absent → don't fail the run
r = subprocess.run(
["ffprobe", "-v", "error", "-select_streams", "a:0",
"-show_entries", "stream=codec_type", "-of", "csv=p=0", str(path)],
capture_output=True, text=True,
)
return r.returncode == 0 and "audio" in r.stdout
def file_duration(path: Path) -> float:
audio = MutagenFile(path)
info = getattr(audio, "info", None)
return float(getattr(info, "length", 0.0) or 0.0)
def has_embedded_art(path: Path) -> bool:
suf = path.suffix.lower()
if suf == ".flac":
try:
return bool(FLAC(path).pictures)
except Exception:
return False
if suf == ".mp3":
try:
return bool(ID3(path).getall("APIC"))
except Exception:
return False
if suf in (".m4a", ".mp4"):
try:
tags = MP4(path).tags
return bool(tags and tags.get("covr"))
except Exception:
return False
return True # unknown format → don't complain
def embed_art(path: Path, jpeg_bytes: bytes) -> None:
suf = path.suffix.lower()
if suf == ".flac":
f = FLAC(path)
pic = Picture()
pic.data = jpeg_bytes
pic.type = 3 # FRONT_COVER
pic.mime = "image/jpeg"
f.clear_pictures()
f.add_picture(pic)
f.save()
elif suf == ".mp3":
try:
tags = ID3(path)
except ID3NoHeaderError:
tags = ID3()
tags.delall("APIC")
tags.add(APIC(encoding=3, mime="image/jpeg", type=3, desc="Cover", data=jpeg_bytes))
tags.save(path)
elif suf in (".m4a", ".mp4"):
f = MP4(path)
f.tags = f.tags or {}
f["covr"] = [MP4Cover(jpeg_bytes, imageformat=MP4Cover.FORMAT_JPEG)]
f.save()
async def fetch_jpeg(session: aiohttp.ClientSession, url: str) -> bytes | None:
try:
async with session.get(url) as resp:
resp.raise_for_status()
data = await resp.read()
except aiohttp.ClientError:
return None
if data[:3] == b"\xff\xd8\xff":
return data
try:
from PIL import Image
img = Image.open(io.BytesIO(data)).convert("RGB")
buf = io.BytesIO()
img.save(buf, format="JPEG", quality=90)
return buf.getvalue()
except Exception:
return None
+82
View File
@@ -0,0 +1,82 @@
"""FLAC -> MP3 V0 joint-stereo conversion via ffmpeg, with source-hash dedup.
The SHA-256 of the source FLAC is stamped into a TXXX:MAPLE_SRC_SHA256 ID3
frame on the produced MP3. Reruns skip conversion when the existing MP3
carries the matching hash, so the cache is content-addressed against inputs.
"""
import hashlib
import shutil
import subprocess
from pathlib import Path
from mutagen import File as MutagenFile
from mutagen.id3 import ID3, ID3NoHeaderError, TXXX
HASH_FRAME_DESC = "MAPLE_SRC_SHA256"
def file_sha256(path: Path) -> str:
with path.open("rb") as f:
return hashlib.file_digest(f, "sha256").hexdigest()
def _read_source_hash(mp3: Path) -> str | None:
try:
tags = ID3(mp3)
except ID3NoHeaderError:
return None
for frame in tags.getall("TXXX"):
if frame.desc == HASH_FRAME_DESC and frame.text:
return str(frame.text[0])
return None
def _write_source_hash(mp3: Path, sha: str) -> None:
try:
tags = ID3(mp3)
except ID3NoHeaderError:
tags = ID3()
tags.delall(f"TXXX:{HASH_FRAME_DESC}")
tags.add(TXXX(encoding=3, desc=HASH_FRAME_DESC, text=sha))
tags.save(mp3)
def convert_flac_to_mp3(flac: Path, mp3: Path) -> bool:
"""Convert flac to mp3 (V0, joint stereo). Returns True if conversion ran."""
src_hash = file_sha256(flac)
if mp3.exists() and _read_source_hash(mp3) == src_hash:
return False
if not shutil.which("ffmpeg"):
raise RuntimeError("ffmpeg not found on PATH")
mp3.parent.mkdir(parents=True, exist_ok=True)
subprocess.run(
[
"ffmpeg", "-nostdin", "-loglevel", "error", "-y",
"-i", str(flac),
"-codec:a", "libmp3lame",
"-q:a", "0",
"-joint_stereo", "1",
"-map_metadata", "0",
str(mp3),
],
check=True,
)
_write_source_hash(mp3, src_hash)
return True
def _duration(path: Path) -> float:
audio = MutagenFile(path)
info = getattr(audio, "info", None)
return float(getattr(info, "length", 0.0) or 0.0)
def verify_duration(src: Path, dst: Path, tolerance: float = 0.1) -> None:
"""Raise if dst's audio length differs from src's by more than `tolerance` seconds."""
a, b = _duration(src), _duration(dst)
if abs(a - b) > tolerance:
raise RuntimeError(
f"duration mismatch: {src.name} = {a:.3f}s vs {dst.name} = {b:.3f}s (Δ {abs(a-b):.3f}s)"
)
+66
View File
@@ -0,0 +1,66 @@
"""Copy audio files into the USB Contents layout.
Targets are assumed to be FAT32-formatted USBs, which constrain us to:
- per-component name length <= 255 LFN chars
- total path (from drive root) <= 260 chars
- no '\\ / : * ? " < > |' or control chars in any component
We truncate per-component to a safe 64 chars (rekordbox firmware is
fussier than raw FAT32) and refuse to emit anything that would still
overflow the 260-char path budget.
"""
import re
import shutil
from pathlib import Path
_BAD = re.compile(r'[\\/:*?"<>|\x00-\x1f]')
MAX_COMPONENT = 48
MAX_PATH = 260
# Pioneer firmware appears to only handle the DeviceSQL short-ASCII form
# for `file_path` in the pdb (length byte < 127). The verified reference
# export uses short-form for every track. Keep the on-pdb path strictly
# inside that budget so every track stays loadable.
MAX_PDB_PATH = 126
def sanitise(component: str, max_len: int = MAX_COMPONENT) -> str:
s = _BAD.sub("_", component).strip(" .")
if len(s) > max_len:
s = s[:max_len].rstrip(" .")
return s or "_"
def _truncate_filename(name: str, max_len: int) -> str:
"""Truncate while preserving the extension."""
cleaned = _BAD.sub("_", name).strip(" .")
if len(cleaned) <= max_len:
return cleaned or "_"
stem, dot, ext = cleaned.rpartition(".")
if dot and len(ext) < max_len - 1:
keep = max_len - len(ext) - 1
return (stem[:keep].rstrip(" .") + "." + ext) or "_"
return cleaned[:max_len].rstrip(" .") or "_"
def staged_path(usb_root: Path, artist: str, album: str, filename: str) -> tuple[Path, str]:
artist_s = sanitise(artist)
album_s = sanitise(album)
prefix = f"/Contents/{artist_s}/{album_s}/"
budget = min(MAX_COMPONENT, MAX_PDB_PATH - len(prefix))
if budget < 8:
raise ValueError(f"no room for filename within MAX_PDB_PATH={MAX_PDB_PATH} (prefix uses {len(prefix)}): {prefix}")
name_s = _truncate_filename(filename, budget)
dest_rel = Path("Contents") / artist_s / album_s / name_s
dest_abs = usb_root / dest_rel
file_path = "/" + dest_rel.as_posix()
if len(file_path) > MAX_PDB_PATH:
raise ValueError(f"pdb path budget exceeded ({len(file_path)} > {MAX_PDB_PATH}): {file_path}")
return dest_abs, file_path
def stage_audio(src: Path, usb_root: Path, artist: str, album: str) -> tuple[Path, str]:
dest_abs, file_path = staged_path(usb_root, artist, album, src.name)
dest_abs.parent.mkdir(parents=True, exist_ok=True)
if not dest_abs.exists():
shutil.copy2(src, dest_abs)
return dest_abs, file_path
+307
View File
@@ -0,0 +1,307 @@
"""Spotify metadata enrichment shared by `just collection` and `just export`.
Per the project convention: a track is "valid" iff every export criterion
holds (artwork, tags, analysis). BOTH pipelines must be capable of
*bringing* a track to that state — validation isn't just reporting,
it's repair.
`just collection`:
1. Fetches Spotify playlists.
2. Persists every track to `<collection>/_metadata/spotify-tracks.json`.
3. Walks the collection, calls `write_spotify_tags` for each match.
`just export`:
1. Loads the manifest written by `just collection`.
2. Walks `data/usb/`'s staged files. For any file missing canonical
tags (typically ISRC), looks up its Spotify entry via the manifest
and calls `write_spotify_tags` BEFORE validation runs.
The shared write function lives here so the two pipelines are
byte-for-byte consistent in what they leave on disk.
"""
from __future__ import annotations
import json
from pathlib import Path
from loguru import logger
from mutagen import File as MutagenFile
from mutagen.flac import FLAC, Picture
from mutagen.id3 import APIC, ID3, ID3NoHeaderError
from mutagen.mp4 import MP4, MP4Cover
from maple.collection.index import first_artist, norm
from maple.collection.validate import embed_art, has_embedded_art
MANIFEST_NAME = "spotify-tracks.json"
# --------------------------------------------------------------------------- #
# Manifest persistence #
# --------------------------------------------------------------------------- #
def save_spotify_manifest(metadata_dir: Path, playlists: list[dict]) -> Path:
"""Persist Spotify-fetched playlists so `just export` can read them later.
Stored as a single JSON blob at `<metadata_dir>/spotify-tracks.json`.
Replaces any prior manifest atomically (tmp + rename).
"""
metadata_dir.mkdir(parents=True, exist_ok=True)
target = metadata_dir / MANIFEST_NAME
tmp = target.with_suffix(target.suffix + ".tmp")
tmp.write_text(
json.dumps({"playlists": playlists}, indent=2, ensure_ascii=False, default=str)
)
tmp.replace(target)
n = sum(len(p.get("tracks") or []) for p in playlists)
logger.info("wrote spotify manifest: {} tracks across {} playlist(s) → {}",
n, len(playlists), target)
return target
def load_spotify_manifest(
metadata_dir: Path,
) -> tuple[dict[tuple[str, str], dict], dict[str, dict]]:
"""Load the manifest written by `just collection`.
Returns `(by_key, by_isrc)` where `by_key` is indexed by
`(first_artist_norm, title_norm)` and `by_isrc` is indexed by the
uppercased ISRC. Empty dicts if the manifest is absent/unreadable.
"""
target = metadata_dir / MANIFEST_NAME
if not target.exists():
return {}, {}
try:
data = json.loads(target.read_text())
except Exception as e:
logger.warning("could not read spotify manifest at {}: {}", target, e)
return {}, {}
by_key: dict[tuple[str, str], dict] = {}
by_isrc: dict[str, dict] = {}
for pl in data.get("playlists", []):
for t in pl.get("tracks", []):
artist = ", ".join(t.get("artists") or [])
k = (norm(first_artist(artist)), norm(t.get("title") or ""))
by_key.setdefault(k, t)
isrc = (t.get("isrc") or "").strip().upper()
if isrc:
by_isrc.setdefault(isrc, t)
return by_key, by_isrc
def find_spotify_track(
path: Path,
by_key: dict[tuple[str, str], dict],
by_isrc: dict[str, dict],
) -> dict | None:
"""Resolve a file to its Spotify track via ISRC tag, then (artist, title)."""
try:
audio = MutagenFile(path, easy=True)
except Exception:
return None
if audio is None or audio.tags is None:
return None
tags = audio.tags
def first(k: str) -> str:
if hasattr(tags, "get"):
v = tags.get(k)
if v:
return str(v[0])
return ""
isrc = first("isrc").upper().strip()
if isrc and isrc in by_isrc:
return by_isrc[isrc]
artist = first("artist")
title = first("title")
if artist or title:
k = (norm(first_artist(artist)), norm(title))
return by_key.get(k)
return None
# --------------------------------------------------------------------------- #
# Tag writing (the canonical write path) #
# --------------------------------------------------------------------------- #
def write_spotify_tags(track: dict, path: Path) -> bool:
"""Overwrite the file's tags with the Spotify-canonical values.
Both pipelines call this. Writes title, artist (`", ".join`),
album, date, genre (list), and ISRC via mutagen's easy interface.
Empty Spotify values leave the existing tag untouched.
Returns True when at least one tag was written + saved.
"""
try:
audio = MutagenFile(path, easy=True)
except Exception as e:
logger.warning("write_spotify_tags: cannot open {}: {}", path.name, e)
return False
if audio is None:
return False
if audio.tags is None:
try:
audio.add_tags()
except Exception as e:
logger.warning("write_spotify_tags: cannot init tags on {}: {}", path.name, e)
return False
wrote_any = False
title = (track.get("title") or "").strip()
if title:
audio.tags["title"] = title
wrote_any = True
artists = track.get("artists") or []
if artists:
audio.tags["artist"] = ", ".join(artists)
wrote_any = True
album = (track.get("album") or "").strip()
if album:
audio.tags["album"] = album
wrote_any = True
date = (track.get("release_date") or "").strip()
if date:
try:
audio.tags["date"] = date
wrote_any = True
except (KeyError, ValueError) as e:
logger.debug("date tag write skipped on {}: {}", path.name, e)
genres = track.get("genres") or []
if genres:
try:
audio.tags["genre"] = list(genres)
wrote_any = True
except (KeyError, ValueError) as e:
logger.debug("genre tag write skipped on {}: {}", path.name, e)
isrc = (track.get("isrc") or "").strip()
if isrc:
try:
audio.tags["isrc"] = isrc
wrote_any = True
except (KeyError, ValueError) as e:
logger.debug("isrc tag write skipped on {}: {}", path.name, e)
if not wrote_any:
return False
try:
audio.save()
return True
except Exception as e:
logger.warning("write_spotify_tags: save failed for {}: {}", path.name, e)
return False
def write_bpm_tag(path: Path, bpm: float, *, overwrite: bool = False) -> bool:
"""Write a `bpm` tag to the file. When `overwrite=False` (the default),
leaves any existing tag value alone — preserves human/label-set BPMs."""
try:
audio = MutagenFile(path, easy=True)
except Exception as e:
logger.warning("write_bpm_tag: cannot open {}: {}", path.name, e)
return False
if audio is None:
return False
if audio.tags is None:
try:
audio.add_tags()
except Exception as e:
logger.warning("write_bpm_tag: cannot init tags on {}: {}", path.name, e)
return False
if not overwrite:
existing = audio.tags.get("bpm") if hasattr(audio.tags, "get") else None
if existing and (existing[0] if isinstance(existing, list) else existing):
return False # keep the existing tag
audio.tags["bpm"] = f"{bpm:.2f}"
try:
audio.save()
return True
except Exception as e:
logger.warning("write_bpm_tag: save failed for {}: {}", path.name, e)
return False
def download_jpeg(url: str, *, timeout: float = 15.0) -> bytes | None:
"""Sync GET of a Spotify album_image_url. Returns the raw bytes or
None on failure. Used by `embed_artwork_from_track` — kept sync so
both `just collection`'s async loop and `just export`'s sync loop
can call into the same helper without contortions."""
if not url:
return None
try:
import urllib.request
req = urllib.request.Request(url, headers={"User-Agent": "maple/1.0"})
with urllib.request.urlopen(req, timeout=timeout) as resp:
return resp.read()
except Exception as e:
logger.warning("artwork fetch failed for {}: {}", url, e)
return None
def embed_artwork_from_track(track: dict, path: Path, *, force: bool = False) -> bool:
"""Fetch the Spotify cover art for `track` and embed it into `path`.
No-op when the file already has embedded art (unless `force=True`).
Skips silently when the track dict has no `album_image_url` (Spotify
occasionally returns tracks without images).
"""
if not force and has_embedded_art(path):
return False
url = (track.get("album_image_url") or "").strip()
if not url:
return False
data = download_jpeg(url)
if not data:
return False
try:
embed_art(path, data)
logger.info(
"embedded artwork for {} (from {})",
path.name, url.rsplit("/", 1)[-1][:40],
)
return True
except Exception as e:
logger.warning("embed_art failed for {}: {}", path.name, e)
return False
def enrich_from_manifest(
path: Path,
by_key: dict[tuple[str, str], dict],
by_isrc: dict[str, dict],
analysis_cache_dir: Path | None = None,
) -> dict | None:
"""Find this file's Spotify entry, bring it to "valid" state, return
the matched track dict (or None if no match).
Steps:
1. Write canonical text tags via `write_spotify_tags`.
2. Fetch + embed album art if the file lacks it (`embed_artwork_from_track`).
3. If `analysis_cache_dir` is supplied and the cache has an entry for
this track's ISRC, write the cached BPM as a tag. This lets the
enrich pass bring a track all the way to "validates" without a
separate aubio-analysis step.
"""
track = find_spotify_track(path, by_key, by_isrc)
if track is None:
return None
if write_spotify_tags(track, path):
logger.info(
"enriched {} from spotify (isrc={!r}, title={!r})",
path.name, (track.get("isrc") or "")[:12], (track.get("title") or "")[:32],
)
embed_artwork_from_track(track, path)
# Backfill BPM tag from the analysis cache when present. Avoids
# re-running aubio here — if the cache is cold, the per-track loop's
# write_anlz will populate it and write the tag in a later pass.
isrc = (track.get("isrc") or "").strip()
if analysis_cache_dir is not None and isrc:
from maple.analyze import Analysis
cached = Analysis.load(analysis_cache_dir, isrc)
if cached is not None and cached.bpm:
write_bpm_tag(path, cached.bpm, overwrite=False)
return track
+20
View File
@@ -0,0 +1,20 @@
"""Loguru sink config for maple's CLIs.
`configure(level)` (re)installs a single stderr sink with a compact format.
Idempotent — safe to call from every CLI entrypoint.
"""
import os
import sys
from loguru import logger
_FMT = "<green>{time:HH:mm:ss}</green> <level>{level:<7}</level> {message}"
def configure(level: str | None = None) -> None:
level = (level or os.environ.get("MAPLE_LOG_LEVEL") or "INFO").upper()
logger.remove()
logger.add(sys.stderr, format=_FMT, level=level, colorize=True, enqueue=False)
__all__ = ["configure", "logger"]
+105
View File
@@ -0,0 +1,105 @@
"""Audio metadata extraction via mutagen."""
from dataclasses import dataclass
from pathlib import Path
from mutagen import File as MutagenFile
DEFAULT_TEMPO_X100 = 12000 # 120.00 BPM, used when no BPM tag is present
@dataclass
class TrackMeta:
title: str
artist: str
album: str
genre: str
track_number: int
year: int
duration: int
bitrate: int
sample_rate: int
sample_depth: int
file_size: int
tempo: int = DEFAULT_TEMPO_X100 # BPM * 100 (rekordbox storage format)
def _first(value) -> str:
if value is None:
return ""
if isinstance(value, list):
return str(value[0]) if value else ""
return str(value)
def _int(value) -> int:
s = _first(value).strip()
if not s:
return 0
# Handle "3/12" track numbers
s = s.split("/")[0]
try:
return int(s)
except ValueError:
return 0
def _tempo_x100(value) -> int:
"""Parse a BPM tag into integer BPM * 100. 0 means "not present"."""
s = _first(value).strip()
if not s:
return 0
try:
return int(round(float(s) * 100))
except ValueError:
return 0
def read_tags(path: Path) -> TrackMeta:
file_size = path.stat().st_size
audio = MutagenFile(path, easy=True)
if audio is None:
return TrackMeta(
title=path.stem,
artist="Unknown Artist",
album="Unknown Album",
genre="",
track_number=0,
year=0,
duration=0,
bitrate=0,
sample_rate=44100,
sample_depth=16,
file_size=file_size,
)
tags = audio.tags or {}
title = _first(tags.get("title")) or path.stem
artist = _first(tags.get("artist")) or "Unknown Artist"
album = _first(tags.get("album")) or "Unknown Album"
genre = _first(tags.get("genre"))
track_number = _int(tags.get("tracknumber"))
year = _int(tags.get("date") or tags.get("year"))
tempo = _tempo_x100(tags.get("bpm")) or DEFAULT_TEMPO_X100
info = getattr(audio, "info", None)
duration = int(getattr(info, "length", 0) or 0)
bitrate_bps = int(getattr(info, "bitrate", 0) or 0)
bitrate = bitrate_bps // 1000 if bitrate_bps > 1000 else bitrate_bps
sample_rate = int(getattr(info, "sample_rate", 44100) or 44100)
sample_depth = int(getattr(info, "bits_per_sample", 16) or 16)
return TrackMeta(
title=title,
artist=artist,
album=album,
genre=genre,
track_number=track_number,
year=year,
duration=duration,
bitrate=bitrate,
sample_rate=sample_rate,
sample_depth=sample_depth,
file_size=file_size,
tempo=tempo,
)
+211
View File
@@ -0,0 +1,211 @@
"""Domain model for the in-memory database that becomes a .pdb file."""
from dataclasses import dataclass, field
from pathlib import Path
@dataclass
class Artist:
id: int
name: str
@dataclass
class Album:
id: int
name: str
artist_id: int
@dataclass
class Genre:
id: int
name: str
@dataclass
class Color:
id: int
name: str
@dataclass
class Key:
id: int
name: str
id2: int | None = None # mirrors id in real exports
@dataclass
class Track:
id: int
title: str
artist_id: int
album_id: int
filename: str
file_path: str
file_size: int = 0
sample_rate: int = 44100
sample_depth: int = 16
bitrate: int = 0
duration: int = 0
track_number: int = 0
year: int = 0
disc_number: int = 0
tempo: int = 0
comment: str = ""
date_added: str = ""
release_date: str = ""
mix_name: str = ""
analyze_path: str = ""
analyze_date: str = ""
artwork_id: int = 0
genre_id: int = 0
source_path: Path | None = None
@dataclass
class Artwork:
id: int
path: str
@dataclass
class Column:
id: int
code: int
name: str
# Reverse-engineered from a verified rekordbox export. Each entry is
# (column_id, internal column code, display name). Code values are
# non-contiguous because rekordbox skips reserved slots.
DEFAULT_COLUMNS: list[Column] = [
Column(1, 0x80, "GENRE"),
Column(2, 0x81, "ARTIST"),
Column(3, 0x82, "ALBUM"),
Column(4, 0x83, "TRACK"),
Column(5, 0x85, "BPM"),
Column(6, 0x86, "RATING"),
Column(7, 0x87, "YEAR"),
Column(8, 0x88, "REMIXER"),
Column(9, 0x89, "LABEL"),
Column(10, 0x8A, "ORIGINAL ARTIST"),
Column(11, 0x8B, "KEY"),
Column(12, 0x8D, "CUE"),
Column(13, 0x8E, "COLOR"),
Column(14, 0x92, "TIME"),
Column(15, 0x93, "BITRATE"),
Column(16, 0x94, "FILE NAME"),
Column(17, 0x84, "PLAYLIST"),
Column(18, 0x98, "HOT CUE BANK"),
Column(19, 0x95, "HISTORY"),
Column(20, 0x91, "SEARCH"),
Column(21, 0x96, "COMMENTS"),
Column(22, 0x8C, "DATE ADDED"),
Column(23, 0x97, "DJ PLAY COUNT"),
Column(24, 0x90, "FOLDER"),
Column(25, 0xA1, "DEFAULT"),
Column(26, 0xA2, "ALPHABET"),
Column(27, 0xAA, "MATCHING"),
]
# Reference-derived rows for the un-decoded `unknown_17` and `unknown_18`
# tables. Each tuple is (a, b, c, d) packed as four little-endian u2s.
# These appear to drive the player's "Filter by ->" menu (which columns
# are filterable, sort order, filter type code). Speculative purpose;
# verified to match the reference byte-for-byte.
UNKNOWN_17_ROWS: list[tuple[int, int, int, int]] = [
(1, 1, 355, 0), (5, 6, 261, 0), (6, 7, 355, 0), (7, 8, 355, 0),
(8, 9, 355, 0), (9, 10, 355, 0), (10, 11, 355, 0), (13, 15, 355, 0),
(14, 19, 260, 0), (15, 20, 262, 0), (16, 21, 355, 0), (18, 23, 355, 0),
(2, 2, 2, 1), (3, 3, 3, 2), (4, 4, 1, 3), (11, 12, 99, 4),
(17, 5, 99, 5), (19, 22, 99, 6), (20, 18, 99, 7), (27, 26, 611, 8),
(24, 17, 99, 9), (22, 27, 99, 10),
]
UNKNOWN_18_ROWS: list[tuple[int, int, int, int]] = [
(1, 6, 1, 0), (21, 7, 1, 0), (14, 8, 1, 0), (8, 9, 1, 0),
(9, 10, 1, 0), (10, 11, 1, 0), (15, 13, 1, 0), (13, 15, 1, 0),
(23, 16, 1, 0), (22, 17, 1, 0), (25, 0, 256, 0), (26, 1, 512, 0),
(2, 2, 768, 0), (3, 3, 1024, 0), (5, 4, 1280, 0), (6, 5, 1536, 0),
(11, 12, 1792, 0),
]
# The standard rekordbox color label palette.
STANDARD_COLORS: list[Color] = [
Color(1, "Pink"),
Color(2, "Red"),
Color(3, "Orange"),
Color(4, "Yellow"),
Color(5, "Green"),
Color(6, "Aqua"),
Color(7, "Blue"),
Color(8, "Purple"),
]
@dataclass
class PlaylistNode:
id: int
parent_id: int
sort_order: int
name: str
is_folder: bool = False
@dataclass
class PlaylistEntry:
playlist_id: int
track_id: int
entry_index: int
@dataclass
class Database:
tracks: list[Track] = field(default_factory=list)
artists: list[Artist] = field(default_factory=list)
albums: list[Album] = field(default_factory=list)
playlist_nodes: list[PlaylistNode] = field(default_factory=list)
playlist_entries: list[PlaylistEntry] = field(default_factory=list)
artwork: list[Artwork] = field(default_factory=list)
genres: list[Genre] = field(default_factory=list)
colors: list[Color] = field(default_factory=list)
keys: list[Key] = field(default_factory=list)
def validate_referential_integrity(db: "Database") -> list[str]:
"""Return a list of dangling-foreign-key errors. Every FK must be 0 ("none")
or resolve to a real row in its target table. A track that references a
genre/artist/album/artwork id with no matching row is freeze-class: the
player follows the FK into an empty/short table and hangs (this is exactly
how build_acemo.py's omitted-genres bug froze the deck). Empty string = OK.
"""
errors: list[str] = []
artist_ids = {a.id for a in db.artists}
album_ids = {a.id for a in db.albums}
artwork_ids = {a.id for a in db.artwork}
genre_ids = {g.id for g in db.genres}
key_ids = {k.id for k in db.keys}
track_ids = {t.id for t in db.tracks}
playlist_ids = {n.id for n in db.playlist_nodes}
def check(kind: str, owner_id: int, field: str, value: int, valid: set[int]) -> None:
# 0 means "unset / no reference" everywhere in the pdb schema.
if value and value not in valid:
errors.append(f"{kind} id={owner_id}: {field}={value} has no matching row")
for t in db.tracks:
check("track", t.id, "artist_id", t.artist_id, artist_ids)
check("track", t.id, "album_id", t.album_id, album_ids)
check("track", t.id, "artwork_id", t.artwork_id, artwork_ids)
check("track", t.id, "genre_id", t.genre_id, genre_ids)
for a in db.albums:
check("album", a.id, "artist_id", a.artist_id, artist_ids)
for e in db.playlist_entries:
check("playlist_entry", e.playlist_id, "track_id", e.track_id, track_ids)
check("playlist_entry", e.playlist_id, "playlist_id", e.playlist_id, playlist_ids)
# key_ids is currently unreferenced by Track in maple's model; included for
# completeness so adding a track.key_id later is covered.
_ = key_ids
return errors
View File
Whitespace-only changes.
+172
View File
@@ -0,0 +1,172 @@
"""Page builder for the rekordbox .pdb format."""
from struct import pack
PAGE_SIZE = 4096
HEADER_SIZE = 40
ROW_GROUP_SIZE = 36
ROWS_PER_GROUP = 16
def _align4(offset: int) -> int:
return (offset + 3) & ~3
def _num_row_groups(n: int) -> int:
return (n + ROWS_PER_GROUP - 1) // ROWS_PER_GROUP
# Strange-page heap is a DeviceSQL free-space map, byte-exact in a fresh export:
# u4(page_index) + u4(data_page_ptr) + 12-byte const tail
# + (f8 ff ff 1f)*1004 + 20 trailing zeros.
# data_page_ptr is the table's first data-page index, or the sentinel
# 0x03ffffff when the table is strange-only (no data page). The 1004 repeat
# count is the same value stored in the header's unknown6 field (0x03ec).
# Verified byte-for-byte against data/acemo_fresh strange pages.
_STRANGE_HEAP_NO_DATA = 0x03FFFFFF
_STRANGE_HEAP_TAIL_CONST = bytes.fromhex("ffffff03000000000000ff1f")
_STRANGE_HEAP_PATTERN = bytes.fromhex("f8ffff1f")
_STRANGE_HEAP_REPEAT = 1004
_STRANGE_HEAP_TAIL = 20
# Strange pages on these table types carry unknown7=1; all others 0. In a fresh
# export this is set for tracks (first table) and history (last table). Other
# fields (free/used=0, txc/txi=0x1fff) match across all strange pages.
STRANGE_UNKNOWN7 = frozenset({0, 19})
def build_strange_page(page_index: int, page_type: int, next_page: int,
data_page: int | None = None) -> bytes:
"""Empty placeholder page that the reference exports use as the first page of
every table chain. Reproduces a FRESH rekordbox export byte-for-byte: header
flags/markers plus the DeviceSQL free-space heap map. `next_page` should be
the reserved data-page index (page_index+1) even for strange-only tables --
rekordbox always points the strange page at its (possibly-blank) data slot.
`data_page` is the table's first data-page index, or None when the table is
strange-only (the free-map then stores the 0x03ffffff sentinel there).
"""
header = b""
header += pack("<I", 0) # magic
header += pack("<I", page_index)
header += pack("<I", page_type)
header += pack("<I", next_page)
header += pack("<I", 1) # sequence
header += pack("<I", 0) # reserved
# packed: num_row_offsets=0, num_rows=0, page_flags=0x64
header += bytes([0, 0, 0, 0x64])
header += pack("<H", 0) # free_size (fresh export = 0)
header += pack("<H", 0) # used_size (fresh export = 0)
header += pack("<H", 0x1fff) # transaction_row_count = failed
header += pack("<H", 0x1fff) # transaction_row_index = failed
header += pack("<H", 0x03ec) # 1004 (free-map repeat count)
header += pack("<H", 1 if page_type in STRANGE_UNKNOWN7 else 0) # unknown7
assert len(header) == HEADER_SIZE
data_ptr = _STRANGE_HEAP_NO_DATA if data_page is None else data_page
heap = pack("<II", page_index, data_ptr) + _STRANGE_HEAP_TAIL_CONST
heap += _STRANGE_HEAP_PATTERN * _STRANGE_HEAP_REPEAT
heap += b"\x00" * _STRANGE_HEAP_TAIL
assert len(heap) == PAGE_SIZE - HEADER_SIZE
return header + heap
class PageBuilder:
def __init__(self, page_index: int, page_type: int, max_rows: int = 0,
data_flag: int = 0x24, has_index_shift: bool = False):
self.page_index = page_index
self.page_type = page_type
self.next_page = page_index # caller patches before finalize
self.max_rows = max_rows
self.data_flag = data_flag # page_flags for a populated data page
# Row types whose layout begins `subtype:u2, index_shift:u2` (tracks,
# artists, albums) carry a slot-based index_shift at row offset 0x02.
# See FORMAT_SPEC.md s1: index_shift = slot_index << 5, where slot_index
# is the row's 0-based position in the page row index. maple never
# deletes rows, so slot_index == the row's ordinal on the page.
self.has_index_shift = has_index_shift
self.row_bytes: list[bytes] = []
self.row_offsets: list[int] = [] # relative to heap_pos
self.heap_cursor = 0 # position relative to heap_pos
def try_add(self, row: bytes) -> bool:
aligned = _align4(self.heap_cursor)
end = aligned + len(row)
footer_after = _num_row_groups(len(self.row_bytes) + 1) * ROW_GROUP_SIZE
if end + HEADER_SIZE + footer_after > PAGE_SIZE:
return False
if self.max_rows and len(self.row_bytes) >= self.max_rows:
return False
self.row_offsets.append(aligned)
self.row_bytes.append(row)
self.heap_cursor = end
return True
def num_rows(self) -> int:
return len(self.row_bytes)
def finalize(self) -> bytes:
n = len(self.row_bytes)
num_rgs = _num_row_groups(n)
footer_size = num_rgs * ROW_GROUP_SIZE
heap_padding = PAGE_SIZE - HEADER_SIZE - footer_size
used_size = self.heap_cursor
free_size = heap_padding - used_size
if free_size < 0:
raise ValueError("page overflow")
# Data pages with rows: rekordbox uses 0x24 (and 0x34 for some tables --
# see writer's PAGE_FLAG_34). maple previously wrote 0x00, which is
# missing the 0x20|0x04 bits every real rekordbox data page sets, and
# the players reject it. Empty data pages stay 0x44.
page_flags = self.data_flag if n > 0 else 0x44
header = b""
header += pack("<I", 0) # magic
header += pack("<I", self.page_index)
header += pack("<I", self.page_type)
header += pack("<I", self.next_page)
header += pack("<I", 0) # unknown1
header += pack("<I", 0) # unknown2
# 24 bits little-endian: low 13 = num_row_offsets, next 11 = num_rows,
# then u1 page_flags. Kaitai reads these as read_bits_int_le.
packed = ((n & 0x7FF) << 13) | (n & 0x1FFF)
header += bytes([packed & 0xFF, (packed >> 8) & 0xFF, (packed >> 16) & 0xFF, page_flags])
header += pack("<HH", free_size, used_size)
header += pack("<H", 0) # transaction_row_count
header += pack("<H", 0) # transaction_row_index
header += pack("<H", 0) # unknown6
header += pack("<H", 0) # unknown7
assert len(header) == HEADER_SIZE
heap = bytearray(heap_padding)
for slot, (ofs, row) in enumerate(zip(self.row_offsets, self.row_bytes)):
if self.has_index_shift:
# Patch index_shift (u2 at row offset 0x02) from the placeholder
# to slot_index<<5 (FORMAT_SPEC.md s1).
row = row[:2] + pack("<H", (slot << 5) & 0xFFFF) + row[4:]
heap[ofs:ofs + len(row)] = row
# Footers go after the heap in the file. Group 0 (rows 0-15) sits closest
# to the end of the page; group num_rgs-1 sits closest to the heap. So
# emit footers in reverse group order.
footer = b""
for g in range(num_rgs - 1, -1, -1):
start = g * ROWS_PER_GROUP
grp_offsets = self.row_offsets[start:start + ROWS_PER_GROUP]
grp_count = len(grp_offsets)
padded = grp_offsets + [0] * (ROWS_PER_GROUP - grp_count)
presence = (1 << grp_count) - 1
rg = b""
for ofs in reversed(padded):
rg += pack("<H", ofs)
rg += pack("<H", presence)
# Trailing u2: on a fresh, fully-packed page (no deleted slots) this
# equals the presence mask. Verified across every clean row group in
# data/acemo_fresh; pages where ref's value differs from presence are
# exactly the dead-slot/edit-history pages (see KNOWN_DIVERGENCE.md).
# Was a constant 16 (rekordcrate default), which never matched.
rg += pack("<H", presence)
assert len(rg) == ROW_GROUP_SIZE
footer += rg
assert len(footer) == footer_size
return header + bytes(heap) + footer
+251
View File
@@ -0,0 +1,251 @@
"""Row serialisers for the rekordbox export.pdb format."""
import zlib
from struct import pack
from maple.model import Album, Artist, Artwork, Color, Column, Genre, PlaylistEntry, PlaylistNode, Track
from maple.pdb.strings import encode_devicesql
TRACK_FIXED_PREFIX = 0x5E
TRACK_OFS_STRINGS_START = TRACK_FIXED_PREFIX
TRACK_HEADER_LEN = TRACK_OFS_STRINGS_START + 21 * 2
# Per-rekordbox-install seed for the track opaque-id blob (row 0x18, u4). It is
# a master.db value, constant across a single install, not derivable from track
# metadata (FORMAT_SPEC.md s2). EVERY real rekordbox export we have carries a
# NON-ZERO seed (this install = 0x21b4a7ab; data/MAPLE = 0xd6d3a567); the
# 0x78f74a68 "unanalyzed default" was a false lead -- it came from maple's own
# earlier output (data/usb), not a real export. Default to this install's real
# value; override per-install if deck testing shows it matters. Players treat
# the whole blob as opaque.
# Per-export "seed" stored at offset 0x18 of every TrackRow. Real rekordbox
# files carry the LE u4 0xd6d3a567 here (parses as opaque_seed_lo=0xa567,
# opaque_seed_hi=0xd6d3). Maple previously used 0x21B4A7AB which the kaitai
# would parse as different seed_lo/hi values — the difference may be benign
# for the CDJ's analyse-path lookup but matches every real reference export
# we have, so we mirror it.
TRACK_OPAQUE_SEED = 0xD6D3A567
# Range maple maps its stand-in opaque_track_id_lo (row 0x14) into, so it has the
# shape of a real rekordbox Content ID. The real AceMo lo values all fall in this
# band (observed 0x02c0a654 .. 0x0f8fd598), i.e. ~134M-261M. We can't reproduce
# the true (encrypted-DB) Content ID, so we map a stable per-track hash into the
# band -- non-zero, in-range, deterministic. Documented divergence (FORMAT_SPEC s2).
OPAQUE_LO_MIN = 0x08000000
OPAQUE_LO_MAX = 0x0F900000
def _pad2(b: bytes) -> bytes:
return b + b"\x00" if len(b) & 1 else b
def serialize_track(t: Track) -> bytes:
strings = [
"", # 0 isrc
"", # 1 texter
"1", # 2 unknown_string_2
"2", # 3 unknown_string_3
"", # 4 unknown_string_4
"", # 5 message
"ON", # 6 kuvo_public
"ON", # 7 autoload_hot_cues
"", # 8 unknown_string_5
"", # 9 unknown_string_6
t.date_added, # 10 date_added
t.release_date, # 11 release_date
t.mix_name, # 12 mix_name
"", # 13 unknown_string_7
t.analyze_path, # 14 analyze_path
t.analyze_date, # 15 analyze_date
t.comment, # 16 comment
t.title, # 17 title
"", # 18 unknown_string_8
t.filename, # 19 filename
t.file_path, # 20 file_path
]
encoded = [encode_devicesql(s) for s in strings]
offsets = []
cursor = TRACK_HEADER_LEN
blob_parts = []
for enc in encoded:
offsets.append(cursor)
blob_parts.append(enc)
cursor += len(enc)
blob = b"".join(blob_parts)
prefix = b""
prefix += pack("<HH", 0x24, 0) # subtype, index_shift
prefix += pack("<I", 788224) # bitmask
prefix += pack("<I", t.sample_rate) # sample_rate
prefix += pack("<I", 0) # composer_id
prefix += pack("<I", t.file_size) # file_size
# offsets 0x14/0x18/0x1a are ONE 8-byte LE opaque id:
# blob_u64 = opaque_track_id_lo | (opaque_seed << 32)
# Both halves are rekordbox-internal stored values (master.db / djmdContent
# + a per-install seed), NOT a function of the track metadata maple has --
# proven across the real exports + an exhaustive negative hash search
# (FORMAT_SPEC.md s2). They cannot be reproduced byte-for-byte without
# rekordbox's DB; that is a documented divergence (KNOWN_DIVERGENCE.md).
# Every REAL export has BOTH halves non-zero, so emit a non-zero stand-in of
# the right SHAPE: the per-install seed, and a per-track lo mapped into the
# real Content-ID band (OPAQUE_LO_MIN..MAX) -- which is where every observed
# AceMo lo lands. Stable hash -> band, so it's deterministic and in-range.
# (Was the 0xa567/0xd6d3 marker, which matched no real export; lo=0/
# seed=0x78f74a68 was also wrong -- that came from maple's own output
# mistaken for an "unanalyzed library", not a real rekordbox export.)
h = zlib.crc32(f"{t.id}|{t.file_path}".encode()) & 0xFFFFFFFF
opaque_lo = OPAQUE_LO_MIN + (h % (OPAQUE_LO_MAX - OPAQUE_LO_MIN))
prefix += pack("<I", opaque_lo) # opaque_track_id_lo (stand-in, per track)
prefix += pack("<I", TRACK_OPAQUE_SEED) # opaque_seed (u4 @0x18 = lo|hi<<16)
prefix += pack("<I", t.artwork_id) # artwork_id
prefix += pack("<I", 0) # key_id
prefix += pack("<I", 0) # original_artist_id
prefix += pack("<I", 0) # label_id
prefix += pack("<I", 0) # remixer_id
prefix += pack("<I", t.bitrate) # bitrate
prefix += pack("<I", t.track_number) # track_number
prefix += pack("<I", t.tempo) # tempo
prefix += pack("<I", t.genre_id) # genre_id
prefix += pack("<I", t.album_id) # album_id
prefix += pack("<I", t.artist_id) # artist_id
prefix += pack("<I", t.id) # id
prefix += pack("<H", t.disc_number) # disc_number
prefix += pack("<H", 0) # play_count
prefix += pack("<H", t.year) # year
prefix += pack("<H", t.sample_depth) # sample_depth
prefix += pack("<H", t.duration) # duration
prefix += pack("<H", 41) # unknown ("always 41?")
prefix += pack("<BB", 0, 0) # color_id, rating
prefix += pack("<H", 1) # unknown ("always 1?")
prefix += pack("<H", 3) # unknown ("alternating 2 or 3")
assert len(prefix) == TRACK_FIXED_PREFIX, len(prefix)
ofs_block = b"".join(pack("<H", o) for o in offsets)
row = prefix + ofs_block + blob
return _pad2(row)
def serialize_artist(a: Artist) -> bytes:
name = encode_devicesql(a.name)
near_ofs = 0x0A
far_threshold = 0x100
if near_ofs < far_threshold:
subtype = 0x60
row = b""
# index_shift is slot-based (slot_index<<5); the pager patches it at
# page-build time -- see FORMAT_SPEC.md s1. Write 0 as a placeholder.
row += pack("<HH", subtype, 0) # subtype, index_shift
row += pack("<I", a.id) # id
row += pack("<B", 0x03) # always 0x03
row += pack("<B", near_ofs) # ofs_name_near
assert len(row) == near_ofs
row += name
else:
subtype = 0x64
# Far form: fixed prefix + u2 ofs_name_far at row+0x0a
# Layout: subtype u2, index_shift u2, id u4, 0x03 u1, ofs_name_near u1, ofs_name_far u2 -> name follows at offset 0x0e
far = 0x0E
row = b""
# index_shift patched by the pager (slot_index<<5); 0 placeholder.
row += pack("<HH", subtype, 0)
row += pack("<I", a.id)
row += pack("<B", 0x03)
row += pack("<B", 0)
row += pack("<H", far)
row += name
return _pad2(row)
def serialize_album(a: Album) -> bytes:
name = encode_devicesql(a.name)
near_ofs = 0x16
subtype = 0x80
row = b""
# index_shift patched by the pager (slot_index<<5); 0 placeholder.
row += pack("<HH", subtype, 0) # subtype, index_shift
row += pack("<I", 0) # unknown
row += pack("<I", a.artist_id) # artist_id
row += pack("<I", a.id) # id
row += pack("<I", 0) # unknown
row += pack("<B", 0x03) # always 0x03
row += pack("<B", near_ofs) # ofs_name_near
assert len(row) == near_ofs
row += name
return _pad2(row)
def serialize_playlist_tree(n: PlaylistNode) -> bytes:
row = b""
row += pack("<I", n.parent_id)
row += pack("<I", 0)
row += pack("<I", n.sort_order)
row += pack("<I", n.id)
row += pack("<I", 1 if n.is_folder else 0)
row += encode_devicesql(n.name)
return _pad2(row)
def serialize_playlist_entry(e: PlaylistEntry) -> bytes:
return pack("<III", e.entry_index, e.track_id, e.playlist_id)
def serialize_artwork(a: Artwork) -> bytes:
row = pack("<I", a.id) + encode_devicesql(a.path)
return _pad2(row)
def serialize_genre(g: Genre) -> bytes:
row = pack("<I", g.id) + encode_devicesql(g.name)
return _pad2(row)
def serialize_key(key_id: int, name: str, id2: int | None = None) -> bytes:
"""key_row: u4 id, u4 id2 (mirrors id in the reference), devicesql name."""
if id2 is None:
id2 = key_id
return _pad2(pack("<II", key_id, id2) + encode_devicesql(name))
def serialize_history_playlist(playlist_id: int, name: str) -> bytes:
"""history_playlist_row: u4 id, devicesql name."""
return _pad2(pack("<I", playlist_id) + encode_devicesql(name))
def serialize_history_entry(track_id: int, playlist_id: int, entry_index: int) -> bytes:
"""history_entry_row: u4 track_id, u4 playlist_id, u4 entry_index."""
return pack("<III", track_id, playlist_id, entry_index)
def serialize_history_blob(date_added: str) -> bytes:
"""The `history` table row format isn't decoded by the kaitai schema.
A 40-byte blob copied from the reference, with the date substituted.
"""
date_str = encode_devicesql(date_added) # short ASCII "2026-05-26"
bucket = encode_devicesql("1000")
empty = encode_devicesql("")
head = pack("<HHH", 0x0280, 0, 0xb0) + b"\x00" * 6
body = head + date_str + b"\x19\x1e" + bucket + empty
# pad to 40 bytes
if len(body) < 40:
body += b"\x00" * (40 - len(body))
return body[:40]
def serialize_color(c: Color) -> bytes:
# Schema: 5 bytes padding, u2 id, u1 (zero), then name string.
# The reference puts the low byte of id at byte 4 (last of the padding).
row = b"\x00\x00\x00\x00" + bytes([c.id & 0xFF]) + pack("<H", c.id & 0xFFFF) + b"\x00" + encode_devicesql(c.name)
return _pad2(row)
def serialize_column(c: Column) -> bytes:
"""A column row is: u2 id, u2 code, then a 'long' devicesql string whose
UTF-16 body is wrapped by the sentinel codepoints U+FFFA / U+FFFB.
"""
body = b"\xfa\xff" + c.name.encode("utf-16-le") + b"\xfb\xff"
total_length = len(body) + 4 # 4 = 0x90 marker + u2 length + u1 pad
name_str = b"\x90" + pack("<H", total_length) + b"\x00" + body
row = pack("<HH", c.id, c.code) + name_str
return _pad2(row)
+35
View File
@@ -0,0 +1,35 @@
"""DeviceSQL variable-length string encoding.
Three on-disk forms (see rekordbox-pdb.ksy `device_sql_string`):
- odd length_and_kind byte -> short ASCII (< 127 chars)
- 0x40 -> long ASCII (any length, ASCII bytes)
- 0x90 -> long UTF-16LE (for non-ASCII content)
We pick the cheapest form that fits the content. Long-but-ASCII paths
must use 0x40, NOT 0x90 -- some player firmwares decode 0x90 strings
as UTF-16 and then fail the case-sensitive filesystem lookup, so any
track whose `file_path` lands in the wrong form silently goes missing.
"""
EMPTY = b"\x03"
def encode_devicesql(s: str) -> bytes:
if not s:
return EMPTY
try:
ascii_bytes = s.encode("ascii")
except UnicodeEncodeError:
ascii_bytes = None
if ascii_bytes is not None:
if len(ascii_bytes) < 127:
length = len(ascii_bytes) + 1
length_and_kind = ((length << 1) | 1) & 0xFF
return bytes([length_and_kind]) + ascii_bytes
total_length = len(ascii_bytes) + 4 # marker + u2 length + u1 pad
return bytes([0x40]) + total_length.to_bytes(2, "little") + b"\x00" + ascii_bytes
payload = s.encode("utf-16-le")
total_length = len(payload) + 4
return bytes([0x90]) + total_length.to_bytes(2, "little") + b"\x00" + payload
+288
View File
@@ -0,0 +1,288 @@
"""Top-level rekordbox .pdb file writer."""
from pathlib import Path
from struct import pack, unpack_from
from loguru import logger
from maple.model import (
DEFAULT_COLUMNS,
STANDARD_COLORS,
UNKNOWN_17_ROWS,
UNKNOWN_18_ROWS,
Database,
validate_referential_integrity,
)
from maple.pdb.pager import PAGE_SIZE, PageBuilder, build_strange_page
from maple.pdb.rows import (
serialize_album,
serialize_artist,
serialize_artwork,
serialize_color,
serialize_column,
serialize_genre,
serialize_history_blob,
serialize_key,
serialize_playlist_entry,
serialize_playlist_tree,
serialize_track,
)
PAGE_TYPE_TRACKS = 0
PAGE_TYPE_GENRES = 1
PAGE_TYPE_ARTISTS = 2
PAGE_TYPE_ALBUMS = 3
PAGE_TYPE_LABELS = 4
PAGE_TYPE_KEYS = 5
PAGE_TYPE_COLORS = 6
PAGE_TYPE_PLAYLIST_TREE = 7
PAGE_TYPE_PLAYLIST_ENTRIES = 8
PAGE_TYPE_UNKNOWN_9 = 9
PAGE_TYPE_UNKNOWN_10 = 10
PAGE_TYPE_HISTORY_PLAYLISTS = 11
PAGE_TYPE_HISTORY_ENTRIES = 12
PAGE_TYPE_ARTWORK = 13
PAGE_TYPE_UNKNOWN_14 = 14
PAGE_TYPE_UNKNOWN_15 = 15
PAGE_TYPE_COLUMNS = 16
PAGE_TYPE_UNKNOWN_17 = 17
PAGE_TYPE_UNKNOWN_18 = 18
PAGE_TYPE_HISTORY = 19
ALL_PAGE_TYPES = list(range(20))
FILE_HEADER_SIZE = 28 # 0,page_size,num_tables,next_unused,unknown,sequence,gap = 7*u4
TABLE_ENTRY_SIZE = 16 # page_type,empty_candidate,first_page,last_page = 4*u4
# Page types whose populated data pages rekordbox tags 0x34 (vs the 0x24 the
# rest get). In a FRESH from-scratch export (data/acemo_fresh) only tracks and
# history carry 0x34; every other table's data page is 0x24. (The wider set
# {0,2,3,13,17,19} was matched against the EDITED reference, whose extra 0x34
# pages were edit-history artifacts.) These are the same two tables that get
# strange-page unknown7=1 -- see pager.STRANGE_UNKNOWN7.
PAGE_FLAG_34 = frozenset({0, 19}) # tracks, history
# Tables that get a real data page even when they have zero rows. A FRESH
# from-scratch rekordbox export (data/acemo_fresh) gives a data page to a table
# IFF it has rows -- every empty table is strange-page-only. The earlier
# non-empty CORE_TABLES set was matched against the EDITED reference
# (data/acemo), whose genres table (type 1) carried a leftover empty data page
# from a deleted genre row -- an edit-history artifact, not a format rule (see
# KNOWN_DIVERGENCE.md). Against the fresh reference the correct rule is
# "data page iff rows>0", i.e. no table is forced. Kept as an (empty) hook in
# case a future fresh export proves some table is always materialised.
CORE_TABLES: frozenset[int] = frozenset()
# Row types whose layout begins `subtype:u2, index_shift:u2`, so they carry a
# slot-based index_shift the pager must patch (FORMAT_SPEC.md s1): tracks,
# artists, albums.
INDEX_SHIFT_TABLES = frozenset({0, 2, 3})
SEQUENCE_OFFSET = 20 # u4 offset of `sequence` in the file header (page 0)
def _next_sequence(path: Path) -> int:
"""rekordbox's file-header `sequence` is an internal transaction counter from
its master.db -- we can't reproduce its exact value. The faithful behaviour is
to monotonically increment: read the sequence already in the destination
export (if any) and add 1, defaulting to 1 for a brand-new file. This keeps
each export's sequence strictly greater than the last, as rekordbox does.
"""
try:
head = path.read_bytes()[:FILE_HEADER_SIZE]
except (FileNotFoundError, OSError):
return 1
if len(head) < FILE_HEADER_SIZE:
return 1
existing = unpack_from("<I", head, SEQUENCE_OFFSET)[0]
return existing + 1 if existing else 1
def write_pdb(db: Database, path: str | Path) -> None:
logger.info(
"building pdb → {} (tracks={} artists={} albums={} genres={} "
"artwork={} playlists={} entries={})",
path,
len(db.tracks), len(db.artists), len(db.albums), len(db.genres),
len(db.artwork), len(db.playlist_nodes), len(db.playlist_entries),
)
# Refuse to emit a pdb with dangling foreign keys -- a track pointing at a
# genre/artist/album/artwork row that doesn't exist makes the player follow
# the FK into an empty table and freeze (freeze-class, like the deck hang).
fk_errors = validate_referential_integrity(db)
if fk_errors:
logger.error("FK validation failed with {} error(s)", len(fk_errors))
for e in fk_errors:
logger.error(" {}", e)
raise ValueError("dangling foreign keys (would freeze the player):\n " + "\n ".join(fk_errors))
# Layout (matching rekordbox): page 0 is the file header. Pages 1..2*N are
# reserved 2-per-table -- strange page at the odd index 1+2k, first data page
# at the even index 2+2k. Overflow data pages come from the pool at 2*N+1.
serialized_tracks = [serialize_track(t) for t in db.tracks]
serialized_artists = [serialize_artist(a) for a in db.artists]
serialized_albums = [serialize_album(a) for a in db.albums]
serialized_pl_tree = [serialize_playlist_tree(n) for n in db.playlist_nodes]
serialized_pl_entries = [serialize_playlist_entry(e) for e in db.playlist_entries]
serialized_artwork = [serialize_artwork(a) for a in db.artwork]
serialized_columns = [serialize_column(c) for c in DEFAULT_COLUMNS]
serialized_genres = [serialize_genre(g) for g in db.genres]
# If the caller didn't populate colors, always emit the standard palette
# so the player's Color filter has a non-empty value set.
colors = db.colors if db.colors else STANDARD_COLORS
serialized_colors = [serialize_color(c) for c in colors]
serialized_keys = [serialize_key(k.id, k.name, k.id2) for k in db.keys]
# unknown_17/unknown_18 row format isn't decoded by the kaitai schema;
# we emit the reference's byte-for-byte rows (four little-endian u2s).
serialized_unknown_17 = [pack("<HHHH", *r) for r in UNKNOWN_17_ROWS]
serialized_unknown_18 = [pack("<HHHH", *r) for r in UNKNOWN_18_ROWS]
# A real rekordbox export leaves history_playlists and history_entries
# EMPTY (only the `history` table itself carries a row). maple used to
# fabricate a dummy "HISTORY 001" playlist + entry referencing a track,
# which the player doesn't expect -- drop them and match the reference.
today = db.tracks[0].date_added if db.tracks else "2026-01-01"
serialized_history_playlists: list[bytes] = []
serialized_history_entries: list[bytes] = []
serialized_history = [serialize_history_blob(today)]
rows_by_type: dict[int, list[bytes]] = {
PAGE_TYPE_TRACKS: serialized_tracks,
PAGE_TYPE_GENRES: serialized_genres,
PAGE_TYPE_ARTISTS: serialized_artists,
PAGE_TYPE_ALBUMS: serialized_albums,
PAGE_TYPE_KEYS: serialized_keys,
PAGE_TYPE_COLORS: serialized_colors,
PAGE_TYPE_PLAYLIST_TREE: serialized_pl_tree,
PAGE_TYPE_PLAYLIST_ENTRIES: serialized_pl_entries,
PAGE_TYPE_ARTWORK: serialized_artwork,
PAGE_TYPE_COLUMNS: serialized_columns,
PAGE_TYPE_UNKNOWN_17: serialized_unknown_17,
PAGE_TYPE_UNKNOWN_18: serialized_unknown_18,
PAGE_TYPE_HISTORY_PLAYLISTS: serialized_history_playlists,
PAGE_TYPE_HISTORY_ENTRIES: serialized_history_entries,
PAGE_TYPE_HISTORY: serialized_history,
}
num_tables = len(ALL_PAGE_TYPES) # 20
overflow_next = 1 + 2 * num_tables # first overflow page (41)
pages_by_index: dict[int, bytes] = {} # page_index -> 4096-byte page
table_dir: list[tuple[int, int, int]] = [] # (page_type, first_page, last_page)
last_builders: list[PageBuilder] = [] # last data page of each chain, patched below
for k, ptype in enumerate(ALL_PAGE_TYPES):
strange_idx = 1 + 2 * k
data_idx = 2 + 2 * k
rows = rows_by_type.get(ptype, [])
data_flag = 0x34 if ptype in PAGE_FLAG_34 else 0x24
has_index_shift = ptype in INDEX_SHIFT_TABLES
# Optional table with no rows: strange page only (its even page stays
# blank). rekordbox still points the strange page at the reserved data
# index (strange_idx+1 == data_idx), and the table dir's last_page is the
# strange page itself.
if not rows and ptype not in CORE_TABLES:
pages_by_index[strange_idx] = build_strange_page(
strange_idx, ptype, next_page=data_idx, data_page=None)
table_dir.append((ptype, strange_idx, strange_idx))
continue
# First data page at the reserved even index; overflow pages from the pool.
builders: list[PageBuilder] = [
PageBuilder(data_idx, ptype, data_flag=data_flag, has_index_shift=has_index_shift)
]
for row in rows:
if not builders[-1].try_add(row):
nb = PageBuilder(overflow_next, ptype, data_flag=data_flag,
has_index_shift=has_index_shift)
overflow_next += 1
if not nb.try_add(row):
raise ValueError(f"row of size {len(row)} too large for an empty page")
builders.append(nb)
pages_by_index[strange_idx] = build_strange_page(
strange_idx, ptype, next_page=data_idx, data_page=data_idx)
for i, b in enumerate(builders):
# Intermediate pages chain forward to the next data page and are
# finalised now. The LAST page's next_page is patched after the loop
# to a free-pool sentinel (FREEZE FIX) -- never at itself -- so it is
# finalised there, not here.
if i != len(builders) - 1:
b.next_page = builders[i + 1].page_index
pages_by_index[b.page_index] = b.finalize()
last_builders.append(builders[-1])
table_dir.append((ptype, strange_idx, builders[-1].page_index))
# FREEZE FIX (#13): no data page may self-loop its next_page. In every
# genuine rekordbox export (data/MAPLE, lib_multi) the last page of each
# chain points FORWARD to a free/unallocated page; the firmware Page Manager
# walks next_page and treats an index at/after next_unused (incl. past EOF)
# as "end of chain", but an index == the current page is an infinite loop ->
# the deck froze. maple used to set last.next_page = its own index. Point
# every last page at a single dedicated free-pool sentinel just past the
# overflow region, materialise that page (zero-filled, as real exports do),
# and set next_unused beyond it.
free_sentinel = overflow_next
for b in last_builders:
b.next_page = free_sentinel
for b in last_builders:
pages_by_index[b.page_index] = b.finalize()
overflow_used = overflow_next - (1 + 2 * num_tables)
# Materialise a small free pool of zero-filled pages and set next_unused PAST
# it, mirroring a real export (e.g. data/MAPLE: 71 pages, next_unused=74 ->
# ~3 free pages past the last live page). A genuine export always keeps
# allocated-but-empty pages after the data and points next_unused beyond the
# file; maple previously had next_unused == EOF with no free pool, which can
# starve the firmware's free-list/allocator walk of a valid sentinel page.
FREE_POOL_PAGES = 4
for j in range(FREE_POOL_PAGES):
pages_by_index[free_sentinel + j] = b"\x00" * PAGE_SIZE
# File ends after the materialised free pool; next_unused points a couple
# pages PAST EOF, exactly as real exports do (data/MAPLE: file=71 pages,
# next_unused=74). The firmware tolerates a past-EOF next_unused.
file_page_count = free_sentinel + FREE_POOL_PAGES # number of pages incl. page 0
next_unused_page = file_page_count + 2
sequence = _next_sequence(Path(path))
logger.info(
"pdb layout: {} pages ({} reserved, {} overflow, {} free), sequence={}",
file_page_count, 2 * num_tables, overflow_used, FREE_POOL_PAGES, sequence,
)
file_header = b""
file_header += pack("<I", 0) # magic
file_header += pack("<I", PAGE_SIZE)
file_header += pack("<I", num_tables)
file_header += pack("<I", next_unused_page)
file_header += pack("<I", 0) # unknown
file_header += pack("<I", sequence) # incrementing transaction counter
file_header += pack("<I", 0) # gap
assert len(file_header) == FILE_HEADER_SIZE
tables_block = b""
for ptype, first, last in table_dir:
tables_block += pack("<I", ptype)
tables_block += pack("<I", 0) # empty_candidate
tables_block += pack("<I", first)
tables_block += pack("<I", last)
assert len(tables_block) == num_tables * TABLE_ENTRY_SIZE
header_full = file_header + tables_block
pad_to_page = PAGE_SIZE - len(header_full)
if pad_to_page < 0:
raise ValueError("too many tables for one page")
page0 = header_full + b"\x00" * pad_to_page
blank = b"\x00" * PAGE_SIZE
out_path = Path(path)
out_path.parent.mkdir(parents=True, exist_ok=True)
with out_path.open("wb") as f:
f.write(page0)
# Write pages 1..file_page_count-1 (reserved-but-unused written blank).
# next_unused_page intentionally points a couple pages PAST this EOF.
for idx in range(1, file_page_count):
page = pages_by_index.get(idx, blank)
assert len(page) == PAGE_SIZE
f.write(page)
logger.info("wrote {} ({} bytes)", out_path, out_path.stat().st_size)
View File
Whitespace-only changes.
+34
View File
@@ -0,0 +1,34 @@
"""Discover audio files and m3u8 playlists in an input directory."""
from pathlib import Path
AUDIO_EXTS = {".mp3", ".flac", ".wav", ".aiff", ".aif", ".m4a"}
def scan_audio(input_dir: Path) -> list[Path]:
out = []
for p in sorted(input_dir.rglob("*")):
if p.is_file() and p.suffix.lower() in AUDIO_EXTS:
out.append(p)
return out
def parse_m3u8(playlist_path: Path) -> list[Path]:
base = playlist_path.parent
entries = []
with playlist_path.open("r", encoding="utf-8", errors="replace") as f:
for line in f:
s = line.strip()
if not s or s.startswith("#"):
continue
p = Path(s)
if not p.is_absolute():
p = (base / p).resolve()
entries.append(p)
return entries
def scan_playlists(input_dir: Path) -> dict[str, list[Path]]:
out: dict[str, list[Path]] = {}
for p in sorted(input_dir.rglob("*.m3u8")):
out[p.stem] = parse_m3u8(p)
return out
+249
View File
@@ -0,0 +1,249 @@
"""Dump tracks from one or more Spotify playlists (Client Credentials flow).
just spotify # uses the placeholder ids below
just spotify "<url-or-id> ..." # explicit playlists
just spotify "--file lists.txt --json"
just spotify "<url-or-id> --csv" # single sockseek CSV (stdout)
just spotify "<url-or-id> --csv --out tracks.csv"
Playlists can be ids, full URLs, or spotify: URIs. Works for any public
playlist; no user login needed. Requires SPOTIFY_CLIENT_ID /
SPOTIFY_CLIENT_SECRET in the environment (loaded from .env by just).
--csv writes one combined CSV with Artist,Title,Album,Length columns (Length
in seconds), the format consumed by sockseek/sldl. Each playlist is preceded
by a `# <name>` comment line as a separator. Writes to stdout, or to a file
with --out <path>.
Networking is async via aiohttp: playlists are fetched concurrently.
"""
import asyncio
import base64
import csv
import json
import os
import re
import sys
import urllib.parse
import aiohttp
async def get_token(session: aiohttp.ClientSession, client_id: str, client_secret: str) -> str:
auth = base64.b64encode(f"{client_id}:{client_secret}".encode()).decode()
async with session.post(
"https://accounts.spotify.com/api/token",
data={"grant_type": "client_credentials"},
headers={"Authorization": f"Basic {auth}"},
) as resp:
resp.raise_for_status()
body = await resp.json()
return body["access_token"]
async def api_get(session: aiohttp.ClientSession, url: str, token: str) -> dict:
async with session.get(url, headers={"Authorization": f"Bearer {token}"}) as resp:
resp.raise_for_status()
return await resp.json()
def parse_playlist_id(arg: str) -> str:
m = re.search(r"playlist[/:]([A-Za-z0-9]+)", arg)
return m.group(1) if m else arg.strip()
def read_list_file(path: str) -> list[str]:
out = []
with open(path) as fh:
for line in fh:
line = line.split("#", 1)[0].strip()
if line:
out.append(line)
return out
def _largest_image(images: list[dict] | None) -> str | None:
if not images:
return None
# Spotify returns images sorted largest first, but defend against absent width.
ranked = sorted(images, key=lambda im: (im.get("width") or 0), reverse=True)
return ranked[0].get("url")
async def fetch_playlist(session: aiohttp.ClientSession, pid: str, token: str) -> dict:
meta = await api_get(
session,
f"https://api.spotify.com/v1/playlists/{pid}"
f"?fields={urllib.parse.quote('name,owner(display_name)')}",
token,
)
fields = (
"items(track(id,name,artists(id,name),album(name,images,release_date,release_date_precision),"
"duration_ms,external_ids(isrc),track_number,disc_number)),next"
)
url = (
f"https://api.spotify.com/v1/playlists/{pid}/tracks"
f"?limit=100&fields={urllib.parse.quote(fields)}"
)
tracks = []
while url:
page = await api_get(session, url, token)
for item in page.get("items", []):
t = item.get("track")
if not t:
continue
tracks.append(
{
"id": t.get("id"),
"title": t["name"],
"artists": [a["name"] for a in t["artists"]],
"artist_ids": [a.get("id") for a in t["artists"] if a.get("id")],
"album": t["album"]["name"],
"album_image_url": _largest_image(t["album"].get("images")),
"release_date": t["album"].get("release_date"),
"release_date_precision": t["album"].get("release_date_precision"),
"duration_ms": t["duration_ms"],
"isrc": (t.get("external_ids") or {}).get("isrc"),
"track_number": t.get("track_number"),
"disc_number": t.get("disc_number"),
"genres": [],
}
)
url = page.get("next")
return {"id": pid, "name": meta.get("name"), "tracks": tracks}
async def fetch_artist_genres(
session: aiohttp.ClientSession,
artist_ids: list[str],
token: str,
) -> dict[str, list[str]]:
"""Batch `/v1/artists?ids=...` (max 50 ids per call). Returns id → genres."""
out: dict[str, list[str]] = {}
seen: set[str] = set()
deduped = [a for a in artist_ids if a and not (a in seen or seen.add(a))]
for i in range(0, len(deduped), 50):
batch = deduped[i:i + 50]
body = await api_get(
session,
f"https://api.spotify.com/v1/artists?ids={','.join(batch)}",
token,
)
for a in body.get("artists", []) or []:
if a and a.get("id"):
out[a["id"]] = a.get("genres") or []
return out
def _enrich_track_genres(playlists: list[dict], genres_by_artist: dict[str, list[str]]) -> None:
"""Populate each track's `genres` field with the union (dedup'd, preserving order)
of its artists' genres."""
for pl in playlists:
for t in pl["tracks"]:
seen: set[str] = set()
union: list[str] = []
for aid in t.get("artist_ids") or []:
for g in genres_by_artist.get(aid, []):
if g not in seen:
seen.add(g)
union.append(g)
t["genres"] = union
def write_csv(playlists, fh):
"""Single combined sockseek/sldl CSV, one `# <name>` comment per playlist."""
w = csv.writer(fh)
w.writerow(["Artist", "Title", "Album", "Length"])
for pl in playlists:
fh.write(f"# {pl['name']}\n")
for t in pl["tracks"]:
w.writerow(
[
", ".join(t["artists"]),
t["title"],
t["album"],
round(t["duration_ms"] / 1000),
]
)
async def fetch_playlists(ids: list[str], with_genres: bool = True) -> list[dict]:
"""Resolve env, open a session, fetch every playlist concurrently.
When `with_genres` is true (default), follow up with a batched
`/v1/artists` call to populate each track's `genres` field.
"""
client_id = os.environ.get("MAPLE_SPOTIFY_CLIENT_ID") or os.environ.get("SPOTIFY_CLIENT_ID")
client_secret = os.environ.get("MAPLE_SPOTIFY_CLIENT_SECRET") or os.environ.get("SPOTIFY_CLIENT_SECRET")
if not (client_id and client_secret):
sys.exit("set MAPLE_SPOTIFY_CLIENT_ID and MAPLE_SPOTIFY_CLIENT_SECRET")
timeout = aiohttp.ClientTimeout(total=30)
async with aiohttp.ClientSession(timeout=timeout) as session:
token = await get_token(session, client_id, client_secret)
playlists = await asyncio.gather(
*(fetch_playlist(session, parse_playlist_id(x), token) for x in ids)
)
if with_genres:
artist_ids: list[str] = []
for pl in playlists:
for t in pl["tracks"]:
artist_ids.extend(t.get("artist_ids") or [])
genres = await fetch_artist_genres(session, artist_ids, token)
_enrich_track_genres(playlists, genres)
return playlists
async def main_async() -> None:
argv = sys.argv[1:]
as_json = "--json" in argv
as_csv = "--csv" in argv
argv = [a for a in argv if a not in ("--json", "--csv")]
inputs: list[str] = []
out_path: str | None = None
i = 0
while i < len(argv):
if argv[i] == "--file":
inputs += read_list_file(argv[i + 1])
i += 2
elif argv[i] == "--out":
out_path = argv[i + 1]
i += 2
else:
inputs.append(argv[i])
i += 1
if not inputs:
sys.exit(
"usage: spotify.py [--json|--csv] <url-or-id ...> "
"[--file list.txt] [--out path]"
)
playlists = await fetch_playlists(inputs)
if as_json:
print(json.dumps(playlists, indent=2, ensure_ascii=False))
return
if as_csv:
if out_path:
with open(out_path, "w", newline="") as fh:
write_csv(playlists, fh)
print(f"wrote {sum(len(p['tracks']) for p in playlists)} tracks to {out_path}")
else:
write_csv(playlists, sys.stdout)
return
for pl in playlists:
print(f"\n=== {pl['name']} ({len(pl['tracks'])} tracks) ===")
for n, t in enumerate(pl["tracks"], 1):
print(f"{n:>3}. {', '.join(t['artists'])} - {t['title']}")
def main() -> None:
asyncio.run(main_async())
if __name__ == "__main__":
main()
+345
View File
@@ -0,0 +1,345 @@
"""Canonical `Track` — the unit shared by `just collection` and `just export`.
A *collection* is a sequence of `Track`s persisted under
`data/collection/`. An *export* reads a sequence of `Track`s and writes a
rekordbox USB at `data/usb/`. This dataclass is the bridge: any code that
talks about "a track" in maple should hold one of these.
`Track.validate()` is the project's single source of truth for what makes
a track "exportable". Every check that used to be scattered across
`maple/validate.py` lives here now; both pipelines call `track.validate()`.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import TYPE_CHECKING
from maple import model
if TYPE_CHECKING:
from maple.analyze import Analysis
@dataclass
class TrackIssue:
"""A single failed validation criterion, bound to the `Track` it
came from. Repr-friendly via `__str__`."""
track: "Track"
field: str
reason: str
def __str__(self) -> str:
return f"{self.track.display_name}: {self.field}: {self.reason}"
@dataclass
class Track:
"""Canonical track metadata. Every field has a concrete default so the
type is always populated — "missing" data shows up as `""`/`0`/`False`
and is caught by `validate()`, not by Optional-handling at every call
site."""
# ---- identity --------------------------------------------------------- #
isrc: str = ""
title: str = ""
artists: list[str] = field(default_factory=list)
album: str = ""
# ---- Spotify-derived ------------------------------------------------- #
spotify_track_id: str = ""
spotify_artist_ids: list[str] = field(default_factory=list)
album_image_url: str = ""
release_date: str = "" # "YYYY", "YYYY-MM", or "YYYY-MM-DD"
release_date_precision: str = ""
genres: list[str] = field(default_factory=list)
track_number: int = 0
disc_number: int = 0
duration_ms: int = 0
# ---- on-disk audio --------------------------------------------------- #
# `path` is the only legitimately-Optional field — a Track may exist
# purely as a Spotify entry before the audio has been downloaded.
path: Path | None = None
file_size: int = 0
bitrate: int = 0 # kbps
sample_rate: int = 0 # Hz
sample_depth: int = 0 # bits per sample
has_artwork: bool = False # embedded cover-art presence
has_cached_analysis: bool = False # collection/_analysis/<isrc>.json exists
# ---- analytical ------------------------------------------------------ #
bpm: float = 0.0 # 0.0 = unknown
# ---- provenance ------------------------------------------------------ #
source: str = "" # "spotify", "local", "bandcamp", …
# ---- views ----------------------------------------------------------- #
@property
def primary_artist(self) -> str:
return self.artists[0] if self.artists else ""
@property
def display_name(self) -> str:
if not self.title:
return self.path.name if self.path else "<unknown>"
if self.artists:
return f"{', '.join(self.artists)} - {self.title}"
return self.title
@property
def primary_genre(self) -> str:
return self.genres[0] if self.genres else ""
@property
def year(self) -> int:
"""First 4 digits of release_date as an int (0 if absent/invalid)."""
if not self.release_date or len(self.release_date) < 4:
return 0
try:
return int(self.release_date[:4])
except ValueError:
return 0
@property
def duration_s(self) -> int:
return int(round(self.duration_ms / 1000)) if self.duration_ms else 0
# ---- ingress --------------------------------------------------------- #
@classmethod
def from_spotify(cls, d: dict) -> "Track":
"""Build from one row of `maple.spotify.fetch_playlist`'s output."""
return cls(
isrc=(d.get("isrc") or "").strip(),
title=d.get("title") or "",
artists=list(d.get("artists") or []),
album=d.get("album") or "",
spotify_track_id=d.get("id") or "",
spotify_artist_ids=list(d.get("artist_ids") or []),
album_image_url=d.get("album_image_url") or "",
release_date=d.get("release_date") or "",
release_date_precision=d.get("release_date_precision") or "",
genres=list(d.get("genres") or []),
track_number=int(d.get("track_number") or 0),
disc_number=int(d.get("disc_number") or 0),
duration_ms=int(d.get("duration_ms") or 0),
source="spotify",
)
@classmethod
def from_file(cls, path: Path, *, analysis_cache_dir: Path | None = None) -> "Track":
"""Construct from on-disk audio via mutagen.
When `analysis_cache_dir` is provided, also sets
`has_cached_analysis` from `<cache_dir>/<isrc>.json` so
`.validate()` can check it without a separate call site.
"""
from mutagen import File as MutagenFile
out = cls(path=path, source="local")
try:
audio = MutagenFile(path, easy=True)
except Exception:
return out
if audio is None:
return out
tags = audio.tags or {}
def first(k: str) -> str:
v = tags.get(k) if hasattr(tags, "get") else None
return (v[0] if v else "") if isinstance(v, list) else (v or "")
out.title = first("title")
artist = first("artist")
if artist:
out.artists = [a.strip() for a in artist.split(",") if a.strip()]
out.album = first("album")
out.isrc = first("isrc").strip()
out.release_date = first("date")
gtag = first("genre")
if gtag:
from maple.cli import split_genre_tags
out.genres = split_genre_tags(gtag)
try:
out.bpm = float(first("bpm")) if first("bpm") else 0.0
except ValueError:
out.bpm = 0.0
try:
out.track_number = int(first("tracknumber").split("/")[0])
except ValueError:
out.track_number = 0
info = getattr(audio, "info", None)
if info is not None:
out.duration_ms = int(round((getattr(info, "length", 0.0) or 0.0) * 1000))
bps = int(getattr(info, "bitrate", 0) or 0)
out.bitrate = bps // 1000 if bps > 1000 else bps
out.sample_rate = int(getattr(info, "sample_rate", 0) or 0)
out.sample_depth = int(getattr(info, "bits_per_sample", 0) or 0)
try:
out.file_size = path.stat().st_size
except OSError:
pass
# Presence flags so validate() can decide without extra plumbing.
from maple.collection.validate import has_embedded_art
out.has_artwork = has_embedded_art(path)
if analysis_cache_dir is not None and out.isrc:
from maple.analyze import cache_path_for
cp = cache_path_for(analysis_cache_dir, out.isrc)
out.has_cached_analysis = cp is not None and cp.exists()
return out
# ---- enrichment ------------------------------------------------------ #
def merge_spotify(self, d: dict) -> "Track":
"""Overlay Spotify-canonical fields onto this Track in place. Empty
Spotify fields are NOT applied (no clobbering)."""
if d.get("title"):
self.title = d["title"]
if d.get("artists"):
self.artists = list(d["artists"])
if d.get("album"):
self.album = d["album"]
if d.get("isrc"):
self.isrc = d["isrc"].strip()
if d.get("id"):
self.spotify_track_id = d["id"]
if d.get("artist_ids"):
self.spotify_artist_ids = list(d["artist_ids"])
if d.get("album_image_url"):
self.album_image_url = d["album_image_url"]
if d.get("release_date"):
self.release_date = d["release_date"]
if d.get("release_date_precision"):
self.release_date_precision = d["release_date_precision"]
if d.get("genres"):
self.genres = list(d["genres"])
if d.get("duration_ms"):
self.duration_ms = int(d["duration_ms"])
if d.get("track_number"):
self.track_number = int(d["track_number"])
if d.get("disc_number"):
self.disc_number = int(d["disc_number"])
if not self.source:
self.source = "spotify"
return self
def merge_analysis(self, analysis: "Analysis") -> "Track":
"""Merge fields from a `maple.analyze.Analysis`."""
if analysis.bpm:
self.bpm = float(analysis.bpm)
return self
# ---- validation ------------------------------------------------------ #
def validate(self) -> list[TrackIssue]:
"""Return the list of failed criteria. Empty list = ready to export.
The single place every "is this track exportable?" check lives.
Both `just collection` and `just export` call this — there is no
separate validation entrypoint.
"""
issues: list[TrackIssue] = []
def fail(field_name: str, reason: str) -> None:
issues.append(TrackIssue(self, field_name, reason))
# Required text tags — the spotify-canonical set.
if not self.title:
fail("title", "tag missing")
if not self.artists:
fail("artist", "tag missing")
if not self.album:
fail("album", "tag missing")
if not self.release_date:
fail("date", "tag missing")
if not self.genres:
fail("genre", "tag missing")
if not self.isrc:
fail("isrc", "tag missing")
# Analytical.
if self.bpm <= 0:
fail("bpm", "tag missing")
# On-disk.
if not self.has_artwork:
fail("artwork", "no embedded cover art")
if not self.has_cached_analysis:
fail("analysis", "no cached analysis")
# Sanity — only check when we have something to check.
if self.path is not None and self.duration_ms <= 0:
fail("duration", "≤ 0s — corrupt file or unreadable container")
return issues
def compare(self, canonical: "Track", *, fuzzy_threshold: float = 0.5) -> list[TrackIssue]:
"""Diff this Track against a canonical one (typically a Spotify
`from_spotify` Track). Returns mismatch issues — empty when the
two agree on every populated field."""
from maple.collection.index import first_artist, fuzzy_ratio, norm, norm_loose
out: list[TrackIssue] = []
def diff(field_name: str, sv: object, fv: object) -> None:
out.append(TrackIssue(self, field_name, f"spotify={sv!r} file={fv!r}"))
sp_artist = ", ".join(canonical.artists)
f_artist = ", ".join(self.artists)
if sp_artist and fuzzy_ratio(norm(first_artist(sp_artist)), norm(first_artist(f_artist))) < fuzzy_threshold:
diff("artist", sp_artist, f_artist)
if canonical.title and fuzzy_ratio(norm_loose(canonical.title), norm_loose(self.title)) < fuzzy_threshold:
diff("title", canonical.title, self.title)
if canonical.album and self.album and fuzzy_ratio(norm_loose(canonical.album), norm_loose(self.album)) < fuzzy_threshold:
diff("album", canonical.album, self.album)
elif canonical.album and not self.album:
diff("album", canonical.album, self.album)
if canonical.isrc and self.isrc and canonical.isrc.upper() != self.isrc.upper():
diff("isrc", canonical.isrc, self.isrc)
if canonical.release_date and self.release_date and canonical.release_date[:4] != self.release_date[:4]:
diff("date", canonical.release_date, self.release_date)
if canonical.duration_ms and self.duration_ms:
delta = abs(canonical.duration_ms - self.duration_ms) / 1000.0
if delta > 5.0:
diff("duration", f"{canonical.duration_s}s", f"{self.duration_s}s")
return out
# ---- export adapter -------------------------------------------------- #
def to_pdb_row(
self,
*,
track_id: int,
artist_id: int,
album_id: int,
genre_id: int = 0,
artwork_id: int = 0,
file_path: str,
analyze_path: str = "",
date_added: str = "",
tempo_x100: int | None = None,
) -> model.Track:
"""Project this canonical Track into the denormalised pdb row shape."""
if tempo_x100 is None:
tempo_x100 = round(self.bpm * 100) if self.bpm else 0
return model.Track(
id=track_id,
title=self.title,
artist_id=artist_id,
album_id=album_id,
filename=Path(file_path).name,
file_path=file_path,
file_size=self.file_size,
sample_rate=self.sample_rate or 44100,
sample_depth=self.sample_depth or 16,
bitrate=self.bitrate,
duration=self.duration_s,
track_number=self.track_number,
year=self.year,
disc_number=self.disc_number,
tempo=tempo_x100,
release_date=self.release_date,
date_added=date_added,
analyze_path=analyze_path,
analyze_date=date_added,
artwork_id=artwork_id,
genre_id=genre_id,
source_path=self.path,
)
+47
View File
@@ -0,0 +1,47 @@
"""Thin compatibility layer over `Track.validate()`.
Validation logic lives on `maple.track.Track` itself — this module just
re-exports `TrackIssue` and offers helpers so callers don't all need to
construct `Track` objects inline.
The `validate_track(path, ...)` wrapper is kept for the few sites that
hand around `Path`s rather than `Track`s; new code should prefer
`Track.from_file(path).validate()`.
"""
from __future__ import annotations
from pathlib import Path
from loguru import logger
from maple.track import Track, TrackIssue
__all__ = ["Track", "TrackIssue", "validate_track", "log_issues"]
def validate_track(
path: Path,
*,
spotify_track: dict | None = None,
analysis_cache_dir: Path | None = None,
playlist: str = "",
) -> list[TrackIssue]:
"""Construct a `Track` from `path` and run its validator.
`playlist` is unused — kept in the signature for API compatibility
with callers that still pass it.
"""
del playlist # not used; TrackIssue carries the track now
track = Track.from_file(path, analysis_cache_dir=analysis_cache_dir)
issues = track.validate()
if spotify_track is not None:
issues.extend(track.compare(Track.from_spotify(spotify_track)))
return issues
def log_issues(issues: list[TrackIssue], *, report: bool = True) -> None:
"""Emit every issue via `loguru.warning`. With `report=True`, also
binds `extra["report"]=True` so a report-file sink can capture them."""
bound = logger.bind(report=True) if report else logger
for i in issues:
bound.warning("{}", i)
+73
View File
@@ -0,0 +1,73 @@
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.14"
# dependencies = []
# ///
"""Thin binding around the `sockseek` CLI.
Creates the data directories and invokes sockseek with the CLI-argument
equivalents of etc/sldl.tpl.conf, so the two stay in sync without rendering a
config file. Reads SOCKSEEK_DATA / SOCKSEEK_USER / SOCKSEEK_PASS from the
environment (loaded from .env by just).
just sockseek <input> # csv path, url, search string, ...
just sockseek "tracks.csv -a" # extra flags pass through / override
Any args after the input are forwarded to sockseek verbatim, placed after the
defaults so they win on conflict.
"""
import os
import subprocess
import sys
from pathlib import Path
# Mirror of etc/sldl.tpl.conf. Keep these in sync with that file.
NAME_FORMAT = "{artist(/)album(/)track( )title|(_missing_tags/)slsk-foldername(/)slsk-filename}"
def env(key):
val = os.environ.get(key)
if not val:
sys.exit(f"missing env var: {key}")
return val
def config_args(data, user, password):
"""CLI equivalents of etc/sldl.tpl.conf (no-progress has no sockseek flag)."""
return [
"--path", data,
"--index-path", f"{data}/_metadata/index.sldl.csv",
"--user", user,
"--pass", password,
"--name-format", NAME_FORMAT,
"--pref-format", "flac,mp3",
"--pref-min-bitrate", "200",
"--pref-max-bitrate", "2500",
"--strict-conditions",
"--fast-search",
"--concurrent-jobs", "4",
]
def main():
if not sys.argv[1:]:
sys.exit("usage: sockseek.py <input> [extra sockseek args...]")
inp, *extra = sys.argv[1:]
data = env("SOCKSEEK_DATA")
user = env("SOCKSEEK_USER")
password = env("SOCKSEEK_PASS")
Path(data, "_metadata").mkdir(parents=True, exist_ok=True)
args = config_args(data, user, password)
cmd = ["sockseek", inp, *args, *extra]
masked = ["***" if a == password else a for a in cmd]
print(" ".join(masked), file=sys.stderr)
raise SystemExit(subprocess.run(cmd).returncode)
if __name__ == "__main__":
main()
+30
View File
@@ -0,0 +1,30 @@
from maple.pdb.strings import EMPTY, encode_devicesql
def test_empty():
assert encode_devicesql("") == EMPTY
def test_short_ascii():
out = encode_devicesql("ON")
# length=3 -> length_and_kind = (3<<1)|1 = 7
assert out == bytes([7]) + b"ON"
def test_longer_ascii():
s = "Hello, World!"
out = encode_devicesql(s)
length = len(s) + 1
assert out[0] == ((length << 1) | 1)
assert out[1:] == s.encode("ascii")
def test_utf16():
s = "ЛаBа"
out = encode_devicesql(s)
assert out[0] == 0x90
# Total length stored = payload_len + 4
payload = s.encode("utf-16-le")
assert out[1:3] == (len(payload) + 4).to_bytes(2, "little")
assert out[3] == 0
assert out[4:] == payload
+29
View File
@@ -0,0 +1,29 @@
from pathlib import Path
from maple.model import Album, Artist, Database, PlaylistEntry, PlaylistNode, Track
from maple.pdb.writer import write_pdb
def test_smoke(tmp_path: Path):
db = Database()
db.artists.append(Artist(id=1, name="AceMo"))
db.albums.append(Album(id=1, name="All My Life", artist_id=1))
db.tracks.append(Track(
id=1, title="Where They At", artist_id=1, album_id=1,
filename="ace.mp3", file_path="/Contents/AceMo/All My Life/ace.mp3",
file_size=14547692, sample_rate=44100, sample_depth=16, bitrate=320,
duration=436, track_number=7, year=2019, date_added="2025-01-04",
))
db.playlist_nodes.append(PlaylistNode(id=1, parent_id=0, sort_order=1, name="MyList"))
db.playlist_entries.append(PlaylistEntry(playlist_id=1, track_id=1, entry_index=1))
out = tmp_path / "export.pdb"
write_pdb(db, out)
assert out.exists()
data = out.read_bytes()
assert len(data) % 4096 == 0
assert data[:4] == b"\x00\x00\x00\x00"
# page_size at offset 4
assert int.from_bytes(data[4:8], "little") == 4096
# num_tables at offset 8 -- rekordbox always emits all 20 page-type tables
assert int.from_bytes(data[8:12], "little") == 20
+80
View File
@@ -0,0 +1,80 @@
"""Build a maple export.pdb from the SAME tracks as the real rekordbox AceMo
export, so the two can be diffed and the maple pdb tested on a player against
the audio/USBANLZ already on the USB.
Reads metadata out of data/acemo/PIONEER/rekordbox/export.pdb (the real export)
and reconstructs an identical-content maple Database, then writes
captures/maple_acemo.pdb.
"""
import sys
from io import BytesIO
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "src"))
from kaitaistruct import KaitaiStream
from kaitai.rekordbox_pdb import RekordboxPdb
from maple.model import Album, Artist, Artwork, Database, Key, Track
from maple.pdb.writer import write_pdb
REAL = ROOT / "data/acemo/PIONEER/rekordbox/export.pdb"
OUT = ROOT / "captures/maple_acemo.pdb"
PT = {t.value: t.name for t in RekordboxPdb.PageType}
def _rows(tbl):
out, ref = [], tbl.first_page
while True:
pg = ref.body
if pg.is_data_page:
for rg in pg.row_groups:
for r in rg.rows:
if r.present:
out.append(r.body)
if ref.index == tbl.last_page.index:
break
ref = pg.next_page
return out
def _txt(v):
try:
return v.body.text
except Exception:
return ""
def main():
pdb = RekordboxPdb(is_ext=False, _io=KaitaiStream(BytesIO(REAL.read_bytes())))
by = {PT.get(t.type.value): _rows(t) for t in pdb.tables}
db = Database()
for a in by.get("artists", []):
db.artists.append(Artist(id=a.id, name=_txt(a.name)))
for a in by.get("albums", []):
db.albums.append(Album(id=a.id, name=_txt(a.name), artist_id=a.artist_id))
for k in by.get("keys", []):
db.keys.append(Key(id=k.id, name=_txt(k.name), id2=k.id2))
for a in by.get("artwork", []):
db.artwork.append(Artwork(id=a.id, path=_txt(a.path)))
for t in by.get("tracks", []):
db.tracks.append(Track(
id=t.id, title=_txt(t.title), artist_id=t.artist_id, album_id=t.album_id,
filename=_txt(t.filename), file_path=_txt(t.file_path), file_size=t.file_size,
sample_rate=t.sample_rate, sample_depth=t.sample_depth, bitrate=t.bitrate,
duration=t.duration, track_number=t.track_number, year=t.year,
disc_number=t.disc_number, tempo=t.tempo, comment=_txt(t.comment),
date_added=_txt(t.date_added), release_date=_txt(t.release_date),
mix_name=_txt(t.mix_name), analyze_path=_txt(t.analyze_path),
analyze_date=_txt(t.analyze_date), artwork_id=t.artwork_id, genre_id=t.genre_id))
write_pdb(db, OUT)
sz = OUT.stat().st_size
print(f"wrote {OUT} ({sz} bytes, page-mult={sz % 4096 == 0}) "
f"tracks={len(db.tracks)} artists={len(db.artists)} albums={len(db.albums)} "
f"keys={len(db.keys)} artwork={len(db.artwork)}")
if __name__ == "__main__":
main()
+104
View File
@@ -0,0 +1,104 @@
// Frida host (Node ESM) for the pdb-capture agent.
// - Attaches the agent (rbx_pdb_capture.js) to rekordbox processes.
// - Streams ALL their output to THIS console (agent lines are [pid]-tagged).
// - STAYS ALIVE via the Node event loop -- no frida-CLI REPL/EOF problem.
// - Uses CHILD-GATING (Windows-supported; spawn-gating is not) so children of a
// gated process are instrumented AT BIRTH. This is how we reach the
// rekordboxAgent that writes export.pdb: it's CIG-protected and refuses
// post-hoc injection, but as a freshly-spawned child it can be hooked.
// - Clears the captures/ folder on start.
//
// Setup (Windows, repo root, once): npm i frida
// Run:
// node tools\capture.mjs --spawn (RECOMMENDED: launch rekordbox under the
// gate so every child incl. the DB agent is
// hooked at birth. Close rekordbox first.)
// node tools\capture.mjs (attach to running procs + gate their
// future children; restart rekordbox to
// respawn the agent as a gated child.)
import * as fridaNs from "frida";
import fs from "node:fs";
import path from "node:path";
import { fileURLToPath } from "node:url";
const frida = fridaNs.default ?? fridaNs;
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const REKORDBOX = "C:\\Program Files\\rekordbox\\rekordbox 7.0.5\\rekordbox.exe";
const AGENT = fs.readFileSync(path.join(__dirname, "rbx_pdb_capture.js"), "utf8");
const CAP = path.join(__dirname, "..", "captures");
const SPAWN = process.argv.includes("--spawn");
function clearCaptures() {
fs.mkdirSync(CAP, { recursive: true });
let n = 0;
for (const f of fs.readdirSync(CAP)) {
try { fs.rmSync(path.join(CAP, f), { force: true, recursive: true }); n++; } catch (e) {}
}
console.log(`[host] cleared ${n} file(s) from ${CAP}`);
}
function isRbx(name) { return /rekordbox/i.test(name || ""); }
// Attach, load the agent, and gate this process's future children too.
async function instrument(device, pid, label) {
try {
const session = await device.attach(pid);
session.detached.connect(reason => console.log(`[host] detached ${label} (${pid}): ${reason}`));
const script = await session.createScript(AGENT);
script.logHandler = (level, text) => console.log(text);
script.message.connect(msg => {
if (msg.type === "error")
console.error(`[host] ${label} (${pid}) script error:`, msg.stack || msg.description || msg);
});
await script.load();
try { await session.enableChildGating(); } catch (e) { /* not always available */ }
console.log(`[host] instrumented ${label} (${pid})`);
return session;
} catch (e) {
console.error(`[host] FAILED to instrument ${label} (${pid}): ${e.message}`);
return null;
}
}
async function main() {
clearCaptures();
const device = await frida.getLocalDevice();
// Children of any gated process are suspended at birth -> instrument, resume.
device.childAdded.connect(async child => {
const name = child.identifier || child.path || String(child.pid);
try {
console.log(`[host] child spawned: ${name} (${child.pid})`);
if (isRbx(name)) await instrument(device, child.pid, "child:" + name);
} catch (e) {
console.error(`[host] child handler error (${child.pid}): ${e.message}`);
} finally {
try { await device.resume(child.pid); } catch (_) {}
}
});
if (SPAWN) {
console.log(`[host] spawning ${REKORDBOX}`);
const pid = await device.spawn([REKORDBOX]);
await instrument(device, pid, "main"); // enables child gating on main
await device.resume(pid);
console.log(`[host] resumed main rekordbox (${pid}); children caught at birth`);
} else {
for (const p of await device.enumerateProcesses()) {
if (isRbx(p.name)) await instrument(device, p.pid, p.name);
}
console.log("[host] attached to running rekordbox processes (+child gating).");
console.log("[host] >>> the export.pdb writer (rekordboxAgent) refuses live injection.");
console.log("[host] >>> RESTART rekordbox -- the new agent spawns as a GATED CHILD and");
console.log("[host] >>> gets hooked at birth. Then Export and watch for [SAVED] ...export.pdb");
console.log("[host] >>> (or re-run with --spawn to launch rekordbox under the gate.)");
}
console.log("[host] ready. Ctrl-C to stop.");
process.stdin.resume();
process.on("SIGINT", () => { console.log("\n[host] stopping."); process.exit(0); });
}
main().catch(e => { console.error("[host] fatal:", e); process.exit(1); });
+184
View File
@@ -0,0 +1,184 @@
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.14"
# dependencies = ["kaitaistruct>=0.11", "pyrekordbox>=0.4.4"]
# ///
"""Decode a rekordbox USB export so we can spot how the analyze-path
bucket/id is derived from track metadata.
Reads `export.pdb` from a rekordbox-built USB stick (or `data/q1-rekordbox`)
plus optionally cross-references the rekordbox machine's `master.db` to
correlate Mac-internal UUIDs with the USB-export short hex ids.
Usage:
uv run tools/decode_rb_export.py <usb-root> [<master.db>]
`<usb-root>` is the directory holding `PIONEER/rekordbox/export.pdb`.
With `<master.db>` we also list the UUID + ContentLink for each track so
the per-track UUID→USB-id derivation can be eye-balled across rows.
"""
import re
import sys
from pathlib import Path
# Make the in-project kaitai schemas importable without an editable install.
HERE = Path(__file__).resolve().parent
sys.path.insert(0, str(HERE.parent / "src"))
from kaitai.rekordbox_pdb import RekordboxPdb # noqa: E402
from kaitaistruct import KaitaiStream # noqa: E402
ANLZ_PATH_RE = re.compile(r"/PIONEER/USBANLZ/([^/]+)/([^/]+)/ANLZ0000\.DAT")
def iter_rows(pdb: RekordboxPdb, page_type):
"""Walk a table's page list, yielding every present row body.
Tolerates the page chain pointing past EOF (real exports do this on
the last page — `next_page.index >= num_pages` is a sentinel)."""
for tref in pdb.tables:
if tref.type != page_type:
continue
pref = tref.first_page
seen = set()
while pref is not None and pref.index not in seen:
seen.add(pref.index)
try:
page = pref.body
except Exception:
break # past-EOF next_page (free-pool sentinel)
if page.is_data_page:
for rg in page.row_groups:
for rref in rg.rows:
if rref.present:
yield rref.body
try:
nxt = page.next_page
except Exception:
break
if nxt is None or nxt.index == 0 or nxt.index == pref.index:
break
pref = nxt
def read_pdb_tracks(pdb_path: Path) -> list[dict]:
"""Return one dict per track row: id, title, file_path, analyze_path,
plus every opaque/seed field plus parsed bucket+anlz_id."""
with pdb_path.open("rb") as f:
pdb = RekordboxPdb(False, KaitaiStream(f))
tracks = []
for r in iter_rows(pdb, RekordboxPdb.PageType.tracks):
ap = r.analyze_path.body.text
m = ANLZ_PATH_RE.match(ap)
bucket = anlz_id = ""
if m:
bucket, anlz_id = m.group(1), m.group(2)
tracks.append({
"track_id": r.id,
"title": r.title.body.text,
"file_path": r.file_path.body.text,
"analyze_path": ap,
"bucket": bucket,
"anlz_id": anlz_id,
"opaque_track_id_lo": r.opaque_track_id_lo,
"opaque_seed_lo": r.opaque_seed_lo,
"opaque_seed_hi": r.opaque_seed_hi,
"tempo": r.tempo,
"duration": r.duration,
"file_size": r.file_size,
"bitmask": r.bitmask,
})
return tracks
def maybe_attach_master_db(tracks: list[dict], master_db_path: Path) -> None:
"""Look up each track in master.db by file path and attach UUID/etc."""
try:
from pyrekordbox import Rekordbox6Database
from pyrekordbox.db6 import tables as t
except ImportError as e:
print(f"(pyrekordbox unavailable: {e}; skipping master.db lookup)", file=sys.stderr)
return
db = Rekordbox6Database(path=str(master_db_path))
for tr in tracks:
# Look up by file name as a fallback — full FolderPath may differ if
# the rekordbox machine mounts the source from a different prefix.
title = tr["title"]
rows = db.session.query(t.DjmdContent).filter(t.DjmdContent.Title == title).limit(3).all()
if not rows:
tr["rb_uuid"] = ""
tr["rb_content_link"] = 0
tr["rb_file_id"] = ""
tr["rb_master_song_id"] = ""
continue
# Prefer one whose FolderPath ends with the same file name
fname = Path(tr["file_path"]).name
match = next((r for r in rows if (r.FolderPath or "").endswith(fname)), rows[0])
tr["rb_uuid"] = match.UUID or ""
tr["rb_content_link"] = match.ContentLink or 0
tr["rb_file_id"] = match.rb_file_id or ""
tr["rb_master_song_id"] = match.MasterSongID or ""
def fmt_int(v, w=10):
if not isinstance(v, int):
return f"{str(v):<{w}}"
return f"{v:>{w-2}d}=0x{v:08x}"[-w:].rjust(w)
def main(argv: list[str]) -> int:
if len(argv) < 1:
sys.exit(__doc__)
usb_root = Path(argv[0]).expanduser()
pdb_path = usb_root / "PIONEER" / "rekordbox" / "export.pdb"
if not pdb_path.exists():
sys.exit(f"export.pdb not found at {pdb_path}")
tracks = read_pdb_tracks(pdb_path)
print(f"# Read {len(tracks)} tracks from {pdb_path}")
if len(argv) >= 2:
master_db = Path(argv[1]).expanduser()
if master_db.exists():
maybe_attach_master_db(tracks, master_db)
print(f"# Cross-referenced against {master_db}")
else:
print(f"# (master.db at {master_db} not found, skipping)")
# Output: one TSV row per track plus a summary at the end.
cols = ["track_id", "bucket", "anlz_id", "opaque_track_id_lo",
"opaque_seed_lo", "opaque_seed_hi",
"rb_uuid", "rb_content_link", "rb_file_id", "rb_master_song_id",
"title"]
print()
print("\t".join(cols))
for tr in tracks:
row = []
for c in cols:
v = tr.get(c, "")
if isinstance(v, int):
row.append(f"{v}=0x{v:x}" if v > 255 else str(v))
else:
row.append(str(v)[:64])
print("\t".join(row))
# Quick correlation tests: print numeric distances + ratios so a
# multi-track export shows obvious patterns at a glance.
if len(tracks) >= 2:
print("\n# Successive anlz_id deltas (probe for sequential counter):")
ids = []
for tr in tracks:
try:
ids.append(int(tr["anlz_id"], 16))
except (ValueError, TypeError):
pass
ids.sort()
for a, b in zip(ids, ids[1:]):
print(f" {a:08x} → {b:08x} Δ = {b-a}")
print(f"\n# anlz_id range: {min(ids):08x} .. {max(ids):08x}")
print(f"# bucket set: {sorted({tr['bucket'] for tr in tracks if tr['bucket']})}")
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv[1:]))
+209
View File
@@ -0,0 +1,209 @@
// Capture rekordbox's .pdb writes with Frida (run on the WINDOWS host).
//
// Hooks at the ntdll layer (NtCreateFile / NtWriteFile / NtClose) so it catches
// file IO no matter whether the app calls kernel32, kernelbase, or the CRT.
// Logs EVERY file opened for writing (so you can see temp names / the real
// filenames even if the .pdb filter is wrong), reconstructs files whose path
// matches SAVE_RE byte-for-byte, and saves them to OUT_DIR via Frida's File API.
// Prints a backtrace the first time each saved file is written -> the
// module+offset of the serializer to open in Ghidra.
//
// Usage (Windows PowerShell, rekordbox running):
// mkdir C:\Users\ntr\code\maple\maple-py\captures # OUT_DIR must exist
// frida-ps | Select-String rekordbox # find all PIDs
// frida -n rekordbox.exe -l tools\rbx_pdb_capture.js # or -p <pid>
// then trigger "Export to USB". If you see [w-open] lines but the export file
// isn't among them, widen SAVE_RE. If you see NOTHING, attach to another PID.
//
// Set VERBOSE = true to log every write-open path; false logs only SAVE_RE hits.
var OUT_DIR = "C:\\Users\\ntr\\code\\maple\\maple-py\\captures\\"; // must exist; Frida won't mkdir
var SAVE_RE = /export.*\.pdb|\.pdb$|PIONEER/i; // which opened files to reconstruct + save
var VERBOSE = true; // log ALL write-opens, not just SAVE_RE
var PID = Process.id;
var TAG = "[" + PID + "] ";
var DEBUG = true; // log every interesting open + all mapping activity
var DEBUG_RE = /export|\.pdb|PIONEER|rekordbox|Contents|USBANLZ/i; // what DEBUG opens to print
var ntdll = Process.getModuleByName("ntdll.dll");
function nt(name) { return ntdll.getExportByName(name); }
// Resolve + confirm a hook target so we can see in the console what attached.
function resolve(name) {
var a = ntdll.findExportByName(name);
console.log(TAG + "hook " + name + (a ? " -> " + a : " MISSING"));
return a;
}
// handle (string) -> { path, pos, end, chunks:[{pos,ab}], save, traced }
var files = {};
var counters = {};
function baseName(path) {
var parts = path.replace(/\\/g, "/").replace(/\/+$/, "").split("/");
return (parts[parts.length - 1] || "capture.bin").replace(/[^A-Za-z0-9._-]/g, "");
}
// Read the wide path out of an OBJECT_ATTRIBUTES* (x64 layout).
function readObjName(pObjAttr) {
if (pObjAttr.isNull()) return null;
var pUni = pObjAttr.add(16).readPointer(); // OBJECT_ATTRIBUTES.ObjectName
if (pUni.isNull()) return null;
var len = pUni.readU16(); // UNICODE_STRING.Length (bytes)
if (!len) return null;
var buf = pUni.add(8).readPointer(); // UNICODE_STRING.Buffer
if (buf.isNull()) return null;
return buf.readUtf16String(len / 2);
}
// NtCreateFile / NtOpenFile: args[0]=PHANDLE out, [1]=ACCESS_MASK, [2]=OBJECT_ATTRIBUTES*
function hookOpen(name) {
var addr = resolve(name);
if (!addr) return;
Interceptor.attach(addr, {
onEnter: function (args) {
this.pHandle = args[0];
this.access = args[1].toUInt32();
this.path = readObjName(args[2]);
},
onLeave: function (retval) {
if (!this.path) return;
var WRITE = 0x40000000 /*GENERIC_WRITE*/ | 0x10000000 /*GENERIC_ALL*/ |
0x2 /*FILE_WRITE_DATA*/ | 0x4 /*FILE_APPEND_DATA*/;
var isWrite = (this.access & WRITE) !== 0;
var save = SAVE_RE.test(this.path);
// DEBUG: show every interesting open (incl. read-only) + its access mask,
// so we can see if/how export.pdb is opened even when our filters miss it.
if (DEBUG && DEBUG_RE.test(this.path))
console.log(TAG + "[open] access=0x" + this.access.toString(16)
+ (isWrite ? " W" : " r") + (save ? " *SAVE" : "") + " " + this.path);
if (!isWrite && !save) return;
var handleStr = this.pHandle.readPointer().toString();
files[handleStr] = { path: this.path, pos: 0, end: 0, chunks: [], save: save, traced: false };
if (!DEBUG && (save || VERBOSE)) console.log(TAG + "[w-open] " + this.path);
},
});
}
hookOpen("NtCreateFile");
hookOpen("NtOpenFile");
// NtWriteFile: [0]=handle [5]=buffer [6]=length [7]=PLARGE_INTEGER ByteOffset
Interceptor.attach(resolve("NtWriteFile"), {
onEnter: function (args) {
var f = files[args[0].toString()];
if (!f) return;
var n = args[6].toUInt32();
if (n <= 0) return;
var pos = f.pos;
var pOff = args[7];
if (!pOff.isNull()) {
var hi = pOff.add(4).readU32();
if (hi !== 0xffffffff) pos = hi * 0x100000000 + pOff.readU32(); // skip sentinels
}
if (f.save) {
f.chunks.push({ pos: pos, ab: args[5].readByteArray(n) });
if (!f.traced) {
f.traced = true;
console.log(TAG + "[trace] writer for " + f.path + ":");
Thread.backtrace(this.context, Backtracer.ACCURATE)
.slice(0, 14)
.forEach(function (a) {
// module+offset maps straight to Ghidra: static = 0x140000000 + offset
var m = Process.findModuleByAddress(a);
var loc = m ? (m.name + "+0x" + a.sub(m.base).toString(16)) : a.toString();
console.log(" " + loc + " " + DebugSymbol.fromAddress(a));
});
}
}
f.pos = pos + n;
if (f.pos > f.end) f.end = f.pos;
},
});
// Shared: write a captured buffer to OUT_DIR with a PID-tagged, de-duped name.
function saveBuffer(path, arrayBuffer, size) {
var name = baseName(path);
var c = counters[name] || 0;
counters[name] = c + 1;
var dest = OUT_DIR + "p" + PID + "_" + name + (c === 0 ? "" : "." + c);
var fh = new File(dest, "wb");
fh.write(arrayBuffer);
fh.close();
console.log(TAG + "[SAVED] " + dest + " (" + size + " bytes)");
}
// NtClose: reconstruct + flush saved files written via NtWriteFile
function flush(handleStr) {
var f = files[handleStr];
if (!f) return;
delete files[handleStr];
if (!f.save || f.end === 0) return;
var out = new Uint8Array(f.end);
for (var i = 0; i < f.chunks.length; i++) out.set(new Uint8Array(f.chunks[i].ab), f.chunks[i].pos);
saveBuffer(f.path, out.buffer, f.end);
}
Interceptor.attach(resolve("NtClose"), { onEnter: function (args) { flush(args[0].toString()); } });
// ---- memory-mapped writes (the .pdb is likely built via a file mapping) ----
// CreateFileMapping/MapViewOfFile funnel through these ntdll calls. We link
// section -> backing file (NtCreateSection takes the file handle we already
// track), record the mapped view, and snapshot its memory when it's unmapped.
var sections = {}; // sectionHandle -> { path, save, maxSize }
var views = {}; // baseAddr string -> { base, size, path, save, maxSize }
function hookSections() {
var cs = resolve("NtCreateSection");
if (cs) Interceptor.attach(cs, {
onEnter: function (args) {
this.pSection = args[0];
this.pMax = args[3]; // PLARGE_INTEGER MaximumSize (may be null)
this.fileHandle = args[6].toString();// HANDLE FileHandle
},
onLeave: function (ret) {
if (ret.toInt32() !== 0) return; // not STATUS_SUCCESS
var f = files[this.fileHandle];
if (!f) return; // only file-backed sections we track
var maxSize = (this.pMax && !this.pMax.isNull()) ? this.pMax.readU32() : 0;
sections[this.pSection.readPointer().toString()] = { path: f.path, save: f.save, maxSize: maxSize };
},
});
var mv = resolve("NtMapViewOfSection");
if (mv) Interceptor.attach(mv, {
onEnter: function (args) {
this.section = args[0].toString();
this.pBase = args[2]; // PVOID* BaseAddress
this.pViewSize = args[6]; // PSIZE_T ViewSize
},
onLeave: function (ret) {
if (ret.toInt32() < 0) return; // negative NTSTATUS = error
var s = sections[this.section];
if (!s) return;
var base = this.pBase.readPointer();
var size = (this.pViewSize && !this.pViewSize.isNull()) ? this.pViewSize.readU32() : 0;
views[base.toString()] = { base: base, size: size, path: s.path, save: s.save, maxSize: s.maxSize };
if (s.save || VERBOSE) console.log(TAG + "[mmap] " + s.path + " @ " + base + " view=" + size + " max=" + s.maxSize);
},
});
function captureView(base) {
var v = views[base.toString()];
if (!v) return;
delete views[base.toString()];
if (!v.save) return;
var size = v.maxSize || v.size;
if (!size) { var r = Process.findRangeByAddress(base); size = r ? r.size : 0; }
if (!size) return;
try { saveBuffer(v.path, Memory.readByteArray(base, size), size); }
catch (e) { console.log(TAG + "[mmap] read failed @ " + base + ": " + e); }
}
var um = ntdll.findExportByName("NtUnmapViewOfSection");
if (um) Interceptor.attach(um, { onEnter: function (args) { captureView(args[1]); } });
}
hookSections();
console.log(TAG + "ntdll + mmap hooks installed -- trigger 'Export to USB' in rekordbox now");
+711
View File
@@ -0,0 +1,711 @@
version = 1
revision = 3
requires-python = ">=3.14"
[[package]]
name = "aiohappyeyeballs"
version = "2.6.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/33/c6/61a2d7b7572279226bb2e7f61d7a19ca7c90da0329c93fa0d560cbf288d8/aiohappyeyeballs-2.6.2.tar.gz", hash = "sha256:e202810ee718bd01fc6ef49e8ea53d023d5cb6b581076d7925aa499fa55dbe64", size = 22591, upload-time = "2026-05-20T15:12:24.631Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/5f/fc/a7bf5b6e4e617b45f90f2d9d2a68519c249c81dd4fc2658c7a2a61c4f4b7/aiohappyeyeballs-2.6.2-py3-none-any.whl", hash = "sha256:4708045e2d7a6c6bdf8aafa8ed39649eaf926a4543b54560659129e3365953c4", size = 15062, upload-time = "2026-05-20T15:12:23.328Z" },
]
[[package]]
name = "aiohttp"
version = "3.14.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohappyeyeballs" },
{ name = "aiosignal" },
{ name = "attrs" },
{ name = "frozenlist" },
{ name = "multidict" },
{ name = "propcache" },
{ name = "yarl" },
]
sdist = { url = "https://files.pythonhosted.org/packages/82/78/8ea7308cac6934de8c74a14f3d5f65d1c89287426688be79538d0e5c013d/aiohttp-3.14.1.tar.gz", hash = "sha256:307f2cff90a764d329e77040603fa032db89c5c24fdad50c4c15334cba744035", size = 7955794, upload-time = "2026-06-07T21:09:35.529Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/c4/a1/5fafa04e1ca91ddb47608699d60649c1c6db3cf41c99e78fc4056f9513db/aiohttp-3.14.1-cp314-cp314-android_24_arm64_v8a.whl", hash = "sha256:7c106c26852ca1c2047c6b80384f17100b4e439af276f21ef3d4e2f450ae7e15", size = 508531, upload-time = "2026-06-07T21:08:02.093Z" },
{ url = "https://files.pythonhosted.org/packages/fa/2e/bfa02f699d87ffc86d5959270b28f1cb410add3ccaced8ed2e0b8a5238fc/aiohttp-3.14.1-cp314-cp314-android_24_x86_64.whl", hash = "sha256:20205f7f5ade7aaec9f4b500549bbc071b046453aed72f9c06dcab87896a83e8", size = 514718, upload-time = "2026-06-07T21:08:04.476Z" },
{ url = "https://files.pythonhosted.org/packages/85/a5/9594ad6289eebbc97d167c44213d557807f90e59115caad24de21ad2c3b1/aiohttp-3.14.1-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:62a759436b29e677181a9e76bab8b8f689a29cb9c535f45f7c48c9c830d3f8c3", size = 487918, upload-time = "2026-06-07T21:08:06.377Z" },
{ url = "https://files.pythonhosted.org/packages/b4/61/16a32c36c3c49edec122a3dc811f2057df2f94d3b14aa107c8017d981618/aiohttp-3.14.1-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:2964cbf553df4d7a57348da44d961d871895fc1ee4e8c322b2a95612c7b17fba", size = 494014, upload-time = "2026-06-07T21:08:08.263Z" },
{ url = "https://files.pythonhosted.org/packages/9b/89/3ebcf96ed99c05bec9c434aaac6963fd3cbab4a786ae739908a144d9ce44/aiohttp-3.14.1-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:237651caadc3a59badd39319c54642b5299e9cc98a3a194310e55d5bb9f5e397", size = 502398, upload-time = "2026-06-07T21:08:10.244Z" },
{ url = "https://files.pythonhosted.org/packages/fd/3d/b74870a0c2d40c355928cd5b96c7a11fa821b8a40fc41365e64479b151fb/aiohttp-3.14.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:896e12dfdbbab9d8f7e16d2b28c6769a60126fa92095d1ebf9473d02593a2448", size = 758018, upload-time = "2026-06-07T21:08:12.447Z" },
{ url = "https://files.pythonhosted.org/packages/d3/66/f42f5c984d99e49c6cff5f26f590750f2e2f7ef1fcfb99966ab5be1b632e/aiohttp-3.14.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d03f281ed22579314ba00821ce20115a7c0ac430660b4cc05704a3f818b3e004", size = 512462, upload-time = "2026-06-07T21:08:14.624Z" },
{ url = "https://files.pythonhosted.org/packages/e9/a7/248e1aebe0c7810b0271e021a0f2a5eb6e78a051885b3c9df49f42a5802d/aiohttp-3.14.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:07eabb979d236335fed927e137a928c9adfb7df3b9ec7aa31726f133a62be983", size = 512824, upload-time = "2026-06-07T21:08:16.572Z" },
{ url = "https://files.pythonhosted.org/packages/26/97/2aa0e5ba0727dc3bd5aaebb7ccbc510f7dfb7fb961ec87497cd496635ab1/aiohttp-3.14.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4fe1f1087cbadb280b5e1bb054a4f00d1423c74d6626c5e48400d871d34ecefe", size = 1749898, upload-time = "2026-06-07T21:08:18.635Z" },
{ url = "https://files.pythonhosted.org/packages/00/8d/e97f6c96c891d457c8479d92a514ba194d0412f981d72c70341ee18488ed/aiohttp-3.14.1-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:367a9314fdc79dab0fac96e216cb41dd73c85bdca85306ce8999118ba7e0f333", size = 1710114, upload-time = "2026-06-07T21:08:20.892Z" },
{ url = "https://files.pythonhosted.org/packages/6f/e6/aa8d7e863048c8fceb5cd6ce74017311cec3ead07847387e12265fb4444e/aiohttp-3.14.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a24f677ebe83749039e7bdf862ff0bbb16818ae4193d4ef96505e269375bcce0", size = 1802541, upload-time = "2026-06-07T21:08:23.044Z" },
{ url = "https://files.pythonhosted.org/packages/83/a8/72193137de57fda4ebfae4563182d082c8856e3b6e9871d0b46f028fb369/aiohttp-3.14.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:c83afe0ba876be7e943d2e0ba645809ad441575d2840c895c21ee5de93b9377a", size = 1875776, upload-time = "2026-06-07T21:08:25.288Z" },
{ url = "https://files.pythonhosted.org/packages/a0/18/938441025db6769a3464596b2410af3afde0b21eb2f204c6f766f68af4bd/aiohttp-3.14.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:634e385930fb6d2d479cf3aa66515955863b77a5e3c2b5894ca259a25b308602", size = 1760329, upload-time = "2026-06-07T21:08:27.363Z" },
{ url = "https://files.pythonhosted.org/packages/60/29/bf2496b4065e76e09fe48015aaffe5ce161d8f089b06ac6982070f653076/aiohttp-3.14.1-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:eeea07c4397bbc57719c4eed8f9c284874d4f175f9b6d57f7a1546b976d455ca", size = 1587293, upload-time = "2026-06-07T21:08:29.805Z" },
{ url = "https://files.pythonhosted.org/packages/49/a2/2136674d52123b1354bd05dd5753c318db47dc0c927cc70b27bab3755456/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:335c0cc3e3545ce98dcb9cfcb836f40c3411f43fa03dab757597d80c89af8a35", size = 1714756, upload-time = "2026-06-07T21:08:32.094Z" },
{ url = "https://files.pythonhosted.org/packages/a7/b9/e5fd2e6f915503081c0f9b1e8540947037929c70c191da2e4d54b31a21a1/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:ae6be797afdef264e8a84864a85b196ca06045586481b3df8a967322fd2fa844", size = 1721052, upload-time = "2026-06-07T21:08:34.167Z" },
{ url = "https://files.pythonhosted.org/packages/63/5a/2833e324a2263e104e31e2e91bc5bbee81bc499afd32203faee048a883f0/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:8560b4d712474335d08907db7973f71912d3a9a8f1dee992ec06b5d2fe359496", size = 1766888, upload-time = "2026-06-07T21:08:36.95Z" },
{ url = "https://files.pythonhosted.org/packages/57/fa/dea6511870913162f3b2e8c42a7614eb203a4540b8c2da43e0bfb0548f3c/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:2b7edd08e0a5deb1e8564a2fcd8f4561014a3f05252334671bbf55ddd47db0e5", size = 1581679, upload-time = "2026-06-07T21:08:39.292Z" },
{ url = "https://files.pythonhosted.org/packages/14/bd/3cf0d55e71784b33534e9710a67d382d900598b4787fbce6cc7317f8c42a/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:b6ff7fcee63287ae57b5df3e4f5957ce032122802509246dec1a5bcc55904c95", size = 1782021, upload-time = "2026-06-07T21:08:41.407Z" },
{ url = "https://files.pythonhosted.org/packages/c1/af/14bb5843eccbe234f4dfb78ab73e549d99727247e62ae5d62cbd22eaf5b0/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6ffbb2f4ec1ceaff7e07d43922954da26b223d188bf30658e561b98e23089444", size = 1742574, upload-time = "2026-06-07T21:08:43.795Z" },
{ url = "https://files.pythonhosted.org/packages/f2/1e/fbeb7af9210a67ac0f9c9bec0f8f4568497924e33137a3d5b48e1cf85f3f/aiohttp-3.14.1-cp314-cp314-win32.whl", hash = "sha256:a9875b46d910cff3ea2f5962f9d266b465459fe634e22556ab9bd6fc1192eea0", size = 457773, upload-time = "2026-06-07T21:08:46.168Z" },
{ url = "https://files.pythonhosted.org/packages/f0/2b/13e8d741a9ec5db7d900c060554cf8352ab85e44e2a4469ebb9d377bda17/aiohttp-3.14.1-cp314-cp314-win_amd64.whl", hash = "sha256:af8b4b81a960eeaf1234971ac3cd0ba5901f3cd42eae42a46b4d089a8b492719", size = 485001, upload-time = "2026-06-07T21:08:48.401Z" },
{ url = "https://files.pythonhosted.org/packages/df/30/491acfa2c4d6c3ff59c49a14fc1b50be3241e25bbb0c84c09e2da4d11395/aiohttp-3.14.1-cp314-cp314-win_arm64.whl", hash = "sha256:cf4491381b1b57425c315a56a439251b1bdac07b2275f19a8c44bc57744532ec", size = 453809, upload-time = "2026-06-07T21:08:50.7Z" },
{ url = "https://files.pythonhosted.org/packages/34/e3/19dbe1a1f4cc6230eb9e314de7fe68053b0992f9302b27d12141a0b5db53/aiohttp-3.14.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:819c054312f1af92947e6a55883d1b66feefab11531a7fc45e0fb9b63880b5c2", size = 793320, upload-time = "2026-06-07T21:08:52.775Z" },
{ url = "https://files.pythonhosted.org/packages/7f/20/1b7182219ba1b108430d6e4dc53d25ae02dcfcf5a045b33af4e8c5167527/aiohttp-3.14.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:10ee9c1753a8f706345b22496c79fbddb5be0599e0823f3738b1534058e25340", size = 529077, upload-time = "2026-06-07T21:08:55Z" },
{ url = "https://files.pythonhosted.org/packages/b9/c8/14ce60ec31a2e5f5274bb17d383a6f7a3aabca31ac04eee05585bbadab16/aiohttp-3.14.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1601cc37baf5750ccacae618ec2daf020769581695550e3b654a911f859c563d", size = 532476, upload-time = "2026-06-07T21:08:57.176Z" },
{ url = "https://files.pythonhosted.org/packages/7e/02/9ac85e081e53da2e061b02fa7758fe0a12d17b8ce2d1f5e6c7cb76730328/aiohttp-3.14.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4d6e0ac9da31c9c04c84e1c0182ad8d6df35965a85cae29cd71d089621b3ae94", size = 1922347, upload-time = "2026-06-07T21:08:59.563Z" },
{ url = "https://files.pythonhosted.org/packages/c0/3e/d3ba07a0ab38b5389e10bec4362d21e10a4f667cba2d79ba30837b3a5059/aiohttp-3.14.1-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:9e8f2d660c350b3d0e259c7a7e3d9b7fc8b41210cbcc3d4a7076ff0a5e5c2fdc", size = 1786465, upload-time = "2026-06-07T21:09:01.909Z" },
{ url = "https://files.pythonhosted.org/packages/0b/cb/e2ee978a00cfb2df829704a69528b18154eba5939f45bc1efa8f33aee4c5/aiohttp-3.14.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:4691802dda97be727f79d86818acaad7eb8e9252626a1d6b519fedbb92d5e251", size = 1909423, upload-time = "2026-06-07T21:09:04.357Z" },
{ url = "https://files.pythonhosted.org/packages/73/5d/1430334858b1022b58ae50399a918f0bd6fe8fa7fa183598d657ff61e040/aiohttp-3.14.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:c389c482a7e9b9dc3ee2701ac46c4125297a3818875b9c305ddb603c04828fd1", size = 2001906, upload-time = "2026-06-07T21:09:06.722Z" },
{ url = "https://files.pythonhosted.org/packages/66/4e/560c7472d3d198a23aa5c8b19a5115bf6a9b77b7d3e4bb363da320430ad2/aiohttp-3.14.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fc0cacab7ba4e56f0f81c82a98c09bed2f39c940107b03a34b168bdf7597edd3", size = 1877095, upload-time = "2026-06-07T21:09:09.011Z" },
{ url = "https://files.pythonhosted.org/packages/0d/f1/4745806578d447db4a784a8591e2dae3afdfc2bcb96f8f81271b13df6543/aiohttp-3.14.1-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:979ed4717f59b8bb12e3963378fa285d93d367e15bcd66c721311826d3c44a6c", size = 1676222, upload-time = "2026-06-07T21:09:11.461Z" },
{ url = "https://files.pythonhosted.org/packages/6a/c9/48255813cca749a229ef0ab476004ec623728ad79a9c0840616f6c076325/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:38e1e7daaea81df51c952e18483f323d878499a1e2bfe564790e0f9701d6f203", size = 1842922, upload-time = "2026-06-07T21:09:14.118Z" },
{ url = "https://files.pythonhosted.org/packages/3d/c0/bbd054e2bee909f529523a5af3891052606af5143c09f5f183ec3b234676/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:4132e72c608fe9fecb8f409113567605915b83e9bdd3ea56538d2f9cd35002f1", size = 1825035, upload-time = "2026-06-07T21:09:16.447Z" },
{ url = "https://files.pythonhosted.org/packages/a8/ae/90395d4376deceb74e09ec26b6adf7d2015a6f8802d6d84446af860fef04/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:eefd9cc9b6d4a2db5f00a26bc3e4f9acf71926a6ec557cd56c9c6f27c290b665", size = 1849512, upload-time = "2026-06-07T21:09:18.742Z" },
{ url = "https://files.pythonhosted.org/packages/93/bd/fb25f3049957553d4ce0ba6ae480aa2f592a6985497fca590837d16c1be0/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:b165790117eea512d7f3fb22f1f6dad3d55a7189571993eb015591c1401276d1", size = 1668571, upload-time = "2026-06-07T21:09:21.458Z" },
{ url = "https://files.pythonhosted.org/packages/3f/22/7f73303d64dd567ff3addca90b556690ed1233a47b8f55d242fb90af3681/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:ed09c7eb1c391271c2ed0314a51903e72a3acb653d5ccfc264cdf3ef11f8269d", size = 1881159, upload-time = "2026-06-07T21:09:23.813Z" },
{ url = "https://files.pythonhosted.org/packages/44/be/0474c5a8b5640e1e4aa1923430a91f4151be82e511373fe764189b89aef5/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:99abd37084b82f5830c635fddd0b4993b9742a66eb746dacf433c8590e8f9e3c", size = 1841409, upload-time = "2026-06-07T21:09:26.207Z" },
{ url = "https://files.pythonhosted.org/packages/7b/3c/bb4a7cba26956cb3da4553cc2056cf67be5b5ff6e6d8fa4fbdff73bfb7ae/aiohttp-3.14.1-cp314-cp314t-win32.whl", hash = "sha256:47ddf841cdecc810749921d25606dee45857d12d2ad5ddb7b5bd7eab12e4b365", size = 494166, upload-time = "2026-06-07T21:09:28.505Z" },
{ url = "https://files.pythonhosted.org/packages/8a/84/ec80c2c1f66a952555a9f86df6b33af65108a6febfa0471b69013a12f807/aiohttp-3.14.1-cp314-cp314t-win_amd64.whl", hash = "sha256:5e78b522b7a6e27e0b25d19b247b75039ac4c94f99823e3c9e53ae1603a9f7e9", size = 530255, upload-time = "2026-06-07T21:09:30.843Z" },
{ url = "https://files.pythonhosted.org/packages/2a/71/6e22be134a4061ada85a92951b842f2657f17d926b727f3f94c56ae963d6/aiohttp-3.14.1-cp314-cp314t-win_arm64.whl", hash = "sha256:90d53f1609c29ccc2193945ef732428382a28f78d0456ae4d3daf0d48b74f0f6", size = 469640, upload-time = "2026-06-07T21:09:33.028Z" },
]
[[package]]
name = "aiosignal"
version = "1.4.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "frozenlist" },
]
sdist = { url = "https://files.pythonhosted.org/packages/61/62/06741b579156360248d1ec624842ad0edf697050bbaf7c3e46394e106ad1/aiosignal-1.4.0.tar.gz", hash = "sha256:f47eecd9468083c2029cc99945502cb7708b082c232f9aca65da147157b251c7", size = 25007, upload-time = "2025-07-03T22:54:43.528Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/fb/76/641ae371508676492379f16e2fa48f4e2c11741bd63c48be4b12a6b09cba/aiosignal-1.4.0-py3-none-any.whl", hash = "sha256:053243f8b92b990551949e63930a839ff0cf0b0ebbe0597b0f3fb19e1a0fe82e", size = 7490, upload-time = "2025-07-03T22:54:42.156Z" },
]
[[package]]
name = "attrs"
version = "26.1.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/9a/8e/82a0fe20a541c03148528be8cac2408564a6c9a0cc7e9171802bc1d26985/attrs-26.1.0.tar.gz", hash = "sha256:d03ceb89cb322a8fd706d4fb91940737b6642aa36998fe130a9bc96c985eff32", size = 952055, upload-time = "2026-03-19T14:22:25.026Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/64/b4/17d4b0b2a2dc85a6df63d1157e028ed19f90d4cd97c36717afef2bc2f395/attrs-26.1.0-py3-none-any.whl", hash = "sha256:c647aa4a12dfbad9333ca4e71fe62ddc36f4e63b2d260a37a8b83d2f043ac309", size = 67548, upload-time = "2026-03-19T14:22:23.645Z" },
]
[[package]]
name = "bidict"
version = "0.23.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/9a/6e/026678aa5a830e07cd9498a05d3e7e650a4f56a42f267a53d22bcda1bdc9/bidict-0.23.1.tar.gz", hash = "sha256:03069d763bc387bbd20e7d49914e75fc4132a41937fa3405417e1a5a2d006d71", size = 29093, upload-time = "2024-02-18T19:09:05.748Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/99/37/e8730c3587a65eb5645d4aba2d27aae48e8003614d6aaf15dda67f702f1f/bidict-0.23.1-py3-none-any.whl", hash = "sha256:5dae8d4d79b552a71cbabc7deb25dfe8ce710b17ff41711e13010ead2abfc3e5", size = 32764, upload-time = "2024-02-18T19:09:04.156Z" },
]
[[package]]
name = "colorama"
version = "0.4.6"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
]
[[package]]
name = "construct"
version = "2.10.70"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/02/77/8c84b98eca70d245a2a956452f21d57930d22ab88cbeed9290ca630cf03f/construct-2.10.70.tar.gz", hash = "sha256:4d2472f9684731e58cc9c56c463be63baa1447d674e0d66aeb5627b22f512c29", size = 86337, upload-time = "2023-11-29T08:44:49.545Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b2/fb/08b3f4bf05da99aba8ffea52a558758def16e8516bc75ca94ff73587e7d3/construct-2.10.70-py3-none-any.whl", hash = "sha256:c80be81ef595a1a821ec69dc16099550ed22197615f4320b57cc9ce2a672cb30", size = 63020, upload-time = "2023-11-29T08:44:46.876Z" },
]
[[package]]
name = "frozenlist"
version = "1.8.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/2d/f5/c831fac6cc817d26fd54c7eaccd04ef7e0288806943f7cc5bbf69f3ac1f0/frozenlist-1.8.0.tar.gz", hash = "sha256:3ede829ed8d842f6cd48fc7081d7a41001a56f1f38603f9d49bf3020d59a31ad", size = 45875, upload-time = "2025-10-06T05:38:17.865Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f1/c8/85da824b7e7b9b6e7f7705b2ecaf9591ba6f79c1177f324c2735e41d36a2/frozenlist-1.8.0-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:cee686f1f4cadeb2136007ddedd0aaf928ab95216e7691c63e50a8ec066336d0", size = 86127, upload-time = "2025-10-06T05:37:08.438Z" },
{ url = "https://files.pythonhosted.org/packages/8e/e8/a1185e236ec66c20afd72399522f142c3724c785789255202d27ae992818/frozenlist-1.8.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:119fb2a1bd47307e899c2fac7f28e85b9a543864df47aa7ec9d3c1b4545f096f", size = 49698, upload-time = "2025-10-06T05:37:09.48Z" },
{ url = "https://files.pythonhosted.org/packages/a1/93/72b1736d68f03fda5fdf0f2180fb6caaae3894f1b854d006ac61ecc727ee/frozenlist-1.8.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:4970ece02dbc8c3a92fcc5228e36a3e933a01a999f7094ff7c23fbd2beeaa67c", size = 49749, upload-time = "2025-10-06T05:37:10.569Z" },
{ url = "https://files.pythonhosted.org/packages/a7/b2/fabede9fafd976b991e9f1b9c8c873ed86f202889b864756f240ce6dd855/frozenlist-1.8.0-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:cba69cb73723c3f329622e34bdbf5ce1f80c21c290ff04256cff1cd3c2036ed2", size = 231298, upload-time = "2025-10-06T05:37:11.993Z" },
{ url = "https://files.pythonhosted.org/packages/3a/3b/d9b1e0b0eed36e70477ffb8360c49c85c8ca8ef9700a4e6711f39a6e8b45/frozenlist-1.8.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:778a11b15673f6f1df23d9586f83c4846c471a8af693a22e066508b77d201ec8", size = 232015, upload-time = "2025-10-06T05:37:13.194Z" },
{ url = "https://files.pythonhosted.org/packages/dc/94/be719d2766c1138148564a3960fc2c06eb688da592bdc25adcf856101be7/frozenlist-1.8.0-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:0325024fe97f94c41c08872db482cf8ac4800d80e79222c6b0b7b162d5b13686", size = 225038, upload-time = "2025-10-06T05:37:14.577Z" },
{ url = "https://files.pythonhosted.org/packages/e4/09/6712b6c5465f083f52f50cf74167b92d4ea2f50e46a9eea0523d658454ae/frozenlist-1.8.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:97260ff46b207a82a7567b581ab4190bd4dfa09f4db8a8b49d1a958f6aa4940e", size = 240130, upload-time = "2025-10-06T05:37:15.781Z" },
{ url = "https://files.pythonhosted.org/packages/f8/d4/cd065cdcf21550b54f3ce6a22e143ac9e4836ca42a0de1022da8498eac89/frozenlist-1.8.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:54b2077180eb7f83dd52c40b2750d0a9f175e06a42e3213ce047219de902717a", size = 242845, upload-time = "2025-10-06T05:37:17.037Z" },
{ url = "https://files.pythonhosted.org/packages/62/c3/f57a5c8c70cd1ead3d5d5f776f89d33110b1addae0ab010ad774d9a44fb9/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:2f05983daecab868a31e1da44462873306d3cbfd76d1f0b5b69c473d21dbb128", size = 229131, upload-time = "2025-10-06T05:37:18.221Z" },
{ url = "https://files.pythonhosted.org/packages/6c/52/232476fe9cb64f0742f3fde2b7d26c1dac18b6d62071c74d4ded55e0ef94/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:33f48f51a446114bc5d251fb2954ab0164d5be02ad3382abcbfe07e2531d650f", size = 240542, upload-time = "2025-10-06T05:37:19.771Z" },
{ url = "https://files.pythonhosted.org/packages/5f/85/07bf3f5d0fb5414aee5f47d33c6f5c77bfe49aac680bfece33d4fdf6a246/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:154e55ec0655291b5dd1b8731c637ecdb50975a2ae70c606d100750a540082f7", size = 237308, upload-time = "2025-10-06T05:37:20.969Z" },
{ url = "https://files.pythonhosted.org/packages/11/99/ae3a33d5befd41ac0ca2cc7fd3aa707c9c324de2e89db0e0f45db9a64c26/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:4314debad13beb564b708b4a496020e5306c7333fa9a3ab90374169a20ffab30", size = 238210, upload-time = "2025-10-06T05:37:22.252Z" },
{ url = "https://files.pythonhosted.org/packages/b2/60/b1d2da22f4970e7a155f0adde9b1435712ece01b3cd45ba63702aea33938/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:073f8bf8becba60aa931eb3bc420b217bb7d5b8f4750e6f8b3be7f3da85d38b7", size = 231972, upload-time = "2025-10-06T05:37:23.5Z" },
{ url = "https://files.pythonhosted.org/packages/3f/ab/945b2f32de889993b9c9133216c068b7fcf257d8595a0ac420ac8677cab0/frozenlist-1.8.0-cp314-cp314-win32.whl", hash = "sha256:bac9c42ba2ac65ddc115d930c78d24ab8d4f465fd3fc473cdedfccadb9429806", size = 40536, upload-time = "2025-10-06T05:37:25.581Z" },
{ url = "https://files.pythonhosted.org/packages/59/ad/9caa9b9c836d9ad6f067157a531ac48b7d36499f5036d4141ce78c230b1b/frozenlist-1.8.0-cp314-cp314-win_amd64.whl", hash = "sha256:3e0761f4d1a44f1d1a47996511752cf3dcec5bbdd9cc2b4fe595caf97754b7a0", size = 44330, upload-time = "2025-10-06T05:37:26.928Z" },
{ url = "https://files.pythonhosted.org/packages/82/13/e6950121764f2676f43534c555249f57030150260aee9dcf7d64efda11dd/frozenlist-1.8.0-cp314-cp314-win_arm64.whl", hash = "sha256:d1eaff1d00c7751b7c6662e9c5ba6eb2c17a2306ba5e2a37f24ddf3cc953402b", size = 40627, upload-time = "2025-10-06T05:37:28.075Z" },
{ url = "https://files.pythonhosted.org/packages/c0/c7/43200656ecc4e02d3f8bc248df68256cd9572b3f0017f0a0c4e93440ae23/frozenlist-1.8.0-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:d3bb933317c52d7ea5004a1c442eef86f426886fba134ef8cf4226ea6ee1821d", size = 89238, upload-time = "2025-10-06T05:37:29.373Z" },
{ url = "https://files.pythonhosted.org/packages/d1/29/55c5f0689b9c0fb765055629f472c0de484dcaf0acee2f7707266ae3583c/frozenlist-1.8.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:8009897cdef112072f93a0efdce29cd819e717fd2f649ee3016efd3cd885a7ed", size = 50738, upload-time = "2025-10-06T05:37:30.792Z" },
{ url = "https://files.pythonhosted.org/packages/ba/7d/b7282a445956506fa11da8c2db7d276adcbf2b17d8bb8407a47685263f90/frozenlist-1.8.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:2c5dcbbc55383e5883246d11fd179782a9d07a986c40f49abe89ddf865913930", size = 51739, upload-time = "2025-10-06T05:37:32.127Z" },
{ url = "https://files.pythonhosted.org/packages/62/1c/3d8622e60d0b767a5510d1d3cf21065b9db874696a51ea6d7a43180a259c/frozenlist-1.8.0-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:39ecbc32f1390387d2aa4f5a995e465e9e2f79ba3adcac92d68e3e0afae6657c", size = 284186, upload-time = "2025-10-06T05:37:33.21Z" },
{ url = "https://files.pythonhosted.org/packages/2d/14/aa36d5f85a89679a85a1d44cd7a6657e0b1c75f61e7cad987b203d2daca8/frozenlist-1.8.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:92db2bf818d5cc8d9c1f1fc56b897662e24ea5adb36ad1f1d82875bd64e03c24", size = 292196, upload-time = "2025-10-06T05:37:36.107Z" },
{ url = "https://files.pythonhosted.org/packages/05/23/6bde59eb55abd407d34f77d39a5126fb7b4f109a3f611d3929f14b700c66/frozenlist-1.8.0-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:2dc43a022e555de94c3b68a4ef0b11c4f747d12c024a520c7101709a2144fb37", size = 273830, upload-time = "2025-10-06T05:37:37.663Z" },
{ url = "https://files.pythonhosted.org/packages/d2/3f/22cff331bfad7a8afa616289000ba793347fcd7bc275f3b28ecea2a27909/frozenlist-1.8.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:cb89a7f2de3602cfed448095bab3f178399646ab7c61454315089787df07733a", size = 294289, upload-time = "2025-10-06T05:37:39.261Z" },
{ url = "https://files.pythonhosted.org/packages/a4/89/5b057c799de4838b6c69aa82b79705f2027615e01be996d2486a69ca99c4/frozenlist-1.8.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:33139dc858c580ea50e7e60a1b0ea003efa1fd42e6ec7fdbad78fff65fad2fd2", size = 300318, upload-time = "2025-10-06T05:37:43.213Z" },
{ url = "https://files.pythonhosted.org/packages/30/de/2c22ab3eb2a8af6d69dc799e48455813bab3690c760de58e1bf43b36da3e/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:168c0969a329b416119507ba30b9ea13688fafffac1b7822802537569a1cb0ef", size = 282814, upload-time = "2025-10-06T05:37:45.337Z" },
{ url = "https://files.pythonhosted.org/packages/59/f7/970141a6a8dbd7f556d94977858cfb36fa9b66e0892c6dd780d2219d8cd8/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:28bd570e8e189d7f7b001966435f9dac6718324b5be2990ac496cf1ea9ddb7fe", size = 291762, upload-time = "2025-10-06T05:37:46.657Z" },
{ url = "https://files.pythonhosted.org/packages/c1/15/ca1adae83a719f82df9116d66f5bb28bb95557b3951903d39135620ef157/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:b2a095d45c5d46e5e79ba1e5b9cb787f541a8dee0433836cea4b96a2c439dcd8", size = 289470, upload-time = "2025-10-06T05:37:47.946Z" },
{ url = "https://files.pythonhosted.org/packages/ac/83/dca6dc53bf657d371fbc88ddeb21b79891e747189c5de990b9dfff2ccba1/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:eab8145831a0d56ec9c4139b6c3e594c7a83c2c8be25d5bcf2d86136a532287a", size = 289042, upload-time = "2025-10-06T05:37:49.499Z" },
{ url = "https://files.pythonhosted.org/packages/96/52/abddd34ca99be142f354398700536c5bd315880ed0a213812bc491cff5e4/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:974b28cf63cc99dfb2188d8d222bc6843656188164848c4f679e63dae4b0708e", size = 283148, upload-time = "2025-10-06T05:37:50.745Z" },
{ url = "https://files.pythonhosted.org/packages/af/d3/76bd4ed4317e7119c2b7f57c3f6934aba26d277acc6309f873341640e21f/frozenlist-1.8.0-cp314-cp314t-win32.whl", hash = "sha256:342c97bf697ac5480c0a7ec73cd700ecfa5a8a40ac923bd035484616efecc2df", size = 44676, upload-time = "2025-10-06T05:37:52.222Z" },
{ url = "https://files.pythonhosted.org/packages/89/76/c615883b7b521ead2944bb3480398cbb07e12b7b4e4d073d3752eb721558/frozenlist-1.8.0-cp314-cp314t-win_amd64.whl", hash = "sha256:06be8f67f39c8b1dc671f5d83aaefd3358ae5cdcf8314552c57e7ed3e6475bdd", size = 49451, upload-time = "2025-10-06T05:37:53.425Z" },
{ url = "https://files.pythonhosted.org/packages/e0/a3/5982da14e113d07b325230f95060e2169f5311b1017ea8af2a29b374c289/frozenlist-1.8.0-cp314-cp314t-win_arm64.whl", hash = "sha256:102e6314ca4da683dca92e3b1355490fed5f313b768500084fbe6371fddfdb79", size = 42507, upload-time = "2025-10-06T05:37:54.513Z" },
{ url = "https://files.pythonhosted.org/packages/9a/9a/e35b4a917281c0b8419d4207f4334c8e8c5dbf4f3f5f9ada73958d937dcc/frozenlist-1.8.0-py3-none-any.whl", hash = "sha256:0c18a16eab41e82c295618a77502e17b195883241c563b00f0aa5106fc4eaa0d", size = 13409, upload-time = "2025-10-06T05:38:16.721Z" },
]
[[package]]
name = "greenlet"
version = "3.5.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/6d/6e/802acd792aebb2256fbbee8cacf2727faaeb6f240ac11008f09eae4414bc/greenlet-3.5.1.tar.gz", hash = "sha256:5a56aeb7d5d9cc4b3a735efb5095bd4b4f6f0e4f93e5ca876d0e2315137b7829", size = 197356, upload-time = "2026-05-20T15:05:03.917Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/8a/cb/c62454606daf5640369c94d8a9dd540599b1bfc090e2d2180cb77f4038d2/greenlet-3.5.1-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:d8ab31c9de8651a2facdd5c5bb0011f2380dd1a7af78ce2adf4b56095294fc07", size = 285579, upload-time = "2026-05-20T13:08:56.396Z" },
{ url = "https://files.pythonhosted.org/packages/ec/71/c4270398c2eba968a6071af1dfbdcaeee6ec1c24bc8b435b8cc452700da6/greenlet-3.5.1-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5e300185139abc337ade480c327183adf42a875ac7181bfe66d7d4efea31fbea", size = 651106, upload-time = "2026-05-20T14:00:09.448Z" },
{ url = "https://files.pythonhosted.org/packages/1a/ab/71e34b78a44ec271fb5f550c17bc46d301ddc5953890d935f270b0dcdb5a/greenlet-3.5.1-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7ffdb990dcaa0234cf9845aead5df2e3c3a8b6507d409274dd87e0d5ab05ffc2", size = 663478, upload-time = "2026-05-20T14:05:45.88Z" },
{ url = "https://files.pythonhosted.org/packages/77/96/4efd6fa5c62c85426a0c19077a586258ebc3a2a146ff2493e4312a697a22/greenlet-3.5.1-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2f82b3597e9d83b63408affed0b48fd0f54935edac4302237b9a837be0dae33c", size = 660800, upload-time = "2026-05-20T13:14:29.129Z" },
{ url = "https://files.pythonhosted.org/packages/7a/e0/6c71401a25cac7000261304e866a2f2cc04dc74810d40e2f118aa4799495/greenlet-3.5.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c0141e37414c10164e702b8fb1473304221ad98f71600850c6ef7ff4880feba0", size = 1617518, upload-time = "2026-05-20T14:02:28.662Z" },
{ url = "https://files.pythonhosted.org/packages/41/26/c5c06643e8c0af9e7bf18e16cb51d0ab7625155f0392e1c9015d66d556cd/greenlet-3.5.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:50ae25a67bea74ea41fb14b960bc532df73eb713417b2d61892dced82fe8d3bc", size = 1681593, upload-time = "2026-05-20T13:14:39.417Z" },
{ url = "https://files.pythonhosted.org/packages/8a/bd/e11a108317485075e68af9d23039619b86b28130c3b50d227d42edece64b/greenlet-3.5.1-cp314-cp314-win_amd64.whl", hash = "sha256:8a17c42330e261299766b75ac1ea32caa437a9453c8f65d16a13140db378ecd3", size = 239800, upload-time = "2026-05-20T13:09:30.128Z" },
{ url = "https://files.pythonhosted.org/packages/47/f8/8e8e8417b7bf28639a5a56356ef934d0375e1d0c70a57e04d7701e870ffe/greenlet-3.5.1-cp314-cp314-win_arm64.whl", hash = "sha256:7b5f5fae05b8ac6d176a61b60c394a8cbdc2b5b91b81793066e68745cf165e54", size = 236862, upload-time = "2026-05-20T13:09:10.498Z" },
{ url = "https://files.pythonhosted.org/packages/90/12/41bf27fde4d3605d3773ae57751eda182b8be2f5398011c041173b1d9534/greenlet-3.5.1-cp314-cp314t-macosx_11_0_universal2.whl", hash = "sha256:ea8da1e900d758d078810d4255d8c6aa572181896a31ec79d779eb79c3adc9ad", size = 293637, upload-time = "2026-05-20T13:12:35.529Z" },
{ url = "https://files.pythonhosted.org/packages/44/44/ba14b23e9757707050c2f397d305bbcae62e5d7cad122f8b6baec5ae4a1f/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a19570c52a21420dcbc94e661994bc325c0b5b11304540fed514586da5dc8f2e", size = 650840, upload-time = "2026-05-20T14:00:11.079Z" },
{ url = "https://files.pythonhosted.org/packages/a8/37/5ddc2b686a6844f91abecef43411842426da2e1573f60b49ecf2547f4ae1/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:3d955c89b75eeca4723d7cc14135f393cd47c32e2a6cb4a8e4c6e760a26b0986", size = 656416, upload-time = "2026-05-20T14:05:47.118Z" },
{ url = "https://files.pythonhosted.org/packages/e1/f0/d17510297c35a2992712f0bf84de3779749999f7d3d63aa1f09db7c62dbe/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:de2daaaebd1a5aa88c49045b6baf9310b3263796bd88db713edf37cf53e7bb4e", size = 654397, upload-time = "2026-05-20T13:14:30.696Z" },
{ url = "https://files.pythonhosted.org/packages/37/eb/147387705bb89092645b012586e7273cb5ed3c90ef7eaf3a69173eaf0209/greenlet-3.5.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:3bfbd69cc349e43bf3a8ae1c85548ff0718efc887615c2db16c3833d7b0b072d", size = 1614469, upload-time = "2026-05-20T14:02:30.192Z" },
{ url = "https://files.pythonhosted.org/packages/a6/4e/37ee0da7732b7aa9896f17e15579a9df34b9fcb9dd494f0adfa749af6623/greenlet-3.5.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:4378720dd888136c27215a0214d32a4d37c3852765d45bc37aad0623423cfd78", size = 1675115, upload-time = "2026-05-20T13:14:40.972Z" },
{ url = "https://files.pythonhosted.org/packages/57/f3/97dfcf4a6eb5077f8a672234216fb5923eb89f2cab7081cb10b2cf75b605/greenlet-3.5.1-cp314-cp314t-win_amd64.whl", hash = "sha256:45718441607f9325d948db98cbc691276059316d0358c188c246da4e1d4d23d2", size = 245246, upload-time = "2026-05-20T13:12:22.646Z" },
{ url = "https://files.pythonhosted.org/packages/5d/73/d7f72e34b582f694f4a9b248162db7b09cc458a259ba8f0c0bfa1a34ea7d/greenlet-3.5.1-cp315-cp315-macosx_11_0_universal2.whl", hash = "sha256:2baee5ca02031757ffe8cc3d69f0cc0aec7065ce362622da74f32d3bcab1c541", size = 285575, upload-time = "2026-05-20T13:12:07.043Z" },
{ url = "https://files.pythonhosted.org/packages/df/59/fa9c6e87dc8ad27a95dabe2f29f372b733d05a8a67470f6c901ed9975655/greenlet-3.5.1-cp315-cp315-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9b1ec3274918a81d3ea778b9e75b56b72b33f300edb6cf7f3a7fe1dae56683de", size = 656428, upload-time = "2026-05-20T14:00:12.556Z" },
{ url = "https://files.pythonhosted.org/packages/f6/f9/e753408871eaa61dfe35e619cfc67512b036fde99893685d50eea9e07146/greenlet-3.5.1-cp315-cp315-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:111e2390ffffc47d5840b01711dd7fac07d4c09283d0283e7f3264b14e284c64", size = 667064, upload-time = "2026-05-20T14:05:48.662Z" },
{ url = "https://files.pythonhosted.org/packages/96/27/5565b5b40389f1c7753003a07e21892fda8660926787036d5bc0308b8113/greenlet-3.5.1-cp315-cp315-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e630136e905fe5ff43e86945ae41220b6d1470956a39220e708110ac48d01ea5", size = 665697, upload-time = "2026-05-20T13:14:32.943Z" },
{ url = "https://files.pythonhosted.org/packages/cf/82/e7de4178c0c2d1c9a5a3be3cc0b33e46a85b3ee4a77c071bf7ad8600e079/greenlet-3.5.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:975eac34b44a7077ca4d421348455b94f0f518246a7f14bc6d2fdcfe5b584368", size = 1621256, upload-time = "2026-05-20T14:02:31.91Z" },
{ url = "https://files.pythonhosted.org/packages/00/10/f2dddcf7dacac17dfc68691809589adad06135eb28930429cf58a6467a2f/greenlet-3.5.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:9ab3c3a0b2ae6198e67c898dad5215a49f9ae0d0081b3c3ec59f333e39eeca26", size = 1685956, upload-time = "2026-05-20T13:14:42.55Z" },
{ url = "https://files.pythonhosted.org/packages/22/17/4a232b32133230ada52f70e9d7f5b65b0caef8772f01849bd8d149e7e4ca/greenlet-3.5.1-cp315-cp315-win_amd64.whl", hash = "sha256:cbfc69be86e10dcfef5b1e6269d1d6926552aa89ee39e1de3353360c1b6989ab", size = 239802, upload-time = "2026-05-20T13:13:15.481Z" },
{ url = "https://files.pythonhosted.org/packages/c2/ae/4e623a7e6d4d2a5f4cb8e4c82de4169fc637942caae68d6e676b8a128ac5/greenlet-3.5.1-cp315-cp315-win_arm64.whl", hash = "sha256:92fd6d44ac5e5a887c8a5dc4a8ba0ba908527c31c12f78c6bc7dcfe8aab279f6", size = 236853, upload-time = "2026-05-20T13:15:37.301Z" },
{ url = "https://files.pythonhosted.org/packages/7a/57/816d9cff29119da3505b3d6a5e14a8af89006ac36f47f891ff293ee05af1/greenlet-3.5.1-cp315-cp315t-macosx_11_0_universal2.whl", hash = "sha256:a6fdf2433a5441ef9a95464f7c3e674775da1c8c1177fff311cee1acad4626ed", size = 293877, upload-time = "2026-05-20T13:10:19.078Z" },
{ url = "https://files.pythonhosted.org/packages/23/a1/59b0a7c7d140ff1a75626680b9a9899b79a9176cab298b394968fb023295/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7546556f0d649f99f6a361098a55f761181bb2ea12ff150bb16d26092ad88244", size = 655333, upload-time = "2026-05-20T14:00:14.758Z" },
{ url = "https://files.pythonhosted.org/packages/72/1b/5efe127597625042218939d01855109f352779050768b670b52edcc16a6c/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d5ee3ea898009fa898f85f9982255d35278c477bebe185beca249cab42d4526c", size = 659443, upload-time = "2026-05-20T14:05:50.159Z" },
{ url = "https://files.pythonhosted.org/packages/6c/6d/c404246ea4d22d097a7426d0efb5b781bd7eb67715f09e79001bd552ab18/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a5c81f74d204d3edd136ebfd50dce53acbb776995d721a0fe801626cfc93b8cd", size = 658356, upload-time = "2026-05-20T13:14:35.091Z" },
{ url = "https://files.pythonhosted.org/packages/51/02/f8ee37fb6d2219329f350af241c27fcf12df57e723d11f6fc6d3bacdadaa/greenlet-3.5.1-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:2c18ef16bf6d4dd410e4dd52996888ea1497be26892fe5bbc73580aba4287b8e", size = 1619216, upload-time = "2026-05-20T14:02:33.403Z" },
{ url = "https://files.pythonhosted.org/packages/93/c5/3dc9475ace2c7a3680da12372cddd7f1ac874eb410a1ac48d3e9dab83782/greenlet-3.5.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:17d86354f0ae6b61bf9be5148d0dd34e06c3cb7c602c671f79f29ac3b150e659", size = 1678427, upload-time = "2026-05-20T13:14:43.71Z" },
{ url = "https://files.pythonhosted.org/packages/df/4e/750c15c317a41ffb36f0bf40b933e3d744a7dede61889f74443ea69690cf/greenlet-3.5.1-cp315-cp315t-win_amd64.whl", hash = "sha256:e7516cf6ae6b8a582c2770a0caed47b8a48373ed732c33d69a72913ae6ac923e", size = 245225, upload-time = "2026-05-20T13:13:59.366Z" },
{ url = "https://files.pythonhosted.org/packages/4f/fd/d3baea2eeb7b617efd47e87ca06e2ec2c6118d303aa9e918e0ce16eadc10/greenlet-3.5.1-cp315-cp315t-win_arm64.whl", hash = "sha256:5028648bf2253ec4745add746129d3904121fa7fe871a76bed23c5720573ce0a", size = 239590, upload-time = "2026-05-20T13:13:37.382Z" },
]
[[package]]
name = "idna"
version = "3.18"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/cd/63/9496c57188a2ee585e0f1db071d75089a11e98aa86eb99d9d7618fc1edce/idna-3.18.tar.gz", hash = "sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848", size = 196711, upload-time = "2026-06-02T14:34:07.794Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" },
]
[[package]]
name = "iniconfig"
version = "2.3.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
]
[[package]]
name = "kaitaistruct"
version = "0.11"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/27/b8/ca7319556912f68832daa4b81425314857ec08dfccd8dbc8c0f65c992108/kaitaistruct-0.11.tar.gz", hash = "sha256:053ee764288e78b8e53acf748e9733268acbd579b8d82a427b1805453625d74b", size = 11519, upload-time = "2025-09-08T15:46:25.037Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/4a/4a/cf14bf3b1f5ffb13c69cf5f0ea78031247790558ee88984a8bdd22fae60d/kaitaistruct-0.11-py2.py3-none-any.whl", hash = "sha256:5c6ce79177b4e193a577ecd359e26516d1d6d000a0bffd6e1010f2a46a62a561", size = 11372, upload-time = "2025-09-08T15:46:23.635Z" },
]
[[package]]
name = "loguru"
version = "0.7.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32'" },
{ name = "win32-setctime", marker = "sys_platform == 'win32'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/3a/05/a1dae3dffd1116099471c643b8924f5aa6524411dc6c63fdae648c4f1aca/loguru-0.7.3.tar.gz", hash = "sha256:19480589e77d47b8d85b2c827ad95d49bf31b0dcde16593892eb51dd18706eb6", size = 63559, upload-time = "2024-12-06T11:20:56.608Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/0c/29/0348de65b8cc732daa3e33e67806420b2ae89bdce2b04af740289c5c6c8c/loguru-0.7.3-py3-none-any.whl", hash = "sha256:31a33c10c8e1e10422bfd431aeb5d351c7cf7fa671e3c4df004162264b28220c", size = 61595, upload-time = "2024-12-06T11:20:54.538Z" },
]
[[package]]
name = "maple"
version = "0.1.0"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },
{ name = "kaitaistruct" },
{ name = "loguru" },
{ name = "mixxx-analyzer" },
{ name = "mutagen" },
{ name = "pillow" },
{ name = "pyrekordbox" },
{ name = "scipy" },
]
[package.dev-dependencies]
dev = [
{ name = "pytest" },
]
[package.metadata]
requires-dist = [
{ name = "aiohttp", specifier = ">=3.10" },
{ name = "kaitaistruct", specifier = ">=0.11" },
{ name = "loguru", specifier = ">=0.7" },
{ name = "mixxx-analyzer", specifier = ">=0.1.2" },
{ name = "mutagen", specifier = ">=1.47.0" },
{ name = "pillow", specifier = ">=12.2.0" },
{ name = "pyrekordbox", specifier = ">=0.4.4" },
{ name = "scipy", specifier = ">=1.10" },
]
[package.metadata.requires-dev]
dev = [{ name = "pytest", specifier = ">=9.0.3" }]
[[package]]
name = "mixxx-analyzer"
version = "0.1.2"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/53/b3/c4026eed48e7685a99212ede8a1ce22bd2442a7d677ec4b970a7718d5b23/mixxx_analyzer-0.1.2-py3-none-macosx_14_0_arm64.whl", hash = "sha256:cb509c06d630fad709f85dd07fbd21a029d337617de0866af74279ca0a1f596b", size = 13932797, upload-time = "2026-03-01T15:44:59.349Z" },
{ url = "https://files.pythonhosted.org/packages/a6/30/bf8afbb52d862d2976045b4f6d00456dcb6320bb5e449384a41a0ce40735/mixxx_analyzer-0.1.2-py3-none-manylinux_2_35_x86_64.whl", hash = "sha256:53d8ec329bae7a7714dfe1cd34cbcf3976ddb55fae0ed74c4872f47085a9d9cf", size = 56567723, upload-time = "2026-03-01T15:45:02.771Z" },
{ url = "https://files.pythonhosted.org/packages/2e/88/e22c9e7c3d862daec693de61c5f2843ce8e567cd4d5e866040c599b57e9d/mixxx_analyzer-0.1.2-py3-none-win_amd64.whl", hash = "sha256:cc6509acfc78f0f3e04ac9f0b1440e9cd31a3c389f888bbd91beae1f249c32b7", size = 44947103, upload-time = "2026-03-01T15:45:05.66Z" },
]
[[package]]
name = "multidict"
version = "6.7.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/1a/c2/c2d94cbe6ac1753f3fc980da97b3d930efe1da3af3c9f5125354436c073d/multidict-6.7.1.tar.gz", hash = "sha256:ec6652a1bee61c53a3e5776b6049172c53b6aaba34f18c9ad04f82712bac623d", size = 102010, upload-time = "2026-01-26T02:46:45.979Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/91/cc/db74228a8be41884a567e88a62fd589a913708fcf180d029898c17a9a371/multidict-6.7.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:8f333ec9c5eb1b7105e3b84b53141e66ca05a19a605368c55450b6ba208cb9ee", size = 75190, upload-time = "2026-01-26T02:45:10.651Z" },
{ url = "https://files.pythonhosted.org/packages/d5/22/492f2246bb5b534abd44804292e81eeaf835388901f0c574bac4eeec73c5/multidict-6.7.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:a407f13c188f804c759fc6a9f88286a565c242a76b27626594c133b82883b5c2", size = 44486, upload-time = "2026-01-26T02:45:11.938Z" },
{ url = "https://files.pythonhosted.org/packages/f1/4f/733c48f270565d78b4544f2baddc2fb2a245e5a8640254b12c36ac7ac68e/multidict-6.7.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:0e161ddf326db5577c3a4cc2d8648f81456e8a20d40415541587a71620d7a7d1", size = 43219, upload-time = "2026-01-26T02:45:14.346Z" },
{ url = "https://files.pythonhosted.org/packages/24/bb/2c0c2287963f4259c85e8bcbba9182ced8d7fca65c780c38e99e61629d11/multidict-6.7.1-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:1e3a8bb24342a8201d178c3b4984c26ba81a577c80d4d525727427460a50c22d", size = 245132, upload-time = "2026-01-26T02:45:15.712Z" },
{ url = "https://files.pythonhosted.org/packages/a7/f9/44d4b3064c65079d2467888794dea218d1601898ac50222ab8a9a8094460/multidict-6.7.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:97231140a50f5d447d3164f994b86a0bed7cd016e2682f8650d6a9158e14fd31", size = 252420, upload-time = "2026-01-26T02:45:17.293Z" },
{ url = "https://files.pythonhosted.org/packages/8b/13/78f7275e73fa17b24c9a51b0bd9d73ba64bb32d0ed51b02a746eb876abe7/multidict-6.7.1-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:6b10359683bd8806a200fd2909e7c8ca3a7b24ec1d8132e483d58e791d881048", size = 233510, upload-time = "2026-01-26T02:45:19.356Z" },
{ url = "https://files.pythonhosted.org/packages/4b/25/8167187f62ae3cbd52da7893f58cb036b47ea3fb67138787c76800158982/multidict-6.7.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:283ddac99f7ac25a4acadbf004cb5ae34480bbeb063520f70ce397b281859362", size = 264094, upload-time = "2026-01-26T02:45:20.834Z" },
{ url = "https://files.pythonhosted.org/packages/a1/e7/69a3a83b7b030cf283fb06ce074a05a02322359783424d7edf0f15fe5022/multidict-6.7.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:538cec1e18c067d0e6103aa9a74f9e832904c957adc260e61cd9d8cf0c3b3d37", size = 260786, upload-time = "2026-01-26T02:45:22.818Z" },
{ url = "https://files.pythonhosted.org/packages/fe/3b/8ec5074bcfc450fe84273713b4b0a0dd47c0249358f5d82eb8104ffe2520/multidict-6.7.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7eee46ccb30ff48a1e35bb818cc90846c6be2b68240e42a78599166722cea709", size = 248483, upload-time = "2026-01-26T02:45:24.368Z" },
{ url = "https://files.pythonhosted.org/packages/48/5a/d5a99e3acbca0e29c5d9cba8f92ceb15dce78bab963b308ae692981e3a5d/multidict-6.7.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fa263a02f4f2dd2d11a7b1bb4362aa7cb1049f84a9235d31adf63f30143469a0", size = 248403, upload-time = "2026-01-26T02:45:25.982Z" },
{ url = "https://files.pythonhosted.org/packages/35/48/e58cd31f6c7d5102f2a4bf89f96b9cf7e00b6c6f3d04ecc44417c00a5a3c/multidict-6.7.1-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:2e1425e2f99ec5bd36c15a01b690a1a2456209c5deed58f95469ffb46039ccbb", size = 240315, upload-time = "2026-01-26T02:45:27.487Z" },
{ url = "https://files.pythonhosted.org/packages/94/33/1cd210229559cb90b6786c30676bb0c58249ff42f942765f88793b41fdce/multidict-6.7.1-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:497394b3239fc6f0e13a78a3e1b61296e72bf1c5f94b4c4eb80b265c37a131cd", size = 245528, upload-time = "2026-01-26T02:45:28.991Z" },
{ url = "https://files.pythonhosted.org/packages/64/f2/6e1107d226278c876c783056b7db43d800bb64c6131cec9c8dfb6903698e/multidict-6.7.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:233b398c29d3f1b9676b4b6f75c518a06fcb2ea0b925119fb2c1bc35c05e1601", size = 258784, upload-time = "2026-01-26T02:45:30.503Z" },
{ url = "https://files.pythonhosted.org/packages/4d/c1/11f664f14d525e4a1b5327a82d4de61a1db604ab34c6603bb3c2cc63ad34/multidict-6.7.1-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:93b1818e4a6e0930454f0f2af7dfce69307ca03cdcfb3739bf4d91241967b6c1", size = 251980, upload-time = "2026-01-26T02:45:32.603Z" },
{ url = "https://files.pythonhosted.org/packages/e1/9f/75a9ac888121d0c5bbd4ecf4eead45668b1766f6baabfb3b7f66a410e231/multidict-6.7.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:f33dc2a3abe9249ea5d8360f969ec7f4142e7ac45ee7014d8f8d5acddf178b7b", size = 243602, upload-time = "2026-01-26T02:45:34.043Z" },
{ url = "https://files.pythonhosted.org/packages/9a/e7/50bf7b004cc8525d80dbbbedfdc7aed3e4c323810890be4413e589074032/multidict-6.7.1-cp314-cp314-win32.whl", hash = "sha256:3ab8b9d8b75aef9df299595d5388b14530839f6422333357af1339443cff777d", size = 40930, upload-time = "2026-01-26T02:45:36.278Z" },
{ url = "https://files.pythonhosted.org/packages/e0/bf/52f25716bbe93745595800f36fb17b73711f14da59ed0bb2eba141bc9f0f/multidict-6.7.1-cp314-cp314-win_amd64.whl", hash = "sha256:5e01429a929600e7dab7b166062d9bb54a5eed752384c7384c968c2afab8f50f", size = 45074, upload-time = "2026-01-26T02:45:37.546Z" },
{ url = "https://files.pythonhosted.org/packages/97/ab/22803b03285fa3a525f48217963da3a65ae40f6a1b6f6cf2768879e208f9/multidict-6.7.1-cp314-cp314-win_arm64.whl", hash = "sha256:4885cb0e817aef5d00a2e8451d4665c1808378dc27c2705f1bf4ef8505c0d2e5", size = 42471, upload-time = "2026-01-26T02:45:38.889Z" },
{ url = "https://files.pythonhosted.org/packages/e0/6d/f9293baa6146ba9507e360ea0292b6422b016907c393e2f63fc40ab7b7b5/multidict-6.7.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:0458c978acd8e6ea53c81eefaddbbee9c6c5e591f41b3f5e8e194780fe026581", size = 82401, upload-time = "2026-01-26T02:45:40.254Z" },
{ url = "https://files.pythonhosted.org/packages/7a/68/53b5494738d83558d87c3c71a486504d8373421c3e0dbb6d0db48ad42ee0/multidict-6.7.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:c0abd12629b0af3cf590982c0b413b1e7395cd4ec026f30986818ab95bfaa94a", size = 48143, upload-time = "2026-01-26T02:45:41.635Z" },
{ url = "https://files.pythonhosted.org/packages/37/e8/5284c53310dcdc99ce5d66563f6e5773531a9b9fe9ec7a615e9bc306b05f/multidict-6.7.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:14525a5f61d7d0c94b368a42cff4c9a4e7ba2d52e2672a7b23d84dc86fb02b0c", size = 46507, upload-time = "2026-01-26T02:45:42.99Z" },
{ url = "https://files.pythonhosted.org/packages/e4/fc/6800d0e5b3875568b4083ecf5f310dcf91d86d52573160834fb4bfcf5e4f/multidict-6.7.1-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:17307b22c217b4cf05033dabefe68255a534d637c6c9b0cc8382718f87be4262", size = 239358, upload-time = "2026-01-26T02:45:44.376Z" },
{ url = "https://files.pythonhosted.org/packages/41/75/4ad0973179361cdf3a113905e6e088173198349131be2b390f9fa4da5fc6/multidict-6.7.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7a7e590ff876a3eaf1c02a4dfe0724b6e69a9e9de6d8f556816f29c496046e59", size = 246884, upload-time = "2026-01-26T02:45:47.167Z" },
{ url = "https://files.pythonhosted.org/packages/c3/9c/095bb28b5da139bd41fb9a5d5caff412584f377914bd8787c2aa98717130/multidict-6.7.1-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:5fa6a95dfee63893d80a34758cd0e0c118a30b8dcb46372bf75106c591b77889", size = 225878, upload-time = "2026-01-26T02:45:48.698Z" },
{ url = "https://files.pythonhosted.org/packages/07/d0/c0a72000243756e8f5a277b6b514fa005f2c73d481b7d9e47cd4568aa2e4/multidict-6.7.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a0543217a6a017692aa6ae5cc39adb75e587af0f3a82288b1492eb73dd6cc2a4", size = 253542, upload-time = "2026-01-26T02:45:50.164Z" },
{ url = "https://files.pythonhosted.org/packages/c0/6b/f69da15289e384ecf2a68837ec8b5ad8c33e973aa18b266f50fe55f24b8c/multidict-6.7.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f99fe611c312b3c1c0ace793f92464d8cd263cc3b26b5721950d977b006b6c4d", size = 252403, upload-time = "2026-01-26T02:45:51.779Z" },
{ url = "https://files.pythonhosted.org/packages/a2/76/b9669547afa5a1a25cd93eaca91c0da1c095b06b6d2d8ec25b713588d3a1/multidict-6.7.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9004d8386d133b7e6135679424c91b0b854d2d164af6ea3f289f8f2761064609", size = 244889, upload-time = "2026-01-26T02:45:53.27Z" },
{ url = "https://files.pythonhosted.org/packages/7e/a9/a50d2669e506dad33cfc45b5d574a205587b7b8a5f426f2fbb2e90882588/multidict-6.7.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e628ef0e6859ffd8273c69412a2465c4be4a9517d07261b33334b5ec6f3c7489", size = 241982, upload-time = "2026-01-26T02:45:54.919Z" },
{ url = "https://files.pythonhosted.org/packages/c5/bb/1609558ad8b456b4827d3c5a5b775c93b87878fd3117ed3db3423dfbce1b/multidict-6.7.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:841189848ba629c3552035a6a7f5bf3b02eb304e9fea7492ca220a8eda6b0e5c", size = 232415, upload-time = "2026-01-26T02:45:56.981Z" },
{ url = "https://files.pythonhosted.org/packages/d8/59/6f61039d2aa9261871e03ab9dc058a550d240f25859b05b67fd70f80d4b3/multidict-6.7.1-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:ce1bbd7d780bb5a0da032e095c951f7014d6b0a205f8318308140f1a6aba159e", size = 240337, upload-time = "2026-01-26T02:45:58.698Z" },
{ url = "https://files.pythonhosted.org/packages/a1/29/fdc6a43c203890dc2ae9249971ecd0c41deaedfe00d25cb6564b2edd99eb/multidict-6.7.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:b26684587228afed0d50cf804cc71062cc9c1cdf55051c4c6345d372947b268c", size = 248788, upload-time = "2026-01-26T02:46:00.862Z" },
{ url = "https://files.pythonhosted.org/packages/a9/14/a153a06101323e4cf086ecee3faadba52ff71633d471f9685c42e3736163/multidict-6.7.1-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:9f9af11306994335398293f9958071019e3ab95e9a707dc1383a35613f6abcb9", size = 242842, upload-time = "2026-01-26T02:46:02.824Z" },
{ url = "https://files.pythonhosted.org/packages/41/5f/604ae839e64a4a6efc80db94465348d3b328ee955e37acb24badbcd24d83/multidict-6.7.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:b4938326284c4f1224178a560987b6cf8b4d38458b113d9b8c1db1a836e640a2", size = 240237, upload-time = "2026-01-26T02:46:05.898Z" },
{ url = "https://files.pythonhosted.org/packages/5f/60/c3a5187bf66f6fb546ff4ab8fb5a077cbdd832d7b1908d4365c7f74a1917/multidict-6.7.1-cp314-cp314t-win32.whl", hash = "sha256:98655c737850c064a65e006a3df7c997cd3b220be4ec8fe26215760b9697d4d7", size = 48008, upload-time = "2026-01-26T02:46:07.468Z" },
{ url = "https://files.pythonhosted.org/packages/0c/f7/addf1087b860ac60e6f382240f64fb99f8bfb532bb06f7c542b83c29ca61/multidict-6.7.1-cp314-cp314t-win_amd64.whl", hash = "sha256:497bde6223c212ba11d462853cfa4f0ae6ef97465033e7dc9940cdb3ab5b48e5", size = 53542, upload-time = "2026-01-26T02:46:08.809Z" },
{ url = "https://files.pythonhosted.org/packages/4c/81/4629d0aa32302ef7b2ec65c75a728cc5ff4fa410c50096174c1632e70b3e/multidict-6.7.1-cp314-cp314t-win_arm64.whl", hash = "sha256:2bbd113e0d4af5db41d5ebfe9ccaff89de2120578164f86a5d17d5a576d1e5b2", size = 44719, upload-time = "2026-01-26T02:46:11.146Z" },
{ url = "https://files.pythonhosted.org/packages/81/08/7036c080d7117f28a4af526d794aab6a84463126db031b007717c1a6676e/multidict-6.7.1-py3-none-any.whl", hash = "sha256:55d97cc6dae627efa6a6e548885712d4864b81110ac76fa4e534c03819fa4a56", size = 12319, upload-time = "2026-01-26T02:46:44.004Z" },
]
[[package]]
name = "mutagen"
version = "1.47.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/81/e6/64bc71b74eef4b68e61eb921dcf72dabd9e4ec4af1e11891bbd312ccbb77/mutagen-1.47.0.tar.gz", hash = "sha256:719fadef0a978c31b4cf3c956261b3c58b6948b32023078a2117b1de09f0fc99", size = 1274186, upload-time = "2023-09-03T16:33:33.411Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b0/7a/620f945b96be1f6ee357d211d5bf74ab1b7fe72a9f1525aafbfe3aee6875/mutagen-1.47.0-py3-none-any.whl", hash = "sha256:edd96f50c5907a9539d8e5bba7245f62c9f520aef333d13392a79a4f70aca719", size = 194391, upload-time = "2023-09-03T16:33:29.955Z" },
]
[[package]]
name = "numpy"
version = "2.4.6"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d0/ad/fed0499ce6a338d2a03ebae59cd15093910c8875328855781952abf6c2fe/numpy-2.4.6.tar.gz", hash = "sha256:f3a3570c4a2a16746ac2c31a7c7c7b0c186b95ce902e33db6f28094ed7387dda", size = 20735807, upload-time = "2026-05-18T23:37:14.07Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f8/91/3ab2044d05fd16d343c5ac2e69b127f1b2854040dd20b193257c78028bd3/numpy-2.4.6-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:06ca2f61ec4385a07a6977c55ba998a4466c123642b4a32694d3128fce18c079", size = 16683458, upload-time = "2026-05-18T23:35:38.353Z" },
{ url = "https://files.pythonhosted.org/packages/8e/62/764ce66fa4147ae6d73071a3abf804ffe606f174618697c571acdf26a7c9/numpy-2.4.6-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:38efbc8de75c7a0fc1ac190162d892787f3f47b57cc291231aafee36b80982b7", size = 14704559, upload-time = "2026-05-18T23:35:42.14Z" },
{ url = "https://files.pythonhosted.org/packages/60/61/23f27c172f022e04025b7dc2367f4d63c1a398120607ec896228649a6f48/numpy-2.4.6-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:d581b735e177fdcdce6fed8e7e8880a3fb6ee4e3653a3ac6af01c6f4c03effc5", size = 5209716, upload-time = "2026-05-18T23:35:45.377Z" },
{ url = "https://files.pythonhosted.org/packages/03/71/21cf70dc6ea3e3acb95fc53a265b2fc248b981f0194ceb5b475271b8809d/numpy-2.4.6-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:0a041d3d761dc3c35cc56ce0351506a02bcbc25f7b169f652435141a17db9096", size = 6543947, upload-time = "2026-05-18T23:35:47.926Z" },
{ url = "https://files.pythonhosted.org/packages/d5/91/64288395ee1799bd2e0b04a305dce9666da90c961e1f3fe982a05ee1c036/numpy-2.4.6-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:40fdc1ae7125e518ea98e53e69a4ebc27e1fd50510c47b7ea130cf21e5e1d42b", size = 15685197, upload-time = "2026-05-18T23:35:50.863Z" },
{ url = "https://files.pythonhosted.org/packages/f3/eb/ebffaa97dc55502df69584a8f0dcf07f69a3e0b3e2323670a2722db9aa39/numpy-2.4.6-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a2c306dea656c12c68f51f4cea133cbe78ca7435eb28c735eac1d3ebe73be6e8", size = 16638245, upload-time = "2026-05-18T23:35:54.752Z" },
{ url = "https://files.pythonhosted.org/packages/b8/0b/54f9da33128d7e350fab89c7455902eeae70349ee52bddb448dc4a576f45/numpy-2.4.6-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:33111801a01c12a8a1e3721f0a9232f8cfc8ae2c6b7098167e6f623c6073f402", size = 17036587, upload-time = "2026-05-18T23:35:58.355Z" },
{ url = "https://files.pythonhosted.org/packages/b6/f0/fdebc1052db1cc37c64beb22072d67cd6d1c71adca1299f53dec2b5e20d3/numpy-2.4.6-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ae506e6902902557576a26ff33eda8695e7ecb3cb36c3b573a0765dee114ebdb", size = 18363226, upload-time = "2026-05-18T23:36:02.845Z" },
{ url = "https://files.pythonhosted.org/packages/aa/b4/298628d98c72b57e57f7165ae6a481a1deaf6f3c28262a6e4c739c275930/numpy-2.4.6-cp314-cp314-win32.whl", hash = "sha256:aaf159caa35993cb1f56fb9b8e4610d35758e7ca005412eb1daa856a78c9c4b1", size = 6010196, upload-time = "2026-05-18T23:36:05.92Z" },
{ url = "https://files.pythonhosted.org/packages/df/ac/46de6dda46478f7942f839e094970be2d4a861e005c4b3bf07c92e291a09/numpy-2.4.6-cp314-cp314-win_amd64.whl", hash = "sha256:b507f5c4c1d508876d1819b6bf9a49d365b96320b5d4993426b33a23ca4b8261", size = 12450334, upload-time = "2026-05-18T23:36:09.107Z" },
{ url = "https://files.pythonhosted.org/packages/78/92/b8b798ac784102c0da830d2257d59358e3d3d90d1e2b3f2575dad976c5cf/numpy-2.4.6-cp314-cp314-win_arm64.whl", hash = "sha256:6f41ae150c4e32db4f3310cdaf64b1593a03dbabe29eec77fc9b50fe64061df6", size = 10495678, upload-time = "2026-05-18T23:36:12.766Z" },
{ url = "https://files.pythonhosted.org/packages/30/34/ec28d1aa8115971537c01469ab2011ee96827930f0a124de1000cc2a7ed7/numpy-2.4.6-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ece3d2cfe132e7d51f44a832b303895e6f2d499c5e74dfbdb06ee246147a304a", size = 14823672, upload-time = "2026-05-18T23:36:16.473Z" },
{ url = "https://files.pythonhosted.org/packages/16/bd/f6d1fede4e54e8042a7ff97bb495510f3c220f94bcd9e8b228e87c92cc0d/numpy-2.4.6-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:e3e5193ef5a3dc73bceee50f7fdc2c90dbb76c42df8d8fae3d1067a583df579e", size = 5328731, upload-time = "2026-05-18T23:36:19.767Z" },
{ url = "https://files.pythonhosted.org/packages/f4/f0/e105b9e2fd728a9910103884decd6951d9dd73896b914a98d9a231de02ee/numpy-2.4.6-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:17f9ade344e7d9b464a084d69bcf18fc691cb1db67c62ed80820bf4926d78f0e", size = 6649805, upload-time = "2026-05-18T23:36:22.266Z" },
{ url = "https://files.pythonhosted.org/packages/82/dd/1206a7ca6ab15e3f02069707ca96222e202af681bb73756da7527f3cb837/numpy-2.4.6-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9cd5ffd25db4e7ba6a375693b3fc0fc1791ec636c17db3720da19bde7180ec43", size = 15730496, upload-time = "2026-05-18T23:36:25.713Z" },
{ url = "https://files.pythonhosted.org/packages/51/e7/38d3ea825dcab85a591734decb2f6c67caa7c8367d374df1a1c3842f9b07/numpy-2.4.6-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7d92c3819208a60205a12a245c91ad70cb0a85336659b19b834205573ac8456e", size = 16679616, upload-time = "2026-05-18T23:36:29.652Z" },
{ url = "https://files.pythonhosted.org/packages/93/b7/caabfdf53edf663e0b4eb74d7d405d83baef09eb5e83bcd32d601d72b93e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e85b752a1e912b70eaad4fafbd4d1238007ab221de2009b9a2f5ae7461239895", size = 17085145, upload-time = "2026-05-18T23:36:33.449Z" },
{ url = "https://files.pythonhosted.org/packages/f9/45/68d7c33a6bcf3e5aa3bdbd57a367e6f615286dfd6482f97e8ffeb734306e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:29cb7f67d10b479ff07c17d33e39f78c07f71c40ef30d63c153d340e96cd3fb4", size = 18403813, upload-time = "2026-05-18T23:36:37.369Z" },
{ url = "https://files.pythonhosted.org/packages/9c/50/0753655aa844c99cd9e018aacf76f130f1bd81d881bb74bc0aef5d73a8ba/numpy-2.4.6-cp314-cp314t-win32.whl", hash = "sha256:260a5d70215b61ab4fadf5c7baacd64821842975eea312125ed3c39a6391b063", size = 6156982, upload-time = "2026-05-18T23:36:40.817Z" },
{ url = "https://files.pythonhosted.org/packages/b2/d4/7c67becf668f973cb490cec3e98dfd799d866f9c989a54d355672cfa0db6/numpy-2.4.6-cp314-cp314t-win_amd64.whl", hash = "sha256:81a1cca95ed5bb92aa8b10dd2cdc9a0d3853a50fad926c28b5d7e8ea54389627", size = 12638908, upload-time = "2026-05-18T23:36:43.996Z" },
{ url = "https://files.pythonhosted.org/packages/43/bb/e1c71a4295b1b1d1393d50dbb4f2a36283c6859d9d3892e84f00ec5a91d5/numpy-2.4.6-cp314-cp314t-win_arm64.whl", hash = "sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66", size = 10565867, upload-time = "2026-05-18T23:36:47.114Z" },
]
[[package]]
name = "packaging"
version = "26.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", size = 228134, upload-time = "2026-04-24T20:15:23.917Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" },
]
[[package]]
name = "pillow"
version = "12.2.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/8c/21/c2bcdd5906101a30244eaffc1b6e6ce71a31bd0742a01eb89e660ebfac2d/pillow-12.2.0.tar.gz", hash = "sha256:a830b1a40919539d07806aa58e1b114df53ddd43213d9c8b75847eee6c0182b5", size = 46987819, upload-time = "2026-04-01T14:46:17.687Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/bf/98/4595daa2365416a86cb0d495248a393dfc84e96d62ad080c8546256cb9c0/pillow-12.2.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:3adc9215e8be0448ed6e814966ecf3d9952f0ea40eb14e89a102b87f450660d8", size = 4100848, upload-time = "2026-04-01T14:44:48.48Z" },
{ url = "https://files.pythonhosted.org/packages/0b/79/40184d464cf89f6663e18dfcf7ca21aae2491fff1a16127681bf1fa9b8cf/pillow-12.2.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:6a9adfc6d24b10f89588096364cc726174118c62130c817c2837c60cf08a392b", size = 4176515, upload-time = "2026-04-01T14:44:51.353Z" },
{ url = "https://files.pythonhosted.org/packages/b0/63/703f86fd4c422a9cf722833670f4f71418fb116b2853ff7da722ea43f184/pillow-12.2.0-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:6a6e67ea2e6feda684ed370f9a1c52e7a243631c025ba42149a2cc5934dec295", size = 3640159, upload-time = "2026-04-01T14:44:53.588Z" },
{ url = "https://files.pythonhosted.org/packages/71/e0/fb22f797187d0be2270f83500aab851536101b254bfa1eae10795709d283/pillow-12.2.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:2bb4a8d594eacdfc59d9e5ad972aa8afdd48d584ffd5f13a937a664c3e7db0ed", size = 5312185, upload-time = "2026-04-01T14:44:56.039Z" },
{ url = "https://files.pythonhosted.org/packages/ba/8c/1a9e46228571de18f8e28f16fabdfc20212a5d019f3e3303452b3f0a580d/pillow-12.2.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:80b2da48193b2f33ed0c32c38140f9d3186583ce7d516526d462645fd98660ae", size = 4695386, upload-time = "2026-04-01T14:44:58.663Z" },
{ url = "https://files.pythonhosted.org/packages/70/62/98f6b7f0c88b9addd0e87c217ded307b36be024d4ff8869a812b241d1345/pillow-12.2.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:22db17c68434de69d8ecfc2fe821569195c0c373b25cccb9cbdacf2c6e53c601", size = 6280384, upload-time = "2026-04-01T14:45:01.5Z" },
{ url = "https://files.pythonhosted.org/packages/5e/03/688747d2e91cfbe0e64f316cd2e8005698f76ada3130d0194664174fa5de/pillow-12.2.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7b14cc0106cd9aecda615dd6903840a058b4700fcb817687d0ee4fc8b6e389be", size = 8091599, upload-time = "2026-04-01T14:45:04.5Z" },
{ url = "https://files.pythonhosted.org/packages/f6/35/577e22b936fcdd66537329b33af0b4ccfefaeabd8aec04b266528cddb33c/pillow-12.2.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8cbeb542b2ebc6fcdacabf8aca8c1a97c9b3ad3927d46b8723f9d4f033288a0f", size = 6396021, upload-time = "2026-04-01T14:45:07.117Z" },
{ url = "https://files.pythonhosted.org/packages/11/8d/d2532ad2a603ca2b93ad9f5135732124e57811d0168155852f37fbce2458/pillow-12.2.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4bfd07bc812fbd20395212969e41931001fd59eb55a60658b0e5710872e95286", size = 7083360, upload-time = "2026-04-01T14:45:09.763Z" },
{ url = "https://files.pythonhosted.org/packages/5e/26/d325f9f56c7e039034897e7380e9cc202b1e368bfd04d4cbe6a441f02885/pillow-12.2.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:9aba9a17b623ef750a4d11b742cbafffeb48a869821252b30ee21b5e91392c50", size = 6507628, upload-time = "2026-04-01T14:45:12.378Z" },
{ url = "https://files.pythonhosted.org/packages/5f/f7/769d5632ffb0988f1c5e7660b3e731e30f7f8ec4318e94d0a5d674eb65a4/pillow-12.2.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:deede7c263feb25dba4e82ea23058a235dcc2fe1f6021025dc71f2b618e26104", size = 7209321, upload-time = "2026-04-01T14:45:15.122Z" },
{ url = "https://files.pythonhosted.org/packages/6a/7a/c253e3c645cd47f1aceea6a8bacdba9991bf45bb7dfe927f7c893e89c93c/pillow-12.2.0-cp314-cp314-win32.whl", hash = "sha256:632ff19b2778e43162304d50da0181ce24ac5bb8180122cbe1bf4673428328c7", size = 6479723, upload-time = "2026-04-01T14:45:17.797Z" },
{ url = "https://files.pythonhosted.org/packages/cd/8b/601e6566b957ca50e28725cb6c355c59c2c8609751efbecd980db44e0349/pillow-12.2.0-cp314-cp314-win_amd64.whl", hash = "sha256:4e6c62e9d237e9b65fac06857d511e90d8461a32adcc1b9065ea0c0fa3a28150", size = 7217400, upload-time = "2026-04-01T14:45:20.529Z" },
{ url = "https://files.pythonhosted.org/packages/d6/94/220e46c73065c3e2951bb91c11a1fb636c8c9ad427ac3ce7d7f3359b9b2f/pillow-12.2.0-cp314-cp314-win_arm64.whl", hash = "sha256:b1c1fbd8a5a1af3412a0810d060a78b5136ec0836c8a4ef9aa11807f2a22f4e1", size = 2554835, upload-time = "2026-04-01T14:45:23.162Z" },
{ url = "https://files.pythonhosted.org/packages/b6/ab/1b426a3974cb0e7da5c29ccff4807871d48110933a57207b5a676cccc155/pillow-12.2.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:57850958fe9c751670e49b2cecf6294acc99e562531f4bd317fa5ddee2068463", size = 5314225, upload-time = "2026-04-01T14:45:25.637Z" },
{ url = "https://files.pythonhosted.org/packages/19/1e/dce46f371be2438eecfee2a1960ee2a243bbe5e961890146d2dee1ff0f12/pillow-12.2.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:d5d38f1411c0ed9f97bcb49b7bd59b6b7c314e0e27420e34d99d844b9ce3b6f3", size = 4698541, upload-time = "2026-04-01T14:45:28.355Z" },
{ url = "https://files.pythonhosted.org/packages/55/c3/7fbecf70adb3a0c33b77a300dc52e424dc22ad8cdc06557a2e49523b703d/pillow-12.2.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:5c0a9f29ca8e79f09de89293f82fc9b0270bb4af1d58bc98f540cc4aedf03166", size = 6322251, upload-time = "2026-04-01T14:45:30.924Z" },
{ url = "https://files.pythonhosted.org/packages/1c/3c/7fbc17cfb7e4fe0ef1642e0abc17fc6c94c9f7a16be41498e12e2ba60408/pillow-12.2.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1610dd6c61621ae1cf811bef44d77e149ce3f7b95afe66a4512f8c59f25d9ebe", size = 8127807, upload-time = "2026-04-01T14:45:33.908Z" },
{ url = "https://files.pythonhosted.org/packages/ff/c3/a8ae14d6defd2e448493ff512fae903b1e9bd40b72efb6ec55ce0048c8ce/pillow-12.2.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0a34329707af4f73cf1782a36cd2289c0368880654a2c11f027bcee9052d35dd", size = 6433935, upload-time = "2026-04-01T14:45:36.623Z" },
{ url = "https://files.pythonhosted.org/packages/6e/32/2880fb3a074847ac159d8f902cb43278a61e85f681661e7419e6596803ed/pillow-12.2.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8e9c4f5b3c546fa3458a29ab22646c1c6c787ea8f5ef51300e5a60300736905e", size = 7116720, upload-time = "2026-04-01T14:45:39.258Z" },
{ url = "https://files.pythonhosted.org/packages/46/87/495cc9c30e0129501643f24d320076f4cc54f718341df18cc70ec94c44e1/pillow-12.2.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:fb043ee2f06b41473269765c2feae53fc2e2fbf96e5e22ca94fb5ad677856f06", size = 6540498, upload-time = "2026-04-01T14:45:41.879Z" },
{ url = "https://files.pythonhosted.org/packages/18/53/773f5edca692009d883a72211b60fdaf8871cbef075eaa9d577f0a2f989e/pillow-12.2.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:f278f034eb75b4e8a13a54a876cc4a5ab39173d2cdd93a638e1b467fc545ac43", size = 7239413, upload-time = "2026-04-01T14:45:44.705Z" },
{ url = "https://files.pythonhosted.org/packages/c9/e4/4b64a97d71b2a83158134abbb2f5bd3f8a2ea691361282f010998f339ec7/pillow-12.2.0-cp314-cp314t-win32.whl", hash = "sha256:6bb77b2dcb06b20f9f4b4a8454caa581cd4dd0643a08bacf821216a16d9c8354", size = 6482084, upload-time = "2026-04-01T14:45:47.568Z" },
{ url = "https://files.pythonhosted.org/packages/ba/13/306d275efd3a3453f72114b7431c877d10b1154014c1ebbedd067770d629/pillow-12.2.0-cp314-cp314t-win_amd64.whl", hash = "sha256:6562ace0d3fb5f20ed7290f1f929cae41b25ae29528f2af1722966a0a02e2aa1", size = 7225152, upload-time = "2026-04-01T14:45:50.032Z" },
{ url = "https://files.pythonhosted.org/packages/ff/6e/cf826fae916b8658848d7b9f38d88da6396895c676e8086fc0988073aaf8/pillow-12.2.0-cp314-cp314t-win_arm64.whl", hash = "sha256:aa88ccfe4e32d362816319ed727a004423aab09c5cea43c01a4b435643fa34eb", size = 2556579, upload-time = "2026-04-01T14:45:52.529Z" },
]
[[package]]
name = "pluggy"
version = "1.6.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
]
[[package]]
name = "propcache"
version = "0.5.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/ec/44/c87281c333769159c50594f22610f77398a47ccbfbbf23074e744e86f87c/propcache-0.5.2.tar.gz", hash = "sha256:01c4fc7480cd0598bb4b57022df55b9ca296da7fc5a8760bd8451a7e63a7d427", size = 50208, upload-time = "2026-05-08T21:02:12.199Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e2/ea/23ee535d90ce8bcc465a3028eb3cc0ce3bd1005f4bb27710b30587de798d/propcache-0.5.2-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:46088abff4cba581dea21ae0467a480526cb25aa5f3c269e909f800328bc3999", size = 94662, upload-time = "2026-05-08T21:01:22.683Z" },
{ url = "https://files.pythonhosted.org/packages/b5/06/c5a52f419b5d8972f8d46a7577476090d8e3263ff589ce40b5ca4968d5be/propcache-0.5.2-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:fc88b26f08d634f7bc819a7852e5214f5802641ab8d9fd5326892292eee1993e", size = 53928, upload-time = "2026-05-08T21:01:23.986Z" },
{ url = "https://files.pythonhosted.org/packages/63/b1/4260d67d6bd85e58a66b72d54ce15d5de789b6f3870cc6bedf8ff9667401/propcache-0.5.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:97797ebb098e670a2f92dd66f32897e30d7615b14e7f59711de23e30a9072539", size = 54650, upload-time = "2026-05-08T21:01:25.305Z" },
{ url = "https://files.pythonhosted.org/packages/70/06/2f46c318e3307cd7a6a7481def374ce838c0fe20084b39dd54b0879d0e99/propcache-0.5.2-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ba57fffe4ac99c5d30076161b5866336d97600769bad35cc68f7774b15298a4e", size = 59912, upload-time = "2026-05-08T21:01:26.545Z" },
{ url = "https://files.pythonhosted.org/packages/4c/29/fe1aebec2ce57ab985a9c382bded1124431f85078113aa222c5d278430d4/propcache-0.5.2-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:583c19759d9eec1e5b69e2fbef36a7d9c326041be9746cb822d335c8cedc2979", size = 63300, upload-time = "2026-05-08T21:01:27.937Z" },
{ url = "https://files.pythonhosted.org/packages/b4/18/2334b26768b6c82be8c69e83671b767d5ef426aa09b0cba6c2ea47816774/propcache-0.5.2-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d0326e2e5e1f3163fa306c834e48e8d490e5fae607a097a40c0648109b47ba80", size = 64208, upload-time = "2026-05-08T21:01:29.484Z" },
{ url = "https://files.pythonhosted.org/packages/2b/76/7f1bfd6afff4c5e38e36a3c6d68eb5f4b7311ea80baf693db78d95b603c4/propcache-0.5.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e00820e192c8dbebcafb383ebbf99030895f09905e7a0eb2e0340a0bcc2bc825", size = 61633, upload-time = "2026-05-08T21:01:31.068Z" },
{ url = "https://files.pythonhosted.org/packages/c4/46/b3ff8aba2b4953a3e50de2cf72f1b5748b8eca93b15f3dc2c84339084c09/propcache-0.5.2-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c66afea89b1e43725731d2004732a046fe6fe955d51f952c3e95a7314a284a39", size = 61724, upload-time = "2026-05-08T21:01:32.374Z" },
{ url = "https://files.pythonhosted.org/packages/c5/01/814cfcafbcff954f94c01cf30e097ddc88a076b5440fbcf4570753437d40/propcache-0.5.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:d4dc37dec6c6cdad0b57881a5658fd14fbf53e333b1a86cf86559f190e1d9ec4", size = 60069, upload-time = "2026-05-08T21:01:33.67Z" },
{ url = "https://files.pythonhosted.org/packages/da/68/5c6f7622d510cc666a300687e06fd060c1a43361c0c9b20d284f06d8096a/propcache-0.5.2-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:5570dbcc97571c15f68068e529c92715a12f8d54030e272d264b377e22bd17a5", size = 57099, upload-time = "2026-05-08T21:01:34.915Z" },
{ url = "https://files.pythonhosted.org/packages/55/27/9cb0b4c679124085327957d42521c99dba04c88c90c3e55a6f0b633ebccc/propcache-0.5.2-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:f814362777a9f841adddb200ecdf8f5cb1e5a3c4b7a86378edbd6ccb26edd702", size = 63391, upload-time = "2026-05-08T21:01:36.231Z" },
{ url = "https://files.pythonhosted.org/packages/f0/9d/7258aaa5bdf60fc6f27591eef6fe52768cb0beda7140be477c8b12c9794a/propcache-0.5.2-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:196913dea116aeb5a2ba95af4ddcb7ea85559ae07d8eee8751688310d09168c3", size = 61626, upload-time = "2026-05-08T21:01:37.545Z" },
{ url = "https://files.pythonhosted.org/packages/8e/0d/41c602003e8a9b16fe1e7eadf62c7bfba9d5474370b24200bf48b315f45f/propcache-0.5.2-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:6e7b8719005dd1175be4ab1cd25e9b98659a5e0347331506ec6760d2773a7fb5", size = 64781, upload-time = "2026-05-08T21:01:38.83Z" },
{ url = "https://files.pythonhosted.org/packages/8b/f3/38e66b1856e9bd079deea015bc4a55f7767c0e4db2f7dcf69e7e680ba4ce/propcache-0.5.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:51f96d685ab16e88cab128cd37a52c5da540809c8b879fa047731bfcb4ad35a4", size = 62570, upload-time = "2026-05-08T21:01:40.415Z" },
{ url = "https://files.pythonhosted.org/packages/95/ca/bbfe9b910ce57dde8bb4876b4520fc02a4e89497c10de26be936758a3aaa/propcache-0.5.2-cp314-cp314-win32.whl", hash = "sha256:cc6fc3cc62e8501d3ed62894425040d2728ecddb1ed072737a5c70bd537aa9f0", size = 39436, upload-time = "2026-05-08T21:01:41.654Z" },
{ url = "https://files.pythonhosted.org/packages/61/d2/45c9defbaa1ea297035d9d4cce9e8f80daafbf19319c6007f157c6256ea9/propcache-0.5.2-cp314-cp314-win_amd64.whl", hash = "sha256:81e3a30b0bb60caa22033dd0f8a3618d1d67356212514f62c57db75cb0ef410c", size = 42373, upload-time = "2026-05-08T21:01:43.041Z" },
{ url = "https://files.pythonhosted.org/packages/44/68/9ea5103f41d5217d7d6ec24db90018e23aebec070c3f9a6e54d12b841fd8/propcache-0.5.2-cp314-cp314-win_arm64.whl", hash = "sha256:0d2c9bf8528f135dbb805ce027567e09164f7efa51a2be07458a2c0420f292d0", size = 38554, upload-time = "2026-05-08T21:01:44.336Z" },
{ url = "https://files.pythonhosted.org/packages/8a/81/fadf555f42d3b762eea8a53950b0489fdc0aa9da5f8ed9e10ce0a4e01b48/propcache-0.5.2-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:4bc8ff1feffc6a61c7002ffe84634c41b822e104990ae009f44a0834430070bb", size = 99395, upload-time = "2026-05-08T21:01:45.883Z" },
{ url = "https://files.pythonhosted.org/packages/f5/c9/c61e134a686949cf7971af3a390148b1156f7be81c73bc0cd12c873e2d48/propcache-0.5.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:79aa3ff0a9b566633b642fa9caf7e21ed1c13d6feca718187873f199e1514078", size = 56653, upload-time = "2026-05-08T21:01:47.307Z" },
{ url = "https://files.pythonhosted.org/packages/cb/73/daf935ea7048ddd7ec8eec5345b4a40b619d2d178b3c0a0900796bc3c794/propcache-0.5.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1b31822f4474c4036bae62de9402710051d431a606d6a0f907fec79935a071aa", size = 56914, upload-time = "2026-05-08T21:01:48.573Z" },
{ url = "https://files.pythonhosted.org/packages/79/9f/aba959b435ea18617edd7cf0a7ad0b9c574b8fc7e3d2cd55fb59cb255d33/propcache-0.5.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:13fef48778b5a2a756523fdb781326b028ca75e32858b04f2cdd19f394564917", size = 62567, upload-time = "2026-05-08T21:01:49.903Z" },
{ url = "https://files.pythonhosted.org/packages/6c/a1/859942de9a791ff42f6141736f5b37749b8f53e65edfa49638c67dd67e6a/propcache-0.5.2-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:8b73ab70f1a3351fbc71f663b3e645af6dd0329100c353081cf69c37433fc6fe", size = 65542, upload-time = "2026-05-08T21:01:51.204Z" },
{ url = "https://files.pythonhosted.org/packages/b5/61/315bc0fd6c0fc7f80a528b8afd209e5fc4a875ea79571b91b8f50f442907/propcache-0.5.2-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5538d2c13d93e4698af7e092b57bc7298fd35d1d58e656ae18f23ee0d0378e03", size = 66845, upload-time = "2026-05-08T21:01:52.539Z" },
{ url = "https://files.pythonhosted.org/packages/47/f7/9f8122e3132e8e354ac41975ef8f1099be7d5a16bc7ae562734e993665c0/propcache-0.5.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cd645f03898405cabe694fb8bc35241e3a9c332ec85627584fe3de201452b335", size = 63985, upload-time = "2026-05-08T21:01:53.847Z" },
{ url = "https://files.pythonhosted.org/packages/c8/54/c317819ec157cbf6f35df9df9657a6f82daf34d5faf15948b2f639c2192e/propcache-0.5.2-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a473b3440261e0c60706e732b2ed2f517857344fc21bf48fdfe211e2d98eb285", size = 63999, upload-time = "2026-05-08T21:01:55.179Z" },
{ url = "https://files.pythonhosted.org/packages/5a/56/387e3f7dfce0a9233df41fb888aa1c30222cb4bbbf09537c02dd9bd85fe2/propcache-0.5.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:7afa37062e6650640e932e4cc9297d81f9f42d9944029cc386b8247dea4da837", size = 62779, upload-time = "2026-05-08T21:01:57.489Z" },
{ url = "https://files.pythonhosted.org/packages/a1/9c/596784cb5824ed61ee960d3f8655a3f0993e107c6e98ab6c818b7fb92ccb/propcache-0.5.2-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:8a90efd5777e996e42d568db9ac740b944d691e565cbfd31b2f7832f9184b2b8", size = 59796, upload-time = "2026-05-08T21:01:58.736Z" },
{ url = "https://files.pythonhosted.org/packages/c2/3d/1a6cfa1726a48542c1e8784a0761421476a5b68e09b7f36bf95eb954aaba/propcache-0.5.2-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:f19bb891234d72535764d703bfed1153cc34f4214d5bd7150aee1eec9e8f4366", size = 66023, upload-time = "2026-05-08T21:02:00.228Z" },
{ url = "https://files.pythonhosted.org/packages/e4/0e/05fd6990369477076e4e280bcb970de760fddf0161a46e988bc95f7940ec/propcache-0.5.2-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:32775082acd2d807ee3db715c7770d38767b817870acfa08c29e057f3c4d5b56", size = 64448, upload-time = "2026-05-08T21:02:01.888Z" },
{ url = "https://files.pythonhosted.org/packages/cd/86/5f8da315a4309c62c10c0b2516b17492d5d3bbe1bb862b96604db67e2a37/propcache-0.5.2-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:9282fb1a3bccd038da9f768b927b24a0c753e466c086b7c4f3c6982851eefb2d", size = 67329, upload-time = "2026-05-08T21:02:03.484Z" },
{ url = "https://files.pythonhosted.org/packages/da/d3/3368efe79ab21f0cdf86ef49895811c9cc933131d4cde1f28a624e22e712/propcache-0.5.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:cc49723e2f60d6b32a0f0b08a3fd6d13203c07f1cd9566cfce0f12a917c967a2", size = 65172, upload-time = "2026-05-08T21:02:04.745Z" },
{ url = "https://files.pythonhosted.org/packages/d5/07/127e8b0bacfb325396196f9d976a22453049b89b9b2b08477cc3145faa44/propcache-0.5.2-cp314-cp314t-win32.whl", hash = "sha256:2d7aa89ebca5acc98cba9d1472d976e394782f587bad6661003602a619fd1821", size = 43813, upload-time = "2026-05-08T21:02:06.025Z" },
{ url = "https://files.pythonhosted.org/packages/88/fb/46dad6c0ae49ed230ab1b16c890c2b6314e2403e6c412976f4a72d64a527/propcache-0.5.2-cp314-cp314t-win_amd64.whl", hash = "sha256:d447bb0b3054be5818458fbb171208b1d9ff11eba14e18ca18b90cbb45767370", size = 47764, upload-time = "2026-05-08T21:02:07.353Z" },
{ url = "https://files.pythonhosted.org/packages/e7/c4/a47d0a63aa309d10d59ede6e9d4cff03a344a79d1f0f4cd0cd74997b53e0/propcache-0.5.2-cp314-cp314t-win_arm64.whl", hash = "sha256:fe67a3d11cd9b4efabfa45c3d00ffba2b26811442a73a581a94b67c2b5faccf6", size = 41140, upload-time = "2026-05-08T21:02:09.065Z" },
{ url = "https://files.pythonhosted.org/packages/3a/ed/1cdcab6ba3d6ab7feca11fc14f0eeea80755bb53ef4e892079f31b10a25f/propcache-0.5.2-py3-none-any.whl", hash = "sha256:be1ddfcbb376e3de5d2e2db1d58d6d67463e6b4f9f040c000de8e300295465fe", size = 14036, upload-time = "2026-05-08T21:02:10.673Z" },
]
[[package]]
name = "psutil"
version = "7.2.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/aa/c6/d1ddf4abb55e93cebc4f2ed8b5d6dbad109ecb8d63748dd2b20ab5e57ebe/psutil-7.2.2.tar.gz", hash = "sha256:0746f5f8d406af344fd547f1c8daa5f5c33dbc293bb8d6a16d80b4bb88f59372", size = 493740, upload-time = "2026-01-28T18:14:54.428Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/81/69/ef179ab5ca24f32acc1dac0c247fd6a13b501fd5534dbae0e05a1c48b66d/psutil-7.2.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:eed63d3b4d62449571547b60578c5b2c4bcccc5387148db46e0c2313dad0ee00", size = 130664, upload-time = "2026-01-28T18:15:09.469Z" },
{ url = "https://files.pythonhosted.org/packages/7b/64/665248b557a236d3fa9efc378d60d95ef56dd0a490c2cd37dafc7660d4a9/psutil-7.2.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7b6d09433a10592ce39b13d7be5a54fbac1d1228ed29abc880fb23df7cb694c9", size = 131087, upload-time = "2026-01-28T18:15:11.724Z" },
{ url = "https://files.pythonhosted.org/packages/d5/2e/e6782744700d6759ebce3043dcfa661fb61e2fb752b91cdeae9af12c2178/psutil-7.2.2-cp314-cp314t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1fa4ecf83bcdf6e6c8f4449aff98eefb5d0604bf88cb883d7da3d8d2d909546a", size = 182383, upload-time = "2026-01-28T18:15:13.445Z" },
{ url = "https://files.pythonhosted.org/packages/57/49/0a41cefd10cb7505cdc04dab3eacf24c0c2cb158a998b8c7b1d27ee2c1f5/psutil-7.2.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e452c464a02e7dc7822a05d25db4cde564444a67e58539a00f929c51eddda0cf", size = 185210, upload-time = "2026-01-28T18:15:16.002Z" },
{ url = "https://files.pythonhosted.org/packages/dd/2c/ff9bfb544f283ba5f83ba725a3c5fec6d6b10b8f27ac1dc641c473dc390d/psutil-7.2.2-cp314-cp314t-win_amd64.whl", hash = "sha256:c7663d4e37f13e884d13994247449e9f8f574bc4655d509c3b95e9ec9e2b9dc1", size = 141228, upload-time = "2026-01-28T18:15:18.385Z" },
{ url = "https://files.pythonhosted.org/packages/f2/fc/f8d9c31db14fcec13748d373e668bc3bed94d9077dbc17fb0eebc073233c/psutil-7.2.2-cp314-cp314t-win_arm64.whl", hash = "sha256:11fe5a4f613759764e79c65cf11ebdf26e33d6dd34336f8a337aa2996d71c841", size = 136284, upload-time = "2026-01-28T18:15:19.912Z" },
{ url = "https://files.pythonhosted.org/packages/e7/36/5ee6e05c9bd427237b11b3937ad82bb8ad2752d72c6969314590dd0c2f6e/psutil-7.2.2-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:ed0cace939114f62738d808fdcecd4c869222507e266e574799e9c0faa17d486", size = 129090, upload-time = "2026-01-28T18:15:22.168Z" },
{ url = "https://files.pythonhosted.org/packages/80/c4/f5af4c1ca8c1eeb2e92ccca14ce8effdeec651d5ab6053c589b074eda6e1/psutil-7.2.2-cp36-abi3-macosx_11_0_arm64.whl", hash = "sha256:1a7b04c10f32cc88ab39cbf606e117fd74721c831c98a27dc04578deb0c16979", size = 129859, upload-time = "2026-01-28T18:15:23.795Z" },
{ url = "https://files.pythonhosted.org/packages/b5/70/5d8df3b09e25bce090399cf48e452d25c935ab72dad19406c77f4e828045/psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:076a2d2f923fd4821644f5ba89f059523da90dc9014e85f8e45a5774ca5bc6f9", size = 155560, upload-time = "2026-01-28T18:15:25.976Z" },
{ url = "https://files.pythonhosted.org/packages/63/65/37648c0c158dc222aba51c089eb3bdfa238e621674dc42d48706e639204f/psutil-7.2.2-cp36-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b0726cecd84f9474419d67252add4ac0cd9811b04d61123054b9fb6f57df6e9e", size = 156997, upload-time = "2026-01-28T18:15:27.794Z" },
{ url = "https://files.pythonhosted.org/packages/8e/13/125093eadae863ce03c6ffdbae9929430d116a246ef69866dad94da3bfbc/psutil-7.2.2-cp36-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:fd04ef36b4a6d599bbdb225dd1d3f51e00105f6d48a28f006da7f9822f2606d8", size = 148972, upload-time = "2026-01-28T18:15:29.342Z" },
{ url = "https://files.pythonhosted.org/packages/04/78/0acd37ca84ce3ddffaa92ef0f571e073faa6d8ff1f0559ab1272188ea2be/psutil-7.2.2-cp36-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:b58fabe35e80b264a4e3bb23e6b96f9e45a3df7fb7eed419ac0e5947c61e47cc", size = 148266, upload-time = "2026-01-28T18:15:31.597Z" },
{ url = "https://files.pythonhosted.org/packages/b4/90/e2159492b5426be0c1fef7acba807a03511f97c5f86b3caeda6ad92351a7/psutil-7.2.2-cp37-abi3-win_amd64.whl", hash = "sha256:eb7e81434c8d223ec4a219b5fc1c47d0417b12be7ea866e24fb5ad6e84b3d988", size = 137737, upload-time = "2026-01-28T18:15:33.849Z" },
{ url = "https://files.pythonhosted.org/packages/8c/c7/7bb2e321574b10df20cbde462a94e2b71d05f9bbda251ef27d104668306a/psutil-7.2.2-cp37-abi3-win_arm64.whl", hash = "sha256:8c233660f575a5a89e6d4cb65d9f938126312bca76d8fe087b947b3a1aaac9ee", size = 134617, upload-time = "2026-01-28T18:15:36.514Z" },
]
[[package]]
name = "pygments"
version = "2.20.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
]
[[package]]
name = "pyrekordbox"
version = "0.4.4"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "bidict" },
{ name = "construct" },
{ name = "numpy" },
{ name = "packaging" },
{ name = "psutil" },
{ name = "python-dateutil" },
{ name = "sqlalchemy" },
{ name = "sqlcipher3-wheels" },
]
sdist = { url = "https://files.pythonhosted.org/packages/39/8b/8fb7f21f4eb676df35c53214129ad33624634e2b678069abbf6a88509c6b/pyrekordbox-0.4.4.tar.gz", hash = "sha256:8bdb59f7c45c743bd2021613a72604b7607092d386e25f20cce78adfe30bf9a4", size = 2310256, upload-time = "2025-08-17T20:09:18.773Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b1/a8/28d28f02683bee40e3cfa32caed7c46e47daffb51482798fea9af0ab23cb/pyrekordbox-0.4.4-py3-none-any.whl", hash = "sha256:2de933870126b9a0ef04bea372f1c5934efcfbb2501efb6b47d8ace0e17fb3d8", size = 76000, upload-time = "2025-08-17T20:09:16.985Z" },
]
[[package]]
name = "pytest"
version = "9.0.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32'" },
{ name = "iniconfig" },
{ name = "packaging" },
{ name = "pluggy" },
{ name = "pygments" },
]
sdist = { url = "https://files.pythonhosted.org/packages/7d/0d/549bd94f1a0a402dc8cf64563a117c0f3765662e2e668477624baeec44d5/pytest-9.0.3.tar.gz", hash = "sha256:b86ada508af81d19edeb213c681b1d48246c1a91d304c6c81a427674c17eb91c", size = 1572165, upload-time = "2026-04-07T17:16:18.027Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl", hash = "sha256:2c5efc453d45394fdd706ade797c0a81091eccd1d6e4bccfcd476e2b8e0ab5d9", size = 375249, upload-time = "2026-04-07T17:16:16.13Z" },
]
[[package]]
name = "python-dateutil"
version = "2.9.0.post0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "six" },
]
sdist = { url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", size = 342432, upload-time = "2024-03-01T18:36:20.211Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", size = 229892, upload-time = "2024-03-01T18:36:18.57Z" },
]
[[package]]
name = "scipy"
version = "1.18.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "numpy" },
]
sdist = { url = "https://files.pythonhosted.org/packages/a7/25/c2700dfaf6442b4effaa91af24ebce5dc9d31bb4a69706313aae70d72cd0/scipy-1.18.0.tar.gz", hash = "sha256:67b2ad2ad54c72ca6d04975a9b2df8c3638c34ddd5b28738e94fc2b57929d378", size = 30774447, upload-time = "2026-06-19T15:01:43.456Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/78/b5/915a19b3de2f7430062b509653563db1633ddbb6f021b06731521115d4e2/scipy-1.18.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:4c256ee70c0d1a8a2ace807e199ccd4e3f57037433842abb3fb36bc17eaa9578", size = 31036253, upload-time = "2026-06-19T15:00:43.216Z" },
{ url = "https://files.pythonhosted.org/packages/d7/88/b72def7262e150d16be13fca37a96481138d624e700340bc3362a7588929/scipy-1.18.0-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:2ef3abc54a4ffc53765374b0d5728532dfdd2585ed23f6b11c206a1f0b1b9af8", size = 28673758, upload-time = "2026-06-19T15:00:46.663Z" },
{ url = "https://files.pythonhosted.org/packages/91/02/2e636a61a525632c373cf6a9c24442a3ffb79e364d38e98b32042964ac32/scipy-1.18.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:f2a6af57bd9e4a75d70e4117e78a1bbee84f79ae3fbb6d0111005d6ebcc4cb8d", size = 20415514, upload-time = "2026-06-19T15:00:49.399Z" },
{ url = "https://files.pythonhosted.org/packages/c9/b6/2135974442f6aba159d9d39d774a1c8cb19947016725d69fecc685df45bf/scipy-1.18.0-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:3f1ac564d3bf6c03d861d2cd87a1bea0da2887136f7fb1bf519c05a8971452d6", size = 23034398, upload-time = "2026-06-19T15:00:51.941Z" },
{ url = "https://files.pythonhosted.org/packages/f6/e6/ba89ec5abf6ee9257c0d1ec985573f3ae32742c24bc03e016388a40b1b15/scipy-1.18.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:40395a5fcd1abee49a5c7aaa98c29db393eedc835138560a588c47ec16156690", size = 33998032, upload-time = "2026-06-19T15:00:54.838Z" },
{ url = "https://files.pythonhosted.org/packages/7f/c4/bc41eb19b0fd0db868f4132920879019318d80cc522ad8f2bca4611af808/scipy-1.18.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8ca01e8ae69f1b18e9a58d91afead31be3cef0dd905a10249dac559ee15460a0", size = 35283333, upload-time = "2026-06-19T15:00:58.152Z" },
{ url = "https://files.pythonhosted.org/packages/53/a4/cbdeef6eb3830a8462a9d4ada814de5fc984345cc9ecf17cbec51a036f1e/scipy-1.18.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7a7f3b01647384dbc3a711e8c6778e0aabbe93959249fef5c7393396bcac0867", size = 35610216, upload-time = "2026-06-19T15:01:01.155Z" },
{ url = "https://files.pythonhosted.org/packages/80/4d/b2b82502b65f661d1b789c1665dcdf315d5f12194e06fc0b37946294ebae/scipy-1.18.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6aa94e78ec192a30063a5e72e561c28af769dc311190b24fe91774eff1969709", size = 37418960, upload-time = "2026-06-19T15:01:04.155Z" },
{ url = "https://files.pythonhosted.org/packages/93/3e/902d836831474b0ab5a37d16404f7bc5fafd9efba632890e271ba952635f/scipy-1.18.0-cp314-cp314-win_amd64.whl", hash = "sha256:2d8bbdc6c817f5b4006a54d799d4f5bab6f910193cbb9a1ff310833d4d270f61", size = 37288845, upload-time = "2026-06-19T15:01:07.822Z" },
{ url = "https://files.pythonhosted.org/packages/b6/43/8d73b337a3bdb14daa0314f0434210747c02d79d729ce1777574a817dcf6/scipy-1.18.0-cp314-cp314-win_arm64.whl", hash = "sha256:18e9575f1569b2c54174e6159d32942e03731177f63dce7975f0a0c88d102f5b", size = 24988971, upload-time = "2026-06-19T15:01:11.076Z" },
{ url = "https://files.pythonhosted.org/packages/b4/b4/f11918b0508a2787031a0499a03fbe3546f3bb5ca05d01038c45b278c09a/scipy-1.18.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:f351e0dd702687d12a402b867a1b4146a256923e1c38317cbc472f6372b94707", size = 31399325, upload-time = "2026-06-19T15:01:13.723Z" },
{ url = "https://files.pythonhosted.org/packages/7b/d1/1f287b57c0ff0ee5185dff3946d92c8017d39b0e431f0ae79a3ff1859512/scipy-1.18.0-cp314-cp314t-macosx_12_0_arm64.whl", hash = "sha256:7c7a51b33ce387193c97f228320cf8e87361daa1bba750638677729598b3e677", size = 29092110, upload-time = "2026-06-19T15:01:16.908Z" },
{ url = "https://files.pythonhosted.org/packages/ff/1a/7b74eb6c392fdcb27d414c0e7558a6d0231eb3b6d73571f479bb81ea8794/scipy-1.18.0-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:84031d7b052a54fae2f8632e0ec802073d385476eb9a63079bce6e23ef9283d4", size = 20833811, upload-time = "2026-06-19T15:01:20.488Z" },
{ url = "https://files.pythonhosted.org/packages/7c/ad/f3941716320a7b9cb4d68734a903b45fe16eff5fb7da7e16f2e619304979/scipy-1.18.0-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:56abf29a7c067dde59be8b9a22d606a4ea1b2f2a4b756d9d903c62818f5dacce", size = 23396644, upload-time = "2026-06-19T15:01:23.364Z" },
{ url = "https://files.pythonhosted.org/packages/22/22/1446b62ffe07f9719b7d9b1b6a4e05a772833ae8f441fe4c22c34c9b250f/scipy-1.18.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1ad44305cfa24b1ba5803cbbebf033590ccbac1aa5d612d727b785325ab408b0", size = 34079318, upload-time = "2026-06-19T15:01:26.002Z" },
{ url = "https://files.pythonhosted.org/packages/56/3b/b87da667098bb470fa30c7011b0ba351ee976dd395c78798c66e941665a3/scipy-1.18.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:945c1761b93f38d7f99ae81ae80c63e621471608c7eeead563f6df025585cd58", size = 35324320, upload-time = "2026-06-19T15:01:28.881Z" },
{ url = "https://files.pythonhosted.org/packages/f8/a1/c7932f91909759b0267f75fdea34e91309f96b895757534b76a90b6b4344/scipy-1.18.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:1a4441f15d620578772a49e5ab48c0ee1f7a0220e387110283062729136b2553", size = 35699541, upload-time = "2026-06-19T15:01:31.968Z" },
{ url = "https://files.pythonhosted.org/packages/f7/86/5185061a1fcc41d18c5dc2463969b3a3964b31d9ac67b2fb05d4c7ff7670/scipy-1.18.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:9aac6192fac56bf2ca534389d24623f07b39ff83317d58287285e7fbd622ff76", size = 37472480, upload-time = "2026-06-19T15:01:35.136Z" },
{ url = "https://files.pythonhosted.org/packages/31/8e/f04c68e39919a010d34f2ee1367fd705b0a25a02f609d755f0bfbc0a15fc/scipy-1.18.0-cp314-cp314t-win_amd64.whl", hash = "sha256:e40baea28ae7f5475c779741e2d90b1247c78531207b49c7030e698ff81cee3f", size = 37365390, upload-time = "2026-06-19T15:01:38.091Z" },
{ url = "https://files.pythonhosted.org/packages/d5/19/969dc072906c84dd0a3b05dcf57ea750936087d7873549e408b35cfc3f97/scipy-1.18.0-cp314-cp314t-win_arm64.whl", hash = "sha256:368e0a705903c466aa5f08eefb39e6b1b6b2d659e7352a31fd9e2438365be0f8", size = 25279661, upload-time = "2026-06-19T15:01:40.817Z" },
]
[[package]]
name = "six"
version = "1.17.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", size = 34031, upload-time = "2024-12-04T17:35:28.174Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050, upload-time = "2024-12-04T17:35:26.475Z" },
]
[[package]]
name = "sqlalchemy"
version = "2.0.50"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "greenlet", marker = "platform_machine == 'AMD64' or platform_machine == 'WIN32' or platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'ppc64le' or platform_machine == 'win32' or platform_machine == 'x86_64'" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/57/da/6fbf010c8ebb347679d0d100b22fe9ba5e13fd04046c5df7280d2f0bf706/sqlalchemy-2.0.50.tar.gz", hash = "sha256:af5607d11ef90fd6a5c0549fe0045dce1663d427426bcfb506dcb5346a85a3b9", size = 9907424, upload-time = "2026-05-24T19:20:04.018Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/df/32/10ac51b4be7cdecd7e93d069251c86dfbf70b7adbd7c67b48ccea6c49e1c/sqlalchemy-2.0.50-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c966932507a4d7d0a37314927dbfcd89720e3f37d2a1e3352e7ae7939fa8e8a0", size = 2158519, upload-time = "2026-05-24T19:27:56.472Z" },
{ url = "https://files.pythonhosted.org/packages/5a/76/e703d2f7681d7d66c4c891af3f07c7ccf4c76ad7f18351de035b5eda007a/sqlalchemy-2.0.50-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:faffef4bcc20a1892e65e155293d99d60855bbbc79250ab712819cfd56a8e6bb", size = 3282063, upload-time = "2026-05-24T20:09:38.57Z" },
{ url = "https://files.pythonhosted.org/packages/31/26/ef168b184a25701f9995e8fb7e503fafd7a99c1c77cda1bc1a26ea2ed486/sqlalchemy-2.0.50-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6c206aec519a2e7bd08abbfb33436e325fd22c632d9c21a9047e376ce241646e", size = 3287069, upload-time = "2026-05-24T20:17:21.942Z" },
{ url = "https://files.pythonhosted.org/packages/c2/15/765acc2bc693bccc43ca4a95d5b69750da8aaf6db1b5c616536e087f8920/sqlalchemy-2.0.50-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:bef4ac756363227ef6402a75fee025a4bc690f92328e825868939b3b3a446a6d", size = 3230453, upload-time = "2026-05-24T20:09:40.398Z" },
{ url = "https://files.pythonhosted.org/packages/63/61/08e03c3adbf5db0087a0b6816746fec8f3032fb2f7fc899a9bb9b2a48ce4/sqlalchemy-2.0.50-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:96fbee6b19c19cd1556c8bf9419447cf2ec149ffcab7ab64348c23e54ef8547f", size = 3252413, upload-time = "2026-05-24T20:17:24.067Z" },
{ url = "https://files.pythonhosted.org/packages/03/0c/370a1f2db38436c615e10134c8a37de3688e74084792380695f3f5083860/sqlalchemy-2.0.50-cp314-cp314-win32.whl", hash = "sha256:8f00e3eb43ba30eb1b238ee03a8a62309486d1321eda3328bb611e0340033ad8", size = 2120063, upload-time = "2026-05-24T19:50:11.08Z" },
{ url = "https://files.pythonhosted.org/packages/7f/a0/fe92bb9817863bc13ba093bda931979a26cc2ca69f8e8f26d07add3d7c6f/sqlalchemy-2.0.50-cp314-cp314-win_amd64.whl", hash = "sha256:15708c613cd5005b7dffe1f66ee6a63ee8f5e46799f71c70ebad74178c676a39", size = 2145830, upload-time = "2026-05-24T19:50:12.452Z" },
{ url = "https://files.pythonhosted.org/packages/cc/ff/e5640a98a0b2f491eb8fde10fb6c773621a2e44340de231fafcc9370f4a9/sqlalchemy-2.0.50-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:3699dac4be410e97049a1658e9480da9cde956594aa0f3aebc60b88f21c5ba70", size = 2178435, upload-time = "2026-05-24T19:42:58.889Z" },
{ url = "https://files.pythonhosted.org/packages/b7/85/337116e186f1236375b5fb70c21cfac98e8e8ab0d3a47be838dc47a59e08/sqlalchemy-2.0.50-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f96233858e3df43932ac11589e22520da6e8aeb624b03fedfeebb0e8ea213086", size = 3566059, upload-time = "2026-05-24T20:01:20.848Z" },
{ url = "https://files.pythonhosted.org/packages/96/34/bb0e190e161c3c2c24314a65add57218be14a4a9486886b7f5047c1ff7c8/sqlalchemy-2.0.50-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c4e70c46fad30c3bcc6a4708bc0130a3173e11a5b25f0ea4a9d8911b450f1f52", size = 3535366, upload-time = "2026-05-24T20:03:56.768Z" },
{ url = "https://files.pythonhosted.org/packages/df/5a/a7f759f97e4fd499c5d4e4488c760d5a7fbecf3028b465a04274fcd52384/sqlalchemy-2.0.50-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:1918a3cf564d16d95bca7301005f41ab2ad50b07cd3b9da50d3ed986db148d6a", size = 3474879, upload-time = "2026-05-24T20:01:23.058Z" },
{ url = "https://files.pythonhosted.org/packages/9d/d9/2907ea38eb60687d297bf9c39e5ee58053c87b57fe8a9cae97090cecbf10/sqlalchemy-2.0.50-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:b00098cdbdbd38c7be3d568b0c9c3122b8c0ec62b911b57cd5e6e0254d60a76d", size = 3486117, upload-time = "2026-05-24T20:03:59.052Z" },
{ url = "https://files.pythonhosted.org/packages/f2/e3/5aa06f167559f8c0bdae487e297d23ba548150ab016a3418265d617a4985/sqlalchemy-2.0.50-cp314-cp314t-win32.whl", hash = "sha256:1fbd55a969d7ac44a98e3dec75016074f809fa08f871585ace58dde110d1bf3e", size = 2150823, upload-time = "2026-05-24T20:08:58.644Z" },
{ url = "https://files.pythonhosted.org/packages/65/9b/112fb8f977582d7489d036e409e3723948bcf5320b3ac465f3c481bbe8f9/sqlalchemy-2.0.50-cp314-cp314t-win_amd64.whl", hash = "sha256:c5c3cdb753a9004183e1ccb634b41611654c989e61bc68617ce878e46d6f1e51", size = 2185794, upload-time = "2026-05-24T20:09:00.319Z" },
{ url = "https://files.pythonhosted.org/packages/d0/10/f7220e9b784d295d241c86ed99aeb537f92afcd469a64861f2717e9bb077/sqlalchemy-2.0.50-py3-none-any.whl", hash = "sha256:92064363517a3ff8212b5a93b8c62876579d8dfd1ca5b561335f30152d884fa9", size = 1943861, upload-time = "2026-05-24T19:59:01.119Z" },
]
[[package]]
name = "sqlcipher3-wheels"
version = "0.5.7"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/61/ec/d79c2e5f5b9246471f217cdd5f703dc8d804bce557a047e1bfad4f33c376/sqlcipher3_wheels-0.5.7.tar.gz", hash = "sha256:5fd344abba62c487c4d857a6fb55cb8dff8a97d4c4fbb4f9394b473e8e315dee", size = 2669821, upload-time = "2026-01-07T09:53:25.48Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/1e/b5/8ca4fe7928d1e7151ecdf221ad757540e9a815ea10d854f047491859cd0c/sqlcipher3_wheels-0.5.7-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:6831fd121adf8374ecfb67b63d9bdfc3bc405e7103e35dd3152f7f2d2bbe87ae", size = 4946253, upload-time = "2026-01-07T09:51:46.013Z" },
{ url = "https://files.pythonhosted.org/packages/cd/39/5eca1cced6b3a78cb20201232103b58ded2d57e87c413212bfd2a30c7209/sqlcipher3_wheels-0.5.7-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:9d03491d9d4023cca969a4c87f01ea845368defcc52ea7cd71ce93deac3df286", size = 2898557, upload-time = "2026-01-07T09:51:47.261Z" },
{ url = "https://files.pythonhosted.org/packages/34/93/8c34dd286ff8bea8785896eee401a5e15ab37901687ba09d2bf0df584712/sqlcipher3_wheels-0.5.7-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:1f3325fee4698fadbfa9b8245f213f7f3ecc3f0c86a761a2b26c787d54d3ecd1", size = 3163380, upload-time = "2026-01-07T09:51:48.384Z" },
{ url = "https://files.pythonhosted.org/packages/6d/2e/5b62f3c4181a64ab041589c6fdb509a70d69c2653f91d35b19c7550ad0d3/sqlcipher3_wheels-0.5.7-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:f80b8c72df20f8e2696e3b23577145e94b3f5a10217fef02d5c78d0ef2e578ba", size = 7259742, upload-time = "2026-01-07T09:51:49.698Z" },
{ url = "https://files.pythonhosted.org/packages/09/18/d78100264c66c96aec0723b2ddaf2b4edc72374e60c25b5ed8f3afb5e4f0/sqlcipher3_wheels-0.5.7-cp314-cp314-manylinux_2_28_i686.whl", hash = "sha256:9feb22952dd44a6e86bbb1e95a0a603bd7cc61332d0b1adad2484d2ccdc61704", size = 6875763, upload-time = "2026-01-07T09:51:51.009Z" },
{ url = "https://files.pythonhosted.org/packages/8b/e0/0529fc2d3cb0d58950557e2ac229d338506e9522a7d3eb03540acd4a6ee7/sqlcipher3_wheels-0.5.7-cp314-cp314-manylinux_2_28_ppc64le.whl", hash = "sha256:79fd014b64fefd85086cff6af85286cfb3b5761473f0a541f973a02a3c0e184b", size = 7413959, upload-time = "2026-01-07T09:51:52.344Z" },
{ url = "https://files.pythonhosted.org/packages/06/81/290107b3bd5a84edcc7a542ce1e7fe42e401cd94f4a8b6ab22e546ba034d/sqlcipher3_wheels-0.5.7-cp314-cp314-manylinux_2_28_s390x.whl", hash = "sha256:92462368d7afc2c396466eeb4cfed85dccc12bef06a5f3a9a43a6598890ddd15", size = 7389084, upload-time = "2026-01-07T09:51:53.786Z" },
{ url = "https://files.pythonhosted.org/packages/54/f1/9237d5d14807d1a6c7900330b55cad40e192f54dd4d5f732239a2271922d/sqlcipher3_wheels-0.5.7-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:663f814ca785d064408ec96c7abdc6e940809efac34cdc1656fb3536cffb8ff5", size = 7039875, upload-time = "2026-01-07T09:51:55.236Z" },
{ url = "https://files.pythonhosted.org/packages/b9/46/ba150f6d1c38f0742055d5a693f5c3532a85c4aac3c8789d130ab8492dd9/sqlcipher3_wheels-0.5.7-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:47b5ae81361c0ca45fab27ce09f94ff795a7763767c9bcbbbce19923c82d2f65", size = 7252142, upload-time = "2026-01-07T09:51:56.581Z" },
{ url = "https://files.pythonhosted.org/packages/b0/d5/e493a4a19dedc2a9c25751bcc1a7eebc29a5cbc8859108b814ef20a8742e/sqlcipher3_wheels-0.5.7-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:869f77c3f8bfc94f05352b6ca42236adfb1b46f6c92082829a608cd561793a54", size = 6948368, upload-time = "2026-01-07T09:51:57.855Z" },
{ url = "https://files.pythonhosted.org/packages/6e/ac/e82f8e26d91859570c79e3c07d2de07c65001dde49b25cb60f058f694929/sqlcipher3_wheels-0.5.7-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:98f1b5f0f54316f105f553ec254961986c8dc617947f67daa11d49a19cbc9a58", size = 7385735, upload-time = "2026-01-07T09:51:59.874Z" },
{ url = "https://files.pythonhosted.org/packages/c2/2e/3152b89c2b5ce3f7c830b675451957349e4e0d2006d0d90354a50f5c8a01/sqlcipher3_wheels-0.5.7-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:e4ca2cd825be2c30e6530c20ba6112545aaa409ad77478fb66285f98f44aac78", size = 7027637, upload-time = "2026-01-07T09:52:03.298Z" },
{ url = "https://files.pythonhosted.org/packages/76/8b/a1b3e8044128ab6a6343ef181a658298b598d0a5a2973d33e7f8fa5e45c0/sqlcipher3_wheels-0.5.7-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:a823f48efe2927a781d28ce31a11263be0e836016ea7ef26e53da18efba2569b", size = 7061293, upload-time = "2026-01-07T09:52:04.997Z" },
{ url = "https://files.pythonhosted.org/packages/cb/2f/350f8ab4e061c588ad90e12dec59f609a7e4b51c7dc921b9c62eee8c020c/sqlcipher3_wheels-0.5.7-cp314-cp314-win32.whl", hash = "sha256:0f3a11b144c46a33cb34235b253a46b1b255968be34430f9bd7f9ed710283d41", size = 2033959, upload-time = "2026-01-07T09:52:06.307Z" },
{ url = "https://files.pythonhosted.org/packages/a0/f4/ecb143774ccf0c6c8b7c718b97ac5a69ef59aa57d89c2422aa564d4a9b2f/sqlcipher3_wheels-0.5.7-cp314-cp314-win_amd64.whl", hash = "sha256:f210fdbbb794cc1b3739c7bff879b8bcc5949567f5c222e251882ec076e69b72", size = 2560254, upload-time = "2026-01-07T09:52:07.614Z" },
{ url = "https://files.pythonhosted.org/packages/2e/bc/82cd7b7ecc0ad647d9641c9115a875ea6fe8bbc04b83bfd583ebf954260f/sqlcipher3_wheels-0.5.7-cp314-cp314-win_arm64.whl", hash = "sha256:30f825607048436ab6134450ddcc49ad05f17479640a830c0ea8d78f1700e6e0", size = 2727306, upload-time = "2026-01-07T09:52:08.83Z" },
{ url = "https://files.pythonhosted.org/packages/8e/06/dcad55d438eeb9b9b46349434a75ee98f2c42196077a63d7e263969195eb/sqlcipher3_wheels-0.5.7-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:b822211ea0d50500bc8dc82863fa570543e0f224359ccb46b7675a24c89983f5", size = 4948699, upload-time = "2026-01-07T09:52:10.513Z" },
{ url = "https://files.pythonhosted.org/packages/31/c2/66fcd2a7c3200323517e69c2ff49d48bb6aad15bc47d6f7d75710531a5c2/sqlcipher3_wheels-0.5.7-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:670f493bf5a9e70e86b0ae680a8bbf96aeaaf526c5018262186e6d7c52f021b1", size = 2900350, upload-time = "2026-01-07T09:52:11.766Z" },
{ url = "https://files.pythonhosted.org/packages/7b/3a/94bebb8260624be45531af2692e9f5329ee1163237f027a3716ce3f2ae39/sqlcipher3_wheels-0.5.7-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:8c4a6dd38e9c4eff2304ea0751e8c5612394f277e63f007ae85118d9ce49b427", size = 3167268, upload-time = "2026-01-07T09:52:13.036Z" },
{ url = "https://files.pythonhosted.org/packages/a1/04/00d4d7e8905d9b53cc3cf490223b151df9a794e10138692fd4822962408b/sqlcipher3_wheels-0.5.7-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:7438567a502bfde64ca8732d1466ee6801deb97856315ac06d168a68456b2783", size = 7301506, upload-time = "2026-01-07T09:52:14.366Z" },
{ url = "https://files.pythonhosted.org/packages/95/81/75e439c4f13cbf79b243de08394916450c812eb8e949b726a758089aaf3b/sqlcipher3_wheels-0.5.7-cp314-cp314t-manylinux_2_28_i686.whl", hash = "sha256:098112cdba65feefdbaba3eebbefe45b5481158882d21a7d133f475614a2fff0", size = 6915748, upload-time = "2026-01-07T09:52:16.166Z" },
{ url = "https://files.pythonhosted.org/packages/48/3c/36e5aa4fbfe25884d7c930215cc2003b4ac7709866992086179982d04bc5/sqlcipher3_wheels-0.5.7-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:3f54e5284ebfeb2b3578ec35d3c4ad209ec1d4a3fd31b910608eadd95169cf83", size = 7453665, upload-time = "2026-01-07T09:52:18.415Z" },
{ url = "https://files.pythonhosted.org/packages/ec/62/9eddf7efdd523b3a109416f31185361608d1db60b550eb6e787a209765bc/sqlcipher3_wheels-0.5.7-cp314-cp314t-manylinux_2_28_s390x.whl", hash = "sha256:8d3bcfbb799abfa4e31e70e61c779e963b993748bf56bd0603007a47cb2a3c3f", size = 7431832, upload-time = "2026-01-07T09:52:22.041Z" },
{ url = "https://files.pythonhosted.org/packages/61/86/3e1f57a9e65d42f67f6e2e07b6c1eee14ef12caa08eedcbeebdba4389ba4/sqlcipher3_wheels-0.5.7-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:97617b4c838ad82877a342df111aecbd8653dd6c4c7dc3dccd1ef6389247d614", size = 7077205, upload-time = "2026-01-07T09:52:23.85Z" },
{ url = "https://files.pythonhosted.org/packages/98/0a/4c062770cb79860cf2cdced2b34887aa27e400fd261c553025b74ba0de74/sqlcipher3_wheels-0.5.7-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:b750c3d51134b82a4c82c9e5a6463de495ad75b7e073d03b1377083332b13ac8", size = 7296097, upload-time = "2026-01-07T09:52:25.622Z" },
{ url = "https://files.pythonhosted.org/packages/58/c7/6a6cec55e666ddd998c2c15e37a40e4fcab3c0b67c9d03c3d12b5f30d56a/sqlcipher3_wheels-0.5.7-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:dbebc62280b9fd005511189d887c080c7595226bb7242c11f1298a0b25a4c123", size = 6988446, upload-time = "2026-01-07T09:52:26.983Z" },
{ url = "https://files.pythonhosted.org/packages/fd/f4/ce2198d1296f63d0917fde374a20c09c5c5e454d65681bf8c2ee4b59591b/sqlcipher3_wheels-0.5.7-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:a806f240ede8da4547f3e50dc3ad88c0d244712270fe361a9b7e3f5d52b4bc39", size = 7425807, upload-time = "2026-01-07T09:52:28.416Z" },
{ url = "https://files.pythonhosted.org/packages/af/4b/d8735ffcf0b463bd54e23be7df46d77d40d43a672bc01a7a8da3e8f6daf3/sqlcipher3_wheels-0.5.7-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:7198ee5b4c51aae6a7dfab93da35fe82a3c8ed37b4bf6cf55d6f1a0eea387d72", size = 7067479, upload-time = "2026-01-07T09:52:29.867Z" },
{ url = "https://files.pythonhosted.org/packages/97/54/663f4c349ae59a1a9b2180f2b06c9579ddc611649b50e2ad88c2e910582c/sqlcipher3_wheels-0.5.7-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:af83b12791a1221f1799d7ad6f79edb74a3be642326a9d780dd9f86a290f9c37", size = 7097843, upload-time = "2026-01-07T09:52:31.172Z" },
{ url = "https://files.pythonhosted.org/packages/f3/8e/f8737c593394f050a73742a81e47be2b46b6413f24e96fa0b10d01dc2555/sqlcipher3_wheels-0.5.7-cp314-cp314t-win32.whl", hash = "sha256:90994590f474975d6b480af96ae5ca3887e17303b1144bfea80a38c2e9fdc157", size = 2036431, upload-time = "2026-01-07T09:52:32.444Z" },
{ url = "https://files.pythonhosted.org/packages/2a/05/573df32b1b1738779dc5326b71d0d611d28d386e7141526ed27043d476a1/sqlcipher3_wheels-0.5.7-cp314-cp314t-win_amd64.whl", hash = "sha256:85422846b4c86e63a2b495a2a60945f7a1a04641d1600859cb65fc68a1ce53ed", size = 2564275, upload-time = "2026-01-07T09:52:33.574Z" },
{ url = "https://files.pythonhosted.org/packages/54/ee/40e79926f469d630bc1df03b131e8e84740b82dc7e202fe272d2da9110e3/sqlcipher3_wheels-0.5.7-cp314-cp314t-win_arm64.whl", hash = "sha256:aeeacd6da1fc293b97641a794d15d862d7928408ddad9ea5072fad99229970aa", size = 2728867, upload-time = "2026-01-07T09:52:34.718Z" },
]
[[package]]
name = "typing-extensions"
version = "4.15.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391, upload-time = "2025-08-25T13:49:26.313Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" },
]
[[package]]
name = "win32-setctime"
version = "1.2.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/b3/8f/705086c9d734d3b663af0e9bb3d4de6578d08f46b1b101c2442fd9aecaa2/win32_setctime-1.2.0.tar.gz", hash = "sha256:ae1fdf948f5640aae05c511ade119313fb6a30d7eabe25fef9764dca5873c4c0", size = 4867, upload-time = "2024-12-07T15:28:28.314Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e1/07/c6fe3ad3e685340704d314d765b7912993bcb8dc198f0e7a89382d37974b/win32_setctime-1.2.0-py3-none-any.whl", hash = "sha256:95d644c4e708aba81dc3704a116d8cbc974d70b3bdb8be1d150e36be6e9d1390", size = 4083, upload-time = "2024-12-07T15:28:26.465Z" },
]
[[package]]
name = "yarl"
version = "1.24.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "idna" },
{ name = "multidict" },
{ name = "propcache" },
]
sdist = { url = "https://files.pythonhosted.org/packages/79/12/1e8f37460ea0f7eb59c221fdaf0ed75e7ac43e97f8093b9c6f411df50a78/yarl-1.24.2.tar.gz", hash = "sha256:9ac374123c6fd7abf64d1fec93962b0bd4ee2c19751755a762a72dd96c0378f8", size = 210798, upload-time = "2026-05-19T21:31:05.599Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/40/0e/e08087695fc12789263821c5dc0f8dc52b5b17efd0887cacf419f8a43ba3/yarl-1.24.2-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:f9312b3c02d9b3d23840f67952913c9c8721d7f1b7db305289faefa878f364c2", size = 129670, upload-time = "2026-05-19T21:29:56.631Z" },
{ url = "https://files.pythonhosted.org/packages/3a/98/ab4b5ed1b1b5cd973c8a3eb994c3a6aefb6ce6d399e21bb5f0316c33815c/yarl-1.24.2-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:a4f4d6cd615823bfc7fb7e9b5987c3f41666371d870d51058f77e2680fbe9630", size = 91916, upload-time = "2026-05-19T21:29:58.645Z" },
{ url = "https://files.pythonhosted.org/packages/ba/b1/5297bb6a7df4782f7605bffc43b31f5044070935fbbcaa6c705a07e6ac65/yarl-1.24.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:0c3063e5c0a8e8e62fae6c2596fa01da1561e4cd1da6fec5789f5cf99a8aefd8", size = 91625, upload-time = "2026-05-19T21:30:00.412Z" },
{ url = "https://files.pythonhosted.org/packages/02/a7/45baabfff76829264e623b185cff0c340d7e11bf3e1cd9ea37e7d17934bd/yarl-1.24.2-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fecd17873a096036c1c87ab3486f1aef7f269ada7f23f7f856f93b1cc7744f14", size = 104574, upload-time = "2026-05-19T21:30:02.544Z" },
{ url = "https://files.pythonhosted.org/packages/f3/40/3a5ab144d3d650ca37d4f4b57e56169be8af3ca34c448793e064b30baaed/yarl-1.24.2-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:a46d1ab4ba4d32e6dc80daf8a28ce0bd83d08df52fbc32f3e288663427734535", size = 97534, upload-time = "2026-05-19T21:30:04.319Z" },
{ url = "https://files.pythonhosted.org/packages/9c/b5/5658fef3681fb5776b4513b052bec750009f47b3a592251c705d75375798/yarl-1.24.2-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:73e68edf6dfd5f73f9ca127d84e2a6f9213c65bdffb736bda19524c0564fcd14", size = 111481, upload-time = "2026-05-19T21:30:05.988Z" },
{ url = "https://files.pythonhosted.org/packages/4c/06/fdcd7dde037f00866dce123ed4ba23dba94beb56fc4cf561668d27be37f2/yarl-1.24.2-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a296ca617f2d25fbceafb962b88750d627e5984e75732c712154d058ae8d79a3", size = 111529, upload-time = "2026-05-19T21:30:07.738Z" },
{ url = "https://files.pythonhosted.org/packages/c2/53/d81269aaafccea0d33396c03035de997b743f11e648e6e27a0df99c72980/yarl-1.24.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e51b2cf5ec89a8b8470177641ed62a3ba22d74e1e898e06ad53aa77972487208", size = 107338, upload-time = "2026-05-19T21:30:09.713Z" },
{ url = "https://files.pythonhosted.org/packages/ae/04/23049463f729bd899df203a7960505a75333edd499cda8aa1d5a82b64df5/yarl-1.24.2-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:310fc687f7b2044ec54e372c8cbe923bb88f5c37bded0d3079e5791c2fc3cf50", size = 106147, upload-time = "2026-05-19T21:30:11.365Z" },
{ url = "https://files.pythonhosted.org/packages/14/18/04a4b5830b43ed5e4c5015b40e9f6241ad91487d71611061b4e111d6ac80/yarl-1.24.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:297a2fe352ecf858b30a98f87948746ec16f001d279f84aebdbd3bd965e2f1bd", size = 104272, upload-time = "2026-05-19T21:30:12.978Z" },
{ url = "https://files.pythonhosted.org/packages/5a/f7/8cffdf319aee7a7c1dbd07b61d91c3e3fda460c7a93b5f93e445f3806c4c/yarl-1.24.2-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:2a263e76b97bc42bdcd7c5f4953dec1f7cd62a1112fa7f869e57255229390d67", size = 99962, upload-time = "2026-05-19T21:30:15.001Z" },
{ url = "https://files.pythonhosted.org/packages/d7/39/b3cce3b7dbef64ac700ad4cea156a207d01bede0f507587616c364b5468e/yarl-1.24.2-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:822519b64cf0b474f1a0aaef1dc621438ea46bb77c94df97a5b4d213a7d8a8b1", size = 111063, upload-time = "2026-05-19T21:30:16.683Z" },
{ url = "https://files.pythonhosted.org/packages/a1/ea/100818505e7ebf165c7242ff17fdf7d9fee79e27234aeca871c1082920d7/yarl-1.24.2-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:b6067060d9dc594899ba83e6db6c48c68d1e494a6dab158156ed86977ca7bcb1", size = 105438, upload-time = "2026-05-19T21:30:18.769Z" },
{ url = "https://files.pythonhosted.org/packages/8f/d2/e075a0b32aa6625087de9e653087df0759fed5de4a435fef594181102a77/yarl-1.24.2-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:0063adad533e57171b79db3943b229d40dfafeeee579767f96541f106bac5f1b", size = 111458, upload-time = "2026-05-19T21:30:21.024Z" },
{ url = "https://files.pythonhosted.org/packages/e6/5c/ceea7ba98b65c8eb8d947fdc52f9bedfcd43c6a57c9e3c90c17be8f324a3/yarl-1.24.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ee8e3fb34513e8dc082b586ef4910c98335d43a6fab688cd44d4851bacfce3e8", size = 107589, upload-time = "2026-05-19T21:30:23.412Z" },
{ url = "https://files.pythonhosted.org/packages/fa/d9/5582d57e2b2db9b85eb6663a22efdd78e08805f3f5389566e9fcad254d1b/yarl-1.24.2-cp314-cp314-win_amd64.whl", hash = "sha256:afb00d7fd8e0f285ca29a44cc50df2d622ff2f7a6d933fa641577b5f9d5f3db0", size = 94424, upload-time = "2026-05-19T21:30:25.425Z" },
{ url = "https://files.pythonhosted.org/packages/92/10/7dc07a0e22806a9280f42a57361395506e800c64e22737cd7b0886feab42/yarl-1.24.2-cp314-cp314-win_arm64.whl", hash = "sha256:68cf6eacd6028ef1142bc4b48376b81566385ca6f9e7dde3b0fa91be08ffcb57", size = 88690, upload-time = "2026-05-19T21:30:27.623Z" },
{ url = "https://files.pythonhosted.org/packages/9e/13/d5b8e2c8667db955bcb3de233f18798fefe7edf1d7429c2c9d4f9c401114/yarl-1.24.2-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:221ce1dd921ac4f603957f17d7c18c5cc0797fbb52f156941f92e04605d1d67b", size = 136248, upload-time = "2026-05-19T21:30:29.297Z" },
{ url = "https://files.pythonhosted.org/packages/de/46/a4a97c05c9c9b8fd266bb2a0df12992c7fbd02391eb9640583411b6dab32/yarl-1.24.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:5f3224db28173a00d7afacdee07045cc4673dfab2b15492c7ae10deddbece761", size = 95084, upload-time = "2026-05-19T21:30:31.031Z" },
{ url = "https://files.pythonhosted.org/packages/95/b2/845cf2074a015e6fe0d0808cf1a2d9e868386c4220d657ebd8302b199043/yarl-1.24.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c557165320d6244ebe3a02431b2a201a20080e02f41f0cfa0ccc47a183765da8", size = 95272, upload-time = "2026-05-19T21:30:33.062Z" },
{ url = "https://files.pythonhosted.org/packages/fe/16/e69d4aa244aef45235ddfebc0e04036a6829842bc5a6a795aedc6c998d23/yarl-1.24.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:904065e6e85b1fa54d0d87438bd58c14c0bad97aad654ad1077fd9d87e8478ed", size = 101497, upload-time = "2026-05-19T21:30:34.842Z" },
{ url = "https://files.pythonhosted.org/packages/15/94/c07107715d621076863ee88b3ddf183fa5e9d4aba5769623c9979828410a/yarl-1.24.2-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:8cec2a38d70edc10e0e856ceda886af5327a017ccbde8e1de1bd44d300357543", size = 94002, upload-time = "2026-05-19T21:30:37.724Z" },
{ url = "https://files.pythonhosted.org/packages/a9/35/fc1bbdd895b5e4010b8fdd037f7ed3aa289d3863e08231b30231ca9a0815/yarl-1.24.2-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e7484b9361ed222ee1ca5b4337aa4cbdcc4618ce5aff57d9ef1582fd95893fc0", size = 106524, upload-time = "2026-05-19T21:30:40.196Z" },
{ url = "https://files.pythonhosted.org/packages/1f/f2/32b66d0a4ba47c296cf86d03e2c67bff58399fe6d6d84d5205c04c66cc6d/yarl-1.24.2-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:84f9670b89f34db07f81e53aee83e0b938a3412329d51c8f922488be7fcc4024", size = 106165, upload-time = "2026-05-19T21:30:41.888Z" },
{ url = "https://files.pythonhosted.org/packages/95/47/37cb5ff50c5e825d4d38e81bb04d1b7e96bf960f7ab89f9850b162f3f114/yarl-1.24.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:abb2759733d63a28b4956500a5dd57140f26486c92b2caedfb964ab7d9b79dbf", size = 103010, upload-time = "2026-05-19T21:30:43.985Z" },
{ url = "https://files.pythonhosted.org/packages/6f/d2/4597912315096f7bb359e46e13bf8b60994fcbb2db29b804c0902ef4eff5/yarl-1.24.2-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:081c2bf54efe03774d0311172bc04fedf9ca01e644d4cd8c805688e527209bdc", size = 101128, upload-time = "2026-05-19T21:30:46.291Z" },
{ url = "https://files.pythonhosted.org/packages/b9/d5/c8e86e120521e646013d02a8e3b8884392e28494be8f392366e50d208efc/yarl-1.24.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:86746bef442aa479107fe28132e1277237f9c24c2f00b0b0cf22b3ee0904f2bb", size = 101382, upload-time = "2026-05-19T21:30:48.085Z" },
{ url = "https://files.pythonhosted.org/packages/fa/98/70b229236118f89dbeb739b76f10225bbf53b5497725502594c9a01d699a/yarl-1.24.2-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:2d07d21d0bc4b17558e8de0b02fbfdf1e347d3bb3699edd00bb92e7c57925420", size = 95964, upload-time = "2026-05-19T21:30:49.785Z" },
{ url = "https://files.pythonhosted.org/packages/87/f8/56c386981e3c8648d279fdef2397ffec577e8320fd5649745e34d54faeb7/yarl-1.24.2-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:4fb1ac3fc5fecd8ae7453ea237e4d22b49befa70266dfe1629924245c21a0c7f", size = 106204, upload-time = "2026-05-19T21:30:51.862Z" },
{ url = "https://files.pythonhosted.org/packages/1a/1e/765afe97811ca35933e2a7de70ac57b1997ea2e4ee895719ee7a231fb7e5/yarl-1.24.2-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:4da31a5512ed1729ca8d8aacde3f7faeb8843cde3165d6bcf7f88f74f17bb8aa", size = 101510, upload-time = "2026-05-19T21:30:53.62Z" },
{ url = "https://files.pythonhosted.org/packages/ee/78/393913f4b9039e1edd09ae8a9bbb9d539be909a8abf6d8a2084585bed4b7/yarl-1.24.2-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:533ded4dceb5f1f3da7906244f4e82cf46cfd40d84c69a1faf5ac506aa65ecbe", size = 105584, upload-time = "2026-05-19T21:30:55.962Z" },
{ url = "https://files.pythonhosted.org/packages/78/87/deb17b7049bbe74ea11a713b86f8f27800cc1c8648b0b797243ebb4830ba/yarl-1.24.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:7b3a85525f6e7eeabcfdd372862b21ee1915db1b498a04e8bf0e389b607ff0bd", size = 103410, upload-time = "2026-05-19T21:30:57.962Z" },
{ url = "https://files.pythonhosted.org/packages/8f/be/f9f7594e23b5b93affff0318e4593c1920331bcaefda326cabcad94296a1/yarl-1.24.2-cp314-cp314t-win_amd64.whl", hash = "sha256:a7624b1ca46ca5d7b864ef0d2f8efe3091454085ee1855b4e992314529972215", size = 102980, upload-time = "2026-05-19T21:30:59.735Z" },
{ url = "https://files.pythonhosted.org/packages/65/a4/ba80dccd3593ff1f01051a818694d07b58cb8232677ee9a22a5a1f93a9fc/yarl-1.24.2-cp314-cp314t-win_arm64.whl", hash = "sha256:e434a45ce2e7a947f951fc5a8944c8cc080b7e59f9c50ae80fd39107cf88126d", size = 91219, upload-time = "2026-05-19T21:31:01.934Z" },
{ url = "https://files.pythonhosted.org/packages/fd/4d/4b880086bd0d3e034d25647be1d830afc3e3f610e98c4ab3490af6b1b6d5/yarl-1.24.2-py3-none-any.whl", hash = "sha256:2783d9226db8797636cd6896e4de81feed252d1db72265686c9558d97a4d94b9", size = 53576, upload-time = "2026-05-19T21:31:03.909Z" },
]
-244
View File
@@ -1,244 +0,0 @@
use rekordcrate::pdb::RowGroup;
use rekordcrate::pdb::Artwork;
use rekordcrate::pdb::ArtworkId;
use rekordcrate::pdb::Page;
use rekordcrate::pdb::PageType;
use rekordcrate::pdb::PageIndex;
use std::io::Write;
use std::io::{Seek, SeekFrom};
use binrw::BinWrite;
use rekordcrate::pdb::LabelId;
use rekordcrate::pdb::KeyId;
use rekordcrate::pdb::GenreId;
use rekordcrate::pdb::ArtistId;
use rekordcrate::pdb::AlbumId;
use rekordcrate::pdb::Album;
use rekordcrate::pdb::Artist;
use rekordcrate::pdb::Genre;
use rekordcrate::pdb::Key;
use rekordcrate::pdb::Label;
use rekordcrate::pdb::Table;
use std::fs::File;
use std::path::Path;
use binrw::BinRead;
use std::collections::BTreeMap;
use binrw::Endian;
use rekordcrate::pdb::{
Header, Row,
Track, TrackId,
PlaylistTreeNode, PlaylistTreeNodeId,
PlaylistEntry,
};
#[derive(Debug)]
pub enum DbError {
UnknownPlaylist,
UnknownTrack,
PositionOutOfBounds,
}
#[derive(Debug)]
pub struct Database {
pub header: Header,
pub tracks: BTreeMap<TrackId, Track>,
pub albums: BTreeMap<AlbumId, Album>,
pub artists: BTreeMap<ArtistId, Artist>,
pub artwork: BTreeMap<ArtworkId, Artwork>,
pub genres: BTreeMap<GenreId, Genre>,
pub keys: BTreeMap<KeyId, Key>,
pub labels: BTreeMap<LabelId, Label>,
pub nodes: BTreeMap<PlaylistTreeNodeId, PlaylistTreeNode>,
// Keep playlist entries ordered by entry_index
pub entries: BTreeMap<PlaylistTreeNodeId, Vec<PlaylistEntry>>,
}
const PAGE_SIZE: u16 = 4096;
impl Database {
pub fn export(self, path: &Path) -> Result<(), binrw::Error> {
let pages = Table::from_rows(PageType::Tracks, 1, PAGE_SIZE, self.tracks.values().cloned().map(Row::Track))?;
println!("{:?}", pages.len());
println!("{:?}", pages.last());
Ok(())
}
// pub fn export(self, path: &Path) -> std::io::Result<()> {
// // 1) Reserve header page
// let mut f = File::create(path)?;
// f.write_all(&vec![0u8; PAGE_SIZE as usize])?;
// // 2) Emit tables linearly, collect table metadata
// let mut next_page_index: u32 = 1;
// // ok now we're getting somewhere
// // smash rows into row_groups of 16
// // try to write those into a page until it is full
// // yield pages until all the rows are in
// // set the indices
// // 420 blaze it
// let (tracks_first, tracks_last) =
// Self::write_table(&mut f, PageType::Tracks, next_page_index, self.tracks.values().cloned().map(Row::Track))?;
// next_page_index = tracks_last + 1;
// let (tree_first, tree_last) =
// Self::write_table(&mut f, PageType::PlaylistTree, next_page_index, self.nodes.values().cloned().map(Row::PlaylistTreeNode))?;
// next_page_index = tree_last + 1;
// let plist_entries = self.entries.iter().flat_map(|(_pid, vec)| {
// // ensure contiguous entry_index
// let mut v = vec.clone();
// v.sort_by_key(|e| e.entry_index);
// v.into_iter().map(Row::PlaylistEntry)
// });
// let (entries_first, entries_last) =
// Self::write_table(&mut f, PageType::PlaylistEntries, next_page_index, plist_entries)?;
// next_page_index = entries_last + 1;
// let (art_first, art_last) =
// Self::write_table(&mut f, PageType::Artwork, next_page_index, self.artwork.values().cloned().map(Row::Artwork))?;
// next_page_index = art_last + 1;
// // 3) Build header and back-patch at page 0
// let header = Header {
// // fill other required fields in your crate
// sequence: 1, // or previous+1
// next_unused_page: PageIndex(next_page_index),
// tables: vec![
// Table { page_type: PageType::Tracks, first_page: PageIndex(tracks_first), last_page: PageIndex(tracks_last), empty_candidate: 0, /* … */ },
// Table { page_type: PageType::PlaylistTree, first_page: PageIndex(tree_first), last_page: PageIndex(tree_last), empty_candidate: 0, /* … */ },
// Table { page_type: PageType::PlaylistEntries, first_page: PageIndex(entries_first), last_page: PageIndex(entries_last), empty_candidate: 0, /* … */ },
// Table { page_type: PageType::Artwork, first_page: PageIndex(art_first), last_page: PageIndex(art_last), empty_candidate: 0, /* … */ },
// ],
// ..Default::default()
// };
// // Write header at offset 0 and pad to one page
// f.seek(SeekFrom::Start(0))?;
// header.write(&mut f);
// // pad residual of page 0 if BinWrite didn’t fill it:
// let pos = f.stream_position()?;
// if pos < PAGE_SIZE as u64 {
// f.write_all(&vec![0u8; (PAGE_SIZE as u64 - pos) as usize])?;
// }
// Ok(())
// }
// fn write_table<I>(
// f: &mut File,
// page_type: PageType,
// start_index: u32,
// rows: I,
// ) -> std::io::Result<(u32, u32)>
// where
// I: IntoIterator<Item = Row>,
// {
// let mut page_index = start_index;
// let mut first = 0;
// let mut last = 0;
// // Pack rows into pages. Use your crate’s Page builder to handle row groups.
// let mut packer = PagePacker::new(page_type, PAGE_SIZE);
// for row in rows {
// if !packer.try_push(&row) {
// // flush full page
// let mut page = packer.finish(page_index, /*next_page=*/0);
// // Temporarily set next_page=0; we will fix it after writing the next page.
// Self::write_page(f, &mut page)?;
// if first == 0 { first = page_index; }
// last = page_index;
// page_index += 1;
// // start new page with the row that didn’t fit
// packer = PagePacker::new(page_type, PAGE_SIZE);
// assert!(packer.try_push(&row));
// }
// }
// // flush final page if it has content or if this is an empty table create one empty page
// if packer.is_empty() && first == 0 {
// let mut page = Page::empty(page_type, page_index, /*next_page=*/0, PAGE_SIZE);
// Self::write_page(f, &mut page)?;
// first = page_index;
// last = page_index;
// } else {
// let mut page = packer.finish(page_index, /*next_page=*/0);
// Self::write_page(f, &mut page)?;
// if first == 0 { first = page_index; }
// last = page_index;
// }
// // Fix `next_page` pointers by rewriting page headers if your Page writer requires it.
// // Simpler: during emission, remember the previous page file offset and seek back
// // to patch its `next_page` to `page_index`.
// Ok((first, last))
// }
// fn write_page(f: &mut File, page: &mut Page) -> std::io::Result<()> {
// page.write(f);
// Ok(())
// }
pub fn load_from_pdb(path: impl AsRef<Path>) -> rekordcrate::Result<Self> {
let mut f = std::fs::File::open(path)?;
let header = Header::read(&mut f)?; // uses the type’s endianness
let mut tracks: BTreeMap<TrackId, Track> = BTreeMap::new();
let mut albums: BTreeMap<AlbumId, Album> = BTreeMap::new();
let mut artists: BTreeMap<ArtistId, Artist> = BTreeMap::new();
let mut artwork: BTreeMap<ArtworkId, Artwork> = BTreeMap::new();
let mut genres: BTreeMap<GenreId, Genre> = BTreeMap::new();
let mut keys: BTreeMap<KeyId, Key> = BTreeMap::new();
let mut labels: BTreeMap<LabelId, Label> = BTreeMap::new();
let mut nodes: BTreeMap<PlaylistTreeNodeId, PlaylistTreeNode> = BTreeMap::new();
let mut entries: BTreeMap<PlaylistTreeNodeId, Vec<PlaylistEntry>> = BTreeMap::new();
for table in &header.tables {
let pages = header.read_pages(&mut f, Endian::NATIVE, (&table.first_page, &table.last_page))?;
for page in pages {
for rg in page.row_groups {
for row in rg.present_rows() {
match row {
Row::Track(t) => {
tracks.insert(t.id, t.clone());
}
Row::Artist(a) => {
artists.insert(a.id, a.clone());
}
Row::Album(a) => {
albums.insert(a.id, a.clone());
}
Row::Genre(g) => {
genres.insert(g.id, g.clone());
}
Row::Key(k) => {
keys.insert(k.id, k.clone());
}
Row::Label(l) => {
labels.insert(l.id, l.clone());
}
Row::PlaylistTreeNode(n) => {
nodes.insert(n.id, n.clone());
}
Row::PlaylistEntry(e) => {
entries.entry(e.playlist_id).or_default().push(e.clone());
}
_ => {}
}
}
}
}
}
// Ensure deterministic ordering within each playlist
for v in entries.values_mut() {
v.sort_by_key(|e| e.entry_index); // needs `pub entry_index`
}
Ok(Self { header, tracks, albums, artists, artwork, genres, keys, labels, nodes, entries })
}
}
+16 -23
View File
@@ -1,31 +1,24 @@
mod db;
extern crate walkdir;
use rekordcrate::pdb::Page;
use rekordcrate::pdb::PageIndex;
use rekordcrate::pdb::Row::PlaylistEntry;
use rekordcrate::pdb::string::DeviceSQLString;
use rekordcrate::pdb::RowGroup;
use rekordcrate::pdb::TrackId;
use rekordcrate::pdb::PageType;
use rekordcrate::pdb::Table;
use rekordcrate::pdb::{Header, PlaylistTreeNode, PlaylistTreeNodeId};
use walkdir::WalkDir;
use std::env;
use std::fs::create_dir;
use std::fs::create_dir_all;
use std::{fs::exists, path::Path};
use db::Database;
use rekordcrate::util::RekordcrateError;
use rekordcrate::pdb::Database;
use std::fs::File;
use std::path::Path;
const PAGE_SIZE: u32 = 4096;
// Example
fn main() -> rekordcrate::Result<()> {
let db = Database::load_from_pdb("ntr.pdb")?;
println!("tracks: {}, playlists: {}", db.tracks.len(), db.nodes.len());
println!("tables: {:?}", db.header);
println!("tables: {:?}", db.header.tables);
fn main() -> Result<(), RekordcrateError> {
let db = Database::load_from_pdb("ntr.pdb").unwrap();
db.export(&Path::new("./data/output.pdb"))?;
println!("{:?}", db.colors);
let output = Path::new("./data/output.pdb");
let mut f = File::create(output)?;
db.export(PAGE_SIZE, &mut f)?;
// let db = Database::load_from_pdb("./data/output.pdb").unwrap();
// println!("{:?}", db);
// for n in db.nodes {
// println!("{:?}", n);
+52 -34
View File
@@ -1,36 +1,54 @@
{
"folders":
[
{
"path": ".",
},
{
"path": "h:ome\\ntr\\code\\rekordcrate",
},
{
"path": "C:\\Users\\ntr\\code\\rekordcrate",
},
{
"path": "/home/ntr/code/rekordcrate"
},
],
"settings":
{
"cargo_build":
{
"paths":
{
"C:\\Users\\ntr\\code\\maple\\maple-rs":
{
"variants":
{
"run":
{
"target": "--bin maple",
},
},
},
},
},
},
"folders":
[
{
"path": ".",
},
],
"settings":
{
"LSP":
{
"LSP-pyright":
{
// Use maple-py/ as the python project root so pyrightconfig.json
// there is picked up (extraPaths=["src"], venvPath=".venv",
// pythonVersion="3.14"). The interpreter is the uv-managed venv.
"settings":
{
"python.pythonPath": "$folder/maple-py/.venv/bin/python",
"python.analysis.extraPaths":
[
"$folder/maple-py/src",
],
"python.analysis.typeCheckingMode": "basic",
"python.analysis.diagnosticMode": "openFilesOnly",
"python.analysis.autoImportCompletions": true,
"python.analysis.indexing": true,
},
// Pin the LSP server's working directory to maple-py/ so file
// lookups resolve relative to where pyrightconfig.json lives.
"initializationOptions":
{
"pythonPath": "$folder/maple-py/.venv/bin/python",
},
},
},
"cargo_build":
{
"paths":
{
"$folder/maple-rs":
{
"variants":
{
"run":
{
"target": "--bin maple",
},
},
},
},
},
},
}