Compare commits
21
Commits
export
...
9fbd222b23
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9fbd222b23 | ||
|
|
d5ef56c3ed | ||
|
|
5810a536bd | ||
|
|
096bce803a | ||
|
|
b85cea496f | ||
|
|
9ee5a70266 | ||
|
|
ea1669babf | ||
|
|
8b4ac2cdaf | ||
|
|
f06d3aa0f9 | ||
|
|
96d0331ff3 | ||
|
|
9ccc3cea95 | ||
|
|
f71a2a3512 | ||
|
|
6a6553077b | ||
|
|
1e49010aa9 | ||
|
|
32adb5ddac | ||
|
|
6a32066069 | ||
|
|
f804c327de | ||
|
|
690f92f565 | ||
|
|
076c60658d | ||
|
|
13ee0dcf01 | ||
|
|
d35b51d4c3 |
No files matched your search
@@ -0,0 +1,50 @@
|
||||
---
|
||||
name: pdb-maple
|
||||
description: Updates maple's pdb serializer (maple-py/src/maple/pdb/) to match decoded format rules from the kaitai schema + FORMAT_SPEC, then regenerates and diffs the AceMo pdb against rekordbox's. Use after pdb-re records a finding.
|
||||
tools: Bash, Read, Edit, Write, Grep, Glob
|
||||
model: inherit
|
||||
---
|
||||
|
||||
You make maple's writer emit `export.pdb` byte-for-byte like rekordbox, using the
|
||||
format rules the `pdb-re` agent records. You implement the writer; you do NOT
|
||||
reverse-engineer the binary.
|
||||
|
||||
## Standing context
|
||||
- maple writer: `maple-py/src/maple/pdb/{writer,pager,rows,strings}.py`, model in
|
||||
`maple-py/src/maple/model.py`.
|
||||
- Format truth (your inputs): `maple-py/src/kaitai/rekordbox_pdb.py` (layout/types)
|
||||
and `maple-py/re/FORMAT_SPEC.md` (value-construction rules). Read both each run.
|
||||
- Reference real export: `maple-py/data/acemo/PIONEER/rekordbox/export.pdb`.
|
||||
- Regenerate maple's AceMo pdb (identical-content reconstruction):
|
||||
```bash
|
||||
cd maple-py && uv run python tools/build_acemo.py # -> captures/maple_acemo.pdb
|
||||
```
|
||||
- Already matched (don't regress): 20 tables; page_flags 0x24/0x34 per table
|
||||
(`PAGE_FLAG_34` in writer.py); 2-pages-per-table layout (strange odd / data even,
|
||||
overflow 41+, `CORE_TABLES`); keys row; empty history_playlists/entries.
|
||||
|
||||
## Your loop, per finding
|
||||
1. Read the latest `FORMAT_SPEC.md` rule + any changed kaitai fields.
|
||||
2. Implement it in the writer (`rows.py` for row fields, `pager.py`/`writer.py`
|
||||
for page/layout). Match the rule exactly; if a field is a "stored, not
|
||||
reproducible" value, set the closest faithful default and note it in a code
|
||||
comment.
|
||||
3. `uv run python tools/build_acemo.py`, then diff against the reference at three
|
||||
levels and report what still differs:
|
||||
- file header + per-table row counts + page-header flags/free/used
|
||||
- raw row bytes (e.g. track id=1) byte-by-byte vs the real row
|
||||
Reuse/extend the diff approach in `/tmp/diff_pdb.py` + `/tmp/rowdiff.py` (or
|
||||
write equivalents). `uv run pytest -q` must still pass.
|
||||
4. Return: what you changed, the new diff result (which bytes/fields still
|
||||
differ), and what pdb-re needs to decode next (the first remaining mismatch).
|
||||
|
||||
Goal: ZERO diff between `captures/maple_acemo.pdb` and the reference. Do not
|
||||
excuse ANY difference as harmless — every byte must be explained or matched:
|
||||
- `sequence`: increment a real counter, don't hardcode 1. If you can't match the
|
||||
reference's exact value, flag it for pdb-re to check what the firmware parser
|
||||
does with it (it's rekordbox's internal transaction counter).
|
||||
- page layout / `next_unused` / overflow-page placement: match it. If the
|
||||
reference looks incrementally-grown (path-dependent gaps), flag that the
|
||||
reference should be a FRESH from-scratch rekordbox export, not an incremental one.
|
||||
- density / free / used: investigate, never assume harmless.
|
||||
Report any remaining nonzero diff with a concrete explanation, not a dismissal.
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
name: pdb-re
|
||||
description: Reverse-engineers rekordbox export.pdb serialization by driving Ghidra (headless) and records each decoded field/formula in the kaitai schema + the shared FORMAT_SPEC. Use to decode an unknown/wrong pdb field.
|
||||
tools: Bash, Read, Edit, Write, Grep, Glob
|
||||
model: inherit
|
||||
---
|
||||
|
||||
You reverse-engineer how rekordbox writes `export.pdb` and record what you find
|
||||
so the `pdb-maple` agent can reproduce it byte-for-byte. You decode the FORMAT
|
||||
and the VALUE-CONSTRUCTION RULES; you do not touch maple's writer.
|
||||
|
||||
## Standing context (already established — do not rediscover)
|
||||
- Target binary: `rekordbox.exe` (89MB, JUCE C++). `export.pdb` is written by its
|
||||
bundled **DeviceSQL "Page Manager"** (`dsql*`/`edb_dyn_*`).
|
||||
- Analyzed Ghidra project: `~/ghidra_proj` (project `rbx`, program `rekordbox.exe`),
|
||||
already imported + analyzed. JDK at `~/jdk/jdk-21.0.11+10`. Ghidra at
|
||||
`/mnt/c/Users/ntr/code/ghidra_12.1.2_PUBLIC_20260605/ghidra_12.1.2_PUBLIC`.
|
||||
- Run a Ghidra script (no re-analysis):
|
||||
```bash
|
||||
export JAVA_HOME=~/jdk/jdk-21.0.11+10; export GHIDRA_HEADLESS_MAXMEM=10G
|
||||
GH=/mnt/c/Users/ntr/code/ghidra_12.1.2_PUBLIC_20260605/ghidra_12.1.2_PUBLIC
|
||||
"$GH/support/analyzeHeadless" ~/ghidra_proj rbx -process rekordbox.exe -noanalysis \
|
||||
-scriptPath ~/ghidra_scripts -postScript DecPdb.java
|
||||
```
|
||||
Edit `~/ghidra_scripts/DecPdb.java`'s ADDRS to decompile any addresses; output
|
||||
→ `~/ghidra_proj/decpdb.txt`. Other scripts: FindPdb, CallTree, FindHash. Copies
|
||||
also live in `maple-py/re/scripts/`.
|
||||
- Address rebasing: Ghidra static = `0x140000000 + module_offset`. Frida traces
|
||||
print `rekordbox.exe+0xOFFSET` (the agent `maple-py/tools/rbx_pdb_capture.js`);
|
||||
never hardcode the ASLR runtime base.
|
||||
- The serializer call stack is in `~/ghidra_proj/decpdb.txt` / `re/findings/`.
|
||||
- Reference real export: `maple-py/data/acemo/PIONEER/rekordbox/export.pdb` (6
|
||||
tracks). Parse with the kaitai schema (`maple-py/src/kaitai/rekordbox_pdb.py`)
|
||||
via `uv run` from `maple-py/`.
|
||||
|
||||
## Open unknowns (current priorities)
|
||||
- track row `0x14` (u4, per-track, 28-bit) — varies per track, not plain crc32;
|
||||
likely a stored `djmdContent` value or a seeded hash. `0x18`/`0x1a` = per-export
|
||||
constant.
|
||||
- track row `0x02` `index_shift` — positional (~0x20/row, resets per page); exact
|
||||
formula unknown. maple writes 0.
|
||||
|
||||
## Your loop, per field
|
||||
1. Pick the unknown/wrong field you were asked about.
|
||||
2. Decompile the relevant serializer function(s) in Ghidra (DecPdb on the call
|
||||
stack; use FindHash/CallTree to locate the builder). Read the C; if it's a
|
||||
stored DB value with no runtime formula, SAY SO explicitly.
|
||||
3. Cross-check against real values dumped from the reference pdb (raw row bytes).
|
||||
4. Record the finding in BOTH:
|
||||
- `maple-py/src/kaitai/rekordbox_pdb.py` — rename the field / fix its type if
|
||||
the layout was wrong.
|
||||
- `maple-py/re/FORMAT_SPEC.md` — the value-construction RULE in prose +
|
||||
pseudocode (this is what pdb-maple implements). Create the file if missing.
|
||||
5. Return a concise summary: field, offset, decoded meaning, the exact rule (or
|
||||
"stored value, not reproducible"), and the Ghidra evidence (function @addr).
|
||||
|
||||
Be rigorous: prefer reading the decompiled code over guessing. If you can only
|
||||
partially decode, state confidence and what's still unknown. Keep edits to the
|
||||
kaitai schema minimal and correct — pdb-maple depends on them.
|
||||
@@ -20,3 +20,9 @@ maple.sublime-workspace
|
||||
*.secret.yaml
|
||||
|
||||
etc/sldl-macos.yaml
|
||||
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
.venv/
|
||||
maple-py/captures
|
||||
maple-py/node_modules
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"mcpServers": {
|
||||
"ghidra": {
|
||||
"command": "uv",
|
||||
"args": [
|
||||
"run",
|
||||
"--script",
|
||||
"/mnt/c/Users/ntr/Downloads/GhidraMCP-release-1-4/GhidraMCP-release-1-4/bridge_mcp_ghidra.py",
|
||||
"--ghidra-server",
|
||||
"http://127.0.0.1:8080/"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,9 +1,31 @@
|
||||
[x] download playlist
|
||||
[x] store everything by artist - album
|
||||
[ ] convert flac to mp3
|
||||
[ ] sync to and from s3 api storage
|
||||
[ ] create entries in rekordbox db for playlists
|
||||
[ ] have a rekordbox vm
|
||||
[ ] analyse files downloaded by all users with rekordbox vm
|
||||
[ ] store analysis metadata in pg
|
||||
[ ]
|
||||
|
||||
## end to end
|
||||
|
||||
### collection management
|
||||
just spot should:
|
||||
download playlist from spotify into data/collection
|
||||
determine files missing from collection
|
||||
download each of those files with sockseek
|
||||
- should be flac
|
||||
- get artwork if missing
|
||||
- correct tags
|
||||
- quick validate
|
||||
update m3u8 matching files in spotify to files in collection
|
||||
|
||||
### prepare for decks
|
||||
|
||||
store ANLZ in metadata directory by isrc
|
||||
create data/usb directory
|
||||
convert to mp3 v0 jstereo
|
||||
- output into the data/usb directory
|
||||
write pdb
|
||||
- match playlist tree structure
|
||||
generate anlz
|
||||
- detect bpm is important
|
||||
- waveforms are nice but not essential
|
||||
|
||||
|
||||
### sync (later)
|
||||
sync collection and usb to cloud
|
||||
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
+4
-4
@@ -1,7 +1,7 @@
|
||||
path = "/data"
|
||||
index-path = "/data/_metadata/index.sldl.csv"
|
||||
username = "${SLDL_USER}"
|
||||
password = "${SLDL_PASS}"
|
||||
path = "${SOCKSEEK_DATA}"
|
||||
index-path = "${SOCKSEEK_DATA}/_metadata/index.sldl.csv"
|
||||
username = "${SOCKSEEK_USER}"
|
||||
password = "${SOCKSEEK_PASS}"
|
||||
|
||||
name-format = "{artist(/)album(/)track( )title|(_missing_tags/)slsk-foldername(/)slsk-filename}"
|
||||
|
||||
|
||||
@@ -28,9 +28,76 @@ dl trg:
|
||||
link plist dir:
|
||||
python3 maple-py/link.py {{plist}} {{dir}}
|
||||
|
||||
# Dump tracks from Spotify playlists (ids, urls, or spotify: uris)
|
||||
spotify ids="7f5tPIfj21UJI4zfDmH4rx 6sAl8LhU6ediFNJRuQQIQX":
|
||||
cd maple-py && uv run python -m maple.spotify {{ids}} --csv
|
||||
|
||||
# Download via sockseek (csv path, url, search string, ...) using sldl.tpl.conf settings
|
||||
sockseek input:
|
||||
cd maple-py && uv run src/sockseek.py {{input}}
|
||||
|
||||
# Sync data/collection with one or more Spotify playlists (download + m3u8s).
|
||||
collection ids="7f5tPIfj21UJI4zfDmH4rx 6sAl8LhU6ediFNJRuQQIQX":
|
||||
cd maple-py && uv run python -m maple.collection {{ids}}
|
||||
|
||||
# Build a rekordbox USB layout at <dst> from the collection at <src>.
|
||||
# Converts FLAC→MP3 V0, extracts artwork, writes ANLZ (with PWAV/PWV3
|
||||
# blue/white waveforms) and PIONEER/rekordbox/export.pdb.
|
||||
export src="data/collection" dst="data/usb":
|
||||
cd maple-py && uv run python -m maple "{{justfile_directory() / src}}" "{{justfile_directory() / dst}}"
|
||||
|
||||
sync args:
|
||||
s5cmd {{args}}
|
||||
|
||||
# Sync data/usb/ → mounted USB stick (default /media/ntr/MAPLE), then
|
||||
# safely unmount so the stick can be pulled without risk to the FAT
|
||||
# write-back cache.
|
||||
usb dst="/media/ntr/MAPLE":
|
||||
@test -d {{dst}} || (echo "USB not mounted at {{dst}}"; exit 1)
|
||||
@# -c (checksum), NOT --size-only: maple re-exports keep the SAME file size
|
||||
@# (ANLZ section lengths are deterministic) but change content (waveforms,
|
||||
@# pdb), so a size-only check silently skips them and leaves stale data.
|
||||
rsync -rtvh --delete -c --no-perms --no-owner --no-group --modify-window=2 "{{justfile_directory()}}/data/usb/" "{{dst}}/"
|
||||
sync
|
||||
@# Try udisksctl first (no sudo, handles the udev event); fall back to
|
||||
@# plain `umount` if udisks isn't available. Either way the recipe
|
||||
@# exits 0 even if the unmount fails (e.g. someone has a file open).
|
||||
@udisksctl unmount -b "$(findmnt -no SOURCE {{dst}})" 2>/dev/null && echo "unmounted {{dst}}" \
|
||||
|| (umount {{dst}} 2>/dev/null && echo "unmounted {{dst}}") \
|
||||
|| echo "warning: could not unmount {{dst}} — close any file browsers + retry"
|
||||
|
||||
# Build a FAT32 image of data/usb/ — ready to dd onto a stick.
|
||||
# `size` must exceed the data/usb/ tree (rough rule: data size + ~50 MB FAT overhead).
|
||||
usb-image size="2G" img="data/maple.img":
|
||||
@command -v mcopy >/dev/null || (echo "mtools not installed (apt install mtools)"; exit 1)
|
||||
@test -d "{{justfile_directory()}}/data/usb" || (echo "no data/usb (run \`just export\` first)"; exit 1)
|
||||
rm -f "{{justfile_directory()}}/{{img}}"
|
||||
truncate -s {{size}} "{{justfile_directory()}}/{{img}}"
|
||||
mkfs.vfat -F 32 -n MAPLE "{{justfile_directory()}}/{{img}}"
|
||||
cd "{{justfile_directory()}}/data/usb" && mcopy -i "{{justfile_directory()}}/{{img}}" -s -Q -p -m * ::/
|
||||
@echo "image ready:" && ls -lh "{{justfile_directory()}}/{{img}}"
|
||||
|
||||
# DESTRUCTIVE: dd data/maple.img onto a block device (e.g. /dev/sdb).
|
||||
# Double-check `lsblk` first — wrong device wipes that drive.
|
||||
[confirm("usb-flash will OVERWRITE the entire device. Proceed?")]
|
||||
usb-flash dev img="data/maple.img":
|
||||
@test -b {{dev}} || (echo "{{dev}} is not a block device"; exit 1)
|
||||
@test -f "{{justfile_directory()}}/{{img}}" || (echo "no image at {{img}}; run \`just usb-image\` first"; exit 1)
|
||||
@lsblk -no NAME,SIZE,LABEL,MODEL {{dev}}
|
||||
sudo dd if="{{justfile_directory()}}/{{img}}" of={{dev}} bs=4M status=progress oflag=direct conv=fsync
|
||||
sudo sync
|
||||
@echo "done — safe to remove (try: udisksctl power-off -b {{dev}})"
|
||||
|
||||
usb-pdb dst="/Volumes/MAPLE":
|
||||
@test -d {{dst}}/PIONEER/rekordbox || (echo "no rekordbox dir at {{dst}}/PIONEER"; exit 1)
|
||||
COPYFILE_DISABLE=1 cp maple-py/data/MAPLE/PIONEER/rekordbox/export.pdb {{dst}}/PIONEER/rekordbox/export.pdb
|
||||
|
||||
usb-reset dst="/Volumes/MAPLE":
|
||||
@test -d {{dst}} || (echo "USB not mounted at {{dst}}"; exit 1)
|
||||
rm -rf {{dst}}/Contents {{dst}}/PIONEER
|
||||
COPYFILE_DISABLE=1 rsync -rtvh --no-perms --no-owner --no-group --modify-window=2 maple-py/data/MAPLE/Contents/ {{dst}}/Contents/
|
||||
COPYFILE_DISABLE=1 rsync -rtvh --no-perms --no-owner --no-group --modify-window=2 maple-py/data/MAPLE/PIONEER/ {{dst}}/PIONEER/
|
||||
|
||||
maple-setup:
|
||||
bash ./etc/maple_setup.sh
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
3.14
|
||||
@@ -0,0 +1,57 @@
|
||||
# maple-py conventions
|
||||
|
||||
Python conventions for the `maple-py/` package. These OVERRIDE default behavior;
|
||||
follow them exactly. (General repo guidance is in the root `CLAUDE.md`; 4-space
|
||||
indentation applies here too.)
|
||||
|
||||
The package is pyright-checked (basic mode, Python 3.14). `Track`
|
||||
(`src/maple/track.py`) and the `Analysis` family (`src/maple/analyze.py`) are the
|
||||
reference examples for all of the below.
|
||||
|
||||
## Imports at the top of the file
|
||||
|
||||
All imports go at module top. No lazy/local imports inside functions. Never wrap
|
||||
an import in `try/except ImportError` to degrade gracefully — if an integral
|
||||
dependency (e.g. `mixxx_analyzer`, `scipy`) is missing, the process must **die at
|
||||
startup** via the top-level import failing. The only acceptable indented import
|
||||
is an `if TYPE_CHECKING:` block to break a circular type-only reference.
|
||||
|
||||
## Fail loud — never swallow into None
|
||||
|
||||
"Don't swallow into None" is a general principle at every level, including call
|
||||
sites. An operation that fails must **let the exception propagate** — never catch
|
||||
it and return `None` / an empty / degraded result. BPM/beat analysis and waveform
|
||||
extraction are integral: `_mixxx_analyze` raises rather than returning None,
|
||||
`_decode_pcm` raises (`RuntimeError`) if ffmpeg is missing or fails to decode
|
||||
rather than returning an empty array, and every `Analysis.analyze` call site (in
|
||||
`anlz.py`, `cli.py`, `collection/core.py`) lets failures abort the run (including
|
||||
not catching around `ThreadPoolExecutor` `fut.result()`).
|
||||
|
||||
Reserve `None` returns for *legitimate absence* (e.g. a cache miss in
|
||||
`Analysis.load`), never for *failure*. The one tolerated catch is a genuinely
|
||||
*optional* side-effect failing (e.g. writing the optimization cache via
|
||||
`analysis.save()`), which logs and continues because it doesn't affect
|
||||
correctness.
|
||||
|
||||
## Specific types over generic ones
|
||||
|
||||
- **Dataclasses, not bare dicts**, for structured values passed around.
|
||||
- **`pathlib.Path`, not `str`**, for local filesystem paths (strings are fine for
|
||||
serialization / rekordbox-USB paths that are inherently string data).
|
||||
- **`typing.NamedTuple` (or a small dataclass), not bare tuples**, for fixed
|
||||
records (e.g. `Beat(number, tempo_x100, time_ms)`, `Bands(low, mid, high)`).
|
||||
- **No `None`-valued fields.** Absent values resolve at instantiation to a
|
||||
concrete sentinel — `0.0` (0.0 = unknown), `""`, `[]`, `b""` — per `Track`.
|
||||
`None` is acceptable only as a *factory return* signalling absence, never as a
|
||||
stored field.
|
||||
|
||||
Idiomatic dataclass shape: plain `@dataclass` (no frozen/slots), `@property` for
|
||||
computed views, `@classmethod from_*()` factories, concrete field defaults.
|
||||
|
||||
## Type every function parameter
|
||||
|
||||
Always annotate every parameter, including nested/inner functions and
|
||||
`@classmethod` factory args (`self`/`cls` exempt). No bare `def f(res):`. Use the
|
||||
real type even from a third-party lib — e.g. `mixxx_analyzer.AnalysisResult`
|
||||
(referenced directly; no extra import needed under
|
||||
`from __future__ import annotations`).
|
||||
@@ -0,0 +1,112 @@
|
||||
# maple-py — rekordbox `export.pdb` writer + reverse-engineering notes
|
||||
|
||||
Python tooling to generate a rekordbox USB export (`PIONEER/rekordbox/export.pdb`
|
||||
+ `exportExt.pdb` + `USBANLZ` analysis files) so a CDJ/XDJ can read a
|
||||
self-assembled music library. The writer lives in `src/maple/pdb/`.
|
||||
|
||||
This README also documents what we learned reverse-engineering rekordbox's own
|
||||
exporter, because the on-disk format has real unknowns / magic numbers that the
|
||||
public kaitai schema (`src/kaitai/rekordbox_pdb.py`) only partially decodes.
|
||||
|
||||
## The format, briefly
|
||||
|
||||
`export.pdb` is a **DeviceSQL** database: a flat file of 4096-byte **pages**,
|
||||
grouped into 20 **tables** (tracks, artists, albums, playlists, artwork,
|
||||
colors, columns, history, and several still-unknown ones). Page 0 is the file
|
||||
header + table directory; each table is a linked list of pages. See
|
||||
`src/maple/pdb/{writer,pager,rows,strings}.py`.
|
||||
|
||||
## Reverse-engineering findings (rekordbox 7.0.5, Windows)
|
||||
|
||||
### Where the export code lives
|
||||
- The exporter is in **`rekordbox.exe`** (the 89 MB main binary), **not**
|
||||
`rekordboxAgent.exe`. Confirmed by string scan: only `rekordbox.exe` contains
|
||||
`USBANLZ`, `PIONEER`, `djmd…` table names, `.pdb`. It's a JUCE C++ app.
|
||||
- `export.pdb` is written by rekordbox's **bundled DeviceSQL "Page Manager"**
|
||||
engine — functions prefixed `dsql*` / `edb_dyn_*` (error strings:
|
||||
`dsqlOpenDeviceDB`, `edb_dyn_CriticalSectionOn ERROR`,
|
||||
`Page Manager (Close/)Open returns ERROR`).
|
||||
- **Key consequence:** the page-level "magic numbers" maple guesses
|
||||
(page flags `0x64`/`0x44`, `0x1fff` markers, `0x3ec`, free/used sizes) are
|
||||
**DeviceSQL internals**, not rekordbox-specific. They are NOT immediate
|
||||
constants in the code — they're produced by the page-manager logic, so they
|
||||
must be read out of the `dsql*`/`edb_dyn_*` functions, not grep'd.
|
||||
|
||||
### A rekordbox USB export writes three databases
|
||||
| file | format | writer |
|
||||
|------|--------|--------|
|
||||
| `export.pdb` | DeviceSQL (CDJ reads this) | DeviceSQL Page Manager |
|
||||
| `exportExt.pdb` | DeviceSQL extended | same |
|
||||
| `exportLibrary.db` | **SQLite** (rekordbox's own) | bundled `sqlite3_*` |
|
||||
| `USBANLZ/**/ANLZ*.DAT/.EXT/.2EX` | per-track waveform/beatgrid | main app |
|
||||
|
||||
### `export.pdb` serializer call stack (live `WriteFile` backtrace → Ghidra)
|
||||
Ghidra static address = `0x140000000 + module_offset` (Ghidra image base is
|
||||
`0x140000000`; the ASLR runtime base varies per launch — never hardcode it,
|
||||
read `module+offset` from Frida).
|
||||
|
||||
```
|
||||
0x1420e8804 calls WriteFile ─┐
|
||||
0x1420d7358 │ DeviceSQL write plumbing
|
||||
0x1420d6e74 │ (transactions, page I/O)
|
||||
0x1420d62a4 │
|
||||
0x1420eac94 │
|
||||
0x1420e07d8 │
|
||||
0x1420de140 ─┘
|
||||
FUN_141d75c20 ← dsqlOpenDeviceDB / Page Manager (export.pdb-specific)
|
||||
0x141cad100
|
||||
0x141cac390
|
||||
0x141b8b4c0
|
||||
0x141aca850
|
||||
0x1423c9ab0 ← outer export driver
|
||||
```
|
||||
Other anchors found by string xref: `FUN_14119d2d0` (export orchestrator,
|
||||
builds `/PIONEER/rekordbox/export.pdb`, walks "Contents"), `FUN_140f37e50`
|
||||
(builds `PIONEER`/`USBANLZ` paths).
|
||||
|
||||
### Gotchas
|
||||
- **Incremental writes:** rekordbox rewrites only the *changed* pages (seeked
|
||||
`WriteFile`s), so a Frida capture is partial. The complete file is on the USB
|
||||
(`F:\PIONEER\rekordbox\export.pdb`) — copy it from there for a full diff.
|
||||
- **CIG / injection:** `rekordboxAgent.exe` refuses live Frida injection
|
||||
(Code Integrity Guard). The pdb writer (main `rekordbox.exe`) is hookable, but
|
||||
only via **child-gating at spawn** — Windows has **no spawn-gating**.
|
||||
|
||||
## Tooling
|
||||
|
||||
### Static analysis (Ghidra, headless, in WSL)
|
||||
- JDK 21 (`~/jdk/jdk-21.0.11+10`), Ghidra 12.1.2. Ghidra rejects JDK 25 — use 21.
|
||||
- Analyzed project: `~/ghidra_proj` (project `rbx`, program `rekordbox.exe`).
|
||||
Full analysis ≈ 1 h with `GHIDRA_HEADLESS_MAXMEM=10G -max-cpu 8`.
|
||||
- Re-run a script without re-analyzing:
|
||||
```bash
|
||||
"$GHIDRA/support/analyzeHeadless" ~/ghidra_proj rbx -process rekordbox.exe \
|
||||
-noanalysis -scriptPath ~/ghidra_scripts -postScript FindPdb.java
|
||||
```
|
||||
Scripts: `~/ghidra_scripts/FindPdb.java` (string+constant xref search),
|
||||
`DecPdb.java` (decompile an address list), `CallTree.java` (call-tree drill).
|
||||
|
||||
### Dynamic capture (Frida, on Windows)
|
||||
- `tools/rbx_pdb_capture.js` — Frida agent. Hooks `NtCreateFile`/`NtWriteFile`/
|
||||
`NtClose` + `NtCreateSection`/`NtMapViewOfSection` (memory-mapped writes).
|
||||
Reconstructs files under `PIONEER`/`*.pdb` to `captures/`, prints each write's
|
||||
backtrace as `rekordbox.exe+0xOFFSET` (→ Ghidra `0x140000000 + offset`).
|
||||
- `tools/capture.mjs` — Node host (`npm i frida`). Attaches to all rekordbox
|
||||
processes, streams `[pid]`-tagged output live, stays alive, **child-gates** so
|
||||
the spawn writer is hooked at birth. Clears `captures/` on start.
|
||||
```powershell
|
||||
npm i frida # once
|
||||
node tools\capture.mjs --spawn # launch rekordbox under the gate, then Export to USB
|
||||
```
|
||||
|
||||
## Diffing maple's output vs rekordbox
|
||||
`compare_pdb.py` parses two pdbs with the kaitai schema and summarises tables/
|
||||
rows. Reference real exports: `data/PIONEER/rekordbox/export.pdb`,
|
||||
`data/acemo/PIONEER/rekordbox/export.pdb`.
|
||||
|
||||
## Status / next steps
|
||||
- Writer fully located; capture + diff pipeline works.
|
||||
- To decode a specific page magic number: search the `dsql*`/`edb_dyn_*`
|
||||
functions (image region `0x142000000+`) for the page-header builder.
|
||||
- To fix a rejected export: diff a complete USB `export.pdb` against maple's
|
||||
output field-by-field; resolve any wrong value via the matching `dsql` function.
|
||||
@@ -0,0 +1,57 @@
|
||||
"""Parse two .pdb files with the kaitai schema and summarise differences."""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent / "src"))
|
||||
from io import BytesIO
|
||||
from kaitaistruct import KaitaiStream
|
||||
from kaitai.rekordbox_pdb import RekordboxPdb
|
||||
|
||||
|
||||
PAGE_TYPE_NAMES = {t.value: t.name for t in RekordboxPdb.PageType}
|
||||
|
||||
|
||||
def summarise(path: Path):
|
||||
data = path.read_bytes()
|
||||
pdb = RekordboxPdb(is_ext=False, _io=KaitaiStream(BytesIO(data)))
|
||||
print(f"\n=== {path} ({path.stat().st_size} bytes) ===")
|
||||
print(f"page_size={pdb.len_page} num_tables={pdb.num_tables} next_unused_page={pdb.next_unused_page} sequence={pdb.sequence}")
|
||||
for tbl in pdb.tables:
|
||||
ptype = PAGE_TYPE_NAMES.get(tbl.type.value, str(tbl.type))
|
||||
page_count = 0
|
||||
row_count = 0
|
||||
err = None
|
||||
try:
|
||||
ref = tbl.first_page
|
||||
while True:
|
||||
page = ref.body
|
||||
page_count += 1
|
||||
if page.is_data_page:
|
||||
for rg in page.row_groups:
|
||||
for rpw in rg.rows:
|
||||
if rpw.present:
|
||||
row_count += 1
|
||||
if ref.index == tbl.last_page.index:
|
||||
break
|
||||
ref = page.next_page
|
||||
if page_count > 5000:
|
||||
break
|
||||
except Exception as e:
|
||||
err = f"{type(e).__name__}: {e}"
|
||||
line = f" {ptype:24s} first={tbl.first_page.index:>4} last={tbl.last_page.index:>4} pages={page_count:>3} rows={row_count}"
|
||||
if err:
|
||||
line += f" ERR={err}"
|
||||
print(line)
|
||||
return pdb
|
||||
|
||||
|
||||
def main():
|
||||
base = Path(__file__).parent
|
||||
ref = base / "data/MAPLE/PIONEER/rekordbox/export.pdb"
|
||||
ours = base / "data/usb/PIONEER/rekordbox/export.pdb"
|
||||
summarise(ref)
|
||||
summarise(ours)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+501
@@ -0,0 +1,501 @@
|
||||
{
|
||||
"name": "maple-py",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"dependencies": {
|
||||
"frida": "^17.11.0"
|
||||
}
|
||||
},
|
||||
"node_modules/balanced-match": {
|
||||
"version": "4.0.4",
|
||||
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-4.0.4.tgz",
|
||||
"integrity": "sha512-BLrgEcRTwX2o6gGxGOCNyMvGSp35YofuYzw9h1IMTRmKqttAZZVU67bdb9Pr2vUHA8+j3i2tJfjO6C6+4myGTA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": "18 || 20 || >=22"
|
||||
}
|
||||
},
|
||||
"node_modules/base64-js": {
|
||||
"version": "1.5.1",
|
||||
"resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",
|
||||
"integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
],
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/bindings": {
|
||||
"version": "1.5.0",
|
||||
"resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",
|
||||
"integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"file-uri-to-path": "1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/bl": {
|
||||
"version": "4.1.0",
|
||||
"resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz",
|
||||
"integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"buffer": "^5.5.0",
|
||||
"inherits": "^2.0.4",
|
||||
"readable-stream": "^3.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/brace-expansion": {
|
||||
"version": "5.0.6",
|
||||
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz",
|
||||
"integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"balanced-match": "^4.0.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": "18 || 20 || >=22"
|
||||
}
|
||||
},
|
||||
"node_modules/buffer": {
|
||||
"version": "5.7.1",
|
||||
"resolved": "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz",
|
||||
"integrity": "sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"base64-js": "^1.3.1",
|
||||
"ieee754": "^1.1.13"
|
||||
}
|
||||
},
|
||||
"node_modules/chownr": {
|
||||
"version": "1.1.4",
|
||||
"resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",
|
||||
"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/decompress-response": {
|
||||
"version": "6.0.0",
|
||||
"resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",
|
||||
"integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"mimic-response": "^3.1.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/deep-extend": {
|
||||
"version": "0.6.0",
|
||||
"resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",
|
||||
"integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=4.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/detect-libc": {
|
||||
"version": "2.1.2",
|
||||
"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
|
||||
"integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",
|
||||
"license": "Apache-2.0",
|
||||
"engines": {
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/end-of-stream": {
|
||||
"version": "1.4.5",
|
||||
"resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz",
|
||||
"integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"once": "^1.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/expand-template": {
|
||||
"version": "2.0.3",
|
||||
"resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz",
|
||||
"integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==",
|
||||
"license": "(MIT OR WTFPL)",
|
||||
"engines": {
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/file-uri-to-path": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz",
|
||||
"integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/frida": {
|
||||
"version": "17.11.0",
|
||||
"resolved": "https://registry.npmjs.org/frida/-/frida-17.11.0.tgz",
|
||||
"integrity": "sha512-ZymNV1mr8lWMP2vo1Ed4D7w1/8VQRVeoZ439sCZN/vKjj/E0UhfQP4BYsRAUrCQZ4nBzfRNN4BouA+WvOZ1z5w==",
|
||||
"hasInstallScript": true,
|
||||
"license": "LGPL-2.0 WITH WxWindows-exception-3.1",
|
||||
"dependencies": {
|
||||
"bindings": "^1.5.0",
|
||||
"minimatch": "^10.0.1",
|
||||
"prebuild-install": "^7.1.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=16"
|
||||
}
|
||||
},
|
||||
"node_modules/fs-constants": {
|
||||
"version": "1.0.0",
|
||||
"resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",
|
||||
"integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/github-from-package": {
|
||||
"version": "0.0.0",
|
||||
"resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",
|
||||
"integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/ieee754": {
|
||||
"version": "1.2.1",
|
||||
"resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz",
|
||||
"integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
],
|
||||
"license": "BSD-3-Clause"
|
||||
},
|
||||
"node_modules/inherits": {
|
||||
"version": "2.0.4",
|
||||
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
|
||||
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/ini": {
|
||||
"version": "1.3.8",
|
||||
"resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",
|
||||
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
|
||||
"license": "ISC"
|
||||
},
|
||||
"node_modules/mimic-response": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",
|
||||
"integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/sindresorhus"
|
||||
}
|
||||
},
|
||||
"node_modules/minimatch": {
|
||||
"version": "10.2.5",
|
||||
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-10.2.5.tgz",
|
||||
"integrity": "sha512-MULkVLfKGYDFYejP07QOurDLLQpcjk7Fw+7jXS2R2czRQzR56yHRveU5NDJEOviH+hETZKSkIk5c+T23GjFUMg==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"dependencies": {
|
||||
"brace-expansion": "^5.0.5"
|
||||
},
|
||||
"engines": {
|
||||
"node": "18 || 20 || >=22"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/isaacs"
|
||||
}
|
||||
},
|
||||
"node_modules/minimist": {
|
||||
"version": "1.2.8",
|
||||
"resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
|
||||
"integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
|
||||
"license": "MIT",
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/mkdirp-classic": {
|
||||
"version": "0.5.3",
|
||||
"resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
|
||||
"integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/napi-build-utils": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",
|
||||
"integrity": "sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/node-abi": {
|
||||
"version": "3.92.0",
|
||||
"resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.92.0.tgz",
|
||||
"integrity": "sha512-KdHvFWZjEKDf0cakgFjebl371GPsISX2oZHcuyKqM7DtogIsHrqKeLTo8wBHxaXRAQlY2PsPlZmfo+9ZCxEREQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"semver": "^7.3.5"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/once": {
|
||||
"version": "1.4.0",
|
||||
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
|
||||
"integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"wrappy": "1"
|
||||
}
|
||||
},
|
||||
"node_modules/prebuild-install": {
|
||||
"version": "7.1.3",
|
||||
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
|
||||
"integrity": "sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==",
|
||||
"deprecated": "No longer maintained. Please contact the author of the relevant native addon; alternatives are available.",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"detect-libc": "^2.0.0",
|
||||
"expand-template": "^2.0.3",
|
||||
"github-from-package": "0.0.0",
|
||||
"minimist": "^1.2.3",
|
||||
"mkdirp-classic": "^0.5.3",
|
||||
"napi-build-utils": "^2.0.0",
|
||||
"node-abi": "^3.3.0",
|
||||
"pump": "^3.0.0",
|
||||
"rc": "^1.2.7",
|
||||
"simple-get": "^4.0.0",
|
||||
"tar-fs": "^2.0.0",
|
||||
"tunnel-agent": "^0.6.0"
|
||||
},
|
||||
"bin": {
|
||||
"prebuild-install": "bin.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/pump": {
|
||||
"version": "3.0.4",
|
||||
"resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",
|
||||
"integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"end-of-stream": "^1.1.0",
|
||||
"once": "^1.3.1"
|
||||
}
|
||||
},
|
||||
"node_modules/rc": {
|
||||
"version": "1.2.8",
|
||||
"resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",
|
||||
"integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",
|
||||
"license": "(BSD-2-Clause OR MIT OR Apache-2.0)",
|
||||
"dependencies": {
|
||||
"deep-extend": "^0.6.0",
|
||||
"ini": "~1.3.0",
|
||||
"minimist": "^1.2.0",
|
||||
"strip-json-comments": "~2.0.1"
|
||||
},
|
||||
"bin": {
|
||||
"rc": "cli.js"
|
||||
}
|
||||
},
|
||||
"node_modules/readable-stream": {
|
||||
"version": "3.6.2",
|
||||
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",
|
||||
"integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"inherits": "^2.0.3",
|
||||
"string_decoder": "^1.1.1",
|
||||
"util-deprecate": "^1.0.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 6"
|
||||
}
|
||||
},
|
||||
"node_modules/safe-buffer": {
|
||||
"version": "5.2.1",
|
||||
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
|
||||
"integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
],
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/semver": {
|
||||
"version": "7.8.2",
|
||||
"resolved": "https://registry.npmjs.org/semver/-/semver-7.8.2.tgz",
|
||||
"integrity": "sha512-c8jsqUZm3omBOI66G90z1Dyw5z622G8oLG+omfsHBJf3CWQTlOcwOjvOG6wtiNfW6anKm/eA39LMwMtMez2TiQ==",
|
||||
"license": "ISC",
|
||||
"bin": {
|
||||
"semver": "bin/semver.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/simple-concat": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz",
|
||||
"integrity": "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
],
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/simple-get": {
|
||||
"version": "4.0.1",
|
||||
"resolved": "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz",
|
||||
"integrity": "sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA==",
|
||||
"funding": [
|
||||
{
|
||||
"type": "github",
|
||||
"url": "https://github.com/sponsors/feross"
|
||||
},
|
||||
{
|
||||
"type": "patreon",
|
||||
"url": "https://www.patreon.com/feross"
|
||||
},
|
||||
{
|
||||
"type": "consulting",
|
||||
"url": "https://feross.org/support"
|
||||
}
|
||||
],
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"decompress-response": "^6.0.0",
|
||||
"once": "^1.3.1",
|
||||
"simple-concat": "^1.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/string_decoder": {
|
||||
"version": "1.3.0",
|
||||
"resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
|
||||
"integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"safe-buffer": "~5.2.0"
|
||||
}
|
||||
},
|
||||
"node_modules/strip-json-comments": {
|
||||
"version": "2.0.1",
|
||||
"resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
|
||||
"integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/tar-fs": {
|
||||
"version": "2.1.4",
|
||||
"resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.4.tgz",
|
||||
"integrity": "sha512-mDAjwmZdh7LTT6pNleZ05Yt65HC3E+NiQzl672vQG38jIrehtJk/J3mNwIg+vShQPcLF/LV7CMnDW6vjj6sfYQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"chownr": "^1.1.1",
|
||||
"mkdirp-classic": "^0.5.2",
|
||||
"pump": "^3.0.0",
|
||||
"tar-stream": "^2.1.4"
|
||||
}
|
||||
},
|
||||
"node_modules/tar-stream": {
|
||||
"version": "2.2.0",
|
||||
"resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz",
|
||||
"integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"bl": "^4.0.3",
|
||||
"end-of-stream": "^1.4.1",
|
||||
"fs-constants": "^1.0.0",
|
||||
"inherits": "^2.0.3",
|
||||
"readable-stream": "^3.1.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/tunnel-agent": {
|
||||
"version": "0.6.0",
|
||||
"resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",
|
||||
"integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"safe-buffer": "^5.0.1"
|
||||
},
|
||||
"engines": {
|
||||
"node": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/util-deprecate": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
|
||||
"integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/wrappy": {
|
||||
"version": "1.0.2",
|
||||
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
|
||||
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
|
||||
"license": "ISC"
|
||||
}
|
||||
}
|
||||
}
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
{
|
||||
"dependencies": {
|
||||
"frida": "^17.11.0"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
[project]
|
||||
name = "maple"
|
||||
version = "0.1.0"
|
||||
description = "Add your description here"
|
||||
readme = "README.md"
|
||||
authors = [
|
||||
{ name = "Nathan Rashleigh", email = "nathan.rashleigh@switchdin.com" }
|
||||
]
|
||||
requires-python = ">=3.14"
|
||||
dependencies = [
|
||||
"aiohttp>=3.10",
|
||||
"kaitaistruct>=0.11",
|
||||
"loguru>=0.7",
|
||||
"mixxx-analyzer>=0.1.2",
|
||||
"mutagen>=1.47.0",
|
||||
"pillow>=12.2.0",
|
||||
"pyrekordbox>=0.4.4",
|
||||
"scipy>=1.10",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
maple = "maple.cli:main"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"pytest>=9.0.3",
|
||||
]
|
||||
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"include": [
|
||||
"src",
|
||||
"tests"
|
||||
],
|
||||
"extraPaths": [
|
||||
"src"
|
||||
],
|
||||
"exclude": [
|
||||
"**/__pycache__",
|
||||
"**/.venv",
|
||||
"data",
|
||||
"re"
|
||||
],
|
||||
"venvPath": ".",
|
||||
"venv": ".venv",
|
||||
"pythonVersion": "3.14",
|
||||
"pythonPlatform": "Linux",
|
||||
"typeCheckingMode": "basic",
|
||||
"reportMissingImports": "error",
|
||||
"reportMissingModuleSource": "warning",
|
||||
"useLibraryCodeForTypes": true
|
||||
}
|
||||
@@ -0,0 +1,609 @@
|
||||
# export.pdb value-construction rules (RE findings)
|
||||
|
||||
This file records the exact rules rekordbox uses to populate `export.pdb`
|
||||
fields that maple currently writes wrong or as zero. Each section is a
|
||||
self-contained spec for the `maple` writer to implement. Evidence is from
|
||||
reverse-engineering `rekordbox.exe` (DeviceSQL "Page Manager", `dsql*` /
|
||||
`edb_dyn_*`) in Ghidra plus byte-level analysis of the reference export
|
||||
`maple-py/data/acemo/PIONEER/rekordbox/export.pdb` (6 tracks, 1 album,
|
||||
1 artist, 1 key, 6 artwork).
|
||||
|
||||
Reference-pdb facts used throughout:
|
||||
- `len_page` = 4096.
|
||||
- Row index grows backward from end of page; each group spans 36 (0x24) bytes:
|
||||
`group_base = len_page - group_index*36`. Within a group, `present_flags`
|
||||
(u2) live at `group_base-4`; the row offset for slot `i` (0..15) is the u2 at
|
||||
`group_base-(6+2*i)`; offsets are relative to the heap start (0x28, i.e. end
|
||||
of the 0x28-byte page header).
|
||||
|
||||
---
|
||||
|
||||
## 1. Track row `index_shift` (offset 0x02, u2) — SOLVED
|
||||
|
||||
**Field:** `TrackRow.index_shift` (also present on artist/album/genre/label/
|
||||
key rows — every row type whose layout begins `subtype:u2, index_shift:u2`).
|
||||
|
||||
**Rule:**
|
||||
|
||||
```
|
||||
index_shift = slot_index * 0x20 # i.e. slot_index << 5
|
||||
```
|
||||
|
||||
where `slot_index` is the row's **0-based position in the page row index**
|
||||
(`group_index*16 + slot_within_group`), NOT its ordinal among present rows and
|
||||
NOT its position in the heap. It resets to 0 at the start of every data page.
|
||||
|
||||
Key consequences for the writer:
|
||||
- The first row index slot (slot 0) gets `index_shift = 0`.
|
||||
- Deleted/absent slots still consume a slot number, so a gap in
|
||||
`present_flags` produces a gap in `index_shift`. (See evidence: the AceMo
|
||||
page has slot 5 deleted, so the row in slot 6 has `index_shift = 0xc0`,
|
||||
skipping `0xa0`.)
|
||||
- `index_shift` is purely a function of the row's slot, so the value is fully
|
||||
reproducible from the page layout maple already controls — there is no
|
||||
external/runtime input. If maple writes each track to slot `i`, set
|
||||
`index_shift = i*0x20`.
|
||||
|
||||
**Evidence:**
|
||||
- Reference AceMo `tracks` page (page 2): present slots are 1,2,3,4,6 (slots 0
|
||||
and 5 are deleted), with `index_shift` = 0x20,0x40,0x60,0x80,0xc0
|
||||
respectively. Overflow page (page 48) has one row in slot 1 →
|
||||
`index_shift = 0x20`. The 0x80→0xc0 jump across the deleted slot 5 proves the
|
||||
value tracks `slot_index`, not present-count.
|
||||
- Cross-checked against `artists` (page 5) and `albums` (page 7) rows: every
|
||||
row satisfies `index_shift == slot_index*0x20` with **0 mismatches** over all
|
||||
present rows in the reference file
|
||||
(`maple-py/re/scripts`-style raw parse, 8 rows checked, see RE notes).
|
||||
- Ghidra: the value is assigned inside the DeviceSQL Page Manager record-insert
|
||||
path, which is reached only through vtable dispatch
|
||||
(`(**(code**)(*obj + 0x4d0))(obj,...)` etc. in `FUN_141b8b4c0` /
|
||||
`FUN_141cac390`), so there is no single statically-anchored
|
||||
`index_shift = slot<<5` instruction to cite; a constant-scan for `shl ,5`
|
||||
paired with the row-index masks (0x1fff/0x7ff) and stride 0x24 produced only
|
||||
CRT/`printf("%a")` false positives (`FUN_14232a86c`). The empirical rule
|
||||
above is exact and zero-mismatch, so confidence is **high** on the formula;
|
||||
confidence on "no hidden runtime input" is high because the value is fully
|
||||
explained by slot position alone.
|
||||
|
||||
**Confidence:** high.
|
||||
|
||||
---
|
||||
|
||||
## 2. Track row 0x14 / 0x18 / 0x1a (opaque id blob) — SOLVED (stored value, not a formula)
|
||||
|
||||
**Fields:** `TrackRow.opaque_track_id_lo` (u4 @0x14), `TrackRow.opaque_seed_lo`
|
||||
(u2 @0x18), `TrackRow.opaque_seed_hi` (u2 @0x1a). Together these are one
|
||||
8-byte little-endian value:
|
||||
|
||||
```
|
||||
blob_u64 = opaque_track_id_lo | (opaque_seed << 32)
|
||||
opaque_seed = opaque_seed_lo | (opaque_seed_hi << 16) # u4 at offset 0x18
|
||||
```
|
||||
|
||||
(@flesniak's schema labels these "Some ID? unknown", "always 19048?",
|
||||
"always 30967?". The "19048"/"30967" = 0x4A68/0x78F7 are only correct for some
|
||||
older exports; they are NOT universal — see below.)
|
||||
|
||||
**What they are (proven empirically across 5 GENUINE rekordbox exports):**
|
||||
|
||||
> NOTE (2026-06-09 correction): a prior draft included a 5th row
|
||||
> "usb (176), seq=1, seed 0x78f74a68, lo all 0" and concluded rekordbox writes
|
||||
> lo=0 / seed 0x78f74a68 for unanalyzed tracks. **That file
|
||||
> (`data/usb/.../export.pdb`) is maple's OWN output, not a rekordbox export**
|
||||
> (it is the compare_pdb.py "ours" path). Reading it back was circular; that
|
||||
> claim is WRONG and has been dropped. Provenance rule: maple output = seed
|
||||
> `0x78f74a68` + ALL per-track lo == 0; genuine rekordbox = per-install nonzero
|
||||
> seed + nonzero per-track lo.
|
||||
|
||||
| export (tracks) | header.seq | opaque_seed | opaque_track_id_lo |
|
||||
|----------------------------|-----------:|---------------|-------------------------------|
|
||||
| acemo (6) | 353 | `0x21b4a7ab` | 6/6 nonzero |
|
||||
| acemo_fresh (7, same lib) | 35 | `0x21b4a7ab` | 7/7 nonzero |
|
||||
| lib_multi (10, same lib) | 56 | `0x21b4a7ab` | 10/10 nonzero |
|
||||
| MAPLE (176, install B) | 546 | `0xd6d3a567` | 176/176 nonzero |
|
||||
| data/PIONEER (847, inst. B)| 4211 | `0xd6d3a567` | 847/847 nonzero |
|
||||
|
||||
Across **all 1046 real tracks in 5 genuine exports: 100% nonzero per-track lo,
|
||||
exactly one nonzero per-install seed per file. There is NO evidence rekordbox
|
||||
ever writes lo=0 or seed `0x78f74a68`.** The lo values span nearly the full
|
||||
28-bit space (`0x0003edeb`..`0x0ffe7f4e`).
|
||||
|
||||
Findings:
|
||||
1. **`opaque_seed` is constant within a single export** (exactly one value per
|
||||
file) and is **per-rekordbox-library/install**: acemo/acemo_fresh/lib_multi
|
||||
are three exports of the same library and all share `0x21b4a7ab`; the separate
|
||||
install behind MAPLE + data/PIONEER uses `0xd6d3a567`. (The old maple hardcode
|
||||
`0xa567`/`0xd6d3` = `0xd6d3a567` was correct only for install B.) The seed is
|
||||
NOT a binary constant (proven: no `0xa7ab`/`0x21b4`/`0xd6d3`/`0xa567`
|
||||
immediate or data-byte sequence in rekordbox.exe), so it is derived at runtime
|
||||
from the per-library DB (most likely a hash/truncation of the master.db's
|
||||
per-library UUID or creation marker). maple cannot reproduce a specific
|
||||
install's seed (see recommendation).
|
||||
2. **`opaque_track_id_lo` is a stable per-track value within a library.** Every
|
||||
track that appears in both the acemo and acemo_fresh exports has an
|
||||
**identical** lo (matched by title): "All My Life"→`0x07ff5a56`,
|
||||
"Smooth Move"→`0x0a20de34`, "Jelly Fish Jam"→`0x0f8fd598`, etc. It is NOT
|
||||
per-export-random.
|
||||
3. **It is NOT derivable from the track's file metadata.** Tested and ruled out
|
||||
(no match, 28-bit and full, utf-8/utf-16, several hash funcs): crc32 / adler32
|
||||
/ md5 / sha1 / fnv1 / fnv1a / djb2 of the title, filename, full
|
||||
`/Contents/...` path, and crc32 of the ANLZ `.DAT`/`.EXT` and the `.mp3`
|
||||
bytes. Also not the USBANLZ folder id, not sequential, not evenly spaced.
|
||||
4. **The values are runtime-derived, not binary constants.** A full
|
||||
instruction-immediate scan of `rekordbox.exe` found NO occurrence of either
|
||||
real seed (`0x21b4a7ab`, `0xd6d3a567`), the per-track lo values, or the
|
||||
track-row `bitmask` constant `0xc0700`. So these come from rekordbox's
|
||||
internal database (master.db / `djmdContent` + a per-library seed), copied
|
||||
into the row by the DeviceSQL record builder (vtable-dispatched, not
|
||||
statically anchorable). (`0x78f74a68` does not appear either — and per the
|
||||
correction above it was never a real rekordbox value here, just maple's
|
||||
placeholder.)
|
||||
|
||||
**Round-2 evidence (2026-06-09, after team-lead supplied acemo_fresh and asked
|
||||
to chase an audio-content hash):**
|
||||
5. **Audio-content-hash hypothesis: DISPROVEN.** Hashed the actual `.mp3`
|
||||
bytes in 8 framings — full file, ID3v2-stripped (audio frames only),
|
||||
first 1k/2k/4k/64k, last 64k, ID3-stripped-first-64k — each with crc32,
|
||||
adler32, md5 (lo/hi, LE+BE), sha1 (LE+BE), sha256, fnv1a, both full-32 and
|
||||
masked-to-28-bit. NONE matched any track's `opaque_track_id_lo`. So it is
|
||||
not a hash of the audio file content (at least not of the file bytes; a hash
|
||||
of decoded PCM cannot be ruled out but there is no evidence for it and it is
|
||||
not reproducible anyway).
|
||||
6. **Source DB confirmed, and it is ENCRYPTED.** Both `exportLibrary.db` (next
|
||||
to the pdb) and the captured `master.db`
|
||||
(`maple-py/captures/p17328_master.db`) fail to open as plain SQLite
|
||||
("file is not a database") — rekordbox 6/7 encrypts them with SQLCipher.
|
||||
The `opaque_track_id_lo` originates there; it cannot be read without the key.
|
||||
7. **The value is a pure stored field, used nowhere else.** It appears in the
|
||||
export.pdb ONLY at each track row's own offset 0x14 (present and stale/deleted
|
||||
rows alike); it does NOT appear in exportExt.pdb (no tag_tracks/ContentID
|
||||
link), nor in the ANLZ `.DAT`/`.EXT` (searched LE+BE), nor does it equal the
|
||||
USBANLZ folder id. So it is not a foreign key the player resolves.
|
||||
8. **Range note (suggestive, not conclusive):** across the 5 real exports the
|
||||
lo values span nearly the full 28-bit space (`0x0003edeb`..`0x0ffe7f4e`),
|
||||
the band of rekordbox-6/7 9-digit `djmdContent.ID` integers, so the most
|
||||
likely identity is "the rekordbox internal Content ID" — a stored
|
||||
random/sequential DB id, NOT a content-derived hash. Consistent with every
|
||||
other finding and with it being unreadable (encrypted DB).
|
||||
|
||||
**Conclusion:** this 8-byte blob is a **stored rekordbox-internal id (most
|
||||
likely the encrypted-DB Content ID), not a function of the track metadata or
|
||||
audio bytes maple has. It cannot be reproduced byte-for-byte** for an arbitrary
|
||||
track without rekordbox's own (encrypted) DB. The audio-content-hash idea is
|
||||
ruled out. There is **no "unanalyzed/zero" form** in any real export — every
|
||||
real track carries a nonzero seed and a nonzero lo.
|
||||
|
||||
**Recommendation for maple's writer:**
|
||||
- Players/crate-digger treat this blob as opaque (never decoded) and play fine
|
||||
regardless — so any value works for playback.
|
||||
- For maximum faithfulness to genuine rekordbox output (which never has
|
||||
lo=0/seed `0x78f74a68`), maple should write a **nonzero per-install seed and a
|
||||
nonzero, stable per-track lo**:
|
||||
- `opaque_seed`: use a real per-install seed. If maple has no install of its
|
||||
own, `0x21b4a7ab` (this RE machine's install) is a known-real value;
|
||||
keep it constant across all tracks in one export. Do NOT use `0x78f74a68`
|
||||
(that was maple's own placeholder, not a rekordbox value).
|
||||
- `opaque_track_id_lo`: a nonzero 28-bit value, stable per track. Since the
|
||||
real value is an opaque DB id maple can't recover, a deterministic stand-in
|
||||
(e.g. a 28-bit hash of a stable per-track key, or the track's own pdb id
|
||||
offset into the ContentID band) is fine — the goal is "looks like a real
|
||||
nonzero ContentID", not byte-parity.
|
||||
- Either way these 8 bytes will NOT match a specific reference export
|
||||
byte-for-byte (the real ids live in that install's encrypted DB) — record the
|
||||
divergence in re/KNOWN_DIVERGENCE.md.
|
||||
|
||||
**Confidence:** high (5-export empirical proof: 1046/1046 real tracks nonzero;
|
||||
exhaustive negative hash search incl. audio bytes; binary const-absence). The
|
||||
earlier "write 0 / 0x78f74a68" advice was based on a maple-output file and is
|
||||
retracted.
|
||||
|
||||
**Round-3 (task #10): can pyrekordbox read the real ids out of master.db?**
|
||||
User insight: pyrekordbox holds the SQLCipher key, so the DBs are not truly
|
||||
unreadable. Validated — partly true, but blocked on the data we have:
|
||||
- pyrekordbox 0.4.4 + `sqlcipher3-wheels` installs and wires up correctly. Its
|
||||
master.db key is the single **global** rekordbox-6 key
|
||||
`402fd482c38817c35ffa8ffb8c7d93143b749e7d315df7a81732a1ff43608497`
|
||||
(`deobfuscate(BLOB)`, must start with `402fd`) — same for every install, NOT
|
||||
per-install. So a VALID master.db *would* open and expose `DjmdContent`.
|
||||
- BUT the captured `maple-py/captures/p17328_master.db` is **corrupt/truncated**:
|
||||
1,024,000 bytes but only 4.4% non-zero, and the first 32 KB (the mandatory
|
||||
SQLCipher salt + page 1) is **all zeros**. No key can decrypt a DB with a
|
||||
zeroed salt. `p17328_master.backup.db` is 0 bytes. The Frida NtWriteFile
|
||||
capture only caught scattered later pages, never the header. → unusable.
|
||||
- The genuine `exportLibrary.db` shipped next to export.pdb (valid 16-byte salt
|
||||
`ed0d3a3c…`) does NOT open with the global master.db key (different key
|
||||
scheme) and pyrekordbox has **no exportLibrary.db support**. → can't read it
|
||||
here.
|
||||
- Strong inference (untested for lack of a readable DB): the per-track
|
||||
`opaque_track_id_lo` is almost certainly `DjmdContent.ID` (the 9-digit
|
||||
rekordbox Content ID; our lo range 0x0003edeb..0x0ffe7f4e fits). If maple can
|
||||
open the user's LIVE `%APPDATA%/Pioneer/rekordbox6/master.db` with pyrekordbox
|
||||
(global key), `db.get_content()` would give `(ID, …)` per track, and the blob
|
||||
would become EXACTLY reproducible: `opaque_track_id_lo = DjmdContent.ID & 0xFFFFFFFF`.
|
||||
- The per-install `opaque_seed` (0x21b4a7ab here) is NOT a DjmdContent column;
|
||||
it is one value for the whole export and most likely derived from a
|
||||
master.db-level per-library marker (a USB/library UUID or an `agentRegistry`/
|
||||
`local_usn`-style value). Identifying it needs a readable master.db to inspect
|
||||
candidate columns; unconfirmed.
|
||||
|
||||
**pyrekordbox write / export capability (asked in #10):**
|
||||
- It CAN write the desktop master.db: `add_content`/`add_album`/`add_artist`/
|
||||
`create_playlist`/`commit`, plus `set_local_usn`, `update_content_path`. But
|
||||
`add_content` lets rekordbox ASSIGN the Content ID; there is no API to force a
|
||||
specific ID. So it is a source of real ids, not a way to dictate them.
|
||||
- It does NOT drive a USB export — only `get_device` (read). The actual
|
||||
`export.pdb`/`exportLibrary.db` writing is rekordbox.exe's DeviceSQL engine
|
||||
(the thing maple reimplements). pyrekordbox manipulates only the desktop
|
||||
library DB, not the USB export. So pyrekordbox cannot replace maple's writer;
|
||||
its only use to maple is as a READ source of the real `DjmdContent.ID` (and
|
||||
potentially the seed) from a valid master.db.
|
||||
|
||||
**Net for #10:** the approach is sound (pyrekordbox + global key reads a real
|
||||
master.db → Content IDs), so `opaque_track_id_lo` is likely reproducible *if
|
||||
maple reads the live master.db*. It is NOT reproducible from the corrupt
|
||||
captured DB we have, and the seed source is still unidentified. pyrekordbox
|
||||
cannot produce the USB pdb itself. Recommended next step for maple/RE: point
|
||||
pyrekordbox at the user's LIVE rekordbox6 master.db
|
||||
(`%APPDATA%/Pioneer/rekordbox6/master.db`), confirm
|
||||
`DjmdContent.ID == opaque_track_id_lo` for the AceMo tracks, then hunt the seed
|
||||
in that DB. Until then, the nonzero stand-in (above) stays.
|
||||
|
||||
---
|
||||
|
||||
## 3. File-header `sequence` (and per-page `sequence`)
|
||||
|
||||
**Field:** `RekordboxPdb.sequence` (u4 @0x14 of the file header) and
|
||||
`Page.sequence` (u4 in each page header).
|
||||
|
||||
**Rule / meaning (confirmed from schema docs + the 5 genuine exports above):**
|
||||
- `sequence` is a monotonically increasing **edit counter** for the database.
|
||||
The file-header value is the **next** page-sequence number; each page records
|
||||
the counter value as of its last edit (so `header.sequence >=
|
||||
max(page.sequence)`). Observed header values in genuine rekordbox exports:
|
||||
35, 56, 353, 546, 4211 — scales with edit history, not track count. (No real
|
||||
export with sequence==1 is available; `data/usb`'s seq==1 is maple's own
|
||||
output, not a rekordbox sample — see §2 correction.)
|
||||
- The firmware/players (and crate-digger) use page `sequence` only as
|
||||
informational ordering metadata; **they do not validate it** for playback.
|
||||
No special value is required beyond keeping `header.sequence` greater than any
|
||||
page's `sequence`.
|
||||
- For maple it is enough to (a) pick a header `sequence` larger than every page
|
||||
`sequence` it writes, and (b) give each page a `sequence` <= header. A simple
|
||||
scheme (all pages = N, header = N+1) is accepted. This is what task #3
|
||||
(incrementing sequence) implements.
|
||||
|
||||
**Confidence:** high for semantics and "not validated by players"; the exact
|
||||
increment rekordbox uses per edit is not needed for a valid export.
|
||||
|
||||
---
|
||||
|
||||
## 4. File-header offset 0x10 (u4, schema `_unnamed4`) — HISTORY-DEPENDENT; not byte-reproducible
|
||||
|
||||
> CORRECTION (2026-06-09): a prior draft cited a "usb fresh, sequence==1,
|
||||
> _unnamed4==0" row as proof that 0 is the fresh value. That file is maple's own
|
||||
> output, not a rekordbox export, so it is removed below. The remaining proof
|
||||
> (same-library divergence) does not depend on it.
|
||||
|
||||
**Field:** the unnamed u4 between `next_unused_page` (0x0c) and `sequence`
|
||||
(0x14). maple writes 0; the AceMo ref has 4.
|
||||
|
||||
**Finding (5 genuine rekordbox exports):** an **edit/allocator accumulator**,
|
||||
not a structural value derivable from the tables. It does NOT equal track count,
|
||||
page count, or `next_unused_page - npages`:
|
||||
|
||||
| export | sequence | _unnamed4@0x10 | next_unused | npages |
|
||||
|---------------|---------:|---------------:|------------:|-------:|
|
||||
| acemo_fresh | 35 | 5 | 52 | 49 |
|
||||
| lib_multi | 56 | 4 | 57 | 52 |
|
||||
| acemo | 353 | 4 | 53 | 49 |
|
||||
| MAPLE | 546 | 5 | — | — |
|
||||
| data/PIONEER | 4211 | 1 | 174 | 174 |
|
||||
|
||||
**Decisive (same-library proof):** acemo and acemo_fresh are the SAME library
|
||||
(identical track set + identical per-track opaque ids) yet have `_unnamed4` =
|
||||
4 vs 5. A structural field would be identical; differing proves it is edit
|
||||
state. (We have NO genuine rekordbox export with `sequence==1`, so the exact
|
||||
never-edited value is unconfirmed — but it is clearly not derivable from the
|
||||
file's structure.)
|
||||
|
||||
**Rule for maple:** there is no reproducible rule. Write **0** (maple already
|
||||
does) as a **reasonable minimal default — NOT a confirmed match** (every genuine
|
||||
export we have is nonzero here, and we have no never-edited rekordbox export to
|
||||
check what fresh rekordbox writes). Record the divergence in
|
||||
re/KNOWN_DIVERGENCE.md. It cannot be matched to the AceMo ref's `4` without that
|
||||
DB's edit history.
|
||||
|
||||
**Confidence:** high that it is edit-history/allocator state and not
|
||||
structurally reproducible; the specific fresh-write value is unconfirmed (no
|
||||
seq==1 rekordbox sample available).
|
||||
|
||||
---
|
||||
|
||||
## 5. Page-header offset 0x26 (u2, "unknown7"/strange-page) — HISTORY-DEPENDENT; maple's 0 is a reasonable default
|
||||
|
||||
> CORRECTION (2026-06-09): do NOT cite `data/usb` (sequence==1) here — it is
|
||||
> maple's OWN output, so "maple's 0 matches usb" would be circular. The
|
||||
> observations below are from the five GENUINE rekordbox exports only.
|
||||
|
||||
**Field:** the second of the two trailing u2s in the page header (after
|
||||
`transaction_row_index` @0x22 and the "1004 for strange blocks" u2 @0x24).
|
||||
@flesniak: "always 0 except 1 for history pages, num entries for strange
|
||||
pages?". maple writes 0; the AceMo ref has 0/1/2 on various tables' first
|
||||
("strange") pages.
|
||||
|
||||
**Finding (5 genuine rekordbox exports — all incrementally edited):**
|
||||
- On DATA pages it is **always 0** (all five exports). This is a solid
|
||||
observation about genuine rekordbox output.
|
||||
- It is nonzero only on a table's first/"strange" page (flags 0x64), and the
|
||||
value does NOT correlate with that table's data-page count or row count
|
||||
(acemo tracks: 2 data pages → 2; data/PIONEER tracks: 105 data pages → 23).
|
||||
It is transaction/edit-tracking state.
|
||||
- **Decisive (same-library proof):** acemo and acemo_fresh hold the identical
|
||||
tracks table, yet its strange-page unknown7 = 2 (acemo) vs 1 (acemo_fresh).
|
||||
Same data → different value ⇒ not structural, it is edit state. (lib_multi:
|
||||
tracks strange unk7 = 2 with 3 data pages / 10 rows — again no count relation.)
|
||||
|
||||
**Rule for maple:** write **0** on every page (maple already does). The data
|
||||
pages being 0 across all five genuine exports makes 0 a well-supported value
|
||||
*for data pages*. For strange pages the real value is per-export edit state
|
||||
(0/1/2…) that cannot be reproduced — maple's 0 there is a **reasonable minimal
|
||||
default, NOT a confirmed match** (we have no never-edited rekordbox export to
|
||||
check what fresh rekordbox stamps on a strange page). Record the strange-page
|
||||
divergence in re/KNOWN_DIVERGENCE.md.
|
||||
|
||||
**Confidence:** high that it is history-dependent and that data-page 0 matches
|
||||
real rekordbox; the fresh-write strange-page value is unconfirmed (no seq==1
|
||||
rekordbox sample).
|
||||
|
||||
---
|
||||
|
||||
## 6. Row-group trailing u2 (`transaction_row_flags`) — write `present_flags` (NOT 16); residual mismatch is edit history
|
||||
|
||||
> CORRECTION (2026-06-09): an earlier draft of this section claimed "a fresh
|
||||
> export writes 16 everywhere, so maple's 16 is correct." That was based on
|
||||
> `data/usb/.../export.pdb`, which on re-check is **maple's OWN output**, not a
|
||||
> rekordbox export (opaque seed `0x78f74a68` + all-zero per-track lo +
|
||||
> `sequence`==low). Reading it back was circular. The analysis below uses ONLY
|
||||
> the five genuine rekordbox exports (provenance verified by nonzero per-track
|
||||
> opaque ids + per-install seed): acemo, acemo_fresh, lib_multi, MAPLE,
|
||||
> data/PIONEER.
|
||||
|
||||
**Field:** the last u2 of each 36-byte row-group block in the page row index
|
||||
(crate-digger `transaction_row_flags`, at `base`; in maple's `pager.py` it is
|
||||
`pack("<H", 16) # unknown (rekordcrate default)`).
|
||||
|
||||
**Finding (5 genuine rekordbox exports):** this u2 is the
|
||||
**"rows touched by the last transaction" bitmask** (same bit layout as
|
||||
`row_present_flags`), and is therefore **edit-history state, not reproducible**:
|
||||
- **No genuine rekordbox export writes 16 as the field's value** on settled
|
||||
pages (count of groups == 16 across the five exports: 0, 0, 0, 1, 8 — and
|
||||
those few are coincidental transaction bitmasks, not a default). So maple's
|
||||
constant **16 does NOT match rekordbox** — it is a rekordcrate invention.
|
||||
- The dominant value in heavily-edited tables is **0x0** (last transaction
|
||||
touched no rows on that page), e.g. data/PIONEER 170/294 groups = 0x0.
|
||||
- Three DIFFERENT patterns coexist in ONE file, proving it is not structural:
|
||||
(a) some pages `trailing == present_flags` (columns/unk17/unk18); (b) artwork
|
||||
pages consistently use `last group = 1<<(rows_in_last_group-1)`, earlier groups
|
||||
= 0 (data/PIONEER p28 g6 10rows→0x200, p147 g2 13rows→0x1000; acemo_fresh
|
||||
artwork 7rows→0x40); (c) churned tracks pages use an arbitrary single bit
|
||||
(0x1/0x2/0x4/0x8/0x10/0x40…) = whatever row that page's last edit touched.
|
||||
A candidate clean rule `[last grp=1<<(rows-1), others=0]` matches only
|
||||
**181/257** fully-present pages in data/PIONEER — confirming NO single formula.
|
||||
- **Same-library proof:** acemo and acemo_fresh hold identical data, yet the
|
||||
tracks page-2 group differs (acemo present=0x5e trailing=0x20; acemo_fresh
|
||||
present=0xbb trailing=0x4) — same rows, different value ⇒ not structural.
|
||||
The previously-flagged artwork `0x40` is likewise transaction state
|
||||
(acemo_fresh artwork=0x40 vs lib_multi artwork p49=0x4), not an open
|
||||
reproducible byte.
|
||||
|
||||
**Rule for maple (DECISION): write `present_flags`, NOT the constant 16.**
|
||||
maple's old constant 16 (rekordcrate's placeholder) matches **0/14** real groups;
|
||||
`trailing = row_present_flags` matches **12/14** clean groups in the real
|
||||
`data/acemo_fresh` export (8 rows→0xff, 16→0xffff, 11→0x7ff, 6→0x3f, 1→0x1). The
|
||||
2 misses (page2→0x4, page28→0x40, page40→0xc0) are the last-transaction bitmask
|
||||
on pages that were edited after creation — i.e. residual edit history, not a
|
||||
different rule. So for a from-scratch single-transaction write the correct value
|
||||
IS `present_flags` (all rows touched in the one creating transaction). maple now
|
||||
ships `present_flags` (pager.py) — this is the best achievable; reverting to 16
|
||||
would regress 12 groups. Remaining mismatch vs an EDITED reference (the per-page
|
||||
last-tx bitmask) is unavoidable edit history → KNOWN_DIVERGENCE.
|
||||
|
||||
> NOTE: the earlier "keep 16 / fresh usb writes 0x10 everywhere" framing was
|
||||
> WRONG — it read `data/usb`, which is MAPLE'S OWN OUTPUT (an old build that
|
||||
> emitted the rekordcrate-default 16), not a real export. `data/usb` must NOT be
|
||||
> used as a 'fresh rekordbox' baseline for ANY field (same contamination that
|
||||
> produced the wrong opaque-blob default in §2). The fresh-write value is
|
||||
> established from the REAL `data/acemo_fresh` as `present_flags`.
|
||||
|
||||
**Confidence:** high that it is history-dependent and that real rekordbox never
|
||||
uses 16; the correct fresh-write value is an open question (no seq==1 sample).
|
||||
|
||||
---
|
||||
|
||||
## Note on the three "reproducible header fields" (task #9)
|
||||
|
||||
After multi-export analysis across the FIVE genuine rekordbox exports (acemo,
|
||||
acemo_fresh, lib_multi, MAPLE, data/PIONEER — provenance verified by nonzero
|
||||
per-track opaque ids + per-install seed), all three fields are
|
||||
**edit-history / transaction state, NOT structurally reproducible**:
|
||||
- §4 file-header 0x10: allocator/edit accumulator. → write 0, DIVERGENCE.
|
||||
- §5 page 0x26 unknown7: 0 on all data pages (exact); per-table 0/1/2 on strange
|
||||
pages is edit state. → write 0, strange-page DIVERGENCE.
|
||||
- §6 row-group trailing u2: "rows touched by last transaction" bitmask. Fresh
|
||||
single-transaction value = `present_flags` (12/14 in real data/acemo_fresh;
|
||||
constant 16 = 0/14). → maple writes `present_flags`; residual mismatch vs an
|
||||
EDITED ref is the per-page last-tx bitmask = edit history, DIVERGENCE.
|
||||
|
||||
The decisive evidence is the **same-library pair** (acemo vs acemo_fresh:
|
||||
identical track data, yet header 0x10 = 4 vs 5, tracks strange unk7 = 2 vs 1,
|
||||
tracks-page trailing differs) — a structural field would be identical.
|
||||
|
||||
maple's values: §4 header0x10 = 0 and §5 unknown7 = 0 are reasonable minimal
|
||||
defaults (confirmed = real exports on data pages; fresh strange-page value
|
||||
unconfirmed). §6 trailing u2 = `present_flags` is the established fresh-write
|
||||
value (NOT the old constant 16). No further writer change needed; remaining
|
||||
mismatches vs edited references are unavoidable edit history.
|
||||
|
||||
IMPORTANT corrections to the earlier draft: (1) `data/usb/.../export.pdb` is
|
||||
maple's OWN output (an old build that emitted the rekordcrate-default 16), not a
|
||||
rekordbox export — so any "fresh usb" claim was circular; `data/usb` is NOT an
|
||||
authority for ANY field (same contamination that produced the wrong opaque-blob
|
||||
default in §2). (2) We have NO genuine rekordbox export with sequence==1, so the
|
||||
never-edited header0x10 / strange-page-unknown7 values are unconfirmed. (3) The
|
||||
row-group trailing u2 fresh value is `present_flags`, established from the REAL
|
||||
data/acemo_fresh (12/14); the rekordcrate "Default=16" is a placeholder that
|
||||
matches 0/14 real groups.
|
||||
|
||||
All three go to re/KNOWN_DIVERGENCE.md, not the writer.
|
||||
|
||||
---
|
||||
|
||||
## 7. Artwork + USBANLZ path schemes (task #11)
|
||||
|
||||
Decoded from genuine `data/MAPLE` (176 tracks/176 artwork) + acemo. The guiding
|
||||
principle (per the task): for **player acceptance only PATH SELF-CONSISTENCY
|
||||
matters** — every path stored in the pdb must resolve to a real on-disk file.
|
||||
Matching rekordbox's exact id-derivation is NOT required; maple's own consistent
|
||||
scheme is acceptable.
|
||||
|
||||
### 7a. Artwork — SOLVED, exactly reproducible (maple already matches)
|
||||
- Path: `/PIONEER/Artwork/{folder:05d}/a{artwork_id}.jpg` (80×80) plus a
|
||||
`a{artwork_id}_m.jpg` (240×240) variant in the same folder (rekordbox writes
|
||||
both; verified on disk in data/MAPLE).
|
||||
- **Folder bucket rule: `folder = artwork_id // 20 + 1`** (20 ids per bucket;
|
||||
bucket 00001 = ids 1–19 because ids are 1-based, 00002 = 20–39, 00003 = 40–59,
|
||||
…). Verified **176/176** in data/MAPLE.
|
||||
- maple's `artwork._bucket()` is literally `f"{artwork_id // 20 + 1:05d}"` and
|
||||
`extract_artwork` writes both jpg variants at that path → **byte-identical to
|
||||
rekordbox AND self-consistent**. No change needed.
|
||||
|
||||
### 7b. USBANLZ — opaque id; maple's own scheme is self-consistent (DIVERGENCE from rekordbox)
|
||||
- Path: `/PIONEER/USBANLZ/P{bucket}/{8-hex}/ANLZ0000.DAT` (+`.EXT`).
|
||||
- rekordbox's `{8-hex}` is a per-track **opaque analysis id** (range seen
|
||||
0x117..0x309fd over 176 tracks; NOT == track id, NOT == the row-0x14
|
||||
`opaque_track_id_lo`, NOT a crc of path/title — tested) and the `P{bucket}`
|
||||
(0–127, 94 distinct over 176) is a spreading bucket that is **NOT a derivable
|
||||
function** of the 8-hex or any track field (tested: `&0x7f`, `>>k`, `%127/128`,
|
||||
low-byte — all fail). Like the row-0x14 blob, these come from rekordbox's
|
||||
internal DB and are **not reproducible** without it.
|
||||
- maple (cli.py `build()`): generates its OWN deterministic scheme —
|
||||
`bucket = P{(crc32(file_path) >> 20) & 0xFFF:03X}`,
|
||||
`analyze_id = {crc32(file_path):08X}`, then `write_anlz()` writes the .DAT at
|
||||
exactly that path. This is **self-consistent by construction** (the path it
|
||||
records is the path it writes) and is the correct approach per the
|
||||
self-consistency principle. It does NOT match rekordbox's id/bucket (expected
|
||||
DIVERGENCE).
|
||||
- Minor cosmetic note (not a correctness issue): maple's bucket is 3 hex
|
||||
digits `P000..PFFF` whereas rekordbox uses `P00..P7F` (0–127). Harmless;
|
||||
optionally narrow to `& 0x7F` (`P{..:02X}`) to look more rekordbox-like.
|
||||
|
||||
### 7c. Self-consistency verification
|
||||
- **Genuine rekordbox (data/MAPLE): 176/176 ANLZ paths and 176/176 artwork
|
||||
paths resolve to on-disk files.** (rekordbox is fully self-consistent — sanity
|
||||
check that confirms the parser and the principle.)
|
||||
- **maple production `build()` (cli.py): self-consistent BY CONSTRUCTION** —
|
||||
`extract_artwork` (artwork.py:62-65) writes `a{id}.jpg`/`a{id}_m.jpg` at the
|
||||
recorded path; `write_anlz` (cli.py:93) writes `ANLZ0000.DAT` at the recorded
|
||||
`analyze_path`. Each pdb path is computed and the file is written there in the
|
||||
same step, so every path resolves.
|
||||
- CAVEAT — the byte-diff harness `tools/build_acemo.py` is NOT the production
|
||||
path: it MIRRORS a reference rekordbox export (copies its `analyze_path`/
|
||||
artwork `path` strings verbatim, writes no ANLZ/artwork files), so the
|
||||
`data/usb` export it produces references the reference's USBANLZ files which
|
||||
are not co-located → those paths don't resolve. This is expected for a
|
||||
structural-diff fixture and says nothing about production self-consistency.
|
||||
|
||||
**Confidence:** high. Artwork rule is exact (176/176). USBANLZ rekordbox ids are
|
||||
opaque/non-reproducible (consistent with the row-0x14 finding); maple's own
|
||||
USBANLZ scheme is self-consistent, which is all the player requires.
|
||||
|
||||
---
|
||||
|
||||
## 8. Page-manager free-space: `empty_candidate` + free pool (task #13 — THE FREEZE)
|
||||
|
||||
DECK TEST: maple's pdb FROZE the player (parser hang, not reject). Decoded the
|
||||
page-manager/free-space divergence against genuine rekordbox exports
|
||||
(`data/working` 9trk seq44, `data/MAPLE` 176trk, `data/PIONEER` 847trk — all
|
||||
load) vs maple output (`captures/maple_acemo.pdb`).
|
||||
|
||||
### Data-page `next_page` SELF-LOOP — ROOT CAUSE (confirmed divergence; high confidence)
|
||||
- The player reads each table by following `first_page → next_page → …`. maple's
|
||||
**last data page of each multi-page chain set `next_page == its own index`**
|
||||
(page 2→2, 6→6, 8→8, 12, 14, 28, 34, 36, 38, 40 — 10 self-loops in the 9-track
|
||||
build). A page whose `next_page` is itself is an **infinite loop → FREEZE**.
|
||||
- **Genuine rekordbox has ZERO self-loops.** A terminal data page uses
|
||||
`next_page = 0` (the "no next" sentinel; seen on pages 10/16/18/20/…/41-47 in
|
||||
`data/working`) OR points FORWARD to a distinct free page (never to itself).
|
||||
- FIX (maple, done in writer.py): point every chain's last page at a single
|
||||
dedicated zero-filled free-pool page (`free_sentinel`, just past the overflow
|
||||
region) instead of itself; materialise that page and set `next_unused` beyond
|
||||
it. Verified on the rebuilt pdb: 10 self-loops → 0; all 20 chains terminate at
|
||||
the blank sentinel; player tolerates last-page→free-page (genuine exports do
|
||||
the same). This is the primary freeze fix.
|
||||
|
||||
### Table-dir `empty_candidate` (u4 after page_type) — SECONDARY divergence (also wrong; lower-confidence freeze contributor)
|
||||
- Layout: each 16-byte table-dir entry is `type:u4, empty_candidate:u4,
|
||||
first_page:u4, last_page:u4`.
|
||||
- `empty_candidate` is the head of that table's **empty/free-page list** (the
|
||||
page the Page Manager grabs when the table needs to grow).
|
||||
- **In EVERY genuine rekordbox export all 20 `empty_candidate`s are NONZERO and
|
||||
distinct**, and each points to an **all-zero "free" page** — either an
|
||||
in-file allocated-but-empty page (e.g. `last_page+1`, or a pooled page like
|
||||
41–47/49) or a **beyond-EOF sentinel** (index ≥ npages, e.g. 50/51/52 when
|
||||
npages=49 — the firmware tolerates beyond-EOF here). It NEVER points to a
|
||||
used data page and NEVER to 0.
|
||||
- **maple writes `empty_candidate = 0` for all 20 tables.** Page 0 is the FILE
|
||||
HEADER. The firmware Page Manager, walking a table's empty-page list, follows
|
||||
`0` → reads the 4 KB file header AS a page → its "next_page" field (header
|
||||
offset 0x0c = `next_unused_page`) and bogus type/flags send the walk into a
|
||||
loop or out-of-bounds read → **hang/FREEZE**. This is the highest-confidence
|
||||
cause: it is the only systematic, data-independent divergence, and `0` is
|
||||
never valid (collides with the header).
|
||||
|
||||
### Free pool / page layout
|
||||
- rekordbox keeps the data pages in chains `1..40,48,…` and pre-allocates a
|
||||
pool of all-zero "free" pages past them; `next_unused_page` = one past the
|
||||
highest allocated page (so `next_unused > npages` is normal, e.g. 53 > 49).
|
||||
- maple **compacts**: 41 pages, every page used, `next_unused == npages == 41`,
|
||||
no free pool. The compaction itself is structurally legal (chains are clean),
|
||||
but it left `empty_candidate = 0` with no free page to point at.
|
||||
- Beyond-EOF links (a page's `next_page` ≥ npages, or `empty_candidate` ≥
|
||||
npages) are TOLERATED — the genuine `working` file has many (page 4→53,
|
||||
46→52, empty_candidate 51/52/50/49 all ≥ 49). So beyond-EOF is a valid
|
||||
"no-more-pages / no-free-page" sentinel; it is NOT the freeze.
|
||||
|
||||
### Strange-page free-map — NOT the cause (decoded, maple matches)
|
||||
- Each table's first/"strange" page (flags 0x64) carries, after a small prefix
|
||||
`[this_page, next_or_0x03ffffff, 0x03ffffff, 0, 0x1fff0000]`, a heap filled
|
||||
with **1004 × `0x1ffffff8`** (`f8 ff ff 1f` LE) — the empty-heap free-map
|
||||
(count == the `m1004`=1004 marker at header 0x24). `0x03ffffff` in the 2nd u4
|
||||
is the legitimate **"this table has no data page"** sentinel (genuine
|
||||
rekordbox writes it for every empty table; e.g. labels/pltree/plentries).
|
||||
- maple reproduces this free-map correctly; its strange-page prefixes match
|
||||
rekordbox's pattern byte-for-byte (data table → u1=first_data_page; empty
|
||||
table → u1=0x03ffffff). **So the free-map is NOT the freeze.**
|
||||
|
||||
### FIX (for maple → deck test)
|
||||
PRIMARY (high confidence, DONE): **eliminate the data-page `next_page` self-loop**
|
||||
— last page of each chain must point to a blank free page (or use `next_page=0`),
|
||||
never to itself. maple now points all last pages at one zero-filled `free_sentinel`
|
||||
page. Verified: 0 self-loops in the rebuilt pdb.
|
||||
|
||||
SECONDARY (cheap, also a real divergence — fold into the same build): **stop
|
||||
writing `empty_candidate = 0`.** Point it at the same `free_sentinel`
|
||||
(`pack("<I", free_sentinel)` instead of 0). Genuine rekordbox NEVER writes 0
|
||||
(0 = the header page); it always points each table's empty_candidate at a blank
|
||||
free page. Lower-confidence as a freeze cause for a read-only player than the
|
||||
self-loop, but wrong and trivially fixable with the page maple already
|
||||
materialised.
|
||||
|
||||
If the deck STILL hangs after both, next suspects (lower confidence):
|
||||
`next_unused_page == npages` (give a little slack), and whether the single shared
|
||||
sentinel needs to be per-table-distinct (rekordbox uses distinct free pages).
|
||||
|
||||
**Confidence:** HIGH that the self-loop was the freeze (self-referential
|
||||
next_page is an unconditional infinite loop; 0/all genuine exports self-loop).
|
||||
empty_candidate=0 is independently wrong (0/all genuine exports use it; collides
|
||||
with header). Exact firmware behaviour is inferred (no firmware to trace); the
|
||||
deck test confirms.
|
||||
is the confirmation.
|
||||
@@ -0,0 +1,167 @@
|
||||
# KNOWN_DIVERGENCE.md — fields maple's writer intentionally does NOT match
|
||||
|
||||
This is the canonical list of bytes that differ between maple's
|
||||
`captures/maple_acemo.pdb` and the AceMo reference
|
||||
`data/acemo/PIONEER/rekordbox/export.pdb` **on purpose**, because they are
|
||||
*history-dependent* artifacts of the particular reference file rather than rules
|
||||
of the `.pdb` format.
|
||||
|
||||
## TL;DR
|
||||
|
||||
The AceMo reference `export.pdb` is **not a fresh from-scratch export — it is an
|
||||
incrementally-edited rekordbox database.** rekordbox's DeviceSQL page manager
|
||||
edits rows *in place*: when a row changes it appends a new copy, clears the old
|
||||
slot's presence bit, and advances `used_size` past the dead bytes **without
|
||||
compacting**. The reference therefore carries the residue of its entire edit
|
||||
history.
|
||||
|
||||
maple writes a **fresh, fully-compacted** database (tightly 4-byte-aligned rows,
|
||||
no dead rows, no leftover slots). That output is byte-correct for a *from-scratch*
|
||||
rekordbox export, but it cannot — and should not — reproduce another file's edit
|
||||
history. The fields below are the consequences of that, and the team has decided
|
||||
**not to chase them**. They only become matchable against a fresh from-scratch
|
||||
reference export (export to an empty USB, never edited afterward).
|
||||
|
||||
Everything NOT listed here is fair game and must match. The task-#7 diff should
|
||||
filter out (and reference) this list, then drive the remaining bytes to zero.
|
||||
|
||||
## Evidence (tracks table, page 2 of the reference)
|
||||
|
||||
5 present rows, each with ~377–417 bytes of real content, yet:
|
||||
|
||||
| row id | ofs | real content len | next ofs | gap | gap bytes are… |
|
||||
|--------|-------|------------------|----------|-------|---------------------------|
|
||||
| 1 | 428 | 377 | 856 | 51 | zero |
|
||||
| 5 | 856 | 377 | 1324 | 91 | partial stale string |
|
||||
| 2 | 1324 | 417 | 1792 | 51 | zero |
|
||||
| 3 | 1792 | 386 | 2656 | 478 | a whole dead track row |
|
||||
| 4 | 2656 | 404 | 3996* | 936 | two whole dead track rows |
|
||||
|
||||
`used_size=3996`, `free_size=38`. The gap after id=3 holds a dead copy of a row
|
||||
ending `"...Jelly Fish Jam.mp3"`; the gap after id=4 holds dead copies of
|
||||
`"Myrtle Ave Party Track"` and `"Smooth Move"` rows. These are previous versions
|
||||
of edited rows, left uncompacted. (Verified by ASCII-dumping the gap bytes.)
|
||||
|
||||
## The fields we do NOT match (and why)
|
||||
|
||||
1. **File / page count and `next_unused_page`.**
|
||||
Reference: 49 pages, `next_unused=53`. maple: 41 pages, `next_unused=41`.
|
||||
The reference is larger purely because dead-row bloat pushed live rows onto
|
||||
extra pages. A compacted export needs fewer pages.
|
||||
|
||||
2. **Overflow / data-page placement (tracks `last_page`).**
|
||||
Reference tracks table spans pages **1 → 48** (live track id=7 spilled onto
|
||||
overflow page 48 because dead rows "filled" page 2). maple spans **1 → 2**
|
||||
(all 6 tracks fit one compacted page). This is the *only* table-directory
|
||||
entry that differs — the other 19 tables match first/last page exactly. The
|
||||
divergence is entirely a function of the dead-row bloat above.
|
||||
|
||||
3. **File-header `sequence` value.**
|
||||
Reference: `353`. maple: an incrementing counter (reads the destination
|
||||
export's sequence + 1, default 1; see `writer._next_sequence`). rekordbox's
|
||||
value comes from its internal `master.db` transaction counter and is not
|
||||
reproducible from content. We match the *semantics* (monotonic increment),
|
||||
not the absolute value.
|
||||
|
||||
4. **Per-page `sequence` (page-header offset 16).**
|
||||
Reference pages carry assorted values (e.g. 26, 45, 320, 342, 347) — each a
|
||||
snapshot of the global counter at the moment that page was last rewritten.
|
||||
maple writes a clean per-build value. Same root cause as (3); not reproducible.
|
||||
|
||||
5. **`transaction_row_count` / `transaction_row_index` (page-header u2 pair).**
|
||||
Reference: nonzero per edited page (e.g. `1/5`, `1/8`, `2/0`). maple: `0/0`.
|
||||
These track the last in-place transaction on the page; meaningless for a
|
||||
freshly written, never-edited page.
|
||||
|
||||
6. **Dead-row offset arrays + `num_row_offsets` > `num_rows`.**
|
||||
Reference row groups list offsets for dead slots too: e.g. tracks page 2 has
|
||||
`nro=9, nrows=5` with offsets `[0,428,856,1324,1792,2224,2656,3112,3568]`
|
||||
(slots 0,5,7,8 dead); artists page 6 has `nro=9, nrows=1`. maple emits exactly
|
||||
`nro == nrows` with no dead slots. Reproducing dead offsets would mean
|
||||
reproducing the edit history.
|
||||
|
||||
7. **Live row in a non-zero slot.**
|
||||
Reference artists page 6 keeps its single live row in **slot 8** (slots 0–7
|
||||
are dead). maple places the single live row in **slot 0**. A from-scratch
|
||||
export also writes slot 0.
|
||||
|
||||
8. **`free_size` / `used_size` totals on data pages.**
|
||||
Reference `used+free` totals vary (4034, 4050, …) because `used_size` covers
|
||||
the dead-row region and the strange-block markers differ. maple's pages total
|
||||
a clean `4020` (= heap capacity for a single 36-byte row group). Direct
|
||||
consequence of (6) — dead bytes are counted as "used".
|
||||
|
||||
9. **Track row opaque id blob — `0x14`/`0x18`/`0x1a` (8 bytes/track).**
|
||||
One 8-byte LE value `opaque_track_id_lo | (opaque_seed << 32)`, both halves
|
||||
stored in rekordbox's internal master.db (a per-install `opaque_seed` + a
|
||||
per-track `opaque_track_id_lo`). Proven non-derivable from track metadata
|
||||
across every real export + an exhaustive negative hash search
|
||||
(FORMAT_SPEC.md s2). **Every real rekordbox export has BOTH halves non-zero**
|
||||
(this install's seed = `0x21b4a7ab`; `data/MAPLE` = `0xd6d3a567`); the
|
||||
"unanalyzed default" lo=0/seed=0x78f74a68 was a false lead — it came from
|
||||
maple's own earlier output (`data/usb`), not a real export. maple emits a
|
||||
plausible non-zero stand-in of the right shape: `opaque_seed = 0x21b4a7ab`
|
||||
(module const `TRACK_OPAQUE_SEED`, overridable per install) and a stable
|
||||
per-track `opaque_track_id_lo` mapped into the real Content-ID band
|
||||
(`OPAQUE_LO_MIN..MAX` = 0x08000000..0x0F900000, where every observed AceMo lo
|
||||
lands) via `crc32(id|file_path)`. This will NOT match rekordbox's master.db
|
||||
id byte-for-byte — unreproducible without its DB — but it is non-zero,
|
||||
in-range and deterministic, the safest shape for deck acceptance. Players
|
||||
treat the blob as opaque.
|
||||
|
||||
10. **Strange-page free-map extension + per-page seq on tracks/history.**
|
||||
On the tracks (type 0) and history (type 19) strange pages only, the
|
||||
free-space map carries an extra accounting word (body offset 16: `01 00`,
|
||||
offset 20: a u4 = the data page's used_size for history=320; tracks differs
|
||||
because of its dead-row/overflow state). maple writes the plain free-map.
|
||||
Tied to the same content/overflow state as items (1)/(2); not reproducible
|
||||
for a compacted writer. (The rest of every strange page — header, free-map
|
||||
head, 1004× `f8 ff ff 1f`, the data-page pointer word, and unknown7 — now
|
||||
matches the fresh reference byte-for-byte.)
|
||||
|
||||
11. **Table-directory `empty_candidate` (u32, table_dir entry +4) + header
|
||||
`next_unused_page`.** Reference: a nonzero per-table `empty_candidate`
|
||||
(e.g. 0x33, 0x04, 0x2e…); maple: 0. rekordcrate documents this field as
|
||||
"maybe links to a chain of empty pages if the database is ever garbage
|
||||
collected", and its fixtures show non-sequential free-pool page indices
|
||||
(47, 4, 49, 8, 50, 46, 42, 16…). It is free-list/GC allocator state, not
|
||||
derivable from content. `next_unused_page` (rekordcrate: "points past end of
|
||||
file", unused) is the same class — see also item (1).
|
||||
|
||||
> Canonical reference note: the diff target is now the FRESH export
|
||||
> `data/acemo_fresh/PIONEER/rekordbox/export.pdb` (7 tracks). It is cleaner than
|
||||
> the edited `data/acemo` (6 tracks) but still incrementally built per-file, so
|
||||
> the divergences above still apply — just smaller (e.g. track 7's dead-row span
|
||||
> is 464B not 3568B, but it still overflows to page 48).
|
||||
|
||||
## What IS reproducible (NOW matched against the fresh reference)
|
||||
|
||||
- The 20-table directory layout and page placement: an empty table is
|
||||
strange-page-only; a table gets a data page IFF it has rows (`CORE_TABLES` is
|
||||
now empty — the old genres data page was an edit-history artifact of the
|
||||
edited ref).
|
||||
- `page_flags`: data pages are `0x34` for **tracks and history only**, `0x24`
|
||||
for every other table (the wider `{0,2,3,13,17,19}` was edited-ref noise);
|
||||
empty data pages `0x44`; strange pages `0x64`.
|
||||
- Strange-page body: the full DeviceSQL free-space map — `u4(page_index)`,
|
||||
`u4(first_data_page or 0x03ffffff)`, the 12-byte const tail, 1004×
|
||||
`f8 ff ff 1f`, 20 zero bytes — plus header `free=used=0`, `txc=txi=0x1fff`,
|
||||
`next_page = page_index+1`, and `unknown7 = 1` for tracks/history else 0.
|
||||
- Track row `index_shift` (`slot_index<<5`) and every other live-row content
|
||||
field except the opaque blob in divergence item (9).
|
||||
- Three header fields `re` decoded (FORMAT_SPEC §4/§5/§6) as edit-history state
|
||||
whose FRESH-export value maple already writes — so they're matched on a fresh
|
||||
export and only diverge against edited refs:
|
||||
- file-header offset 16 (`_unnamed4`): fresh=`0` (maple writes 0); edited DBs
|
||||
accumulate a small nonzero with no structural meaning.
|
||||
- strange/data-page `unknown7` (0x26): fresh=`0` everywhere (maple writes 0);
|
||||
nonzero only on edited DBs' strange pages.
|
||||
- row-group trailing u2: it is the "rows-touched-in-last-transaction" bitmask.
|
||||
On a fresh export it equals the row-present mask, so maple writes
|
||||
`presence` (matches 12/14 groups in `data/acemo_fresh`; the 2 misses are
|
||||
edited pages — divergence). NOTE: `re`'s "keep constant 16" advice came from
|
||||
reading `data/usb`, which is maple's OWN output, not a real export — a real
|
||||
fresh export uses the presence value, not 16.
|
||||
|
||||
See `re/FORMAT_SPEC.md` for the value-construction rules and `re/README.md` for
|
||||
the open RE questions.
|
||||
@@ -0,0 +1,54 @@
|
||||
# re/ — rekordbox `export.pdb` reverse-engineering
|
||||
|
||||
Shared record of the Ghidra/Frida work behind maple's pdb writer. See the
|
||||
top-level `README.md` for the high-level findings; this dir holds the raw
|
||||
artifacts so others can reproduce and extend the analysis.
|
||||
|
||||
## What's here
|
||||
|
||||
`scripts/` — Ghidra headless scripts (run against an analyzed `rekordbox.exe`):
|
||||
- `FindPdb.java` — locate the writer via string + magic-constant xrefs
|
||||
- `DecPdb.java` — decompile an explicit list of addresses
|
||||
- `CallTree.java` — walk a call tree, flag file-I/O / 4096-using functions
|
||||
- `FindHash.java` — hunt the per-track hash: decompile `formatForSearch` + callers + content-string xrefs
|
||||
|
||||
`findings/` — script output (decompiled C + xref maps):
|
||||
- `findpdb.txt` — string xrefs → the export functions; magic-constant hit counts
|
||||
- `decpdb.txt` — decompiled `export.pdb` serializer call stack (DeviceSQL Page Manager)
|
||||
- `calltree.txt` — call tree under the export orchestrator `FUN_14119d2d0`
|
||||
- `hash.txt` — `formatForSearch` + callers + `djmdContent`/`ContentID` xrefs
|
||||
|
||||
## Reproduce (not committed: the binary + Ghidra project)
|
||||
|
||||
`rekordbox.exe` and the analyzed Ghidra project are **not** in the repo
|
||||
(copyright + size). To rebuild from your own rekordbox install (Windows path
|
||||
visible from WSL):
|
||||
|
||||
```bash
|
||||
export JAVA_HOME=~/jdk/jdk-21.0.11+10 # Ghidra needs JDK 21
|
||||
GH=/path/to/ghidra_12.1.2_PUBLIC
|
||||
# one-time: import + analyze (~1h, 10G heap, 8 cores)
|
||||
GHIDRA_HEADLESS_MAXMEM=10G "$GH/support/analyzeHeadless" ~/ghidra_proj rbx \
|
||||
-import "/mnt/c/Program Files/rekordbox/rekordbox 7.0.5/rekordbox.exe" \
|
||||
-overwrite -max-cpu 8 -analysisTimeoutPerFile 7200
|
||||
# then run any script without re-analyzing:
|
||||
"$GH/support/analyzeHeadless" ~/ghidra_proj rbx -process rekordbox.exe -noanalysis \
|
||||
-scriptPath re/scripts -postScript FindPdb.java
|
||||
```
|
||||
|
||||
Runtime→Ghidra address map: `static = 0x140000000 + module_offset` (Ghidra image
|
||||
base `0x140000000`; ASLR runtime base varies — read `rekordbox.exe+0xOFFSET` from
|
||||
the Frida agent, never hardcode it).
|
||||
|
||||
## Live capture (Frida, Windows)
|
||||
|
||||
`../tools/capture.mjs` (Node, `npm i frida`) + `../tools/rbx_pdb_capture.js`
|
||||
(agent). Child-gates so the CIG-protected writer is hooked at spawn:
|
||||
`node tools/capture.mjs --spawn`.
|
||||
|
||||
## Open questions
|
||||
|
||||
- per-track `0x14` (28-bit) — varies per track, not a plain crc32; likely a
|
||||
stored `djmdContent` value. `0x18`/`0x1a` are a per-export constant.
|
||||
- `index_shift` (row `0x02`) — positional (`~0x20`/row, resets per page) but the
|
||||
exact formula isn't pinned. maple writes `0`; rekordbox doesn't.
|
||||
@@ -0,0 +1,470 @@
|
||||
# REFERENCE_COMPARISON.md — pyrekordbox & rekordcrate vs maple
|
||||
|
||||
What the two mature open-source rekordbox libraries can decode/encode, how
|
||||
that lines up with maple's writer and our RE findings, and the concrete GAPS
|
||||
maple must close for a full from-scratch USB export.
|
||||
|
||||
- **pyrekordbox** (Python, dylanljones) — https://pyrekordbox.readthedocs.io
|
||||
- **rekordcrate** (Rust, Holzhaus) — https://holzhaus.github.io/rekordcrate/
|
||||
local source at `/mnt/c/Users/ntr/code/rekordcrate/src`
|
||||
- **maple** — `maple-py/src/maple/*`, this `re/` dir.
|
||||
|
||||
Scope note (coordination): the per-format **library-capability** comparison and
|
||||
the from-scratch-export gaps are mine (task #12). The master.db SQLCipher key
|
||||
mechanics and opaque Content ID identity are owned by `re` (#10); the exact
|
||||
USBANLZ/artwork on-disk path schemes by `re` (#11). Where those are cited below
|
||||
they are tagged `[see re #10]` / `[see re #11]` and should be treated as
|
||||
`re`'s authority, not re-derived here.
|
||||
|
||||
---
|
||||
|
||||
## 0. One-screen capability matrix
|
||||
|
||||
| Format | pyrekordbox | rekordcrate | maple |
|
||||
|---|---|---|---|
|
||||
| **export.pdb** | parse only (via the `pdb` parser, read) | **parse + WRITE** (`BinWrite` on rows/pages, full roundtrip) | **WRITE only** (from-scratch, the whole point) |
|
||||
| **exportExt.pdb** | not supported | not supported | not written |
|
||||
| **ANLZ (.DAT/.EXT/.2EX)** | **parse only** ("creating … is planned") | parse + *partial* write (binrw `BinWrite` derived on waveform/cue structs; not a whole-file builder) | **WRITE** (minimal: PMAI + PPTH only) |
|
||||
| **artwork (.jpg thumbs)** | reads `ImagePath` from db only | n/a (just the pdb `Artwork` row path string) | **WRITE** (extracts embedded art) |
|
||||
| **master.db (db6/7)** | **READ + WRITE** (SQLCipher unlock) | not supported | not used |
|
||||
| **mysetting (\*SETTING.DAT)** | **READ + WRITE** (build from scratch) | parse only (`setting` module) | not written |
|
||||
| **rekordbox XML** | **READ + WRITE** | parse only (`xml` module) | not used |
|
||||
|
||||
The headline: **no existing library writes a complete from-scratch USB export.**
|
||||
rekordcrate writes `export.pdb` rows but not ANLZ/exportExt and has no
|
||||
metadata source; pyrekordbox writes master.db + mysetting + XML but not the USB
|
||||
`export.pdb`/ANLZ. maple is filling a real gap. The reference libs are most
|
||||
valuable to maple as (a) a **read-back oracle** (parse maple's output and diff)
|
||||
and (b) for pyrekordbox, a **source of the real opaque Content IDs** from
|
||||
master.db.
|
||||
|
||||
---
|
||||
|
||||
## 1. export.pdb
|
||||
|
||||
### What rekordcrate does (the closest reference writer)
|
||||
`pdb/mod.rs` decodes **and writes** every row type maple emits: Track, Artist,
|
||||
Album, Artwork, Color, Genre, Key, Label, PlaylistTreeNode, PlaylistEntry,
|
||||
ColumnEntry, HistoryPlaylist, HistoryEntry (`pub enum Row`, lines ~1064-1103).
|
||||
It has a genuine page serializer: `write_page_contents` (mod.rs:212),
|
||||
`write_options_and_get_row_offset` (mod.rs:463), per-row-type alignment
|
||||
(`align_by`, mod.rs:1121-1133), and a `Track` `BinWrite` impl (mod.rs:934).
|
||||
|
||||
### Where rekordcrate CONFIRMS maple's layout (high-value confirmations)
|
||||
- **Track fixed prefix = 0x5e bytes, then a 21×u2 (0x2a) offset block.**
|
||||
rekordcrate asserts `start_of_string_section - base_position == 0x5e` and
|
||||
seeks `0x2a` for 21 string offsets (mod.rs ~986-996). maple's
|
||||
`TRACK_FIXED_PREFIX = 0x5E`, `TRACK_HEADER_LEN = 0x5E + 21*2`
|
||||
(`rows.py:8-10`) and its 21-string list match field-for-field, including the
|
||||
exact string order (isrc, …, analyze_path, analyze_date, comment, title,
|
||||
unknown_string8, filename, file_path). **Independent confirmation of maple's
|
||||
track row.**
|
||||
- **Field order of the 0x5e prefix** is identical between rekordcrate's
|
||||
`Track::write_options` and maple's `serialize_track`: unknown1/index_shift/
|
||||
bitmask/sample_rate/composer_id/file_size/`unknown2`(=opaque lo @0x14)/
|
||||
`unknown3`+`unknown4`(=seed @0x18/0x1a)/artwork_id/key_id/orig_artist_id/
|
||||
label_id/remixer_id/bitrate/track_number/tempo/genre_id/album_id/artist_id/
|
||||
id/disc/play_count/year/sample_depth/duration/unknown5/color/rating/unknown6/
|
||||
unknown7.
|
||||
- **DeviceSQL string encoding.** rekordcrate `pdb/string.rs` roundtrips the
|
||||
same three forms maple's `strings.py` writes: short-ASCII
|
||||
(`(len<<1)|1`), long-ASCII (`0x40`), long-UTF16LE (`0x90`). Its tests encode
|
||||
`"I ❤ Rust"` as 0x90 and a long string as 0x40 — exactly maple's selection
|
||||
rule. The empty string is `0x03` in both. **Confirms strings.py.**
|
||||
- **Row-group / page-index mechanics.** rekordcrate's `RowGroup` (mod.rs:451,
|
||||
2102) uses the same 16-rows-per-group, backward-growing offset index, and
|
||||
presence bitmask maple builds in `pager.py`. `Header` (mod.rs:162) carries
|
||||
the same 13-bit num_row_offsets / 11-bit num_rows packing maple writes
|
||||
(`pager.py:131`).
|
||||
|
||||
### Where rekordcrate caught a maple BUG — `next_page` chain termination (the FREEZE)
|
||||
rekordcrate `pdb/mod.rs` documents `Page::next_page` (lines 262-265) precisely:
|
||||
> *"Index of the next page with the same page type. If this page is the last one
|
||||
> of that type, the page index stored in the field will point **past the end of
|
||||
> the file**."*
|
||||
|
||||
maple's writer originally set the **last** page of each table chain to
|
||||
`next_page = its own page_index` (a self-pointer). That is exactly what
|
||||
rekordcrate says must NOT happen — the last page must point *forward, past the
|
||||
allocated pages*. A self-referencing `next_page` makes a firmware page-walk that
|
||||
terminates on "next_page past EOF" **loop forever on the last page → the deck
|
||||
freezes.** This is the root cause being fixed in task #13: `writer.py` now points
|
||||
every last page at a dedicated free-pool sentinel page just past the overflow
|
||||
region (`free_sentinel`, materialised zero-filled) and sets `next_unused_page`
|
||||
beyond it — which removes the self-loop.
|
||||
(rekordcrate's own `read_pages`, mod.rs:185-196, terminates the walk on
|
||||
`page_index == table.last_page` from the directory, so its *parser* tolerates a
|
||||
self-pointer — which is why maple's read-back oracle never flagged it. The
|
||||
*firmware* uses the chain-walk, so only a real deck exposed the freeze.
|
||||
Lesson: a clean rekordcrate parse is NOT proof the chain is firmware-valid.)
|
||||
|
||||
> **Verified against a real export — the EXACT rule (not just "past EOF").**
|
||||
> I decoded `data/PIONEER/rekordbox/export.pdb` (847 tracks, 174 pages,
|
||||
> `next_unused=177`). rekordcrate's "points past the end of file" is a
|
||||
> simplification; the real, consistent rule across **all 20 tables** is:
|
||||
> **each table's last/only page sets `next_page == that table's
|
||||
> `empty_candidate`**, where `empty_candidate` (table-dir entry +4) is a
|
||||
> *distinct, nonzero* page index `< next_unused` — the head of that table's
|
||||
> free-page chain. Evidence: every last page's `next_page` equalled its own
|
||||
> dir `empty_candidate` (type0 173→174=ec; type1 4→53=ec; type19 40→41=ec; …),
|
||||
> and those targets (174,53,158,163,54,49,42,46,170,20,22,175,176,165,30,32,
|
||||
> 43,44,45,41) are all `< next_unused`. Strange-only tables already match
|
||||
> (next_page = page_idx+1 = their ec; free-map `data_ptr=0x03ffffff`) — maple's
|
||||
> `build_strange_page` does this correctly.
|
||||
>
|
||||
> So maple's current #13 fix (a **single shared sentinel** for every data
|
||||
> table + `empty_candidate=0` in the directory) removes the freeze but still
|
||||
> **diverges** from real rekordbox, which uses a **distinct per-table
|
||||
> `empty_candidate`** that the directory entry also records (nonzero). If
|
||||
> firmware ever cross-checks `next_page` against the directory's
|
||||
> `empty_candidate`, the shared sentinel + zero ec could still misbehave.
|
||||
> Recommended (reported to `maple`): give each data table its own free page,
|
||||
> set `next_page = that page` AND write that index into the table-dir
|
||||
> `empty_candidate`, keep them all `< next_unused`. This is straightforwardly
|
||||
> reproducible (unlike the *value* of `empty_candidate` on an edited ref, which
|
||||
> is GC state — here we only need a valid distinct free page, which a
|
||||
> from-scratch writer fully controls).
|
||||
|
||||
### Where rekordcrate is WRONG and maple is RIGHT — page row-count header (>255 rows)
|
||||
Audited because rekordcrate's `Page` header (mod.rs:277-377) reads the row count
|
||||
as **`num_rows_small: u8` @0x18** plus **`num_rows_large: u16` @0x22** with a
|
||||
`calculate_num_rows` rule (`large if large>small && large!=0x1fff else small`).
|
||||
maple instead writes the **canonical crate-digger .ksy bitfield** at 0x18:
|
||||
13-bit `num_row_offsets` + 11-bit `num_rows`, then `page_flags` @0x1b
|
||||
(`pager.py:131`). These are two different readings of the same bytes — so I
|
||||
**verified against a real export** (`data/PIONEER`, which has 284-row
|
||||
playlist-entry pages):
|
||||
- Real page 18 bytes @0x18 = `1c 81 23 24`. The 24-bit LE value `0x23811c`
|
||||
decodes (kaitai/maple way) to `num_row_offsets=284, num_rows=284` — **correct**.
|
||||
- maple's packer reproduces `1c 81 23` **exactly** for n=284 (and `a6 c0 14` for
|
||||
166, `7e c0 0f` for 126 — all matching real pages byte-for-byte).
|
||||
- rekordcrate's reading of the *same* bytes: `num_rows_small = 0x1c = 28` (just
|
||||
the low byte) and `num_rows_large @0x22 = 283` (which is really the
|
||||
`transaction_row_index` edit field), giving `calculate_num_rows = 283` — i.e.
|
||||
rekordcrate would report **283 rows on a 284-row page, and 28 if the
|
||||
transaction field were 0.** Its small/large model only coincidentally agrees
|
||||
when `n <= 255`.
|
||||
- **Conclusion:** maple's header encoding is correct and firmware-faithful for
|
||||
pages up to 2047 rows (the 11-bit max); rekordcrate's `num_rows_small`/
|
||||
`num_rows_large` is an older/divergent decode that breaks for `n > 255`. This
|
||||
matters because maple *can* emit >255-row pages (playlist_entries fits ~284/
|
||||
page, colors ~394). No maple change needed — this is a maple-is-right result.
|
||||
|
||||
### Where rekordcrate's row alignment diverges (maple matches real rekordbox)
|
||||
rekordcrate aligns each row by its Rust struct's natural `align_of_val`
|
||||
(mod.rs:1114-1136), which would 2-align u16-only row types. maple aligns **every
|
||||
row to 4** (`pager._align4`). Checked real exports: **every row in
|
||||
`data/PIONEER` (3567/3567) and `rb_ref` (101/101) is 4-aligned.** So maple's
|
||||
universal 4-alignment matches rekordbox; rekordcrate's per-type scheme is the
|
||||
divergent one (its own code comments admit "we should survey all the different
|
||||
kind of rows and ensure they're aligned properly", mod.rs:1110). No maple change
|
||||
needed.
|
||||
|
||||
### Where maple's RE goes BEYOND rekordcrate (we know more than the reference)
|
||||
These are fields rekordcrate leaves as opaque caller-supplied "unknown"s but
|
||||
maple's RE solved/characterised — important because it means **rekordcrate
|
||||
cannot be used to validate these; maple's FORMAT_SPEC is the authority**:
|
||||
|
||||
- **`index_shift` (row +0x02).** rekordcrate calls it `index_shift` but writes
|
||||
whatever the caller stored — it does **not** know the value (mod.rs:104-105
|
||||
"Unknown field, called `index_shift` by flesniak"). maple **derives** it as
|
||||
`slot_index << 5` and patches it at page-build time (`pager.py:142-145`,
|
||||
FORMAT_SPEC.md §1). maple is correct where rekordcrate is agnostic.
|
||||
- **The 8-byte opaque blob (0x14 lo / 0x18+0x1a seed).** rekordcrate:
|
||||
`unknown2: u32`, `unknown3: u16` ("always 19048?"), `unknown4: u16`
|
||||
("always 30967?") — three blind unknowns it round-trips verbatim
|
||||
(mod.rs:115-119). maple's RE (FORMAT_SPEC.md §2) proved this is one LE u64 =
|
||||
`opaque_track_id_lo | (opaque_seed<<32)`, a per-track Content ID + per-install
|
||||
seed stored in master.db, **non-derivable** from metadata, and that the
|
||||
"19048/30967" constants are NOT universal (they only hold for some old
|
||||
exports). maple emits a non-zero stand-in; the real values live in master.db
|
||||
`[see re #10]`. maple's understanding is strictly deeper than rekordcrate's.
|
||||
- **`bitmask` (0x06).** rekordcrate: opaque "unknown" round-tripped. maple
|
||||
writes a fixed `788224` (0xC0700) — itself not yet RE'd to a formula, but
|
||||
maple at least pins a known-good value.
|
||||
- **Strange/"first" pages.** rekordcrate leaves the strange-page heap **zero**
|
||||
(it round-trips parsed input); maple **constructs** the full DeviceSQL
|
||||
free-space map from scratch byte-for-byte (`pager.build_strange_page`,
|
||||
verified vs `data/acemo_fresh` and the real strange pages in data/PIONEER),
|
||||
including the `f8 ff ff 1f`×1004 map, the `0x03ffffff` no-data sentinel, and
|
||||
`unknown7=1` for tracks/history. `re` confirms maple is **strictly more
|
||||
faithful than rekordcrate on strange pages.** This is from-scratch generation
|
||||
rekordcrate never had to do.
|
||||
- **RowGroup trailing u2.** rekordcrate writes a constant `16` placeholder
|
||||
(`Default::default`, mod.rs:454) and does **not** decode it; `re` confirms
|
||||
genuine rekordbox **never** uses 16 there — it is the "rows touched by the
|
||||
last transaction" bitmask (FORMAT_SPEC §6). maple writes `present_flags` (the
|
||||
single-transaction value), closer to real output; either way it is
|
||||
edit-history state (KNOWN_DIVERGENCE §6), not a rekordcrate-validatable field.
|
||||
|
||||
### Where rekordcrate (and our RE) AGREE the field is unreproducible — with a caveat
|
||||
rekordcrate documents `empty_candidate` ("maybe links to a chain of empty
|
||||
pages if the database is ever garbage collected") and `next_unused_page`
|
||||
("points past end of file") as allocator/GC state — matching
|
||||
KNOWN_DIVERGENCE.md items 11 and 1. The **historical VALUE** on an edited
|
||||
reference is indeed unreproducible. **BUT** my data-page audit (see the FREEZE
|
||||
section above) found `empty_candidate` is **load-bearing structure**, not just
|
||||
GC trivia: in data/PIONEER every table's last page has
|
||||
`next_page == that table's empty_candidate`, a distinct free page `< next_unused`.
|
||||
So a from-scratch writer must still emit a *valid* distinct `empty_candidate`
|
||||
per table (which it fully controls) even though it can't match an edited ref's
|
||||
specific value. **Confirms** these are correctly on
|
||||
maple's divergence list, not bugs.
|
||||
|
||||
### GAP — none structural for export.pdb
|
||||
maple writes a complete, players-accepted export.pdb. Remaining export.pdb
|
||||
divergences from a reference file are all **edit-history artifacts**
|
||||
(KNOWN_DIVERGENCE.md) that neither library can reproduce from content either.
|
||||
The one content-level gap is the opaque blob (see §4).
|
||||
|
||||
---
|
||||
|
||||
## 2. exportExt.pdb — GAP (nobody writes it; maybe nobody needs to)
|
||||
|
||||
- **pyrekordbox:** no support.
|
||||
- **rekordcrate:** no support (its `pdb` module is export.pdb only).
|
||||
- **maple:** does not emit it.
|
||||
|
||||
Format facts (Deep Symmetry export analysis; reverse-engineered by Dominik
|
||||
Stolz, crate-digger issue #11): `exportExt.pdb` is a *second, smaller*
|
||||
DeviceSQL file beside `export.pdb` holding only two table types — **Tags
|
||||
(type 03)** and **Tag Tracks (type 04)** — i.e. the "My Tag" categorisation
|
||||
feature and its tag→track associations. It uses the same page/row container
|
||||
format as export.pdb.
|
||||
|
||||
**Assessment / recommendation:** exportExt.pdb is **optional**. It only carries
|
||||
the My-Tag feature; players that don't find it simply show no custom tags, and
|
||||
tracks/playlists from export.pdb play normally. maple has no tag data to
|
||||
populate anyway. **Do not block a v1 export on this.** If maple ever supports
|
||||
tags, the existing `pager.py`/`PageBuilder` already produces the container
|
||||
format — only two new row serialisers (tag row, tag-track row) and a second
|
||||
`write_pdb`-style call writing the 2-table directory would be needed.
|
||||
Confidence the file is skippable: high (it post-dates the core format and is
|
||||
absent from older exports that still play).
|
||||
|
||||
---
|
||||
|
||||
## 3. ANLZ (.DAT / .EXT / .2EX) — the biggest real GAP
|
||||
|
||||
### What the references decode
|
||||
- **pyrekordbox** (`pyrekordbox.anlz`): full **parser** for PMAI + all section
|
||||
tags — PPTH, PQTZ/PQT2 (beat grids), PCOB/PCO2 (cues/loops), PVBR, PSSI
|
||||
(phrase/song-structure, XOR-masked), PWAV/PWV2 (previews), PWV3/4/5
|
||||
(detailed + colour waveforms), PWV6/7/PWVC (.2EX). Writing is **"planned …
|
||||
but the full structure has to be understood first"** — i.e. **read-only
|
||||
today.**
|
||||
- **rekordcrate** (`anlz.rs`): parses the same tag set; magic constants present
|
||||
in source: PMAI, PPTH, PQTZ, PVBR, PWAV, PWV2, PCOB, PCO2, PCPT, PCP2, PSSI,
|
||||
PWV3/4/5. It derives `BinWrite` on the waveform-column structs
|
||||
(`WaveformPreviewColumn` etc., anlz.rs:368-428) and has a `write_encrypted`
|
||||
for the XOR-masked PSSI (anlz.rs:902) — so it has *piece-level* write code,
|
||||
but **no whole-file ANLZ builder / CLI to emit a valid .DAT/.EXT from track
|
||||
audio.**
|
||||
|
||||
### What maple does
|
||||
`anlz.py` writes a **minimal** `ANLZ0000.DAT`: a 28-byte PMAI header + a single
|
||||
**PPTH** section (u4 len + UTF-16BE path). No beat grid, no waveform, no cues,
|
||||
no `.EXT`, no `.2EX`. Rationale in the module docstring: firmware reads PPTH to
|
||||
confirm which audio file the analysis belongs to, and a track whose
|
||||
`analyze_path` points at *nothing* can be hidden from the library, so PPTH
|
||||
alone makes the track "appear analysed enough to surface."
|
||||
|
||||
### CONFIRMATIONS
|
||||
- maple's PMAI/PPTH layout (4-cc + u4 len_header + u4 len_tag; PPTH = u4
|
||||
len_path + UTF-16**BE**, big-endian throughout) matches the format both
|
||||
references parse. The big-endian-for-ANLZ-vs-little-endian-for-pdb split that
|
||||
`anlz.py` calls out is corroborated by both libraries.
|
||||
|
||||
### GAPS maple must close for full faithfulness
|
||||
1. **No `.EXT` file.** Modern CDJs/XDJs (NXS2+) read `ANLZ0000.EXT` for colour
|
||||
waveforms (PWV4/5), extended cues (PCO2), and phrase data (PSSI). Without it
|
||||
maple tracks show **no colour waveform and no phrase/structure** on newer
|
||||
players. The `analyze_path` in the pdb points at the `.DAT`; the player
|
||||
derives the `.EXT` sibling. maple writes neither beyond the `.DAT` PPTH stub.
|
||||
2. **No beat grid (PQTZ).** Without a beat grid, beat-sync / quantize / loop
|
||||
features have nothing to lock to; the track plays but degrades the CDJ
|
||||
experience. This is the highest-value ANLZ section to add after PPTH.
|
||||
3. **No waveform (PWAV/PWV3-5).** Cosmetic but conspicuous — tracks show a flat
|
||||
or absent waveform. pyrekordbox can *parse* these to learn the exact byte
|
||||
layout, and rekordcrate's `BinWrite` waveform-column structs are a concrete
|
||||
reference for the encoding, but **maple must generate the waveform data
|
||||
itself** (decode audio → downsample → colour-band), which neither library
|
||||
provides.
|
||||
4. **Generation, not just layout, is the hard part.** The byte format is fully
|
||||
documented (both libs + Deep Symmetry); what's missing everywhere is the
|
||||
DSP that *produces* beat positions and waveform columns from PCM. That DSP
|
||||
is maple's to build — the references only validate the container.
|
||||
|
||||
**Recommendation:** keep the PPTH-only `.DAT` as the working baseline (it is
|
||||
what gets tracks visible), and treat PQTZ beat grid + a `.EXT` with PWV5 colour
|
||||
waveform as the next ANLZ milestones. Use pyrekordbox's parser as the
|
||||
round-trip oracle (write a section, parse it back, compare). Confidence the
|
||||
PPTH-only stub is sufficient for *visibility/playback*: medium-high (matches
|
||||
maple's own deck testing rationale); confidence it's sufficient for a *complete*
|
||||
experience: low — beat grid/waveform are expected by users.
|
||||
|
||||
---
|
||||
|
||||
## 4. master.db (db6/7) + the opaque Content IDs — GAP maple can close via pyrekordbox
|
||||
|
||||
### What pyrekordbox does (this is its core strength) — VALIDATED by re #10
|
||||
- Reads **and writes** the encrypted `master.db`. The db is **SQLCipher4**
|
||||
encrypted with a **single GLOBAL key, identical on every install** (not
|
||||
machine/license-derived). `re` validated the concrete key (FORMAT_SPEC §2
|
||||
Round-3): `402fd482c38817c35ffa8ffb8c7d93143b749e7d315df7a81732a1ff43608497`.
|
||||
pyrekordbox 0.4.4 + `sqlcipher3-wheels` installs and wires up correctly, so a
|
||||
**valid** master.db opens and exposes `DjmdContent` `[see re #10]`.
|
||||
- Exposes `DjmdContent` (one row per track) with: **`ID`** (the rekordbox
|
||||
internal Content ID), `FolderPath`, `FileNameL`, **`AnalysisDataPath`** (the
|
||||
`/PIONEER/USBANLZ/...` analysis path), **`ImagePath`** (artwork), and FK ids
|
||||
to Artist/Album/Genre/Label/Key/Composer; plus `djmdCue`, `djmdPlaylist`/
|
||||
`djmdSongPlaylist`, `djmdHistory`, `djmdMixerParam`, `djmdDevice`, etc.
|
||||
|
||||
> **Reality check from re #10 — currently BLOCKED on data, not on method:**
|
||||
> - The captured `maple-py/captures/p17328_master.db` is **corrupt/truncated**
|
||||
> (zeroed SQLCipher salt + page 1) — no key can open it. The genuine
|
||||
> `exportLibrary.db` shipped beside export.pdb uses a *different* key scheme
|
||||
> that the global master.db key does not open, and pyrekordbox has no
|
||||
> exportLibrary.db support. So the IDs have **not yet been read** on this RE
|
||||
> machine — the path works only against a **live**
|
||||
> `%APPDATA%/Pioneer/rekordbox6/master.db`.
|
||||
> - `opaque_track_id_lo == DjmdContent.ID & 0xFFFFFFFF` is a **strong inference,
|
||||
> still UNTESTED** (the lo range fits the 9-digit Content-ID band), not a
|
||||
> confirmed equality.
|
||||
> - The per-install **seed** (`0x21b4a7ab`) is **NOT** a DjmdContent column and
|
||||
> its source is **still unidentified** (likely a library/USB UUID or
|
||||
> agentRegistry/local_usn-style marker; needs a readable master.db to find).
|
||||
> - pyrekordbox `add_content` lets *rekordbox* assign the Content ID — there is
|
||||
> **no API to force a specific ID** — and it does **not** drive a USB export
|
||||
> (`get_device` is read-only). So pyrekordbox is a **read source of real IDs**,
|
||||
> not a writer/exporter and not a way to dictate IDs.
|
||||
|
||||
### Why this matters for maple (ties to FORMAT_SPEC §2 / KNOWN_DIVERGENCE #9)
|
||||
maple's RE proved the export.pdb track blob at 0x14/0x18 is the rekordbox
|
||||
**Content ID** (per-track, 28-bit band) plus a **per-install seed**, both stored
|
||||
in master.db and **not derivable** from the audio/metadata maple has. maple
|
||||
currently emits a non-zero *stand-in* (`crc32(id|file_path)` lo +
|
||||
`TRACK_OPAQUE_SEED = 0x21b4a7ab`). Therefore:
|
||||
|
||||
- **GAP (closable in principle, not yet proven): real Content IDs.** If maple
|
||||
runs on a machine with rekordbox installed, it can use **pyrekordbox + the
|
||||
global key to open the LIVE `%APPDATA%/Pioneer/rekordbox6/master.db` and read
|
||||
`DjmdContent.ID`** per track, then write those into the pdb blob instead of
|
||||
the stand-in. `re` validated the *mechanism* (global key, DjmdContent schema)
|
||||
but could NOT confirm `ID == opaque_track_id_lo` here because the only
|
||||
captured master.db is corrupt — so this is a **strong-but-untested** lead, not
|
||||
a closed item. Next step (re #10): point pyrekordbox at a live master.db and
|
||||
confirm the equality for the AceMo tracks. The per-install **seed**
|
||||
`0x21b4a7ab` source is still unidentified, so maple keeps the configurable
|
||||
`TRACK_OPAQUE_SEED` either way.
|
||||
- **Caveats:** (a) works only *with a rekordbox install present and a readable
|
||||
master.db* (not the corrupt capture), and only for tracks rekordbox already
|
||||
knows; (b) pyrekordbox `add_content` cannot **force** a chosen ID (rekordbox
|
||||
assigns it) and does **not** drive a USB export, so it can supply IDs but
|
||||
cannot replace maple's writer. Pure from-scratch (no rekordbox install) stays
|
||||
a documented divergence (KNOWN_DIVERGENCE #9), with maple's non-zero stand-in.
|
||||
|
||||
### rekordcrate
|
||||
No master.db support at all — it is a *device-export* (USB) library, not a
|
||||
*library-database* library. So it cannot help with Content IDs.
|
||||
|
||||
---
|
||||
|
||||
## 5. mysettings (\*SETTING.DAT) — GAP (small, easy) maple doesn't write
|
||||
|
||||
- **pyrekordbox:** reads **and writes/builds** `DEVSETTING.DAT`,
|
||||
`MYSETTING.DAT` (40B, CDJ play settings), `MYSETTING2.DAT` (40B), and
|
||||
`DJMMYSETTING.DAT` (52B, DJM mixer) — dict-style get/set + save. (`DEVSETTING`
|
||||
payload still partly ununderstood.)
|
||||
- **rekordcrate:** `setting` module **parses** these (read-only).
|
||||
- **maple:** does **not** write any of them.
|
||||
|
||||
These live in `PIONEER/` on the USB alongside `rekordbox/`. They are **optional
|
||||
preference files** — absent, the player just uses its own defaults; tracks and
|
||||
playlists are unaffected. **Recommendation:** low priority. If desired,
|
||||
pyrekordbox can *build* default ones in a few lines (it's read/write), so maple
|
||||
could shell out to / import pyrekordbox to drop standard `MYSETTING.DAT` +
|
||||
`MYSETTING2.DAT` rather than re-implementing the 40-byte structs. Confidence
|
||||
they're optional for playback: high.
|
||||
|
||||
---
|
||||
|
||||
## 6. Overall directory layout + path schemes (cross-check) `[see re #11 / FORMAT_SPEC §7]`
|
||||
|
||||
Both references and Deep Symmetry agree on the USB tree maple targets:
|
||||
`/PIONEER/rekordbox/export.pdb` (+ optional `exportExt.pdb`),
|
||||
`/PIONEER/USBANLZ/<...>/ANLZ0000.{DAT,EXT,2EX}`,
|
||||
`/PIONEER/Artwork/<...>/a<id>.jpg`, plus optional `/PIONEER/*SETTING.DAT`.
|
||||
maple writes `PIONEER/rekordbox/export.pdb` (`cli.py:156`) and
|
||||
`PIONEER/USBANLZ/<bucket>/<id>/ANLZ0000.DAT` (`cli.py:92`).
|
||||
|
||||
`re` resolved both path schemes (#11, FORMAT_SPEC §7):
|
||||
|
||||
- **Artwork — EXACTLY reproducible, and maple already matches.** Real rekordbox
|
||||
uses `/PIONEER/Artwork/{id//20+1:05d}/a{id}.jpg` (+ a `a{id}_m.jpg` 240px
|
||||
thumb), 20 ids per bucket folder (bucket 1 = ids 1-19). `re` verified
|
||||
176/176 in data/MAPLE, and maple's `artwork._bucket()` already implements this
|
||||
rule. **Reproducible, confirmed — no gap.**
|
||||
|
||||
- **USBANLZ — NOT reproducible; maple's own scheme is correct-by-construction.**
|
||||
Real rekordbox uses `/PIONEER/USBANLZ/P{bucket}/{8hex}/ANLZ0000.DAT` where the
|
||||
8-hex is an **opaque per-track analysis id** (NOT the track id, NOT the row
|
||||
0x14 lo, NOT any crc of the path — `re` ruled these out) and the P-bucket
|
||||
(0-127) is **not a derivable function** of it. Like the 0x14 blob, it is
|
||||
DB-sourced and non-reproducible. maple synthesises its own
|
||||
`P{(crc32(path)>>20)&0xFFF:03X}/{crc32(path):08X}` and writes the `.DAT`
|
||||
there, so the pdb's `analyze_path` and the on-disk file always agree. **This
|
||||
is fine:** the player only requires every pdb path to resolve to a real
|
||||
on-disk file, not to match rekordbox's id scheme.
|
||||
|
||||
> **Gotcha for the gaps below (from `re`):** `tools/build_acemo.py` MIRRORS a
|
||||
> reference export's paths verbatim and writes **no** ANLZ/artwork files (it's a
|
||||
> byte-diff fixture builder). The **production** `cli.py build()` is the one that
|
||||
> actually writes each ANLZ/artwork file at the recorded path. Don't conflate
|
||||
> the two when reasoning about what maple emits on disk.
|
||||
|
||||
---
|
||||
|
||||
## 7. Bottom line — prioritised GAP list for a full from-scratch export
|
||||
|
||||
1. **(high, lead — untested) Real opaque Content IDs via pyrekordbox/master.db**
|
||||
— mechanism validated by `re` (global SQLCipher key `402fd482…` +
|
||||
`Rekordbox6Database` reads `DjmdContent`), but `DjmdContent.ID ==
|
||||
opaque_track_id_lo` is a **strong INFERENCE, NOT validated** (the only
|
||||
captured master.db is corrupt — zeroed salt — so `re` never confirmed the
|
||||
equality on real data). Would close KNOWN_DIVERGENCE #9 *if* confirmed against
|
||||
a readable **live** `%APPDATA%/Pioneer/rekordbox6/master.db`.
|
||||
`[see re #10 / FORMAT_SPEC §2 Round-3]`
|
||||
2. **(OPEN — Ghidra trace pending) Per-install `opaque_seed` source.** The seed
|
||||
(`0x21b4a7ab` / `0xd6d3a567`) is **NOT** a `DjmdContent` column and `re` has
|
||||
**not located it** — likely a per-library UUID/agent/usn marker or a computed
|
||||
value. It needs a readable master.db to inspect candidate rows, OR a Ghidra
|
||||
trace of rekordbox.exe (which `re` can now attempt — the Ghidra MCP just came
|
||||
online), so it is **not permanently blocked.** Until resolved, maple keeps the
|
||||
configurable `TRACK_OPAQUE_SEED` stand-in. This is the one genuinely open
|
||||
question touching the opaque blob. `[see re #10 — OPEN; FORMAT_SPEC §2 Round-3]`
|
||||
3. **(high) ANLZ beat grid (PQTZ) + an `.EXT` colour waveform (PWV5)** — needs
|
||||
DSP maple must build; references validate the byte format only. Biggest
|
||||
functional gap on modern players.
|
||||
4. **(structural — in progress) Per-table `empty_candidate` / last-page
|
||||
`next_page`** — the FREEZE fix (#13). maple must point each table's last page
|
||||
at a distinct free page recorded in the dir's `empty_candidate`, not a shared
|
||||
sentinel with `empty_candidate=0`. See the FREEZE section in §1.
|
||||
5. **(low) mysettings** — optional; pyrekordbox can build them.
|
||||
6. **(low/skip) exportExt.pdb** — optional My-Tag data; only if maple adds tags.
|
||||
7. **(reference, not a gap) Use rekordcrate as a read-back oracle** — parse
|
||||
maple's `export.pdb` with rekordcrate's CLI to catch container regressions;
|
||||
use pyrekordbox's anlz parser as the oracle for any new ANLZ sections.
|
||||
**Caveat (proven this session):** rekordcrate's parser tolerates things the
|
||||
firmware rejects (the `next_page` self-loop) and is itself wrong on some
|
||||
fields (>255-row count, per-type alignment) — so it is a *regression* oracle,
|
||||
NOT a correctness authority. Validate load-critical fields against REAL
|
||||
export bytes (e.g. data/PIONEER), not against rekordcrate.
|
||||
|
||||
**Headline confirmation for the team:** rekordcrate's *writer* independently
|
||||
validates maple's export.pdb track-row layout, DeviceSQL string encoding, and
|
||||
page/row-group container down to the 0x5e/0x2a offsets — but it is *agnostic*
|
||||
exactly where maple's RE added knowledge (`index_shift` formula, the opaque-id
|
||||
semantics). So maple's FORMAT_SPEC is the authority for those fields, and the
|
||||
remaining real work is **ANLZ generation** and **pulling the real Content IDs
|
||||
from master.db** — neither of which any existing library does end-to-end.
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,595 @@
|
||||
// HIT 0x4a68 @141f0a08b in FUN_141f09e70@141f09e70 : LEA RCX,[RDI + 0x4a68]
|
||||
// HIT 0x4a68 @141f0a08b in FUN_141f09e70@141f09e70 : LEA RCX,[RDI + 0x4a68]
|
||||
// HIT 0x4a68 @141f15d17 in FUN_141f15bf0@141f15bf0 : LEA RCX,[RBX + 0x4a68]
|
||||
// HIT 0x4a68 @141f15d17 in FUN_141f15bf0@141f15bf0 : LEA RCX,[RBX + 0x4a68]
|
||||
// HIT 0x4a68 @141f637b7 in FUN_141f63210@141f63210 : MOV RAX,qword ptr [RBX + 0x4a68]
|
||||
// HIT 0x4a68 @141f637b7 in FUN_141f63210@141f63210 : MOV RAX,qword ptr [RBX + 0x4a68]
|
||||
// HIT 0x4a68 @141f637be in FUN_141f63210@141f63210 : LEA RCX,[RBX + 0x4a68]
|
||||
// HIT 0x4a68 @141f637be in FUN_141f63210@141f63210 : LEA RCX,[RBX + 0x4a68]
|
||||
// HIT 0x4a68 @141f637da in FUN_141f63210@141f63210 : LEA RCX,[RBX + 0x4a68]
|
||||
// HIT 0x4a68 @141f637da in FUN_141f63210@141f63210 : LEA RCX,[RBX + 0x4a68]
|
||||
// HIT 0x4a68 @141f637e1 in FUN_141f63210@141f63210 : MOV RAX,qword ptr [RBX + 0x4a68]
|
||||
// HIT 0x4a68 @141f637e1 in FUN_141f63210@141f63210 : MOV RAX,qword ptr [RBX + 0x4a68]
|
||||
// --- decompiling containing functions ---
|
||||
|
||||
//===== FUN_141f09e70 @ 141f09e70 =====
|
||||
|
||||
undefined8 *
|
||||
FUN_141f09e70(undefined8 *param_1,undefined4 param_2,undefined8 param_3,longlong param_4)
|
||||
|
||||
{
|
||||
undefined8 *puVar1;
|
||||
char cVar2;
|
||||
undefined8 uVar3;
|
||||
undefined8 *puVar4;
|
||||
undefined8 *puVar5;
|
||||
longlong lVar6;
|
||||
undefined8 *puVar7;
|
||||
undefined8 *puVar8;
|
||||
longlong local_res20;
|
||||
undefined1 local_f8 [4];
|
||||
undefined8 local_f4;
|
||||
undefined1 local_ec;
|
||||
undefined4 local_e8;
|
||||
undefined1 local_e0 [8];
|
||||
undefined1 local_d8 [8];
|
||||
undefined1 local_d0 [8];
|
||||
undefined1 local_c8 [8];
|
||||
undefined2 local_c0;
|
||||
undefined1 local_b8 [8];
|
||||
undefined1 local_b0;
|
||||
undefined1 local_a8 [8];
|
||||
undefined4 local_a0;
|
||||
undefined1 local_98 [8];
|
||||
undefined1 local_90 [8];
|
||||
undefined1 local_88 [8];
|
||||
undefined1 local_80 [8];
|
||||
char *local_78;
|
||||
undefined1 local_70 [16];
|
||||
undefined1 local_60 [16];
|
||||
undefined1 local_50 [24];
|
||||
|
||||
puVar8 = (undefined8 *)0x0;
|
||||
uVar3 = FUN_1423d5980(&local_res20,param_3);
|
||||
FUN_141f09b00(param_1,0xffffffff,uVar3);
|
||||
FUN_14246e8b0(param_1 + 0x134);
|
||||
*param_1 = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
param_1[0x14] = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
puVar1 = param_1 + 0x15;
|
||||
*puVar1 = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
param_1[0x134] = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
param_1[0x138] = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
param_1[0x139] = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
param_1[0x13a] = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
*(undefined4 *)(param_1 + 0x13b) = 0x25f8;
|
||||
param_1[0x1a3] = 0;
|
||||
param_1[0x1a4] = 0;
|
||||
param_1[0x1bb] = 0;
|
||||
param_1[0x1bd] = 0;
|
||||
param_1[0x1be] = 0;
|
||||
param_1[0x1bf] = 0;
|
||||
param_1[0x1c0] = 0;
|
||||
*(undefined4 *)(param_1 + 0x1c1) = 0;
|
||||
*(undefined2 *)((longlong)param_1 + 0xe0c) = 0;
|
||||
*(undefined1 *)((longlong)param_1 + 0xe0e) = 0;
|
||||
*(undefined4 *)(param_1 + 0x1c2) = 3;
|
||||
puVar7 = param_1 + 0x1c4;
|
||||
*puVar7 = 0;
|
||||
param_1[0x1c5] = 0;
|
||||
param_1[0x1c6] = 0;
|
||||
param_1[0x1c8] = 0;
|
||||
FUN_141ee27c0(param_1 + 0x1c9);
|
||||
_eh_vector_constructor_iterator_(param_1 + 0x2d5,0x470,4,FUN_141ee3f50,FUN_141ee3fe0);
|
||||
FUN_141ee3f50(param_1 + 0x50d);
|
||||
FUN_141ee4d90(param_1 + 0x59c);
|
||||
FUN_141ee5080(param_1 + 0x62b);
|
||||
FUN_141ee5400(param_1 + 0x6ba);
|
||||
FUN_141ee5970(param_1 + 0x74a);
|
||||
FUN_141ee5db0(param_1 + 0x7da);
|
||||
FUN_141ee6210(param_1 + 0x86a);
|
||||
FUN_141ee97f0(param_1 + 0x8fb);
|
||||
FUN_141ee9a10(param_1 + 0x907);
|
||||
FUN_141ee9c10(param_1 + 0x915);
|
||||
FUN_141ee9c10(param_1 + 0x920);
|
||||
FUN_141ee9c10(param_1 + 0x92b);
|
||||
FUN_141ee9c10(param_1 + 0x936);
|
||||
FUN_141ee9e30(param_1 + 0x941);
|
||||
FUN_141eea060(param_1 + 0x94d);
|
||||
FUN_141eea150(param_1 + 0x959);
|
||||
FUN_141eea330(param_1 + 0x964);
|
||||
*(undefined2 *)(param_1 + 0x96f) = 0;
|
||||
*(undefined1 *)((longlong)param_1 + 0x4b7a) = 0;
|
||||
*(undefined4 *)((longlong)param_1 + 0x4b7c) = 0;
|
||||
*(undefined1 *)(param_1 + 0x970) = 1;
|
||||
param_1[0x197] = 0;
|
||||
puVar4 = (undefined8 *)FUN_1423ce840(local_98,"CDJ-TOUR1");
|
||||
cVar2 = FUN_1423d2890(param_3,*puVar4);
|
||||
if (cVar2 != '\0') {
|
||||
*(undefined1 *)((longlong)param_1 + 0xe0b) = 1;
|
||||
}
|
||||
*(undefined4 *)((longlong)param_1 + 0xe1a) = 0;
|
||||
*(undefined1 *)((longlong)param_1 + 0xe1e) = 0;
|
||||
LOCK();
|
||||
*(undefined4 *)(param_1 + 0x1c8) = 0;
|
||||
UNLOCK();
|
||||
LOCK();
|
||||
*(undefined4 *)((longlong)param_1 + 0xe44) = 0;
|
||||
UNLOCK();
|
||||
puVar5 = (undefined8 *)FUN_14217c644(0x40);
|
||||
puVar4 = puVar8;
|
||||
if (puVar5 != (undefined8 *)0x0) {
|
||||
FUN_1423aa4f0(puVar5);
|
||||
puVar5[6] = 0;
|
||||
*(undefined4 *)(puVar5 + 7) = 8;
|
||||
puVar4 = puVar5;
|
||||
}
|
||||
param_1[0x1c7] = puVar4;
|
||||
lVar6 = FUN_14217c644(0x478);
|
||||
puVar4 = puVar8;
|
||||
if (lVar6 != 0) {
|
||||
puVar4 = (undefined8 *)FUN_141ee41c0(lVar6,8);
|
||||
}
|
||||
param_1[0x59b] = puVar4;
|
||||
local_f8[0] = 0;
|
||||
local_f4 = 0;
|
||||
local_ec = 0;
|
||||
local_e8 = 0;
|
||||
FUN_1423d59e0(local_e0);
|
||||
FUN_1423d59e0(local_d8);
|
||||
FUN_1423d59e0(local_d0);
|
||||
FUN_1423d59e0(local_c8);
|
||||
local_c0 = 0;
|
||||
FUN_1423d59e0(local_b8);
|
||||
local_b0 = 0;
|
||||
FUN_1423d59e0(local_a8);
|
||||
local_a0 = 0;
|
||||
lVar6 = param_1[0x1c5];
|
||||
if (lVar6 == param_1[0x1c6]) {
|
||||
FUN_141edc340(puVar7,lVar6,local_f8);
|
||||
}
|
||||
else {
|
||||
FUN_141068df0(lVar6,local_f8);
|
||||
param_1[0x1c5] = param_1[0x1c5] + 0x60;
|
||||
}
|
||||
lVar6 = param_1[0x1c5];
|
||||
if (lVar6 == param_1[0x1c6]) {
|
||||
FUN_141edc340(puVar7,lVar6,local_f8);
|
||||
}
|
||||
else {
|
||||
FUN_141068df0(lVar6,local_f8);
|
||||
param_1[0x1c5] = param_1[0x1c5] + 0x60;
|
||||
}
|
||||
lVar6 = param_1[0x1c5];
|
||||
if (lVar6 == param_1[0x1c6]) {
|
||||
FUN_141edc340(puVar7,lVar6,local_f8);
|
||||
}
|
||||
else {
|
||||
FUN_141068df0(lVar6,local_f8);
|
||||
param_1[0x1c5] = param_1[0x1c5] + 0x60;
|
||||
}
|
||||
lVar6 = param_1[0x1c5];
|
||||
if (lVar6 == param_1[0x1c6]) {
|
||||
FUN_141edc340(puVar7,lVar6,local_f8);
|
||||
}
|
||||
else {
|
||||
FUN_141068df0(lVar6,local_f8);
|
||||
param_1[0x1c5] = param_1[0x1c5] + 0x60;
|
||||
}
|
||||
lVar6 = param_1[0x1c5];
|
||||
if (lVar6 == param_1[0x1c6]) {
|
||||
FUN_141edc340(puVar7,lVar6,local_f8);
|
||||
}
|
||||
else {
|
||||
FUN_141068df0(lVar6,local_f8);
|
||||
param_1[0x1c5] = param_1[0x1c5] + 0x60;
|
||||
}
|
||||
lVar6 = param_1[0x1c5];
|
||||
if (lVar6 == param_1[0x1c6]) {
|
||||
FUN_141edc340(puVar7,lVar6,local_f8);
|
||||
}
|
||||
else {
|
||||
FUN_141068df0(lVar6,local_f8);
|
||||
param_1[0x1c5] = param_1[0x1c5] + 0x60;
|
||||
}
|
||||
lVar6 = param_1[0x1c5];
|
||||
if (lVar6 == param_1[0x1c6]) {
|
||||
FUN_141edc340(puVar7,lVar6,local_f8);
|
||||
}
|
||||
else {
|
||||
FUN_141068df0(lVar6,local_f8);
|
||||
param_1[0x1c5] = param_1[0x1c5] + 0x60;
|
||||
}
|
||||
lVar6 = param_1[0x1c5];
|
||||
if (lVar6 == param_1[0x1c6]) {
|
||||
FUN_141edc340(puVar7,lVar6,local_f8);
|
||||
}
|
||||
else {
|
||||
FUN_141068df0(lVar6,local_f8);
|
||||
param_1[0x1c5] = param_1[0x1c5] + 0x60;
|
||||
}
|
||||
if (param_4 == 0) {
|
||||
uVar3 = FUN_1423e00f0(local_60,0x41,1);
|
||||
local_78 = "@Activate";
|
||||
FUN_1423dfec0(local_70,uVar3);
|
||||
FUN_141f07e80(param_1 + 0x132,local_50,&local_78);
|
||||
FUN_1423dffc0(local_70);
|
||||
FUN_1423dffc0(local_60);
|
||||
puVar4 = (undefined8 *)FUN_14217c644(0x30);
|
||||
puVar7 = puVar8;
|
||||
if (puVar4 != (undefined8 *)0x0) {
|
||||
*puVar4 = djplay::HidExtentFormat::vftable;
|
||||
*(undefined2 *)(puVar4 + 1) = 0xf0;
|
||||
puVar4[2] = 0;
|
||||
puVar4[3] = 0;
|
||||
puVar4[4] = 0;
|
||||
puVar4[5] = 0;
|
||||
puVar7 = puVar4;
|
||||
}
|
||||
puVar4 = (undefined8 *)param_1[0x1be];
|
||||
param_1[0x1be] = puVar7;
|
||||
if (puVar4 != (undefined8 *)0x0) {
|
||||
(**(code **)*puVar4)(puVar4,1);
|
||||
}
|
||||
uVar3 = FUN_1423d5720(&local_res20,"@AuthReq");
|
||||
FUN_141f1be20(param_1,0,0x11,uVar3,puVar1);
|
||||
uVar3 = FUN_1423d5720(local_90,"@AuthChallengeA");
|
||||
FUN_141f1be20(param_1,0,0x12,uVar3,puVar1);
|
||||
uVar3 = FUN_1423d5720(local_88,"@AuthResponseA");
|
||||
FUN_141f1be20(param_1,0,0x13,uVar3,puVar1);
|
||||
uVar3 = FUN_1423d5720(local_80,"@AuthResponseE");
|
||||
FUN_141f1be20(param_1,0,0x14,uVar3,puVar1);
|
||||
uVar3 = FUN_1423d5720(local_98,"@AuthEnd");
|
||||
FUN_141f1be20(param_1,0,0x15,uVar3,puVar1);
|
||||
local_res20 = FUN_14217c644(0x1d0);
|
||||
if (local_res20 != 0) {
|
||||
puVar8 = (undefined8 *)FUN_141f093a0(local_res20,puVar1,0);
|
||||
}
|
||||
param_1[0x1bd] = puVar8;
|
||||
*(undefined4 *)(puVar8 + 0x2f) = 4;
|
||||
if (param_1[0x1bd] != 0) {
|
||||
FUN_141f69980(param_1[0x1bd],param_1 + 0x13a);
|
||||
}
|
||||
}
|
||||
else {
|
||||
param_1[0x1bd] = param_4;
|
||||
*(undefined1 *)((longlong)param_1 + 0xe09) = 1;
|
||||
*(undefined4 *)((longlong)param_1 + 0x4b7c) = param_2;
|
||||
FUN_141f40750(param_1);
|
||||
}
|
||||
FUN_1423d59b0(local_a8);
|
||||
FUN_1423d59b0(local_b8);
|
||||
FUN_1423d59b0(local_c8);
|
||||
FUN_1423d59b0(local_d0);
|
||||
FUN_1423d59b0(local_d8);
|
||||
FUN_1423d59b0(local_e0);
|
||||
FUN_1423d59b0(param_3);
|
||||
return param_1;
|
||||
}
|
||||
|
||||
|
||||
|
||||
//===== FUN_141f15bf0 @ 141f15bf0 =====
|
||||
|
||||
void FUN_141f15bf0(undefined8 *param_1)
|
||||
|
||||
{
|
||||
undefined8 *puVar1;
|
||||
LPCRITICAL_SECTION lpCriticalSection;
|
||||
longlong *plVar2;
|
||||
undefined8 uVar3;
|
||||
int iVar4;
|
||||
|
||||
*param_1 = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
param_1[0x14] = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
param_1[0x15] = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
param_1[0x134] = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
param_1[0x138] = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
param_1[0x139] = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
param_1[0x13a] = djplay::HidMapCDJ2000NXS2::vftable;
|
||||
if (*(char *)((longlong)param_1 + 0xe09) == '\0') {
|
||||
if (param_1[0x1bd] != 0) {
|
||||
FUN_141f72e10(param_1[0x1bd],param_1 + 0x13a);
|
||||
}
|
||||
puVar1 = (undefined8 *)param_1[0x1bd];
|
||||
if (puVar1 != (undefined8 *)0x0) {
|
||||
(**(code **)*puVar1)(puVar1,1);
|
||||
}
|
||||
}
|
||||
uVar3 = FUN_14106db40();
|
||||
FUN_141ed5ce0(uVar3,param_1 + 0x138);
|
||||
uVar3 = FUN_1414518c0();
|
||||
FUN_141ede760(uVar3,param_1 + 0x139);
|
||||
lpCriticalSection = (LPCRITICAL_SECTION)param_1[0x1c7];
|
||||
if (lpCriticalSection != (LPCRITICAL_SECTION)0x0) {
|
||||
DeleteCriticalSection(lpCriticalSection);
|
||||
thunk_FUN_14217cc5c(lpCriticalSection,0x40);
|
||||
}
|
||||
puVar1 = (undefined8 *)param_1[0x59b];
|
||||
if (puVar1 != (undefined8 *)0x0) {
|
||||
(**(code **)*puVar1)(puVar1,1);
|
||||
}
|
||||
FUN_141eea370(param_1 + 0x964);
|
||||
FUN_141eea1a0(param_1 + 0x959);
|
||||
FUN_141eea0a0(param_1 + 0x94d);
|
||||
FUN_141ee9e80(param_1 + 0x941);
|
||||
FUN_141ee9c60(param_1 + 0x936);
|
||||
FUN_141ee9c60(param_1 + 0x92b);
|
||||
FUN_141ee9c60(param_1 + 0x920);
|
||||
FUN_141ee9c60(param_1 + 0x915);
|
||||
FUN_141ee9a70(param_1 + 0x907);
|
||||
FUN_141ee9860(param_1 + 0x8fb);
|
||||
FUN_141ee6340(param_1 + 0x86a);
|
||||
FUN_141ee5ee0(param_1 + 0x7da);
|
||||
FUN_141ee5aa0(param_1 + 0x74a);
|
||||
FUN_141ee54a0(param_1 + 0x6ba);
|
||||
FUN_141ee5120(param_1 + 0x62b);
|
||||
FUN_141ee4ea0(param_1 + 0x59c);
|
||||
FUN_141ee3fe0(param_1 + 0x50d);
|
||||
_eh_vector_destructor_iterator_(param_1 + 0x2d5,0x470,4,FUN_141ee3fe0);
|
||||
FUN_141ee2880(param_1 + 0x1c9);
|
||||
FUN_141edd2a0(param_1 + 0x1c4);
|
||||
iVar4 = 0;
|
||||
if (0 < *(int *)((longlong)param_1 + 0xe04)) {
|
||||
do {
|
||||
FUN_1423dffc0((longlong)iVar4 * 0x10 + param_1[0x1bf]);
|
||||
iVar4 = iVar4 + 1;
|
||||
} while (iVar4 < *(int *)((longlong)param_1 + 0xe04));
|
||||
}
|
||||
*(undefined4 *)((longlong)param_1 + 0xe04) = 0;
|
||||
FUN_14230889c(param_1[0x1bf]);
|
||||
puVar1 = (undefined8 *)param_1[0x1be];
|
||||
if (puVar1 != (undefined8 *)0x0) {
|
||||
(**(code **)*puVar1)(puVar1,1);
|
||||
}
|
||||
plVar2 = (longlong *)param_1[0x1bb];
|
||||
if (plVar2 != (longlong *)0x0) {
|
||||
(**(code **)(*plVar2 + 0x38))(plVar2,1);
|
||||
}
|
||||
puVar1 = (undefined8 *)param_1[0x1a4];
|
||||
if (puVar1 != (undefined8 *)0x0) {
|
||||
(**(code **)*puVar1)(puVar1,1);
|
||||
}
|
||||
puVar1 = (undefined8 *)param_1[0x1a3];
|
||||
if (puVar1 != (undefined8 *)0x0) {
|
||||
(**(code **)*puVar1)(puVar1,1);
|
||||
}
|
||||
param_1[0x13a] = djplay::DeviceAuth::Listener::vftable;
|
||||
param_1[0x139] = djplay::TrackInformationManager::Listener::vftable;
|
||||
param_1[0x138] = djplay::DisplayDataManager::Listener::vftable;
|
||||
FUN_14246e800(param_1 + 0x134);
|
||||
FUN_141f15b10(param_1);
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
|
||||
//===== FUN_141f63210 @ 141f63210 =====
|
||||
|
||||
void FUN_141f63210(longlong param_1,undefined8 *param_2,undefined4 *param_3,undefined8 param_4,
|
||||
undefined1 param_5)
|
||||
|
||||
{
|
||||
char cVar1;
|
||||
undefined8 uVar2;
|
||||
undefined8 local_18 [2];
|
||||
|
||||
switch(param_5) {
|
||||
case 0x21:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0xe48) + 0x10))(param_1 + 0xe48);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0xe48) + 8))(param_1 + 0xe48,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x22:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x16a8) + 0x10))(param_1 + 0x16a8);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x16a8) + 8))(param_1 + 0x16a8,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x23:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x1b18) + 0x10))(param_1 + 0x1b18);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x1b18) + 8))(param_1 + 0x1b18,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x24:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x1f88) + 0x10))(param_1 + 0x1f88);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x1f88) + 8))(param_1 + 0x1f88,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x25:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x23f8) + 0x10))(param_1 + 0x23f8);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x23f8) + 8))(param_1 + 0x23f8,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x26:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x2868) + 0x10))(param_1 + 0x2868);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x2868) + 8))(param_1 + 0x2868,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x2a:
|
||||
if ((*(longlong **)(param_1 + 0x2cd8) != (longlong *)0x0) &&
|
||||
(cVar1 = (**(code **)(**(longlong **)(param_1 + 0x2cd8) + 0x10))(), cVar1 != '\0')) {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(**(longlong **)(param_1 + 0x2cd8) + 8))
|
||||
(*(longlong **)(param_1 + 0x2cd8),local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x2b:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x2ce0) + 0x10))(param_1 + 0x2ce0);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x2ce0) + 8))(param_1 + 0x2ce0,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x2c:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x3158) + 0x10))(param_1 + 0x3158);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x3158) + 8))(param_1 + 0x3158,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x2d:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x35d0) + 0x10))(param_1 + 0x35d0);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x35d0) + 8))(param_1 + 0x35d0,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x2e:
|
||||
if ((*(char *)(param_1 + 0xe0c) == '\0') &&
|
||||
(cVar1 = (**(code **)(*(longlong *)(param_1 + 0x3a50) + 0x10))(param_1 + 0x3a50),
|
||||
cVar1 != '\0')) {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x3a50) + 8))(param_1 + 0x3a50,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x2f:
|
||||
if ((*(char *)(param_1 + 0xe0c) == '\0') &&
|
||||
(cVar1 = (**(code **)(*(longlong *)(param_1 + 0x3ed0) + 0x10))(param_1 + 0x3ed0),
|
||||
cVar1 != '\0')) {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x3ed0) + 8))(param_1 + 0x3ed0,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x30:
|
||||
if ((*(char *)(param_1 + 0xe0c) == '\0') &&
|
||||
(cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4350) + 0x10))(param_1 + 0x4350),
|
||||
cVar1 != '\0')) {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4350) + 8))(param_1 + 0x4350,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x40:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x47d8) + 0x10))(param_1 + 0x47d8);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x47d8) + 8))(param_1 + 0x47d8,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x41:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4838) + 0x10))(param_1 + 0x4838);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4838) + 8))(param_1 + 0x4838,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x42:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x48a8) + 0x10))(param_1 + 0x48a8);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x48a8) + 8))(param_1 + 0x48a8,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x43:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4900) + 0x10))(param_1 + 0x4900);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4900) + 8))(param_1 + 0x4900,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x44:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4958) + 0x10))(param_1 + 0x4958);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4958) + 8))(param_1 + 0x4958,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x45:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x49b0) + 0x10))(param_1 + 0x49b0);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x49b0) + 8))(param_1 + 0x49b0,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x46:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4a08) + 0x10))(param_1 + 0x4a08);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4a08) + 8))(param_1 + 0x4a08,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x47:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4a68) + 0x10))(param_1 + 0x4a68);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4a68) + 8))(param_1 + 0x4a68,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x48:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4ac8) + 0x10))(param_1 + 0x4ac8);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4ac8) + 8))(param_1 + 0x4ac8,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
break;
|
||||
case 0x4f:
|
||||
cVar1 = (**(code **)(*(longlong *)(param_1 + 0x4b20) + 0x10))(param_1 + 0x4b20);
|
||||
if (cVar1 != '\0') {
|
||||
local_18[0] = 0;
|
||||
uVar2 = (**(code **)(*(longlong *)(param_1 + 0x4b20) + 8))(param_1 + 0x4b20,local_18,param_4);
|
||||
*param_2 = uVar2;
|
||||
goto LAB_141f63871;
|
||||
}
|
||||
}
|
||||
local_18[0]._0_4_ = 0;
|
||||
*param_2 = 0;
|
||||
LAB_141f63871:
|
||||
*param_3 = (undefined4)local_18[0];
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,85 @@
|
||||
// Walk the call tree down from the export orchestrator and surface the page
|
||||
// serializer: flag functions that call file-I/O APIs or use the 4096 page size,
|
||||
// then decompile those candidates. Output -> ~/ghidra_proj/calltree.txt
|
||||
import java.io.PrintWriter;
|
||||
import java.util.*;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.app.decompiler.*;
|
||||
import ghidra.program.model.listing.*;
|
||||
import ghidra.program.model.scalar.Scalar;
|
||||
|
||||
public class CallTree extends GhidraScript {
|
||||
static final String ROOT = "0x14119d2d0";
|
||||
static final int MAX_DEPTH = 4;
|
||||
|
||||
boolean hasPageConst(Function f) {
|
||||
InstructionIterator it = currentProgram.getListing().getInstructions(f.getBody(), true);
|
||||
while (it.hasNext()) {
|
||||
Instruction insn = it.next();
|
||||
for (int op = 0; op < insn.getNumOperands(); op++)
|
||||
for (Object o : insn.getOpObjects(op))
|
||||
if (o instanceof Scalar) {
|
||||
long v = ((Scalar) o).getUnsignedValue();
|
||||
if (v == 0x1000L || v == 4096L) return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
String ioCalls(Function f) {
|
||||
StringBuilder sb = new StringBuilder();
|
||||
for (Function c : f.getCalledFunctions(monitor)) {
|
||||
String n = c.getName().toLowerCase();
|
||||
if (n.contains("write") || n.contains("createfile") || n.contains("fopen")
|
||||
|| n.contains("fwrite") || n.contains("mapviewof") || n.contains("setfilepointer")
|
||||
|| n.contains("flushview") || n.contains("fclose"))
|
||||
sb.append(c.getName()).append(" ");
|
||||
}
|
||||
return sb.toString();
|
||||
}
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
PrintWriter out = new PrintWriter("/home/ntr/ghidra_proj/calltree.txt");
|
||||
FunctionManager fm = currentProgram.getFunctionManager();
|
||||
Function root = fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(ROOT));
|
||||
|
||||
Map<Function, Integer> depth = new LinkedHashMap<>();
|
||||
Deque<Function> q = new ArrayDeque<>();
|
||||
depth.put(root, 0); q.add(root);
|
||||
while (!q.isEmpty() && !monitor.isCancelled()) {
|
||||
Function f = q.poll();
|
||||
int d = depth.get(f);
|
||||
if (d >= MAX_DEPTH) continue;
|
||||
for (Function c : f.getCalledFunctions(monitor))
|
||||
if (!depth.containsKey(c)) { depth.put(c, d + 1); q.add(c); }
|
||||
}
|
||||
out.println("call tree from " + root.getName() + " depth<=" + MAX_DEPTH + ": " + depth.size() + " funcs\n");
|
||||
|
||||
List<Function> candidates = new ArrayList<>();
|
||||
for (Map.Entry<Function, Integer> e : depth.entrySet()) {
|
||||
Function f = e.getKey();
|
||||
boolean page = hasPageConst(f);
|
||||
String io = ioCalls(f);
|
||||
String flags = (page ? "[4096]" : "") + (io.isEmpty() ? "" : "[io:" + io.trim() + "]");
|
||||
out.println(String.format("d%d %-12s size=%-6d %s %s",
|
||||
e.getValue(), f.getEntryPoint(), f.getBody().getNumAddresses(), f.getName(), flags));
|
||||
if (page || !io.isEmpty()) candidates.add(f);
|
||||
}
|
||||
|
||||
DecompInterface di = new DecompInterface();
|
||||
di.openProgram(currentProgram);
|
||||
out.println("\n===== CANDIDATE DECOMPILES (" + candidates.size() + ") =====");
|
||||
for (Function f : candidates) {
|
||||
if (monitor.isCancelled()) break;
|
||||
out.println("\n//===== " + f.getName() + " @ " + f.getEntryPoint() + " =====");
|
||||
DecompileResults r = di.decompileFunction(f, 60, monitor);
|
||||
if (r != null && r.decompileCompleted())
|
||||
out.println(r.getDecompiledFunction().getC());
|
||||
else
|
||||
out.println("// decompile failed");
|
||||
}
|
||||
out.flush(); out.close();
|
||||
println("wrote calltree.txt: " + depth.size() + " funcs, " + candidates.size() + " candidates");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
// Decompile the pdb export functions found via string xrefs.
|
||||
import java.io.PrintWriter;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.app.decompiler.*;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.listing.Function;
|
||||
|
||||
public class DecPdb extends GhidraScript {
|
||||
// export.pdb serializer call stack from a live Frida WriteFile backtrace.
|
||||
// module+offset -> Ghidra static = 0x140000000 + offset (base confirmed).
|
||||
static final String[] ADDRS = {
|
||||
"0x1420e883e", // calls WriteFile
|
||||
"0x1420d73a6",
|
||||
"0x1420d6e92",
|
||||
"0x1420d6368",
|
||||
"0x1420eae07",
|
||||
"0x1420e0852",
|
||||
"0x1420de1c4",
|
||||
"0x141d75d21", // export.pdb-specific frame
|
||||
"0x141cad188",
|
||||
"0x141cac585",
|
||||
"0x141b8ba75",
|
||||
"0x141acaf8c",
|
||||
"0x1423c9b3b",
|
||||
};
|
||||
@Override public void run() throws Exception {
|
||||
PrintWriter out = new PrintWriter("/home/ntr/ghidra_proj/decpdb.txt");
|
||||
DecompInterface di = new DecompInterface();
|
||||
di.openProgram(currentProgram);
|
||||
for (String a : ADDRS) {
|
||||
Address addr = currentProgram.getAddressFactory().getAddress(a);
|
||||
Function f = currentProgram.getFunctionManager().getFunctionContaining(addr);
|
||||
if (f == null) { out.println("// no function at " + a); continue; }
|
||||
out.println("\n//================ " + f.getName() + " @ " + f.getEntryPoint() + " ================");
|
||||
DecompileResults r = di.decompileFunction(f, 90, monitor);
|
||||
if (r != null && r.decompileCompleted())
|
||||
out.println(r.getDecompiledFunction().getC());
|
||||
else
|
||||
out.println("// decompile failed: " + (r != null ? r.getErrorMessage() : "null"));
|
||||
}
|
||||
out.flush(); out.close();
|
||||
println("wrote decpdb.txt");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
// Decompile the export root FUN_14119d2d0 and selected callees, to find the
|
||||
// per-track row-insert that sets index_shift (0x02) and the 0x14/0x18/0x1a blob.
|
||||
// Also dump callees of FUN_14117be20 (rekordbox/export.pdb string xref).
|
||||
// Output -> ~/ghidra_proj/decroot.txt
|
||||
import java.io.PrintWriter;
|
||||
import java.util.*;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.app.decompiler.*;
|
||||
import ghidra.program.model.listing.*;
|
||||
|
||||
public class DecRoot extends GhidraScript {
|
||||
PrintWriter out; DecompInterface di; FunctionManager fm;
|
||||
Set<String> done = new HashSet<>();
|
||||
void dec(String a, String why) {
|
||||
Function f = fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(a));
|
||||
if (f == null) { out.println("// no fn at " + a + " (" + why + ")"); return; }
|
||||
if (!done.add(f.getEntryPoint().toString())) return;
|
||||
out.println("\n//===== " + f.getName() + " @ " + f.getEntryPoint() + " [" + why + "] =====");
|
||||
DecompileResults r = di.decompileFunction(f, 120, monitor);
|
||||
out.println(r != null && r.decompileCompleted() ? r.getDecompiledFunction().getC() : "// decompile failed: "+(r!=null?r.getErrorMessage():"null"));
|
||||
}
|
||||
@Override public void run() throws Exception {
|
||||
out = new PrintWriter("/home/ntr/ghidra_proj/decroot.txt");
|
||||
di = new DecompInterface(); di.openProgram(currentProgram);
|
||||
fm = currentProgram.getFunctionManager();
|
||||
dec("0x14119d2d0", "export root (/.PIONEER/rekordbox/export.pdb)");
|
||||
dec("0x14117be20", "rekordbox/export.pdb xref");
|
||||
out.flush(); out.close();
|
||||
println("wrote decroot.txt");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
// Decode the track-row builder: the function that maps a djmdContent record into
|
||||
// the on-disk DeviceSQL row (offset 0x02 index_shift, 0x14 u4 hash, 0x18/0x1a u2).
|
||||
// Decompiles FUN_141c04be0 (AnalysisDataPath/SearchStr/MasterContentID mapper) and
|
||||
// FUN_141ae4860 (djmdContent), plus scans for code referencing the constant
|
||||
// 0x21b4a7ab / 0xa7ab and for `<<5` row-index assignment.
|
||||
// Output -> ~/ghidra_proj/decrow.txt
|
||||
import java.io.PrintWriter;
|
||||
import java.util.*;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.app.decompiler.*;
|
||||
import ghidra.program.model.address.*;
|
||||
import ghidra.program.model.listing.*;
|
||||
import ghidra.program.model.scalar.Scalar;
|
||||
import ghidra.program.model.lang.OperandType;
|
||||
|
||||
public class DecRow extends GhidraScript {
|
||||
PrintWriter out;
|
||||
DecompInterface di;
|
||||
Set<String> done = new HashSet<>();
|
||||
FunctionManager fm;
|
||||
|
||||
void dec(String a, String why) {
|
||||
Function f = fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(a));
|
||||
if (f == null) { out.println("// no function at " + a + " (" + why + ")"); return; }
|
||||
if (!done.add(f.getEntryPoint().toString())) return;
|
||||
out.println("\n//===== " + f.getName() + " @ " + f.getEntryPoint() + " [" + why + "] =====");
|
||||
DecompileResults r = di.decompileFunction(f, 90, monitor);
|
||||
out.println(r != null && r.decompileCompleted() ? r.getDecompiledFunction().getC() : "// decompile failed");
|
||||
}
|
||||
|
||||
@Override public void run() throws Exception {
|
||||
out = new PrintWriter("/home/ntr/ghidra_proj/decrow.txt");
|
||||
di = new DecompInterface(); di.openProgram(currentProgram);
|
||||
fm = currentProgram.getFunctionManager();
|
||||
|
||||
dec("0x141c04be0", "AnalysisDataPath/SearchStr/MasterContentID mapper");
|
||||
dec("0x141c07230", "SearchStr 2");
|
||||
dec("0x141c076d0", "SearchStr 3");
|
||||
dec("0x141ae4860", "djmdContent xrefs");
|
||||
|
||||
// Scan all instructions for the per-export constants used in 0x18/0x1a
|
||||
// (0xa7ab=42923, 0x21b4=8628, combined u4 0x21b4a7ab=565353387) and for
|
||||
// an immediate 0x20 / shift-by-5 pattern that would build index_shift.
|
||||
long[] consts = {0xa7abL, 0x21b4L, 0x21b4a7abL, 0xa7ab21b4L};
|
||||
Set<String> hitFns = new LinkedHashSet<>();
|
||||
Listing lst = currentProgram.getListing();
|
||||
InstructionIterator it = lst.getInstructions(true);
|
||||
int scanned = 0;
|
||||
while (it.hasNext() && !monitor.isCancelled()) {
|
||||
Instruction ins = it.next();
|
||||
scanned++;
|
||||
for (int op = 0; op < ins.getNumOperands(); op++) {
|
||||
Object[] objs = ins.getOpObjects(op);
|
||||
for (Object o : objs) {
|
||||
if (o instanceof Scalar) {
|
||||
long v = ((Scalar) o).getUnsignedValue();
|
||||
for (long c : consts) {
|
||||
if (v == c) {
|
||||
Function f = fm.getFunctionContaining(ins.getAddress());
|
||||
String tag = "const 0x" + Long.toHexString(c) + " @" + ins.getAddress()
|
||||
+ " in " + (f != null ? f.getName() + "@" + f.getEntryPoint() : "?")
|
||||
+ " : " + ins.toString();
|
||||
out.println("// SCAN " + tag);
|
||||
if (f != null) hitFns.add(f.getEntryPoint().toString() + "|const0x" + Long.toHexString(c));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
out.println("// scanned " + scanned + " instructions");
|
||||
for (String h : hitFns) {
|
||||
String[] parts = h.split("\\|");
|
||||
dec("0x" + parts[0], parts[1]);
|
||||
}
|
||||
out.flush(); out.close();
|
||||
println("wrote decrow.txt");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
// Decompile the prime index_shift candidate FUN_14232a86c (shl5 + mask 0x7ff) and
|
||||
// a few neighboring DeviceSQL page-manager functions. Output -> ~/ghidra_proj/decshift.txt
|
||||
import java.io.PrintWriter;
|
||||
import java.util.*;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.app.decompiler.*;
|
||||
import ghidra.program.model.listing.*;
|
||||
|
||||
public class DecShift extends GhidraScript {
|
||||
PrintWriter out; DecompInterface di; FunctionManager fm; Set<String> done=new HashSet<>();
|
||||
void dec(String a, String why) {
|
||||
Function f = fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(a));
|
||||
if (f == null) { out.println("// no fn at " + a + " (" + why + ")"); return; }
|
||||
if (!done.add(f.getEntryPoint().toString())) return;
|
||||
out.println("\n//===== " + f.getName() + " @ " + f.getEntryPoint() + " [" + why + "] =====");
|
||||
DecompileResults r = di.decompileFunction(f, 120, monitor);
|
||||
out.println(r != null && r.decompileCompleted() ? r.getDecompiledFunction().getC() : "// decompile failed");
|
||||
}
|
||||
@Override public void run() throws Exception {
|
||||
out = new PrintWriter("/home/ntr/ghidra_proj/decshift.txt");
|
||||
di = new DecompInterface(); di.openProgram(currentProgram);
|
||||
fm = currentProgram.getFunctionManager();
|
||||
dec("0x14232a86c", "shl5+mask7ff PRIME index_shift candidate");
|
||||
// neighbors in DeviceSQL page-manager range
|
||||
dec("0x1423e1550", "page-mgr neighbor");
|
||||
dec("0x1423e12a0", "page-mgr neighbor");
|
||||
out.flush(); out.close();
|
||||
println("wrote decshift.txt");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
// @flesniak documented track-row 0x18=u2 "always 19048" (0x4A68) and 0x1a=u2
|
||||
// "always 30967" (0x78F7). Our AceMo export has 0xa7ab / 0x21b4 instead, and a
|
||||
// const-scan found no 0xa7ab/0x21b4 immediates. So locate every reference to the
|
||||
// documented defaults 0x4A68 and 0x78F7 (and combined 0x78F74A68 / 0x4A6878F7),
|
||||
// decompile the containing functions, and report. This should reveal how 0x18/0x1a
|
||||
// are produced (literal default vs computed). Output -> ~/ghidra_proj/find4a68.txt
|
||||
import java.io.PrintWriter;
|
||||
import java.util.*;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.app.decompiler.*;
|
||||
import ghidra.program.model.listing.*;
|
||||
import ghidra.program.model.scalar.Scalar;
|
||||
|
||||
public class Find4A68 extends GhidraScript {
|
||||
@Override public void run() throws Exception {
|
||||
PrintWriter out = new PrintWriter("/home/ntr/ghidra_proj/find4a68.txt");
|
||||
FunctionManager fm = currentProgram.getFunctionManager();
|
||||
DecompInterface di = new DecompInterface(); di.openProgram(currentProgram);
|
||||
long[] consts = {0x4A68L, 0x78F7L, 0x78F74A68L, 0x4A6878F7L, 19048L, 30967L};
|
||||
Set<String> fns = new LinkedHashSet<>();
|
||||
InstructionIterator it = currentProgram.getListing().getInstructions(true);
|
||||
while (it.hasNext() && !monitor.isCancelled()) {
|
||||
Instruction ins = it.next();
|
||||
for (int op=0; op<ins.getNumOperands(); op++)
|
||||
for (Object o : ins.getOpObjects(op))
|
||||
if (o instanceof Scalar) {
|
||||
long v=((Scalar)o).getUnsignedValue();
|
||||
for (long c: consts) if (v==c) {
|
||||
Function f=fm.getFunctionContaining(ins.getAddress());
|
||||
out.println("// HIT 0x"+Long.toHexString(c)+" @"+ins.getAddress()
|
||||
+" in "+(f!=null?f.getName()+"@"+f.getEntryPoint():"?")+" : "+ins.toString());
|
||||
if (f!=null) fns.add(f.getEntryPoint().toString());
|
||||
}
|
||||
}
|
||||
}
|
||||
// also search defined data for the byte patterns
|
||||
out.println("// --- decompiling containing functions ---");
|
||||
for (String a: fns) {
|
||||
Function f=fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(a));
|
||||
out.println("\n//===== "+f.getName()+" @ "+f.getEntryPoint()+" =====");
|
||||
DecompileResults r=di.decompileFunction(f,90,monitor);
|
||||
out.println(r!=null&&r.decompileCompleted()?r.getDecompiledFunction().getC():"// decompile failed");
|
||||
}
|
||||
out.flush(); out.close();
|
||||
println("wrote find4a68.txt");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
// Anchor the track-row builder via the track-row bitmask 0xc0700 (constant in
|
||||
// every AceMo track row at offset 0x04) and the subtype/0x24 nearby. Find any
|
||||
// instruction with immediate 0xc0700 and decompile the function. Output ->
|
||||
// ~/ghidra_proj/findbitmask.txt
|
||||
import java.io.PrintWriter;
|
||||
import java.util.*;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.app.decompiler.*;
|
||||
import ghidra.program.model.listing.*;
|
||||
import ghidra.program.model.scalar.Scalar;
|
||||
|
||||
public class FindBitmask extends GhidraScript {
|
||||
@Override public void run() throws Exception {
|
||||
PrintWriter out = new PrintWriter("/home/ntr/ghidra_proj/findbitmask.txt");
|
||||
FunctionManager fm = currentProgram.getFunctionManager();
|
||||
DecompInterface di = new DecompInterface(); di.openProgram(currentProgram);
|
||||
long[] consts = {0xc0700L, 0x700L, 0xc07L};
|
||||
Set<String> fns = new LinkedHashSet<>();
|
||||
InstructionIterator it = currentProgram.getListing().getInstructions(true);
|
||||
while (it.hasNext() && !monitor.isCancelled()) {
|
||||
Instruction ins = it.next();
|
||||
for (int op=0; op<ins.getNumOperands(); op++)
|
||||
for (Object o : ins.getOpObjects(op))
|
||||
if (o instanceof Scalar) {
|
||||
long v=((Scalar)o).getUnsignedValue();
|
||||
if (v==0xc0700L) {
|
||||
Function f=fm.getFunctionContaining(ins.getAddress());
|
||||
out.println("// HIT 0xc0700 @"+ins.getAddress()+" in "
|
||||
+(f!=null?f.getName()+"@"+f.getEntryPoint():"?")+" : "+ins.toString());
|
||||
if (f!=null) fns.add(f.getEntryPoint().toString());
|
||||
}
|
||||
}
|
||||
}
|
||||
out.println("// --- decompiling ---");
|
||||
for (String a: fns) {
|
||||
Function f=fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(a));
|
||||
out.println("\n//===== "+f.getName()+" @ "+f.getEntryPoint()+" =====");
|
||||
DecompileResults r=di.decompileFunction(f,120,monitor);
|
||||
out.println(r!=null&&r.decompileCompleted()?r.getDecompiledFunction().getC():"// fail");
|
||||
}
|
||||
out.flush(); out.close();
|
||||
println("wrote findbitmask.txt");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
// Confirm 0x78f74a68 / 0x4a68 / 0x78f7 is rekordbox's default opaque_seed by
|
||||
// searching DEFINED DATA (not just instruction immediates) for the byte pattern
|
||||
// 68 4a f7 78 (LE u4) and the two u2s, and report containing context.
|
||||
// Output -> ~/ghidra_proj/finddefault.txt
|
||||
import java.io.PrintWriter;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.model.address.*;
|
||||
import ghidra.program.model.mem.*;
|
||||
|
||||
public class FindDefault extends GhidraScript {
|
||||
@Override public void run() throws Exception {
|
||||
PrintWriter out = new PrintWriter("/home/ntr/ghidra_proj/finddefault.txt");
|
||||
Memory mem = currentProgram.getMemory();
|
||||
byte[] pat = { (byte)0x68,(byte)0x4a,(byte)0xf7,(byte)0x78 }; // 0x78f74a68 LE
|
||||
Address a = mem.getMinAddress();
|
||||
int found=0;
|
||||
while (a != null && found < 50) {
|
||||
Address hit = mem.findBytes(a, pat, null, true, monitor);
|
||||
if (hit == null) break;
|
||||
out.println("// 0x78f74a68 bytes @ " + hit);
|
||||
found++;
|
||||
a = hit.add(1);
|
||||
}
|
||||
out.println("// total occurrences of 0x78f74a68 LE in memory image: " + found);
|
||||
out.flush(); out.close();
|
||||
println("wrote finddefault.txt found="+found);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
// Hunt for the track-row builder / the per-track 8-byte hash (row offsets
|
||||
// 0x14 u4, 0x18 u2, 0x1a u2) and index_shift (0x02). Decompiles the string-hash
|
||||
// function formatForSearch + its callers, and dumps xrefs of track/content/
|
||||
// search-related strings. Output -> ~/ghidra_proj/hash.txt
|
||||
import java.io.PrintWriter;
|
||||
import java.util.*;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.app.decompiler.*;
|
||||
import ghidra.program.model.listing.*;
|
||||
import ghidra.program.model.symbol.*;
|
||||
|
||||
public class FindHash extends GhidraScript {
|
||||
PrintWriter out;
|
||||
DecompInterface di;
|
||||
Set<String> done = new HashSet<>();
|
||||
FunctionManager fm;
|
||||
ReferenceManager rm;
|
||||
|
||||
void dec(Function f, String why) {
|
||||
if (f == null) return;
|
||||
if (!done.add(f.getEntryPoint().toString())) return;
|
||||
out.println("\n//===== " + f.getName() + " @ " + f.getEntryPoint() + " [" + why + "] =====");
|
||||
DecompileResults r = di.decompileFunction(f, 60, monitor);
|
||||
out.println(r != null && r.decompileCompleted() ? r.getDecompiledFunction().getC() : "// decompile failed");
|
||||
}
|
||||
|
||||
Function fn(String a) {
|
||||
return fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(a));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
out = new PrintWriter("/home/ntr/ghidra_proj/hash.txt");
|
||||
di = new DecompInterface(); di.openProgram(currentProgram);
|
||||
fm = currentProgram.getFunctionManager();
|
||||
rm = currentProgram.getReferenceManager();
|
||||
|
||||
// 1. formatForSearch (string hash) + its callers
|
||||
Function fmt = fn("0x141a1afc0");
|
||||
dec(fmt, "formatForSearch");
|
||||
if (fmt != null) {
|
||||
int n = 0;
|
||||
for (Reference ref : rm.getReferencesTo(fmt.getEntryPoint())) {
|
||||
if (n++ >= 10) break;
|
||||
dec(fm.getFunctionContaining(ref.getFromAddress()), "caller of formatForSearch");
|
||||
}
|
||||
}
|
||||
|
||||
// 2. strings that hint at the track/content row builder -> xref functions
|
||||
String[] kw = {"djmdContent", "SearchStr", "formatForSearch", "AnalysisDataPath",
|
||||
"ContentID", "rb_data_status", "rb_local", "dsqlInsert", "AddTrack"};
|
||||
DataIterator dit = currentProgram.getListing().getDefinedData(true);
|
||||
int shown = 0;
|
||||
while (dit.hasNext() && !monitor.isCancelled()) {
|
||||
Data d = dit.next();
|
||||
if (d == null || !d.hasStringValue()) continue;
|
||||
Object v = d.getValue();
|
||||
if (!(v instanceof String)) continue;
|
||||
String s = (String) v;
|
||||
for (String k : kw) {
|
||||
if (s.contains(k)) {
|
||||
out.println("\n// STRING @" + d.getAddress() + " : " + s);
|
||||
int n = 0;
|
||||
for (Reference ref : rm.getReferencesTo(d.getAddress())) {
|
||||
if (n++ >= 5) break;
|
||||
Function c = fm.getFunctionContaining(ref.getFromAddress());
|
||||
out.println("// xref in " + (c != null ? c.getName() + " @" + c.getEntryPoint() : ref.getFromAddress()));
|
||||
}
|
||||
shown++;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (shown >= 30) break;
|
||||
}
|
||||
out.flush(); out.close();
|
||||
println("wrote hash.txt");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
// Locate the rekordbox .pdb serializer via known strings + magic-number constants.
|
||||
// Run headless: analyzeHeadless <proj> rbx -process rekordbox.exe -noanalysis \
|
||||
// -scriptPath ~/ghidra_scripts -postScript FindPdb.java
|
||||
// Output goes to ~/ghidra_proj/findpdb.txt (and the log).
|
||||
import java.io.PrintWriter;
|
||||
import java.util.*;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.listing.*;
|
||||
import ghidra.program.model.scalar.Scalar;
|
||||
import ghidra.program.model.symbol.*;
|
||||
|
||||
public class FindPdb extends GhidraScript {
|
||||
|
||||
// Strings that mark the export code paths.
|
||||
static final String[] KEYWORDS = {
|
||||
"export.pdb", "exportExt.pdb", "USBANLZ", "PIONEER",
|
||||
"ANLZ", "HOT CUE BANK", "ORIGINAL ARTIST", "rekordbox/export"
|
||||
};
|
||||
|
||||
// Magic numbers maple currently guesses (various widths/combinations).
|
||||
static final long[] MAGICS = {
|
||||
42343L, 54995L, // 0xA567 / 0xD6D3 per-track markers
|
||||
0xD6D3A567L, 0xA567D6D3L, // combined 32-bit forms
|
||||
788224L, // 0x000C0700 track bitmask
|
||||
1004L, // 0x03EC "strange page" marker
|
||||
0x64L, 0x44L, 0x24L, // page-flag bytes
|
||||
19048L, 30967L // the schema author's original guesses
|
||||
};
|
||||
|
||||
PrintWriter out;
|
||||
|
||||
void emit(String s) { println(s); if (out != null) out.println(s); }
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
out = new PrintWriter("/home/ntr/ghidra_proj/findpdb.txt");
|
||||
Listing listing = currentProgram.getListing();
|
||||
ReferenceManager rm = currentProgram.getReferenceManager();
|
||||
FunctionManager fm = currentProgram.getFunctionManager();
|
||||
|
||||
emit("===== STRING XREFS =====");
|
||||
DataIterator di = listing.getDefinedData(true);
|
||||
while (di.hasNext() && !monitor.isCancelled()) {
|
||||
Data d = di.next();
|
||||
if (d == null || !d.hasStringValue()) continue;
|
||||
Object val = d.getValue();
|
||||
if (!(val instanceof String)) continue;
|
||||
String s = (String) val;
|
||||
for (String kw : KEYWORDS) {
|
||||
if (s.contains(kw)) {
|
||||
emit("\nSTRING @ " + d.getAddress() + " : " + s.replace("\n", "\\n"));
|
||||
ReferenceIterator ri = rm.getReferencesTo(d.getAddress());
|
||||
int n = 0;
|
||||
while (ri.hasNext() && n++ < 25) {
|
||||
Reference r = ri.next();
|
||||
Function f = fm.getFunctionContaining(r.getFromAddress());
|
||||
emit(" xref from " + r.getFromAddress()
|
||||
+ (f != null ? " in " + f.getName() + " @ " + f.getEntryPoint() : " (no func)"));
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
emit("\n===== CONSTANT XREFS =====");
|
||||
Set<Long> targets = new HashSet<>();
|
||||
for (long m : MAGICS) targets.add(m);
|
||||
Map<Long, Integer> counts = new TreeMap<>();
|
||||
InstructionIterator ii = listing.getInstructions(true);
|
||||
while (ii.hasNext() && !monitor.isCancelled()) {
|
||||
Instruction insn = ii.next();
|
||||
for (int op = 0; op < insn.getNumOperands(); op++) {
|
||||
for (Object o : insn.getOpObjects(op)) {
|
||||
if (o instanceof Scalar) {
|
||||
long v = ((Scalar) o).getUnsignedValue();
|
||||
if (targets.contains(v)) {
|
||||
Function f = fm.getFunctionContaining(insn.getAddress());
|
||||
emit(String.format("CONST 0x%X (%d) @ %s %s in %s", v, v,
|
||||
insn.getAddress(), insn.toString(),
|
||||
f != null ? f.getName() + " @ " + f.getEntryPoint() : "(no func)"));
|
||||
counts.merge(v, 1, Integer::sum);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
emit("\n===== CONSTANT HIT COUNTS =====");
|
||||
for (Map.Entry<Long, Integer> e : counts.entrySet())
|
||||
emit(String.format("0x%X (%d): %d hits", e.getKey(), e.getKey(), e.getValue()));
|
||||
|
||||
out.flush();
|
||||
out.close();
|
||||
emit("\nwrote /home/ntr/ghidra_proj/findpdb.txt");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
// Find the DeviceSQL row-index code that computes index_shift = slot << 5.
|
||||
// Heuristic: functions that contain BOTH a left-shift-by-5 (SHL r,5 / LEA r,[r*0x20])
|
||||
// AND reference the row-index masks (0x1fff / 0x7ff) or stride 0x24, AND a u2 store.
|
||||
// Output -> ~/ghidra_proj/findshift.txt
|
||||
import java.io.PrintWriter;
|
||||
import java.util.*;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.model.listing.*;
|
||||
import ghidra.program.model.scalar.Scalar;
|
||||
|
||||
public class FindShift extends GhidraScript {
|
||||
@Override public void run() throws Exception {
|
||||
PrintWriter out = new PrintWriter("/home/ntr/ghidra_proj/findshift.txt");
|
||||
FunctionManager fm = currentProgram.getFunctionManager();
|
||||
Listing lst = currentProgram.getListing();
|
||||
// accumulate per-function evidence
|
||||
Map<String,Set<String>> ev = new HashMap<>();
|
||||
InstructionIterator it = lst.getInstructions(true);
|
||||
while (it.hasNext() && !monitor.isCancelled()) {
|
||||
Instruction ins = it.next();
|
||||
Function f = fm.getFunctionContaining(ins.getAddress());
|
||||
if (f == null) continue;
|
||||
String key = f.getEntryPoint().toString();
|
||||
String mn = ins.getMnemonicString();
|
||||
String txt = ins.toString();
|
||||
boolean shl5 = (mn.equalsIgnoreCase("SHL") && txt.endsWith(",0x5"))
|
||||
|| (mn.equalsIgnoreCase("LEA") && txt.contains("*0x20"));
|
||||
if (shl5) ev.computeIfAbsent(key,k->new HashSet<>()).add("shl5@"+ins.getAddress());
|
||||
for (int op=0; op<ins.getNumOperands(); op++)
|
||||
for (Object o : ins.getOpObjects(op))
|
||||
if (o instanceof Scalar) {
|
||||
long v = ((Scalar)o).getUnsignedValue();
|
||||
if (v==0x1fff) ev.computeIfAbsent(key,k->new HashSet<>()).add("mask1fff@"+ins.getAddress());
|
||||
if (v==0x7ff) ev.computeIfAbsent(key,k->new HashSet<>()).add("mask7ff@"+ins.getAddress());
|
||||
if (v==0x24) ev.computeIfAbsent(key,k->new HashSet<>()).add("stride24@"+ins.getAddress());
|
||||
}
|
||||
}
|
||||
// report functions that have shl5 AND (mask or stride)
|
||||
for (Map.Entry<String,Set<String>> e : ev.entrySet()) {
|
||||
Set<String> tags = e.getValue();
|
||||
boolean hasShl = tags.stream().anyMatch(t->t.startsWith("shl5"));
|
||||
boolean hasCtx = tags.stream().anyMatch(t->t.startsWith("mask1fff")||t.startsWith("mask7ff")||t.startsWith("stride24"));
|
||||
if (hasShl && hasCtx) {
|
||||
Function f = fm.getFunctionContaining(currentProgram.getAddressFactory().getAddress(e.getKey()));
|
||||
out.println("CANDIDATE " + (f!=null?f.getName():"?") + " @" + e.getKey() + " : " + tags);
|
||||
}
|
||||
}
|
||||
out.flush(); out.close();
|
||||
println("wrote findshift.txt");
|
||||
}
|
||||
}
|
||||
Whitespace-only changes.
@@ -0,0 +1,673 @@
|
||||
meta:
|
||||
id: rekordbox_anlz
|
||||
title: rekordbox track analysis file
|
||||
application: rekordbox
|
||||
file-extension:
|
||||
- dat
|
||||
- ext
|
||||
license: EPL-1.0
|
||||
endian: be
|
||||
|
||||
doc: |
|
||||
These files are created by rekordbox when analyzing audio tracks
|
||||
to facilitate DJ performance. They include waveforms, beat grids
|
||||
(information about the precise time at which each beat occurs),
|
||||
time indices to allow efficient seeking to specific positions
|
||||
inside variable bit-rate audio streams, and lists of memory cues
|
||||
and loop points. They are used by Pioneer professional DJ
|
||||
equipment.
|
||||
|
||||
The format has been reverse-engineered to facilitate sophisticated
|
||||
integrations with light and laser shows, videos, and other musical
|
||||
instruments, by supporting deep knowledge of what is playing and
|
||||
what is coming next through monitoring the network communications
|
||||
of the players.
|
||||
|
||||
doc-ref: https://reverseengineering.stackexchange.com/questions/4311/help-reversing-a-edb-database-file-for-pioneers-rekordbox-software
|
||||
|
||||
seq:
|
||||
- id: magic
|
||||
contents: "PMAI"
|
||||
doc: Identifies this as an analysis file.
|
||||
- id: len_header
|
||||
type: u4
|
||||
doc: |
|
||||
The number of bytes of this header section.
|
||||
- id: len_file
|
||||
type: u4
|
||||
doc: |
|
||||
The number of bytes in the entire file.
|
||||
- size: len_header - _io.pos
|
||||
- id: sections
|
||||
type: tagged_section
|
||||
repeat: eos
|
||||
doc: |
|
||||
The remainder of the file is a sequence of type-tagged sections,
|
||||
identified by a four-byte magic sequence.
|
||||
|
||||
types:
|
||||
tagged_section:
|
||||
doc: |
|
||||
A type-tagged file section, identified by a four-byte magic
|
||||
sequence, with a header specifying its length, and whose payload
|
||||
is determined by the type tag.
|
||||
seq:
|
||||
- id: fourcc
|
||||
type: s4
|
||||
enum: section_tags
|
||||
doc: |
|
||||
A tag value indicating what kind of section this is.
|
||||
- id: len_header
|
||||
type: u4
|
||||
doc: |
|
||||
The size, in bytes, of the header portion of the tag.
|
||||
- id: len_tag
|
||||
type: u4
|
||||
doc: |
|
||||
The size, in bytes, of this entire tag, counting the header.
|
||||
- id: body
|
||||
size: len_tag - 12
|
||||
type:
|
||||
switch-on: fourcc
|
||||
cases:
|
||||
'section_tags::cues_2': cue_extended_tag # PCO2
|
||||
'section_tags::cues': cue_tag # PCOB
|
||||
'section_tags::path': path_tag # PPTH
|
||||
'section_tags::beat_grid': beat_grid_tag # PQTZ
|
||||
'section_tags::vbr': vbr_tag # PVBR
|
||||
'section_tags::wave_preview': wave_preview_tag # PWAV
|
||||
'section_tags::wave_tiny': wave_preview_tag # PWV2
|
||||
'section_tags::wave_scroll': wave_scroll_tag # PWV3, seen in .EXT
|
||||
'section_tags::wave_color_preview': wave_color_preview_tag # PWV4, in .EXT
|
||||
'section_tags::wave_color_scroll': wave_color_scroll_tag # PWV5, in .EXT
|
||||
'section_tags::wave_3band_preview': wave_3band_preview_tag # P@V6, in .2EX
|
||||
'section_tags::wave_3band_scroll': wave_3band_scroll_tag # PWV7, in .2EX
|
||||
'section_tags::song_structure': song_structure_tag # PSSI, in .EXT
|
||||
_: unknown_tag
|
||||
-webide-representation: '{fourcc}'
|
||||
|
||||
|
||||
beat_grid_tag:
|
||||
doc: |
|
||||
Holds a list of all the beats found within the track, recording
|
||||
their bar position, the time at which they occur, and the tempo
|
||||
at that point.
|
||||
seq:
|
||||
- type: u4
|
||||
- type: u4 # @flesniak says this is always 0x80000
|
||||
- id: num_beats
|
||||
type: u4
|
||||
doc: |
|
||||
The number of beat entries which follow.
|
||||
- id: beats
|
||||
type: beat_grid_beat
|
||||
repeat: expr
|
||||
repeat-expr: num_beats
|
||||
doc: The entries of the beat grid.
|
||||
|
||||
beat_grid_beat:
|
||||
doc: |
|
||||
Describes an individual beat in a beat grid.
|
||||
seq:
|
||||
- id: beat_number
|
||||
type: u2
|
||||
doc: |
|
||||
The position of the beat within its musical bar, where beat 1
|
||||
is the down beat.
|
||||
- id: tempo
|
||||
type: u2
|
||||
doc: |
|
||||
The tempo at the time of this beat, in beats per minute,
|
||||
multiplied by 100.
|
||||
- id: time
|
||||
type: u4
|
||||
doc: |
|
||||
The time, in milliseconds, at which this beat occurs when
|
||||
the track is played at normal (100%) pitch.
|
||||
|
||||
cue_tag:
|
||||
doc: |
|
||||
Stores either a list of ordinary memory cues and loop points, or
|
||||
a list of hot cues and loop points.
|
||||
seq:
|
||||
- id: type
|
||||
type: u4
|
||||
enum: cue_list_type
|
||||
doc: |
|
||||
Identifies whether this tag stores ordinary or hot cues.
|
||||
- size: 2
|
||||
- id: num_cues
|
||||
type: u2
|
||||
doc: |
|
||||
The length of the cue list.
|
||||
- id: memory_count
|
||||
type: u4
|
||||
doc: |
|
||||
Unsure what this means.
|
||||
- id: cues
|
||||
type: cue_entry
|
||||
repeat: expr
|
||||
repeat-expr: num_cues
|
||||
|
||||
cue_entry:
|
||||
doc: |
|
||||
A cue list entry. Can either represent a memory cue or a loop.
|
||||
seq:
|
||||
- id: magic
|
||||
contents: "PCPT"
|
||||
doc: Identifies this as a cue list entry (cue point).
|
||||
- id: len_header
|
||||
type: u4
|
||||
- id: len_entry
|
||||
type: u4
|
||||
- id: hot_cue
|
||||
type: u4
|
||||
doc: |
|
||||
If zero, this is an ordinary memory cue, otherwise this a
|
||||
hot cue with the specified number.
|
||||
- id: status
|
||||
type: u4
|
||||
enum: cue_entry_status
|
||||
doc: |
|
||||
Indicates if this is an active loop.
|
||||
- type: u4 # Seems to always be 0x10000
|
||||
- id: order_first
|
||||
type: u2
|
||||
doc: |
|
||||
@flesniak says: "0xffff for first cue, 0,1,3 for next"
|
||||
- id: order_last
|
||||
type: u2
|
||||
doc: |
|
||||
@flesniak says: "1,2,3 for first, second, third cue, 0xffff for last"
|
||||
- id: type
|
||||
type: u1
|
||||
enum: cue_entry_type
|
||||
doc: |
|
||||
Indicates whether this is a memory cue or a loop.
|
||||
- size: 3 # seems to always be 1000
|
||||
- id: time
|
||||
type: u4
|
||||
doc: |
|
||||
The position, in milliseconds, at which the cue point lies
|
||||
in the track.
|
||||
- id: loop_time
|
||||
type: u4
|
||||
doc: |
|
||||
The position, in milliseconds, at which the player loops
|
||||
back to the cue time if this is a loop.
|
||||
- size: 16
|
||||
|
||||
cue_extended_tag:
|
||||
doc: |
|
||||
A variation of cue_tag which was introduced with the nxs2 line,
|
||||
and adds descriptive names. (Still comes in two forms, either
|
||||
holding memory cues and loop points, or holding hot cues and
|
||||
loop points.) Also includes hot cues D through H and color assignment.
|
||||
seq:
|
||||
- id: type
|
||||
type: u4
|
||||
enum: cue_list_type
|
||||
doc: |
|
||||
Identifies whether this tag stores ordinary or hot cues.
|
||||
- id: num_cues
|
||||
type: u2
|
||||
doc: |
|
||||
The length of the cue comment list.
|
||||
- size: 2
|
||||
- id: cues
|
||||
type: cue_extended_entry
|
||||
repeat: expr
|
||||
repeat-expr: num_cues
|
||||
|
||||
cue_extended_entry:
|
||||
doc: |
|
||||
A cue extended list entry. Can either describe a memory cue or a
|
||||
loop.
|
||||
seq:
|
||||
- id: magic
|
||||
contents: "PCP2"
|
||||
doc: Identifies this as an extended cue list entry (cue point).
|
||||
- id: len_header
|
||||
type: u4
|
||||
- id: len_entry
|
||||
type: u4
|
||||
- id: hot_cue
|
||||
type: u4
|
||||
doc: |
|
||||
If zero, this is an ordinary memory cue, otherwise this a
|
||||
hot cue with the specified number.
|
||||
- id: type
|
||||
type: u1
|
||||
enum: cue_entry_type
|
||||
doc: |
|
||||
Indicates whether this is a regular cue point or a loop.
|
||||
- size: 3 # seems to always be 1000
|
||||
- id: time
|
||||
type: u4
|
||||
doc: |
|
||||
The position, in milliseconds, at which the cue point lies
|
||||
in the track.
|
||||
- id: loop_time
|
||||
type: u4
|
||||
doc: |
|
||||
The position, in milliseconds, at which the player loops
|
||||
back to the cue time if this is a loop.
|
||||
- id: color_id
|
||||
type: u1
|
||||
doc: |
|
||||
References a row in the colors table if this is a memory cue or loop
|
||||
and has been assigned a color.
|
||||
- size: 7
|
||||
- id: loop_numerator
|
||||
type: u2
|
||||
doc: |
|
||||
The numerator of the loop length in beats.
|
||||
Zero if the loop is not quantized.
|
||||
- id: loop_denominator
|
||||
type: u2
|
||||
doc: |
|
||||
The denominator of the loop length in beats.
|
||||
Zero if the loop is not quantized.
|
||||
- id: len_comment
|
||||
type: u4
|
||||
if: len_entry > 43
|
||||
- id: comment
|
||||
type: str
|
||||
size: len_comment
|
||||
encoding: utf-16be
|
||||
doc: |
|
||||
The comment assigned to this cue by the DJ, if any, with a trailing NUL.
|
||||
if: len_entry > 43
|
||||
- id: color_code
|
||||
type: u1
|
||||
doc: |
|
||||
A lookup value for a color table? We use this to index to the hot cue colors shown in rekordbox.
|
||||
if: (len_entry - len_comment) > 44
|
||||
- id: color_red
|
||||
type: u1
|
||||
doc: |
|
||||
The red component of the hot cue color to be displayed.
|
||||
if: (len_entry - len_comment) > 45
|
||||
- id: color_green
|
||||
type: u1
|
||||
doc: |
|
||||
The green component of the hot cue color to be displayed.
|
||||
if: (len_entry - len_comment) > 46
|
||||
- id: color_blue
|
||||
type: u1
|
||||
doc: |
|
||||
The blue component of the hot cue color to be displayed.
|
||||
if: (len_entry - len_comment) > 47
|
||||
- size: len_entry - 48 - len_comment # The remainder after the color
|
||||
if: (len_entry - len_comment) > 48
|
||||
|
||||
path_tag:
|
||||
doc: |
|
||||
Stores the file path of the audio file to which this analysis
|
||||
applies.
|
||||
seq:
|
||||
- id: len_path
|
||||
type: u4
|
||||
- id: path
|
||||
type: str
|
||||
size: len_path - 2
|
||||
encoding: utf-16be
|
||||
if: len_path > 1
|
||||
-webide-representation: '{path}'
|
||||
|
||||
vbr_tag:
|
||||
doc: |
|
||||
Stores an index allowing rapid seeking to particular times
|
||||
within a variable-bitrate audio file.
|
||||
seq:
|
||||
- type: u4
|
||||
- id: index
|
||||
type: u4
|
||||
repeat: expr
|
||||
repeat-expr: 400
|
||||
|
||||
wave_preview_tag:
|
||||
doc: |
|
||||
Stores a waveform preview image suitable for display above
|
||||
the touch strip for jumping to a track position.
|
||||
seq:
|
||||
- id: len_data
|
||||
type: u4
|
||||
doc: |
|
||||
The length, in bytes, of the preview data itself. This is
|
||||
slightly redundant because it can be computed from the
|
||||
length of the tag.
|
||||
- type: u4 # This seems to always have the value 0x10000
|
||||
- id: data
|
||||
size: len_data
|
||||
doc: |
|
||||
The actual bytes of the waveform preview.
|
||||
if: _parent.len_tag > _parent.len_header
|
||||
|
||||
wave_scroll_tag:
|
||||
doc: |
|
||||
A larger waveform image suitable for scrolling along as a track
|
||||
plays.
|
||||
seq:
|
||||
- id: len_entry_bytes
|
||||
type: u4
|
||||
doc: |
|
||||
The size of each entry, in bytes. Seems to always be 1.
|
||||
- id: len_entries
|
||||
type: u4
|
||||
doc: |
|
||||
The number of waveform data points, each of which takes one
|
||||
byte.
|
||||
- type: u4 # Always 0x960000?
|
||||
- id: entries
|
||||
size: len_entries * len_entry_bytes
|
||||
|
||||
wave_color_preview_tag:
|
||||
doc: |
|
||||
A larger, colorful waveform preview image suitable for display
|
||||
above the touch strip for jumping to a track position on newer
|
||||
high-resolution players.
|
||||
seq:
|
||||
- id: len_entry_bytes
|
||||
type: u4
|
||||
doc: |
|
||||
The size of each entry, in bytes. Seems to always be 6.
|
||||
- id: len_entries
|
||||
type: u4
|
||||
doc: |
|
||||
The number of waveform data points, each of which takes one
|
||||
byte for each of six channels of information.
|
||||
- type: u4
|
||||
- id: entries
|
||||
size: len_entries * len_entry_bytes
|
||||
|
||||
wave_color_scroll_tag:
|
||||
doc: |
|
||||
A larger, colorful waveform image suitable for scrolling along
|
||||
as a track plays on newer high-resolution hardware. Also
|
||||
contains a higher-resolution blue/white waveform.
|
||||
seq:
|
||||
- id: len_entry_bytes
|
||||
type: u4
|
||||
doc: |
|
||||
The size of each entry, in bytes. Seems to always be 2.
|
||||
- id: len_entries
|
||||
type: u4
|
||||
doc: |
|
||||
The number of columns of waveform data (this matches the
|
||||
non-color waveform length.
|
||||
- type: u4
|
||||
- id: entries
|
||||
size: len_entries * len_entry_bytes
|
||||
|
||||
wave_3band_preview_tag:
|
||||
doc: |
|
||||
The minimalist CDJ-3000 waveform preview image suitable for display
|
||||
above the touch strip for jumping to a track position.
|
||||
seq:
|
||||
- id: len_entry_bytes
|
||||
type: u4
|
||||
doc: |
|
||||
The size of each entry, in bytes. Seems to always be 3.
|
||||
- id: len_entries
|
||||
type: u4
|
||||
doc: |
|
||||
The number of waveform data points, each of which takes one
|
||||
byte for each of six channels of information.
|
||||
- id: entries
|
||||
size: len_entries * len_entry_bytes
|
||||
|
||||
wave_3band_scroll_tag:
|
||||
doc: |
|
||||
The minimalist CDJ-3000 waveform image suitable for scrolling along
|
||||
as a track plays on newer high-resolution hardware.
|
||||
seq:
|
||||
- id: len_entry_bytes
|
||||
type: u4
|
||||
doc: |
|
||||
The size of each entry, in bytes. Seems to always be 3.
|
||||
- id: len_entries
|
||||
type: u4
|
||||
doc: |
|
||||
The number of columns of waveform data (this matches the
|
||||
non-color waveform length.
|
||||
- type: u4
|
||||
- id: entries
|
||||
size: len_entries * len_entry_bytes
|
||||
|
||||
song_structure_tag:
|
||||
doc: |
|
||||
Stores the song structure, also known as phrases (intro, verse,
|
||||
bridge, chorus, up, down, outro).
|
||||
seq:
|
||||
- id: len_entry_bytes
|
||||
type: u4
|
||||
doc: |
|
||||
The size of each entry, in bytes. Seems to always be 24.
|
||||
- id: len_entries
|
||||
type: u2
|
||||
doc: |
|
||||
The number of phrases.
|
||||
- id: body
|
||||
type: song_structure_body
|
||||
doc: |
|
||||
The rest of the tag, which needs to be unmasked before it
|
||||
can be parsed.
|
||||
size-eos: true
|
||||
process: 'xor(is_masked ? mask : [0])'
|
||||
instances:
|
||||
c:
|
||||
value: len_entries
|
||||
mask:
|
||||
value: |
|
||||
[
|
||||
(0xCB+c).as<s1>, (0xE1+c).as<s1>, (0xEE+c).as<s1>, (0xFA+c).as<s1>, (0xE5+c).as<s1>, (0xEE+c).as<s1>, (0xAD+c).as<s1>, (0xEE+c).as<s1>,
|
||||
(0xE9+c).as<s1>, (0xD2+c).as<s1>, (0xE9+c).as<s1>, (0xEB+c).as<s1>, (0xE1+c).as<s1>, (0xE9+c).as<s1>, (0xF3+c).as<s1>, (0xE8+c).as<s1>,
|
||||
(0xE9+c).as<s1>, (0xF4+c).as<s1>, (0xE1+c).as<s1>
|
||||
].as<bytes>
|
||||
raw_mood:
|
||||
type: u2
|
||||
pos: 6
|
||||
doc: |
|
||||
This is a way to tell whether the rest of the tag has been masked. The value is supposed
|
||||
to range from 1 to 3, but in masked files it will be much larger.
|
||||
is_masked:
|
||||
value: 'raw_mood > 20' # This is almost certainly not true for an unmasked file.
|
||||
-webide-representation: '{body.mood}'
|
||||
|
||||
|
||||
song_structure_body:
|
||||
doc: |
|
||||
Stores the rest of the song structure tag, which can only be
|
||||
parsed after unmasking.
|
||||
seq:
|
||||
- id: mood
|
||||
type: u2
|
||||
enum: track_mood
|
||||
doc: |
|
||||
The mood which rekordbox assigns the track as a whole during phrase analysis.
|
||||
- size: 6
|
||||
- id: end_beat
|
||||
type: u2
|
||||
doc: |
|
||||
The beat number at which the last phrase ends. The track may
|
||||
continue after the last phrase ends. If this is the case, it will
|
||||
mostly be silence.
|
||||
- size: 2
|
||||
- id: raw_bank
|
||||
type: u1
|
||||
doc: |
|
||||
Number identifying a stylistic bank which can be assigned to the track in rekordbox Lighting mode.
|
||||
- size: 1
|
||||
- id: entries
|
||||
type: song_structure_entry
|
||||
repeat: expr
|
||||
repeat-expr: _parent.len_entries
|
||||
instances:
|
||||
bank:
|
||||
value: raw_bank
|
||||
enum: track_bank
|
||||
doc: |
|
||||
The stylistic bank which can be assigned to the track in rekordbox Lighting mode, if raw_bank has a legal value.
|
||||
if: 'raw_bank < 9'
|
||||
|
||||
|
||||
song_structure_entry:
|
||||
doc: |
|
||||
A song structure entry, represents a single phrase.
|
||||
seq:
|
||||
- id: index
|
||||
type: u2
|
||||
doc: |
|
||||
The absolute number of the phrase, starting at one.
|
||||
- id: beat
|
||||
type: u2
|
||||
doc: |
|
||||
The beat number at which the phrase starts.
|
||||
- id: kind
|
||||
type:
|
||||
switch-on: _parent.mood
|
||||
cases:
|
||||
'track_mood::high': phrase_high
|
||||
'track_mood::mid': phrase_mid
|
||||
'track_mood::low': phrase_low
|
||||
_: phrase_mid # We don't recognize this mood, so pick a generic interpretation.
|
||||
doc: |
|
||||
The kind of phrase as displayed in rekordbox.
|
||||
- size: 1
|
||||
- id: k1
|
||||
type: u1
|
||||
doc: One of three flags that identify phrase kind variants in high-mood tracks.
|
||||
- size: 1
|
||||
- id: k2
|
||||
type: u1
|
||||
doc: One of three flags that identify phrase kind variants in high-mood tracks.
|
||||
- size: 1
|
||||
- id: b
|
||||
type: u1
|
||||
doc: |
|
||||
Flags how many more beat numbers are in a high-mood "Up 3" phrase.
|
||||
- id: beat2
|
||||
type: u2
|
||||
doc: |
|
||||
Extra beat number (falling within phrase) always present in high-mood "Up 3" phrases.
|
||||
- id: beat3
|
||||
type: u2
|
||||
doc: |
|
||||
Extra beat number (falling within phrase, larger than beat2)
|
||||
present in high-mood "Up 3" phrases when b has value 1.
|
||||
- id: beat4
|
||||
type: u2
|
||||
doc: |
|
||||
Extra beat number (falling within phrase, larger than beat3)
|
||||
present in high-mood "Up 3" phrases when b has value 1.
|
||||
- size: 1
|
||||
- id: k3
|
||||
type: u1
|
||||
doc: One of three flags that identify phrase kind variants in high-mood tracks.
|
||||
- size: 1
|
||||
- id: fill
|
||||
type: u1
|
||||
doc: |
|
||||
If nonzero, fill-in is present at end of phrase.
|
||||
- id: beat_fill
|
||||
type: u2
|
||||
doc: |
|
||||
The beat number at which fill-in starts.
|
||||
-webide-representation: '{kind.id}'
|
||||
|
||||
phrase_high:
|
||||
seq:
|
||||
- id: id
|
||||
type: u2
|
||||
enum: mood_high_phrase
|
||||
|
||||
phrase_mid:
|
||||
seq:
|
||||
- id: id
|
||||
type: u2
|
||||
enum: mood_mid_phrase
|
||||
|
||||
phrase_low:
|
||||
seq:
|
||||
- id: id
|
||||
type: u2
|
||||
enum: mood_low_phrase
|
||||
|
||||
unknown_tag: {}
|
||||
|
||||
enums:
|
||||
section_tags:
|
||||
0x50434f42: cues # PCOB
|
||||
0x50434f32: cues_2 # PCO2 (seen in .EXT)
|
||||
0x50505448: path # PPTH
|
||||
0x50564252: vbr # PVBR
|
||||
0x5051545a: beat_grid # PQTZ
|
||||
0x50574156: wave_preview # PWAV
|
||||
0x50575632: wave_tiny # PWV2
|
||||
0x50575633: wave_scroll # PWV3 (seen in .EXT)
|
||||
0x50575634: wave_color_preview # PWV4 (seen in .EXT)
|
||||
0x50575635: wave_color_scroll # PWV5 (seen in .EXT)
|
||||
0x50535349: song_structure # PSSI (seen in .EXT)
|
||||
0x50575636: wave_3band_preview # PWV6 (seen in .2EX)
|
||||
0x50575637: wave_3band_scroll # PWV7 (seen in .2EX)
|
||||
|
||||
cue_list_type:
|
||||
0: memory_cues
|
||||
1: hot_cues
|
||||
|
||||
cue_entry_type:
|
||||
1: memory_cue
|
||||
2: loop
|
||||
|
||||
cue_entry_status:
|
||||
0: disabled
|
||||
1: enabled
|
||||
4: active_loop
|
||||
|
||||
track_mood:
|
||||
1: high
|
||||
2: mid
|
||||
3: low
|
||||
|
||||
mood_low_phrase:
|
||||
1: intro
|
||||
2: verse_1
|
||||
3: verse_1b # Just displayed as "Verse 1" in rekordbox.
|
||||
4: verse_1c # Just displayed as "Verse 1" in rekordbox.
|
||||
5: verse_2
|
||||
6: verse_2b # Just displayed as "Verse 2" in rekordbox.
|
||||
7: verse_2c # Just displayed as "Verse 2" in rekordbox.
|
||||
8: bridge
|
||||
9: chorus
|
||||
10: outro
|
||||
|
||||
mood_mid_phrase:
|
||||
1: intro
|
||||
2: verse_1
|
||||
3: verse_2
|
||||
4: verse_3
|
||||
5: verse_4
|
||||
6: verse_5
|
||||
7: verse_6
|
||||
8: bridge
|
||||
9: chorus
|
||||
10: outro
|
||||
|
||||
mood_high_phrase:
|
||||
1: intro
|
||||
2: up
|
||||
3: down
|
||||
5: chorus
|
||||
6: outro
|
||||
|
||||
track_bank:
|
||||
0: default
|
||||
1: cool
|
||||
2: natural
|
||||
3: hot
|
||||
4: subtle
|
||||
5: warm
|
||||
6: vivid
|
||||
7: club_1
|
||||
8: club_2
|
||||
@@ -0,0 +1,897 @@
|
||||
# This is a generated file! Please edit source .ksy file and use kaitai-struct-compiler to rebuild
|
||||
# type: ignore
|
||||
|
||||
import kaitaistruct
|
||||
from kaitaistruct import KaitaiStruct, KaitaiStream, BytesIO
|
||||
from enum import IntEnum
|
||||
import struct
|
||||
|
||||
|
||||
if getattr(kaitaistruct, 'API_VERSION', (0, 9)) < (0, 11):
|
||||
raise Exception("Incompatible Kaitai Struct Python API: 0.11 or later is required, but you have %s" % (kaitaistruct.__version__))
|
||||
|
||||
class RekordboxAnlz(KaitaiStruct):
|
||||
"""These files are created by rekordbox when analyzing audio tracks
|
||||
to facilitate DJ performance. They include waveforms, beat grids
|
||||
(information about the precise time at which each beat occurs),
|
||||
time indices to allow efficient seeking to specific positions
|
||||
inside variable bit-rate audio streams, and lists of memory cues
|
||||
and loop points. They are used by Pioneer professional DJ
|
||||
equipment.
|
||||
|
||||
The format has been reverse-engineered to facilitate sophisticated
|
||||
integrations with light and laser shows, videos, and other musical
|
||||
instruments, by supporting deep knowledge of what is playing and
|
||||
what is coming next through monitoring the network communications
|
||||
of the players.
|
||||
|
||||
.. seealso::
|
||||
Source - https://reverseengineering.stackexchange.com/questions/4311/help-reversing-a-edb-database-file-for-pioneers-rekordbox-software
|
||||
"""
|
||||
|
||||
class CueEntryStatus(IntEnum):
|
||||
disabled = 0
|
||||
enabled = 1
|
||||
active_loop = 4
|
||||
|
||||
class CueEntryType(IntEnum):
|
||||
memory_cue = 1
|
||||
loop = 2
|
||||
|
||||
class CueListType(IntEnum):
|
||||
memory_cues = 0
|
||||
hot_cues = 1
|
||||
|
||||
class MoodHighPhrase(IntEnum):
|
||||
intro = 1
|
||||
up = 2
|
||||
down = 3
|
||||
chorus = 5
|
||||
outro = 6
|
||||
|
||||
class MoodLowPhrase(IntEnum):
|
||||
intro = 1
|
||||
verse_1 = 2
|
||||
verse_1b = 3
|
||||
verse_1c = 4
|
||||
verse_2 = 5
|
||||
verse_2b = 6
|
||||
verse_2c = 7
|
||||
bridge = 8
|
||||
chorus = 9
|
||||
outro = 10
|
||||
|
||||
class MoodMidPhrase(IntEnum):
|
||||
intro = 1
|
||||
verse_1 = 2
|
||||
verse_2 = 3
|
||||
verse_3 = 4
|
||||
verse_4 = 5
|
||||
verse_5 = 6
|
||||
verse_6 = 7
|
||||
bridge = 8
|
||||
chorus = 9
|
||||
outro = 10
|
||||
|
||||
class SectionTags(IntEnum):
|
||||
cues_2 = 1346588466
|
||||
cues = 1346588482
|
||||
path = 1347441736
|
||||
beat_grid = 1347507290
|
||||
song_structure = 1347638089
|
||||
vbr = 1347830354
|
||||
wave_preview = 1347895638
|
||||
wave_tiny = 1347900978
|
||||
wave_scroll = 1347900979
|
||||
wave_color_preview = 1347900980
|
||||
wave_color_scroll = 1347900981
|
||||
wave_3band_preview = 1347900982
|
||||
wave_3band_scroll = 1347900983
|
||||
|
||||
class TrackBank(IntEnum):
|
||||
default = 0
|
||||
cool = 1
|
||||
natural = 2
|
||||
hot = 3
|
||||
subtle = 4
|
||||
warm = 5
|
||||
vivid = 6
|
||||
club_1 = 7
|
||||
club_2 = 8
|
||||
|
||||
class TrackMood(IntEnum):
|
||||
high = 1
|
||||
mid = 2
|
||||
low = 3
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root or self
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.magic = self._io.read_bytes(4)
|
||||
if not self.magic == b"\x50\x4D\x41\x49":
|
||||
raise kaitaistruct.ValidationNotEqualError(b"\x50\x4D\x41\x49", self.magic, self._io, u"/seq/0")
|
||||
self.len_header = self._io.read_u4be()
|
||||
self.len_file = self._io.read_u4be()
|
||||
self._unnamed3 = self._io.read_bytes(self.len_header - self._io.pos())
|
||||
self.sections = []
|
||||
i = 0
|
||||
while not self._io.is_eof():
|
||||
self.sections.append(RekordboxAnlz.TaggedSection(self._io, self, self._root))
|
||||
i += 1
|
||||
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
for i in range(len(self.sections)):
|
||||
pass
|
||||
self.sections[i]._fetch_instances()
|
||||
|
||||
|
||||
class BeatGridBeat(KaitaiStruct):
|
||||
"""Describes an individual beat in a beat grid.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.BeatGridBeat, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.beat_number = self._io.read_u2be()
|
||||
self.tempo = self._io.read_u2be()
|
||||
self.time = self._io.read_u4be()
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
|
||||
|
||||
class BeatGridTag(KaitaiStruct):
|
||||
"""Holds a list of all the beats found within the track, recording
|
||||
their bar position, the time at which they occur, and the tempo
|
||||
at that point.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.BeatGridTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self._unnamed0 = self._io.read_u4be()
|
||||
self._unnamed1 = self._io.read_u4be()
|
||||
self.num_beats = self._io.read_u4be()
|
||||
self.beats = []
|
||||
for i in range(self.num_beats):
|
||||
self.beats.append(RekordboxAnlz.BeatGridBeat(self._io, self, self._root))
|
||||
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
for i in range(len(self.beats)):
|
||||
pass
|
||||
self.beats[i]._fetch_instances()
|
||||
|
||||
|
||||
|
||||
class CueEntry(KaitaiStruct):
|
||||
"""A cue list entry. Can either represent a memory cue or a loop.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.CueEntry, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.magic = self._io.read_bytes(4)
|
||||
if not self.magic == b"\x50\x43\x50\x54":
|
||||
raise kaitaistruct.ValidationNotEqualError(b"\x50\x43\x50\x54", self.magic, self._io, u"/types/cue_entry/seq/0")
|
||||
self.len_header = self._io.read_u4be()
|
||||
self.len_entry = self._io.read_u4be()
|
||||
self.hot_cue = self._io.read_u4be()
|
||||
self.status = KaitaiStream.resolve_enum(RekordboxAnlz.CueEntryStatus, self._io.read_u4be())
|
||||
self._unnamed5 = self._io.read_u4be()
|
||||
self.order_first = self._io.read_u2be()
|
||||
self.order_last = self._io.read_u2be()
|
||||
self.type = KaitaiStream.resolve_enum(RekordboxAnlz.CueEntryType, self._io.read_u1())
|
||||
self._unnamed9 = self._io.read_bytes(3)
|
||||
self.time = self._io.read_u4be()
|
||||
self.loop_time = self._io.read_u4be()
|
||||
self._unnamed12 = self._io.read_bytes(16)
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
|
||||
|
||||
class CueExtendedEntry(KaitaiStruct):
|
||||
"""A cue extended list entry. Can either describe a memory cue or a
|
||||
loop.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.CueExtendedEntry, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.magic = self._io.read_bytes(4)
|
||||
if not self.magic == b"\x50\x43\x50\x32":
|
||||
raise kaitaistruct.ValidationNotEqualError(b"\x50\x43\x50\x32", self.magic, self._io, u"/types/cue_extended_entry/seq/0")
|
||||
self.len_header = self._io.read_u4be()
|
||||
self.len_entry = self._io.read_u4be()
|
||||
self.hot_cue = self._io.read_u4be()
|
||||
self.type = KaitaiStream.resolve_enum(RekordboxAnlz.CueEntryType, self._io.read_u1())
|
||||
self._unnamed5 = self._io.read_bytes(3)
|
||||
self.time = self._io.read_u4be()
|
||||
self.loop_time = self._io.read_u4be()
|
||||
self.color_id = self._io.read_u1()
|
||||
self._unnamed9 = self._io.read_bytes(7)
|
||||
self.loop_numerator = self._io.read_u2be()
|
||||
self.loop_denominator = self._io.read_u2be()
|
||||
if self.len_entry > 43:
|
||||
pass
|
||||
self.len_comment = self._io.read_u4be()
|
||||
|
||||
if self.len_entry > 43:
|
||||
pass
|
||||
self.comment = (self._io.read_bytes(self.len_comment)).decode(u"UTF-16BE")
|
||||
|
||||
if self.len_entry - self.len_comment > 44:
|
||||
pass
|
||||
self.color_code = self._io.read_u1()
|
||||
|
||||
if self.len_entry - self.len_comment > 45:
|
||||
pass
|
||||
self.color_red = self._io.read_u1()
|
||||
|
||||
if self.len_entry - self.len_comment > 46:
|
||||
pass
|
||||
self.color_green = self._io.read_u1()
|
||||
|
||||
if self.len_entry - self.len_comment > 47:
|
||||
pass
|
||||
self.color_blue = self._io.read_u1()
|
||||
|
||||
if self.len_entry - self.len_comment > 48:
|
||||
pass
|
||||
self._unnamed18 = self._io.read_bytes((self.len_entry - 48) - self.len_comment)
|
||||
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
if self.len_entry > 43:
|
||||
pass
|
||||
|
||||
if self.len_entry > 43:
|
||||
pass
|
||||
|
||||
if self.len_entry - self.len_comment > 44:
|
||||
pass
|
||||
|
||||
if self.len_entry - self.len_comment > 45:
|
||||
pass
|
||||
|
||||
if self.len_entry - self.len_comment > 46:
|
||||
pass
|
||||
|
||||
if self.len_entry - self.len_comment > 47:
|
||||
pass
|
||||
|
||||
if self.len_entry - self.len_comment > 48:
|
||||
pass
|
||||
|
||||
|
||||
|
||||
class CueExtendedTag(KaitaiStruct):
|
||||
"""A variation of cue_tag which was introduced with the nxs2 line,
|
||||
and adds descriptive names. (Still comes in two forms, either
|
||||
holding memory cues and loop points, or holding hot cues and
|
||||
loop points.) Also includes hot cues D through H and color assignment.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.CueExtendedTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.type = KaitaiStream.resolve_enum(RekordboxAnlz.CueListType, self._io.read_u4be())
|
||||
self.num_cues = self._io.read_u2be()
|
||||
self._unnamed2 = self._io.read_bytes(2)
|
||||
self.cues = []
|
||||
for i in range(self.num_cues):
|
||||
self.cues.append(RekordboxAnlz.CueExtendedEntry(self._io, self, self._root))
|
||||
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
for i in range(len(self.cues)):
|
||||
pass
|
||||
self.cues[i]._fetch_instances()
|
||||
|
||||
|
||||
|
||||
class CueTag(KaitaiStruct):
|
||||
"""Stores either a list of ordinary memory cues and loop points, or
|
||||
a list of hot cues and loop points.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.CueTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.type = KaitaiStream.resolve_enum(RekordboxAnlz.CueListType, self._io.read_u4be())
|
||||
self._unnamed1 = self._io.read_bytes(2)
|
||||
self.num_cues = self._io.read_u2be()
|
||||
self.memory_count = self._io.read_u4be()
|
||||
self.cues = []
|
||||
for i in range(self.num_cues):
|
||||
self.cues.append(RekordboxAnlz.CueEntry(self._io, self, self._root))
|
||||
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
for i in range(len(self.cues)):
|
||||
pass
|
||||
self.cues[i]._fetch_instances()
|
||||
|
||||
|
||||
|
||||
class PathTag(KaitaiStruct):
|
||||
"""Stores the file path of the audio file to which this analysis
|
||||
applies.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.PathTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.len_path = self._io.read_u4be()
|
||||
if self.len_path > 1:
|
||||
pass
|
||||
self.path = (self._io.read_bytes(self.len_path - 2)).decode(u"UTF-16BE")
|
||||
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
if self.len_path > 1:
|
||||
pass
|
||||
|
||||
|
||||
|
||||
class PhraseHigh(KaitaiStruct):
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.PhraseHigh, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.id = KaitaiStream.resolve_enum(RekordboxAnlz.MoodHighPhrase, self._io.read_u2be())
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
|
||||
|
||||
class PhraseLow(KaitaiStruct):
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.PhraseLow, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.id = KaitaiStream.resolve_enum(RekordboxAnlz.MoodLowPhrase, self._io.read_u2be())
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
|
||||
|
||||
class PhraseMid(KaitaiStruct):
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.PhraseMid, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.id = KaitaiStream.resolve_enum(RekordboxAnlz.MoodMidPhrase, self._io.read_u2be())
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
|
||||
|
||||
class SongStructureBody(KaitaiStruct):
|
||||
"""Stores the rest of the song structure tag, which can only be
|
||||
parsed after unmasking.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.SongStructureBody, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.mood = KaitaiStream.resolve_enum(RekordboxAnlz.TrackMood, self._io.read_u2be())
|
||||
self._unnamed1 = self._io.read_bytes(6)
|
||||
self.end_beat = self._io.read_u2be()
|
||||
self._unnamed3 = self._io.read_bytes(2)
|
||||
self.raw_bank = self._io.read_u1()
|
||||
self._unnamed5 = self._io.read_bytes(1)
|
||||
self.entries = []
|
||||
for i in range(self._parent.len_entries):
|
||||
self.entries.append(RekordboxAnlz.SongStructureEntry(self._io, self, self._root))
|
||||
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
for i in range(len(self.entries)):
|
||||
pass
|
||||
self.entries[i]._fetch_instances()
|
||||
|
||||
|
||||
@property
|
||||
def bank(self):
|
||||
"""The stylistic bank which can be assigned to the track in rekordbox Lighting mode, if raw_bank has a legal value.
|
||||
"""
|
||||
if hasattr(self, '_m_bank'):
|
||||
return self._m_bank
|
||||
|
||||
if self.raw_bank < 9:
|
||||
pass
|
||||
self._m_bank = KaitaiStream.resolve_enum(RekordboxAnlz.TrackBank, self.raw_bank)
|
||||
|
||||
return getattr(self, '_m_bank', None)
|
||||
|
||||
|
||||
class SongStructureEntry(KaitaiStruct):
|
||||
"""A song structure entry, represents a single phrase.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.SongStructureEntry, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.index = self._io.read_u2be()
|
||||
self.beat = self._io.read_u2be()
|
||||
_on = self._parent.mood
|
||||
if _on == RekordboxAnlz.TrackMood.high:
|
||||
pass
|
||||
self.kind = RekordboxAnlz.PhraseHigh(self._io, self, self._root)
|
||||
elif _on == RekordboxAnlz.TrackMood.low:
|
||||
pass
|
||||
self.kind = RekordboxAnlz.PhraseLow(self._io, self, self._root)
|
||||
elif _on == RekordboxAnlz.TrackMood.mid:
|
||||
pass
|
||||
self.kind = RekordboxAnlz.PhraseMid(self._io, self, self._root)
|
||||
else:
|
||||
pass
|
||||
self.kind = RekordboxAnlz.PhraseMid(self._io, self, self._root)
|
||||
self._unnamed3 = self._io.read_bytes(1)
|
||||
self.k1 = self._io.read_u1()
|
||||
self._unnamed5 = self._io.read_bytes(1)
|
||||
self.k2 = self._io.read_u1()
|
||||
self._unnamed7 = self._io.read_bytes(1)
|
||||
self.b = self._io.read_u1()
|
||||
self.beat2 = self._io.read_u2be()
|
||||
self.beat3 = self._io.read_u2be()
|
||||
self.beat4 = self._io.read_u2be()
|
||||
self._unnamed12 = self._io.read_bytes(1)
|
||||
self.k3 = self._io.read_u1()
|
||||
self._unnamed14 = self._io.read_bytes(1)
|
||||
self.fill = self._io.read_u1()
|
||||
self.beat_fill = self._io.read_u2be()
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
_on = self._parent.mood
|
||||
if _on == RekordboxAnlz.TrackMood.high:
|
||||
pass
|
||||
self.kind._fetch_instances()
|
||||
elif _on == RekordboxAnlz.TrackMood.low:
|
||||
pass
|
||||
self.kind._fetch_instances()
|
||||
elif _on == RekordboxAnlz.TrackMood.mid:
|
||||
pass
|
||||
self.kind._fetch_instances()
|
||||
else:
|
||||
pass
|
||||
self.kind._fetch_instances()
|
||||
|
||||
|
||||
class SongStructureTag(KaitaiStruct):
|
||||
"""Stores the song structure, also known as phrases (intro, verse,
|
||||
bridge, chorus, up, down, outro).
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.SongStructureTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.len_entry_bytes = self._io.read_u4be()
|
||||
self.len_entries = self._io.read_u2be()
|
||||
self._raw__raw_body = self._io.read_bytes_full()
|
||||
self._raw_body = KaitaiStream.process_xor_many(self._raw__raw_body, (self.mask if self.is_masked else b"\x00"))
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.SongStructureBody(_io__raw_body, self, self._root)
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
_ = self.raw_mood
|
||||
if hasattr(self, '_m_raw_mood'):
|
||||
pass
|
||||
|
||||
|
||||
@property
|
||||
def c(self):
|
||||
if hasattr(self, '_m_c'):
|
||||
return self._m_c
|
||||
|
||||
self._m_c = self.len_entries
|
||||
return getattr(self, '_m_c', None)
|
||||
|
||||
@property
|
||||
def is_masked(self):
|
||||
if hasattr(self, '_m_is_masked'):
|
||||
return self._m_is_masked
|
||||
|
||||
self._m_is_masked = self.raw_mood > 20
|
||||
return getattr(self, '_m_is_masked', None)
|
||||
|
||||
@property
|
||||
def mask(self):
|
||||
if hasattr(self, '_m_mask'):
|
||||
return self._m_mask
|
||||
|
||||
self._m_mask = struct.pack('19B', (203 + self.c), (225 + self.c), (238 + self.c), (250 + self.c), (229 + self.c), (238 + self.c), (173 + self.c), (238 + self.c), (233 + self.c), (210 + self.c), (233 + self.c), (235 + self.c), (225 + self.c), (233 + self.c), (243 + self.c), (232 + self.c), (233 + self.c), (244 + self.c), (225 + self.c))
|
||||
return getattr(self, '_m_mask', None)
|
||||
|
||||
@property
|
||||
def raw_mood(self):
|
||||
"""This is a way to tell whether the rest of the tag has been masked. The value is supposed
|
||||
to range from 1 to 3, but in masked files it will be much larger.
|
||||
"""
|
||||
if hasattr(self, '_m_raw_mood'):
|
||||
return self._m_raw_mood
|
||||
|
||||
_pos = self._io.pos()
|
||||
self._io.seek(6)
|
||||
self._m_raw_mood = self._io.read_u2be()
|
||||
self._io.seek(_pos)
|
||||
return getattr(self, '_m_raw_mood', None)
|
||||
|
||||
|
||||
class TaggedSection(KaitaiStruct):
|
||||
"""A type-tagged file section, identified by a four-byte magic
|
||||
sequence, with a header specifying its length, and whose payload
|
||||
is determined by the type tag.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.TaggedSection, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.fourcc = KaitaiStream.resolve_enum(RekordboxAnlz.SectionTags, self._io.read_s4be())
|
||||
self.len_header = self._io.read_u4be()
|
||||
self.len_tag = self._io.read_u4be()
|
||||
_on = self.fourcc
|
||||
if _on == RekordboxAnlz.SectionTags.beat_grid:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.BeatGridTag(_io__raw_body, self, self._root)
|
||||
elif _on == RekordboxAnlz.SectionTags.cues:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.CueTag(_io__raw_body, self, self._root)
|
||||
elif _on == RekordboxAnlz.SectionTags.cues_2:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.CueExtendedTag(_io__raw_body, self, self._root)
|
||||
elif _on == RekordboxAnlz.SectionTags.path:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.PathTag(_io__raw_body, self, self._root)
|
||||
elif _on == RekordboxAnlz.SectionTags.song_structure:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.SongStructureTag(_io__raw_body, self, self._root)
|
||||
elif _on == RekordboxAnlz.SectionTags.vbr:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.VbrTag(_io__raw_body, self, self._root)
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_3band_preview:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.Wave3bandPreviewTag(_io__raw_body, self, self._root)
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_3band_scroll:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.Wave3bandScrollTag(_io__raw_body, self, self._root)
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_color_preview:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.WaveColorPreviewTag(_io__raw_body, self, self._root)
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_color_scroll:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.WaveColorScrollTag(_io__raw_body, self, self._root)
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_preview:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.WavePreviewTag(_io__raw_body, self, self._root)
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_scroll:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.WaveScrollTag(_io__raw_body, self, self._root)
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_tiny:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.WavePreviewTag(_io__raw_body, self, self._root)
|
||||
else:
|
||||
pass
|
||||
self._raw_body = self._io.read_bytes(self.len_tag - 12)
|
||||
_io__raw_body = KaitaiStream(BytesIO(self._raw_body))
|
||||
self.body = RekordboxAnlz.UnknownTag(_io__raw_body, self, self._root)
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
_on = self.fourcc
|
||||
if _on == RekordboxAnlz.SectionTags.beat_grid:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
elif _on == RekordboxAnlz.SectionTags.cues:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
elif _on == RekordboxAnlz.SectionTags.cues_2:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
elif _on == RekordboxAnlz.SectionTags.path:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
elif _on == RekordboxAnlz.SectionTags.song_structure:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
elif _on == RekordboxAnlz.SectionTags.vbr:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_3band_preview:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_3band_scroll:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_color_preview:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_color_scroll:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_preview:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_scroll:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
elif _on == RekordboxAnlz.SectionTags.wave_tiny:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
else:
|
||||
pass
|
||||
self.body._fetch_instances()
|
||||
|
||||
|
||||
class UnknownTag(KaitaiStruct):
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.UnknownTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
pass
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
|
||||
|
||||
class VbrTag(KaitaiStruct):
|
||||
"""Stores an index allowing rapid seeking to particular times
|
||||
within a variable-bitrate audio file.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.VbrTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self._unnamed0 = self._io.read_u4be()
|
||||
self.index = []
|
||||
for i in range(400):
|
||||
self.index.append(self._io.read_u4be())
|
||||
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
for i in range(len(self.index)):
|
||||
pass
|
||||
|
||||
|
||||
|
||||
class Wave3bandPreviewTag(KaitaiStruct):
|
||||
"""The minimalist CDJ-3000 waveform preview image suitable for display
|
||||
above the touch strip for jumping to a track position.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.Wave3bandPreviewTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.len_entry_bytes = self._io.read_u4be()
|
||||
self.len_entries = self._io.read_u4be()
|
||||
self.entries = self._io.read_bytes(self.len_entries * self.len_entry_bytes)
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
|
||||
|
||||
class Wave3bandScrollTag(KaitaiStruct):
|
||||
"""The minimalist CDJ-3000 waveform image suitable for scrolling along
|
||||
as a track plays on newer high-resolution hardware.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.Wave3bandScrollTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.len_entry_bytes = self._io.read_u4be()
|
||||
self.len_entries = self._io.read_u4be()
|
||||
self._unnamed2 = self._io.read_u4be()
|
||||
self.entries = self._io.read_bytes(self.len_entries * self.len_entry_bytes)
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
|
||||
|
||||
class WaveColorPreviewTag(KaitaiStruct):
|
||||
"""A larger, colorful waveform preview image suitable for display
|
||||
above the touch strip for jumping to a track position on newer
|
||||
high-resolution players.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.WaveColorPreviewTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.len_entry_bytes = self._io.read_u4be()
|
||||
self.len_entries = self._io.read_u4be()
|
||||
self._unnamed2 = self._io.read_u4be()
|
||||
self.entries = self._io.read_bytes(self.len_entries * self.len_entry_bytes)
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
|
||||
|
||||
class WaveColorScrollTag(KaitaiStruct):
|
||||
"""A larger, colorful waveform image suitable for scrolling along
|
||||
as a track plays on newer high-resolution hardware. Also
|
||||
contains a higher-resolution blue/white waveform.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.WaveColorScrollTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.len_entry_bytes = self._io.read_u4be()
|
||||
self.len_entries = self._io.read_u4be()
|
||||
self._unnamed2 = self._io.read_u4be()
|
||||
self.entries = self._io.read_bytes(self.len_entries * self.len_entry_bytes)
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
|
||||
|
||||
class WavePreviewTag(KaitaiStruct):
|
||||
"""Stores a waveform preview image suitable for display above
|
||||
the touch strip for jumping to a track position.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.WavePreviewTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.len_data = self._io.read_u4be()
|
||||
self._unnamed1 = self._io.read_u4be()
|
||||
if self._parent.len_tag > self._parent.len_header:
|
||||
pass
|
||||
self.data = self._io.read_bytes(self.len_data)
|
||||
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
if self._parent.len_tag > self._parent.len_header:
|
||||
pass
|
||||
|
||||
|
||||
|
||||
class WaveScrollTag(KaitaiStruct):
|
||||
"""A larger waveform image suitable for scrolling along as a track
|
||||
plays.
|
||||
"""
|
||||
def __init__(self, _io, _parent=None, _root=None):
|
||||
super(RekordboxAnlz.WaveScrollTag, self).__init__(_io)
|
||||
self._parent = _parent
|
||||
self._root = _root
|
||||
self._read()
|
||||
|
||||
def _read(self):
|
||||
self.len_entry_bytes = self._io.read_u4be()
|
||||
self.len_entries = self._io.read_u4be()
|
||||
self._unnamed2 = self._io.read_u4be()
|
||||
self.entries = self._io.read_bytes(self.len_entries * self.len_entry_bytes)
|
||||
|
||||
|
||||
def _fetch_instances(self):
|
||||
pass
|
||||
|
||||
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,2 @@
|
||||
def hello() -> str:
|
||||
return "Hello from maple!"
|
||||
@@ -0,0 +1,3 @@
|
||||
from maple.cli import main
|
||||
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,500 @@
|
||||
"""Audio analysis: BPM + beat grid via mixxx-analyzer, plus waveform envelopes.
|
||||
|
||||
BPM and the beat grid come from `mixxx-analyzer` — Python bindings around
|
||||
Mixxx's own analysis (the Queen Mary tempo tracker) — which produces a
|
||||
CDJ-grade beat grid. Its wheel bundles a native binary, so there's no extra
|
||||
system dependency. ffmpeg still decodes the source to 44.1 kHz mono float32 PCM
|
||||
for the per-column waveform envelopes (PWAV / PWV3 / PWV4 / PWV5).
|
||||
|
||||
The decoded audio never touches disk — ffmpeg's `-f f32le` output is piped
|
||||
straight to numpy.
|
||||
|
||||
Everything an analysis produces is held by the `Analysis` dataclass, composed of
|
||||
`BeatGrid` (BPM + PQTZ beats), `MonoWaveforms` (PWAV/PWV2/PWV3) and
|
||||
`BandWaveforms` (the 3-band colour streams). Build one with
|
||||
`Analysis.analyze(path)` (or `Analysis.from_track(track)`); persist it with
|
||||
`.save(cache_dir)` and read it back with `Analysis.load(cache_dir, isrc)`.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
from base64 import b64decode, b64encode
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, NamedTuple
|
||||
|
||||
import mixxx_analyzer
|
||||
import numpy as np
|
||||
from loguru import logger
|
||||
from scipy.signal import butter, sosfilt
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from maple.track import Track
|
||||
|
||||
SAMPLE_RATE = 44100
|
||||
|
||||
# Waveform envelope dimensions.
|
||||
# PWAV preview is **200 segments × 2 bytes** = 400 bytes — per Beat Link's
|
||||
# reader, height = data[seg*2] & 0x1f, brightness = data[seg*2+1] & 0x07. (An
|
||||
# earlier single-packed-byte reading was wrong; the player decoded it as garbled
|
||||
# height/intensity pairs → "always full, never bright".) PWV2 = 100 columns × 1
|
||||
# byte, 4-bit height. PWV3 (detail/scroll) genuinely IS 1 byte/col.
|
||||
PWAV_SEGMENTS = 200 # PWAV: 200 segments × 2 bytes (height, brightness)
|
||||
PWV2_COLS = 100 # PWV2: 100 single bytes (4-bit height)
|
||||
SCROLL_FPS = 150 # PWV3 / PWV5 / PWV7 = ~150 cols/sec
|
||||
PWAV_HEIGHT_LEVELS = 31 # 5-bit (0-31), bits 0-4
|
||||
PWV2_HEIGHT_LEVELS = 15 # 4-bit (0-15), bits 0-3
|
||||
PWV6_COLS = 1200 # 3-band / colour preview width (fixed by spec)
|
||||
|
||||
CACHE_VERSION = 9 # PWAV paired (2B/seg); treble-based brightness (not crest)
|
||||
|
||||
|
||||
class Band(NamedTuple):
|
||||
"""A band-pass frequency range, in Hz."""
|
||||
lo_hz: int
|
||||
hi_hz: int
|
||||
|
||||
|
||||
# 3-band waveform split: bass (kick + sub), mid (snare, vocals, chords),
|
||||
# high (hats, cymbals).
|
||||
BAND_LOW = Band(20, 200)
|
||||
BAND_MID = Band(200, 2000)
|
||||
BAND_HIGH = Band(2000, 15000)
|
||||
|
||||
|
||||
class Beat(NamedTuple):
|
||||
"""One PQTZ beat-grid row. Unpacks straight into `struct.pack(">HHI", *beat)`."""
|
||||
number: int # 1..4 (cycles from the anchor; no downbeat info)
|
||||
tempo_x100: int # BPM × 100
|
||||
time_ms: int # position in milliseconds
|
||||
|
||||
|
||||
class Bands(NamedTuple):
|
||||
"""A low/mid/high triple of per-column waveform byte-streams."""
|
||||
low: bytes = b""
|
||||
mid: bytes = b""
|
||||
high: bytes = b""
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# mixxx-analyzer + ffmpeg decode #
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def _mixxx_analyze(path: Path) -> mixxx_analyzer.AnalysisResult:
|
||||
"""Run mixxx-analyzer (Mixxx's BPM / beat-grid / key algorithms) on `path`.
|
||||
|
||||
Returns the `AnalysisResult` (fields incl. `bpm` and `beatgrid` — beat
|
||||
positions in seconds). BPM/beat detection is integral to an export, so a
|
||||
failure here propagates rather than degrading to an analysis with no grid."""
|
||||
return mixxx_analyzer.analyze(str(path))
|
||||
|
||||
|
||||
def _project_pqtz(beatgrid: list[float], bpm: float) -> list[Beat]:
|
||||
"""Project a continuous PQTZ grid from mixxx's tempo + phase.
|
||||
|
||||
Returns `[Beat(number 1-4, tempo×100, time_ms), ...]`. mixxx gives an
|
||||
accurate BPM and beat phase, but its raw beat list can have GAPS in quiet
|
||||
sections (e.g. 907 detected beats where a continuous 160 BPM grid has
|
||||
~1010). rekordbox/CDJs expect a gap-free grid — beat-sync and quantise lock
|
||||
to it — so we anchor on mixxx's first beat, walk back to t≥0 to cover the
|
||||
intro, and emit one beat every 60/bpm seconds through to the last detected
|
||||
beat. beat_number cycles 1→4 from the anchor (no downbeat info)."""
|
||||
if not beatgrid or bpm <= 0:
|
||||
return []
|
||||
interval_ms = 60000.0 / bpm
|
||||
first_ms = beatgrid[0] * 1000.0
|
||||
last_ms = beatgrid[-1] * 1000.0
|
||||
n_before = int(first_ms / interval_ms) # preserve phase back to t≥0
|
||||
anchor = first_ms - n_before * interval_ms
|
||||
tempo_x100 = round(bpm * 100)
|
||||
grid: list[Beat] = []
|
||||
i = 0
|
||||
t = anchor
|
||||
while t <= last_ms + interval_ms:
|
||||
grid.append(Beat((i % 4) + 1, tempo_x100, int(round(t))))
|
||||
i += 1
|
||||
t = anchor + i * interval_ms
|
||||
return grid
|
||||
|
||||
|
||||
def _decode_pcm(path: Path) -> np.ndarray:
|
||||
"""Decode `path` to a 1D float32 numpy array of mono PCM at SAMPLE_RATE.
|
||||
|
||||
Raises `RuntimeError` if ffmpeg is missing or fails to decode — waveform
|
||||
extraction is integral, so a failure here is fatal rather than swallowed
|
||||
into an empty array (which would yield a waveform-less ANLZ).
|
||||
"""
|
||||
if not shutil.which("ffmpeg"):
|
||||
raise RuntimeError("ffmpeg not on PATH; cannot extract waveform")
|
||||
proc = subprocess.run(
|
||||
["ffmpeg", "-nostdin", "-v", "error",
|
||||
"-i", str(path),
|
||||
"-f", "f32le", "-ac", "1", "-ar", str(SAMPLE_RATE), "-"],
|
||||
capture_output=True,
|
||||
)
|
||||
if proc.returncode != 0:
|
||||
raise RuntimeError(
|
||||
f"ffmpeg failed for {path.name}: "
|
||||
f"{proc.stderr.decode('utf-8', 'replace').strip()}"
|
||||
)
|
||||
return np.frombuffer(proc.stdout, dtype=np.float32)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Quantisation + per-column binning (the waveform DSP) #
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
# Height: gate near-silence to 0, then a linear map to the level range
|
||||
# (QUANT_GAMMA=1.0). Heights use per-track-normalised RMS loudness, which on the
|
||||
# reference export spreads low (scroll ~5.7/31, overview ~13/31) with no
|
||||
# saturation. Colour/whiteness use a sub-linear lift (COLOR_GAMMA) so quiet
|
||||
# bands still read vivid mid-range instead of near-zero.
|
||||
SILENCE_GATE_PCT = 0.02 # < 2% of peak → 0
|
||||
QUANT_GAMMA = 1.0 # HEIGHT: linear; do NOT lift mids toward max
|
||||
COLOR_GAMMA = 0.4 # COLOUR/whiteness: sub-linear lift
|
||||
# PWAV (preview) height uses an ABSOLUTE loudness scale instead of per-track
|
||||
# normalisation, so a quiet track/section reads short and a loud one tall
|
||||
# (per-track norm always pinned the loudest column to full → "always too high").
|
||||
# height = round(rms * PWAV_HEIGHT_SCALE), clamped to 31. The scale was DERIVED
|
||||
# by least-squares fitting the genuine reference export's PWAV heights against
|
||||
# per-column audio RMS across 3 tracks: 42.2 / 43.0 / 39.9 → 42 (mean err ~2/31;
|
||||
# reference peaks at ~24, i.e. full ≈ RMS 0.74). (PWV3 keeps the per-track scale
|
||||
# — it's good as-is.)
|
||||
PWAV_HEIGHT_SCALE = 42
|
||||
|
||||
|
||||
def _quantise(peaks: np.ndarray, levels: int, peak_max: float | None = None,
|
||||
gamma: float | None = None) -> np.ndarray:
|
||||
"""Gate + (optional gamma) + quantise to 0..levels uint8.
|
||||
|
||||
`peak_max` lets callers pass a SHARED normaliser (e.g. across bands) so the
|
||||
relative energy between channels is preserved; defaults to this array's own
|
||||
max. `gamma` overrides QUANT_GAMMA (use COLOR_GAMMA to lift colour)."""
|
||||
g = QUANT_GAMMA if gamma is None else gamma
|
||||
pm = float(peaks.max()) if peak_max is None else peak_max
|
||||
if pm <= 0:
|
||||
return np.zeros(peaks.size, dtype=np.uint8)
|
||||
gated = np.where(peaks < pm * SILENCE_GATE_PCT, 0.0, peaks)
|
||||
normalised = np.clip(gated / pm, 0.0, 1.0)
|
||||
curved = normalised if g == 1.0 else np.power(normalised, g)
|
||||
return np.clip(np.round(curved * levels), 0, levels).astype(np.uint8)
|
||||
|
||||
|
||||
def _col_rms(values: np.ndarray, n_cols: int) -> np.ndarray:
|
||||
"""Per-column RMS over n_cols evenly-split chunks (loudness, not peak)."""
|
||||
chunks = np.array_split(values.astype(np.float64), n_cols)
|
||||
return np.array([float(np.sqrt(np.mean(c * c))) if len(c) else 0.0 for c in chunks])
|
||||
|
||||
|
||||
def _env_peaks(env: np.ndarray, n_cols: int) -> np.ndarray:
|
||||
chunks = np.array_split(env, n_cols)
|
||||
return np.array([float(c.max()) if len(c) else 0.0 for c in chunks])
|
||||
|
||||
|
||||
def _packed_mono(samples: np.ndarray, hi_env: np.ndarray, n_cols: int,
|
||||
height_levels: int = PWAV_HEIGHT_LEVELS,
|
||||
abs_scale: float | None = None) -> bytes:
|
||||
"""One byte per column = (whiteness << 5) | height — the real PWAV/PWV3
|
||||
on-disk format. whiteness = lifted treble (high-band) peak.
|
||||
|
||||
height: when `abs_scale` is given, an ABSOLUTE loudness map
|
||||
`round(rms * abs_scale)` (quiet → short, loud → tall — used by PWAV); when
|
||||
None, per-track-normalised RMS via `_quantise` (loudest column → full —
|
||||
used by PWV3)."""
|
||||
if n_cols <= 0 or samples.size == 0:
|
||||
return bytes(n_cols)
|
||||
rms = _col_rms(np.abs(samples), n_cols)
|
||||
if abs_scale is not None:
|
||||
height = np.clip(np.round(rms * abs_scale), 0, height_levels).astype(np.uint8)
|
||||
else:
|
||||
height = _quantise(rms, height_levels)
|
||||
white = _quantise(_env_peaks(hi_env, n_cols), 7, gamma=COLOR_GAMMA) # lifted
|
||||
white = np.where(height > 0, white, 0).astype(np.uint8) # silence → no white
|
||||
return ((white << 5) | height).astype(np.uint8).tobytes()
|
||||
|
||||
|
||||
def _packed_preview(samples: np.ndarray, hi_env: np.ndarray,
|
||||
segments: int = PWAV_SEGMENTS) -> bytes:
|
||||
"""PWAV preview — **2 bytes per segment** (NOT one packed byte). This is the
|
||||
format Beat Link / the CDJ actually read:
|
||||
|
||||
even byte: bits 0-4 = height (0-31) ← data[seg*2] & 0x1f
|
||||
odd byte: bits 0-2 = brightness (0-7) ← data[seg*2+1] & 0x07 (>=5 → cyan)
|
||||
|
||||
height = absolute loudness `round(rms*PWAV_HEIGHT_SCALE)` (fit to the
|
||||
reference: full ≈ RMS 0.74). brightness = the same treble (high-band)
|
||||
whiteness PWV3 uses — it matches the reference brightness distribution
|
||||
(~14-38% ≥5) far better than a crest factor (which sat ≥5 on ~all columns).
|
||||
Whiteness is mirrored into the high 3 bits of both bytes, matching the
|
||||
reference layout (even≈odd).
|
||||
"""
|
||||
if segments <= 0 or samples.size == 0:
|
||||
return bytes(segments * 2)
|
||||
rms = _col_rms(np.abs(samples), segments)
|
||||
height = np.clip(np.round(rms * PWAV_HEIGHT_SCALE), 0, 31).astype(np.uint8)
|
||||
# brightness: modest treble level (linear, no lift). The reference's exact
|
||||
# brightness channel doesn't track any computable feature (uncorrelated with
|
||||
# treble/height/crest across reference tracks), so this just gives some
|
||||
# transient/treble sparkle without washing the whole preview cyan.
|
||||
bright = _quantise(_env_peaks(hi_env, segments), 7)
|
||||
bright = np.where(height > 0, bright, 0).astype(np.uint8) # silence → no bright
|
||||
even = ((bright << 5) | height).astype(np.uint8) # &0x1f → height
|
||||
odd = ((bright << 5) | bright).astype(np.uint8) # &0x07 → brightness
|
||||
out = np.empty(segments * 2, dtype=np.uint8)
|
||||
out[0::2] = even
|
||||
out[1::2] = odd
|
||||
return out.tobytes()
|
||||
|
||||
|
||||
def _packed_tiny(samples: np.ndarray, n_cols: int) -> bytes:
|
||||
"""PWV2: one byte per column, 4-bit height (0-15) in the low nibble."""
|
||||
if n_cols <= 0 or samples.size == 0:
|
||||
return bytes(n_cols)
|
||||
return _quantise(_col_rms(np.abs(samples), n_cols), PWV2_HEIGHT_LEVELS).astype(np.uint8).tobytes()
|
||||
|
||||
|
||||
def _bin_bands(lo_env: np.ndarray, mid_env: np.ndarray, hi_env: np.ndarray,
|
||||
n_cols: int, shared: bool, levels: int = 255) -> Bands:
|
||||
"""Bin three band envelopes (per-column PEAK) into n_cols, lifted by
|
||||
COLOR_GAMMA. `shared=True` normalises all three by one SHARED peak → natural
|
||||
bass→treble rolloff (PWV4 overview). `shared=False` normalises each band to
|
||||
its OWN peak → vivid balanced colour where sustained bands read strong
|
||||
(PWV5 colour scroll)."""
|
||||
lp, mp, hp = _env_peaks(lo_env, n_cols), _env_peaks(mid_env, n_cols), _env_peaks(hi_env, n_cols)
|
||||
if shared:
|
||||
s = max(float(lp.max()), float(mp.max()), float(hp.max()), 1e-9)
|
||||
nlo = nmd = nhi = s
|
||||
else:
|
||||
nlo, nmd, nhi = None, None, None
|
||||
return Bands(
|
||||
_quantise(lp, levels, nlo, gamma=COLOR_GAMMA).tobytes(),
|
||||
_quantise(mp, levels, nmd, gamma=COLOR_GAMMA).tobytes(),
|
||||
_quantise(hp, levels, nhi, gamma=COLOR_GAMMA).tobytes(),
|
||||
)
|
||||
|
||||
|
||||
def _band_envelopes(samples: np.ndarray, sr: int) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
|
||||
"""Per-sample low/mid/high envelopes (4-pole Butterworth band-pass + abs)."""
|
||||
def env(band: Band) -> np.ndarray:
|
||||
sos = butter(4, [band.lo_hz, band.hi_hz], btype="band", fs=sr, output="sos")
|
||||
return np.abs(sosfilt(sos, samples)).astype(np.float32)
|
||||
return env(BAND_LOW), env(BAND_MID), env(BAND_HIGH)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# ISRC-keyed cache paths #
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def _sanitise_isrc(isrc: str) -> str:
|
||||
"""Reduce an ISRC to a filename-safe form (uppercase, alnum only)."""
|
||||
return "".join(c for c in (isrc or "").upper() if c.isalnum())
|
||||
|
||||
|
||||
def cache_path_for(cache_dir: Path, isrc: str) -> Path | None:
|
||||
"""Return the path the cache file for `isrc` lives at, or None if no isrc."""
|
||||
s = _sanitise_isrc(isrc)
|
||||
if not s:
|
||||
return None
|
||||
return cache_dir / f"{s}.json"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Composed analysis dataclasses #
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
@dataclass
|
||||
class BeatGrid:
|
||||
"""BPM + the projected PQTZ beat grid. `bpm == 0.0` means unknown — there is
|
||||
no `None` here, an absent grid is simply empty."""
|
||||
|
||||
bpm: float = 0.0
|
||||
beats: list[Beat] = field(default_factory=list)
|
||||
|
||||
@property
|
||||
def tempo_x100(self) -> int:
|
||||
"""BPM × 100, the integer tempo every PQTZ/PQT2 row carries."""
|
||||
return round(self.bpm * 100)
|
||||
|
||||
@classmethod
|
||||
def from_mixxx(cls, res: mixxx_analyzer.AnalysisResult) -> "BeatGrid":
|
||||
"""Build from a mixxx `AnalysisResult`. `_project_pqtz` tolerates an
|
||||
empty beat list, so a result with no usable tempo/phase yields an empty
|
||||
grid (bpm 0.0) rather than raising."""
|
||||
bpm = float(res.bpm) if res.bpm else 0.0
|
||||
beats = _project_pqtz(list(res.beatgrid), bpm) if res.beatgrid else []
|
||||
return cls(bpm=bpm, beats=beats)
|
||||
|
||||
|
||||
@dataclass
|
||||
class MonoWaveforms:
|
||||
"""The single-channel waveform streams.
|
||||
|
||||
`preview` = PWAV (200 segments × 2 bytes), `tiny` = PWV2 (100 × 1 byte,
|
||||
4-bit height), `scroll` = PWV3 (~150 cols/sec, packed whiteness|height)."""
|
||||
|
||||
preview: bytes = b""
|
||||
tiny: bytes = b""
|
||||
scroll: bytes = b""
|
||||
|
||||
@classmethod
|
||||
def from_audio(cls, samples: np.ndarray, hi_env: np.ndarray, scroll_cols: int) -> "MonoWaveforms":
|
||||
return cls(
|
||||
preview=_packed_preview(samples, hi_env, PWAV_SEGMENTS),
|
||||
tiny=_packed_tiny(samples, PWV2_COLS),
|
||||
scroll=_packed_mono(samples, hi_env, scroll_cols, PWAV_HEIGHT_LEVELS),
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class BandWaveforms:
|
||||
"""The 3-band colour streams. `preview` is SHARED-normalised (energy rolloff,
|
||||
feeds PWV4/PWV6); `scroll` is per-band-normalised (vivid, feeds PWV5/PWV7)."""
|
||||
|
||||
preview: Bands = Bands()
|
||||
scroll: Bands = Bands()
|
||||
|
||||
@classmethod
|
||||
def from_envelopes(cls, lo_env: np.ndarray, mid_env: np.ndarray,
|
||||
hi_env: np.ndarray, scroll_cols: int) -> "BandWaveforms":
|
||||
return cls(
|
||||
preview=_bin_bands(lo_env, mid_env, hi_env, PWV6_COLS, shared=True),
|
||||
scroll=_bin_bands(lo_env, mid_env, hi_env, scroll_cols, shared=False),
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Analysis:
|
||||
"""A track's full audio analysis: BPM + beat grid and every waveform stream
|
||||
the rekordbox ANLZ writer needs. Build with `Analysis.analyze(path)` or
|
||||
`Analysis.from_track(track)`; cache with `.save()` / `Analysis.load()`."""
|
||||
|
||||
isrc: str = ""
|
||||
grid: BeatGrid = field(default_factory=BeatGrid)
|
||||
mono: MonoWaveforms = field(default_factory=MonoWaveforms)
|
||||
bands: BandWaveforms = field(default_factory=BandWaveforms)
|
||||
|
||||
# ---- views ----------------------------------------------------------- #
|
||||
|
||||
@property
|
||||
def bpm(self) -> float:
|
||||
return self.grid.bpm
|
||||
|
||||
@property
|
||||
def beats(self) -> list[Beat]:
|
||||
return self.grid.beats
|
||||
|
||||
# ---- construction ---------------------------------------------------- #
|
||||
|
||||
@classmethod
|
||||
def analyze(cls, path: Path, *, isrc: str = "") -> "Analysis":
|
||||
"""One-pass decode + analysis of an audio file.
|
||||
|
||||
BPM + beat grid come from mixxx-analyzer (independent of the waveform
|
||||
decode, so a track still gets its grid even if ffmpeg fails). Always
|
||||
returns a populated `Analysis` — empty waveforms when ffmpeg can't
|
||||
decode the source."""
|
||||
grid = BeatGrid.from_mixxx(_mixxx_analyze(path))
|
||||
samples = _decode_pcm(path)
|
||||
if samples.size == 0:
|
||||
return cls(isrc=isrc, grid=grid)
|
||||
scroll_cols = max(1, int(round(samples.size / SAMPLE_RATE * SCROLL_FPS)))
|
||||
# Band envelopes first — the high band feeds the mono whiteness channel,
|
||||
# and all three feed the colour sections.
|
||||
lo_env, mid_env, hi_env = _band_envelopes(samples, SAMPLE_RATE)
|
||||
return cls(
|
||||
isrc=isrc,
|
||||
grid=grid,
|
||||
mono=MonoWaveforms.from_audio(samples, hi_env, scroll_cols),
|
||||
bands=BandWaveforms.from_envelopes(lo_env, mid_env, hi_env, scroll_cols),
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_track(cls, track: "Track") -> "Analysis":
|
||||
"""Seed `isrc` from the track and compute the rest from `track.path`.
|
||||
|
||||
Returns a bare (isrc-only) `Analysis` when the track has no audio
|
||||
downloaded yet (`track.path is None`)."""
|
||||
if track.path is None:
|
||||
return cls(isrc=track.isrc)
|
||||
return cls.analyze(track.path, isrc=track.isrc)
|
||||
|
||||
# ---- cache I/O ------------------------------------------------------- #
|
||||
|
||||
def to_cache(self) -> dict:
|
||||
"""Serialise to the on-disk JSON payload (bytes → base64). Mirrors the
|
||||
`CACHE_VERSION` v9 layout so existing caches stay readable."""
|
||||
return {
|
||||
"version": CACHE_VERSION,
|
||||
"isrc": _sanitise_isrc(self.isrc),
|
||||
"bpm": self.grid.bpm or None,
|
||||
"beats": [list(b) for b in self.grid.beats],
|
||||
"preview_b64": b64encode(self.mono.preview).decode("ascii"),
|
||||
"tiny_b64": b64encode(self.mono.tiny).decode("ascii"),
|
||||
"scroll_b64": b64encode(self.mono.scroll).decode("ascii"),
|
||||
# 3-band envelopes — kept separate from the mono streams so callers
|
||||
# can ask for whichever they need without re-decoding.
|
||||
"preview_low_b64": b64encode(self.bands.preview.low).decode("ascii"),
|
||||
"preview_mid_b64": b64encode(self.bands.preview.mid).decode("ascii"),
|
||||
"preview_high_b64": b64encode(self.bands.preview.high).decode("ascii"),
|
||||
"scroll_low_b64": b64encode(self.bands.scroll.low).decode("ascii"),
|
||||
"scroll_mid_b64": b64encode(self.bands.scroll.mid).decode("ascii"),
|
||||
"scroll_high_b64": b64encode(self.bands.scroll.high).decode("ascii"),
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def from_cache(cls, d: dict) -> "Analysis | None":
|
||||
"""Rebuild from a `to_cache()` payload, or None if the version is stale."""
|
||||
if d.get("version") != CACHE_VERSION:
|
||||
return None
|
||||
return cls(
|
||||
isrc=d.get("isrc") or "",
|
||||
grid=BeatGrid(
|
||||
bpm=float(d.get("bpm") or 0.0),
|
||||
beats=[Beat(*b) for b in d.get("beats", [])],
|
||||
),
|
||||
mono=MonoWaveforms(
|
||||
preview=b64decode(d.get("preview_b64") or ""),
|
||||
tiny=b64decode(d.get("tiny_b64") or ""),
|
||||
scroll=b64decode(d.get("scroll_b64") or ""),
|
||||
),
|
||||
bands=BandWaveforms(
|
||||
preview=Bands(
|
||||
b64decode(d.get("preview_low_b64") or ""),
|
||||
b64decode(d.get("preview_mid_b64") or ""),
|
||||
b64decode(d.get("preview_high_b64") or ""),
|
||||
),
|
||||
scroll=Bands(
|
||||
b64decode(d.get("scroll_low_b64") or ""),
|
||||
b64decode(d.get("scroll_mid_b64") or ""),
|
||||
b64decode(d.get("scroll_high_b64") or ""),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
def save(self, cache_dir: Path) -> Path | None:
|
||||
"""Persist this analysis under `cache_dir`, keyed by ISRC. No-op
|
||||
(returns None) when the ISRC is empty."""
|
||||
target = cache_path_for(cache_dir, self.isrc)
|
||||
if target is None:
|
||||
return None
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(json.dumps(self.to_cache(), separators=(",", ":")))
|
||||
return target
|
||||
|
||||
@classmethod
|
||||
def load(cls, cache_dir: Path, isrc: str) -> "Analysis | None":
|
||||
"""Return the cached analysis for `isrc`, or None if
|
||||
absent / stale / unreadable."""
|
||||
target = cache_path_for(cache_dir, isrc)
|
||||
if target is None or not target.exists():
|
||||
return None
|
||||
try:
|
||||
return cls.from_cache(json.loads(target.read_text()))
|
||||
except (json.JSONDecodeError, OSError, ValueError) as e:
|
||||
logger.warning("analysis cache read failed for {}: {}", isrc, e)
|
||||
return None
|
||||
@@ -0,0 +1,499 @@
|
||||
"""Generate a rekordbox ANLZ0000.{DAT,EXT} pair.
|
||||
|
||||
ANLZ0000.DAT — what CDJ-900-era through CDJ-3000 firmware reads to display
|
||||
the loaded-deck BPM, the touch-strip preview waveform, and the cue list:
|
||||
|
||||
PPTH player-relative audio path
|
||||
PVBR variable-bitrate seek index (zero-filled placeholder; matches the
|
||||
shape rekordbox writes for files where the seek index isn't useful)
|
||||
PQTZ beat grid (per-beat: beat_number, tempo*100, time_ms)
|
||||
PWAV 400-byte blue/white touch-strip preview
|
||||
PWV2 100-byte "tiny" overview waveform
|
||||
PCOB memory-cue list (empty placeholder)
|
||||
PCOB hot-cue list (empty placeholder)
|
||||
|
||||
ANLZ0000.EXT — extended data; we write a minimal PPTH-only file so newer
|
||||
firmware that opens .EXT as part of its load flow finds a valid container.
|
||||
Color / 3-band waveforms (PWV4/PWV5/PWV6/PWV7) belong in .EXT/.2EX and
|
||||
aren't generated yet.
|
||||
|
||||
Field discoveries documented by byte-diffing maple's output against a real
|
||||
rekordbox export at data/q1-rekordbox/.
|
||||
|
||||
All multi-byte fields in ANLZ files are BIG-endian, unlike the pdb.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from struct import pack
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from maple.analyze import Analysis, Beat
|
||||
|
||||
# Real rekordbox PMAI header carries three non-zero u4 markers in the bytes
|
||||
# after the standard PMAI/len_header/len_file triple. Verified by hex-dump
|
||||
# of data/q1-rekordbox/PIONEER/USBANLZ/P020/0002E10A/ANLZ0000.{DAT,EXT,2EX}
|
||||
# — all three files share the same 16-byte trailer.
|
||||
PMAI_HEADER_TRAILER = pack(">IIII", 0x00000001, 0x00010000, 0x00010000, 0x00000000)
|
||||
ANLZ_HEADER_LEN = 28 # PMAI + 2 size u4s + the 16-byte trailer above
|
||||
|
||||
|
||||
def _section_pth(path: str) -> bytes:
|
||||
"""PPTH section: u4 len_path, then UTF-16BE bytes (null-terminated)."""
|
||||
utf16 = path.encode("utf-16-be") + b"\x00\x00"
|
||||
len_path = len(utf16)
|
||||
body = pack(">I", len_path) + utf16
|
||||
fourcc = b"PPTH"
|
||||
len_header = 16 # fourcc + 2 u4 size fields + len_path u4
|
||||
len_tag = 12 + len(body)
|
||||
return fourcc + pack(">II", len_header, len_tag) + body
|
||||
|
||||
|
||||
def _section_pvbr() -> bytes:
|
||||
"""PVBR section: 400-entry VBR seek index. We write a zero-filled
|
||||
placeholder — rekordbox accepts this for fixed-bitrate sources and it
|
||||
matches the section's presence in real exports."""
|
||||
fourcc = b"PVBR"
|
||||
len_header = 16 # prelude(12) + u4 unnamed0(4)
|
||||
# 400 u4 seek index + 4 trailer bytes = 1604 bytes of variable data
|
||||
# (verified by len_tag=1620 in the real Q1.1 export).
|
||||
body_var = b"\x00" * 1604
|
||||
unnamed0 = b"\x00\x00\x00\x00"
|
||||
len_tag = len_header + len(body_var) # 16 + 1604 = 1620
|
||||
return fourcc + pack(">II", len_header, len_tag) + unnamed0 + body_var
|
||||
|
||||
|
||||
def _section_pqtz(beats: list[Beat]) -> bytes:
|
||||
"""PQTZ: BeatGridTag — per-beat (beat_number, tempo×100, time_ms).
|
||||
|
||||
The CDJ's loaded-deck BPM readout, tempo slider and sync engine all
|
||||
read from this section. Real rekordbox sets `_unnamed1=0x00080000`
|
||||
(was 0 in maple's earlier writer — likely the cause of the deck-side
|
||||
BPM display staying blank even when the grid was otherwise valid).
|
||||
"""
|
||||
fourcc = b"PQTZ"
|
||||
len_header = 24 # prelude(12) + 3 u4 body-header fields
|
||||
num_beats = len(beats)
|
||||
body = pack(">III", 0x00000000, 0x00080000, num_beats)
|
||||
for bn, tempo_x100, t_ms in beats:
|
||||
body += pack(">HHI", bn, tempo_x100, t_ms)
|
||||
len_tag = 12 + len(body)
|
||||
return fourcc + pack(">II", len_header, len_tag) + body
|
||||
|
||||
|
||||
def _section_pwav(preview: bytes) -> bytes:
|
||||
"""PWAV: WavePreviewTag — 400-byte touch-strip preview, **400 columns ×
|
||||
1 byte**, each byte = (whiteness << 5) | height (3-bit whiteness, 5-bit
|
||||
height). `preview` is produced already in that form by
|
||||
`maple.analyze._packed_mono`. (The earlier "200 height/intensity col-pairs"
|
||||
reading was wrong — the player decoded those as garbled half-columns with
|
||||
no whiteness.)
|
||||
"""
|
||||
fourcc = b"PWAV"
|
||||
len_header = 20
|
||||
len_tag = len_header + len(preview)
|
||||
return fourcc + pack(">IIII", len_header, len_tag, len(preview), 0x00010000) + preview
|
||||
|
||||
|
||||
def _section_pwv2(tiny: bytes) -> bytes:
|
||||
"""PWV2: 100-byte tiny preview — 100 columns × 1 byte, 4-bit height
|
||||
(bits 0-3, 0-15) in the low nibble. `tiny` is produced by
|
||||
`maple.analyze._packed_tiny`."""
|
||||
fourcc = b"PWV2"
|
||||
len_header = 20
|
||||
len_tag = len_header + len(tiny)
|
||||
return fourcc + pack(">IIII", len_header, len_tag, len(tiny), 0x00010000) + tiny
|
||||
|
||||
|
||||
def _section_pcob(cue_list_type: int) -> bytes:
|
||||
"""PCOB: CueTag — cue/loop list. Real exports always include both
|
||||
sections in a fixed order — hot-cues (type=1) FIRST, then memory-cues
|
||||
(type=0) — and use `0xffffffff` as the memory_count sentinel for
|
||||
empty lists. Body: u4 type, 2 bytes unnamed, u2 num_cues, u4
|
||||
memory_count. Verified against data/q1-rekordbox/.../ANLZ0000.DAT
|
||||
byte-for-byte."""
|
||||
fourcc = b"PCOB"
|
||||
len_header = 24 # prelude(12) + body fixed prefix(12)
|
||||
body = pack(">I", cue_list_type) + b"\x00\x00" + pack(">HI", 0, 0xFFFFFFFF)
|
||||
len_tag = 12 + len(body)
|
||||
return fourcc + pack(">II", len_header, len_tag) + body
|
||||
|
||||
|
||||
def _section_pwv3(scroll: bytes) -> bytes:
|
||||
"""PWV3: WaveScrollTag — full-track mono scroll waveform. Written
|
||||
into ANLZ0000.EXT (real rekordbox puts PWV3 there, not in .DAT).
|
||||
The loaded-deck scroll display reads from here."""
|
||||
fourcc = b"PWV3"
|
||||
len_header = 24
|
||||
len_entry_bytes = 1
|
||||
len_entries = len(scroll)
|
||||
len_tag = len_header + len(scroll)
|
||||
return fourcc + pack(">IIIII", len_header, len_tag, len_entry_bytes, len_entries, 0x00960000) + scroll
|
||||
|
||||
|
||||
def _section_pco2(cue_list_type: int) -> bytes:
|
||||
"""PCO2: ExtendedCueTag — empty extended cue list. Real exports ship
|
||||
both type=1 (hot) and type=0 (memory) variants in .EXT alongside the
|
||||
PCOB sections. Body: u4 type, u4 reserved."""
|
||||
fourcc = b"PCO2"
|
||||
len_header = 20 # prelude(12) + body fixed prefix(8)
|
||||
body = pack(">II", cue_list_type, 0)
|
||||
len_tag = 12 + len(body)
|
||||
return fourcc + pack(">II", len_header, len_tag) + body
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Colour + 3-band waveform sections (live in .EXT / .2EX) #
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def _section_pwv4(low: bytes, mid: bytes, high: bytes) -> bytes:
|
||||
"""PWV4: WaveColorPreviewTag — colour overview, 1200 cols × 6 bytes/col:
|
||||
|
||||
byte 0/1 : whiteness/brightness (= overall)
|
||||
byte 2 : overall (full-spectrum) energy
|
||||
byte 3 : bass (low band)
|
||||
byte 4 : mid (mid band)
|
||||
byte 5 : treble (high band)
|
||||
|
||||
`low/mid/high` arrive SHARED-normalised (0-255); `overall = max(bands)`."""
|
||||
fourcc = b"PWV4"
|
||||
len_header = 24
|
||||
n = min(len(low), len(mid), len(high))
|
||||
lo, md, hi = bytearray(low[:n]), bytearray(mid[:n]), bytearray(high[:n])
|
||||
body = bytearray(n * 6)
|
||||
for i in range(n):
|
||||
overall = max(lo[i], md[i], hi[i])
|
||||
body[i * 6 + 0] = overall
|
||||
body[i * 6 + 1] = overall
|
||||
body[i * 6 + 2] = overall
|
||||
body[i * 6 + 3] = lo[i] # bass
|
||||
body[i * 6 + 4] = md[i] # mid
|
||||
body[i * 6 + 5] = hi[i] # treble
|
||||
len_tag = len_header + len(body)
|
||||
return fourcc + pack(">IIIII", len_header, len_tag, 6, n, 0x00000000) + bytes(body)
|
||||
|
||||
|
||||
def _section_pwv5(scroll: bytes, low: bytes, mid: bytes, high: bytes) -> bytes:
|
||||
"""PWV5: WaveColorScrollTag — colour scroll, u16 BE per col:
|
||||
|
||||
fedcba9876543210
|
||||
RRRGGGBBBHHHHH--
|
||||
|
||||
Colour **R=treble, G=mid, B=bass** (per spec). Bands arrive per-band
|
||||
normalised 0-255 → 3-bit via >>5. Height is the broadband value from the
|
||||
mono `scroll` byte's low 5 bits, so colour and mono scrolls share a height.
|
||||
"""
|
||||
fourcc = b"PWV5"
|
||||
len_header = 24
|
||||
n = min(len(scroll), len(low), len(mid), len(high))
|
||||
body = bytearray(n * 2)
|
||||
for i in range(n):
|
||||
r = high[i] >> 5 # red = treble
|
||||
g = mid[i] >> 5 # green = mid
|
||||
b = low[i] >> 5 # blue = bass
|
||||
h = scroll[i] & 0x1F # broadband height (0-31), shared with PWV3
|
||||
word = (r & 0x7) << 13 | (g & 0x7) << 10 | (b & 0x7) << 7 | (h & 0x1F) << 2
|
||||
body[i * 2] = (word >> 8) & 0xFF
|
||||
body[i * 2 + 1] = word & 0xFF
|
||||
len_tag = len_header + len(body)
|
||||
return fourcc + pack(">IIIII", len_header, len_tag, 2, n, 0x00960305) + bytes(body)
|
||||
|
||||
|
||||
def _section_pwv6(low: bytes, mid: bytes, high: bytes) -> bytes:
|
||||
"""PWV6: Wave3bandPreviewTag (CDJ-3000). 3 bytes/col, order **mid,
|
||||
high, low** per the docs (NOT low-mid-high). Fixed 1200 cols."""
|
||||
fourcc = b"PWV6"
|
||||
len_header = 20
|
||||
n = min(len(low), len(mid), len(high))
|
||||
body = bytearray(n * 3)
|
||||
for i in range(n):
|
||||
body[i * 3 + 0] = mid[i]
|
||||
body[i * 3 + 1] = high[i]
|
||||
body[i * 3 + 2] = low[i]
|
||||
len_entry_bytes = 3
|
||||
len_entries = n
|
||||
len_tag = len_header + len(body)
|
||||
return fourcc + pack(">IIII", len_header, len_tag, len_entry_bytes, len_entries) + bytes(body)
|
||||
|
||||
|
||||
def _section_pwv7(low: bytes, mid: bytes, high: bytes) -> bytes:
|
||||
"""PWV7: Wave3bandScrollTag (CDJ-3000). Same 3-byte mid/high/low
|
||||
encoding as PWV6 but variable length at ~150 cols/sec."""
|
||||
fourcc = b"PWV7"
|
||||
len_header = 24
|
||||
n = min(len(low), len(mid), len(high))
|
||||
body = bytearray(n * 3)
|
||||
for i in range(n):
|
||||
body[i * 3 + 0] = mid[i]
|
||||
body[i * 3 + 1] = high[i]
|
||||
body[i * 3 + 2] = low[i]
|
||||
len_entry_bytes = 3
|
||||
len_entries = n
|
||||
len_tag = len_header + len(body)
|
||||
return fourcc + pack(">IIIII", len_header, len_tag, len_entry_bytes, len_entries, 0x00960000) + bytes(body)
|
||||
|
||||
|
||||
def _section_pqt2(beats: list[Beat]) -> bytes:
|
||||
"""PQT2: extended beat grid (CDJ-3000+). Layout is only partially
|
||||
documented; the format isn't in the kaitai schema or the deepsymmetry
|
||||
spec page. The hex dump of a real Q1 PQT2 shows a 56-byte header
|
||||
carrying num_beats, first_beat_ms, mid_ms, end_ms (matching the
|
||||
corresponding PQTZ values), then per-beat 2-byte entries.
|
||||
|
||||
We write a placeholder: header fields mirror PQTZ where we know
|
||||
them, the rest zero-filled. Per-beat entries are 2 bytes of zero
|
||||
each (encoding unknown). Empirically the CDJ-2000NXS2+ uses the
|
||||
presence of PQT2 — independently of its content — as a gate for the
|
||||
deck-side waveform / BPM display.
|
||||
"""
|
||||
fourcc = b"PQT2"
|
||||
len_header = 56
|
||||
num_beats = len(beats)
|
||||
first_beat_ms = beats[0][2] if beats else 0
|
||||
if num_beats >= 2:
|
||||
interval = beats[-1][2] - beats[-2][2]
|
||||
end_ms = beats[-1][2] + interval
|
||||
else:
|
||||
end_ms = first_beat_ms
|
||||
mid_ms = end_ms // 2
|
||||
# 44 bytes of body header (11 × u4). First few fields known from the
|
||||
# real Q1 PQT2 dump; later u4s look like an analyser-timestamp that
|
||||
# we can't reproduce, so we zero them.
|
||||
body_header = pack(
|
||||
">11I",
|
||||
0, # unknown1
|
||||
0x01000002, # unknown2 — observed constant in real exports
|
||||
0, # unknown3
|
||||
0x00013200, # unknown4 — observed constant
|
||||
first_beat_ms,
|
||||
mid_ms,
|
||||
end_ms,
|
||||
num_beats,
|
||||
0, # unknown9 (date/timestamp; not reproduced)
|
||||
0, # unknown10
|
||||
0, # unknown11
|
||||
)
|
||||
entries = b"\x00\x00" * num_beats
|
||||
body = body_header + entries
|
||||
len_tag = 12 + len(body)
|
||||
return fourcc + pack(">II", len_header, len_tag) + body
|
||||
|
||||
|
||||
def _section_pwvc() -> bytes:
|
||||
"""PWVC: 20-byte tag at the tail of `.2EX`. Undocumented. Real exports
|
||||
always include it; we emit a zero-filled placeholder of the expected
|
||||
length so the file ends with the same section list."""
|
||||
fourcc = b"PWVC"
|
||||
len_header = 14
|
||||
body = b"\x00" * 8
|
||||
len_tag = 12 + len(body)
|
||||
return fourcc + pack(">II", len_header, len_tag) + body
|
||||
|
||||
|
||||
# XOR mask used by rekordbox to obfuscate the PSSI body. Mask is offset
|
||||
# per byte by the (unmasked) `len_entries` value before XOR'ing in.
|
||||
_PSSI_MASK = bytes.fromhex("CBE1EEFAE5EEADEEE9D2E9EBE1E9F3E8E9F4E1")
|
||||
|
||||
|
||||
def _section_pssi(num_beats: int) -> bytes:
|
||||
"""PSSI: SongStructureTag — phrase analysis. We emit an empty (0
|
||||
phrases) placeholder so the CDJ-3000 sees a valid PSSI and doesn't
|
||||
blank its phrase indicators. Mood=2 (mid), bank=0 (default).
|
||||
|
||||
Per spec, bytes 0x12 onwards are XOR-masked with `_PSSI_MASK[i] +
|
||||
len_entries` (mod 256). Since `len_entries == 0`, the mask itself is
|
||||
the XOR key.
|
||||
"""
|
||||
fourcc = b"PSSI"
|
||||
len_header = 32
|
||||
# Unmasked body fields:
|
||||
len_entry_bytes = 0x18
|
||||
len_entries = 0 # no phrases
|
||||
mood = 2 # mid
|
||||
unknown_u4 = 0
|
||||
end_beat = num_beats # final phrase end beat = last beat in PQTZ
|
||||
unknown2_u4 = 0
|
||||
bank = 0 # default
|
||||
unknown3 = 0
|
||||
|
||||
unmasked = pack(">I", len_entry_bytes) # offset 0x0c-0x0f (NOT masked per spec)
|
||||
masked_block = (
|
||||
pack(">H", len_entries) # 0x10-0x11
|
||||
+ pack(">H", mood) # 0x12-0x13
|
||||
+ pack(">I", unknown_u4) # 0x14-0x17
|
||||
+ pack(">H", end_beat) # 0x18-0x19
|
||||
+ pack(">I", unknown2_u4) # 0x1a-0x1d
|
||||
+ pack(">B", bank) # 0x1e
|
||||
+ pack(">B", unknown3) # 0x1f
|
||||
)
|
||||
# Apply mask: spec says "all bytes from offset 0x12 onwards" — that's
|
||||
# the 2-byte mood field onward. Skip the first 2 bytes of masked_block
|
||||
# (which are len_entries at 0x10).
|
||||
masked_out = bytearray(masked_block[:2]) # len_entries unmasked
|
||||
for i, b in enumerate(masked_block[2:]):
|
||||
masked_out.append(b ^ ((_PSSI_MASK[i % len(_PSSI_MASK)] + len_entries) & 0xFF))
|
||||
|
||||
body = unmasked + bytes(masked_out)
|
||||
# No entries to append (len_entries=0).
|
||||
len_tag = 12 + len(body)
|
||||
return fourcc + pack(">II", len_header, len_tag) + body
|
||||
|
||||
|
||||
def _pmai_header(file_len: int) -> bytes:
|
||||
"""28-byte PMAI header — fourcc + sizes + the canonical trailer that
|
||||
rekordbox writes (was zero-padded in maple's earlier output)."""
|
||||
return b"PMAI" + pack(">II", ANLZ_HEADER_LEN, file_len) + PMAI_HEADER_TRAILER
|
||||
|
||||
|
||||
def build_dat(rb_path: str, analysis: Analysis | None) -> bytes:
|
||||
"""Build ANLZ0000.DAT — the file every CDJ reads on track load.
|
||||
|
||||
Section order matches rekordbox's own output: PPTH, PVBR, PQTZ, PWAV,
|
||||
PWV2, PCOB(memory), PCOB(hot).
|
||||
"""
|
||||
sections = [_section_pth(rb_path), _section_pvbr()]
|
||||
if analysis is not None:
|
||||
if analysis.beats:
|
||||
sections.append(_section_pqtz(analysis.beats))
|
||||
if analysis.mono.preview:
|
||||
sections.append(_section_pwav(analysis.mono.preview))
|
||||
if analysis.mono.tiny:
|
||||
sections.append(_section_pwv2(analysis.mono.tiny))
|
||||
sections.append(_section_pcob(1)) # hot cues (empty) — real writes hot-cues first
|
||||
sections.append(_section_pcob(0)) # memory cues (empty)
|
||||
body = b"".join(sections)
|
||||
return _pmai_header(ANLZ_HEADER_LEN + len(body)) + body
|
||||
|
||||
|
||||
def build_ext(rb_path: str, analysis: Analysis | None = None) -> bytes:
|
||||
"""Build ANLZ0000.EXT — extended analysis data the loaded-deck reads.
|
||||
|
||||
Section order matches the real Q1 reference:
|
||||
PPTH, PWV3 (mono scroll), PCOB×2, PCO2×2,
|
||||
[PQT2 — skipped, format undocumented],
|
||||
PWV5 (colour scroll), PWV4 (colour preview), PSSI (empty).
|
||||
"""
|
||||
sections = [_section_pth(rb_path)]
|
||||
if analysis is not None and analysis.mono.scroll:
|
||||
sections.append(_section_pwv3(analysis.mono.scroll))
|
||||
sections.append(_section_pcob(1))
|
||||
sections.append(_section_pcob(0))
|
||||
sections.append(_section_pco2(1))
|
||||
sections.append(_section_pco2(0))
|
||||
if analysis is not None and analysis.beats:
|
||||
# PQT2 between PCO2 and PWV5 — matches real-export section order
|
||||
# and (empirically) gates the CDJ's deck-side waveform/BPM display.
|
||||
sections.append(_section_pqt2(analysis.beats))
|
||||
if analysis is not None:
|
||||
scroll = analysis.mono.scroll
|
||||
s_lo, s_md, s_hi = analysis.bands.scroll
|
||||
if scroll and s_lo and s_md and s_hi:
|
||||
sections.append(_section_pwv5(scroll, s_lo, s_md, s_hi))
|
||||
p_lo, p_md, p_hi = analysis.bands.preview
|
||||
if p_lo and p_md and p_hi:
|
||||
sections.append(_section_pwv4(p_lo, p_md, p_hi))
|
||||
if analysis.beats:
|
||||
sections.append(_section_pssi(len(analysis.beats)))
|
||||
body = b"".join(sections)
|
||||
return _pmai_header(ANLZ_HEADER_LEN + len(body)) + body
|
||||
|
||||
|
||||
def build_2ex(rb_path: str, analysis: Analysis | None = None) -> bytes:
|
||||
"""Build ANLZ0000.2EX — CDJ-3000 3-band waveforms.
|
||||
|
||||
Section order matches the real Q1 reference:
|
||||
PPTH, PWV7 (3-band scroll), PWV6 (3-band preview), PWVC (skipped).
|
||||
"""
|
||||
sections = [_section_pth(rb_path)]
|
||||
if analysis is not None:
|
||||
s_lo, s_md, s_hi = analysis.bands.scroll
|
||||
if s_lo and s_md and s_hi:
|
||||
sections.append(_section_pwv7(s_lo, s_md, s_hi))
|
||||
p_lo, p_md, p_hi = analysis.bands.preview
|
||||
if p_lo and p_md and p_hi:
|
||||
sections.append(_section_pwv6(p_lo, p_md, p_hi))
|
||||
sections.append(_section_pwvc()) # 20-byte trailer real exports always include
|
||||
body = b"".join(sections)
|
||||
return _pmai_header(ANLZ_HEADER_LEN + len(body)) + body
|
||||
|
||||
|
||||
def build_anlz(rb_path: str, source_audio: Path | None = None) -> bytes:
|
||||
"""Compatibility wrapper for callers that want the .DAT bytes only.
|
||||
|
||||
Analysis failures propagate — a broken analysis is fatal, not swallowed."""
|
||||
analysis = Analysis.analyze(source_audio) if source_audio is not None else None
|
||||
return build_dat(rb_path, analysis)
|
||||
|
||||
|
||||
def build_minimal_anlz(rb_path: str) -> bytes:
|
||||
"""PPTH-only-plus-empties-DAT. Used when no source audio is available."""
|
||||
return build_dat(rb_path, analysis=None)
|
||||
|
||||
|
||||
def _isrc_for(path: Path) -> str | None:
|
||||
"""Best-effort ISRC read from a file tag (mp3/flac/m4a)."""
|
||||
try:
|
||||
from mutagen import File as MutagenFile
|
||||
audio = MutagenFile(path, easy=True)
|
||||
if audio is None or audio.tags is None:
|
||||
return None
|
||||
v = audio.tags.get("isrc")
|
||||
if not v:
|
||||
return None
|
||||
return str(v[0]).strip() or None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def write_anlz(
|
||||
rb_path: str,
|
||||
dest: Path,
|
||||
source_audio: Path | None = None,
|
||||
analysis_cache_dir: Path | None = None,
|
||||
) -> Analysis | None:
|
||||
"""Write ANLZ0000.DAT (and a minimal ANLZ0000.EXT alongside).
|
||||
|
||||
`dest` is the .DAT path. The .EXT companion is written next to it
|
||||
automatically — every real CDJ export ships both files.
|
||||
|
||||
Returns the `Analysis` (cache hit or fresh mixxx run) or None when
|
||||
no source audio was given.
|
||||
"""
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
analysis: Analysis | None = None
|
||||
cache_hit = False
|
||||
if source_audio is not None:
|
||||
isrc = _isrc_for(source_audio) if analysis_cache_dir else None
|
||||
if isrc and analysis_cache_dir:
|
||||
cached = Analysis.load(analysis_cache_dir, isrc)
|
||||
if cached is not None:
|
||||
analysis = cached
|
||||
cache_hit = True
|
||||
logger.info("anlz: cache hit for {} (isrc {})", source_audio.name, isrc)
|
||||
if analysis is None:
|
||||
# Not a cache hit — analyse fresh. Failures propagate (a broken
|
||||
# analysis is fatal, not swallowed into a grid-less ANLZ).
|
||||
analysis = Analysis.analyze(source_audio, isrc=isrc or "")
|
||||
# Populate the cache on every successful fresh analysis so subsequent
|
||||
# exports (and collection runs) can skip the ffmpeg+mixxx pass. The
|
||||
# cache is the single source of truth — populated by whichever
|
||||
# pipeline (`just collection` or `just export`) sees a track first.
|
||||
if analysis is not None and not cache_hit and analysis_cache_dir is not None and isrc:
|
||||
try:
|
||||
cp = analysis.save(analysis_cache_dir)
|
||||
if cp is not None:
|
||||
logger.info("anlz: cached analysis for {} → {}", source_audio.name, cp)
|
||||
except Exception as e:
|
||||
logger.warning("could not write analysis cache for {}: {}", source_audio.name, e)
|
||||
|
||||
dest.write_bytes(build_dat(rb_path, analysis))
|
||||
ext_path = dest.with_suffix(".EXT")
|
||||
ext_path.write_bytes(build_ext(rb_path, analysis))
|
||||
# .2EX carries the CDJ-3000's 3-band waveforms (PWV6/PWV7). Older
|
||||
# firmware ignores the file; CDJ-3000 reads it for the three-band
|
||||
# strip display.
|
||||
two_ex_path = dest.with_suffix(".2EX")
|
||||
two_ex_path.write_bytes(build_2ex(rb_path, analysis))
|
||||
return analysis
|
||||
@@ -0,0 +1,70 @@
|
||||
"""Extract embedded album art and write Pioneer-format JPEGs.
|
||||
|
||||
Pioneer's CDJ/XDJ players expect two square JPEGs per artwork:
|
||||
/PIONEER/Artwork/{bucket:05d}/a{id}.jpg -> 80x80
|
||||
/PIONEER/Artwork/{bucket:05d}/a{id}_m.jpg -> 240x240
|
||||
where bucket = id // 20 + 1. The pdb's artwork_row.path points at the
|
||||
small (80x80) variant; the player auto-resolves the `_m` companion.
|
||||
"""
|
||||
import hashlib
|
||||
import io
|
||||
from pathlib import Path
|
||||
|
||||
from mutagen.id3 import ID3, ID3NoHeaderError
|
||||
from PIL import Image
|
||||
|
||||
SMALL = 80
|
||||
LARGE = 240
|
||||
|
||||
|
||||
def _read_apic(mp3: Path) -> bytes | None:
|
||||
try:
|
||||
tags = ID3(mp3)
|
||||
except ID3NoHeaderError:
|
||||
return None
|
||||
frames = tags.getall("APIC")
|
||||
if not frames:
|
||||
return None
|
||||
return frames[0].data
|
||||
|
||||
|
||||
def _bucket(artwork_id: int) -> str:
|
||||
return f"{artwork_id // 20 + 1:05d}"
|
||||
|
||||
|
||||
def _resize_jpeg(data: bytes, size: int) -> bytes:
|
||||
img = Image.open(io.BytesIO(data))
|
||||
if img.mode != "RGB":
|
||||
img = img.convert("RGB")
|
||||
img = img.resize((size, size), Image.LANCZOS)
|
||||
out = io.BytesIO()
|
||||
img.save(out, format="JPEG", quality=85)
|
||||
return out.getvalue()
|
||||
|
||||
|
||||
def extract_artwork(mp3: Path, usb_root: Path, next_id: int, cache: dict[str, int]) -> tuple[int, str | None]:
|
||||
"""Return (artwork_id, file_path_for_pdb). artwork_id is 0 if no art."""
|
||||
data = _read_apic(mp3)
|
||||
if data is None:
|
||||
return 0, None
|
||||
sha = hashlib.sha256(data).hexdigest()
|
||||
cached = cache.get(sha)
|
||||
if cached is not None:
|
||||
return cached, None
|
||||
|
||||
aid = next_id
|
||||
bucket = _bucket(aid)
|
||||
rel_small = f"/PIONEER/Artwork/{bucket}/a{aid}.jpg"
|
||||
rel_large = f"/PIONEER/Artwork/{bucket}/a{aid}_m.jpg"
|
||||
abs_small = usb_root / rel_small.lstrip("/")
|
||||
abs_large = usb_root / rel_large.lstrip("/")
|
||||
abs_small.parent.mkdir(parents=True, exist_ok=True)
|
||||
# Always overwrite — the artwork_id↔file mapping is recomputed every
|
||||
# export, and a stale a<id>.jpg left over from a previous run would
|
||||
# silently render the wrong cover on the CDJ. The intra-run dedup
|
||||
# (cache hit at the top of this function) already prevents wasteful
|
||||
# rewrites within one export.
|
||||
abs_small.write_bytes(_resize_jpeg(data, SMALL))
|
||||
abs_large.write_bytes(_resize_jpeg(data, LARGE))
|
||||
cache[sha] = aid
|
||||
return aid, rel_small
|
||||
@@ -0,0 +1,528 @@
|
||||
"""maple CLI: scan a directory of audio + m3u8 playlists and build a rekordbox USB."""
|
||||
import argparse
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
from collections import defaultdict
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from maple.analyze import Analysis, cache_path_for
|
||||
from maple.anlz import _isrc_for, write_anlz
|
||||
from maple.artwork import _bucket as _artwork_bucket
|
||||
from maple.artwork import extract_artwork
|
||||
from maple.convert import convert_flac_to_mp3, verify_duration
|
||||
from maple.copy import stage_audio, staged_path
|
||||
from maple.log import configure as configure_logging
|
||||
from maple.meta import read_tags
|
||||
from maple.model import Album, Artist, Artwork, Database, Genre, PlaylistEntry, PlaylistNode, Track
|
||||
from maple.enrich import enrich_from_manifest, load_spotify_manifest, write_bpm_tag
|
||||
from maple.pdb.writer import write_pdb
|
||||
from maple.scan import scan_audio, scan_playlists
|
||||
from maple.validate import TrackIssue, log_issues, validate_track
|
||||
|
||||
# Genre tag delimiters seen across our sources: Spotify-derived uses
|
||||
# ';', European labels often use ',', some store '/'-separated lists.
|
||||
_GENRE_SPLIT_RE = re.compile(r"\s*[;,/]\s*")
|
||||
|
||||
|
||||
def usbanlz_location(usb_path: str) -> tuple[int, int]:
|
||||
"""rekordbox's ANLZ folder derivation from a USB-relative track path.
|
||||
|
||||
Returns ``(bucket, anlz_id)`` for ``/PIONEER/USBANLZ/P{bucket:03X}/{anlz_id:08X}/``.
|
||||
|
||||
`usb_path` is the forward-slashed, leading-slash USB-relative path that the
|
||||
pdb stores as `file_path` (e.g. ``/Contents/Artist/Album/01 Title.mp3``).
|
||||
|
||||
CRITICAL: the CDJ/XDJ firmware recomputes this from `file_path` and reads a
|
||||
track's analysis from the resulting folder, *ignoring* the pdb's
|
||||
`analyze_path` string. So maple must write its ANLZ files here and nowhere
|
||||
else, or the player shows no waveform/beatgrid. Verified 1002/1002 against a
|
||||
genuine rekordbox export and confirmed on XDJ-RX2 hardware.
|
||||
|
||||
`anlz_id` is a polynomial hash of the path's UTF-16 code units mod 0x30D43
|
||||
(200003 — hence ids never exceed 0x30D42); `bucket` is a fixed bit-extraction
|
||||
from that id. (Algorithm: github.com/Radexito/DjManager protocol.md §2.)
|
||||
"""
|
||||
h = 0
|
||||
for ch in usb_path:
|
||||
h = (h * 0x34F5501D + ord(ch) * 0x93B6) & 0xFFFFFFFF
|
||||
anlz_id = h % 0x30D43
|
||||
bucket = (
|
||||
((((((((((anlz_id >> 2) & 0x4000) | (anlz_id & 0x2000)) >> 3)
|
||||
| (anlz_id & 0x200)) >> 1) | (anlz_id & 0xC0)) >> 3)
|
||||
| (anlz_id & 0x4)) >> 1) | (anlz_id & 0x1)
|
||||
)
|
||||
return bucket, anlz_id
|
||||
|
||||
|
||||
def split_genre_tags(raw: str) -> list[str]:
|
||||
"""Tokenise a possibly multi-value genre string into normalised parts.
|
||||
|
||||
Each token's casing is preserved as-given. Empty parts are dropped.
|
||||
Duplicates within the same source string are collapsed (first wins).
|
||||
"""
|
||||
if not raw:
|
||||
return []
|
||||
seen: set[str] = set()
|
||||
out: list[str] = []
|
||||
for token in _GENRE_SPLIT_RE.split(raw):
|
||||
t = token.strip()
|
||||
if not t:
|
||||
continue
|
||||
key = t.casefold()
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
out.append(t)
|
||||
return out
|
||||
|
||||
|
||||
def build(input_dir: Path, usb_root: Path, with_playlists: bool = True) -> tuple[Database, set[Path]]:
|
||||
"""Build the rekordbox USB layout under `usb_root`.
|
||||
|
||||
Returns the populated `Database` plus the set of absolute paths that
|
||||
were written (audio files, ANLZ files, artwork JPEGs). The caller adds
|
||||
the pdb path and calls `prune_stale` to remove anything else under
|
||||
`usb_root` left over from a previous export.
|
||||
"""
|
||||
db = Database()
|
||||
today = date.today().isoformat()
|
||||
|
||||
artist_index: dict[str, int] = {}
|
||||
album_index: dict[tuple[int, str], int] = {}
|
||||
# Key by casefolded name so "Dub Techno" and "dub techno" collapse to one row.
|
||||
genre_index: dict[str, int] = {}
|
||||
# The user-facing genre→tracks map. Populated as tracks are processed.
|
||||
genre_to_tracks: dict[str, list[int]] = defaultdict(list)
|
||||
track_by_src: dict[Path, int] = {}
|
||||
artwork_cache: dict[str, int] = {}
|
||||
# USBANLZ folder occupancy. Two distinct paths can hash to the same
|
||||
# P{bucket}/{id} folder (a hash collision; also happens for genuine
|
||||
# duplicate paths). rekordbox keeps both by bumping the filename to
|
||||
# ANLZ0001.DAT, ANLZ0002.DAT, … inside the shared folder (verified: a
|
||||
# real export had `Celestica` + `Soak It Up` co-located at P049/00030991).
|
||||
# Mirror that so colliding tracks don't clobber each other's analysis.
|
||||
anlz_folder_seen: dict[str, int] = defaultdict(int)
|
||||
written: set[Path] = set()
|
||||
|
||||
audio_files = scan_audio(input_dir)
|
||||
logger.info("scanned {}: {} audio file(s)", input_dir, len(audio_files))
|
||||
|
||||
# ---------- Spotify enrichment (symmetric with `just collection`) -------- #
|
||||
# Per the validation convention: both pipelines must be capable of
|
||||
# bringing a track to "valid" state. Validate first; only files that
|
||||
# actually need it pay the enrichment cost (mutagen tag rewrite +
|
||||
# possible artwork fetch). Repeated exports of an already-valid
|
||||
# collection skip the enrichment loop entirely.
|
||||
by_key, by_isrc = load_spotify_manifest(input_dir / "_metadata")
|
||||
cache_dir = input_dir / "_analysis"
|
||||
if by_key or by_isrc:
|
||||
logger.info(
|
||||
"loaded spotify manifest: {} (artist,title) keys, {} ISRCs",
|
||||
len(by_key), len(by_isrc),
|
||||
)
|
||||
enriched = 0
|
||||
skipped = 0
|
||||
for p in audio_files:
|
||||
if not validate_track(p, analysis_cache_dir=cache_dir):
|
||||
skipped += 1
|
||||
continue
|
||||
if enrich_from_manifest(p, by_key, by_isrc, analysis_cache_dir=cache_dir) is not None:
|
||||
enriched += 1
|
||||
logger.info(
|
||||
"enrichment: {} enriched, {} already valid (skipped)",
|
||||
enriched, skipped,
|
||||
)
|
||||
else:
|
||||
logger.warning(
|
||||
"no spotify manifest at {} — tracks won't be Spotify-enriched. "
|
||||
"Run `just collection` first to fetch + persist the manifest.",
|
||||
input_dir / "_metadata" / "spotify-tracks.json",
|
||||
)
|
||||
|
||||
t = time.perf_counter()
|
||||
initial_meta = {p: read_tags(p) for p in audio_files}
|
||||
logger.debug("read tags in {:.2f}s", time.perf_counter() - t)
|
||||
|
||||
flac_jobs: list[tuple[Path, Path]] = []
|
||||
for original in audio_files:
|
||||
if original.suffix.lower() == ".flac":
|
||||
m = initial_meta[original]
|
||||
dest_abs, _ = staged_path(usb_root, m.artist, m.album, original.with_suffix(".mp3").name)
|
||||
flac_jobs.append((original, dest_abs))
|
||||
|
||||
def _job(pair: tuple[Path, Path]) -> tuple[Path, Path, bool]:
|
||||
src, dst = pair
|
||||
t0 = time.perf_counter()
|
||||
converted = convert_flac_to_mp3(src, dst)
|
||||
return src, dst, converted, time.perf_counter() - t0
|
||||
|
||||
workers = max(1, os.cpu_count() or 1)
|
||||
if flac_jobs:
|
||||
logger.info("converting {} flac(s) → mp3 v0 (workers={})", len(flac_jobs), workers)
|
||||
t = time.perf_counter()
|
||||
n_converted = 0
|
||||
with ThreadPoolExecutor(max_workers=workers) as ex:
|
||||
for i, (src, dst, converted, dt) in enumerate(ex.map(_job, flac_jobs), 1):
|
||||
tag = "convert" if converted else "cached "
|
||||
logger.info("[{}/{}] {} {} ({:.1f}s)", i, len(flac_jobs), tag, src.relative_to(input_dir), dt)
|
||||
if converted:
|
||||
n_converted += 1
|
||||
verify_duration(src, dst)
|
||||
logger.info(
|
||||
"flac→mp3 pass: {} converted, {} cached in {:.1f}s total",
|
||||
n_converted, len(flac_jobs) - n_converted, time.perf_counter() - t,
|
||||
)
|
||||
|
||||
# Parallel pre-analysis. The per-track build loop below is sequential
|
||||
# (the pdb / artwork-id assignments depend on iteration order), but
|
||||
# `Analysis.analyze` is the slow step and trivially parallel: ffmpeg +
|
||||
# mixxx + scipy each release the GIL, so threads scale linearly with
|
||||
# cores. After this pass every track has its ISRC-keyed analysis
|
||||
# cached, so write_anlz in the build loop is a cache hit.
|
||||
def _dest_for(p: Path) -> Path:
|
||||
m = initial_meta[p]
|
||||
if p.suffix.lower() == ".flac":
|
||||
dest_abs, _ = staged_path(usb_root, m.artist, m.album, p.with_suffix(".mp3").name)
|
||||
else:
|
||||
dest_abs, _ = staged_path(usb_root, m.artist, m.album, p.name)
|
||||
return dest_abs
|
||||
|
||||
# Make sure non-FLAC sources are staged to data/usb first so the
|
||||
# analysis pass operates on the file the CDJ will actually read.
|
||||
for p in audio_files:
|
||||
if p.suffix.lower() != ".flac":
|
||||
m = initial_meta[p]
|
||||
stage_audio(p, usb_root, m.artist, m.album)
|
||||
|
||||
def _ensure_analysis_cache(p: Path) -> bool:
|
||||
"""Populate `<cache>/<isrc>.json` if missing. Returns True iff a
|
||||
fresh analysis actually ran (i.e. cache was cold). Analysis failures
|
||||
propagate — a broken analysis is fatal, not skipped."""
|
||||
dest = _dest_for(p)
|
||||
if not dest.exists():
|
||||
return False
|
||||
isrc = _isrc_for(dest)
|
||||
if not isrc:
|
||||
return False
|
||||
cp = cache_path_for(cache_dir, isrc)
|
||||
if cp is None or cp.exists():
|
||||
return False
|
||||
a = Analysis.analyze(dest, isrc=isrc)
|
||||
a.save(cache_dir)
|
||||
return True
|
||||
|
||||
pre_t0 = time.perf_counter()
|
||||
logger.info("pre-analysing {} tracks (workers={})...", len(audio_files), workers)
|
||||
new_count = 0
|
||||
done = 0
|
||||
with ThreadPoolExecutor(max_workers=workers) as ex:
|
||||
futures = {ex.submit(_ensure_analysis_cache, p): p for p in audio_files}
|
||||
for fut in as_completed(futures):
|
||||
done += 1
|
||||
if fut.result(): # re-raises any worker analysis failure
|
||||
new_count += 1
|
||||
if done % 8 == 0 or done == len(audio_files):
|
||||
logger.info(" [{}/{}] cache misses analysed: {}", done, len(audio_files), new_count)
|
||||
logger.info(
|
||||
"pre-analysis: {} new in {:.1f}s ({} already cached)",
|
||||
new_count, time.perf_counter() - pre_t0, len(audio_files) - new_count,
|
||||
)
|
||||
|
||||
logger.info("building track table ({} files)", len(audio_files))
|
||||
build_t0 = time.perf_counter()
|
||||
validation_issues: list[TrackIssue] = []
|
||||
for track_idx, original in enumerate(audio_files, 1):
|
||||
track_t0 = time.perf_counter()
|
||||
meta = initial_meta[original]
|
||||
artist_id = artist_index.get(meta.artist)
|
||||
if artist_id is None:
|
||||
artist_id = len(db.artists) + 1
|
||||
artist_index[meta.artist] = artist_id
|
||||
db.artists.append(Artist(id=artist_id, name=meta.artist))
|
||||
|
||||
album_key = (artist_id, meta.album)
|
||||
album_id = album_index.get(album_key)
|
||||
if album_id is None:
|
||||
album_id = len(db.albums) + 1
|
||||
album_index[album_key] = album_id
|
||||
db.albums.append(Album(id=album_id, name=meta.album, artist_id=artist_id))
|
||||
|
||||
if original.suffix.lower() == ".flac":
|
||||
dest_abs, file_path = staged_path(usb_root, meta.artist, meta.album, original.with_suffix(".mp3").name)
|
||||
meta = read_tags(dest_abs)
|
||||
else:
|
||||
dest_abs, file_path = stage_audio(original, usb_root, meta.artist, meta.album)
|
||||
written.add(dest_abs.resolve())
|
||||
|
||||
# Re-enrich the STAGED file too — `stage_audio` and the FLAC→MP3
|
||||
# converter cache by hash, so a cached dest may still carry stale
|
||||
# tags from a previous export. Skip when the staged file already
|
||||
# passes validation so re-exports of an unchanged collection don't
|
||||
# re-mutate every track.
|
||||
if (by_key or by_isrc) and validate_track(dest_abs, analysis_cache_dir=cache_dir):
|
||||
if enrich_from_manifest(dest_abs, by_key, by_isrc, analysis_cache_dir=cache_dir) is not None:
|
||||
meta = read_tags(dest_abs)
|
||||
|
||||
next_artwork_id = len(db.artwork) + 1
|
||||
artwork_id, art_file_path = extract_artwork(dest_abs, usb_root, next_artwork_id, artwork_cache)
|
||||
if art_file_path is not None:
|
||||
db.artwork.append(Artwork(id=artwork_id, path=art_file_path))
|
||||
# Track artwork paths for the prune sweep — both this run's writes
|
||||
# and any artwork_cache hits (where extract_artwork returned None
|
||||
# for art_file_path because the jpeg already exists from a prior
|
||||
# iteration). _bucket() is the same id-to-folder mapping artwork.py
|
||||
# uses, so this rebuilds the absolute on-disk path exactly.
|
||||
if artwork_id:
|
||||
art_bucket = _artwork_bucket(artwork_id)
|
||||
for fname in (f"a{artwork_id}.jpg", f"a{artwork_id}_m.jpg"):
|
||||
written.add((usb_root / "PIONEER" / "Artwork" / art_bucket / fname).resolve())
|
||||
|
||||
# Genre: split multi-value tag strings ("dub techno;minimal techno",
|
||||
# "Drum & Bass, Électronique", etc.) into individual genre rows so the
|
||||
# CDJ's Genre filter is usable. The track FK points at the FIRST
|
||||
# (most-specific) token; every other token still gets its own row in
|
||||
# the pdb's genre table so it appears in the browse list, and we
|
||||
# record the {genre→track_ids} map so we can log + reuse it later.
|
||||
primary_genre_id = 0
|
||||
tokens = split_genre_tags(meta.genre)
|
||||
for n, token in enumerate(tokens):
|
||||
key = token.casefold()
|
||||
gid = genre_index.get(key)
|
||||
if gid is None:
|
||||
gid = len(db.genres) + 1
|
||||
genre_index[key] = gid
|
||||
db.genres.append(Genre(id=gid, name=token))
|
||||
if n == 0:
|
||||
primary_genre_id = gid
|
||||
genre_to_tracks[token].append(len(db.tracks) + 1) # this track's id-to-be
|
||||
genre_id = primary_genre_id
|
||||
|
||||
track_id = len(db.tracks) + 1
|
||||
# The USBANLZ folder is derived from the USB-relative `file_path` by
|
||||
# rekordbox's bespoke path hash (see `usbanlz_location`). The CDJ/XDJ
|
||||
# firmware RE-DERIVES this same path from `file_path` and reads the
|
||||
# analysis from there — it IGNORES the pdb's `analyze_path` string —
|
||||
# so the folder MUST match rekordbox exactly or the player silently
|
||||
# shows no waveform/beatgrid. Verified 1002/1002 vs a real export and
|
||||
# on XDJ-RX2 hardware. (Earlier crc32 scheme matched 0/984 → grids
|
||||
# missing.)
|
||||
bucket_n, anlz_id = usbanlz_location(file_path)
|
||||
anlz_folder = f"/PIONEER/USBANLZ/P{bucket_n:03X}/{anlz_id:08X}"
|
||||
anlz_index = anlz_folder_seen[anlz_folder] # 0, then 1, 2, … on collision
|
||||
anlz_folder_seen[anlz_folder] += 1
|
||||
analyze_path = f"{anlz_folder}/ANLZ{anlz_index:04d}.DAT"
|
||||
anlz_abs = (usb_root / analyze_path.lstrip("/")).resolve()
|
||||
# `_analysis/` sits at the input collection root; ISRC-keyed cache
|
||||
# populated during `just collection`. write_anlz reads it before
|
||||
# falling back to a fresh ffmpeg+mixxx pass.
|
||||
analysis = write_anlz(
|
||||
file_path, anlz_abs, source_audio=dest_abs,
|
||||
analysis_cache_dir=input_dir / "_analysis",
|
||||
)
|
||||
written.add(anlz_abs)
|
||||
# .EXT and .2EX sister files are written automatically next to .DAT.
|
||||
written.add(anlz_abs.with_suffix(".EXT"))
|
||||
written.add(anlz_abs.with_suffix(".2EX"))
|
||||
# Prefer the analytically detected BPM over the file tag when we have one.
|
||||
# CDJ browse panel reads pdb.tempo; loaded-deck readout reads the PQTZ
|
||||
# we just wrote — keeping the two in sync avoids the "120.0 in browse,
|
||||
# blank on deck" UX bug.
|
||||
if analysis is not None and analysis.bpm:
|
||||
tempo_x100 = analysis.grid.tempo_x100
|
||||
# Also write the bpm TAG to the file (symmetric with `just
|
||||
# collection`'s _write_bpm_tag). overwrite=False so any
|
||||
# pre-existing label-set BPM is preserved.
|
||||
write_bpm_tag(dest_abs, analysis.bpm, overwrite=False)
|
||||
else:
|
||||
tempo_x100 = meta.tempo
|
||||
db.tracks.append(Track(
|
||||
id=track_id,
|
||||
title=meta.title,
|
||||
artist_id=artist_id,
|
||||
album_id=album_id,
|
||||
filename=Path(file_path).name,
|
||||
file_path=file_path,
|
||||
file_size=meta.file_size,
|
||||
sample_rate=meta.sample_rate,
|
||||
sample_depth=meta.sample_depth,
|
||||
bitrate=meta.bitrate,
|
||||
duration=meta.duration,
|
||||
track_number=meta.track_number,
|
||||
year=meta.year,
|
||||
tempo=tempo_x100,
|
||||
date_added=today,
|
||||
analyze_path=analyze_path,
|
||||
analyze_date=today,
|
||||
artwork_id=artwork_id,
|
||||
genre_id=genre_id,
|
||||
source_path=original,
|
||||
))
|
||||
track_by_src[original.resolve()] = track_id
|
||||
|
||||
# Cross-cutting validation: same checks as `just collection`. The
|
||||
# input audio_path is the staged file under data/usb/ so its tags
|
||||
# are what end up on the player. analysis cache lives at the source
|
||||
# collection root (input_dir/_analysis/<isrc>.json).
|
||||
track_issues = validate_track(
|
||||
dest_abs,
|
||||
analysis_cache_dir=input_dir / "_analysis",
|
||||
)
|
||||
validation_issues.extend(track_issues)
|
||||
|
||||
track_dt = time.perf_counter() - track_t0
|
||||
bpm_str = f"{analysis.bpm:.1f} bpm" if (analysis is not None and analysis.bpm) else f"tag {tempo_x100/100:.1f} bpm"
|
||||
issue_tag = f" [{len(track_issues)} issue(s)]" if track_issues else ""
|
||||
logger.info(
|
||||
"[{}/{}] {} - {} ({}, {:.1f}s){}",
|
||||
track_idx, len(audio_files),
|
||||
meta.artist, meta.title,
|
||||
bpm_str, track_dt, issue_tag,
|
||||
)
|
||||
logger.info(
|
||||
"track table built in {:.1f}s ({} tracks, {} artists, {} albums, "
|
||||
"{} artwork, {} genres)",
|
||||
time.perf_counter() - build_t0,
|
||||
len(db.tracks), len(db.artists), len(db.albums),
|
||||
len(db.artwork), len(db.genres),
|
||||
)
|
||||
# Genre map summary: largest buckets first; helps spot mistagged tracks.
|
||||
if genre_to_tracks:
|
||||
logger.info("genre→tracks map ({} genres):", len(genre_to_tracks))
|
||||
for genre, track_ids in sorted(
|
||||
genre_to_tracks.items(), key=lambda kv: (-len(kv[1]), kv[0])
|
||||
):
|
||||
logger.info(" {:<32s} {:>3} track(s)", genre[:32], len(track_ids))
|
||||
no_genre = sum(1 for t in db.tracks if t.genre_id == 0)
|
||||
if no_genre:
|
||||
logger.warning("{} of {} tracks have no genre tag", no_genre, len(db.tracks))
|
||||
|
||||
# Emit the validation report. Same TrackIssue shape as `just collection`,
|
||||
# surfaced through loguru so the user gets visibility plus a roll-up at
|
||||
# the end.
|
||||
if validation_issues:
|
||||
logger.warning(
|
||||
"{} validation issue(s) across {} track(s)",
|
||||
len(validation_issues),
|
||||
len({i.track.display_name for i in validation_issues}),
|
||||
)
|
||||
log_issues(validation_issues, report=False)
|
||||
else:
|
||||
logger.info("validation: all {} tracks passed", len(audio_files))
|
||||
|
||||
if not with_playlists:
|
||||
return db, written, validation_issues
|
||||
playlists = scan_playlists(input_dir)
|
||||
if playlists:
|
||||
logger.info("scanning {} playlist(s)", len(playlists))
|
||||
for sort_order, (name, paths) in enumerate(playlists.items(), start=1):
|
||||
playlist_id = len(db.playlist_nodes) + 1
|
||||
db.playlist_nodes.append(PlaylistNode(
|
||||
id=playlist_id,
|
||||
parent_id=0,
|
||||
sort_order=sort_order,
|
||||
name=name,
|
||||
is_folder=False,
|
||||
))
|
||||
resolved = 0
|
||||
for entry_index, p in enumerate(paths, start=1):
|
||||
tid = track_by_src.get(p.resolve())
|
||||
if tid is None:
|
||||
logger.warning("playlist {!r} references missing file {}", name, p)
|
||||
continue
|
||||
db.playlist_entries.append(PlaylistEntry(
|
||||
playlist_id=playlist_id,
|
||||
track_id=tid,
|
||||
entry_index=entry_index,
|
||||
))
|
||||
resolved += 1
|
||||
logger.info(" {}: {} entries ({} resolved, {} missing)",
|
||||
name, len(paths), resolved, len(paths) - resolved)
|
||||
|
||||
return db, written, validation_issues
|
||||
|
||||
|
||||
def prune_stale(usb_root: Path, kept: set[Path]) -> tuple[int, int]:
|
||||
"""Delete every file under `usb_root` not in `kept`, then drop empty dirs.
|
||||
|
||||
Returns (files_removed, dirs_removed). Operates only under `usb_root`,
|
||||
so anything outside is untouched. The `kept` set is expected to hold
|
||||
fully-resolved absolute paths.
|
||||
"""
|
||||
files_removed = 0
|
||||
for p in usb_root.rglob("*"):
|
||||
if not p.is_file():
|
||||
continue
|
||||
if p.resolve() in kept:
|
||||
continue
|
||||
try:
|
||||
p.unlink()
|
||||
files_removed += 1
|
||||
except OSError as e:
|
||||
logger.warning("failed to remove {}: {}", p, e)
|
||||
|
||||
# Bottom-up empty-dir removal: deepest first so each parent gets a
|
||||
# chance to become empty once its children are gone.
|
||||
dirs = sorted(
|
||||
(d for d in usb_root.rglob("*") if d.is_dir()),
|
||||
key=lambda d: len(d.parts),
|
||||
reverse=True,
|
||||
)
|
||||
dirs_removed = 0
|
||||
for d in dirs:
|
||||
try:
|
||||
d.rmdir()
|
||||
dirs_removed += 1
|
||||
except OSError:
|
||||
pass # not empty; expected for the directories we still want
|
||||
return files_removed, dirs_removed
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(prog="maple", description="Build a rekordbox USB from a folder of audio + m3u8 playlists.")
|
||||
parser.add_argument("input_dir", nargs="?", type=Path, default=Path("./data/input"))
|
||||
parser.add_argument("output_root", nargs="?", type=Path, default=Path("./data/usb"))
|
||||
parser.add_argument("--no-playlists", action="store_true", help="Skip m3u8 playlist scanning")
|
||||
parser.add_argument("--no-prune", action="store_true", help="Skip the post-build sweep of stale files")
|
||||
parser.add_argument("--strict", action="store_true", help="Fail with non-zero exit if any track fails validation")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
configure_logging()
|
||||
if not args.input_dir.is_dir():
|
||||
parser.error(f"input_dir {args.input_dir} is not a directory")
|
||||
|
||||
overall_t0 = time.perf_counter()
|
||||
logger.info("export: {} → {}", args.input_dir, args.output_root)
|
||||
db, written, validation_issues = build(args.input_dir, args.output_root, with_playlists=not args.no_playlists)
|
||||
|
||||
pdb_path = args.output_root / "PIONEER" / "rekordbox" / "export.pdb"
|
||||
write_pdb(db, pdb_path)
|
||||
written.add(pdb_path.resolve())
|
||||
|
||||
if not args.no_prune:
|
||||
t = time.perf_counter()
|
||||
n_files, n_dirs = prune_stale(args.output_root, written)
|
||||
if n_files or n_dirs:
|
||||
logger.info(
|
||||
"pruned {} stale file(s) + {} empty dir(s) in {:.1f}s",
|
||||
n_files, n_dirs, time.perf_counter() - t,
|
||||
)
|
||||
else:
|
||||
logger.debug("nothing to prune ({:.1f}s)", time.perf_counter() - t)
|
||||
|
||||
logger.info(
|
||||
"done in {:.1f}s: {} tracks, {} artists, {} albums, "
|
||||
"{} playlists ({} entries) → {}",
|
||||
time.perf_counter() - overall_t0,
|
||||
len(db.tracks), len(db.artists), len(db.albums),
|
||||
len(db.playlist_nodes), len(db.playlist_entries), pdb_path,
|
||||
)
|
||||
if args.strict and validation_issues:
|
||||
logger.error("--strict: refusing to succeed with {} validation issue(s)", len(validation_issues))
|
||||
return 1
|
||||
return 0
|
||||
@@ -0,0 +1,38 @@
|
||||
"""Sync data/collection with one or more Spotify playlists.
|
||||
|
||||
Pipeline (one shot, in `maple.collection.cli.main_async`):
|
||||
1. Async-fetch each Spotify playlist via aiohttp (concurrently).
|
||||
2. Diff against the Collection's index (sldl `index.sldl.csv` + tag scan).
|
||||
3. Write the missing rows as a CSV and shell out to `sockseek` to download
|
||||
them (streaming `--progress-json` events back so each completed track
|
||||
updates its playlist's m3u8 immediately).
|
||||
4. Validate every track that now resolves to a file: ffprobe-decodable,
|
||||
duration within 5s of Spotify, fuzzy tag match, embedded cover art.
|
||||
If artwork is missing, fetch the largest image from
|
||||
`track.album.images` and embed it.
|
||||
5. Write one m3u8 per playlist into `<root>/_playlists/<name>.m3u8` with
|
||||
paths relative to the m3u8 file (portable across synced devices).
|
||||
|
||||
Usage:
|
||||
uv run python -m maple.collection <id-or-url ...> [--collection PATH]
|
||||
[--no-download] [--no-validate]
|
||||
|
||||
Env (from .env, loaded by just):
|
||||
MAPLE_SPOTIFY_CLIENT_ID, MAPLE_SPOTIFY_CLIENT_SECRET (spotify.py reads)
|
||||
SOCKSEEK_USER, SOCKSEEK_PASS (forwarded to sockseek)
|
||||
"""
|
||||
# Re-exports are lazy via __getattr__ so importing `maple.collection.index`
|
||||
# (or any other submodule) doesn't drag the whole package — `maple.validate`
|
||||
# depends on `maple.collection.index` and would otherwise circular-import
|
||||
# through `maple.collection.core` → `maple.validate` → here.
|
||||
__all__ = ["Collection", "CollectionIndex", "TrackIssue"]
|
||||
|
||||
|
||||
def __getattr__(name):
|
||||
if name == "Collection" or name == "TrackIssue":
|
||||
from maple.collection.core import Collection, TrackIssue
|
||||
return {"Collection": Collection, "TrackIssue": TrackIssue}[name]
|
||||
if name == "CollectionIndex":
|
||||
from maple.collection.index import CollectionIndex
|
||||
return CollectionIndex
|
||||
raise AttributeError(name)
|
||||
@@ -0,0 +1,4 @@
|
||||
"""Entrypoint for `python -m maple.collection`."""
|
||||
from maple.collection.cli import main
|
||||
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,196 @@
|
||||
"""`just collection` CLI entrypoint.
|
||||
|
||||
End-to-end pipeline: fetch Spotify playlists, diff against the on-disk
|
||||
collection, run sockseek for the missing tracks, validate the result,
|
||||
write per-playlist m3u8s, and emit a report.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import aiohttp
|
||||
from loguru import logger
|
||||
|
||||
from maple.collection.core import Collection, TrackIssue
|
||||
from maple.collection.index import first_artist, norm
|
||||
from maple.enrich import save_spotify_manifest
|
||||
from maple.log import configure as configure_logging
|
||||
from maple.spotify import fetch_playlists
|
||||
|
||||
|
||||
def parse_args(argv: list[str]) -> argparse.Namespace:
|
||||
p = argparse.ArgumentParser(prog="maple.collection", description="Sync data/collection with Spotify playlists.")
|
||||
p.add_argument("ids", nargs="+", help="Spotify playlist ids, URLs, or spotify: URIs")
|
||||
p.add_argument("--collection", help="Override collection root (default: <repo>/data/collection)")
|
||||
p.add_argument("--no-download", action="store_true", help="Skip the sockseek download step")
|
||||
p.add_argument("--no-validate", action="store_true", help="Skip ffprobe/duration/tag/art validation")
|
||||
return p.parse_args(argv)
|
||||
|
||||
|
||||
async def main_async(argv: list[str]) -> int:
|
||||
args = parse_args(argv)
|
||||
configure_logging()
|
||||
collection = Collection(Path(args.collection).resolve()) if args.collection else Collection.from_repo()
|
||||
collection.ensure_dirs()
|
||||
collection.attach_report_sink()
|
||||
logger.info("collection root: {}", collection.root)
|
||||
|
||||
# Startup hygiene: drop unfinished downloads, prune the sldl index, and
|
||||
# rewrite any stale entries out of existing m3u8s.
|
||||
removed_incomplete = collection.remove_unfinished_downloads()
|
||||
if removed_incomplete:
|
||||
logger.warning("removed {} unfinished download(s)", removed_incomplete)
|
||||
kept, removed = collection.validate_index_file()
|
||||
if removed:
|
||||
logger.warning("index: pruned {} row(s) ({} kept)", removed, kept)
|
||||
pruned_pl, rewritten_pl = collection.validate_playlists()
|
||||
if pruned_pl or rewritten_pl:
|
||||
logger.warning(
|
||||
"playlists: pruned {} stale entr(ies), rewrote {} absolute path(s)",
|
||||
pruned_pl, rewritten_pl,
|
||||
)
|
||||
|
||||
playlists = await fetch_playlists(args.ids)
|
||||
total = sum(len(p["tracks"]) for p in playlists)
|
||||
logger.info("fetched {} playlist(s), {} tracks", len(playlists), total)
|
||||
# Persist the full set of Spotify tracks so `just export` (and any
|
||||
# later `just collection` run) can enrich tags without re-fetching.
|
||||
save_spotify_manifest(collection.metadata_dir, playlists)
|
||||
|
||||
collection.refresh_index()
|
||||
logger.info(
|
||||
"existing in collection: {} strict / {} loose keys",
|
||||
len(collection.index.by_strict),
|
||||
len(collection.index.by_loose),
|
||||
)
|
||||
|
||||
missing = collection.find_missing(playlists)
|
||||
logger.info("missing tracks to download: {}", len(missing))
|
||||
|
||||
# Issues accumulate from both the streaming callback (per-track on
|
||||
# completion) and the end-of-run validate sweep. process_track dedups
|
||||
# paths so neither path double-reports.
|
||||
issues: list[TrackIssue] = []
|
||||
|
||||
if missing and not args.no_download:
|
||||
# (artist_norm, title_norm) -> [(playlist_dict, spotify_track)] so the
|
||||
# callback can both rewrite the right m3u8 and reconcile tags against
|
||||
# the canonical Spotify entry (sockseek echoes back only artist/title).
|
||||
track_to_playlists: dict[tuple[str, str], list[tuple[dict, dict]]] = {}
|
||||
for pl in playlists:
|
||||
for t in pl["tracks"]:
|
||||
key = (
|
||||
norm(first_artist(", ".join(t["artists"]))),
|
||||
norm(t["title"]),
|
||||
)
|
||||
track_to_playlists.setdefault(key, []).append((pl, t))
|
||||
|
||||
def _human_size(n: int) -> str:
|
||||
for unit in ("B", "KiB", "MiB", "GiB"):
|
||||
if n < 1024:
|
||||
return f"{n:.0f}{unit}" if unit == "B" else f"{n:.1f}{unit}"
|
||||
n /= 1024
|
||||
return f"{n:.1f}TiB"
|
||||
|
||||
async def on_event(evt: dict) -> None:
|
||||
etype = evt.get("type")
|
||||
data = evt.get("data") or {}
|
||||
|
||||
if etype == "search_start":
|
||||
logger.info(
|
||||
"search: {} - {}",
|
||||
data.get("artist") or "?", data.get("title") or "?",
|
||||
)
|
||||
return
|
||||
|
||||
if etype == "download_start":
|
||||
size = data.get("size") or 0
|
||||
logger.info(
|
||||
"downloading: {} - {} ({} {}, from {})",
|
||||
data.get("artist") or "?",
|
||||
data.get("title") or "?",
|
||||
_human_size(size) if size else "?",
|
||||
(data.get("extension") or "").lstrip(".") or "?",
|
||||
data.get("username") or "?",
|
||||
)
|
||||
return
|
||||
|
||||
if etype != "track_state":
|
||||
return
|
||||
|
||||
outcome = (data.get("terminalOutcome") or "").lower()
|
||||
skip_reason = (data.get("skipReason") or "").lower()
|
||||
failure_reason = (data.get("failureReason") or "").strip()
|
||||
artist = data.get("artist") or ""
|
||||
title = data.get("title") or ""
|
||||
success = outcome == "succeeded" or (
|
||||
outcome == "skipped" and skip_reason == "alreadyexists"
|
||||
)
|
||||
if not success:
|
||||
# Surface failures/skips so the user sees why a track didn't land.
|
||||
if outcome == "skipped":
|
||||
logger.info("skipped: {} - {} ({})", artist, title, skip_reason or "skipped")
|
||||
else:
|
||||
logger.warning(
|
||||
"failed: {} - {} ({})",
|
||||
artist, title, failure_reason or outcome or "unknown",
|
||||
)
|
||||
return
|
||||
|
||||
download_path = data.get("downloadPath")
|
||||
if not download_path:
|
||||
return
|
||||
path = Path(download_path)
|
||||
if not path.exists():
|
||||
logger.debug("track_state success but path missing: {}", download_path)
|
||||
return
|
||||
collection.index.add(artist, title, path)
|
||||
key = (norm(first_artist(artist)), norm(title))
|
||||
affected = track_to_playlists.get(key, [])
|
||||
if not affected:
|
||||
logger.info("downloaded {} - {} ({})", artist, title, path.name)
|
||||
return
|
||||
# Reconcile against the Spotify track (writes Spotify tags
|
||||
# + records any mismatches). One spotify track might appear
|
||||
# in several playlists; process_track dedups via collection._reconciled.
|
||||
spotify_track = affected[0][1]
|
||||
new_issues = collection.process_track(
|
||||
affected[0][0]["name"] or affected[0][0]["id"],
|
||||
spotify_track,
|
||||
path,
|
||||
)
|
||||
for issue in new_issues:
|
||||
logger.warning(
|
||||
"{}: {}: {}", issue.track.display_name, issue.field, issue.reason,
|
||||
)
|
||||
issues.extend(new_issues)
|
||||
for pl, _ in affected:
|
||||
out = collection.write_m3u8_for(pl)
|
||||
logger.info(
|
||||
"downloaded {} - {} → updated {}",
|
||||
artist, title, out.name,
|
||||
)
|
||||
|
||||
rc = await collection.download(missing, on_event=on_event)
|
||||
if rc != 0:
|
||||
logger.warning("sockseek exited with {}", rc)
|
||||
collection.refresh_index()
|
||||
|
||||
if not args.no_validate:
|
||||
timeout = aiohttp.ClientTimeout(total=30)
|
||||
async with aiohttp.ClientSession(timeout=timeout) as session:
|
||||
issues.extend(await collection.validate(session, playlists))
|
||||
|
||||
for out in collection.write_m3u8s(playlists, issues):
|
||||
logger.info("wrote {}", out)
|
||||
|
||||
collection.report_issues(issues)
|
||||
logger.info("{} issue(s) recorded in {}", len(issues), collection.report_path)
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
return asyncio.run(main_async(sys.argv[1:]))
|
||||
@@ -0,0 +1,608 @@
|
||||
"""`Collection` dataclass: the on-disk music collection rooted at `root`.
|
||||
|
||||
Owns dir layout, the in-memory `CollectionIndex` of (artist, title) →
|
||||
file paths, startup validation (drop unfinished downloads, prune the
|
||||
sldl index, prune stale m3u8 entries), the missing-track diff, validation
|
||||
of downloaded tracks, the per-playlist m3u8 writer, and the loguru file
|
||||
sink used for `collection-report.txt`.
|
||||
|
||||
Sockseek subprocess plumbing lives in `maple.collection.sockseek`; this
|
||||
module just calls into it from `Collection.download()`.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import csv
|
||||
import os
|
||||
import re
|
||||
from collections.abc import Awaitable, Callable
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
import aiohttp
|
||||
from loguru import logger
|
||||
from mutagen import File as MutagenFile
|
||||
|
||||
from maple.analyze import Analysis
|
||||
from maple.enrich import write_spotify_tags as _shared_write_spotify_tags
|
||||
from maple.validate import TrackIssue as _ValidateTrackIssue
|
||||
from maple.validate import validate_track as _validate_track
|
||||
from maple.collection.index import (
|
||||
CollectionIndex,
|
||||
first_artist,
|
||||
fuzzy_ratio,
|
||||
norm,
|
||||
norm_loose,
|
||||
)
|
||||
from maple.collection.sockseek import download as run_sockseek
|
||||
from maple.collection.validate import (
|
||||
ARTWORK_CONCURRENCY,
|
||||
DURATION_TOLERANCE_S,
|
||||
embed_art,
|
||||
fetch_jpeg,
|
||||
ffprobe_ok,
|
||||
file_duration,
|
||||
has_embedded_art,
|
||||
)
|
||||
from maple.scan import AUDIO_EXTS
|
||||
|
||||
BPM_DIVERGENCE_WARN = 0.05 # warn when aubio vs existing tag differs >5%
|
||||
|
||||
|
||||
# Re-export the shared validator's TrackIssue so callers that imported it
|
||||
# from maple.collection.core continue to work. There is one TrackIssue
|
||||
# definition in the codebase — `maple.validate.TrackIssue` — and both the
|
||||
# `just collection` and `just export` flows produce/consume the same shape.
|
||||
TrackIssue = _ValidateTrackIssue
|
||||
|
||||
|
||||
_M3U_BAD = re.compile(r"[\\/\x00]+")
|
||||
|
||||
|
||||
def _sanitise_name(name: str) -> str:
|
||||
s = _M3U_BAD.sub("_", name or "playlist").strip().lstrip(".")
|
||||
return (s or "playlist")[:200]
|
||||
|
||||
|
||||
@dataclass
|
||||
class Collection:
|
||||
"""A music collection rooted at `root` on disk."""
|
||||
|
||||
root: Path
|
||||
index: CollectionIndex = field(default_factory=CollectionIndex)
|
||||
# Paths whose tags have already been reconciled against Spotify this run.
|
||||
# `process_track` consults this so the on_event hot path and the
|
||||
# end-of-run `validate` sweep don't double-handle the same file.
|
||||
_reconciled: set[Path] = field(default_factory=set, repr=False)
|
||||
|
||||
# --- paths ------------------------------------------------------------- #
|
||||
|
||||
@classmethod
|
||||
def from_repo(cls) -> "Collection":
|
||||
"""Default location: `<repo>/data/collection`."""
|
||||
here = Path(__file__).resolve()
|
||||
# src/maple/collection/core.py → src/maple/collection → src/maple → src → maple-py → repo
|
||||
return cls((here.parents[4] / "data" / "collection").resolve())
|
||||
|
||||
@property
|
||||
def metadata_dir(self) -> Path:
|
||||
return self.root / "_metadata"
|
||||
|
||||
@property
|
||||
def playlists_dir(self) -> Path:
|
||||
return self.root / "_playlists"
|
||||
|
||||
@property
|
||||
def index_csv(self) -> Path:
|
||||
return self.metadata_dir / "index.sldl.csv"
|
||||
|
||||
@property
|
||||
def report_path(self) -> Path:
|
||||
return self.metadata_dir / "collection-report.txt"
|
||||
|
||||
@property
|
||||
def analysis_dir(self) -> Path:
|
||||
"""`<root>/_analysis/<isrc>.json` — cached aubio output keyed on ISRC,
|
||||
consumed by `just export` to skip the decode+detect pass."""
|
||||
return self.root / "_analysis"
|
||||
|
||||
def ensure_dirs(self) -> None:
|
||||
self.root.mkdir(parents=True, exist_ok=True)
|
||||
self.metadata_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.playlists_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.analysis_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# --- startup validation ----------------------------------------------- #
|
||||
|
||||
def remove_unfinished_downloads(self) -> int:
|
||||
"""Delete sldl's `*.incomplete` leftover files. Returns count removed."""
|
||||
if not self.root.exists():
|
||||
return 0
|
||||
count = 0
|
||||
for p in self.root.rglob("*.incomplete"):
|
||||
if "_metadata" in p.parts or "_playlists" in p.parts:
|
||||
continue
|
||||
try:
|
||||
p.unlink()
|
||||
count += 1
|
||||
except OSError as e:
|
||||
logger.warning("failed to remove {}: {}", p, e)
|
||||
return count
|
||||
|
||||
def validate_index_file(self) -> tuple[int, int]:
|
||||
"""Rewrite `index.sldl.csv` dropping duplicate rows and orphaned-path rows.
|
||||
|
||||
Returns (kept, removed). Failure rows (no filepath) are kept so
|
||||
sldl can still honour `--skip-not-found` on the next run.
|
||||
"""
|
||||
if not self.index_csv.exists():
|
||||
return (0, 0)
|
||||
with self.index_csv.open(newline="", encoding="utf-8", errors="replace") as fh:
|
||||
rows = list(csv.reader(fh))
|
||||
if len(rows) <= 1:
|
||||
return (max(0, len(rows) - 1), 0)
|
||||
header = rows[0]
|
||||
cols = {h.strip().lower(): i for i, h in enumerate(header)}
|
||||
path_i = cols.get("filepath", 0)
|
||||
|
||||
kept: list[list[str]] = []
|
||||
seen: set[tuple[str, ...]] = set()
|
||||
removed = 0
|
||||
for row in rows[1:]:
|
||||
key = tuple(row)
|
||||
if key in seen:
|
||||
removed += 1
|
||||
continue
|
||||
seen.add(key)
|
||||
rel = row[path_i] if path_i < len(row) else ""
|
||||
if rel:
|
||||
p = self.root / rel if not Path(rel).is_absolute() else Path(rel)
|
||||
if not p.exists():
|
||||
removed += 1
|
||||
continue
|
||||
kept.append(row)
|
||||
|
||||
if removed > 0:
|
||||
with self.index_csv.open("w", newline="", encoding="utf-8") as fh:
|
||||
w = csv.writer(fh)
|
||||
w.writerow(header)
|
||||
w.writerows(kept)
|
||||
return (len(kept), removed)
|
||||
|
||||
def validate_playlists(self) -> tuple[int, int]:
|
||||
"""Sweep `_playlists/*.m3u8`. Two passes per entry:
|
||||
|
||||
1. Lines whose file doesn't resolve are replaced with a
|
||||
`# NOT FOUND: <title>` comment so the playlist still tells you
|
||||
what's missing.
|
||||
2. Lines that resolve but were written with an absolute path get
|
||||
rewritten relative to the m3u8's parent dir — playlists need to
|
||||
survive being synced to another machine where the collection
|
||||
sits at a different root.
|
||||
|
||||
Returns (pruned, rewritten)."""
|
||||
if not self.playlists_dir.exists():
|
||||
return (0, 0)
|
||||
total_pruned = 0
|
||||
total_rewritten = 0
|
||||
for m3u in self.playlists_dir.glob("*.m3u8"):
|
||||
lines = m3u.read_text(encoding="utf-8").splitlines()
|
||||
out: list[str] = []
|
||||
pending_extinf: str | None = None
|
||||
pruned_here = 0
|
||||
rewritten_here = 0
|
||||
for line in lines:
|
||||
s = line.strip()
|
||||
if not s:
|
||||
out.append(line)
|
||||
continue
|
||||
if s.startswith("#EXTINF"):
|
||||
pending_extinf = line
|
||||
continue
|
||||
if s.startswith("#"):
|
||||
if pending_extinf is not None:
|
||||
out.append(pending_extinf)
|
||||
pending_extinf = None
|
||||
out.append(line)
|
||||
continue
|
||||
# Bare file path (relative to m3u or absolute).
|
||||
was_absolute = Path(s).is_absolute()
|
||||
target = Path(s) if was_absolute else (m3u.parent / s)
|
||||
if target.exists():
|
||||
if pending_extinf is not None:
|
||||
out.append(pending_extinf)
|
||||
pending_extinf = None
|
||||
if was_absolute:
|
||||
rel = os.path.relpath(target.resolve(), start=m3u.parent.resolve())
|
||||
out.append(rel)
|
||||
rewritten_here += 1
|
||||
else:
|
||||
out.append(line)
|
||||
else:
|
||||
desc = s
|
||||
if pending_extinf and "," in pending_extinf:
|
||||
desc = pending_extinf.split(",", 1)[1].strip() or s
|
||||
out.append(f"# NOT FOUND: {desc}")
|
||||
pending_extinf = None
|
||||
pruned_here += 1
|
||||
if pruned_here or rewritten_here:
|
||||
m3u.write_text("\n".join(out) + "\n", encoding="utf-8")
|
||||
if pruned_here:
|
||||
logger.warning("pruned {} missing entries from {}", pruned_here, m3u.name)
|
||||
if rewritten_here:
|
||||
logger.warning("rewrote {} absolute path(s) → relative in {}", rewritten_here, m3u.name)
|
||||
total_pruned += pruned_here
|
||||
total_rewritten += rewritten_here
|
||||
return (total_pruned, total_rewritten)
|
||||
|
||||
# --- report sink ------------------------------------------------------- #
|
||||
|
||||
def attach_report_sink(self) -> int:
|
||||
"""Install a loguru sink that mirrors every reported issue to disk.
|
||||
|
||||
Filters on `extra["report"] is True` so unrelated log lines don't
|
||||
leak into the report. Returns the loguru handle so callers can
|
||||
detach it later.
|
||||
"""
|
||||
self.metadata_dir.mkdir(parents=True, exist_ok=True)
|
||||
if self.report_path.exists():
|
||||
self.report_path.unlink()
|
||||
return logger.add(
|
||||
self.report_path,
|
||||
format="{message}",
|
||||
filter=lambda r: r["extra"].get("report") is True,
|
||||
level="INFO",
|
||||
)
|
||||
|
||||
def report_issues(self, issues: list[TrackIssue]) -> None:
|
||||
"""Emit each issue through the report sink; emit `ok` if none."""
|
||||
bound = logger.bind(report=True)
|
||||
if not issues:
|
||||
bound.info("ok")
|
||||
return
|
||||
for i in issues:
|
||||
bound.warning(f"{i.track.display_name}: {i.field}: {i.reason}")
|
||||
|
||||
# --- indexing ---------------------------------------------------------- #
|
||||
|
||||
def refresh_index(self) -> CollectionIndex:
|
||||
"""Rebuild `self.index` from sldl's csv + an on-disk tag scan."""
|
||||
idx = CollectionIndex()
|
||||
self._load_sldl_index(idx)
|
||||
self._scan_collection_tags(idx)
|
||||
self.index = idx
|
||||
return idx
|
||||
|
||||
def _load_sldl_index(self, idx: CollectionIndex) -> None:
|
||||
if not self.index_csv.exists():
|
||||
return
|
||||
with self.index_csv.open(newline="", encoding="utf-8", errors="replace") as fh:
|
||||
reader = csv.reader(fh)
|
||||
try:
|
||||
header = next(reader)
|
||||
except StopIteration:
|
||||
return
|
||||
cols = {h.strip().lower(): i for i, h in enumerate(header)}
|
||||
path_i = cols.get("filepath") or cols.get("path") or 0
|
||||
artist_i = cols.get("artist")
|
||||
title_i = cols.get("title")
|
||||
state_i = cols.get("state")
|
||||
for row in reader:
|
||||
if not row:
|
||||
continue
|
||||
if state_i is not None and state_i < len(row):
|
||||
state = row[state_i].strip().lower()
|
||||
if state in {"2", "failed", "notfounderror", "error"}:
|
||||
continue
|
||||
try:
|
||||
rel = row[path_i]
|
||||
except IndexError:
|
||||
continue
|
||||
if not rel:
|
||||
continue
|
||||
path = self.root / rel if not Path(rel).is_absolute() else Path(rel)
|
||||
if not path.exists():
|
||||
continue
|
||||
artist = row[artist_i] if artist_i is not None and artist_i < len(row) else ""
|
||||
title = row[title_i] if title_i is not None and title_i < len(row) else ""
|
||||
idx.add(artist, title, path)
|
||||
|
||||
def _scan_collection_tags(self, idx: CollectionIndex) -> None:
|
||||
if not self.root.exists():
|
||||
return
|
||||
for p in self.root.rglob("*"):
|
||||
if not p.is_file() or p.suffix.lower() not in AUDIO_EXTS:
|
||||
continue
|
||||
if "_metadata" in p.parts or "_playlists" in p.parts:
|
||||
continue
|
||||
try:
|
||||
audio = MutagenFile(p, easy=True)
|
||||
except Exception:
|
||||
continue
|
||||
if audio is None:
|
||||
continue
|
||||
tags = audio.tags or {}
|
||||
artist = (tags.get("artist") or [""])[0] if hasattr(tags, "get") else ""
|
||||
title = (tags.get("title") or [""])[0] if hasattr(tags, "get") else ""
|
||||
if not title:
|
||||
title = p.stem
|
||||
idx.add(str(artist), str(title), p)
|
||||
|
||||
# --- planning + downloads --------------------------------------------- #
|
||||
|
||||
def find_missing(self, playlists: list[dict]) -> list[dict]:
|
||||
"""Return the Spotify tracks not currently resolvable in this collection."""
|
||||
seen: set[tuple[str, str]] = set()
|
||||
missing: list[dict] = []
|
||||
for pl in playlists:
|
||||
for t in pl["tracks"]:
|
||||
artist = ", ".join(t["artists"])
|
||||
if self.index.lookup(artist, t["title"]) is not None:
|
||||
continue
|
||||
key = (norm(first_artist(artist)), norm(t["title"]))
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
missing.append(t)
|
||||
return missing
|
||||
|
||||
async def download(
|
||||
self,
|
||||
missing: list[dict],
|
||||
on_event: Callable[[dict], Awaitable[None]] | None = None,
|
||||
) -> int:
|
||||
"""Thin wrapper around `maple.collection.sockseek.download`."""
|
||||
return await run_sockseek(self.root, self.index_csv, missing, on_event)
|
||||
|
||||
# --- per-track Spotify reconciliation --------------------------------- #
|
||||
|
||||
def compare_against_spotify(
|
||||
self,
|
||||
playlist_name: str,
|
||||
track: dict,
|
||||
path: Path,
|
||||
) -> list[TrackIssue]:
|
||||
"""Compare a downloaded file's tags + duration against Spotify.
|
||||
|
||||
Spotify is the source of truth. Each mismatched field yields a
|
||||
`TrackIssue` whose `reason` reads as
|
||||
`<field> mismatch: spotify=<sv> file=<fv>`. Empty/missing file
|
||||
tags count as mismatches against a non-empty Spotify value.
|
||||
"""
|
||||
sp_artist = ", ".join(track.get("artists") or [])
|
||||
sp_title = (track.get("title") or "").strip()
|
||||
sp_album = (track.get("album") or "").strip()
|
||||
sp_isrc = (track.get("isrc") or "").strip()
|
||||
sp_duration = (track.get("duration_ms") or 0) / 1000.0
|
||||
sp_date = (track.get("release_date") or "").strip()
|
||||
|
||||
try:
|
||||
audio = MutagenFile(path, easy=True)
|
||||
tags = audio.tags if audio is not None else None
|
||||
except Exception:
|
||||
tags = None
|
||||
f_artist = (tags.get("artist") or [""])[0] if tags and hasattr(tags, "get") else ""
|
||||
f_title = (tags.get("title") or [""])[0] if tags and hasattr(tags, "get") else ""
|
||||
f_album = (tags.get("album") or [""])[0] if tags and hasattr(tags, "get") else ""
|
||||
f_isrc = (tags.get("isrc") or [""])[0] if tags and hasattr(tags, "get") else ""
|
||||
f_date = (tags.get("date") or [""])[0] if tags and hasattr(tags, "get") else ""
|
||||
f_duration = file_duration(path)
|
||||
|
||||
issues: list[TrackIssue] = []
|
||||
def mismatch(field: str, sv: object, fv: object) -> None:
|
||||
issues.append(TrackIssue(
|
||||
playlist_name, sp_artist, sp_title,
|
||||
f"{field} mismatch: spotify={sv!r} file={fv!r} ({path.name})",
|
||||
))
|
||||
|
||||
# Artist + title use fuzzy compare because tags & spotify quote
|
||||
# remixers / features differently. Album + ISRC + date are exact-ish.
|
||||
if sp_artist and fuzzy_ratio(norm(first_artist(sp_artist)), norm(first_artist(f_artist))) < 0.5:
|
||||
mismatch("artist", sp_artist, f_artist)
|
||||
if sp_title and fuzzy_ratio(norm_loose(sp_title), norm_loose(f_title)) < 0.5:
|
||||
mismatch("title", sp_title, f_title)
|
||||
if sp_album and f_album and fuzzy_ratio(norm_loose(sp_album), norm_loose(f_album)) < 0.5:
|
||||
mismatch("album", sp_album, f_album)
|
||||
elif sp_album and not f_album:
|
||||
mismatch("album", sp_album, f_album)
|
||||
if sp_isrc and f_isrc and sp_isrc.upper() != f_isrc.upper():
|
||||
mismatch("isrc", sp_isrc, f_isrc)
|
||||
# Compare only the year portion of `date` — file tags are usually "YYYY"
|
||||
# while Spotify is "YYYY[-MM[-DD]]".
|
||||
if sp_date and f_date and sp_date[:4] != f_date[:4]:
|
||||
mismatch("date", sp_date, f_date)
|
||||
if sp_duration > 0 and abs(f_duration - sp_duration) > DURATION_TOLERANCE_S:
|
||||
mismatch("duration", f"{sp_duration:.1f}s", f"{f_duration:.1f}s")
|
||||
|
||||
return issues
|
||||
|
||||
def write_spotify_tags(self, track: dict, path: Path) -> None:
|
||||
"""Delegate to the shared `maple.enrich.write_spotify_tags` so the
|
||||
collection-side and export-side writes are byte-for-byte identical
|
||||
and can't drift."""
|
||||
_shared_write_spotify_tags(track, path)
|
||||
|
||||
def _existing_bpm(self, path: Path) -> float | None:
|
||||
"""Return the parsed `bpm` tag value, or None if absent/invalid."""
|
||||
try:
|
||||
audio = MutagenFile(path, easy=True)
|
||||
if audio is None or audio.tags is None:
|
||||
return None
|
||||
v = audio.tags.get("bpm")
|
||||
if not v:
|
||||
return None
|
||||
return float(v[0])
|
||||
except (ValueError, TypeError, Exception):
|
||||
return None
|
||||
|
||||
def _write_bpm_tag(self, path: Path, bpm: float) -> None:
|
||||
try:
|
||||
audio = MutagenFile(path, easy=True)
|
||||
if audio is None:
|
||||
return
|
||||
if audio.tags is None:
|
||||
try:
|
||||
audio.add_tags()
|
||||
except Exception:
|
||||
return
|
||||
audio.tags["bpm"] = f"{bpm:.2f}"
|
||||
audio.save()
|
||||
except Exception as e:
|
||||
logger.warning("failed to write bpm to {}: {}", path.name, e)
|
||||
|
||||
def process_track(
|
||||
self,
|
||||
playlist_name: str,
|
||||
track: dict,
|
||||
path: Path,
|
||||
) -> list[TrackIssue]:
|
||||
"""Reconcile a single downloaded file against Spotify + analyse it.
|
||||
|
||||
Steps:
|
||||
1. ffprobe sanity check (one issue + early return on failure).
|
||||
2. Compare every relevant field; collect mismatch issues.
|
||||
3. Write Spotify's values back into the file's tags.
|
||||
4. If the file has no `bpm` tag yet, run aubio BPM detection
|
||||
and write the analytical value. Existing tags are left alone.
|
||||
|
||||
Idempotent within a run — paths already processed are no-ops.
|
||||
"""
|
||||
if path in self._reconciled:
|
||||
return []
|
||||
self._reconciled.add(path)
|
||||
|
||||
# Source-of-truth tag writes first: ensures the file mirrors Spotify
|
||||
# before validation reads its tags back. Mismatches caught here are
|
||||
# written to the report regardless of the validator's subsequent
|
||||
# findings (the validator may dedup them if Spotify still disagrees).
|
||||
self.write_spotify_tags(track, path)
|
||||
|
||||
# Single audio analysis pass: BPM + beat grid + waveform envelopes.
|
||||
# Cached under collection/_analysis/<isrc>.json so `just export`
|
||||
# reads the bytes back instead of redoing ffmpeg+mixxx. Analysis
|
||||
# failures propagate — a broken analysis is fatal, not skipped.
|
||||
isrc = (track.get("isrc") or "").strip()
|
||||
existing_bpm = self._existing_bpm(path)
|
||||
analysis = Analysis.analyze(path, isrc=isrc)
|
||||
if analysis.bpm:
|
||||
if existing_bpm is None:
|
||||
self._write_bpm_tag(path, analysis.bpm)
|
||||
logger.info("{}: bpm {:.2f} (mixxx)", path.name, analysis.bpm)
|
||||
else:
|
||||
logger.debug(
|
||||
"{}: bpm {:.2f} (existing tag, kept; mixxx measured {:.2f})",
|
||||
path.name, existing_bpm, analysis.bpm,
|
||||
)
|
||||
if isrc:
|
||||
cp = analysis.save(self.analysis_dir)
|
||||
logger.info("cached analysis for {}: {} → {}", isrc, path.name, cp)
|
||||
else:
|
||||
logger.warning(
|
||||
"skipping analysis cache for {}: no ISRC in spotify track ({} - {})",
|
||||
path.name, ", ".join(track.get("artists") or []), track.get("title") or "?",
|
||||
)
|
||||
|
||||
# Full cross-cutting validation pass. Runs the SAME checks as
|
||||
# `just export`, so a clean run here means the track will export
|
||||
# without issues — and vice versa.
|
||||
return _validate_track(
|
||||
path,
|
||||
spotify_track=track,
|
||||
analysis_cache_dir=self.analysis_dir,
|
||||
playlist=playlist_name,
|
||||
)
|
||||
|
||||
# --- validation -------------------------------------------------------- #
|
||||
|
||||
async def validate(
|
||||
self,
|
||||
session: aiohttp.ClientSession,
|
||||
playlists: list[dict],
|
||||
) -> list[TrackIssue]:
|
||||
"""Per-track Spotify reconciliation + artwork embed.
|
||||
|
||||
Spotify is the source of truth: any field that differs from the
|
||||
downloaded file's tags is recorded as a mismatch issue, then the
|
||||
file's tags are overwritten with the Spotify values. Missing
|
||||
embedded artwork is queued for an async batch fetch.
|
||||
"""
|
||||
resolved: list[tuple[str, dict, Path]] = []
|
||||
for pl in playlists:
|
||||
for t in pl["tracks"]:
|
||||
path = self.index.lookup(", ".join(t["artists"]), t["title"])
|
||||
if path is not None:
|
||||
resolved.append((pl["name"] or pl["id"], t, path))
|
||||
|
||||
issues: list[TrackIssue] = []
|
||||
sem = asyncio.Semaphore(ARTWORK_CONCURRENCY)
|
||||
art_jobs: list[tuple[str, dict, Path]] = []
|
||||
|
||||
for playlist_name, track, path in resolved:
|
||||
issues.extend(self.process_track(playlist_name, track, path))
|
||||
if not has_embedded_art(path) and track.get("album_image_url"):
|
||||
art_jobs.append((playlist_name, track, path))
|
||||
|
||||
async def _embed(job: tuple[str, dict, Path]) -> None:
|
||||
playlist_name, track, path = job
|
||||
async with sem:
|
||||
data = await fetch_jpeg(session, track["album_image_url"])
|
||||
if not data:
|
||||
issues.append(TrackIssue(
|
||||
playlist_name, ", ".join(track["artists"]), track["title"],
|
||||
"artwork download failed",
|
||||
))
|
||||
return
|
||||
try:
|
||||
embed_art(path, data)
|
||||
except Exception as e:
|
||||
issues.append(TrackIssue(
|
||||
playlist_name, ", ".join(track["artists"]), track["title"],
|
||||
f"artwork embed failed: {e}",
|
||||
))
|
||||
|
||||
if art_jobs:
|
||||
await asyncio.gather(*(_embed(j) for j in art_jobs))
|
||||
return issues
|
||||
|
||||
# --- output ------------------------------------------------------------ #
|
||||
|
||||
def write_m3u8_for(
|
||||
self,
|
||||
playlist: dict,
|
||||
issues: list[TrackIssue] | None = None,
|
||||
) -> Path:
|
||||
"""Write a single playlist's m3u8 against the current index.
|
||||
|
||||
Track paths are emitted relative to the m3u8's parent directory so
|
||||
the playlist is portable across devices that sync the collection.
|
||||
|
||||
Pass `issues=None` for incremental rewrites during download — the
|
||||
list of unresolved tracks gets rebuilt on the final pass instead.
|
||||
"""
|
||||
self.playlists_dir.mkdir(parents=True, exist_ok=True)
|
||||
playlist_name = playlist["name"] or playlist["id"]
|
||||
out_path = self.playlists_dir / f"{_sanitise_name(playlist_name)}.m3u8"
|
||||
lines = ["#EXTM3U"]
|
||||
for t in playlist["tracks"]:
|
||||
artist = ", ".join(t["artists"])
|
||||
title = t["title"]
|
||||
secs = round(t["duration_ms"] / 1000)
|
||||
path = self.index.lookup(artist, title)
|
||||
if path is None:
|
||||
lines.append(f"# NOT FOUND: {artist} - {title}")
|
||||
if issues is not None:
|
||||
issues.append(TrackIssue(playlist_name, artist, title, "unresolved in collection"))
|
||||
continue
|
||||
rel = os.path.relpath(path.resolve(), start=out_path.parent.resolve())
|
||||
lines.append(f"#EXTINF:{secs},{artist} - {title}")
|
||||
lines.append(rel)
|
||||
out_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
return out_path
|
||||
|
||||
def write_m3u8s(
|
||||
self,
|
||||
playlists: list[dict],
|
||||
issues: list[TrackIssue],
|
||||
) -> list[Path]:
|
||||
"""One m3u8 per playlist in `<root>/_playlists/`. Returns the paths written."""
|
||||
return [self.write_m3u8_for(pl, issues) for pl in playlists]
|
||||
@@ -0,0 +1,81 @@
|
||||
"""Artist/title normalisation + `CollectionIndex` lookup table.
|
||||
|
||||
No I/O. Pure functions plus a small dataclass keyed on normalised
|
||||
(artist, title) tuples so we can answer "is this track already in the
|
||||
collection?" against an in-memory map.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import unicodedata
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
FUZZY_THRESHOLD = 0.85
|
||||
|
||||
_PARENS = re.compile(r"\s*[\(\[][^)\]]*[\)\]]\s*")
|
||||
_NONWORD = re.compile(r"[^\w\s]+", flags=re.UNICODE)
|
||||
_WS = re.compile(r"\s+")
|
||||
|
||||
|
||||
def norm(s: str) -> str:
|
||||
"""Lowercase + NFKC + strip non-word, preserves remix/version suffixes."""
|
||||
s = unicodedata.normalize("NFKC", s or "").lower()
|
||||
s = _NONWORD.sub(" ", s)
|
||||
s = _WS.sub(" ", s).strip()
|
||||
return s
|
||||
|
||||
|
||||
def norm_loose(s: str) -> str:
|
||||
"""Same as norm() but with parenthesised suffixes removed."""
|
||||
s = unicodedata.normalize("NFKC", s or "").lower()
|
||||
s = _PARENS.sub(" ", s)
|
||||
s = _NONWORD.sub(" ", s)
|
||||
s = _WS.sub(" ", s).strip()
|
||||
return s
|
||||
|
||||
|
||||
def first_artist(artists: str) -> str:
|
||||
"""Split sldl/spotify multi-artist strings on common delimiters."""
|
||||
if not artists:
|
||||
return ""
|
||||
for sep in (",", " feat.", " ft.", " & ", "&", " and ", ";"):
|
||||
if sep in artists:
|
||||
return artists.split(sep, 1)[0].strip()
|
||||
return artists.strip()
|
||||
|
||||
|
||||
def fuzzy_ratio(a: str, b: str) -> float:
|
||||
"""Cheap token-overlap ratio in [0, 1]. No external dep."""
|
||||
ta, tb = set(a.split()), set(b.split())
|
||||
if not ta or not tb:
|
||||
return 0.0
|
||||
return len(ta & tb) / max(len(ta), len(tb))
|
||||
|
||||
|
||||
@dataclass
|
||||
class CollectionIndex:
|
||||
"""Maps normalised (artist, title) pairs to file paths in the collection."""
|
||||
|
||||
by_strict: dict[tuple[str, str], Path] = field(default_factory=dict)
|
||||
by_loose: dict[tuple[str, str], Path] = field(default_factory=dict)
|
||||
|
||||
def add(self, artist: str, title: str, path: Path) -> None:
|
||||
a, t = norm(first_artist(artist)), norm(title)
|
||||
if a and t:
|
||||
self.by_strict.setdefault((a, t), path)
|
||||
la, lt = norm_loose(first_artist(artist)), norm_loose(title)
|
||||
if la and lt:
|
||||
self.by_loose.setdefault((la, lt), path)
|
||||
|
||||
def lookup(self, artist: str, title: str) -> Path | None:
|
||||
a, t = norm(first_artist(artist)), norm(title)
|
||||
if (a, t) in self.by_strict:
|
||||
return self.by_strict[(a, t)]
|
||||
la, lt = norm_loose(first_artist(artist)), norm_loose(title)
|
||||
if (la, lt) in self.by_loose:
|
||||
return self.by_loose[(la, lt)]
|
||||
for (ka, kt), p in self.by_loose.items():
|
||||
if ka == la and fuzzy_ratio(kt, lt) >= FUZZY_THRESHOLD:
|
||||
return p
|
||||
return None
|
||||
@@ -0,0 +1,149 @@
|
||||
"""Sockseek subprocess plumbing.
|
||||
|
||||
Owns:
|
||||
* the CLI argument shape (`args_for`) used to run sockseek/sldl,
|
||||
* the temp-CSV writer that feeds tracks in,
|
||||
* the async `download()` entrypoint that streams `--progress-json`
|
||||
events out and forwards sockseek stderr through loguru.
|
||||
|
||||
`Collection.download()` is a thin wrapper around `download()` here so the
|
||||
subprocess concerns stay in one file.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import csv
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
from collections.abc import Awaitable, Callable
|
||||
from pathlib import Path
|
||||
|
||||
from loguru import logger
|
||||
|
||||
NAME_FORMAT = "{artist(/)album(/)track( )title|(_missing_tags/)slsk-foldername(/)slsk-filename}"
|
||||
|
||||
_ANSI_RE = re.compile(r"\x1b\[[0-9;]*[A-Za-z]")
|
||||
_LEVEL_RE = re.compile(
|
||||
r"^\s*\[?(?P<lvl>TRACE|TRC|DEBUG|DBG|INFO|INF|WARN|WRN|ERROR|ERR|FATAL|CRITICAL|CRT)\]?\b",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
_LEVEL_MAP = {
|
||||
"TRC": "TRACE", "TRACE": "TRACE",
|
||||
"DBG": "DEBUG", "DEBUG": "DEBUG",
|
||||
"INF": "INFO", "INFO": "INFO",
|
||||
"WRN": "WARNING", "WARN": "WARNING",
|
||||
"ERR": "ERROR", "ERROR": "ERROR",
|
||||
"FATAL": "CRITICAL", "CRT": "CRITICAL", "CRITICAL": "CRITICAL",
|
||||
}
|
||||
|
||||
|
||||
def log_level_for(line: str) -> str:
|
||||
"""Map a sockseek/.NET-style log prefix to a loguru level. Default WARNING."""
|
||||
m = _LEVEL_RE.match(line)
|
||||
if not m:
|
||||
return "WARNING"
|
||||
return _LEVEL_MAP.get(m.group("lvl").upper(), "WARNING")
|
||||
|
||||
|
||||
def args_for(root: Path, index_csv: Path, user: str, password: str) -> list[str]:
|
||||
"""Mirror of src/sockseek.py:config_args(), pinned for FLAC-strict."""
|
||||
return [
|
||||
"--path", str(root),
|
||||
"--index-path", str(index_csv),
|
||||
"--user", user,
|
||||
"--pass", password,
|
||||
"--name-format", NAME_FORMAT,
|
||||
"--format", "flac",
|
||||
"--pref-format", "flac",
|
||||
"--pref-min-bitrate", "200",
|
||||
"--pref-max-bitrate", "2500",
|
||||
"--strict-conditions",
|
||||
"--fast-search",
|
||||
"--concurrent-jobs", "4",
|
||||
]
|
||||
|
||||
|
||||
def write_download_csv(missing: list[dict], fh) -> None:
|
||||
"""sldl-compatible CSV (Artist,Title,Album,Length)."""
|
||||
w = csv.writer(fh)
|
||||
w.writerow(["Artist", "Title", "Album", "Length"])
|
||||
for t in missing:
|
||||
w.writerow([
|
||||
", ".join(t["artists"]),
|
||||
t["title"],
|
||||
t["album"],
|
||||
round(t["duration_ms"] / 1000),
|
||||
])
|
||||
|
||||
|
||||
async def download(
|
||||
root: Path,
|
||||
index_csv: Path,
|
||||
missing: list[dict],
|
||||
on_event: Callable[[dict], Awaitable[None]] | None = None,
|
||||
) -> int:
|
||||
"""Shell out to sockseek for every track in `missing`.
|
||||
|
||||
When `on_event` is provided, sockseek is run with `--progress-json` and
|
||||
every NDJSON envelope on stdout is parsed and awaited through the
|
||||
callback. stderr is always piped and forwarded line-by-line through
|
||||
loguru with a best-effort log-level guess.
|
||||
"""
|
||||
if not missing:
|
||||
return 0
|
||||
binary = shutil.which("sockseek") or shutil.which("sldl")
|
||||
if binary is None:
|
||||
logger.error("sockseek/sldl not on PATH; install slsk-batchdl first")
|
||||
sys.exit(1)
|
||||
user = os.environ.get("SOCKSEEK_USER")
|
||||
password = os.environ.get("SOCKSEEK_PASS")
|
||||
if not (user and password):
|
||||
logger.error("set SOCKSEEK_USER and SOCKSEEK_PASS for downloads")
|
||||
sys.exit(1)
|
||||
index_csv.parent.mkdir(parents=True, exist_ok=True)
|
||||
with tempfile.NamedTemporaryFile("w", suffix=".csv", newline="", delete=False) as tf:
|
||||
write_download_csv(missing, tf)
|
||||
tf_path = Path(tf.name)
|
||||
try:
|
||||
args = [binary, str(tf_path), *args_for(root, index_csv, user, password)]
|
||||
if on_event is not None:
|
||||
args.append("--progress-json")
|
||||
masked = ["***" if a == password else a for a in args]
|
||||
logger.info("$ {}", " ".join(masked))
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*args,
|
||||
stdout=asyncio.subprocess.PIPE if on_event is not None else None,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
|
||||
async def _forward_stderr() -> None:
|
||||
assert proc.stderr is not None
|
||||
async for raw in proc.stderr:
|
||||
line = _ANSI_RE.sub("", raw.decode("utf-8", errors="replace")).rstrip()
|
||||
if not line:
|
||||
continue
|
||||
logger.log(log_level_for(line), "sockseek: {}", line)
|
||||
|
||||
stderr_task = asyncio.create_task(_forward_stderr())
|
||||
|
||||
if on_event is not None:
|
||||
assert proc.stdout is not None
|
||||
async for raw in proc.stdout:
|
||||
line = raw.decode("utf-8", errors="replace").strip()
|
||||
if not line or not line.startswith("{"):
|
||||
continue
|
||||
try:
|
||||
evt = json.loads(line)
|
||||
except ValueError:
|
||||
continue
|
||||
await on_event(evt)
|
||||
|
||||
rc = await proc.wait()
|
||||
await stderr_task
|
||||
return rc
|
||||
finally:
|
||||
tf_path.unlink(missing_ok=True)
|
||||
@@ -0,0 +1,103 @@
|
||||
"""Per-track validation helpers: ffprobe sanity, duration probe, embedded
|
||||
artwork inspection + embed, async JPEG fetch for missing cover art.
|
||||
|
||||
Pure helpers — no Collection coupling.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import aiohttp
|
||||
from mutagen import File as MutagenFile
|
||||
from mutagen.flac import FLAC, Picture
|
||||
from mutagen.id3 import APIC, ID3, ID3NoHeaderError
|
||||
from mutagen.mp4 import MP4, MP4Cover
|
||||
|
||||
DURATION_TOLERANCE_S = 5.0
|
||||
ARTWORK_CONCURRENCY = 8
|
||||
|
||||
|
||||
def ffprobe_ok(path: Path) -> bool:
|
||||
if not shutil.which("ffprobe"):
|
||||
return True # ffprobe absent → don't fail the run
|
||||
r = subprocess.run(
|
||||
["ffprobe", "-v", "error", "-select_streams", "a:0",
|
||||
"-show_entries", "stream=codec_type", "-of", "csv=p=0", str(path)],
|
||||
capture_output=True, text=True,
|
||||
)
|
||||
return r.returncode == 0 and "audio" in r.stdout
|
||||
|
||||
|
||||
def file_duration(path: Path) -> float:
|
||||
audio = MutagenFile(path)
|
||||
info = getattr(audio, "info", None)
|
||||
return float(getattr(info, "length", 0.0) or 0.0)
|
||||
|
||||
|
||||
def has_embedded_art(path: Path) -> bool:
|
||||
suf = path.suffix.lower()
|
||||
if suf == ".flac":
|
||||
try:
|
||||
return bool(FLAC(path).pictures)
|
||||
except Exception:
|
||||
return False
|
||||
if suf == ".mp3":
|
||||
try:
|
||||
return bool(ID3(path).getall("APIC"))
|
||||
except Exception:
|
||||
return False
|
||||
if suf in (".m4a", ".mp4"):
|
||||
try:
|
||||
tags = MP4(path).tags
|
||||
return bool(tags and tags.get("covr"))
|
||||
except Exception:
|
||||
return False
|
||||
return True # unknown format → don't complain
|
||||
|
||||
|
||||
def embed_art(path: Path, jpeg_bytes: bytes) -> None:
|
||||
suf = path.suffix.lower()
|
||||
if suf == ".flac":
|
||||
f = FLAC(path)
|
||||
pic = Picture()
|
||||
pic.data = jpeg_bytes
|
||||
pic.type = 3 # FRONT_COVER
|
||||
pic.mime = "image/jpeg"
|
||||
f.clear_pictures()
|
||||
f.add_picture(pic)
|
||||
f.save()
|
||||
elif suf == ".mp3":
|
||||
try:
|
||||
tags = ID3(path)
|
||||
except ID3NoHeaderError:
|
||||
tags = ID3()
|
||||
tags.delall("APIC")
|
||||
tags.add(APIC(encoding=3, mime="image/jpeg", type=3, desc="Cover", data=jpeg_bytes))
|
||||
tags.save(path)
|
||||
elif suf in (".m4a", ".mp4"):
|
||||
f = MP4(path)
|
||||
f.tags = f.tags or {}
|
||||
f["covr"] = [MP4Cover(jpeg_bytes, imageformat=MP4Cover.FORMAT_JPEG)]
|
||||
f.save()
|
||||
|
||||
|
||||
async def fetch_jpeg(session: aiohttp.ClientSession, url: str) -> bytes | None:
|
||||
try:
|
||||
async with session.get(url) as resp:
|
||||
resp.raise_for_status()
|
||||
data = await resp.read()
|
||||
except aiohttp.ClientError:
|
||||
return None
|
||||
if data[:3] == b"\xff\xd8\xff":
|
||||
return data
|
||||
try:
|
||||
from PIL import Image
|
||||
img = Image.open(io.BytesIO(data)).convert("RGB")
|
||||
buf = io.BytesIO()
|
||||
img.save(buf, format="JPEG", quality=90)
|
||||
return buf.getvalue()
|
||||
except Exception:
|
||||
return None
|
||||
@@ -0,0 +1,82 @@
|
||||
"""FLAC -> MP3 V0 joint-stereo conversion via ffmpeg, with source-hash dedup.
|
||||
|
||||
The SHA-256 of the source FLAC is stamped into a TXXX:MAPLE_SRC_SHA256 ID3
|
||||
frame on the produced MP3. Reruns skip conversion when the existing MP3
|
||||
carries the matching hash, so the cache is content-addressed against inputs.
|
||||
"""
|
||||
import hashlib
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from mutagen import File as MutagenFile
|
||||
from mutagen.id3 import ID3, ID3NoHeaderError, TXXX
|
||||
|
||||
HASH_FRAME_DESC = "MAPLE_SRC_SHA256"
|
||||
|
||||
|
||||
def file_sha256(path: Path) -> str:
|
||||
with path.open("rb") as f:
|
||||
return hashlib.file_digest(f, "sha256").hexdigest()
|
||||
|
||||
|
||||
def _read_source_hash(mp3: Path) -> str | None:
|
||||
try:
|
||||
tags = ID3(mp3)
|
||||
except ID3NoHeaderError:
|
||||
return None
|
||||
for frame in tags.getall("TXXX"):
|
||||
if frame.desc == HASH_FRAME_DESC and frame.text:
|
||||
return str(frame.text[0])
|
||||
return None
|
||||
|
||||
|
||||
def _write_source_hash(mp3: Path, sha: str) -> None:
|
||||
try:
|
||||
tags = ID3(mp3)
|
||||
except ID3NoHeaderError:
|
||||
tags = ID3()
|
||||
tags.delall(f"TXXX:{HASH_FRAME_DESC}")
|
||||
tags.add(TXXX(encoding=3, desc=HASH_FRAME_DESC, text=sha))
|
||||
tags.save(mp3)
|
||||
|
||||
|
||||
def convert_flac_to_mp3(flac: Path, mp3: Path) -> bool:
|
||||
"""Convert flac to mp3 (V0, joint stereo). Returns True if conversion ran."""
|
||||
src_hash = file_sha256(flac)
|
||||
if mp3.exists() and _read_source_hash(mp3) == src_hash:
|
||||
return False
|
||||
|
||||
if not shutil.which("ffmpeg"):
|
||||
raise RuntimeError("ffmpeg not found on PATH")
|
||||
|
||||
mp3.parent.mkdir(parents=True, exist_ok=True)
|
||||
subprocess.run(
|
||||
[
|
||||
"ffmpeg", "-nostdin", "-loglevel", "error", "-y",
|
||||
"-i", str(flac),
|
||||
"-codec:a", "libmp3lame",
|
||||
"-q:a", "0",
|
||||
"-joint_stereo", "1",
|
||||
"-map_metadata", "0",
|
||||
str(mp3),
|
||||
],
|
||||
check=True,
|
||||
)
|
||||
_write_source_hash(mp3, src_hash)
|
||||
return True
|
||||
|
||||
|
||||
def _duration(path: Path) -> float:
|
||||
audio = MutagenFile(path)
|
||||
info = getattr(audio, "info", None)
|
||||
return float(getattr(info, "length", 0.0) or 0.0)
|
||||
|
||||
|
||||
def verify_duration(src: Path, dst: Path, tolerance: float = 0.1) -> None:
|
||||
"""Raise if dst's audio length differs from src's by more than `tolerance` seconds."""
|
||||
a, b = _duration(src), _duration(dst)
|
||||
if abs(a - b) > tolerance:
|
||||
raise RuntimeError(
|
||||
f"duration mismatch: {src.name} = {a:.3f}s vs {dst.name} = {b:.3f}s (Δ {abs(a-b):.3f}s)"
|
||||
)
|
||||
@@ -0,0 +1,66 @@
|
||||
"""Copy audio files into the USB Contents layout.
|
||||
|
||||
Targets are assumed to be FAT32-formatted USBs, which constrain us to:
|
||||
- per-component name length <= 255 LFN chars
|
||||
- total path (from drive root) <= 260 chars
|
||||
- no '\\ / : * ? " < > |' or control chars in any component
|
||||
We truncate per-component to a safe 64 chars (rekordbox firmware is
|
||||
fussier than raw FAT32) and refuse to emit anything that would still
|
||||
overflow the 260-char path budget.
|
||||
"""
|
||||
import re
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
_BAD = re.compile(r'[\\/:*?"<>|\x00-\x1f]')
|
||||
|
||||
MAX_COMPONENT = 48
|
||||
MAX_PATH = 260
|
||||
# Pioneer firmware appears to only handle the DeviceSQL short-ASCII form
|
||||
# for `file_path` in the pdb (length byte < 127). The verified reference
|
||||
# export uses short-form for every track. Keep the on-pdb path strictly
|
||||
# inside that budget so every track stays loadable.
|
||||
MAX_PDB_PATH = 126
|
||||
|
||||
|
||||
def sanitise(component: str, max_len: int = MAX_COMPONENT) -> str:
|
||||
s = _BAD.sub("_", component).strip(" .")
|
||||
if len(s) > max_len:
|
||||
s = s[:max_len].rstrip(" .")
|
||||
return s or "_"
|
||||
|
||||
|
||||
def _truncate_filename(name: str, max_len: int) -> str:
|
||||
"""Truncate while preserving the extension."""
|
||||
cleaned = _BAD.sub("_", name).strip(" .")
|
||||
if len(cleaned) <= max_len:
|
||||
return cleaned or "_"
|
||||
stem, dot, ext = cleaned.rpartition(".")
|
||||
if dot and len(ext) < max_len - 1:
|
||||
keep = max_len - len(ext) - 1
|
||||
return (stem[:keep].rstrip(" .") + "." + ext) or "_"
|
||||
return cleaned[:max_len].rstrip(" .") or "_"
|
||||
|
||||
|
||||
def staged_path(usb_root: Path, artist: str, album: str, filename: str) -> tuple[Path, str]:
|
||||
artist_s = sanitise(artist)
|
||||
album_s = sanitise(album)
|
||||
prefix = f"/Contents/{artist_s}/{album_s}/"
|
||||
budget = min(MAX_COMPONENT, MAX_PDB_PATH - len(prefix))
|
||||
if budget < 8:
|
||||
raise ValueError(f"no room for filename within MAX_PDB_PATH={MAX_PDB_PATH} (prefix uses {len(prefix)}): {prefix}")
|
||||
name_s = _truncate_filename(filename, budget)
|
||||
dest_rel = Path("Contents") / artist_s / album_s / name_s
|
||||
dest_abs = usb_root / dest_rel
|
||||
file_path = "/" + dest_rel.as_posix()
|
||||
if len(file_path) > MAX_PDB_PATH:
|
||||
raise ValueError(f"pdb path budget exceeded ({len(file_path)} > {MAX_PDB_PATH}): {file_path}")
|
||||
return dest_abs, file_path
|
||||
|
||||
|
||||
def stage_audio(src: Path, usb_root: Path, artist: str, album: str) -> tuple[Path, str]:
|
||||
dest_abs, file_path = staged_path(usb_root, artist, album, src.name)
|
||||
dest_abs.parent.mkdir(parents=True, exist_ok=True)
|
||||
if not dest_abs.exists():
|
||||
shutil.copy2(src, dest_abs)
|
||||
return dest_abs, file_path
|
||||
@@ -0,0 +1,307 @@
|
||||
"""Spotify metadata enrichment shared by `just collection` and `just export`.
|
||||
|
||||
Per the project convention: a track is "valid" iff every export criterion
|
||||
holds (artwork, tags, analysis). BOTH pipelines must be capable of
|
||||
*bringing* a track to that state — validation isn't just reporting,
|
||||
it's repair.
|
||||
|
||||
`just collection`:
|
||||
1. Fetches Spotify playlists.
|
||||
2. Persists every track to `<collection>/_metadata/spotify-tracks.json`.
|
||||
3. Walks the collection, calls `write_spotify_tags` for each match.
|
||||
|
||||
`just export`:
|
||||
1. Loads the manifest written by `just collection`.
|
||||
2. Walks `data/usb/`'s staged files. For any file missing canonical
|
||||
tags (typically ISRC), looks up its Spotify entry via the manifest
|
||||
and calls `write_spotify_tags` BEFORE validation runs.
|
||||
|
||||
The shared write function lives here so the two pipelines are
|
||||
byte-for-byte consistent in what they leave on disk.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from loguru import logger
|
||||
from mutagen import File as MutagenFile
|
||||
from mutagen.flac import FLAC, Picture
|
||||
from mutagen.id3 import APIC, ID3, ID3NoHeaderError
|
||||
from mutagen.mp4 import MP4, MP4Cover
|
||||
|
||||
from maple.collection.index import first_artist, norm
|
||||
from maple.collection.validate import embed_art, has_embedded_art
|
||||
|
||||
MANIFEST_NAME = "spotify-tracks.json"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Manifest persistence #
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def save_spotify_manifest(metadata_dir: Path, playlists: list[dict]) -> Path:
|
||||
"""Persist Spotify-fetched playlists so `just export` can read them later.
|
||||
|
||||
Stored as a single JSON blob at `<metadata_dir>/spotify-tracks.json`.
|
||||
Replaces any prior manifest atomically (tmp + rename).
|
||||
"""
|
||||
metadata_dir.mkdir(parents=True, exist_ok=True)
|
||||
target = metadata_dir / MANIFEST_NAME
|
||||
tmp = target.with_suffix(target.suffix + ".tmp")
|
||||
tmp.write_text(
|
||||
json.dumps({"playlists": playlists}, indent=2, ensure_ascii=False, default=str)
|
||||
)
|
||||
tmp.replace(target)
|
||||
n = sum(len(p.get("tracks") or []) for p in playlists)
|
||||
logger.info("wrote spotify manifest: {} tracks across {} playlist(s) → {}",
|
||||
n, len(playlists), target)
|
||||
return target
|
||||
|
||||
|
||||
def load_spotify_manifest(
|
||||
metadata_dir: Path,
|
||||
) -> tuple[dict[tuple[str, str], dict], dict[str, dict]]:
|
||||
"""Load the manifest written by `just collection`.
|
||||
|
||||
Returns `(by_key, by_isrc)` where `by_key` is indexed by
|
||||
`(first_artist_norm, title_norm)` and `by_isrc` is indexed by the
|
||||
uppercased ISRC. Empty dicts if the manifest is absent/unreadable.
|
||||
"""
|
||||
target = metadata_dir / MANIFEST_NAME
|
||||
if not target.exists():
|
||||
return {}, {}
|
||||
try:
|
||||
data = json.loads(target.read_text())
|
||||
except Exception as e:
|
||||
logger.warning("could not read spotify manifest at {}: {}", target, e)
|
||||
return {}, {}
|
||||
by_key: dict[tuple[str, str], dict] = {}
|
||||
by_isrc: dict[str, dict] = {}
|
||||
for pl in data.get("playlists", []):
|
||||
for t in pl.get("tracks", []):
|
||||
artist = ", ".join(t.get("artists") or [])
|
||||
k = (norm(first_artist(artist)), norm(t.get("title") or ""))
|
||||
by_key.setdefault(k, t)
|
||||
isrc = (t.get("isrc") or "").strip().upper()
|
||||
if isrc:
|
||||
by_isrc.setdefault(isrc, t)
|
||||
return by_key, by_isrc
|
||||
|
||||
|
||||
def find_spotify_track(
|
||||
path: Path,
|
||||
by_key: dict[tuple[str, str], dict],
|
||||
by_isrc: dict[str, dict],
|
||||
) -> dict | None:
|
||||
"""Resolve a file to its Spotify track via ISRC tag, then (artist, title)."""
|
||||
try:
|
||||
audio = MutagenFile(path, easy=True)
|
||||
except Exception:
|
||||
return None
|
||||
if audio is None or audio.tags is None:
|
||||
return None
|
||||
tags = audio.tags
|
||||
|
||||
def first(k: str) -> str:
|
||||
if hasattr(tags, "get"):
|
||||
v = tags.get(k)
|
||||
if v:
|
||||
return str(v[0])
|
||||
return ""
|
||||
|
||||
isrc = first("isrc").upper().strip()
|
||||
if isrc and isrc in by_isrc:
|
||||
return by_isrc[isrc]
|
||||
artist = first("artist")
|
||||
title = first("title")
|
||||
if artist or title:
|
||||
k = (norm(first_artist(artist)), norm(title))
|
||||
return by_key.get(k)
|
||||
return None
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Tag writing (the canonical write path) #
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def write_spotify_tags(track: dict, path: Path) -> bool:
|
||||
"""Overwrite the file's tags with the Spotify-canonical values.
|
||||
|
||||
Both pipelines call this. Writes title, artist (`", ".join`),
|
||||
album, date, genre (list), and ISRC via mutagen's easy interface.
|
||||
Empty Spotify values leave the existing tag untouched.
|
||||
|
||||
Returns True when at least one tag was written + saved.
|
||||
"""
|
||||
try:
|
||||
audio = MutagenFile(path, easy=True)
|
||||
except Exception as e:
|
||||
logger.warning("write_spotify_tags: cannot open {}: {}", path.name, e)
|
||||
return False
|
||||
if audio is None:
|
||||
return False
|
||||
if audio.tags is None:
|
||||
try:
|
||||
audio.add_tags()
|
||||
except Exception as e:
|
||||
logger.warning("write_spotify_tags: cannot init tags on {}: {}", path.name, e)
|
||||
return False
|
||||
|
||||
wrote_any = False
|
||||
title = (track.get("title") or "").strip()
|
||||
if title:
|
||||
audio.tags["title"] = title
|
||||
wrote_any = True
|
||||
artists = track.get("artists") or []
|
||||
if artists:
|
||||
audio.tags["artist"] = ", ".join(artists)
|
||||
wrote_any = True
|
||||
album = (track.get("album") or "").strip()
|
||||
if album:
|
||||
audio.tags["album"] = album
|
||||
wrote_any = True
|
||||
date = (track.get("release_date") or "").strip()
|
||||
if date:
|
||||
try:
|
||||
audio.tags["date"] = date
|
||||
wrote_any = True
|
||||
except (KeyError, ValueError) as e:
|
||||
logger.debug("date tag write skipped on {}: {}", path.name, e)
|
||||
genres = track.get("genres") or []
|
||||
if genres:
|
||||
try:
|
||||
audio.tags["genre"] = list(genres)
|
||||
wrote_any = True
|
||||
except (KeyError, ValueError) as e:
|
||||
logger.debug("genre tag write skipped on {}: {}", path.name, e)
|
||||
isrc = (track.get("isrc") or "").strip()
|
||||
if isrc:
|
||||
try:
|
||||
audio.tags["isrc"] = isrc
|
||||
wrote_any = True
|
||||
except (KeyError, ValueError) as e:
|
||||
logger.debug("isrc tag write skipped on {}: {}", path.name, e)
|
||||
|
||||
if not wrote_any:
|
||||
return False
|
||||
try:
|
||||
audio.save()
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.warning("write_spotify_tags: save failed for {}: {}", path.name, e)
|
||||
return False
|
||||
|
||||
|
||||
def write_bpm_tag(path: Path, bpm: float, *, overwrite: bool = False) -> bool:
|
||||
"""Write a `bpm` tag to the file. When `overwrite=False` (the default),
|
||||
leaves any existing tag value alone — preserves human/label-set BPMs."""
|
||||
try:
|
||||
audio = MutagenFile(path, easy=True)
|
||||
except Exception as e:
|
||||
logger.warning("write_bpm_tag: cannot open {}: {}", path.name, e)
|
||||
return False
|
||||
if audio is None:
|
||||
return False
|
||||
if audio.tags is None:
|
||||
try:
|
||||
audio.add_tags()
|
||||
except Exception as e:
|
||||
logger.warning("write_bpm_tag: cannot init tags on {}: {}", path.name, e)
|
||||
return False
|
||||
if not overwrite:
|
||||
existing = audio.tags.get("bpm") if hasattr(audio.tags, "get") else None
|
||||
if existing and (existing[0] if isinstance(existing, list) else existing):
|
||||
return False # keep the existing tag
|
||||
audio.tags["bpm"] = f"{bpm:.2f}"
|
||||
try:
|
||||
audio.save()
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.warning("write_bpm_tag: save failed for {}: {}", path.name, e)
|
||||
return False
|
||||
|
||||
|
||||
def download_jpeg(url: str, *, timeout: float = 15.0) -> bytes | None:
|
||||
"""Sync GET of a Spotify album_image_url. Returns the raw bytes or
|
||||
None on failure. Used by `embed_artwork_from_track` — kept sync so
|
||||
both `just collection`'s async loop and `just export`'s sync loop
|
||||
can call into the same helper without contortions."""
|
||||
if not url:
|
||||
return None
|
||||
try:
|
||||
import urllib.request
|
||||
req = urllib.request.Request(url, headers={"User-Agent": "maple/1.0"})
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
return resp.read()
|
||||
except Exception as e:
|
||||
logger.warning("artwork fetch failed for {}: {}", url, e)
|
||||
return None
|
||||
|
||||
|
||||
def embed_artwork_from_track(track: dict, path: Path, *, force: bool = False) -> bool:
|
||||
"""Fetch the Spotify cover art for `track` and embed it into `path`.
|
||||
|
||||
No-op when the file already has embedded art (unless `force=True`).
|
||||
Skips silently when the track dict has no `album_image_url` (Spotify
|
||||
occasionally returns tracks without images).
|
||||
"""
|
||||
if not force and has_embedded_art(path):
|
||||
return False
|
||||
url = (track.get("album_image_url") or "").strip()
|
||||
if not url:
|
||||
return False
|
||||
data = download_jpeg(url)
|
||||
if not data:
|
||||
return False
|
||||
try:
|
||||
embed_art(path, data)
|
||||
logger.info(
|
||||
"embedded artwork for {} (from {})",
|
||||
path.name, url.rsplit("/", 1)[-1][:40],
|
||||
)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.warning("embed_art failed for {}: {}", path.name, e)
|
||||
return False
|
||||
|
||||
|
||||
def enrich_from_manifest(
|
||||
path: Path,
|
||||
by_key: dict[tuple[str, str], dict],
|
||||
by_isrc: dict[str, dict],
|
||||
analysis_cache_dir: Path | None = None,
|
||||
) -> dict | None:
|
||||
"""Find this file's Spotify entry, bring it to "valid" state, return
|
||||
the matched track dict (or None if no match).
|
||||
|
||||
Steps:
|
||||
1. Write canonical text tags via `write_spotify_tags`.
|
||||
2. Fetch + embed album art if the file lacks it (`embed_artwork_from_track`).
|
||||
3. If `analysis_cache_dir` is supplied and the cache has an entry for
|
||||
this track's ISRC, write the cached BPM as a tag. This lets the
|
||||
enrich pass bring a track all the way to "validates" without a
|
||||
separate aubio-analysis step.
|
||||
"""
|
||||
track = find_spotify_track(path, by_key, by_isrc)
|
||||
if track is None:
|
||||
return None
|
||||
if write_spotify_tags(track, path):
|
||||
logger.info(
|
||||
"enriched {} from spotify (isrc={!r}, title={!r})",
|
||||
path.name, (track.get("isrc") or "")[:12], (track.get("title") or "")[:32],
|
||||
)
|
||||
embed_artwork_from_track(track, path)
|
||||
|
||||
# Backfill BPM tag from the analysis cache when present. Avoids
|
||||
# re-running aubio here — if the cache is cold, the per-track loop's
|
||||
# write_anlz will populate it and write the tag in a later pass.
|
||||
isrc = (track.get("isrc") or "").strip()
|
||||
if analysis_cache_dir is not None and isrc:
|
||||
from maple.analyze import Analysis
|
||||
cached = Analysis.load(analysis_cache_dir, isrc)
|
||||
if cached is not None and cached.bpm:
|
||||
write_bpm_tag(path, cached.bpm, overwrite=False)
|
||||
return track
|
||||
@@ -0,0 +1,20 @@
|
||||
"""Loguru sink config for maple's CLIs.
|
||||
|
||||
`configure(level)` (re)installs a single stderr sink with a compact format.
|
||||
Idempotent — safe to call from every CLI entrypoint.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
from loguru import logger
|
||||
|
||||
_FMT = "<green>{time:HH:mm:ss}</green> <level>{level:<7}</level> {message}"
|
||||
|
||||
|
||||
def configure(level: str | None = None) -> None:
|
||||
level = (level or os.environ.get("MAPLE_LOG_LEVEL") or "INFO").upper()
|
||||
logger.remove()
|
||||
logger.add(sys.stderr, format=_FMT, level=level, colorize=True, enqueue=False)
|
||||
|
||||
|
||||
__all__ = ["configure", "logger"]
|
||||
@@ -0,0 +1,105 @@
|
||||
"""Audio metadata extraction via mutagen."""
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from mutagen import File as MutagenFile
|
||||
|
||||
|
||||
DEFAULT_TEMPO_X100 = 12000 # 120.00 BPM, used when no BPM tag is present
|
||||
|
||||
|
||||
@dataclass
|
||||
class TrackMeta:
|
||||
title: str
|
||||
artist: str
|
||||
album: str
|
||||
genre: str
|
||||
track_number: int
|
||||
year: int
|
||||
duration: int
|
||||
bitrate: int
|
||||
sample_rate: int
|
||||
sample_depth: int
|
||||
file_size: int
|
||||
tempo: int = DEFAULT_TEMPO_X100 # BPM * 100 (rekordbox storage format)
|
||||
|
||||
|
||||
def _first(value) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
if isinstance(value, list):
|
||||
return str(value[0]) if value else ""
|
||||
return str(value)
|
||||
|
||||
|
||||
def _int(value) -> int:
|
||||
s = _first(value).strip()
|
||||
if not s:
|
||||
return 0
|
||||
# Handle "3/12" track numbers
|
||||
s = s.split("/")[0]
|
||||
try:
|
||||
return int(s)
|
||||
except ValueError:
|
||||
return 0
|
||||
|
||||
|
||||
def _tempo_x100(value) -> int:
|
||||
"""Parse a BPM tag into integer BPM * 100. 0 means "not present"."""
|
||||
s = _first(value).strip()
|
||||
if not s:
|
||||
return 0
|
||||
try:
|
||||
return int(round(float(s) * 100))
|
||||
except ValueError:
|
||||
return 0
|
||||
|
||||
|
||||
def read_tags(path: Path) -> TrackMeta:
|
||||
file_size = path.stat().st_size
|
||||
audio = MutagenFile(path, easy=True)
|
||||
if audio is None:
|
||||
return TrackMeta(
|
||||
title=path.stem,
|
||||
artist="Unknown Artist",
|
||||
album="Unknown Album",
|
||||
genre="",
|
||||
track_number=0,
|
||||
year=0,
|
||||
duration=0,
|
||||
bitrate=0,
|
||||
sample_rate=44100,
|
||||
sample_depth=16,
|
||||
file_size=file_size,
|
||||
)
|
||||
|
||||
tags = audio.tags or {}
|
||||
title = _first(tags.get("title")) or path.stem
|
||||
artist = _first(tags.get("artist")) or "Unknown Artist"
|
||||
album = _first(tags.get("album")) or "Unknown Album"
|
||||
genre = _first(tags.get("genre"))
|
||||
track_number = _int(tags.get("tracknumber"))
|
||||
year = _int(tags.get("date") or tags.get("year"))
|
||||
tempo = _tempo_x100(tags.get("bpm")) or DEFAULT_TEMPO_X100
|
||||
|
||||
info = getattr(audio, "info", None)
|
||||
duration = int(getattr(info, "length", 0) or 0)
|
||||
bitrate_bps = int(getattr(info, "bitrate", 0) or 0)
|
||||
bitrate = bitrate_bps // 1000 if bitrate_bps > 1000 else bitrate_bps
|
||||
sample_rate = int(getattr(info, "sample_rate", 44100) or 44100)
|
||||
sample_depth = int(getattr(info, "bits_per_sample", 16) or 16)
|
||||
|
||||
return TrackMeta(
|
||||
title=title,
|
||||
artist=artist,
|
||||
album=album,
|
||||
genre=genre,
|
||||
track_number=track_number,
|
||||
year=year,
|
||||
duration=duration,
|
||||
bitrate=bitrate,
|
||||
sample_rate=sample_rate,
|
||||
sample_depth=sample_depth,
|
||||
file_size=file_size,
|
||||
tempo=tempo,
|
||||
)
|
||||
@@ -0,0 +1,211 @@
|
||||
"""Domain model for the in-memory database that becomes a .pdb file."""
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@dataclass
|
||||
class Artist:
|
||||
id: int
|
||||
name: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class Album:
|
||||
id: int
|
||||
name: str
|
||||
artist_id: int
|
||||
|
||||
|
||||
@dataclass
|
||||
class Genre:
|
||||
id: int
|
||||
name: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class Color:
|
||||
id: int
|
||||
name: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class Key:
|
||||
id: int
|
||||
name: str
|
||||
id2: int | None = None # mirrors id in real exports
|
||||
|
||||
|
||||
@dataclass
|
||||
class Track:
|
||||
id: int
|
||||
title: str
|
||||
artist_id: int
|
||||
album_id: int
|
||||
filename: str
|
||||
file_path: str
|
||||
file_size: int = 0
|
||||
sample_rate: int = 44100
|
||||
sample_depth: int = 16
|
||||
bitrate: int = 0
|
||||
duration: int = 0
|
||||
track_number: int = 0
|
||||
year: int = 0
|
||||
disc_number: int = 0
|
||||
tempo: int = 0
|
||||
comment: str = ""
|
||||
date_added: str = ""
|
||||
release_date: str = ""
|
||||
mix_name: str = ""
|
||||
analyze_path: str = ""
|
||||
analyze_date: str = ""
|
||||
artwork_id: int = 0
|
||||
genre_id: int = 0
|
||||
source_path: Path | None = None
|
||||
|
||||
|
||||
@dataclass
|
||||
class Artwork:
|
||||
id: int
|
||||
path: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class Column:
|
||||
id: int
|
||||
code: int
|
||||
name: str
|
||||
|
||||
|
||||
# Reverse-engineered from a verified rekordbox export. Each entry is
|
||||
# (column_id, internal column code, display name). Code values are
|
||||
# non-contiguous because rekordbox skips reserved slots.
|
||||
DEFAULT_COLUMNS: list[Column] = [
|
||||
Column(1, 0x80, "GENRE"),
|
||||
Column(2, 0x81, "ARTIST"),
|
||||
Column(3, 0x82, "ALBUM"),
|
||||
Column(4, 0x83, "TRACK"),
|
||||
Column(5, 0x85, "BPM"),
|
||||
Column(6, 0x86, "RATING"),
|
||||
Column(7, 0x87, "YEAR"),
|
||||
Column(8, 0x88, "REMIXER"),
|
||||
Column(9, 0x89, "LABEL"),
|
||||
Column(10, 0x8A, "ORIGINAL ARTIST"),
|
||||
Column(11, 0x8B, "KEY"),
|
||||
Column(12, 0x8D, "CUE"),
|
||||
Column(13, 0x8E, "COLOR"),
|
||||
Column(14, 0x92, "TIME"),
|
||||
Column(15, 0x93, "BITRATE"),
|
||||
Column(16, 0x94, "FILE NAME"),
|
||||
Column(17, 0x84, "PLAYLIST"),
|
||||
Column(18, 0x98, "HOT CUE BANK"),
|
||||
Column(19, 0x95, "HISTORY"),
|
||||
Column(20, 0x91, "SEARCH"),
|
||||
Column(21, 0x96, "COMMENTS"),
|
||||
Column(22, 0x8C, "DATE ADDED"),
|
||||
Column(23, 0x97, "DJ PLAY COUNT"),
|
||||
Column(24, 0x90, "FOLDER"),
|
||||
Column(25, 0xA1, "DEFAULT"),
|
||||
Column(26, 0xA2, "ALPHABET"),
|
||||
Column(27, 0xAA, "MATCHING"),
|
||||
]
|
||||
|
||||
|
||||
# Reference-derived rows for the un-decoded `unknown_17` and `unknown_18`
|
||||
# tables. Each tuple is (a, b, c, d) packed as four little-endian u2s.
|
||||
# These appear to drive the player's "Filter by ->" menu (which columns
|
||||
# are filterable, sort order, filter type code). Speculative purpose;
|
||||
# verified to match the reference byte-for-byte.
|
||||
UNKNOWN_17_ROWS: list[tuple[int, int, int, int]] = [
|
||||
(1, 1, 355, 0), (5, 6, 261, 0), (6, 7, 355, 0), (7, 8, 355, 0),
|
||||
(8, 9, 355, 0), (9, 10, 355, 0), (10, 11, 355, 0), (13, 15, 355, 0),
|
||||
(14, 19, 260, 0), (15, 20, 262, 0), (16, 21, 355, 0), (18, 23, 355, 0),
|
||||
(2, 2, 2, 1), (3, 3, 3, 2), (4, 4, 1, 3), (11, 12, 99, 4),
|
||||
(17, 5, 99, 5), (19, 22, 99, 6), (20, 18, 99, 7), (27, 26, 611, 8),
|
||||
(24, 17, 99, 9), (22, 27, 99, 10),
|
||||
]
|
||||
UNKNOWN_18_ROWS: list[tuple[int, int, int, int]] = [
|
||||
(1, 6, 1, 0), (21, 7, 1, 0), (14, 8, 1, 0), (8, 9, 1, 0),
|
||||
(9, 10, 1, 0), (10, 11, 1, 0), (15, 13, 1, 0), (13, 15, 1, 0),
|
||||
(23, 16, 1, 0), (22, 17, 1, 0), (25, 0, 256, 0), (26, 1, 512, 0),
|
||||
(2, 2, 768, 0), (3, 3, 1024, 0), (5, 4, 1280, 0), (6, 5, 1536, 0),
|
||||
(11, 12, 1792, 0),
|
||||
]
|
||||
|
||||
|
||||
# The standard rekordbox color label palette.
|
||||
STANDARD_COLORS: list[Color] = [
|
||||
Color(1, "Pink"),
|
||||
Color(2, "Red"),
|
||||
Color(3, "Orange"),
|
||||
Color(4, "Yellow"),
|
||||
Color(5, "Green"),
|
||||
Color(6, "Aqua"),
|
||||
Color(7, "Blue"),
|
||||
Color(8, "Purple"),
|
||||
]
|
||||
|
||||
|
||||
@dataclass
|
||||
class PlaylistNode:
|
||||
id: int
|
||||
parent_id: int
|
||||
sort_order: int
|
||||
name: str
|
||||
is_folder: bool = False
|
||||
|
||||
|
||||
@dataclass
|
||||
class PlaylistEntry:
|
||||
playlist_id: int
|
||||
track_id: int
|
||||
entry_index: int
|
||||
|
||||
|
||||
@dataclass
|
||||
class Database:
|
||||
tracks: list[Track] = field(default_factory=list)
|
||||
artists: list[Artist] = field(default_factory=list)
|
||||
albums: list[Album] = field(default_factory=list)
|
||||
playlist_nodes: list[PlaylistNode] = field(default_factory=list)
|
||||
playlist_entries: list[PlaylistEntry] = field(default_factory=list)
|
||||
artwork: list[Artwork] = field(default_factory=list)
|
||||
genres: list[Genre] = field(default_factory=list)
|
||||
colors: list[Color] = field(default_factory=list)
|
||||
keys: list[Key] = field(default_factory=list)
|
||||
|
||||
|
||||
def validate_referential_integrity(db: "Database") -> list[str]:
|
||||
"""Return a list of dangling-foreign-key errors. Every FK must be 0 ("none")
|
||||
or resolve to a real row in its target table. A track that references a
|
||||
genre/artist/album/artwork id with no matching row is freeze-class: the
|
||||
player follows the FK into an empty/short table and hangs (this is exactly
|
||||
how build_acemo.py's omitted-genres bug froze the deck). Empty string = OK.
|
||||
"""
|
||||
errors: list[str] = []
|
||||
artist_ids = {a.id for a in db.artists}
|
||||
album_ids = {a.id for a in db.albums}
|
||||
artwork_ids = {a.id for a in db.artwork}
|
||||
genre_ids = {g.id for g in db.genres}
|
||||
key_ids = {k.id for k in db.keys}
|
||||
track_ids = {t.id for t in db.tracks}
|
||||
playlist_ids = {n.id for n in db.playlist_nodes}
|
||||
|
||||
def check(kind: str, owner_id: int, field: str, value: int, valid: set[int]) -> None:
|
||||
# 0 means "unset / no reference" everywhere in the pdb schema.
|
||||
if value and value not in valid:
|
||||
errors.append(f"{kind} id={owner_id}: {field}={value} has no matching row")
|
||||
|
||||
for t in db.tracks:
|
||||
check("track", t.id, "artist_id", t.artist_id, artist_ids)
|
||||
check("track", t.id, "album_id", t.album_id, album_ids)
|
||||
check("track", t.id, "artwork_id", t.artwork_id, artwork_ids)
|
||||
check("track", t.id, "genre_id", t.genre_id, genre_ids)
|
||||
for a in db.albums:
|
||||
check("album", a.id, "artist_id", a.artist_id, artist_ids)
|
||||
for e in db.playlist_entries:
|
||||
check("playlist_entry", e.playlist_id, "track_id", e.track_id, track_ids)
|
||||
check("playlist_entry", e.playlist_id, "playlist_id", e.playlist_id, playlist_ids)
|
||||
# key_ids is currently unreferenced by Track in maple's model; included for
|
||||
# completeness so adding a track.key_id later is covered.
|
||||
_ = key_ids
|
||||
return errors
|
||||
Whitespace-only changes.
@@ -0,0 +1,172 @@
|
||||
"""Page builder for the rekordbox .pdb format."""
|
||||
from struct import pack
|
||||
|
||||
PAGE_SIZE = 4096
|
||||
HEADER_SIZE = 40
|
||||
ROW_GROUP_SIZE = 36
|
||||
ROWS_PER_GROUP = 16
|
||||
|
||||
|
||||
def _align4(offset: int) -> int:
|
||||
return (offset + 3) & ~3
|
||||
|
||||
|
||||
def _num_row_groups(n: int) -> int:
|
||||
return (n + ROWS_PER_GROUP - 1) // ROWS_PER_GROUP
|
||||
|
||||
|
||||
# Strange-page heap is a DeviceSQL free-space map, byte-exact in a fresh export:
|
||||
# u4(page_index) + u4(data_page_ptr) + 12-byte const tail
|
||||
# + (f8 ff ff 1f)*1004 + 20 trailing zeros.
|
||||
# data_page_ptr is the table's first data-page index, or the sentinel
|
||||
# 0x03ffffff when the table is strange-only (no data page). The 1004 repeat
|
||||
# count is the same value stored in the header's unknown6 field (0x03ec).
|
||||
# Verified byte-for-byte against data/acemo_fresh strange pages.
|
||||
_STRANGE_HEAP_NO_DATA = 0x03FFFFFF
|
||||
_STRANGE_HEAP_TAIL_CONST = bytes.fromhex("ffffff03000000000000ff1f")
|
||||
_STRANGE_HEAP_PATTERN = bytes.fromhex("f8ffff1f")
|
||||
_STRANGE_HEAP_REPEAT = 1004
|
||||
_STRANGE_HEAP_TAIL = 20
|
||||
|
||||
# Strange pages on these table types carry unknown7=1; all others 0. In a fresh
|
||||
# export this is set for tracks (first table) and history (last table). Other
|
||||
# fields (free/used=0, txc/txi=0x1fff) match across all strange pages.
|
||||
STRANGE_UNKNOWN7 = frozenset({0, 19})
|
||||
|
||||
|
||||
def build_strange_page(page_index: int, page_type: int, next_page: int,
|
||||
data_page: int | None = None) -> bytes:
|
||||
"""Empty placeholder page that the reference exports use as the first page of
|
||||
every table chain. Reproduces a FRESH rekordbox export byte-for-byte: header
|
||||
flags/markers plus the DeviceSQL free-space heap map. `next_page` should be
|
||||
the reserved data-page index (page_index+1) even for strange-only tables --
|
||||
rekordbox always points the strange page at its (possibly-blank) data slot.
|
||||
`data_page` is the table's first data-page index, or None when the table is
|
||||
strange-only (the free-map then stores the 0x03ffffff sentinel there).
|
||||
"""
|
||||
header = b""
|
||||
header += pack("<I", 0) # magic
|
||||
header += pack("<I", page_index)
|
||||
header += pack("<I", page_type)
|
||||
header += pack("<I", next_page)
|
||||
header += pack("<I", 1) # sequence
|
||||
header += pack("<I", 0) # reserved
|
||||
# packed: num_row_offsets=0, num_rows=0, page_flags=0x64
|
||||
header += bytes([0, 0, 0, 0x64])
|
||||
header += pack("<H", 0) # free_size (fresh export = 0)
|
||||
header += pack("<H", 0) # used_size (fresh export = 0)
|
||||
header += pack("<H", 0x1fff) # transaction_row_count = failed
|
||||
header += pack("<H", 0x1fff) # transaction_row_index = failed
|
||||
header += pack("<H", 0x03ec) # 1004 (free-map repeat count)
|
||||
header += pack("<H", 1 if page_type in STRANGE_UNKNOWN7 else 0) # unknown7
|
||||
assert len(header) == HEADER_SIZE
|
||||
|
||||
data_ptr = _STRANGE_HEAP_NO_DATA if data_page is None else data_page
|
||||
heap = pack("<II", page_index, data_ptr) + _STRANGE_HEAP_TAIL_CONST
|
||||
heap += _STRANGE_HEAP_PATTERN * _STRANGE_HEAP_REPEAT
|
||||
heap += b"\x00" * _STRANGE_HEAP_TAIL
|
||||
assert len(heap) == PAGE_SIZE - HEADER_SIZE
|
||||
return header + heap
|
||||
|
||||
|
||||
class PageBuilder:
|
||||
def __init__(self, page_index: int, page_type: int, max_rows: int = 0,
|
||||
data_flag: int = 0x24, has_index_shift: bool = False):
|
||||
self.page_index = page_index
|
||||
self.page_type = page_type
|
||||
self.next_page = page_index # caller patches before finalize
|
||||
self.max_rows = max_rows
|
||||
self.data_flag = data_flag # page_flags for a populated data page
|
||||
# Row types whose layout begins `subtype:u2, index_shift:u2` (tracks,
|
||||
# artists, albums) carry a slot-based index_shift at row offset 0x02.
|
||||
# See FORMAT_SPEC.md s1: index_shift = slot_index << 5, where slot_index
|
||||
# is the row's 0-based position in the page row index. maple never
|
||||
# deletes rows, so slot_index == the row's ordinal on the page.
|
||||
self.has_index_shift = has_index_shift
|
||||
self.row_bytes: list[bytes] = []
|
||||
self.row_offsets: list[int] = [] # relative to heap_pos
|
||||
self.heap_cursor = 0 # position relative to heap_pos
|
||||
|
||||
def try_add(self, row: bytes) -> bool:
|
||||
aligned = _align4(self.heap_cursor)
|
||||
end = aligned + len(row)
|
||||
footer_after = _num_row_groups(len(self.row_bytes) + 1) * ROW_GROUP_SIZE
|
||||
if end + HEADER_SIZE + footer_after > PAGE_SIZE:
|
||||
return False
|
||||
if self.max_rows and len(self.row_bytes) >= self.max_rows:
|
||||
return False
|
||||
self.row_offsets.append(aligned)
|
||||
self.row_bytes.append(row)
|
||||
self.heap_cursor = end
|
||||
return True
|
||||
|
||||
def num_rows(self) -> int:
|
||||
return len(self.row_bytes)
|
||||
|
||||
def finalize(self) -> bytes:
|
||||
n = len(self.row_bytes)
|
||||
num_rgs = _num_row_groups(n)
|
||||
footer_size = num_rgs * ROW_GROUP_SIZE
|
||||
heap_padding = PAGE_SIZE - HEADER_SIZE - footer_size
|
||||
used_size = self.heap_cursor
|
||||
free_size = heap_padding - used_size
|
||||
if free_size < 0:
|
||||
raise ValueError("page overflow")
|
||||
|
||||
# Data pages with rows: rekordbox uses 0x24 (and 0x34 for some tables --
|
||||
# see writer's PAGE_FLAG_34). maple previously wrote 0x00, which is
|
||||
# missing the 0x20|0x04 bits every real rekordbox data page sets, and
|
||||
# the players reject it. Empty data pages stay 0x44.
|
||||
page_flags = self.data_flag if n > 0 else 0x44
|
||||
|
||||
header = b""
|
||||
header += pack("<I", 0) # magic
|
||||
header += pack("<I", self.page_index)
|
||||
header += pack("<I", self.page_type)
|
||||
header += pack("<I", self.next_page)
|
||||
header += pack("<I", 0) # unknown1
|
||||
header += pack("<I", 0) # unknown2
|
||||
# 24 bits little-endian: low 13 = num_row_offsets, next 11 = num_rows,
|
||||
# then u1 page_flags. Kaitai reads these as read_bits_int_le.
|
||||
packed = ((n & 0x7FF) << 13) | (n & 0x1FFF)
|
||||
header += bytes([packed & 0xFF, (packed >> 8) & 0xFF, (packed >> 16) & 0xFF, page_flags])
|
||||
header += pack("<HH", free_size, used_size)
|
||||
header += pack("<H", 0) # transaction_row_count
|
||||
header += pack("<H", 0) # transaction_row_index
|
||||
header += pack("<H", 0) # unknown6
|
||||
header += pack("<H", 0) # unknown7
|
||||
assert len(header) == HEADER_SIZE
|
||||
|
||||
heap = bytearray(heap_padding)
|
||||
for slot, (ofs, row) in enumerate(zip(self.row_offsets, self.row_bytes)):
|
||||
if self.has_index_shift:
|
||||
# Patch index_shift (u2 at row offset 0x02) from the placeholder
|
||||
# to slot_index<<5 (FORMAT_SPEC.md s1).
|
||||
row = row[:2] + pack("<H", (slot << 5) & 0xFFFF) + row[4:]
|
||||
heap[ofs:ofs + len(row)] = row
|
||||
|
||||
# Footers go after the heap in the file. Group 0 (rows 0-15) sits closest
|
||||
# to the end of the page; group num_rgs-1 sits closest to the heap. So
|
||||
# emit footers in reverse group order.
|
||||
footer = b""
|
||||
for g in range(num_rgs - 1, -1, -1):
|
||||
start = g * ROWS_PER_GROUP
|
||||
grp_offsets = self.row_offsets[start:start + ROWS_PER_GROUP]
|
||||
grp_count = len(grp_offsets)
|
||||
padded = grp_offsets + [0] * (ROWS_PER_GROUP - grp_count)
|
||||
presence = (1 << grp_count) - 1
|
||||
rg = b""
|
||||
for ofs in reversed(padded):
|
||||
rg += pack("<H", ofs)
|
||||
rg += pack("<H", presence)
|
||||
# Trailing u2: on a fresh, fully-packed page (no deleted slots) this
|
||||
# equals the presence mask. Verified across every clean row group in
|
||||
# data/acemo_fresh; pages where ref's value differs from presence are
|
||||
# exactly the dead-slot/edit-history pages (see KNOWN_DIVERGENCE.md).
|
||||
# Was a constant 16 (rekordcrate default), which never matched.
|
||||
rg += pack("<H", presence)
|
||||
assert len(rg) == ROW_GROUP_SIZE
|
||||
footer += rg
|
||||
assert len(footer) == footer_size
|
||||
|
||||
return header + bytes(heap) + footer
|
||||
@@ -0,0 +1,251 @@
|
||||
"""Row serialisers for the rekordbox export.pdb format."""
|
||||
import zlib
|
||||
from struct import pack
|
||||
|
||||
from maple.model import Album, Artist, Artwork, Color, Column, Genre, PlaylistEntry, PlaylistNode, Track
|
||||
from maple.pdb.strings import encode_devicesql
|
||||
|
||||
TRACK_FIXED_PREFIX = 0x5E
|
||||
TRACK_OFS_STRINGS_START = TRACK_FIXED_PREFIX
|
||||
TRACK_HEADER_LEN = TRACK_OFS_STRINGS_START + 21 * 2
|
||||
|
||||
# Per-rekordbox-install seed for the track opaque-id blob (row 0x18, u4). It is
|
||||
# a master.db value, constant across a single install, not derivable from track
|
||||
# metadata (FORMAT_SPEC.md s2). EVERY real rekordbox export we have carries a
|
||||
# NON-ZERO seed (this install = 0x21b4a7ab; data/MAPLE = 0xd6d3a567); the
|
||||
# 0x78f74a68 "unanalyzed default" was a false lead -- it came from maple's own
|
||||
# earlier output (data/usb), not a real export. Default to this install's real
|
||||
# value; override per-install if deck testing shows it matters. Players treat
|
||||
# the whole blob as opaque.
|
||||
# Per-export "seed" stored at offset 0x18 of every TrackRow. Real rekordbox
|
||||
# files carry the LE u4 0xd6d3a567 here (parses as opaque_seed_lo=0xa567,
|
||||
# opaque_seed_hi=0xd6d3). Maple previously used 0x21B4A7AB which the kaitai
|
||||
# would parse as different seed_lo/hi values — the difference may be benign
|
||||
# for the CDJ's analyse-path lookup but matches every real reference export
|
||||
# we have, so we mirror it.
|
||||
TRACK_OPAQUE_SEED = 0xD6D3A567
|
||||
|
||||
# Range maple maps its stand-in opaque_track_id_lo (row 0x14) into, so it has the
|
||||
# shape of a real rekordbox Content ID. The real AceMo lo values all fall in this
|
||||
# band (observed 0x02c0a654 .. 0x0f8fd598), i.e. ~134M-261M. We can't reproduce
|
||||
# the true (encrypted-DB) Content ID, so we map a stable per-track hash into the
|
||||
# band -- non-zero, in-range, deterministic. Documented divergence (FORMAT_SPEC s2).
|
||||
OPAQUE_LO_MIN = 0x08000000
|
||||
OPAQUE_LO_MAX = 0x0F900000
|
||||
|
||||
|
||||
def _pad2(b: bytes) -> bytes:
|
||||
return b + b"\x00" if len(b) & 1 else b
|
||||
|
||||
|
||||
def serialize_track(t: Track) -> bytes:
|
||||
strings = [
|
||||
"", # 0 isrc
|
||||
"", # 1 texter
|
||||
"1", # 2 unknown_string_2
|
||||
"2", # 3 unknown_string_3
|
||||
"", # 4 unknown_string_4
|
||||
"", # 5 message
|
||||
"ON", # 6 kuvo_public
|
||||
"ON", # 7 autoload_hot_cues
|
||||
"", # 8 unknown_string_5
|
||||
"", # 9 unknown_string_6
|
||||
t.date_added, # 10 date_added
|
||||
t.release_date, # 11 release_date
|
||||
t.mix_name, # 12 mix_name
|
||||
"", # 13 unknown_string_7
|
||||
t.analyze_path, # 14 analyze_path
|
||||
t.analyze_date, # 15 analyze_date
|
||||
t.comment, # 16 comment
|
||||
t.title, # 17 title
|
||||
"", # 18 unknown_string_8
|
||||
t.filename, # 19 filename
|
||||
t.file_path, # 20 file_path
|
||||
]
|
||||
|
||||
encoded = [encode_devicesql(s) for s in strings]
|
||||
|
||||
offsets = []
|
||||
cursor = TRACK_HEADER_LEN
|
||||
blob_parts = []
|
||||
for enc in encoded:
|
||||
offsets.append(cursor)
|
||||
blob_parts.append(enc)
|
||||
cursor += len(enc)
|
||||
blob = b"".join(blob_parts)
|
||||
|
||||
prefix = b""
|
||||
prefix += pack("<HH", 0x24, 0) # subtype, index_shift
|
||||
prefix += pack("<I", 788224) # bitmask
|
||||
prefix += pack("<I", t.sample_rate) # sample_rate
|
||||
prefix += pack("<I", 0) # composer_id
|
||||
prefix += pack("<I", t.file_size) # file_size
|
||||
# offsets 0x14/0x18/0x1a are ONE 8-byte LE opaque id:
|
||||
# blob_u64 = opaque_track_id_lo | (opaque_seed << 32)
|
||||
# Both halves are rekordbox-internal stored values (master.db / djmdContent
|
||||
# + a per-install seed), NOT a function of the track metadata maple has --
|
||||
# proven across the real exports + an exhaustive negative hash search
|
||||
# (FORMAT_SPEC.md s2). They cannot be reproduced byte-for-byte without
|
||||
# rekordbox's DB; that is a documented divergence (KNOWN_DIVERGENCE.md).
|
||||
# Every REAL export has BOTH halves non-zero, so emit a non-zero stand-in of
|
||||
# the right SHAPE: the per-install seed, and a per-track lo mapped into the
|
||||
# real Content-ID band (OPAQUE_LO_MIN..MAX) -- which is where every observed
|
||||
# AceMo lo lands. Stable hash -> band, so it's deterministic and in-range.
|
||||
# (Was the 0xa567/0xd6d3 marker, which matched no real export; lo=0/
|
||||
# seed=0x78f74a68 was also wrong -- that came from maple's own output
|
||||
# mistaken for an "unanalyzed library", not a real rekordbox export.)
|
||||
h = zlib.crc32(f"{t.id}|{t.file_path}".encode()) & 0xFFFFFFFF
|
||||
opaque_lo = OPAQUE_LO_MIN + (h % (OPAQUE_LO_MAX - OPAQUE_LO_MIN))
|
||||
prefix += pack("<I", opaque_lo) # opaque_track_id_lo (stand-in, per track)
|
||||
prefix += pack("<I", TRACK_OPAQUE_SEED) # opaque_seed (u4 @0x18 = lo|hi<<16)
|
||||
prefix += pack("<I", t.artwork_id) # artwork_id
|
||||
prefix += pack("<I", 0) # key_id
|
||||
prefix += pack("<I", 0) # original_artist_id
|
||||
prefix += pack("<I", 0) # label_id
|
||||
prefix += pack("<I", 0) # remixer_id
|
||||
prefix += pack("<I", t.bitrate) # bitrate
|
||||
prefix += pack("<I", t.track_number) # track_number
|
||||
prefix += pack("<I", t.tempo) # tempo
|
||||
prefix += pack("<I", t.genre_id) # genre_id
|
||||
prefix += pack("<I", t.album_id) # album_id
|
||||
prefix += pack("<I", t.artist_id) # artist_id
|
||||
prefix += pack("<I", t.id) # id
|
||||
prefix += pack("<H", t.disc_number) # disc_number
|
||||
prefix += pack("<H", 0) # play_count
|
||||
prefix += pack("<H", t.year) # year
|
||||
prefix += pack("<H", t.sample_depth) # sample_depth
|
||||
prefix += pack("<H", t.duration) # duration
|
||||
prefix += pack("<H", 41) # unknown ("always 41?")
|
||||
prefix += pack("<BB", 0, 0) # color_id, rating
|
||||
prefix += pack("<H", 1) # unknown ("always 1?")
|
||||
prefix += pack("<H", 3) # unknown ("alternating 2 or 3")
|
||||
|
||||
assert len(prefix) == TRACK_FIXED_PREFIX, len(prefix)
|
||||
|
||||
ofs_block = b"".join(pack("<H", o) for o in offsets)
|
||||
row = prefix + ofs_block + blob
|
||||
return _pad2(row)
|
||||
|
||||
|
||||
def serialize_artist(a: Artist) -> bytes:
|
||||
name = encode_devicesql(a.name)
|
||||
near_ofs = 0x0A
|
||||
far_threshold = 0x100
|
||||
if near_ofs < far_threshold:
|
||||
subtype = 0x60
|
||||
row = b""
|
||||
# index_shift is slot-based (slot_index<<5); the pager patches it at
|
||||
# page-build time -- see FORMAT_SPEC.md s1. Write 0 as a placeholder.
|
||||
row += pack("<HH", subtype, 0) # subtype, index_shift
|
||||
row += pack("<I", a.id) # id
|
||||
row += pack("<B", 0x03) # always 0x03
|
||||
row += pack("<B", near_ofs) # ofs_name_near
|
||||
assert len(row) == near_ofs
|
||||
row += name
|
||||
else:
|
||||
subtype = 0x64
|
||||
# Far form: fixed prefix + u2 ofs_name_far at row+0x0a
|
||||
# Layout: subtype u2, index_shift u2, id u4, 0x03 u1, ofs_name_near u1, ofs_name_far u2 -> name follows at offset 0x0e
|
||||
far = 0x0E
|
||||
row = b""
|
||||
# index_shift patched by the pager (slot_index<<5); 0 placeholder.
|
||||
row += pack("<HH", subtype, 0)
|
||||
row += pack("<I", a.id)
|
||||
row += pack("<B", 0x03)
|
||||
row += pack("<B", 0)
|
||||
row += pack("<H", far)
|
||||
row += name
|
||||
return _pad2(row)
|
||||
|
||||
|
||||
def serialize_album(a: Album) -> bytes:
|
||||
name = encode_devicesql(a.name)
|
||||
near_ofs = 0x16
|
||||
subtype = 0x80
|
||||
row = b""
|
||||
# index_shift patched by the pager (slot_index<<5); 0 placeholder.
|
||||
row += pack("<HH", subtype, 0) # subtype, index_shift
|
||||
row += pack("<I", 0) # unknown
|
||||
row += pack("<I", a.artist_id) # artist_id
|
||||
row += pack("<I", a.id) # id
|
||||
row += pack("<I", 0) # unknown
|
||||
row += pack("<B", 0x03) # always 0x03
|
||||
row += pack("<B", near_ofs) # ofs_name_near
|
||||
assert len(row) == near_ofs
|
||||
row += name
|
||||
return _pad2(row)
|
||||
|
||||
|
||||
def serialize_playlist_tree(n: PlaylistNode) -> bytes:
|
||||
row = b""
|
||||
row += pack("<I", n.parent_id)
|
||||
row += pack("<I", 0)
|
||||
row += pack("<I", n.sort_order)
|
||||
row += pack("<I", n.id)
|
||||
row += pack("<I", 1 if n.is_folder else 0)
|
||||
row += encode_devicesql(n.name)
|
||||
return _pad2(row)
|
||||
|
||||
|
||||
def serialize_playlist_entry(e: PlaylistEntry) -> bytes:
|
||||
return pack("<III", e.entry_index, e.track_id, e.playlist_id)
|
||||
|
||||
|
||||
def serialize_artwork(a: Artwork) -> bytes:
|
||||
row = pack("<I", a.id) + encode_devicesql(a.path)
|
||||
return _pad2(row)
|
||||
|
||||
|
||||
def serialize_genre(g: Genre) -> bytes:
|
||||
row = pack("<I", g.id) + encode_devicesql(g.name)
|
||||
return _pad2(row)
|
||||
|
||||
|
||||
def serialize_key(key_id: int, name: str, id2: int | None = None) -> bytes:
|
||||
"""key_row: u4 id, u4 id2 (mirrors id in the reference), devicesql name."""
|
||||
if id2 is None:
|
||||
id2 = key_id
|
||||
return _pad2(pack("<II", key_id, id2) + encode_devicesql(name))
|
||||
|
||||
|
||||
def serialize_history_playlist(playlist_id: int, name: str) -> bytes:
|
||||
"""history_playlist_row: u4 id, devicesql name."""
|
||||
return _pad2(pack("<I", playlist_id) + encode_devicesql(name))
|
||||
|
||||
|
||||
def serialize_history_entry(track_id: int, playlist_id: int, entry_index: int) -> bytes:
|
||||
"""history_entry_row: u4 track_id, u4 playlist_id, u4 entry_index."""
|
||||
return pack("<III", track_id, playlist_id, entry_index)
|
||||
|
||||
|
||||
def serialize_history_blob(date_added: str) -> bytes:
|
||||
"""The `history` table row format isn't decoded by the kaitai schema.
|
||||
A 40-byte blob copied from the reference, with the date substituted.
|
||||
"""
|
||||
date_str = encode_devicesql(date_added) # short ASCII "2026-05-26"
|
||||
bucket = encode_devicesql("1000")
|
||||
empty = encode_devicesql("")
|
||||
head = pack("<HHH", 0x0280, 0, 0xb0) + b"\x00" * 6
|
||||
body = head + date_str + b"\x19\x1e" + bucket + empty
|
||||
# pad to 40 bytes
|
||||
if len(body) < 40:
|
||||
body += b"\x00" * (40 - len(body))
|
||||
return body[:40]
|
||||
|
||||
|
||||
def serialize_color(c: Color) -> bytes:
|
||||
# Schema: 5 bytes padding, u2 id, u1 (zero), then name string.
|
||||
# The reference puts the low byte of id at byte 4 (last of the padding).
|
||||
row = b"\x00\x00\x00\x00" + bytes([c.id & 0xFF]) + pack("<H", c.id & 0xFFFF) + b"\x00" + encode_devicesql(c.name)
|
||||
return _pad2(row)
|
||||
|
||||
|
||||
def serialize_column(c: Column) -> bytes:
|
||||
"""A column row is: u2 id, u2 code, then a 'long' devicesql string whose
|
||||
UTF-16 body is wrapped by the sentinel codepoints U+FFFA / U+FFFB.
|
||||
"""
|
||||
body = b"\xfa\xff" + c.name.encode("utf-16-le") + b"\xfb\xff"
|
||||
total_length = len(body) + 4 # 4 = 0x90 marker + u2 length + u1 pad
|
||||
name_str = b"\x90" + pack("<H", total_length) + b"\x00" + body
|
||||
row = pack("<HH", c.id, c.code) + name_str
|
||||
return _pad2(row)
|
||||
@@ -0,0 +1,35 @@
|
||||
"""DeviceSQL variable-length string encoding.
|
||||
|
||||
Three on-disk forms (see rekordbox-pdb.ksy `device_sql_string`):
|
||||
- odd length_and_kind byte -> short ASCII (< 127 chars)
|
||||
- 0x40 -> long ASCII (any length, ASCII bytes)
|
||||
- 0x90 -> long UTF-16LE (for non-ASCII content)
|
||||
|
||||
We pick the cheapest form that fits the content. Long-but-ASCII paths
|
||||
must use 0x40, NOT 0x90 -- some player firmwares decode 0x90 strings
|
||||
as UTF-16 and then fail the case-sensitive filesystem lookup, so any
|
||||
track whose `file_path` lands in the wrong form silently goes missing.
|
||||
"""
|
||||
|
||||
EMPTY = b"\x03"
|
||||
|
||||
|
||||
def encode_devicesql(s: str) -> bytes:
|
||||
if not s:
|
||||
return EMPTY
|
||||
try:
|
||||
ascii_bytes = s.encode("ascii")
|
||||
except UnicodeEncodeError:
|
||||
ascii_bytes = None
|
||||
|
||||
if ascii_bytes is not None:
|
||||
if len(ascii_bytes) < 127:
|
||||
length = len(ascii_bytes) + 1
|
||||
length_and_kind = ((length << 1) | 1) & 0xFF
|
||||
return bytes([length_and_kind]) + ascii_bytes
|
||||
total_length = len(ascii_bytes) + 4 # marker + u2 length + u1 pad
|
||||
return bytes([0x40]) + total_length.to_bytes(2, "little") + b"\x00" + ascii_bytes
|
||||
|
||||
payload = s.encode("utf-16-le")
|
||||
total_length = len(payload) + 4
|
||||
return bytes([0x90]) + total_length.to_bytes(2, "little") + b"\x00" + payload
|
||||
@@ -0,0 +1,288 @@
|
||||
"""Top-level rekordbox .pdb file writer."""
|
||||
from pathlib import Path
|
||||
from struct import pack, unpack_from
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from maple.model import (
|
||||
DEFAULT_COLUMNS,
|
||||
STANDARD_COLORS,
|
||||
UNKNOWN_17_ROWS,
|
||||
UNKNOWN_18_ROWS,
|
||||
Database,
|
||||
validate_referential_integrity,
|
||||
)
|
||||
from maple.pdb.pager import PAGE_SIZE, PageBuilder, build_strange_page
|
||||
from maple.pdb.rows import (
|
||||
serialize_album,
|
||||
serialize_artist,
|
||||
serialize_artwork,
|
||||
serialize_color,
|
||||
serialize_column,
|
||||
serialize_genre,
|
||||
serialize_history_blob,
|
||||
serialize_key,
|
||||
serialize_playlist_entry,
|
||||
serialize_playlist_tree,
|
||||
serialize_track,
|
||||
)
|
||||
|
||||
PAGE_TYPE_TRACKS = 0
|
||||
PAGE_TYPE_GENRES = 1
|
||||
PAGE_TYPE_ARTISTS = 2
|
||||
PAGE_TYPE_ALBUMS = 3
|
||||
PAGE_TYPE_LABELS = 4
|
||||
PAGE_TYPE_KEYS = 5
|
||||
PAGE_TYPE_COLORS = 6
|
||||
PAGE_TYPE_PLAYLIST_TREE = 7
|
||||
PAGE_TYPE_PLAYLIST_ENTRIES = 8
|
||||
PAGE_TYPE_UNKNOWN_9 = 9
|
||||
PAGE_TYPE_UNKNOWN_10 = 10
|
||||
PAGE_TYPE_HISTORY_PLAYLISTS = 11
|
||||
PAGE_TYPE_HISTORY_ENTRIES = 12
|
||||
PAGE_TYPE_ARTWORK = 13
|
||||
PAGE_TYPE_UNKNOWN_14 = 14
|
||||
PAGE_TYPE_UNKNOWN_15 = 15
|
||||
PAGE_TYPE_COLUMNS = 16
|
||||
PAGE_TYPE_UNKNOWN_17 = 17
|
||||
PAGE_TYPE_UNKNOWN_18 = 18
|
||||
PAGE_TYPE_HISTORY = 19
|
||||
|
||||
ALL_PAGE_TYPES = list(range(20))
|
||||
|
||||
FILE_HEADER_SIZE = 28 # 0,page_size,num_tables,next_unused,unknown,sequence,gap = 7*u4
|
||||
TABLE_ENTRY_SIZE = 16 # page_type,empty_candidate,first_page,last_page = 4*u4
|
||||
|
||||
|
||||
# Page types whose populated data pages rekordbox tags 0x34 (vs the 0x24 the
|
||||
# rest get). In a FRESH from-scratch export (data/acemo_fresh) only tracks and
|
||||
# history carry 0x34; every other table's data page is 0x24. (The wider set
|
||||
# {0,2,3,13,17,19} was matched against the EDITED reference, whose extra 0x34
|
||||
# pages were edit-history artifacts.) These are the same two tables that get
|
||||
# strange-page unknown7=1 -- see pager.STRANGE_UNKNOWN7.
|
||||
PAGE_FLAG_34 = frozenset({0, 19}) # tracks, history
|
||||
|
||||
# Tables that get a real data page even when they have zero rows. A FRESH
|
||||
# from-scratch rekordbox export (data/acemo_fresh) gives a data page to a table
|
||||
# IFF it has rows -- every empty table is strange-page-only. The earlier
|
||||
# non-empty CORE_TABLES set was matched against the EDITED reference
|
||||
# (data/acemo), whose genres table (type 1) carried a leftover empty data page
|
||||
# from a deleted genre row -- an edit-history artifact, not a format rule (see
|
||||
# KNOWN_DIVERGENCE.md). Against the fresh reference the correct rule is
|
||||
# "data page iff rows>0", i.e. no table is forced. Kept as an (empty) hook in
|
||||
# case a future fresh export proves some table is always materialised.
|
||||
CORE_TABLES: frozenset[int] = frozenset()
|
||||
|
||||
# Row types whose layout begins `subtype:u2, index_shift:u2`, so they carry a
|
||||
# slot-based index_shift the pager must patch (FORMAT_SPEC.md s1): tracks,
|
||||
# artists, albums.
|
||||
INDEX_SHIFT_TABLES = frozenset({0, 2, 3})
|
||||
|
||||
|
||||
SEQUENCE_OFFSET = 20 # u4 offset of `sequence` in the file header (page 0)
|
||||
|
||||
|
||||
def _next_sequence(path: Path) -> int:
|
||||
"""rekordbox's file-header `sequence` is an internal transaction counter from
|
||||
its master.db -- we can't reproduce its exact value. The faithful behaviour is
|
||||
to monotonically increment: read the sequence already in the destination
|
||||
export (if any) and add 1, defaulting to 1 for a brand-new file. This keeps
|
||||
each export's sequence strictly greater than the last, as rekordbox does.
|
||||
"""
|
||||
try:
|
||||
head = path.read_bytes()[:FILE_HEADER_SIZE]
|
||||
except (FileNotFoundError, OSError):
|
||||
return 1
|
||||
if len(head) < FILE_HEADER_SIZE:
|
||||
return 1
|
||||
existing = unpack_from("<I", head, SEQUENCE_OFFSET)[0]
|
||||
return existing + 1 if existing else 1
|
||||
|
||||
|
||||
def write_pdb(db: Database, path: str | Path) -> None:
|
||||
logger.info(
|
||||
"building pdb → {} (tracks={} artists={} albums={} genres={} "
|
||||
"artwork={} playlists={} entries={})",
|
||||
path,
|
||||
len(db.tracks), len(db.artists), len(db.albums), len(db.genres),
|
||||
len(db.artwork), len(db.playlist_nodes), len(db.playlist_entries),
|
||||
)
|
||||
# Refuse to emit a pdb with dangling foreign keys -- a track pointing at a
|
||||
# genre/artist/album/artwork row that doesn't exist makes the player follow
|
||||
# the FK into an empty table and freeze (freeze-class, like the deck hang).
|
||||
fk_errors = validate_referential_integrity(db)
|
||||
if fk_errors:
|
||||
logger.error("FK validation failed with {} error(s)", len(fk_errors))
|
||||
for e in fk_errors:
|
||||
logger.error(" {}", e)
|
||||
raise ValueError("dangling foreign keys (would freeze the player):\n " + "\n ".join(fk_errors))
|
||||
|
||||
# Layout (matching rekordbox): page 0 is the file header. Pages 1..2*N are
|
||||
# reserved 2-per-table -- strange page at the odd index 1+2k, first data page
|
||||
# at the even index 2+2k. Overflow data pages come from the pool at 2*N+1.
|
||||
serialized_tracks = [serialize_track(t) for t in db.tracks]
|
||||
serialized_artists = [serialize_artist(a) for a in db.artists]
|
||||
serialized_albums = [serialize_album(a) for a in db.albums]
|
||||
serialized_pl_tree = [serialize_playlist_tree(n) for n in db.playlist_nodes]
|
||||
serialized_pl_entries = [serialize_playlist_entry(e) for e in db.playlist_entries]
|
||||
serialized_artwork = [serialize_artwork(a) for a in db.artwork]
|
||||
serialized_columns = [serialize_column(c) for c in DEFAULT_COLUMNS]
|
||||
serialized_genres = [serialize_genre(g) for g in db.genres]
|
||||
# If the caller didn't populate colors, always emit the standard palette
|
||||
# so the player's Color filter has a non-empty value set.
|
||||
colors = db.colors if db.colors else STANDARD_COLORS
|
||||
serialized_colors = [serialize_color(c) for c in colors]
|
||||
serialized_keys = [serialize_key(k.id, k.name, k.id2) for k in db.keys]
|
||||
# unknown_17/unknown_18 row format isn't decoded by the kaitai schema;
|
||||
# we emit the reference's byte-for-byte rows (four little-endian u2s).
|
||||
serialized_unknown_17 = [pack("<HHHH", *r) for r in UNKNOWN_17_ROWS]
|
||||
serialized_unknown_18 = [pack("<HHHH", *r) for r in UNKNOWN_18_ROWS]
|
||||
|
||||
# A real rekordbox export leaves history_playlists and history_entries
|
||||
# EMPTY (only the `history` table itself carries a row). maple used to
|
||||
# fabricate a dummy "HISTORY 001" playlist + entry referencing a track,
|
||||
# which the player doesn't expect -- drop them and match the reference.
|
||||
today = db.tracks[0].date_added if db.tracks else "2026-01-01"
|
||||
serialized_history_playlists: list[bytes] = []
|
||||
serialized_history_entries: list[bytes] = []
|
||||
serialized_history = [serialize_history_blob(today)]
|
||||
|
||||
rows_by_type: dict[int, list[bytes]] = {
|
||||
PAGE_TYPE_TRACKS: serialized_tracks,
|
||||
PAGE_TYPE_GENRES: serialized_genres,
|
||||
PAGE_TYPE_ARTISTS: serialized_artists,
|
||||
PAGE_TYPE_ALBUMS: serialized_albums,
|
||||
PAGE_TYPE_KEYS: serialized_keys,
|
||||
PAGE_TYPE_COLORS: serialized_colors,
|
||||
PAGE_TYPE_PLAYLIST_TREE: serialized_pl_tree,
|
||||
PAGE_TYPE_PLAYLIST_ENTRIES: serialized_pl_entries,
|
||||
PAGE_TYPE_ARTWORK: serialized_artwork,
|
||||
PAGE_TYPE_COLUMNS: serialized_columns,
|
||||
PAGE_TYPE_UNKNOWN_17: serialized_unknown_17,
|
||||
PAGE_TYPE_UNKNOWN_18: serialized_unknown_18,
|
||||
PAGE_TYPE_HISTORY_PLAYLISTS: serialized_history_playlists,
|
||||
PAGE_TYPE_HISTORY_ENTRIES: serialized_history_entries,
|
||||
PAGE_TYPE_HISTORY: serialized_history,
|
||||
}
|
||||
num_tables = len(ALL_PAGE_TYPES) # 20
|
||||
overflow_next = 1 + 2 * num_tables # first overflow page (41)
|
||||
pages_by_index: dict[int, bytes] = {} # page_index -> 4096-byte page
|
||||
table_dir: list[tuple[int, int, int]] = [] # (page_type, first_page, last_page)
|
||||
last_builders: list[PageBuilder] = [] # last data page of each chain, patched below
|
||||
|
||||
for k, ptype in enumerate(ALL_PAGE_TYPES):
|
||||
strange_idx = 1 + 2 * k
|
||||
data_idx = 2 + 2 * k
|
||||
rows = rows_by_type.get(ptype, [])
|
||||
data_flag = 0x34 if ptype in PAGE_FLAG_34 else 0x24
|
||||
has_index_shift = ptype in INDEX_SHIFT_TABLES
|
||||
|
||||
# Optional table with no rows: strange page only (its even page stays
|
||||
# blank). rekordbox still points the strange page at the reserved data
|
||||
# index (strange_idx+1 == data_idx), and the table dir's last_page is the
|
||||
# strange page itself.
|
||||
if not rows and ptype not in CORE_TABLES:
|
||||
pages_by_index[strange_idx] = build_strange_page(
|
||||
strange_idx, ptype, next_page=data_idx, data_page=None)
|
||||
table_dir.append((ptype, strange_idx, strange_idx))
|
||||
continue
|
||||
|
||||
# First data page at the reserved even index; overflow pages from the pool.
|
||||
builders: list[PageBuilder] = [
|
||||
PageBuilder(data_idx, ptype, data_flag=data_flag, has_index_shift=has_index_shift)
|
||||
]
|
||||
for row in rows:
|
||||
if not builders[-1].try_add(row):
|
||||
nb = PageBuilder(overflow_next, ptype, data_flag=data_flag,
|
||||
has_index_shift=has_index_shift)
|
||||
overflow_next += 1
|
||||
if not nb.try_add(row):
|
||||
raise ValueError(f"row of size {len(row)} too large for an empty page")
|
||||
builders.append(nb)
|
||||
|
||||
pages_by_index[strange_idx] = build_strange_page(
|
||||
strange_idx, ptype, next_page=data_idx, data_page=data_idx)
|
||||
for i, b in enumerate(builders):
|
||||
# Intermediate pages chain forward to the next data page and are
|
||||
# finalised now. The LAST page's next_page is patched after the loop
|
||||
# to a free-pool sentinel (FREEZE FIX) -- never at itself -- so it is
|
||||
# finalised there, not here.
|
||||
if i != len(builders) - 1:
|
||||
b.next_page = builders[i + 1].page_index
|
||||
pages_by_index[b.page_index] = b.finalize()
|
||||
last_builders.append(builders[-1])
|
||||
table_dir.append((ptype, strange_idx, builders[-1].page_index))
|
||||
|
||||
# FREEZE FIX (#13): no data page may self-loop its next_page. In every
|
||||
# genuine rekordbox export (data/MAPLE, lib_multi) the last page of each
|
||||
# chain points FORWARD to a free/unallocated page; the firmware Page Manager
|
||||
# walks next_page and treats an index at/after next_unused (incl. past EOF)
|
||||
# as "end of chain", but an index == the current page is an infinite loop ->
|
||||
# the deck froze. maple used to set last.next_page = its own index. Point
|
||||
# every last page at a single dedicated free-pool sentinel just past the
|
||||
# overflow region, materialise that page (zero-filled, as real exports do),
|
||||
# and set next_unused beyond it.
|
||||
free_sentinel = overflow_next
|
||||
for b in last_builders:
|
||||
b.next_page = free_sentinel
|
||||
for b in last_builders:
|
||||
pages_by_index[b.page_index] = b.finalize()
|
||||
overflow_used = overflow_next - (1 + 2 * num_tables)
|
||||
|
||||
# Materialise a small free pool of zero-filled pages and set next_unused PAST
|
||||
# it, mirroring a real export (e.g. data/MAPLE: 71 pages, next_unused=74 ->
|
||||
# ~3 free pages past the last live page). A genuine export always keeps
|
||||
# allocated-but-empty pages after the data and points next_unused beyond the
|
||||
# file; maple previously had next_unused == EOF with no free pool, which can
|
||||
# starve the firmware's free-list/allocator walk of a valid sentinel page.
|
||||
FREE_POOL_PAGES = 4
|
||||
for j in range(FREE_POOL_PAGES):
|
||||
pages_by_index[free_sentinel + j] = b"\x00" * PAGE_SIZE
|
||||
# File ends after the materialised free pool; next_unused points a couple
|
||||
# pages PAST EOF, exactly as real exports do (data/MAPLE: file=71 pages,
|
||||
# next_unused=74). The firmware tolerates a past-EOF next_unused.
|
||||
file_page_count = free_sentinel + FREE_POOL_PAGES # number of pages incl. page 0
|
||||
next_unused_page = file_page_count + 2
|
||||
|
||||
sequence = _next_sequence(Path(path))
|
||||
logger.info(
|
||||
"pdb layout: {} pages ({} reserved, {} overflow, {} free), sequence={}",
|
||||
file_page_count, 2 * num_tables, overflow_used, FREE_POOL_PAGES, sequence,
|
||||
)
|
||||
|
||||
file_header = b""
|
||||
file_header += pack("<I", 0) # magic
|
||||
file_header += pack("<I", PAGE_SIZE)
|
||||
file_header += pack("<I", num_tables)
|
||||
file_header += pack("<I", next_unused_page)
|
||||
file_header += pack("<I", 0) # unknown
|
||||
file_header += pack("<I", sequence) # incrementing transaction counter
|
||||
file_header += pack("<I", 0) # gap
|
||||
assert len(file_header) == FILE_HEADER_SIZE
|
||||
|
||||
tables_block = b""
|
||||
for ptype, first, last in table_dir:
|
||||
tables_block += pack("<I", ptype)
|
||||
tables_block += pack("<I", 0) # empty_candidate
|
||||
tables_block += pack("<I", first)
|
||||
tables_block += pack("<I", last)
|
||||
assert len(tables_block) == num_tables * TABLE_ENTRY_SIZE
|
||||
|
||||
header_full = file_header + tables_block
|
||||
pad_to_page = PAGE_SIZE - len(header_full)
|
||||
if pad_to_page < 0:
|
||||
raise ValueError("too many tables for one page")
|
||||
page0 = header_full + b"\x00" * pad_to_page
|
||||
|
||||
blank = b"\x00" * PAGE_SIZE
|
||||
out_path = Path(path)
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with out_path.open("wb") as f:
|
||||
f.write(page0)
|
||||
# Write pages 1..file_page_count-1 (reserved-but-unused written blank).
|
||||
# next_unused_page intentionally points a couple pages PAST this EOF.
|
||||
for idx in range(1, file_page_count):
|
||||
page = pages_by_index.get(idx, blank)
|
||||
assert len(page) == PAGE_SIZE
|
||||
f.write(page)
|
||||
logger.info("wrote {} ({} bytes)", out_path, out_path.stat().st_size)
|
||||
Whitespace-only changes.
@@ -0,0 +1,34 @@
|
||||
"""Discover audio files and m3u8 playlists in an input directory."""
|
||||
from pathlib import Path
|
||||
|
||||
AUDIO_EXTS = {".mp3", ".flac", ".wav", ".aiff", ".aif", ".m4a"}
|
||||
|
||||
|
||||
def scan_audio(input_dir: Path) -> list[Path]:
|
||||
out = []
|
||||
for p in sorted(input_dir.rglob("*")):
|
||||
if p.is_file() and p.suffix.lower() in AUDIO_EXTS:
|
||||
out.append(p)
|
||||
return out
|
||||
|
||||
|
||||
def parse_m3u8(playlist_path: Path) -> list[Path]:
|
||||
base = playlist_path.parent
|
||||
entries = []
|
||||
with playlist_path.open("r", encoding="utf-8", errors="replace") as f:
|
||||
for line in f:
|
||||
s = line.strip()
|
||||
if not s or s.startswith("#"):
|
||||
continue
|
||||
p = Path(s)
|
||||
if not p.is_absolute():
|
||||
p = (base / p).resolve()
|
||||
entries.append(p)
|
||||
return entries
|
||||
|
||||
|
||||
def scan_playlists(input_dir: Path) -> dict[str, list[Path]]:
|
||||
out: dict[str, list[Path]] = {}
|
||||
for p in sorted(input_dir.rglob("*.m3u8")):
|
||||
out[p.stem] = parse_m3u8(p)
|
||||
return out
|
||||
@@ -0,0 +1,249 @@
|
||||
"""Dump tracks from one or more Spotify playlists (Client Credentials flow).
|
||||
|
||||
just spotify # uses the placeholder ids below
|
||||
just spotify "<url-or-id> ..." # explicit playlists
|
||||
just spotify "--file lists.txt --json"
|
||||
just spotify "<url-or-id> --csv" # single sockseek CSV (stdout)
|
||||
just spotify "<url-or-id> --csv --out tracks.csv"
|
||||
|
||||
Playlists can be ids, full URLs, or spotify: URIs. Works for any public
|
||||
playlist; no user login needed. Requires SPOTIFY_CLIENT_ID /
|
||||
SPOTIFY_CLIENT_SECRET in the environment (loaded from .env by just).
|
||||
|
||||
--csv writes one combined CSV with Artist,Title,Album,Length columns (Length
|
||||
in seconds), the format consumed by sockseek/sldl. Each playlist is preceded
|
||||
by a `# <name>` comment line as a separator. Writes to stdout, or to a file
|
||||
with --out <path>.
|
||||
|
||||
Networking is async via aiohttp: playlists are fetched concurrently.
|
||||
"""
|
||||
import asyncio
|
||||
import base64
|
||||
import csv
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.parse
|
||||
|
||||
import aiohttp
|
||||
|
||||
|
||||
async def get_token(session: aiohttp.ClientSession, client_id: str, client_secret: str) -> str:
|
||||
auth = base64.b64encode(f"{client_id}:{client_secret}".encode()).decode()
|
||||
async with session.post(
|
||||
"https://accounts.spotify.com/api/token",
|
||||
data={"grant_type": "client_credentials"},
|
||||
headers={"Authorization": f"Basic {auth}"},
|
||||
) as resp:
|
||||
resp.raise_for_status()
|
||||
body = await resp.json()
|
||||
return body["access_token"]
|
||||
|
||||
|
||||
async def api_get(session: aiohttp.ClientSession, url: str, token: str) -> dict:
|
||||
async with session.get(url, headers={"Authorization": f"Bearer {token}"}) as resp:
|
||||
resp.raise_for_status()
|
||||
return await resp.json()
|
||||
|
||||
|
||||
def parse_playlist_id(arg: str) -> str:
|
||||
m = re.search(r"playlist[/:]([A-Za-z0-9]+)", arg)
|
||||
return m.group(1) if m else arg.strip()
|
||||
|
||||
|
||||
def read_list_file(path: str) -> list[str]:
|
||||
out = []
|
||||
with open(path) as fh:
|
||||
for line in fh:
|
||||
line = line.split("#", 1)[0].strip()
|
||||
if line:
|
||||
out.append(line)
|
||||
return out
|
||||
|
||||
|
||||
def _largest_image(images: list[dict] | None) -> str | None:
|
||||
if not images:
|
||||
return None
|
||||
# Spotify returns images sorted largest first, but defend against absent width.
|
||||
ranked = sorted(images, key=lambda im: (im.get("width") or 0), reverse=True)
|
||||
return ranked[0].get("url")
|
||||
|
||||
|
||||
async def fetch_playlist(session: aiohttp.ClientSession, pid: str, token: str) -> dict:
|
||||
meta = await api_get(
|
||||
session,
|
||||
f"https://api.spotify.com/v1/playlists/{pid}"
|
||||
f"?fields={urllib.parse.quote('name,owner(display_name)')}",
|
||||
token,
|
||||
)
|
||||
fields = (
|
||||
"items(track(id,name,artists(id,name),album(name,images,release_date,release_date_precision),"
|
||||
"duration_ms,external_ids(isrc),track_number,disc_number)),next"
|
||||
)
|
||||
url = (
|
||||
f"https://api.spotify.com/v1/playlists/{pid}/tracks"
|
||||
f"?limit=100&fields={urllib.parse.quote(fields)}"
|
||||
)
|
||||
tracks = []
|
||||
while url:
|
||||
page = await api_get(session, url, token)
|
||||
for item in page.get("items", []):
|
||||
t = item.get("track")
|
||||
if not t:
|
||||
continue
|
||||
tracks.append(
|
||||
{
|
||||
"id": t.get("id"),
|
||||
"title": t["name"],
|
||||
"artists": [a["name"] for a in t["artists"]],
|
||||
"artist_ids": [a.get("id") for a in t["artists"] if a.get("id")],
|
||||
"album": t["album"]["name"],
|
||||
"album_image_url": _largest_image(t["album"].get("images")),
|
||||
"release_date": t["album"].get("release_date"),
|
||||
"release_date_precision": t["album"].get("release_date_precision"),
|
||||
"duration_ms": t["duration_ms"],
|
||||
"isrc": (t.get("external_ids") or {}).get("isrc"),
|
||||
"track_number": t.get("track_number"),
|
||||
"disc_number": t.get("disc_number"),
|
||||
"genres": [],
|
||||
}
|
||||
)
|
||||
url = page.get("next")
|
||||
return {"id": pid, "name": meta.get("name"), "tracks": tracks}
|
||||
|
||||
|
||||
async def fetch_artist_genres(
|
||||
session: aiohttp.ClientSession,
|
||||
artist_ids: list[str],
|
||||
token: str,
|
||||
) -> dict[str, list[str]]:
|
||||
"""Batch `/v1/artists?ids=...` (max 50 ids per call). Returns id → genres."""
|
||||
out: dict[str, list[str]] = {}
|
||||
seen: set[str] = set()
|
||||
deduped = [a for a in artist_ids if a and not (a in seen or seen.add(a))]
|
||||
for i in range(0, len(deduped), 50):
|
||||
batch = deduped[i:i + 50]
|
||||
body = await api_get(
|
||||
session,
|
||||
f"https://api.spotify.com/v1/artists?ids={','.join(batch)}",
|
||||
token,
|
||||
)
|
||||
for a in body.get("artists", []) or []:
|
||||
if a and a.get("id"):
|
||||
out[a["id"]] = a.get("genres") or []
|
||||
return out
|
||||
|
||||
|
||||
def _enrich_track_genres(playlists: list[dict], genres_by_artist: dict[str, list[str]]) -> None:
|
||||
"""Populate each track's `genres` field with the union (dedup'd, preserving order)
|
||||
of its artists' genres."""
|
||||
for pl in playlists:
|
||||
for t in pl["tracks"]:
|
||||
seen: set[str] = set()
|
||||
union: list[str] = []
|
||||
for aid in t.get("artist_ids") or []:
|
||||
for g in genres_by_artist.get(aid, []):
|
||||
if g not in seen:
|
||||
seen.add(g)
|
||||
union.append(g)
|
||||
t["genres"] = union
|
||||
|
||||
|
||||
def write_csv(playlists, fh):
|
||||
"""Single combined sockseek/sldl CSV, one `# <name>` comment per playlist."""
|
||||
w = csv.writer(fh)
|
||||
w.writerow(["Artist", "Title", "Album", "Length"])
|
||||
for pl in playlists:
|
||||
fh.write(f"# {pl['name']}\n")
|
||||
for t in pl["tracks"]:
|
||||
w.writerow(
|
||||
[
|
||||
", ".join(t["artists"]),
|
||||
t["title"],
|
||||
t["album"],
|
||||
round(t["duration_ms"] / 1000),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
async def fetch_playlists(ids: list[str], with_genres: bool = True) -> list[dict]:
|
||||
"""Resolve env, open a session, fetch every playlist concurrently.
|
||||
|
||||
When `with_genres` is true (default), follow up with a batched
|
||||
`/v1/artists` call to populate each track's `genres` field.
|
||||
"""
|
||||
client_id = os.environ.get("MAPLE_SPOTIFY_CLIENT_ID") or os.environ.get("SPOTIFY_CLIENT_ID")
|
||||
client_secret = os.environ.get("MAPLE_SPOTIFY_CLIENT_SECRET") or os.environ.get("SPOTIFY_CLIENT_SECRET")
|
||||
if not (client_id and client_secret):
|
||||
sys.exit("set MAPLE_SPOTIFY_CLIENT_ID and MAPLE_SPOTIFY_CLIENT_SECRET")
|
||||
|
||||
timeout = aiohttp.ClientTimeout(total=30)
|
||||
async with aiohttp.ClientSession(timeout=timeout) as session:
|
||||
token = await get_token(session, client_id, client_secret)
|
||||
playlists = await asyncio.gather(
|
||||
*(fetch_playlist(session, parse_playlist_id(x), token) for x in ids)
|
||||
)
|
||||
if with_genres:
|
||||
artist_ids: list[str] = []
|
||||
for pl in playlists:
|
||||
for t in pl["tracks"]:
|
||||
artist_ids.extend(t.get("artist_ids") or [])
|
||||
genres = await fetch_artist_genres(session, artist_ids, token)
|
||||
_enrich_track_genres(playlists, genres)
|
||||
return playlists
|
||||
|
||||
|
||||
async def main_async() -> None:
|
||||
argv = sys.argv[1:]
|
||||
as_json = "--json" in argv
|
||||
as_csv = "--csv" in argv
|
||||
argv = [a for a in argv if a not in ("--json", "--csv")]
|
||||
|
||||
inputs: list[str] = []
|
||||
out_path: str | None = None
|
||||
i = 0
|
||||
while i < len(argv):
|
||||
if argv[i] == "--file":
|
||||
inputs += read_list_file(argv[i + 1])
|
||||
i += 2
|
||||
elif argv[i] == "--out":
|
||||
out_path = argv[i + 1]
|
||||
i += 2
|
||||
else:
|
||||
inputs.append(argv[i])
|
||||
i += 1
|
||||
|
||||
if not inputs:
|
||||
sys.exit(
|
||||
"usage: spotify.py [--json|--csv] <url-or-id ...> "
|
||||
"[--file list.txt] [--out path]"
|
||||
)
|
||||
|
||||
playlists = await fetch_playlists(inputs)
|
||||
|
||||
if as_json:
|
||||
print(json.dumps(playlists, indent=2, ensure_ascii=False))
|
||||
return
|
||||
|
||||
if as_csv:
|
||||
if out_path:
|
||||
with open(out_path, "w", newline="") as fh:
|
||||
write_csv(playlists, fh)
|
||||
print(f"wrote {sum(len(p['tracks']) for p in playlists)} tracks to {out_path}")
|
||||
else:
|
||||
write_csv(playlists, sys.stdout)
|
||||
return
|
||||
|
||||
for pl in playlists:
|
||||
print(f"\n=== {pl['name']} ({len(pl['tracks'])} tracks) ===")
|
||||
for n, t in enumerate(pl["tracks"], 1):
|
||||
print(f"{n:>3}. {', '.join(t['artists'])} - {t['title']}")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
asyncio.run(main_async())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,345 @@
|
||||
"""Canonical `Track` — the unit shared by `just collection` and `just export`.
|
||||
|
||||
A *collection* is a sequence of `Track`s persisted under
|
||||
`data/collection/`. An *export* reads a sequence of `Track`s and writes a
|
||||
rekordbox USB at `data/usb/`. This dataclass is the bridge: any code that
|
||||
talks about "a track" in maple should hold one of these.
|
||||
|
||||
`Track.validate()` is the project's single source of truth for what makes
|
||||
a track "exportable". Every check that used to be scattered across
|
||||
`maple/validate.py` lives here now; both pipelines call `track.validate()`.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from maple import model
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from maple.analyze import Analysis
|
||||
|
||||
|
||||
@dataclass
|
||||
class TrackIssue:
|
||||
"""A single failed validation criterion, bound to the `Track` it
|
||||
came from. Repr-friendly via `__str__`."""
|
||||
|
||||
track: "Track"
|
||||
field: str
|
||||
reason: str
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f"{self.track.display_name}: {self.field}: {self.reason}"
|
||||
|
||||
|
||||
@dataclass
|
||||
class Track:
|
||||
"""Canonical track metadata. Every field has a concrete default so the
|
||||
type is always populated — "missing" data shows up as `""`/`0`/`False`
|
||||
and is caught by `validate()`, not by Optional-handling at every call
|
||||
site."""
|
||||
|
||||
# ---- identity --------------------------------------------------------- #
|
||||
isrc: str = ""
|
||||
title: str = ""
|
||||
artists: list[str] = field(default_factory=list)
|
||||
album: str = ""
|
||||
|
||||
# ---- Spotify-derived ------------------------------------------------- #
|
||||
spotify_track_id: str = ""
|
||||
spotify_artist_ids: list[str] = field(default_factory=list)
|
||||
album_image_url: str = ""
|
||||
release_date: str = "" # "YYYY", "YYYY-MM", or "YYYY-MM-DD"
|
||||
release_date_precision: str = ""
|
||||
genres: list[str] = field(default_factory=list)
|
||||
track_number: int = 0
|
||||
disc_number: int = 0
|
||||
duration_ms: int = 0
|
||||
|
||||
# ---- on-disk audio --------------------------------------------------- #
|
||||
# `path` is the only legitimately-Optional field — a Track may exist
|
||||
# purely as a Spotify entry before the audio has been downloaded.
|
||||
path: Path | None = None
|
||||
file_size: int = 0
|
||||
bitrate: int = 0 # kbps
|
||||
sample_rate: int = 0 # Hz
|
||||
sample_depth: int = 0 # bits per sample
|
||||
has_artwork: bool = False # embedded cover-art presence
|
||||
has_cached_analysis: bool = False # collection/_analysis/<isrc>.json exists
|
||||
|
||||
# ---- analytical ------------------------------------------------------ #
|
||||
bpm: float = 0.0 # 0.0 = unknown
|
||||
|
||||
# ---- provenance ------------------------------------------------------ #
|
||||
source: str = "" # "spotify", "local", "bandcamp", …
|
||||
|
||||
# ---- views ----------------------------------------------------------- #
|
||||
|
||||
@property
|
||||
def primary_artist(self) -> str:
|
||||
return self.artists[0] if self.artists else ""
|
||||
|
||||
@property
|
||||
def display_name(self) -> str:
|
||||
if not self.title:
|
||||
return self.path.name if self.path else "<unknown>"
|
||||
if self.artists:
|
||||
return f"{', '.join(self.artists)} - {self.title}"
|
||||
return self.title
|
||||
|
||||
@property
|
||||
def primary_genre(self) -> str:
|
||||
return self.genres[0] if self.genres else ""
|
||||
|
||||
@property
|
||||
def year(self) -> int:
|
||||
"""First 4 digits of release_date as an int (0 if absent/invalid)."""
|
||||
if not self.release_date or len(self.release_date) < 4:
|
||||
return 0
|
||||
try:
|
||||
return int(self.release_date[:4])
|
||||
except ValueError:
|
||||
return 0
|
||||
|
||||
@property
|
||||
def duration_s(self) -> int:
|
||||
return int(round(self.duration_ms / 1000)) if self.duration_ms else 0
|
||||
|
||||
# ---- ingress --------------------------------------------------------- #
|
||||
|
||||
@classmethod
|
||||
def from_spotify(cls, d: dict) -> "Track":
|
||||
"""Build from one row of `maple.spotify.fetch_playlist`'s output."""
|
||||
return cls(
|
||||
isrc=(d.get("isrc") or "").strip(),
|
||||
title=d.get("title") or "",
|
||||
artists=list(d.get("artists") or []),
|
||||
album=d.get("album") or "",
|
||||
spotify_track_id=d.get("id") or "",
|
||||
spotify_artist_ids=list(d.get("artist_ids") or []),
|
||||
album_image_url=d.get("album_image_url") or "",
|
||||
release_date=d.get("release_date") or "",
|
||||
release_date_precision=d.get("release_date_precision") or "",
|
||||
genres=list(d.get("genres") or []),
|
||||
track_number=int(d.get("track_number") or 0),
|
||||
disc_number=int(d.get("disc_number") or 0),
|
||||
duration_ms=int(d.get("duration_ms") or 0),
|
||||
source="spotify",
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_file(cls, path: Path, *, analysis_cache_dir: Path | None = None) -> "Track":
|
||||
"""Construct from on-disk audio via mutagen.
|
||||
|
||||
When `analysis_cache_dir` is provided, also sets
|
||||
`has_cached_analysis` from `<cache_dir>/<isrc>.json` so
|
||||
`.validate()` can check it without a separate call site.
|
||||
"""
|
||||
from mutagen import File as MutagenFile
|
||||
|
||||
out = cls(path=path, source="local")
|
||||
try:
|
||||
audio = MutagenFile(path, easy=True)
|
||||
except Exception:
|
||||
return out
|
||||
if audio is None:
|
||||
return out
|
||||
tags = audio.tags or {}
|
||||
def first(k: str) -> str:
|
||||
v = tags.get(k) if hasattr(tags, "get") else None
|
||||
return (v[0] if v else "") if isinstance(v, list) else (v or "")
|
||||
out.title = first("title")
|
||||
artist = first("artist")
|
||||
if artist:
|
||||
out.artists = [a.strip() for a in artist.split(",") if a.strip()]
|
||||
out.album = first("album")
|
||||
out.isrc = first("isrc").strip()
|
||||
out.release_date = first("date")
|
||||
gtag = first("genre")
|
||||
if gtag:
|
||||
from maple.cli import split_genre_tags
|
||||
out.genres = split_genre_tags(gtag)
|
||||
try:
|
||||
out.bpm = float(first("bpm")) if first("bpm") else 0.0
|
||||
except ValueError:
|
||||
out.bpm = 0.0
|
||||
try:
|
||||
out.track_number = int(first("tracknumber").split("/")[0])
|
||||
except ValueError:
|
||||
out.track_number = 0
|
||||
info = getattr(audio, "info", None)
|
||||
if info is not None:
|
||||
out.duration_ms = int(round((getattr(info, "length", 0.0) or 0.0) * 1000))
|
||||
bps = int(getattr(info, "bitrate", 0) or 0)
|
||||
out.bitrate = bps // 1000 if bps > 1000 else bps
|
||||
out.sample_rate = int(getattr(info, "sample_rate", 0) or 0)
|
||||
out.sample_depth = int(getattr(info, "bits_per_sample", 0) or 0)
|
||||
try:
|
||||
out.file_size = path.stat().st_size
|
||||
except OSError:
|
||||
pass
|
||||
# Presence flags so validate() can decide without extra plumbing.
|
||||
from maple.collection.validate import has_embedded_art
|
||||
out.has_artwork = has_embedded_art(path)
|
||||
if analysis_cache_dir is not None and out.isrc:
|
||||
from maple.analyze import cache_path_for
|
||||
cp = cache_path_for(analysis_cache_dir, out.isrc)
|
||||
out.has_cached_analysis = cp is not None and cp.exists()
|
||||
return out
|
||||
|
||||
# ---- enrichment ------------------------------------------------------ #
|
||||
|
||||
def merge_spotify(self, d: dict) -> "Track":
|
||||
"""Overlay Spotify-canonical fields onto this Track in place. Empty
|
||||
Spotify fields are NOT applied (no clobbering)."""
|
||||
if d.get("title"):
|
||||
self.title = d["title"]
|
||||
if d.get("artists"):
|
||||
self.artists = list(d["artists"])
|
||||
if d.get("album"):
|
||||
self.album = d["album"]
|
||||
if d.get("isrc"):
|
||||
self.isrc = d["isrc"].strip()
|
||||
if d.get("id"):
|
||||
self.spotify_track_id = d["id"]
|
||||
if d.get("artist_ids"):
|
||||
self.spotify_artist_ids = list(d["artist_ids"])
|
||||
if d.get("album_image_url"):
|
||||
self.album_image_url = d["album_image_url"]
|
||||
if d.get("release_date"):
|
||||
self.release_date = d["release_date"]
|
||||
if d.get("release_date_precision"):
|
||||
self.release_date_precision = d["release_date_precision"]
|
||||
if d.get("genres"):
|
||||
self.genres = list(d["genres"])
|
||||
if d.get("duration_ms"):
|
||||
self.duration_ms = int(d["duration_ms"])
|
||||
if d.get("track_number"):
|
||||
self.track_number = int(d["track_number"])
|
||||
if d.get("disc_number"):
|
||||
self.disc_number = int(d["disc_number"])
|
||||
if not self.source:
|
||||
self.source = "spotify"
|
||||
return self
|
||||
|
||||
def merge_analysis(self, analysis: "Analysis") -> "Track":
|
||||
"""Merge fields from a `maple.analyze.Analysis`."""
|
||||
if analysis.bpm:
|
||||
self.bpm = float(analysis.bpm)
|
||||
return self
|
||||
|
||||
# ---- validation ------------------------------------------------------ #
|
||||
|
||||
def validate(self) -> list[TrackIssue]:
|
||||
"""Return the list of failed criteria. Empty list = ready to export.
|
||||
|
||||
The single place every "is this track exportable?" check lives.
|
||||
Both `just collection` and `just export` call this — there is no
|
||||
separate validation entrypoint.
|
||||
"""
|
||||
issues: list[TrackIssue] = []
|
||||
def fail(field_name: str, reason: str) -> None:
|
||||
issues.append(TrackIssue(self, field_name, reason))
|
||||
|
||||
# Required text tags — the spotify-canonical set.
|
||||
if not self.title:
|
||||
fail("title", "tag missing")
|
||||
if not self.artists:
|
||||
fail("artist", "tag missing")
|
||||
if not self.album:
|
||||
fail("album", "tag missing")
|
||||
if not self.release_date:
|
||||
fail("date", "tag missing")
|
||||
if not self.genres:
|
||||
fail("genre", "tag missing")
|
||||
if not self.isrc:
|
||||
fail("isrc", "tag missing")
|
||||
|
||||
# Analytical.
|
||||
if self.bpm <= 0:
|
||||
fail("bpm", "tag missing")
|
||||
|
||||
# On-disk.
|
||||
if not self.has_artwork:
|
||||
fail("artwork", "no embedded cover art")
|
||||
if not self.has_cached_analysis:
|
||||
fail("analysis", "no cached analysis")
|
||||
|
||||
# Sanity — only check when we have something to check.
|
||||
if self.path is not None and self.duration_ms <= 0:
|
||||
fail("duration", "≤ 0s — corrupt file or unreadable container")
|
||||
return issues
|
||||
|
||||
def compare(self, canonical: "Track", *, fuzzy_threshold: float = 0.5) -> list[TrackIssue]:
|
||||
"""Diff this Track against a canonical one (typically a Spotify
|
||||
`from_spotify` Track). Returns mismatch issues — empty when the
|
||||
two agree on every populated field."""
|
||||
from maple.collection.index import first_artist, fuzzy_ratio, norm, norm_loose
|
||||
out: list[TrackIssue] = []
|
||||
def diff(field_name: str, sv: object, fv: object) -> None:
|
||||
out.append(TrackIssue(self, field_name, f"spotify={sv!r} file={fv!r}"))
|
||||
|
||||
sp_artist = ", ".join(canonical.artists)
|
||||
f_artist = ", ".join(self.artists)
|
||||
if sp_artist and fuzzy_ratio(norm(first_artist(sp_artist)), norm(first_artist(f_artist))) < fuzzy_threshold:
|
||||
diff("artist", sp_artist, f_artist)
|
||||
if canonical.title and fuzzy_ratio(norm_loose(canonical.title), norm_loose(self.title)) < fuzzy_threshold:
|
||||
diff("title", canonical.title, self.title)
|
||||
if canonical.album and self.album and fuzzy_ratio(norm_loose(canonical.album), norm_loose(self.album)) < fuzzy_threshold:
|
||||
diff("album", canonical.album, self.album)
|
||||
elif canonical.album and not self.album:
|
||||
diff("album", canonical.album, self.album)
|
||||
if canonical.isrc and self.isrc and canonical.isrc.upper() != self.isrc.upper():
|
||||
diff("isrc", canonical.isrc, self.isrc)
|
||||
if canonical.release_date and self.release_date and canonical.release_date[:4] != self.release_date[:4]:
|
||||
diff("date", canonical.release_date, self.release_date)
|
||||
if canonical.duration_ms and self.duration_ms:
|
||||
delta = abs(canonical.duration_ms - self.duration_ms) / 1000.0
|
||||
if delta > 5.0:
|
||||
diff("duration", f"{canonical.duration_s}s", f"{self.duration_s}s")
|
||||
return out
|
||||
|
||||
# ---- export adapter -------------------------------------------------- #
|
||||
|
||||
def to_pdb_row(
|
||||
self,
|
||||
*,
|
||||
track_id: int,
|
||||
artist_id: int,
|
||||
album_id: int,
|
||||
genre_id: int = 0,
|
||||
artwork_id: int = 0,
|
||||
file_path: str,
|
||||
analyze_path: str = "",
|
||||
date_added: str = "",
|
||||
tempo_x100: int | None = None,
|
||||
) -> model.Track:
|
||||
"""Project this canonical Track into the denormalised pdb row shape."""
|
||||
if tempo_x100 is None:
|
||||
tempo_x100 = round(self.bpm * 100) if self.bpm else 0
|
||||
return model.Track(
|
||||
id=track_id,
|
||||
title=self.title,
|
||||
artist_id=artist_id,
|
||||
album_id=album_id,
|
||||
filename=Path(file_path).name,
|
||||
file_path=file_path,
|
||||
file_size=self.file_size,
|
||||
sample_rate=self.sample_rate or 44100,
|
||||
sample_depth=self.sample_depth or 16,
|
||||
bitrate=self.bitrate,
|
||||
duration=self.duration_s,
|
||||
track_number=self.track_number,
|
||||
year=self.year,
|
||||
disc_number=self.disc_number,
|
||||
tempo=tempo_x100,
|
||||
release_date=self.release_date,
|
||||
date_added=date_added,
|
||||
analyze_path=analyze_path,
|
||||
analyze_date=date_added,
|
||||
artwork_id=artwork_id,
|
||||
genre_id=genre_id,
|
||||
source_path=self.path,
|
||||
)
|
||||
@@ -0,0 +1,47 @@
|
||||
"""Thin compatibility layer over `Track.validate()`.
|
||||
|
||||
Validation logic lives on `maple.track.Track` itself — this module just
|
||||
re-exports `TrackIssue` and offers helpers so callers don't all need to
|
||||
construct `Track` objects inline.
|
||||
|
||||
The `validate_track(path, ...)` wrapper is kept for the few sites that
|
||||
hand around `Path`s rather than `Track`s; new code should prefer
|
||||
`Track.from_file(path).validate()`.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from loguru import logger
|
||||
|
||||
from maple.track import Track, TrackIssue
|
||||
|
||||
__all__ = ["Track", "TrackIssue", "validate_track", "log_issues"]
|
||||
|
||||
|
||||
def validate_track(
|
||||
path: Path,
|
||||
*,
|
||||
spotify_track: dict | None = None,
|
||||
analysis_cache_dir: Path | None = None,
|
||||
playlist: str = "",
|
||||
) -> list[TrackIssue]:
|
||||
"""Construct a `Track` from `path` and run its validator.
|
||||
|
||||
`playlist` is unused — kept in the signature for API compatibility
|
||||
with callers that still pass it.
|
||||
"""
|
||||
del playlist # not used; TrackIssue carries the track now
|
||||
track = Track.from_file(path, analysis_cache_dir=analysis_cache_dir)
|
||||
issues = track.validate()
|
||||
if spotify_track is not None:
|
||||
issues.extend(track.compare(Track.from_spotify(spotify_track)))
|
||||
return issues
|
||||
|
||||
|
||||
def log_issues(issues: list[TrackIssue], *, report: bool = True) -> None:
|
||||
"""Emit every issue via `loguru.warning`. With `report=True`, also
|
||||
binds `extra["report"]=True` so a report-file sink can capture them."""
|
||||
bound = logger.bind(report=True) if report else logger
|
||||
for i in issues:
|
||||
bound.warning("{}", i)
|
||||
@@ -0,0 +1,73 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.14"
|
||||
# dependencies = []
|
||||
# ///
|
||||
"""Thin binding around the `sockseek` CLI.
|
||||
|
||||
Creates the data directories and invokes sockseek with the CLI-argument
|
||||
equivalents of etc/sldl.tpl.conf, so the two stay in sync without rendering a
|
||||
config file. Reads SOCKSEEK_DATA / SOCKSEEK_USER / SOCKSEEK_PASS from the
|
||||
environment (loaded from .env by just).
|
||||
|
||||
just sockseek <input> # csv path, url, search string, ...
|
||||
just sockseek "tracks.csv -a" # extra flags pass through / override
|
||||
|
||||
Any args after the input are forwarded to sockseek verbatim, placed after the
|
||||
defaults so they win on conflict.
|
||||
"""
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Mirror of etc/sldl.tpl.conf. Keep these in sync with that file.
|
||||
NAME_FORMAT = "{artist(/)album(/)track( )title|(_missing_tags/)slsk-foldername(/)slsk-filename}"
|
||||
|
||||
|
||||
def env(key):
|
||||
val = os.environ.get(key)
|
||||
if not val:
|
||||
sys.exit(f"missing env var: {key}")
|
||||
return val
|
||||
|
||||
|
||||
def config_args(data, user, password):
|
||||
"""CLI equivalents of etc/sldl.tpl.conf (no-progress has no sockseek flag)."""
|
||||
return [
|
||||
"--path", data,
|
||||
"--index-path", f"{data}/_metadata/index.sldl.csv",
|
||||
"--user", user,
|
||||
"--pass", password,
|
||||
"--name-format", NAME_FORMAT,
|
||||
"--pref-format", "flac,mp3",
|
||||
"--pref-min-bitrate", "200",
|
||||
"--pref-max-bitrate", "2500",
|
||||
"--strict-conditions",
|
||||
"--fast-search",
|
||||
"--concurrent-jobs", "4",
|
||||
]
|
||||
|
||||
|
||||
def main():
|
||||
if not sys.argv[1:]:
|
||||
sys.exit("usage: sockseek.py <input> [extra sockseek args...]")
|
||||
inp, *extra = sys.argv[1:]
|
||||
|
||||
data = env("SOCKSEEK_DATA")
|
||||
user = env("SOCKSEEK_USER")
|
||||
password = env("SOCKSEEK_PASS")
|
||||
|
||||
Path(data, "_metadata").mkdir(parents=True, exist_ok=True)
|
||||
|
||||
args = config_args(data, user, password)
|
||||
cmd = ["sockseek", inp, *args, *extra]
|
||||
|
||||
masked = ["***" if a == password else a for a in cmd]
|
||||
print(" ".join(masked), file=sys.stderr)
|
||||
|
||||
raise SystemExit(subprocess.run(cmd).returncode)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,30 @@
|
||||
from maple.pdb.strings import EMPTY, encode_devicesql
|
||||
|
||||
|
||||
def test_empty():
|
||||
assert encode_devicesql("") == EMPTY
|
||||
|
||||
|
||||
def test_short_ascii():
|
||||
out = encode_devicesql("ON")
|
||||
# length=3 -> length_and_kind = (3<<1)|1 = 7
|
||||
assert out == bytes([7]) + b"ON"
|
||||
|
||||
|
||||
def test_longer_ascii():
|
||||
s = "Hello, World!"
|
||||
out = encode_devicesql(s)
|
||||
length = len(s) + 1
|
||||
assert out[0] == ((length << 1) | 1)
|
||||
assert out[1:] == s.encode("ascii")
|
||||
|
||||
|
||||
def test_utf16():
|
||||
s = "ЛаBа"
|
||||
out = encode_devicesql(s)
|
||||
assert out[0] == 0x90
|
||||
# Total length stored = payload_len + 4
|
||||
payload = s.encode("utf-16-le")
|
||||
assert out[1:3] == (len(payload) + 4).to_bytes(2, "little")
|
||||
assert out[3] == 0
|
||||
assert out[4:] == payload
|
||||
@@ -0,0 +1,29 @@
|
||||
from pathlib import Path
|
||||
|
||||
from maple.model import Album, Artist, Database, PlaylistEntry, PlaylistNode, Track
|
||||
from maple.pdb.writer import write_pdb
|
||||
|
||||
|
||||
def test_smoke(tmp_path: Path):
|
||||
db = Database()
|
||||
db.artists.append(Artist(id=1, name="AceMo"))
|
||||
db.albums.append(Album(id=1, name="All My Life", artist_id=1))
|
||||
db.tracks.append(Track(
|
||||
id=1, title="Where They At", artist_id=1, album_id=1,
|
||||
filename="ace.mp3", file_path="/Contents/AceMo/All My Life/ace.mp3",
|
||||
file_size=14547692, sample_rate=44100, sample_depth=16, bitrate=320,
|
||||
duration=436, track_number=7, year=2019, date_added="2025-01-04",
|
||||
))
|
||||
db.playlist_nodes.append(PlaylistNode(id=1, parent_id=0, sort_order=1, name="MyList"))
|
||||
db.playlist_entries.append(PlaylistEntry(playlist_id=1, track_id=1, entry_index=1))
|
||||
|
||||
out = tmp_path / "export.pdb"
|
||||
write_pdb(db, out)
|
||||
assert out.exists()
|
||||
data = out.read_bytes()
|
||||
assert len(data) % 4096 == 0
|
||||
assert data[:4] == b"\x00\x00\x00\x00"
|
||||
# page_size at offset 4
|
||||
assert int.from_bytes(data[4:8], "little") == 4096
|
||||
# num_tables at offset 8 -- rekordbox always emits all 20 page-type tables
|
||||
assert int.from_bytes(data[8:12], "little") == 20
|
||||
@@ -0,0 +1,80 @@
|
||||
"""Build a maple export.pdb from the SAME tracks as the real rekordbox AceMo
|
||||
export, so the two can be diffed and the maple pdb tested on a player against
|
||||
the audio/USBANLZ already on the USB.
|
||||
|
||||
Reads metadata out of data/acemo/PIONEER/rekordbox/export.pdb (the real export)
|
||||
and reconstructs an identical-content maple Database, then writes
|
||||
captures/maple_acemo.pdb.
|
||||
"""
|
||||
import sys
|
||||
from io import BytesIO
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(ROOT / "src"))
|
||||
|
||||
from kaitaistruct import KaitaiStream
|
||||
from kaitai.rekordbox_pdb import RekordboxPdb
|
||||
from maple.model import Album, Artist, Artwork, Database, Key, Track
|
||||
from maple.pdb.writer import write_pdb
|
||||
|
||||
REAL = ROOT / "data/acemo/PIONEER/rekordbox/export.pdb"
|
||||
OUT = ROOT / "captures/maple_acemo.pdb"
|
||||
PT = {t.value: t.name for t in RekordboxPdb.PageType}
|
||||
|
||||
|
||||
def _rows(tbl):
|
||||
out, ref = [], tbl.first_page
|
||||
while True:
|
||||
pg = ref.body
|
||||
if pg.is_data_page:
|
||||
for rg in pg.row_groups:
|
||||
for r in rg.rows:
|
||||
if r.present:
|
||||
out.append(r.body)
|
||||
if ref.index == tbl.last_page.index:
|
||||
break
|
||||
ref = pg.next_page
|
||||
return out
|
||||
|
||||
|
||||
def _txt(v):
|
||||
try:
|
||||
return v.body.text
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
def main():
|
||||
pdb = RekordboxPdb(is_ext=False, _io=KaitaiStream(BytesIO(REAL.read_bytes())))
|
||||
by = {PT.get(t.type.value): _rows(t) for t in pdb.tables}
|
||||
|
||||
db = Database()
|
||||
for a in by.get("artists", []):
|
||||
db.artists.append(Artist(id=a.id, name=_txt(a.name)))
|
||||
for a in by.get("albums", []):
|
||||
db.albums.append(Album(id=a.id, name=_txt(a.name), artist_id=a.artist_id))
|
||||
for k in by.get("keys", []):
|
||||
db.keys.append(Key(id=k.id, name=_txt(k.name), id2=k.id2))
|
||||
for a in by.get("artwork", []):
|
||||
db.artwork.append(Artwork(id=a.id, path=_txt(a.path)))
|
||||
for t in by.get("tracks", []):
|
||||
db.tracks.append(Track(
|
||||
id=t.id, title=_txt(t.title), artist_id=t.artist_id, album_id=t.album_id,
|
||||
filename=_txt(t.filename), file_path=_txt(t.file_path), file_size=t.file_size,
|
||||
sample_rate=t.sample_rate, sample_depth=t.sample_depth, bitrate=t.bitrate,
|
||||
duration=t.duration, track_number=t.track_number, year=t.year,
|
||||
disc_number=t.disc_number, tempo=t.tempo, comment=_txt(t.comment),
|
||||
date_added=_txt(t.date_added), release_date=_txt(t.release_date),
|
||||
mix_name=_txt(t.mix_name), analyze_path=_txt(t.analyze_path),
|
||||
analyze_date=_txt(t.analyze_date), artwork_id=t.artwork_id, genre_id=t.genre_id))
|
||||
|
||||
write_pdb(db, OUT)
|
||||
sz = OUT.stat().st_size
|
||||
print(f"wrote {OUT} ({sz} bytes, page-mult={sz % 4096 == 0}) "
|
||||
f"tracks={len(db.tracks)} artists={len(db.artists)} albums={len(db.albums)} "
|
||||
f"keys={len(db.keys)} artwork={len(db.artwork)}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,104 @@
|
||||
// Frida host (Node ESM) for the pdb-capture agent.
|
||||
// - Attaches the agent (rbx_pdb_capture.js) to rekordbox processes.
|
||||
// - Streams ALL their output to THIS console (agent lines are [pid]-tagged).
|
||||
// - STAYS ALIVE via the Node event loop -- no frida-CLI REPL/EOF problem.
|
||||
// - Uses CHILD-GATING (Windows-supported; spawn-gating is not) so children of a
|
||||
// gated process are instrumented AT BIRTH. This is how we reach the
|
||||
// rekordboxAgent that writes export.pdb: it's CIG-protected and refuses
|
||||
// post-hoc injection, but as a freshly-spawned child it can be hooked.
|
||||
// - Clears the captures/ folder on start.
|
||||
//
|
||||
// Setup (Windows, repo root, once): npm i frida
|
||||
// Run:
|
||||
// node tools\capture.mjs --spawn (RECOMMENDED: launch rekordbox under the
|
||||
// gate so every child incl. the DB agent is
|
||||
// hooked at birth. Close rekordbox first.)
|
||||
// node tools\capture.mjs (attach to running procs + gate their
|
||||
// future children; restart rekordbox to
|
||||
// respawn the agent as a gated child.)
|
||||
|
||||
import * as fridaNs from "frida";
|
||||
import fs from "node:fs";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
const frida = fridaNs.default ?? fridaNs;
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
|
||||
const REKORDBOX = "C:\\Program Files\\rekordbox\\rekordbox 7.0.5\\rekordbox.exe";
|
||||
const AGENT = fs.readFileSync(path.join(__dirname, "rbx_pdb_capture.js"), "utf8");
|
||||
const CAP = path.join(__dirname, "..", "captures");
|
||||
const SPAWN = process.argv.includes("--spawn");
|
||||
|
||||
function clearCaptures() {
|
||||
fs.mkdirSync(CAP, { recursive: true });
|
||||
let n = 0;
|
||||
for (const f of fs.readdirSync(CAP)) {
|
||||
try { fs.rmSync(path.join(CAP, f), { force: true, recursive: true }); n++; } catch (e) {}
|
||||
}
|
||||
console.log(`[host] cleared ${n} file(s) from ${CAP}`);
|
||||
}
|
||||
|
||||
function isRbx(name) { return /rekordbox/i.test(name || ""); }
|
||||
|
||||
// Attach, load the agent, and gate this process's future children too.
|
||||
async function instrument(device, pid, label) {
|
||||
try {
|
||||
const session = await device.attach(pid);
|
||||
session.detached.connect(reason => console.log(`[host] detached ${label} (${pid}): ${reason}`));
|
||||
const script = await session.createScript(AGENT);
|
||||
script.logHandler = (level, text) => console.log(text);
|
||||
script.message.connect(msg => {
|
||||
if (msg.type === "error")
|
||||
console.error(`[host] ${label} (${pid}) script error:`, msg.stack || msg.description || msg);
|
||||
});
|
||||
await script.load();
|
||||
try { await session.enableChildGating(); } catch (e) { /* not always available */ }
|
||||
console.log(`[host] instrumented ${label} (${pid})`);
|
||||
return session;
|
||||
} catch (e) {
|
||||
console.error(`[host] FAILED to instrument ${label} (${pid}): ${e.message}`);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function main() {
|
||||
clearCaptures();
|
||||
const device = await frida.getLocalDevice();
|
||||
|
||||
// Children of any gated process are suspended at birth -> instrument, resume.
|
||||
device.childAdded.connect(async child => {
|
||||
const name = child.identifier || child.path || String(child.pid);
|
||||
try {
|
||||
console.log(`[host] child spawned: ${name} (${child.pid})`);
|
||||
if (isRbx(name)) await instrument(device, child.pid, "child:" + name);
|
||||
} catch (e) {
|
||||
console.error(`[host] child handler error (${child.pid}): ${e.message}`);
|
||||
} finally {
|
||||
try { await device.resume(child.pid); } catch (_) {}
|
||||
}
|
||||
});
|
||||
|
||||
if (SPAWN) {
|
||||
console.log(`[host] spawning ${REKORDBOX}`);
|
||||
const pid = await device.spawn([REKORDBOX]);
|
||||
await instrument(device, pid, "main"); // enables child gating on main
|
||||
await device.resume(pid);
|
||||
console.log(`[host] resumed main rekordbox (${pid}); children caught at birth`);
|
||||
} else {
|
||||
for (const p of await device.enumerateProcesses()) {
|
||||
if (isRbx(p.name)) await instrument(device, p.pid, p.name);
|
||||
}
|
||||
console.log("[host] attached to running rekordbox processes (+child gating).");
|
||||
console.log("[host] >>> the export.pdb writer (rekordboxAgent) refuses live injection.");
|
||||
console.log("[host] >>> RESTART rekordbox -- the new agent spawns as a GATED CHILD and");
|
||||
console.log("[host] >>> gets hooked at birth. Then Export and watch for [SAVED] ...export.pdb");
|
||||
console.log("[host] >>> (or re-run with --spawn to launch rekordbox under the gate.)");
|
||||
}
|
||||
|
||||
console.log("[host] ready. Ctrl-C to stop.");
|
||||
process.stdin.resume();
|
||||
process.on("SIGINT", () => { console.log("\n[host] stopping."); process.exit(0); });
|
||||
}
|
||||
|
||||
main().catch(e => { console.error("[host] fatal:", e); process.exit(1); });
|
||||
@@ -0,0 +1,184 @@
|
||||
#!/usr/bin/env -S uv run --script
|
||||
# /// script
|
||||
# requires-python = ">=3.14"
|
||||
# dependencies = ["kaitaistruct>=0.11", "pyrekordbox>=0.4.4"]
|
||||
# ///
|
||||
"""Decode a rekordbox USB export so we can spot how the analyze-path
|
||||
bucket/id is derived from track metadata.
|
||||
|
||||
Reads `export.pdb` from a rekordbox-built USB stick (or `data/q1-rekordbox`)
|
||||
plus optionally cross-references the rekordbox machine's `master.db` to
|
||||
correlate Mac-internal UUIDs with the USB-export short hex ids.
|
||||
|
||||
Usage:
|
||||
uv run tools/decode_rb_export.py <usb-root> [<master.db>]
|
||||
|
||||
`<usb-root>` is the directory holding `PIONEER/rekordbox/export.pdb`.
|
||||
With `<master.db>` we also list the UUID + ContentLink for each track so
|
||||
the per-track UUID→USB-id derivation can be eye-balled across rows.
|
||||
"""
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Make the in-project kaitai schemas importable without an editable install.
|
||||
HERE = Path(__file__).resolve().parent
|
||||
sys.path.insert(0, str(HERE.parent / "src"))
|
||||
|
||||
from kaitai.rekordbox_pdb import RekordboxPdb # noqa: E402
|
||||
from kaitaistruct import KaitaiStream # noqa: E402
|
||||
|
||||
ANLZ_PATH_RE = re.compile(r"/PIONEER/USBANLZ/([^/]+)/([^/]+)/ANLZ0000\.DAT")
|
||||
|
||||
|
||||
def iter_rows(pdb: RekordboxPdb, page_type):
|
||||
"""Walk a table's page list, yielding every present row body.
|
||||
Tolerates the page chain pointing past EOF (real exports do this on
|
||||
the last page — `next_page.index >= num_pages` is a sentinel)."""
|
||||
for tref in pdb.tables:
|
||||
if tref.type != page_type:
|
||||
continue
|
||||
pref = tref.first_page
|
||||
seen = set()
|
||||
while pref is not None and pref.index not in seen:
|
||||
seen.add(pref.index)
|
||||
try:
|
||||
page = pref.body
|
||||
except Exception:
|
||||
break # past-EOF next_page (free-pool sentinel)
|
||||
if page.is_data_page:
|
||||
for rg in page.row_groups:
|
||||
for rref in rg.rows:
|
||||
if rref.present:
|
||||
yield rref.body
|
||||
try:
|
||||
nxt = page.next_page
|
||||
except Exception:
|
||||
break
|
||||
if nxt is None or nxt.index == 0 or nxt.index == pref.index:
|
||||
break
|
||||
pref = nxt
|
||||
|
||||
|
||||
def read_pdb_tracks(pdb_path: Path) -> list[dict]:
|
||||
"""Return one dict per track row: id, title, file_path, analyze_path,
|
||||
plus every opaque/seed field plus parsed bucket+anlz_id."""
|
||||
with pdb_path.open("rb") as f:
|
||||
pdb = RekordboxPdb(False, KaitaiStream(f))
|
||||
tracks = []
|
||||
for r in iter_rows(pdb, RekordboxPdb.PageType.tracks):
|
||||
ap = r.analyze_path.body.text
|
||||
m = ANLZ_PATH_RE.match(ap)
|
||||
bucket = anlz_id = ""
|
||||
if m:
|
||||
bucket, anlz_id = m.group(1), m.group(2)
|
||||
tracks.append({
|
||||
"track_id": r.id,
|
||||
"title": r.title.body.text,
|
||||
"file_path": r.file_path.body.text,
|
||||
"analyze_path": ap,
|
||||
"bucket": bucket,
|
||||
"anlz_id": anlz_id,
|
||||
"opaque_track_id_lo": r.opaque_track_id_lo,
|
||||
"opaque_seed_lo": r.opaque_seed_lo,
|
||||
"opaque_seed_hi": r.opaque_seed_hi,
|
||||
"tempo": r.tempo,
|
||||
"duration": r.duration,
|
||||
"file_size": r.file_size,
|
||||
"bitmask": r.bitmask,
|
||||
})
|
||||
return tracks
|
||||
|
||||
|
||||
def maybe_attach_master_db(tracks: list[dict], master_db_path: Path) -> None:
|
||||
"""Look up each track in master.db by file path and attach UUID/etc."""
|
||||
try:
|
||||
from pyrekordbox import Rekordbox6Database
|
||||
from pyrekordbox.db6 import tables as t
|
||||
except ImportError as e:
|
||||
print(f"(pyrekordbox unavailable: {e}; skipping master.db lookup)", file=sys.stderr)
|
||||
return
|
||||
db = Rekordbox6Database(path=str(master_db_path))
|
||||
for tr in tracks:
|
||||
# Look up by file name as a fallback — full FolderPath may differ if
|
||||
# the rekordbox machine mounts the source from a different prefix.
|
||||
title = tr["title"]
|
||||
rows = db.session.query(t.DjmdContent).filter(t.DjmdContent.Title == title).limit(3).all()
|
||||
if not rows:
|
||||
tr["rb_uuid"] = ""
|
||||
tr["rb_content_link"] = 0
|
||||
tr["rb_file_id"] = ""
|
||||
tr["rb_master_song_id"] = ""
|
||||
continue
|
||||
# Prefer one whose FolderPath ends with the same file name
|
||||
fname = Path(tr["file_path"]).name
|
||||
match = next((r for r in rows if (r.FolderPath or "").endswith(fname)), rows[0])
|
||||
tr["rb_uuid"] = match.UUID or ""
|
||||
tr["rb_content_link"] = match.ContentLink or 0
|
||||
tr["rb_file_id"] = match.rb_file_id or ""
|
||||
tr["rb_master_song_id"] = match.MasterSongID or ""
|
||||
|
||||
|
||||
def fmt_int(v, w=10):
|
||||
if not isinstance(v, int):
|
||||
return f"{str(v):<{w}}"
|
||||
return f"{v:>{w-2}d}=0x{v:08x}"[-w:].rjust(w)
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
if len(argv) < 1:
|
||||
sys.exit(__doc__)
|
||||
usb_root = Path(argv[0]).expanduser()
|
||||
pdb_path = usb_root / "PIONEER" / "rekordbox" / "export.pdb"
|
||||
if not pdb_path.exists():
|
||||
sys.exit(f"export.pdb not found at {pdb_path}")
|
||||
|
||||
tracks = read_pdb_tracks(pdb_path)
|
||||
print(f"# Read {len(tracks)} tracks from {pdb_path}")
|
||||
|
||||
if len(argv) >= 2:
|
||||
master_db = Path(argv[1]).expanduser()
|
||||
if master_db.exists():
|
||||
maybe_attach_master_db(tracks, master_db)
|
||||
print(f"# Cross-referenced against {master_db}")
|
||||
else:
|
||||
print(f"# (master.db at {master_db} not found, skipping)")
|
||||
|
||||
# Output: one TSV row per track plus a summary at the end.
|
||||
cols = ["track_id", "bucket", "anlz_id", "opaque_track_id_lo",
|
||||
"opaque_seed_lo", "opaque_seed_hi",
|
||||
"rb_uuid", "rb_content_link", "rb_file_id", "rb_master_song_id",
|
||||
"title"]
|
||||
print()
|
||||
print("\t".join(cols))
|
||||
for tr in tracks:
|
||||
row = []
|
||||
for c in cols:
|
||||
v = tr.get(c, "")
|
||||
if isinstance(v, int):
|
||||
row.append(f"{v}=0x{v:x}" if v > 255 else str(v))
|
||||
else:
|
||||
row.append(str(v)[:64])
|
||||
print("\t".join(row))
|
||||
|
||||
# Quick correlation tests: print numeric distances + ratios so a
|
||||
# multi-track export shows obvious patterns at a glance.
|
||||
if len(tracks) >= 2:
|
||||
print("\n# Successive anlz_id deltas (probe for sequential counter):")
|
||||
ids = []
|
||||
for tr in tracks:
|
||||
try:
|
||||
ids.append(int(tr["anlz_id"], 16))
|
||||
except (ValueError, TypeError):
|
||||
pass
|
||||
ids.sort()
|
||||
for a, b in zip(ids, ids[1:]):
|
||||
print(f" {a:08x} → {b:08x} Δ = {b-a}")
|
||||
print(f"\n# anlz_id range: {min(ids):08x} .. {max(ids):08x}")
|
||||
print(f"# bucket set: {sorted({tr['bucket'] for tr in tracks if tr['bucket']})}")
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main(sys.argv[1:]))
|
||||
@@ -0,0 +1,209 @@
|
||||
// Capture rekordbox's .pdb writes with Frida (run on the WINDOWS host).
|
||||
//
|
||||
// Hooks at the ntdll layer (NtCreateFile / NtWriteFile / NtClose) so it catches
|
||||
// file IO no matter whether the app calls kernel32, kernelbase, or the CRT.
|
||||
// Logs EVERY file opened for writing (so you can see temp names / the real
|
||||
// filenames even if the .pdb filter is wrong), reconstructs files whose path
|
||||
// matches SAVE_RE byte-for-byte, and saves them to OUT_DIR via Frida's File API.
|
||||
// Prints a backtrace the first time each saved file is written -> the
|
||||
// module+offset of the serializer to open in Ghidra.
|
||||
//
|
||||
// Usage (Windows PowerShell, rekordbox running):
|
||||
// mkdir C:\Users\ntr\code\maple\maple-py\captures # OUT_DIR must exist
|
||||
// frida-ps | Select-String rekordbox # find all PIDs
|
||||
// frida -n rekordbox.exe -l tools\rbx_pdb_capture.js # or -p <pid>
|
||||
// then trigger "Export to USB". If you see [w-open] lines but the export file
|
||||
// isn't among them, widen SAVE_RE. If you see NOTHING, attach to another PID.
|
||||
//
|
||||
// Set VERBOSE = true to log every write-open path; false logs only SAVE_RE hits.
|
||||
|
||||
var OUT_DIR = "C:\\Users\\ntr\\code\\maple\\maple-py\\captures\\"; // must exist; Frida won't mkdir
|
||||
var SAVE_RE = /export.*\.pdb|\.pdb$|PIONEER/i; // which opened files to reconstruct + save
|
||||
var VERBOSE = true; // log ALL write-opens, not just SAVE_RE
|
||||
|
||||
var PID = Process.id;
|
||||
var TAG = "[" + PID + "] ";
|
||||
var DEBUG = true; // log every interesting open + all mapping activity
|
||||
var DEBUG_RE = /export|\.pdb|PIONEER|rekordbox|Contents|USBANLZ/i; // what DEBUG opens to print
|
||||
var ntdll = Process.getModuleByName("ntdll.dll");
|
||||
function nt(name) { return ntdll.getExportByName(name); }
|
||||
|
||||
// Resolve + confirm a hook target so we can see in the console what attached.
|
||||
function resolve(name) {
|
||||
var a = ntdll.findExportByName(name);
|
||||
console.log(TAG + "hook " + name + (a ? " -> " + a : " MISSING"));
|
||||
return a;
|
||||
}
|
||||
|
||||
|
||||
// handle (string) -> { path, pos, end, chunks:[{pos,ab}], save, traced }
|
||||
var files = {};
|
||||
var counters = {};
|
||||
|
||||
function baseName(path) {
|
||||
var parts = path.replace(/\\/g, "/").replace(/\/+$/, "").split("/");
|
||||
return (parts[parts.length - 1] || "capture.bin").replace(/[^A-Za-z0-9._-]/g, "");
|
||||
}
|
||||
|
||||
// Read the wide path out of an OBJECT_ATTRIBUTES* (x64 layout).
|
||||
function readObjName(pObjAttr) {
|
||||
if (pObjAttr.isNull()) return null;
|
||||
var pUni = pObjAttr.add(16).readPointer(); // OBJECT_ATTRIBUTES.ObjectName
|
||||
if (pUni.isNull()) return null;
|
||||
var len = pUni.readU16(); // UNICODE_STRING.Length (bytes)
|
||||
if (!len) return null;
|
||||
var buf = pUni.add(8).readPointer(); // UNICODE_STRING.Buffer
|
||||
if (buf.isNull()) return null;
|
||||
return buf.readUtf16String(len / 2);
|
||||
}
|
||||
|
||||
// NtCreateFile / NtOpenFile: args[0]=PHANDLE out, [1]=ACCESS_MASK, [2]=OBJECT_ATTRIBUTES*
|
||||
function hookOpen(name) {
|
||||
var addr = resolve(name);
|
||||
if (!addr) return;
|
||||
Interceptor.attach(addr, {
|
||||
onEnter: function (args) {
|
||||
this.pHandle = args[0];
|
||||
this.access = args[1].toUInt32();
|
||||
this.path = readObjName(args[2]);
|
||||
},
|
||||
onLeave: function (retval) {
|
||||
if (!this.path) return;
|
||||
var WRITE = 0x40000000 /*GENERIC_WRITE*/ | 0x10000000 /*GENERIC_ALL*/ |
|
||||
0x2 /*FILE_WRITE_DATA*/ | 0x4 /*FILE_APPEND_DATA*/;
|
||||
var isWrite = (this.access & WRITE) !== 0;
|
||||
var save = SAVE_RE.test(this.path);
|
||||
// DEBUG: show every interesting open (incl. read-only) + its access mask,
|
||||
// so we can see if/how export.pdb is opened even when our filters miss it.
|
||||
if (DEBUG && DEBUG_RE.test(this.path))
|
||||
console.log(TAG + "[open] access=0x" + this.access.toString(16)
|
||||
+ (isWrite ? " W" : " r") + (save ? " *SAVE" : "") + " " + this.path);
|
||||
if (!isWrite && !save) return;
|
||||
var handleStr = this.pHandle.readPointer().toString();
|
||||
files[handleStr] = { path: this.path, pos: 0, end: 0, chunks: [], save: save, traced: false };
|
||||
if (!DEBUG && (save || VERBOSE)) console.log(TAG + "[w-open] " + this.path);
|
||||
},
|
||||
});
|
||||
}
|
||||
hookOpen("NtCreateFile");
|
||||
hookOpen("NtOpenFile");
|
||||
|
||||
// NtWriteFile: [0]=handle [5]=buffer [6]=length [7]=PLARGE_INTEGER ByteOffset
|
||||
Interceptor.attach(resolve("NtWriteFile"), {
|
||||
onEnter: function (args) {
|
||||
var f = files[args[0].toString()];
|
||||
if (!f) return;
|
||||
var n = args[6].toUInt32();
|
||||
if (n <= 0) return;
|
||||
|
||||
var pos = f.pos;
|
||||
var pOff = args[7];
|
||||
if (!pOff.isNull()) {
|
||||
var hi = pOff.add(4).readU32();
|
||||
if (hi !== 0xffffffff) pos = hi * 0x100000000 + pOff.readU32(); // skip sentinels
|
||||
}
|
||||
|
||||
if (f.save) {
|
||||
f.chunks.push({ pos: pos, ab: args[5].readByteArray(n) });
|
||||
if (!f.traced) {
|
||||
f.traced = true;
|
||||
console.log(TAG + "[trace] writer for " + f.path + ":");
|
||||
Thread.backtrace(this.context, Backtracer.ACCURATE)
|
||||
.slice(0, 14)
|
||||
.forEach(function (a) {
|
||||
// module+offset maps straight to Ghidra: static = 0x140000000 + offset
|
||||
var m = Process.findModuleByAddress(a);
|
||||
var loc = m ? (m.name + "+0x" + a.sub(m.base).toString(16)) : a.toString();
|
||||
console.log(" " + loc + " " + DebugSymbol.fromAddress(a));
|
||||
});
|
||||
}
|
||||
}
|
||||
f.pos = pos + n;
|
||||
if (f.pos > f.end) f.end = f.pos;
|
||||
},
|
||||
});
|
||||
|
||||
// Shared: write a captured buffer to OUT_DIR with a PID-tagged, de-duped name.
|
||||
function saveBuffer(path, arrayBuffer, size) {
|
||||
var name = baseName(path);
|
||||
var c = counters[name] || 0;
|
||||
counters[name] = c + 1;
|
||||
var dest = OUT_DIR + "p" + PID + "_" + name + (c === 0 ? "" : "." + c);
|
||||
var fh = new File(dest, "wb");
|
||||
fh.write(arrayBuffer);
|
||||
fh.close();
|
||||
console.log(TAG + "[SAVED] " + dest + " (" + size + " bytes)");
|
||||
}
|
||||
|
||||
// NtClose: reconstruct + flush saved files written via NtWriteFile
|
||||
function flush(handleStr) {
|
||||
var f = files[handleStr];
|
||||
if (!f) return;
|
||||
delete files[handleStr];
|
||||
if (!f.save || f.end === 0) return;
|
||||
var out = new Uint8Array(f.end);
|
||||
for (var i = 0; i < f.chunks.length; i++) out.set(new Uint8Array(f.chunks[i].ab), f.chunks[i].pos);
|
||||
saveBuffer(f.path, out.buffer, f.end);
|
||||
}
|
||||
Interceptor.attach(resolve("NtClose"), { onEnter: function (args) { flush(args[0].toString()); } });
|
||||
|
||||
// ---- memory-mapped writes (the .pdb is likely built via a file mapping) ----
|
||||
// CreateFileMapping/MapViewOfFile funnel through these ntdll calls. We link
|
||||
// section -> backing file (NtCreateSection takes the file handle we already
|
||||
// track), record the mapped view, and snapshot its memory when it's unmapped.
|
||||
var sections = {}; // sectionHandle -> { path, save, maxSize }
|
||||
var views = {}; // baseAddr string -> { base, size, path, save, maxSize }
|
||||
|
||||
function hookSections() {
|
||||
var cs = resolve("NtCreateSection");
|
||||
if (cs) Interceptor.attach(cs, {
|
||||
onEnter: function (args) {
|
||||
this.pSection = args[0];
|
||||
this.pMax = args[3]; // PLARGE_INTEGER MaximumSize (may be null)
|
||||
this.fileHandle = args[6].toString();// HANDLE FileHandle
|
||||
},
|
||||
onLeave: function (ret) {
|
||||
if (ret.toInt32() !== 0) return; // not STATUS_SUCCESS
|
||||
var f = files[this.fileHandle];
|
||||
if (!f) return; // only file-backed sections we track
|
||||
var maxSize = (this.pMax && !this.pMax.isNull()) ? this.pMax.readU32() : 0;
|
||||
sections[this.pSection.readPointer().toString()] = { path: f.path, save: f.save, maxSize: maxSize };
|
||||
},
|
||||
});
|
||||
|
||||
var mv = resolve("NtMapViewOfSection");
|
||||
if (mv) Interceptor.attach(mv, {
|
||||
onEnter: function (args) {
|
||||
this.section = args[0].toString();
|
||||
this.pBase = args[2]; // PVOID* BaseAddress
|
||||
this.pViewSize = args[6]; // PSIZE_T ViewSize
|
||||
},
|
||||
onLeave: function (ret) {
|
||||
if (ret.toInt32() < 0) return; // negative NTSTATUS = error
|
||||
var s = sections[this.section];
|
||||
if (!s) return;
|
||||
var base = this.pBase.readPointer();
|
||||
var size = (this.pViewSize && !this.pViewSize.isNull()) ? this.pViewSize.readU32() : 0;
|
||||
views[base.toString()] = { base: base, size: size, path: s.path, save: s.save, maxSize: s.maxSize };
|
||||
if (s.save || VERBOSE) console.log(TAG + "[mmap] " + s.path + " @ " + base + " view=" + size + " max=" + s.maxSize);
|
||||
},
|
||||
});
|
||||
|
||||
function captureView(base) {
|
||||
var v = views[base.toString()];
|
||||
if (!v) return;
|
||||
delete views[base.toString()];
|
||||
if (!v.save) return;
|
||||
var size = v.maxSize || v.size;
|
||||
if (!size) { var r = Process.findRangeByAddress(base); size = r ? r.size : 0; }
|
||||
if (!size) return;
|
||||
try { saveBuffer(v.path, Memory.readByteArray(base, size), size); }
|
||||
catch (e) { console.log(TAG + "[mmap] read failed @ " + base + ": " + e); }
|
||||
}
|
||||
|
||||
var um = ntdll.findExportByName("NtUnmapViewOfSection");
|
||||
if (um) Interceptor.attach(um, { onEnter: function (args) { captureView(args[1]); } });
|
||||
}
|
||||
hookSections();
|
||||
|
||||
console.log(TAG + "ntdll + mmap hooks installed -- trigger 'Export to USB' in rekordbox now");
|
||||
Generated
+711
@@ -0,0 +1,711 @@
|
||||
version = 1
|
||||
revision = 3
|
||||
requires-python = ">=3.14"
|
||||
|
||||
[[package]]
|
||||
name = "aiohappyeyeballs"
|
||||
version = "2.6.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/33/c6/61a2d7b7572279226bb2e7f61d7a19ca7c90da0329c93fa0d560cbf288d8/aiohappyeyeballs-2.6.2.tar.gz", hash = "sha256:e202810ee718bd01fc6ef49e8ea53d023d5cb6b581076d7925aa499fa55dbe64", size = 22591, upload-time = "2026-05-20T15:12:24.631Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/5f/fc/a7bf5b6e4e617b45f90f2d9d2a68519c249c81dd4fc2658c7a2a61c4f4b7/aiohappyeyeballs-2.6.2-py3-none-any.whl", hash = "sha256:4708045e2d7a6c6bdf8aafa8ed39649eaf926a4543b54560659129e3365953c4", size = 15062, upload-time = "2026-05-20T15:12:23.328Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aiohttp"
|
||||
version = "3.14.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "aiohappyeyeballs" },
|
||||
{ name = "aiosignal" },
|
||||
{ name = "attrs" },
|
||||
{ name = "frozenlist" },
|
||||
{ name = "multidict" },
|
||||
{ name = "propcache" },
|
||||
{ name = "yarl" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/82/78/8ea7308cac6934de8c74a14f3d5f65d1c89287426688be79538d0e5c013d/aiohttp-3.14.1.tar.gz", hash = "sha256:307f2cff90a764d329e77040603fa032db89c5c24fdad50c4c15334cba744035", size = 7955794, upload-time = "2026-06-07T21:09:35.529Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c4/a1/5fafa04e1ca91ddb47608699d60649c1c6db3cf41c99e78fc4056f9513db/aiohttp-3.14.1-cp314-cp314-android_24_arm64_v8a.whl", hash = "sha256:7c106c26852ca1c2047c6b80384f17100b4e439af276f21ef3d4e2f450ae7e15", size = 508531, upload-time = "2026-06-07T21:08:02.093Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fa/2e/bfa02f699d87ffc86d5959270b28f1cb410add3ccaced8ed2e0b8a5238fc/aiohttp-3.14.1-cp314-cp314-android_24_x86_64.whl", hash = "sha256:20205f7f5ade7aaec9f4b500549bbc071b046453aed72f9c06dcab87896a83e8", size = 514718, upload-time = "2026-06-07T21:08:04.476Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/85/a5/9594ad6289eebbc97d167c44213d557807f90e59115caad24de21ad2c3b1/aiohttp-3.14.1-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:62a759436b29e677181a9e76bab8b8f689a29cb9c535f45f7c48c9c830d3f8c3", size = 487918, upload-time = "2026-06-07T21:08:06.377Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b4/61/16a32c36c3c49edec122a3dc811f2057df2f94d3b14aa107c8017d981618/aiohttp-3.14.1-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:2964cbf553df4d7a57348da44d961d871895fc1ee4e8c322b2a95612c7b17fba", size = 494014, upload-time = "2026-06-07T21:08:08.263Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9b/89/3ebcf96ed99c05bec9c434aaac6963fd3cbab4a786ae739908a144d9ce44/aiohttp-3.14.1-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:237651caadc3a59badd39319c54642b5299e9cc98a3a194310e55d5bb9f5e397", size = 502398, upload-time = "2026-06-07T21:08:10.244Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fd/3d/b74870a0c2d40c355928cd5b96c7a11fa821b8a40fc41365e64479b151fb/aiohttp-3.14.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:896e12dfdbbab9d8f7e16d2b28c6769a60126fa92095d1ebf9473d02593a2448", size = 758018, upload-time = "2026-06-07T21:08:12.447Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d3/66/f42f5c984d99e49c6cff5f26f590750f2e2f7ef1fcfb99966ab5be1b632e/aiohttp-3.14.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d03f281ed22579314ba00821ce20115a7c0ac430660b4cc05704a3f818b3e004", size = 512462, upload-time = "2026-06-07T21:08:14.624Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e9/a7/248e1aebe0c7810b0271e021a0f2a5eb6e78a051885b3c9df49f42a5802d/aiohttp-3.14.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:07eabb979d236335fed927e137a928c9adfb7df3b9ec7aa31726f133a62be983", size = 512824, upload-time = "2026-06-07T21:08:16.572Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/26/97/2aa0e5ba0727dc3bd5aaebb7ccbc510f7dfb7fb961ec87497cd496635ab1/aiohttp-3.14.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4fe1f1087cbadb280b5e1bb054a4f00d1423c74d6626c5e48400d871d34ecefe", size = 1749898, upload-time = "2026-06-07T21:08:18.635Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/00/8d/e97f6c96c891d457c8479d92a514ba194d0412f981d72c70341ee18488ed/aiohttp-3.14.1-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:367a9314fdc79dab0fac96e216cb41dd73c85bdca85306ce8999118ba7e0f333", size = 1710114, upload-time = "2026-06-07T21:08:20.892Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6f/e6/aa8d7e863048c8fceb5cd6ce74017311cec3ead07847387e12265fb4444e/aiohttp-3.14.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a24f677ebe83749039e7bdf862ff0bbb16818ae4193d4ef96505e269375bcce0", size = 1802541, upload-time = "2026-06-07T21:08:23.044Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/83/a8/72193137de57fda4ebfae4563182d082c8856e3b6e9871d0b46f028fb369/aiohttp-3.14.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:c83afe0ba876be7e943d2e0ba645809ad441575d2840c895c21ee5de93b9377a", size = 1875776, upload-time = "2026-06-07T21:08:25.288Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a0/18/938441025db6769a3464596b2410af3afde0b21eb2f204c6f766f68af4bd/aiohttp-3.14.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:634e385930fb6d2d479cf3aa66515955863b77a5e3c2b5894ca259a25b308602", size = 1760329, upload-time = "2026-06-07T21:08:27.363Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/60/29/bf2496b4065e76e09fe48015aaffe5ce161d8f089b06ac6982070f653076/aiohttp-3.14.1-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:eeea07c4397bbc57719c4eed8f9c284874d4f175f9b6d57f7a1546b976d455ca", size = 1587293, upload-time = "2026-06-07T21:08:29.805Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/49/a2/2136674d52123b1354bd05dd5753c318db47dc0c927cc70b27bab3755456/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:335c0cc3e3545ce98dcb9cfcb836f40c3411f43fa03dab757597d80c89af8a35", size = 1714756, upload-time = "2026-06-07T21:08:32.094Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a7/b9/e5fd2e6f915503081c0f9b1e8540947037929c70c191da2e4d54b31a21a1/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:ae6be797afdef264e8a84864a85b196ca06045586481b3df8a967322fd2fa844", size = 1721052, upload-time = "2026-06-07T21:08:34.167Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/63/5a/2833e324a2263e104e31e2e91bc5bbee81bc499afd32203faee048a883f0/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:8560b4d712474335d08907db7973f71912d3a9a8f1dee992ec06b5d2fe359496", size = 1766888, upload-time = "2026-06-07T21:08:36.95Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/57/fa/dea6511870913162f3b2e8c42a7614eb203a4540b8c2da43e0bfb0548f3c/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:2b7edd08e0a5deb1e8564a2fcd8f4561014a3f05252334671bbf55ddd47db0e5", size = 1581679, upload-time = "2026-06-07T21:08:39.292Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/14/bd/3cf0d55e71784b33534e9710a67d382d900598b4787fbce6cc7317f8c42a/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:b6ff7fcee63287ae57b5df3e4f5957ce032122802509246dec1a5bcc55904c95", size = 1782021, upload-time = "2026-06-07T21:08:41.407Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c1/af/14bb5843eccbe234f4dfb78ab73e549d99727247e62ae5d62cbd22eaf5b0/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6ffbb2f4ec1ceaff7e07d43922954da26b223d188bf30658e561b98e23089444", size = 1742574, upload-time = "2026-06-07T21:08:43.795Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f2/1e/fbeb7af9210a67ac0f9c9bec0f8f4568497924e33137a3d5b48e1cf85f3f/aiohttp-3.14.1-cp314-cp314-win32.whl", hash = "sha256:a9875b46d910cff3ea2f5962f9d266b465459fe634e22556ab9bd6fc1192eea0", size = 457773, upload-time = "2026-06-07T21:08:46.168Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f0/2b/13e8d741a9ec5db7d900c060554cf8352ab85e44e2a4469ebb9d377bda17/aiohttp-3.14.1-cp314-cp314-win_amd64.whl", hash = "sha256:af8b4b81a960eeaf1234971ac3cd0ba5901f3cd42eae42a46b4d089a8b492719", size = 485001, upload-time = "2026-06-07T21:08:48.401Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/df/30/491acfa2c4d6c3ff59c49a14fc1b50be3241e25bbb0c84c09e2da4d11395/aiohttp-3.14.1-cp314-cp314-win_arm64.whl", hash = "sha256:cf4491381b1b57425c315a56a439251b1bdac07b2275f19a8c44bc57744532ec", size = 453809, upload-time = "2026-06-07T21:08:50.7Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/34/e3/19dbe1a1f4cc6230eb9e314de7fe68053b0992f9302b27d12141a0b5db53/aiohttp-3.14.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:819c054312f1af92947e6a55883d1b66feefab11531a7fc45e0fb9b63880b5c2", size = 793320, upload-time = "2026-06-07T21:08:52.775Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7f/20/1b7182219ba1b108430d6e4dc53d25ae02dcfcf5a045b33af4e8c5167527/aiohttp-3.14.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:10ee9c1753a8f706345b22496c79fbddb5be0599e0823f3738b1534058e25340", size = 529077, upload-time = "2026-06-07T21:08:55Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b9/c8/14ce60ec31a2e5f5274bb17d383a6f7a3aabca31ac04eee05585bbadab16/aiohttp-3.14.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1601cc37baf5750ccacae618ec2daf020769581695550e3b654a911f859c563d", size = 532476, upload-time = "2026-06-07T21:08:57.176Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7e/02/9ac85e081e53da2e061b02fa7758fe0a12d17b8ce2d1f5e6c7cb76730328/aiohttp-3.14.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4d6e0ac9da31c9c04c84e1c0182ad8d6df35965a85cae29cd71d089621b3ae94", size = 1922347, upload-time = "2026-06-07T21:08:59.563Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c0/3e/d3ba07a0ab38b5389e10bec4362d21e10a4f667cba2d79ba30837b3a5059/aiohttp-3.14.1-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:9e8f2d660c350b3d0e259c7a7e3d9b7fc8b41210cbcc3d4a7076ff0a5e5c2fdc", size = 1786465, upload-time = "2026-06-07T21:09:01.909Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0b/cb/e2ee978a00cfb2df829704a69528b18154eba5939f45bc1efa8f33aee4c5/aiohttp-3.14.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:4691802dda97be727f79d86818acaad7eb8e9252626a1d6b519fedbb92d5e251", size = 1909423, upload-time = "2026-06-07T21:09:04.357Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/73/5d/1430334858b1022b58ae50399a918f0bd6fe8fa7fa183598d657ff61e040/aiohttp-3.14.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:c389c482a7e9b9dc3ee2701ac46c4125297a3818875b9c305ddb603c04828fd1", size = 2001906, upload-time = "2026-06-07T21:09:06.722Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/66/4e/560c7472d3d198a23aa5c8b19a5115bf6a9b77b7d3e4bb363da320430ad2/aiohttp-3.14.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fc0cacab7ba4e56f0f81c82a98c09bed2f39c940107b03a34b168bdf7597edd3", size = 1877095, upload-time = "2026-06-07T21:09:09.011Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0d/f1/4745806578d447db4a784a8591e2dae3afdfc2bcb96f8f81271b13df6543/aiohttp-3.14.1-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:979ed4717f59b8bb12e3963378fa285d93d367e15bcd66c721311826d3c44a6c", size = 1676222, upload-time = "2026-06-07T21:09:11.461Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6a/c9/48255813cca749a229ef0ab476004ec623728ad79a9c0840616f6c076325/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:38e1e7daaea81df51c952e18483f323d878499a1e2bfe564790e0f9701d6f203", size = 1842922, upload-time = "2026-06-07T21:09:14.118Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3d/c0/bbd054e2bee909f529523a5af3891052606af5143c09f5f183ec3b234676/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:4132e72c608fe9fecb8f409113567605915b83e9bdd3ea56538d2f9cd35002f1", size = 1825035, upload-time = "2026-06-07T21:09:16.447Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a8/ae/90395d4376deceb74e09ec26b6adf7d2015a6f8802d6d84446af860fef04/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:eefd9cc9b6d4a2db5f00a26bc3e4f9acf71926a6ec557cd56c9c6f27c290b665", size = 1849512, upload-time = "2026-06-07T21:09:18.742Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/93/bd/fb25f3049957553d4ce0ba6ae480aa2f592a6985497fca590837d16c1be0/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:b165790117eea512d7f3fb22f1f6dad3d55a7189571993eb015591c1401276d1", size = 1668571, upload-time = "2026-06-07T21:09:21.458Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3f/22/7f73303d64dd567ff3addca90b556690ed1233a47b8f55d242fb90af3681/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:ed09c7eb1c391271c2ed0314a51903e72a3acb653d5ccfc264cdf3ef11f8269d", size = 1881159, upload-time = "2026-06-07T21:09:23.813Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/44/be/0474c5a8b5640e1e4aa1923430a91f4151be82e511373fe764189b89aef5/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:99abd37084b82f5830c635fddd0b4993b9742a66eb746dacf433c8590e8f9e3c", size = 1841409, upload-time = "2026-06-07T21:09:26.207Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7b/3c/bb4a7cba26956cb3da4553cc2056cf67be5b5ff6e6d8fa4fbdff73bfb7ae/aiohttp-3.14.1-cp314-cp314t-win32.whl", hash = "sha256:47ddf841cdecc810749921d25606dee45857d12d2ad5ddb7b5bd7eab12e4b365", size = 494166, upload-time = "2026-06-07T21:09:28.505Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8a/84/ec80c2c1f66a952555a9f86df6b33af65108a6febfa0471b69013a12f807/aiohttp-3.14.1-cp314-cp314t-win_amd64.whl", hash = "sha256:5e78b522b7a6e27e0b25d19b247b75039ac4c94f99823e3c9e53ae1603a9f7e9", size = 530255, upload-time = "2026-06-07T21:09:30.843Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2a/71/6e22be134a4061ada85a92951b842f2657f17d926b727f3f94c56ae963d6/aiohttp-3.14.1-cp314-cp314t-win_arm64.whl", hash = "sha256:90d53f1609c29ccc2193945ef732428382a28f78d0456ae4d3daf0d48b74f0f6", size = 469640, upload-time = "2026-06-07T21:09:33.028Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aiosignal"
|
||||
version = "1.4.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "frozenlist" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/61/62/06741b579156360248d1ec624842ad0edf697050bbaf7c3e46394e106ad1/aiosignal-1.4.0.tar.gz", hash = "sha256:f47eecd9468083c2029cc99945502cb7708b082c232f9aca65da147157b251c7", size = 25007, upload-time = "2025-07-03T22:54:43.528Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/fb/76/641ae371508676492379f16e2fa48f4e2c11741bd63c48be4b12a6b09cba/aiosignal-1.4.0-py3-none-any.whl", hash = "sha256:053243f8b92b990551949e63930a839ff0cf0b0ebbe0597b0f3fb19e1a0fe82e", size = 7490, upload-time = "2025-07-03T22:54:42.156Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "attrs"
|
||||
version = "26.1.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/9a/8e/82a0fe20a541c03148528be8cac2408564a6c9a0cc7e9171802bc1d26985/attrs-26.1.0.tar.gz", hash = "sha256:d03ceb89cb322a8fd706d4fb91940737b6642aa36998fe130a9bc96c985eff32", size = 952055, upload-time = "2026-03-19T14:22:25.026Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/64/b4/17d4b0b2a2dc85a6df63d1157e028ed19f90d4cd97c36717afef2bc2f395/attrs-26.1.0-py3-none-any.whl", hash = "sha256:c647aa4a12dfbad9333ca4e71fe62ddc36f4e63b2d260a37a8b83d2f043ac309", size = 67548, upload-time = "2026-03-19T14:22:23.645Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "bidict"
|
||||
version = "0.23.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/9a/6e/026678aa5a830e07cd9498a05d3e7e650a4f56a42f267a53d22bcda1bdc9/bidict-0.23.1.tar.gz", hash = "sha256:03069d763bc387bbd20e7d49914e75fc4132a41937fa3405417e1a5a2d006d71", size = 29093, upload-time = "2024-02-18T19:09:05.748Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/99/37/e8730c3587a65eb5645d4aba2d27aae48e8003614d6aaf15dda67f702f1f/bidict-0.23.1-py3-none-any.whl", hash = "sha256:5dae8d4d79b552a71cbabc7deb25dfe8ce710b17ff41711e13010ead2abfc3e5", size = 32764, upload-time = "2024-02-18T19:09:04.156Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "colorama"
|
||||
version = "0.4.6"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "construct"
|
||||
version = "2.10.70"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/02/77/8c84b98eca70d245a2a956452f21d57930d22ab88cbeed9290ca630cf03f/construct-2.10.70.tar.gz", hash = "sha256:4d2472f9684731e58cc9c56c463be63baa1447d674e0d66aeb5627b22f512c29", size = 86337, upload-time = "2023-11-29T08:44:49.545Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b2/fb/08b3f4bf05da99aba8ffea52a558758def16e8516bc75ca94ff73587e7d3/construct-2.10.70-py3-none-any.whl", hash = "sha256:c80be81ef595a1a821ec69dc16099550ed22197615f4320b57cc9ce2a672cb30", size = 63020, upload-time = "2023-11-29T08:44:46.876Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "frozenlist"
|
||||
version = "1.8.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/2d/f5/c831fac6cc817d26fd54c7eaccd04ef7e0288806943f7cc5bbf69f3ac1f0/frozenlist-1.8.0.tar.gz", hash = "sha256:3ede829ed8d842f6cd48fc7081d7a41001a56f1f38603f9d49bf3020d59a31ad", size = 45875, upload-time = "2025-10-06T05:38:17.865Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/f1/c8/85da824b7e7b9b6e7f7705b2ecaf9591ba6f79c1177f324c2735e41d36a2/frozenlist-1.8.0-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:cee686f1f4cadeb2136007ddedd0aaf928ab95216e7691c63e50a8ec066336d0", size = 86127, upload-time = "2025-10-06T05:37:08.438Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8e/e8/a1185e236ec66c20afd72399522f142c3724c785789255202d27ae992818/frozenlist-1.8.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:119fb2a1bd47307e899c2fac7f28e85b9a543864df47aa7ec9d3c1b4545f096f", size = 49698, upload-time = "2025-10-06T05:37:09.48Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a1/93/72b1736d68f03fda5fdf0f2180fb6caaae3894f1b854d006ac61ecc727ee/frozenlist-1.8.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:4970ece02dbc8c3a92fcc5228e36a3e933a01a999f7094ff7c23fbd2beeaa67c", size = 49749, upload-time = "2025-10-06T05:37:10.569Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a7/b2/fabede9fafd976b991e9f1b9c8c873ed86f202889b864756f240ce6dd855/frozenlist-1.8.0-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:cba69cb73723c3f329622e34bdbf5ce1f80c21c290ff04256cff1cd3c2036ed2", size = 231298, upload-time = "2025-10-06T05:37:11.993Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3a/3b/d9b1e0b0eed36e70477ffb8360c49c85c8ca8ef9700a4e6711f39a6e8b45/frozenlist-1.8.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:778a11b15673f6f1df23d9586f83c4846c471a8af693a22e066508b77d201ec8", size = 232015, upload-time = "2025-10-06T05:37:13.194Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/dc/94/be719d2766c1138148564a3960fc2c06eb688da592bdc25adcf856101be7/frozenlist-1.8.0-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:0325024fe97f94c41c08872db482cf8ac4800d80e79222c6b0b7b162d5b13686", size = 225038, upload-time = "2025-10-06T05:37:14.577Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e4/09/6712b6c5465f083f52f50cf74167b92d4ea2f50e46a9eea0523d658454ae/frozenlist-1.8.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:97260ff46b207a82a7567b581ab4190bd4dfa09f4db8a8b49d1a958f6aa4940e", size = 240130, upload-time = "2025-10-06T05:37:15.781Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f8/d4/cd065cdcf21550b54f3ce6a22e143ac9e4836ca42a0de1022da8498eac89/frozenlist-1.8.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:54b2077180eb7f83dd52c40b2750d0a9f175e06a42e3213ce047219de902717a", size = 242845, upload-time = "2025-10-06T05:37:17.037Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/62/c3/f57a5c8c70cd1ead3d5d5f776f89d33110b1addae0ab010ad774d9a44fb9/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:2f05983daecab868a31e1da44462873306d3cbfd76d1f0b5b69c473d21dbb128", size = 229131, upload-time = "2025-10-06T05:37:18.221Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6c/52/232476fe9cb64f0742f3fde2b7d26c1dac18b6d62071c74d4ded55e0ef94/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:33f48f51a446114bc5d251fb2954ab0164d5be02ad3382abcbfe07e2531d650f", size = 240542, upload-time = "2025-10-06T05:37:19.771Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5f/85/07bf3f5d0fb5414aee5f47d33c6f5c77bfe49aac680bfece33d4fdf6a246/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:154e55ec0655291b5dd1b8731c637ecdb50975a2ae70c606d100750a540082f7", size = 237308, upload-time = "2025-10-06T05:37:20.969Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/11/99/ae3a33d5befd41ac0ca2cc7fd3aa707c9c324de2e89db0e0f45db9a64c26/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:4314debad13beb564b708b4a496020e5306c7333fa9a3ab90374169a20ffab30", size = 238210, upload-time = "2025-10-06T05:37:22.252Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b2/60/b1d2da22f4970e7a155f0adde9b1435712ece01b3cd45ba63702aea33938/frozenlist-1.8.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:073f8bf8becba60aa931eb3bc420b217bb7d5b8f4750e6f8b3be7f3da85d38b7", size = 231972, upload-time = "2025-10-06T05:37:23.5Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3f/ab/945b2f32de889993b9c9133216c068b7fcf257d8595a0ac420ac8677cab0/frozenlist-1.8.0-cp314-cp314-win32.whl", hash = "sha256:bac9c42ba2ac65ddc115d930c78d24ab8d4f465fd3fc473cdedfccadb9429806", size = 40536, upload-time = "2025-10-06T05:37:25.581Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/59/ad/9caa9b9c836d9ad6f067157a531ac48b7d36499f5036d4141ce78c230b1b/frozenlist-1.8.0-cp314-cp314-win_amd64.whl", hash = "sha256:3e0761f4d1a44f1d1a47996511752cf3dcec5bbdd9cc2b4fe595caf97754b7a0", size = 44330, upload-time = "2025-10-06T05:37:26.928Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/82/13/e6950121764f2676f43534c555249f57030150260aee9dcf7d64efda11dd/frozenlist-1.8.0-cp314-cp314-win_arm64.whl", hash = "sha256:d1eaff1d00c7751b7c6662e9c5ba6eb2c17a2306ba5e2a37f24ddf3cc953402b", size = 40627, upload-time = "2025-10-06T05:37:28.075Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c0/c7/43200656ecc4e02d3f8bc248df68256cd9572b3f0017f0a0c4e93440ae23/frozenlist-1.8.0-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:d3bb933317c52d7ea5004a1c442eef86f426886fba134ef8cf4226ea6ee1821d", size = 89238, upload-time = "2025-10-06T05:37:29.373Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d1/29/55c5f0689b9c0fb765055629f472c0de484dcaf0acee2f7707266ae3583c/frozenlist-1.8.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:8009897cdef112072f93a0efdce29cd819e717fd2f649ee3016efd3cd885a7ed", size = 50738, upload-time = "2025-10-06T05:37:30.792Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ba/7d/b7282a445956506fa11da8c2db7d276adcbf2b17d8bb8407a47685263f90/frozenlist-1.8.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:2c5dcbbc55383e5883246d11fd179782a9d07a986c40f49abe89ddf865913930", size = 51739, upload-time = "2025-10-06T05:37:32.127Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/62/1c/3d8622e60d0b767a5510d1d3cf21065b9db874696a51ea6d7a43180a259c/frozenlist-1.8.0-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:39ecbc32f1390387d2aa4f5a995e465e9e2f79ba3adcac92d68e3e0afae6657c", size = 284186, upload-time = "2025-10-06T05:37:33.21Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2d/14/aa36d5f85a89679a85a1d44cd7a6657e0b1c75f61e7cad987b203d2daca8/frozenlist-1.8.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:92db2bf818d5cc8d9c1f1fc56b897662e24ea5adb36ad1f1d82875bd64e03c24", size = 292196, upload-time = "2025-10-06T05:37:36.107Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/05/23/6bde59eb55abd407d34f77d39a5126fb7b4f109a3f611d3929f14b700c66/frozenlist-1.8.0-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:2dc43a022e555de94c3b68a4ef0b11c4f747d12c024a520c7101709a2144fb37", size = 273830, upload-time = "2025-10-06T05:37:37.663Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d2/3f/22cff331bfad7a8afa616289000ba793347fcd7bc275f3b28ecea2a27909/frozenlist-1.8.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:cb89a7f2de3602cfed448095bab3f178399646ab7c61454315089787df07733a", size = 294289, upload-time = "2025-10-06T05:37:39.261Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a4/89/5b057c799de4838b6c69aa82b79705f2027615e01be996d2486a69ca99c4/frozenlist-1.8.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:33139dc858c580ea50e7e60a1b0ea003efa1fd42e6ec7fdbad78fff65fad2fd2", size = 300318, upload-time = "2025-10-06T05:37:43.213Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/30/de/2c22ab3eb2a8af6d69dc799e48455813bab3690c760de58e1bf43b36da3e/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:168c0969a329b416119507ba30b9ea13688fafffac1b7822802537569a1cb0ef", size = 282814, upload-time = "2025-10-06T05:37:45.337Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/59/f7/970141a6a8dbd7f556d94977858cfb36fa9b66e0892c6dd780d2219d8cd8/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:28bd570e8e189d7f7b001966435f9dac6718324b5be2990ac496cf1ea9ddb7fe", size = 291762, upload-time = "2025-10-06T05:37:46.657Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c1/15/ca1adae83a719f82df9116d66f5bb28bb95557b3951903d39135620ef157/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:b2a095d45c5d46e5e79ba1e5b9cb787f541a8dee0433836cea4b96a2c439dcd8", size = 289470, upload-time = "2025-10-06T05:37:47.946Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ac/83/dca6dc53bf657d371fbc88ddeb21b79891e747189c5de990b9dfff2ccba1/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:eab8145831a0d56ec9c4139b6c3e594c7a83c2c8be25d5bcf2d86136a532287a", size = 289042, upload-time = "2025-10-06T05:37:49.499Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/96/52/abddd34ca99be142f354398700536c5bd315880ed0a213812bc491cff5e4/frozenlist-1.8.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:974b28cf63cc99dfb2188d8d222bc6843656188164848c4f679e63dae4b0708e", size = 283148, upload-time = "2025-10-06T05:37:50.745Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/af/d3/76bd4ed4317e7119c2b7f57c3f6934aba26d277acc6309f873341640e21f/frozenlist-1.8.0-cp314-cp314t-win32.whl", hash = "sha256:342c97bf697ac5480c0a7ec73cd700ecfa5a8a40ac923bd035484616efecc2df", size = 44676, upload-time = "2025-10-06T05:37:52.222Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/89/76/c615883b7b521ead2944bb3480398cbb07e12b7b4e4d073d3752eb721558/frozenlist-1.8.0-cp314-cp314t-win_amd64.whl", hash = "sha256:06be8f67f39c8b1dc671f5d83aaefd3358ae5cdcf8314552c57e7ed3e6475bdd", size = 49451, upload-time = "2025-10-06T05:37:53.425Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e0/a3/5982da14e113d07b325230f95060e2169f5311b1017ea8af2a29b374c289/frozenlist-1.8.0-cp314-cp314t-win_arm64.whl", hash = "sha256:102e6314ca4da683dca92e3b1355490fed5f313b768500084fbe6371fddfdb79", size = 42507, upload-time = "2025-10-06T05:37:54.513Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9a/9a/e35b4a917281c0b8419d4207f4334c8e8c5dbf4f3f5f9ada73958d937dcc/frozenlist-1.8.0-py3-none-any.whl", hash = "sha256:0c18a16eab41e82c295618a77502e17b195883241c563b00f0aa5106fc4eaa0d", size = 13409, upload-time = "2025-10-06T05:38:16.721Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "greenlet"
|
||||
version = "3.5.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/6d/6e/802acd792aebb2256fbbee8cacf2727faaeb6f240ac11008f09eae4414bc/greenlet-3.5.1.tar.gz", hash = "sha256:5a56aeb7d5d9cc4b3a735efb5095bd4b4f6f0e4f93e5ca876d0e2315137b7829", size = 197356, upload-time = "2026-05-20T15:05:03.917Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/8a/cb/c62454606daf5640369c94d8a9dd540599b1bfc090e2d2180cb77f4038d2/greenlet-3.5.1-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:d8ab31c9de8651a2facdd5c5bb0011f2380dd1a7af78ce2adf4b56095294fc07", size = 285579, upload-time = "2026-05-20T13:08:56.396Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ec/71/c4270398c2eba968a6071af1dfbdcaeee6ec1c24bc8b435b8cc452700da6/greenlet-3.5.1-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5e300185139abc337ade480c327183adf42a875ac7181bfe66d7d4efea31fbea", size = 651106, upload-time = "2026-05-20T14:00:09.448Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1a/ab/71e34b78a44ec271fb5f550c17bc46d301ddc5953890d935f270b0dcdb5a/greenlet-3.5.1-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7ffdb990dcaa0234cf9845aead5df2e3c3a8b6507d409274dd87e0d5ab05ffc2", size = 663478, upload-time = "2026-05-20T14:05:45.88Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/77/96/4efd6fa5c62c85426a0c19077a586258ebc3a2a146ff2493e4312a697a22/greenlet-3.5.1-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2f82b3597e9d83b63408affed0b48fd0f54935edac4302237b9a837be0dae33c", size = 660800, upload-time = "2026-05-20T13:14:29.129Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7a/e0/6c71401a25cac7000261304e866a2f2cc04dc74810d40e2f118aa4799495/greenlet-3.5.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c0141e37414c10164e702b8fb1473304221ad98f71600850c6ef7ff4880feba0", size = 1617518, upload-time = "2026-05-20T14:02:28.662Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/41/26/c5c06643e8c0af9e7bf18e16cb51d0ab7625155f0392e1c9015d66d556cd/greenlet-3.5.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:50ae25a67bea74ea41fb14b960bc532df73eb713417b2d61892dced82fe8d3bc", size = 1681593, upload-time = "2026-05-20T13:14:39.417Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8a/bd/e11a108317485075e68af9d23039619b86b28130c3b50d227d42edece64b/greenlet-3.5.1-cp314-cp314-win_amd64.whl", hash = "sha256:8a17c42330e261299766b75ac1ea32caa437a9453c8f65d16a13140db378ecd3", size = 239800, upload-time = "2026-05-20T13:09:30.128Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/47/f8/8e8e8417b7bf28639a5a56356ef934d0375e1d0c70a57e04d7701e870ffe/greenlet-3.5.1-cp314-cp314-win_arm64.whl", hash = "sha256:7b5f5fae05b8ac6d176a61b60c394a8cbdc2b5b91b81793066e68745cf165e54", size = 236862, upload-time = "2026-05-20T13:09:10.498Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/90/12/41bf27fde4d3605d3773ae57751eda182b8be2f5398011c041173b1d9534/greenlet-3.5.1-cp314-cp314t-macosx_11_0_universal2.whl", hash = "sha256:ea8da1e900d758d078810d4255d8c6aa572181896a31ec79d779eb79c3adc9ad", size = 293637, upload-time = "2026-05-20T13:12:35.529Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/44/44/ba14b23e9757707050c2f397d305bbcae62e5d7cad122f8b6baec5ae4a1f/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a19570c52a21420dcbc94e661994bc325c0b5b11304540fed514586da5dc8f2e", size = 650840, upload-time = "2026-05-20T14:00:11.079Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a8/37/5ddc2b686a6844f91abecef43411842426da2e1573f60b49ecf2547f4ae1/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:3d955c89b75eeca4723d7cc14135f393cd47c32e2a6cb4a8e4c6e760a26b0986", size = 656416, upload-time = "2026-05-20T14:05:47.118Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e1/f0/d17510297c35a2992712f0bf84de3779749999f7d3d63aa1f09db7c62dbe/greenlet-3.5.1-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:de2daaaebd1a5aa88c49045b6baf9310b3263796bd88db713edf37cf53e7bb4e", size = 654397, upload-time = "2026-05-20T13:14:30.696Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/37/eb/147387705bb89092645b012586e7273cb5ed3c90ef7eaf3a69173eaf0209/greenlet-3.5.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:3bfbd69cc349e43bf3a8ae1c85548ff0718efc887615c2db16c3833d7b0b072d", size = 1614469, upload-time = "2026-05-20T14:02:30.192Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a6/4e/37ee0da7732b7aa9896f17e15579a9df34b9fcb9dd494f0adfa749af6623/greenlet-3.5.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:4378720dd888136c27215a0214d32a4d37c3852765d45bc37aad0623423cfd78", size = 1675115, upload-time = "2026-05-20T13:14:40.972Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/57/f3/97dfcf4a6eb5077f8a672234216fb5923eb89f2cab7081cb10b2cf75b605/greenlet-3.5.1-cp314-cp314t-win_amd64.whl", hash = "sha256:45718441607f9325d948db98cbc691276059316d0358c188c246da4e1d4d23d2", size = 245246, upload-time = "2026-05-20T13:12:22.646Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5d/73/d7f72e34b582f694f4a9b248162db7b09cc458a259ba8f0c0bfa1a34ea7d/greenlet-3.5.1-cp315-cp315-macosx_11_0_universal2.whl", hash = "sha256:2baee5ca02031757ffe8cc3d69f0cc0aec7065ce362622da74f32d3bcab1c541", size = 285575, upload-time = "2026-05-20T13:12:07.043Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/df/59/fa9c6e87dc8ad27a95dabe2f29f372b733d05a8a67470f6c901ed9975655/greenlet-3.5.1-cp315-cp315-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9b1ec3274918a81d3ea778b9e75b56b72b33f300edb6cf7f3a7fe1dae56683de", size = 656428, upload-time = "2026-05-20T14:00:12.556Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f6/f9/e753408871eaa61dfe35e619cfc67512b036fde99893685d50eea9e07146/greenlet-3.5.1-cp315-cp315-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:111e2390ffffc47d5840b01711dd7fac07d4c09283d0283e7f3264b14e284c64", size = 667064, upload-time = "2026-05-20T14:05:48.662Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/96/27/5565b5b40389f1c7753003a07e21892fda8660926787036d5bc0308b8113/greenlet-3.5.1-cp315-cp315-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e630136e905fe5ff43e86945ae41220b6d1470956a39220e708110ac48d01ea5", size = 665697, upload-time = "2026-05-20T13:14:32.943Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cf/82/e7de4178c0c2d1c9a5a3be3cc0b33e46a85b3ee4a77c071bf7ad8600e079/greenlet-3.5.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:975eac34b44a7077ca4d421348455b94f0f518246a7f14bc6d2fdcfe5b584368", size = 1621256, upload-time = "2026-05-20T14:02:31.91Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/00/10/f2dddcf7dacac17dfc68691809589adad06135eb28930429cf58a6467a2f/greenlet-3.5.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:9ab3c3a0b2ae6198e67c898dad5215a49f9ae0d0081b3c3ec59f333e39eeca26", size = 1685956, upload-time = "2026-05-20T13:14:42.55Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/22/17/4a232b32133230ada52f70e9d7f5b65b0caef8772f01849bd8d149e7e4ca/greenlet-3.5.1-cp315-cp315-win_amd64.whl", hash = "sha256:cbfc69be86e10dcfef5b1e6269d1d6926552aa89ee39e1de3353360c1b6989ab", size = 239802, upload-time = "2026-05-20T13:13:15.481Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c2/ae/4e623a7e6d4d2a5f4cb8e4c82de4169fc637942caae68d6e676b8a128ac5/greenlet-3.5.1-cp315-cp315-win_arm64.whl", hash = "sha256:92fd6d44ac5e5a887c8a5dc4a8ba0ba908527c31c12f78c6bc7dcfe8aab279f6", size = 236853, upload-time = "2026-05-20T13:15:37.301Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7a/57/816d9cff29119da3505b3d6a5e14a8af89006ac36f47f891ff293ee05af1/greenlet-3.5.1-cp315-cp315t-macosx_11_0_universal2.whl", hash = "sha256:a6fdf2433a5441ef9a95464f7c3e674775da1c8c1177fff311cee1acad4626ed", size = 293877, upload-time = "2026-05-20T13:10:19.078Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/23/a1/59b0a7c7d140ff1a75626680b9a9899b79a9176cab298b394968fb023295/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7546556f0d649f99f6a361098a55f761181bb2ea12ff150bb16d26092ad88244", size = 655333, upload-time = "2026-05-20T14:00:14.758Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/72/1b/5efe127597625042218939d01855109f352779050768b670b52edcc16a6c/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d5ee3ea898009fa898f85f9982255d35278c477bebe185beca249cab42d4526c", size = 659443, upload-time = "2026-05-20T14:05:50.159Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6c/6d/c404246ea4d22d097a7426d0efb5b781bd7eb67715f09e79001bd552ab18/greenlet-3.5.1-cp315-cp315t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a5c81f74d204d3edd136ebfd50dce53acbb776995d721a0fe801626cfc93b8cd", size = 658356, upload-time = "2026-05-20T13:14:35.091Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/51/02/f8ee37fb6d2219329f350af241c27fcf12df57e723d11f6fc6d3bacdadaa/greenlet-3.5.1-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:2c18ef16bf6d4dd410e4dd52996888ea1497be26892fe5bbc73580aba4287b8e", size = 1619216, upload-time = "2026-05-20T14:02:33.403Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/93/c5/3dc9475ace2c7a3680da12372cddd7f1ac874eb410a1ac48d3e9dab83782/greenlet-3.5.1-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:17d86354f0ae6b61bf9be5148d0dd34e06c3cb7c602c671f79f29ac3b150e659", size = 1678427, upload-time = "2026-05-20T13:14:43.71Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/df/4e/750c15c317a41ffb36f0bf40b933e3d744a7dede61889f74443ea69690cf/greenlet-3.5.1-cp315-cp315t-win_amd64.whl", hash = "sha256:e7516cf6ae6b8a582c2770a0caed47b8a48373ed732c33d69a72913ae6ac923e", size = 245225, upload-time = "2026-05-20T13:13:59.366Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4f/fd/d3baea2eeb7b617efd47e87ca06e2ec2c6118d303aa9e918e0ce16eadc10/greenlet-3.5.1-cp315-cp315t-win_arm64.whl", hash = "sha256:5028648bf2253ec4745add746129d3904121fa7fe871a76bed23c5720573ce0a", size = 239590, upload-time = "2026-05-20T13:13:37.382Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "idna"
|
||||
version = "3.18"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/cd/63/9496c57188a2ee585e0f1db071d75089a11e98aa86eb99d9d7618fc1edce/idna-3.18.tar.gz", hash = "sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848", size = 196711, upload-time = "2026-06-02T14:34:07.794Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "iniconfig"
|
||||
version = "2.3.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "kaitaistruct"
|
||||
version = "0.11"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/27/b8/ca7319556912f68832daa4b81425314857ec08dfccd8dbc8c0f65c992108/kaitaistruct-0.11.tar.gz", hash = "sha256:053ee764288e78b8e53acf748e9733268acbd579b8d82a427b1805453625d74b", size = 11519, upload-time = "2025-09-08T15:46:25.037Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/4a/4a/cf14bf3b1f5ffb13c69cf5f0ea78031247790558ee88984a8bdd22fae60d/kaitaistruct-0.11-py2.py3-none-any.whl", hash = "sha256:5c6ce79177b4e193a577ecd359e26516d1d6d000a0bffd6e1010f2a46a62a561", size = 11372, upload-time = "2025-09-08T15:46:23.635Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "loguru"
|
||||
version = "0.7.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "colorama", marker = "sys_platform == 'win32'" },
|
||||
{ name = "win32-setctime", marker = "sys_platform == 'win32'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/3a/05/a1dae3dffd1116099471c643b8924f5aa6524411dc6c63fdae648c4f1aca/loguru-0.7.3.tar.gz", hash = "sha256:19480589e77d47b8d85b2c827ad95d49bf31b0dcde16593892eb51dd18706eb6", size = 63559, upload-time = "2024-12-06T11:20:56.608Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/0c/29/0348de65b8cc732daa3e33e67806420b2ae89bdce2b04af740289c5c6c8c/loguru-0.7.3-py3-none-any.whl", hash = "sha256:31a33c10c8e1e10422bfd431aeb5d351c7cf7fa671e3c4df004162264b28220c", size = 61595, upload-time = "2024-12-06T11:20:54.538Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "maple"
|
||||
version = "0.1.0"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "aiohttp" },
|
||||
{ name = "kaitaistruct" },
|
||||
{ name = "loguru" },
|
||||
{ name = "mixxx-analyzer" },
|
||||
{ name = "mutagen" },
|
||||
{ name = "pillow" },
|
||||
{ name = "pyrekordbox" },
|
||||
{ name = "scipy" },
|
||||
]
|
||||
|
||||
[package.dev-dependencies]
|
||||
dev = [
|
||||
{ name = "pytest" },
|
||||
]
|
||||
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "aiohttp", specifier = ">=3.10" },
|
||||
{ name = "kaitaistruct", specifier = ">=0.11" },
|
||||
{ name = "loguru", specifier = ">=0.7" },
|
||||
{ name = "mixxx-analyzer", specifier = ">=0.1.2" },
|
||||
{ name = "mutagen", specifier = ">=1.47.0" },
|
||||
{ name = "pillow", specifier = ">=12.2.0" },
|
||||
{ name = "pyrekordbox", specifier = ">=0.4.4" },
|
||||
{ name = "scipy", specifier = ">=1.10" },
|
||||
]
|
||||
|
||||
[package.metadata.requires-dev]
|
||||
dev = [{ name = "pytest", specifier = ">=9.0.3" }]
|
||||
|
||||
[[package]]
|
||||
name = "mixxx-analyzer"
|
||||
version = "0.1.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/53/b3/c4026eed48e7685a99212ede8a1ce22bd2442a7d677ec4b970a7718d5b23/mixxx_analyzer-0.1.2-py3-none-macosx_14_0_arm64.whl", hash = "sha256:cb509c06d630fad709f85dd07fbd21a029d337617de0866af74279ca0a1f596b", size = 13932797, upload-time = "2026-03-01T15:44:59.349Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a6/30/bf8afbb52d862d2976045b4f6d00456dcb6320bb5e449384a41a0ce40735/mixxx_analyzer-0.1.2-py3-none-manylinux_2_35_x86_64.whl", hash = "sha256:53d8ec329bae7a7714dfe1cd34cbcf3976ddb55fae0ed74c4872f47085a9d9cf", size = 56567723, upload-time = "2026-03-01T15:45:02.771Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2e/88/e22c9e7c3d862daec693de61c5f2843ce8e567cd4d5e866040c599b57e9d/mixxx_analyzer-0.1.2-py3-none-win_amd64.whl", hash = "sha256:cc6509acfc78f0f3e04ac9f0b1440e9cd31a3c389f888bbd91beae1f249c32b7", size = 44947103, upload-time = "2026-03-01T15:45:05.66Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "multidict"
|
||||
version = "6.7.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/1a/c2/c2d94cbe6ac1753f3fc980da97b3d930efe1da3af3c9f5125354436c073d/multidict-6.7.1.tar.gz", hash = "sha256:ec6652a1bee61c53a3e5776b6049172c53b6aaba34f18c9ad04f82712bac623d", size = 102010, upload-time = "2026-01-26T02:46:45.979Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/91/cc/db74228a8be41884a567e88a62fd589a913708fcf180d029898c17a9a371/multidict-6.7.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:8f333ec9c5eb1b7105e3b84b53141e66ca05a19a605368c55450b6ba208cb9ee", size = 75190, upload-time = "2026-01-26T02:45:10.651Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d5/22/492f2246bb5b534abd44804292e81eeaf835388901f0c574bac4eeec73c5/multidict-6.7.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:a407f13c188f804c759fc6a9f88286a565c242a76b27626594c133b82883b5c2", size = 44486, upload-time = "2026-01-26T02:45:11.938Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f1/4f/733c48f270565d78b4544f2baddc2fb2a245e5a8640254b12c36ac7ac68e/multidict-6.7.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:0e161ddf326db5577c3a4cc2d8648f81456e8a20d40415541587a71620d7a7d1", size = 43219, upload-time = "2026-01-26T02:45:14.346Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/24/bb/2c0c2287963f4259c85e8bcbba9182ced8d7fca65c780c38e99e61629d11/multidict-6.7.1-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:1e3a8bb24342a8201d178c3b4984c26ba81a577c80d4d525727427460a50c22d", size = 245132, upload-time = "2026-01-26T02:45:15.712Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a7/f9/44d4b3064c65079d2467888794dea218d1601898ac50222ab8a9a8094460/multidict-6.7.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:97231140a50f5d447d3164f994b86a0bed7cd016e2682f8650d6a9158e14fd31", size = 252420, upload-time = "2026-01-26T02:45:17.293Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8b/13/78f7275e73fa17b24c9a51b0bd9d73ba64bb32d0ed51b02a746eb876abe7/multidict-6.7.1-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:6b10359683bd8806a200fd2909e7c8ca3a7b24ec1d8132e483d58e791d881048", size = 233510, upload-time = "2026-01-26T02:45:19.356Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4b/25/8167187f62ae3cbd52da7893f58cb036b47ea3fb67138787c76800158982/multidict-6.7.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:283ddac99f7ac25a4acadbf004cb5ae34480bbeb063520f70ce397b281859362", size = 264094, upload-time = "2026-01-26T02:45:20.834Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a1/e7/69a3a83b7b030cf283fb06ce074a05a02322359783424d7edf0f15fe5022/multidict-6.7.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:538cec1e18c067d0e6103aa9a74f9e832904c957adc260e61cd9d8cf0c3b3d37", size = 260786, upload-time = "2026-01-26T02:45:22.818Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fe/3b/8ec5074bcfc450fe84273713b4b0a0dd47c0249358f5d82eb8104ffe2520/multidict-6.7.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7eee46ccb30ff48a1e35bb818cc90846c6be2b68240e42a78599166722cea709", size = 248483, upload-time = "2026-01-26T02:45:24.368Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/48/5a/d5a99e3acbca0e29c5d9cba8f92ceb15dce78bab963b308ae692981e3a5d/multidict-6.7.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fa263a02f4f2dd2d11a7b1bb4362aa7cb1049f84a9235d31adf63f30143469a0", size = 248403, upload-time = "2026-01-26T02:45:25.982Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/35/48/e58cd31f6c7d5102f2a4bf89f96b9cf7e00b6c6f3d04ecc44417c00a5a3c/multidict-6.7.1-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:2e1425e2f99ec5bd36c15a01b690a1a2456209c5deed58f95469ffb46039ccbb", size = 240315, upload-time = "2026-01-26T02:45:27.487Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/94/33/1cd210229559cb90b6786c30676bb0c58249ff42f942765f88793b41fdce/multidict-6.7.1-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:497394b3239fc6f0e13a78a3e1b61296e72bf1c5f94b4c4eb80b265c37a131cd", size = 245528, upload-time = "2026-01-26T02:45:28.991Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/64/f2/6e1107d226278c876c783056b7db43d800bb64c6131cec9c8dfb6903698e/multidict-6.7.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:233b398c29d3f1b9676b4b6f75c518a06fcb2ea0b925119fb2c1bc35c05e1601", size = 258784, upload-time = "2026-01-26T02:45:30.503Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4d/c1/11f664f14d525e4a1b5327a82d4de61a1db604ab34c6603bb3c2cc63ad34/multidict-6.7.1-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:93b1818e4a6e0930454f0f2af7dfce69307ca03cdcfb3739bf4d91241967b6c1", size = 251980, upload-time = "2026-01-26T02:45:32.603Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e1/9f/75a9ac888121d0c5bbd4ecf4eead45668b1766f6baabfb3b7f66a410e231/multidict-6.7.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:f33dc2a3abe9249ea5d8360f969ec7f4142e7ac45ee7014d8f8d5acddf178b7b", size = 243602, upload-time = "2026-01-26T02:45:34.043Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9a/e7/50bf7b004cc8525d80dbbbedfdc7aed3e4c323810890be4413e589074032/multidict-6.7.1-cp314-cp314-win32.whl", hash = "sha256:3ab8b9d8b75aef9df299595d5388b14530839f6422333357af1339443cff777d", size = 40930, upload-time = "2026-01-26T02:45:36.278Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e0/bf/52f25716bbe93745595800f36fb17b73711f14da59ed0bb2eba141bc9f0f/multidict-6.7.1-cp314-cp314-win_amd64.whl", hash = "sha256:5e01429a929600e7dab7b166062d9bb54a5eed752384c7384c968c2afab8f50f", size = 45074, upload-time = "2026-01-26T02:45:37.546Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/97/ab/22803b03285fa3a525f48217963da3a65ae40f6a1b6f6cf2768879e208f9/multidict-6.7.1-cp314-cp314-win_arm64.whl", hash = "sha256:4885cb0e817aef5d00a2e8451d4665c1808378dc27c2705f1bf4ef8505c0d2e5", size = 42471, upload-time = "2026-01-26T02:45:38.889Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e0/6d/f9293baa6146ba9507e360ea0292b6422b016907c393e2f63fc40ab7b7b5/multidict-6.7.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:0458c978acd8e6ea53c81eefaddbbee9c6c5e591f41b3f5e8e194780fe026581", size = 82401, upload-time = "2026-01-26T02:45:40.254Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7a/68/53b5494738d83558d87c3c71a486504d8373421c3e0dbb6d0db48ad42ee0/multidict-6.7.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:c0abd12629b0af3cf590982c0b413b1e7395cd4ec026f30986818ab95bfaa94a", size = 48143, upload-time = "2026-01-26T02:45:41.635Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/37/e8/5284c53310dcdc99ce5d66563f6e5773531a9b9fe9ec7a615e9bc306b05f/multidict-6.7.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:14525a5f61d7d0c94b368a42cff4c9a4e7ba2d52e2672a7b23d84dc86fb02b0c", size = 46507, upload-time = "2026-01-26T02:45:42.99Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e4/fc/6800d0e5b3875568b4083ecf5f310dcf91d86d52573160834fb4bfcf5e4f/multidict-6.7.1-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:17307b22c217b4cf05033dabefe68255a534d637c6c9b0cc8382718f87be4262", size = 239358, upload-time = "2026-01-26T02:45:44.376Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/41/75/4ad0973179361cdf3a113905e6e088173198349131be2b390f9fa4da5fc6/multidict-6.7.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:7a7e590ff876a3eaf1c02a4dfe0724b6e69a9e9de6d8f556816f29c496046e59", size = 246884, upload-time = "2026-01-26T02:45:47.167Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c3/9c/095bb28b5da139bd41fb9a5d5caff412584f377914bd8787c2aa98717130/multidict-6.7.1-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:5fa6a95dfee63893d80a34758cd0e0c118a30b8dcb46372bf75106c591b77889", size = 225878, upload-time = "2026-01-26T02:45:48.698Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/07/d0/c0a72000243756e8f5a277b6b514fa005f2c73d481b7d9e47cd4568aa2e4/multidict-6.7.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a0543217a6a017692aa6ae5cc39adb75e587af0f3a82288b1492eb73dd6cc2a4", size = 253542, upload-time = "2026-01-26T02:45:50.164Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c0/6b/f69da15289e384ecf2a68837ec8b5ad8c33e973aa18b266f50fe55f24b8c/multidict-6.7.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f99fe611c312b3c1c0ace793f92464d8cd263cc3b26b5721950d977b006b6c4d", size = 252403, upload-time = "2026-01-26T02:45:51.779Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a2/76/b9669547afa5a1a25cd93eaca91c0da1c095b06b6d2d8ec25b713588d3a1/multidict-6.7.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9004d8386d133b7e6135679424c91b0b854d2d164af6ea3f289f8f2761064609", size = 244889, upload-time = "2026-01-26T02:45:53.27Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7e/a9/a50d2669e506dad33cfc45b5d574a205587b7b8a5f426f2fbb2e90882588/multidict-6.7.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e628ef0e6859ffd8273c69412a2465c4be4a9517d07261b33334b5ec6f3c7489", size = 241982, upload-time = "2026-01-26T02:45:54.919Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c5/bb/1609558ad8b456b4827d3c5a5b775c93b87878fd3117ed3db3423dfbce1b/multidict-6.7.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:841189848ba629c3552035a6a7f5bf3b02eb304e9fea7492ca220a8eda6b0e5c", size = 232415, upload-time = "2026-01-26T02:45:56.981Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d8/59/6f61039d2aa9261871e03ab9dc058a550d240f25859b05b67fd70f80d4b3/multidict-6.7.1-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:ce1bbd7d780bb5a0da032e095c951f7014d6b0a205f8318308140f1a6aba159e", size = 240337, upload-time = "2026-01-26T02:45:58.698Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a1/29/fdc6a43c203890dc2ae9249971ecd0c41deaedfe00d25cb6564b2edd99eb/multidict-6.7.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:b26684587228afed0d50cf804cc71062cc9c1cdf55051c4c6345d372947b268c", size = 248788, upload-time = "2026-01-26T02:46:00.862Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a9/14/a153a06101323e4cf086ecee3faadba52ff71633d471f9685c42e3736163/multidict-6.7.1-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:9f9af11306994335398293f9958071019e3ab95e9a707dc1383a35613f6abcb9", size = 242842, upload-time = "2026-01-26T02:46:02.824Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/41/5f/604ae839e64a4a6efc80db94465348d3b328ee955e37acb24badbcd24d83/multidict-6.7.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:b4938326284c4f1224178a560987b6cf8b4d38458b113d9b8c1db1a836e640a2", size = 240237, upload-time = "2026-01-26T02:46:05.898Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5f/60/c3a5187bf66f6fb546ff4ab8fb5a077cbdd832d7b1908d4365c7f74a1917/multidict-6.7.1-cp314-cp314t-win32.whl", hash = "sha256:98655c737850c064a65e006a3df7c997cd3b220be4ec8fe26215760b9697d4d7", size = 48008, upload-time = "2026-01-26T02:46:07.468Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0c/f7/addf1087b860ac60e6f382240f64fb99f8bfb532bb06f7c542b83c29ca61/multidict-6.7.1-cp314-cp314t-win_amd64.whl", hash = "sha256:497bde6223c212ba11d462853cfa4f0ae6ef97465033e7dc9940cdb3ab5b48e5", size = 53542, upload-time = "2026-01-26T02:46:08.809Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4c/81/4629d0aa32302ef7b2ec65c75a728cc5ff4fa410c50096174c1632e70b3e/multidict-6.7.1-cp314-cp314t-win_arm64.whl", hash = "sha256:2bbd113e0d4af5db41d5ebfe9ccaff89de2120578164f86a5d17d5a576d1e5b2", size = 44719, upload-time = "2026-01-26T02:46:11.146Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/81/08/7036c080d7117f28a4af526d794aab6a84463126db031b007717c1a6676e/multidict-6.7.1-py3-none-any.whl", hash = "sha256:55d97cc6dae627efa6a6e548885712d4864b81110ac76fa4e534c03819fa4a56", size = 12319, upload-time = "2026-01-26T02:46:44.004Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mutagen"
|
||||
version = "1.47.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/81/e6/64bc71b74eef4b68e61eb921dcf72dabd9e4ec4af1e11891bbd312ccbb77/mutagen-1.47.0.tar.gz", hash = "sha256:719fadef0a978c31b4cf3c956261b3c58b6948b32023078a2117b1de09f0fc99", size = 1274186, upload-time = "2023-09-03T16:33:33.411Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b0/7a/620f945b96be1f6ee357d211d5bf74ab1b7fe72a9f1525aafbfe3aee6875/mutagen-1.47.0-py3-none-any.whl", hash = "sha256:edd96f50c5907a9539d8e5bba7245f62c9f520aef333d13392a79a4f70aca719", size = 194391, upload-time = "2023-09-03T16:33:29.955Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "numpy"
|
||||
version = "2.4.6"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d0/ad/fed0499ce6a338d2a03ebae59cd15093910c8875328855781952abf6c2fe/numpy-2.4.6.tar.gz", hash = "sha256:f3a3570c4a2a16746ac2c31a7c7c7b0c186b95ce902e33db6f28094ed7387dda", size = 20735807, upload-time = "2026-05-18T23:37:14.07Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/f8/91/3ab2044d05fd16d343c5ac2e69b127f1b2854040dd20b193257c78028bd3/numpy-2.4.6-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:06ca2f61ec4385a07a6977c55ba998a4466c123642b4a32694d3128fce18c079", size = 16683458, upload-time = "2026-05-18T23:35:38.353Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8e/62/764ce66fa4147ae6d73071a3abf804ffe606f174618697c571acdf26a7c9/numpy-2.4.6-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:38efbc8de75c7a0fc1ac190162d892787f3f47b57cc291231aafee36b80982b7", size = 14704559, upload-time = "2026-05-18T23:35:42.14Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/60/61/23f27c172f022e04025b7dc2367f4d63c1a398120607ec896228649a6f48/numpy-2.4.6-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:d581b735e177fdcdce6fed8e7e8880a3fb6ee4e3653a3ac6af01c6f4c03effc5", size = 5209716, upload-time = "2026-05-18T23:35:45.377Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/03/71/21cf70dc6ea3e3acb95fc53a265b2fc248b981f0194ceb5b475271b8809d/numpy-2.4.6-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:0a041d3d761dc3c35cc56ce0351506a02bcbc25f7b169f652435141a17db9096", size = 6543947, upload-time = "2026-05-18T23:35:47.926Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d5/91/64288395ee1799bd2e0b04a305dce9666da90c961e1f3fe982a05ee1c036/numpy-2.4.6-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:40fdc1ae7125e518ea98e53e69a4ebc27e1fd50510c47b7ea130cf21e5e1d42b", size = 15685197, upload-time = "2026-05-18T23:35:50.863Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f3/eb/ebffaa97dc55502df69584a8f0dcf07f69a3e0b3e2323670a2722db9aa39/numpy-2.4.6-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a2c306dea656c12c68f51f4cea133cbe78ca7435eb28c735eac1d3ebe73be6e8", size = 16638245, upload-time = "2026-05-18T23:35:54.752Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b8/0b/54f9da33128d7e350fab89c7455902eeae70349ee52bddb448dc4a576f45/numpy-2.4.6-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:33111801a01c12a8a1e3721f0a9232f8cfc8ae2c6b7098167e6f623c6073f402", size = 17036587, upload-time = "2026-05-18T23:35:58.355Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b6/f0/fdebc1052db1cc37c64beb22072d67cd6d1c71adca1299f53dec2b5e20d3/numpy-2.4.6-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ae506e6902902557576a26ff33eda8695e7ecb3cb36c3b573a0765dee114ebdb", size = 18363226, upload-time = "2026-05-18T23:36:02.845Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/aa/b4/298628d98c72b57e57f7165ae6a481a1deaf6f3c28262a6e4c739c275930/numpy-2.4.6-cp314-cp314-win32.whl", hash = "sha256:aaf159caa35993cb1f56fb9b8e4610d35758e7ca005412eb1daa856a78c9c4b1", size = 6010196, upload-time = "2026-05-18T23:36:05.92Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/df/ac/46de6dda46478f7942f839e094970be2d4a861e005c4b3bf07c92e291a09/numpy-2.4.6-cp314-cp314-win_amd64.whl", hash = "sha256:b507f5c4c1d508876d1819b6bf9a49d365b96320b5d4993426b33a23ca4b8261", size = 12450334, upload-time = "2026-05-18T23:36:09.107Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/78/92/b8b798ac784102c0da830d2257d59358e3d3d90d1e2b3f2575dad976c5cf/numpy-2.4.6-cp314-cp314-win_arm64.whl", hash = "sha256:6f41ae150c4e32db4f3310cdaf64b1593a03dbabe29eec77fc9b50fe64061df6", size = 10495678, upload-time = "2026-05-18T23:36:12.766Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/30/34/ec28d1aa8115971537c01469ab2011ee96827930f0a124de1000cc2a7ed7/numpy-2.4.6-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ece3d2cfe132e7d51f44a832b303895e6f2d499c5e74dfbdb06ee246147a304a", size = 14823672, upload-time = "2026-05-18T23:36:16.473Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/16/bd/f6d1fede4e54e8042a7ff97bb495510f3c220f94bcd9e8b228e87c92cc0d/numpy-2.4.6-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:e3e5193ef5a3dc73bceee50f7fdc2c90dbb76c42df8d8fae3d1067a583df579e", size = 5328731, upload-time = "2026-05-18T23:36:19.767Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f4/f0/e105b9e2fd728a9910103884decd6951d9dd73896b914a98d9a231de02ee/numpy-2.4.6-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:17f9ade344e7d9b464a084d69bcf18fc691cb1db67c62ed80820bf4926d78f0e", size = 6649805, upload-time = "2026-05-18T23:36:22.266Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/82/dd/1206a7ca6ab15e3f02069707ca96222e202af681bb73756da7527f3cb837/numpy-2.4.6-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9cd5ffd25db4e7ba6a375693b3fc0fc1791ec636c17db3720da19bde7180ec43", size = 15730496, upload-time = "2026-05-18T23:36:25.713Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/51/e7/38d3ea825dcab85a591734decb2f6c67caa7c8367d374df1a1c3842f9b07/numpy-2.4.6-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7d92c3819208a60205a12a245c91ad70cb0a85336659b19b834205573ac8456e", size = 16679616, upload-time = "2026-05-18T23:36:29.652Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/93/b7/caabfdf53edf663e0b4eb74d7d405d83baef09eb5e83bcd32d601d72b93e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e85b752a1e912b70eaad4fafbd4d1238007ab221de2009b9a2f5ae7461239895", size = 17085145, upload-time = "2026-05-18T23:36:33.449Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f9/45/68d7c33a6bcf3e5aa3bdbd57a367e6f615286dfd6482f97e8ffeb734306e/numpy-2.4.6-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:29cb7f67d10b479ff07c17d33e39f78c07f71c40ef30d63c153d340e96cd3fb4", size = 18403813, upload-time = "2026-05-18T23:36:37.369Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9c/50/0753655aa844c99cd9e018aacf76f130f1bd81d881bb74bc0aef5d73a8ba/numpy-2.4.6-cp314-cp314t-win32.whl", hash = "sha256:260a5d70215b61ab4fadf5c7baacd64821842975eea312125ed3c39a6391b063", size = 6156982, upload-time = "2026-05-18T23:36:40.817Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b2/d4/7c67becf668f973cb490cec3e98dfd799d866f9c989a54d355672cfa0db6/numpy-2.4.6-cp314-cp314t-win_amd64.whl", hash = "sha256:81a1cca95ed5bb92aa8b10dd2cdc9a0d3853a50fad926c28b5d7e8ea54389627", size = 12638908, upload-time = "2026-05-18T23:36:43.996Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/43/bb/e1c71a4295b1b1d1393d50dbb4f2a36283c6859d9d3892e84f00ec5a91d5/numpy-2.4.6-cp314-cp314t-win_arm64.whl", hash = "sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66", size = 10565867, upload-time = "2026-05-18T23:36:47.114Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "packaging"
|
||||
version = "26.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", size = 228134, upload-time = "2026-04-24T20:15:23.917Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pillow"
|
||||
version = "12.2.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/8c/21/c2bcdd5906101a30244eaffc1b6e6ce71a31bd0742a01eb89e660ebfac2d/pillow-12.2.0.tar.gz", hash = "sha256:a830b1a40919539d07806aa58e1b114df53ddd43213d9c8b75847eee6c0182b5", size = 46987819, upload-time = "2026-04-01T14:46:17.687Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/bf/98/4595daa2365416a86cb0d495248a393dfc84e96d62ad080c8546256cb9c0/pillow-12.2.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:3adc9215e8be0448ed6e814966ecf3d9952f0ea40eb14e89a102b87f450660d8", size = 4100848, upload-time = "2026-04-01T14:44:48.48Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0b/79/40184d464cf89f6663e18dfcf7ca21aae2491fff1a16127681bf1fa9b8cf/pillow-12.2.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:6a9adfc6d24b10f89588096364cc726174118c62130c817c2837c60cf08a392b", size = 4176515, upload-time = "2026-04-01T14:44:51.353Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b0/63/703f86fd4c422a9cf722833670f4f71418fb116b2853ff7da722ea43f184/pillow-12.2.0-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:6a6e67ea2e6feda684ed370f9a1c52e7a243631c025ba42149a2cc5934dec295", size = 3640159, upload-time = "2026-04-01T14:44:53.588Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/71/e0/fb22f797187d0be2270f83500aab851536101b254bfa1eae10795709d283/pillow-12.2.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:2bb4a8d594eacdfc59d9e5ad972aa8afdd48d584ffd5f13a937a664c3e7db0ed", size = 5312185, upload-time = "2026-04-01T14:44:56.039Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ba/8c/1a9e46228571de18f8e28f16fabdfc20212a5d019f3e3303452b3f0a580d/pillow-12.2.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:80b2da48193b2f33ed0c32c38140f9d3186583ce7d516526d462645fd98660ae", size = 4695386, upload-time = "2026-04-01T14:44:58.663Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/70/62/98f6b7f0c88b9addd0e87c217ded307b36be024d4ff8869a812b241d1345/pillow-12.2.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:22db17c68434de69d8ecfc2fe821569195c0c373b25cccb9cbdacf2c6e53c601", size = 6280384, upload-time = "2026-04-01T14:45:01.5Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5e/03/688747d2e91cfbe0e64f316cd2e8005698f76ada3130d0194664174fa5de/pillow-12.2.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:7b14cc0106cd9aecda615dd6903840a058b4700fcb817687d0ee4fc8b6e389be", size = 8091599, upload-time = "2026-04-01T14:45:04.5Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f6/35/577e22b936fcdd66537329b33af0b4ccfefaeabd8aec04b266528cddb33c/pillow-12.2.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8cbeb542b2ebc6fcdacabf8aca8c1a97c9b3ad3927d46b8723f9d4f033288a0f", size = 6396021, upload-time = "2026-04-01T14:45:07.117Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/11/8d/d2532ad2a603ca2b93ad9f5135732124e57811d0168155852f37fbce2458/pillow-12.2.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4bfd07bc812fbd20395212969e41931001fd59eb55a60658b0e5710872e95286", size = 7083360, upload-time = "2026-04-01T14:45:09.763Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5e/26/d325f9f56c7e039034897e7380e9cc202b1e368bfd04d4cbe6a441f02885/pillow-12.2.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:9aba9a17b623ef750a4d11b742cbafffeb48a869821252b30ee21b5e91392c50", size = 6507628, upload-time = "2026-04-01T14:45:12.378Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5f/f7/769d5632ffb0988f1c5e7660b3e731e30f7f8ec4318e94d0a5d674eb65a4/pillow-12.2.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:deede7c263feb25dba4e82ea23058a235dcc2fe1f6021025dc71f2b618e26104", size = 7209321, upload-time = "2026-04-01T14:45:15.122Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6a/7a/c253e3c645cd47f1aceea6a8bacdba9991bf45bb7dfe927f7c893e89c93c/pillow-12.2.0-cp314-cp314-win32.whl", hash = "sha256:632ff19b2778e43162304d50da0181ce24ac5bb8180122cbe1bf4673428328c7", size = 6479723, upload-time = "2026-04-01T14:45:17.797Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cd/8b/601e6566b957ca50e28725cb6c355c59c2c8609751efbecd980db44e0349/pillow-12.2.0-cp314-cp314-win_amd64.whl", hash = "sha256:4e6c62e9d237e9b65fac06857d511e90d8461a32adcc1b9065ea0c0fa3a28150", size = 7217400, upload-time = "2026-04-01T14:45:20.529Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d6/94/220e46c73065c3e2951bb91c11a1fb636c8c9ad427ac3ce7d7f3359b9b2f/pillow-12.2.0-cp314-cp314-win_arm64.whl", hash = "sha256:b1c1fbd8a5a1af3412a0810d060a78b5136ec0836c8a4ef9aa11807f2a22f4e1", size = 2554835, upload-time = "2026-04-01T14:45:23.162Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b6/ab/1b426a3974cb0e7da5c29ccff4807871d48110933a57207b5a676cccc155/pillow-12.2.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:57850958fe9c751670e49b2cecf6294acc99e562531f4bd317fa5ddee2068463", size = 5314225, upload-time = "2026-04-01T14:45:25.637Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/19/1e/dce46f371be2438eecfee2a1960ee2a243bbe5e961890146d2dee1ff0f12/pillow-12.2.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:d5d38f1411c0ed9f97bcb49b7bd59b6b7c314e0e27420e34d99d844b9ce3b6f3", size = 4698541, upload-time = "2026-04-01T14:45:28.355Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/55/c3/7fbecf70adb3a0c33b77a300dc52e424dc22ad8cdc06557a2e49523b703d/pillow-12.2.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:5c0a9f29ca8e79f09de89293f82fc9b0270bb4af1d58bc98f540cc4aedf03166", size = 6322251, upload-time = "2026-04-01T14:45:30.924Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1c/3c/7fbc17cfb7e4fe0ef1642e0abc17fc6c94c9f7a16be41498e12e2ba60408/pillow-12.2.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1610dd6c61621ae1cf811bef44d77e149ce3f7b95afe66a4512f8c59f25d9ebe", size = 8127807, upload-time = "2026-04-01T14:45:33.908Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ff/c3/a8ae14d6defd2e448493ff512fae903b1e9bd40b72efb6ec55ce0048c8ce/pillow-12.2.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0a34329707af4f73cf1782a36cd2289c0368880654a2c11f027bcee9052d35dd", size = 6433935, upload-time = "2026-04-01T14:45:36.623Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6e/32/2880fb3a074847ac159d8f902cb43278a61e85f681661e7419e6596803ed/pillow-12.2.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8e9c4f5b3c546fa3458a29ab22646c1c6c787ea8f5ef51300e5a60300736905e", size = 7116720, upload-time = "2026-04-01T14:45:39.258Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/46/87/495cc9c30e0129501643f24d320076f4cc54f718341df18cc70ec94c44e1/pillow-12.2.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:fb043ee2f06b41473269765c2feae53fc2e2fbf96e5e22ca94fb5ad677856f06", size = 6540498, upload-time = "2026-04-01T14:45:41.879Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/18/53/773f5edca692009d883a72211b60fdaf8871cbef075eaa9d577f0a2f989e/pillow-12.2.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:f278f034eb75b4e8a13a54a876cc4a5ab39173d2cdd93a638e1b467fc545ac43", size = 7239413, upload-time = "2026-04-01T14:45:44.705Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c9/e4/4b64a97d71b2a83158134abbb2f5bd3f8a2ea691361282f010998f339ec7/pillow-12.2.0-cp314-cp314t-win32.whl", hash = "sha256:6bb77b2dcb06b20f9f4b4a8454caa581cd4dd0643a08bacf821216a16d9c8354", size = 6482084, upload-time = "2026-04-01T14:45:47.568Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ba/13/306d275efd3a3453f72114b7431c877d10b1154014c1ebbedd067770d629/pillow-12.2.0-cp314-cp314t-win_amd64.whl", hash = "sha256:6562ace0d3fb5f20ed7290f1f929cae41b25ae29528f2af1722966a0a02e2aa1", size = 7225152, upload-time = "2026-04-01T14:45:50.032Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ff/6e/cf826fae916b8658848d7b9f38d88da6396895c676e8086fc0988073aaf8/pillow-12.2.0-cp314-cp314t-win_arm64.whl", hash = "sha256:aa88ccfe4e32d362816319ed727a004423aab09c5cea43c01a4b435643fa34eb", size = 2556579, upload-time = "2026-04-01T14:45:52.529Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pluggy"
|
||||
version = "1.6.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "propcache"
|
||||
version = "0.5.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/ec/44/c87281c333769159c50594f22610f77398a47ccbfbbf23074e744e86f87c/propcache-0.5.2.tar.gz", hash = "sha256:01c4fc7480cd0598bb4b57022df55b9ca296da7fc5a8760bd8451a7e63a7d427", size = 50208, upload-time = "2026-05-08T21:02:12.199Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/e2/ea/23ee535d90ce8bcc465a3028eb3cc0ce3bd1005f4bb27710b30587de798d/propcache-0.5.2-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:46088abff4cba581dea21ae0467a480526cb25aa5f3c269e909f800328bc3999", size = 94662, upload-time = "2026-05-08T21:01:22.683Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b5/06/c5a52f419b5d8972f8d46a7577476090d8e3263ff589ce40b5ca4968d5be/propcache-0.5.2-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:fc88b26f08d634f7bc819a7852e5214f5802641ab8d9fd5326892292eee1993e", size = 53928, upload-time = "2026-05-08T21:01:23.986Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/63/b1/4260d67d6bd85e58a66b72d54ce15d5de789b6f3870cc6bedf8ff9667401/propcache-0.5.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:97797ebb098e670a2f92dd66f32897e30d7615b14e7f59711de23e30a9072539", size = 54650, upload-time = "2026-05-08T21:01:25.305Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/70/06/2f46c318e3307cd7a6a7481def374ce838c0fe20084b39dd54b0879d0e99/propcache-0.5.2-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ba57fffe4ac99c5d30076161b5866336d97600769bad35cc68f7774b15298a4e", size = 59912, upload-time = "2026-05-08T21:01:26.545Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4c/29/fe1aebec2ce57ab985a9c382bded1124431f85078113aa222c5d278430d4/propcache-0.5.2-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:583c19759d9eec1e5b69e2fbef36a7d9c326041be9746cb822d335c8cedc2979", size = 63300, upload-time = "2026-05-08T21:01:27.937Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b4/18/2334b26768b6c82be8c69e83671b767d5ef426aa09b0cba6c2ea47816774/propcache-0.5.2-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d0326e2e5e1f3163fa306c834e48e8d490e5fae607a097a40c0648109b47ba80", size = 64208, upload-time = "2026-05-08T21:01:29.484Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2b/76/7f1bfd6afff4c5e38e36a3c6d68eb5f4b7311ea80baf693db78d95b603c4/propcache-0.5.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e00820e192c8dbebcafb383ebbf99030895f09905e7a0eb2e0340a0bcc2bc825", size = 61633, upload-time = "2026-05-08T21:01:31.068Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c4/46/b3ff8aba2b4953a3e50de2cf72f1b5748b8eca93b15f3dc2c84339084c09/propcache-0.5.2-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c66afea89b1e43725731d2004732a046fe6fe955d51f952c3e95a7314a284a39", size = 61724, upload-time = "2026-05-08T21:01:32.374Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c5/01/814cfcafbcff954f94c01cf30e097ddc88a076b5440fbcf4570753437d40/propcache-0.5.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:d4dc37dec6c6cdad0b57881a5658fd14fbf53e333b1a86cf86559f190e1d9ec4", size = 60069, upload-time = "2026-05-08T21:01:33.67Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/da/68/5c6f7622d510cc666a300687e06fd060c1a43361c0c9b20d284f06d8096a/propcache-0.5.2-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:5570dbcc97571c15f68068e529c92715a12f8d54030e272d264b377e22bd17a5", size = 57099, upload-time = "2026-05-08T21:01:34.915Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/55/27/9cb0b4c679124085327957d42521c99dba04c88c90c3e55a6f0b633ebccc/propcache-0.5.2-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:f814362777a9f841adddb200ecdf8f5cb1e5a3c4b7a86378edbd6ccb26edd702", size = 63391, upload-time = "2026-05-08T21:01:36.231Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f0/9d/7258aaa5bdf60fc6f27591eef6fe52768cb0beda7140be477c8b12c9794a/propcache-0.5.2-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:196913dea116aeb5a2ba95af4ddcb7ea85559ae07d8eee8751688310d09168c3", size = 61626, upload-time = "2026-05-08T21:01:37.545Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8e/0d/41c602003e8a9b16fe1e7eadf62c7bfba9d5474370b24200bf48b315f45f/propcache-0.5.2-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:6e7b8719005dd1175be4ab1cd25e9b98659a5e0347331506ec6760d2773a7fb5", size = 64781, upload-time = "2026-05-08T21:01:38.83Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8b/f3/38e66b1856e9bd079deea015bc4a55f7767c0e4db2f7dcf69e7e680ba4ce/propcache-0.5.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:51f96d685ab16e88cab128cd37a52c5da540809c8b879fa047731bfcb4ad35a4", size = 62570, upload-time = "2026-05-08T21:01:40.415Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/95/ca/bbfe9b910ce57dde8bb4876b4520fc02a4e89497c10de26be936758a3aaa/propcache-0.5.2-cp314-cp314-win32.whl", hash = "sha256:cc6fc3cc62e8501d3ed62894425040d2728ecddb1ed072737a5c70bd537aa9f0", size = 39436, upload-time = "2026-05-08T21:01:41.654Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/61/d2/45c9defbaa1ea297035d9d4cce9e8f80daafbf19319c6007f157c6256ea9/propcache-0.5.2-cp314-cp314-win_amd64.whl", hash = "sha256:81e3a30b0bb60caa22033dd0f8a3618d1d67356212514f62c57db75cb0ef410c", size = 42373, upload-time = "2026-05-08T21:01:43.041Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/44/68/9ea5103f41d5217d7d6ec24db90018e23aebec070c3f9a6e54d12b841fd8/propcache-0.5.2-cp314-cp314-win_arm64.whl", hash = "sha256:0d2c9bf8528f135dbb805ce027567e09164f7efa51a2be07458a2c0420f292d0", size = 38554, upload-time = "2026-05-08T21:01:44.336Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8a/81/fadf555f42d3b762eea8a53950b0489fdc0aa9da5f8ed9e10ce0a4e01b48/propcache-0.5.2-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:4bc8ff1feffc6a61c7002ffe84634c41b822e104990ae009f44a0834430070bb", size = 99395, upload-time = "2026-05-08T21:01:45.883Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f5/c9/c61e134a686949cf7971af3a390148b1156f7be81c73bc0cd12c873e2d48/propcache-0.5.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:79aa3ff0a9b566633b642fa9caf7e21ed1c13d6feca718187873f199e1514078", size = 56653, upload-time = "2026-05-08T21:01:47.307Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cb/73/daf935ea7048ddd7ec8eec5345b4a40b619d2d178b3c0a0900796bc3c794/propcache-0.5.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1b31822f4474c4036bae62de9402710051d431a606d6a0f907fec79935a071aa", size = 56914, upload-time = "2026-05-08T21:01:48.573Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/79/9f/aba959b435ea18617edd7cf0a7ad0b9c574b8fc7e3d2cd55fb59cb255d33/propcache-0.5.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:13fef48778b5a2a756523fdb781326b028ca75e32858b04f2cdd19f394564917", size = 62567, upload-time = "2026-05-08T21:01:49.903Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6c/a1/859942de9a791ff42f6141736f5b37749b8f53e65edfa49638c67dd67e6a/propcache-0.5.2-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:8b73ab70f1a3351fbc71f663b3e645af6dd0329100c353081cf69c37433fc6fe", size = 65542, upload-time = "2026-05-08T21:01:51.204Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b5/61/315bc0fd6c0fc7f80a528b8afd209e5fc4a875ea79571b91b8f50f442907/propcache-0.5.2-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5538d2c13d93e4698af7e092b57bc7298fd35d1d58e656ae18f23ee0d0378e03", size = 66845, upload-time = "2026-05-08T21:01:52.539Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/47/f7/9f8122e3132e8e354ac41975ef8f1099be7d5a16bc7ae562734e993665c0/propcache-0.5.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cd645f03898405cabe694fb8bc35241e3a9c332ec85627584fe3de201452b335", size = 63985, upload-time = "2026-05-08T21:01:53.847Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c8/54/c317819ec157cbf6f35df9df9657a6f82daf34d5faf15948b2f639c2192e/propcache-0.5.2-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a473b3440261e0c60706e732b2ed2f517857344fc21bf48fdfe211e2d98eb285", size = 63999, upload-time = "2026-05-08T21:01:55.179Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5a/56/387e3f7dfce0a9233df41fb888aa1c30222cb4bbbf09537c02dd9bd85fe2/propcache-0.5.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:7afa37062e6650640e932e4cc9297d81f9f42d9944029cc386b8247dea4da837", size = 62779, upload-time = "2026-05-08T21:01:57.489Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a1/9c/596784cb5824ed61ee960d3f8655a3f0993e107c6e98ab6c818b7fb92ccb/propcache-0.5.2-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:8a90efd5777e996e42d568db9ac740b944d691e565cbfd31b2f7832f9184b2b8", size = 59796, upload-time = "2026-05-08T21:01:58.736Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c2/3d/1a6cfa1726a48542c1e8784a0761421476a5b68e09b7f36bf95eb954aaba/propcache-0.5.2-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:f19bb891234d72535764d703bfed1153cc34f4214d5bd7150aee1eec9e8f4366", size = 66023, upload-time = "2026-05-08T21:02:00.228Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e4/0e/05fd6990369477076e4e280bcb970de760fddf0161a46e988bc95f7940ec/propcache-0.5.2-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:32775082acd2d807ee3db715c7770d38767b817870acfa08c29e057f3c4d5b56", size = 64448, upload-time = "2026-05-08T21:02:01.888Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cd/86/5f8da315a4309c62c10c0b2516b17492d5d3bbe1bb862b96604db67e2a37/propcache-0.5.2-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:9282fb1a3bccd038da9f768b927b24a0c753e466c086b7c4f3c6982851eefb2d", size = 67329, upload-time = "2026-05-08T21:02:03.484Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/da/d3/3368efe79ab21f0cdf86ef49895811c9cc933131d4cde1f28a624e22e712/propcache-0.5.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:cc49723e2f60d6b32a0f0b08a3fd6d13203c07f1cd9566cfce0f12a917c967a2", size = 65172, upload-time = "2026-05-08T21:02:04.745Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d5/07/127e8b0bacfb325396196f9d976a22453049b89b9b2b08477cc3145faa44/propcache-0.5.2-cp314-cp314t-win32.whl", hash = "sha256:2d7aa89ebca5acc98cba9d1472d976e394782f587bad6661003602a619fd1821", size = 43813, upload-time = "2026-05-08T21:02:06.025Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/88/fb/46dad6c0ae49ed230ab1b16c890c2b6314e2403e6c412976f4a72d64a527/propcache-0.5.2-cp314-cp314t-win_amd64.whl", hash = "sha256:d447bb0b3054be5818458fbb171208b1d9ff11eba14e18ca18b90cbb45767370", size = 47764, upload-time = "2026-05-08T21:02:07.353Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e7/c4/a47d0a63aa309d10d59ede6e9d4cff03a344a79d1f0f4cd0cd74997b53e0/propcache-0.5.2-cp314-cp314t-win_arm64.whl", hash = "sha256:fe67a3d11cd9b4efabfa45c3d00ffba2b26811442a73a581a94b67c2b5faccf6", size = 41140, upload-time = "2026-05-08T21:02:09.065Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3a/ed/1cdcab6ba3d6ab7feca11fc14f0eeea80755bb53ef4e892079f31b10a25f/propcache-0.5.2-py3-none-any.whl", hash = "sha256:be1ddfcbb376e3de5d2e2db1d58d6d67463e6b4f9f040c000de8e300295465fe", size = 14036, upload-time = "2026-05-08T21:02:10.673Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "psutil"
|
||||
version = "7.2.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/aa/c6/d1ddf4abb55e93cebc4f2ed8b5d6dbad109ecb8d63748dd2b20ab5e57ebe/psutil-7.2.2.tar.gz", hash = "sha256:0746f5f8d406af344fd547f1c8daa5f5c33dbc293bb8d6a16d80b4bb88f59372", size = 493740, upload-time = "2026-01-28T18:14:54.428Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/81/69/ef179ab5ca24f32acc1dac0c247fd6a13b501fd5534dbae0e05a1c48b66d/psutil-7.2.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:eed63d3b4d62449571547b60578c5b2c4bcccc5387148db46e0c2313dad0ee00", size = 130664, upload-time = "2026-01-28T18:15:09.469Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7b/64/665248b557a236d3fa9efc378d60d95ef56dd0a490c2cd37dafc7660d4a9/psutil-7.2.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7b6d09433a10592ce39b13d7be5a54fbac1d1228ed29abc880fb23df7cb694c9", size = 131087, upload-time = "2026-01-28T18:15:11.724Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d5/2e/e6782744700d6759ebce3043dcfa661fb61e2fb752b91cdeae9af12c2178/psutil-7.2.2-cp314-cp314t-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1fa4ecf83bcdf6e6c8f4449aff98eefb5d0604bf88cb883d7da3d8d2d909546a", size = 182383, upload-time = "2026-01-28T18:15:13.445Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/57/49/0a41cefd10cb7505cdc04dab3eacf24c0c2cb158a998b8c7b1d27ee2c1f5/psutil-7.2.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e452c464a02e7dc7822a05d25db4cde564444a67e58539a00f929c51eddda0cf", size = 185210, upload-time = "2026-01-28T18:15:16.002Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/dd/2c/ff9bfb544f283ba5f83ba725a3c5fec6d6b10b8f27ac1dc641c473dc390d/psutil-7.2.2-cp314-cp314t-win_amd64.whl", hash = "sha256:c7663d4e37f13e884d13994247449e9f8f574bc4655d509c3b95e9ec9e2b9dc1", size = 141228, upload-time = "2026-01-28T18:15:18.385Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f2/fc/f8d9c31db14fcec13748d373e668bc3bed94d9077dbc17fb0eebc073233c/psutil-7.2.2-cp314-cp314t-win_arm64.whl", hash = "sha256:11fe5a4f613759764e79c65cf11ebdf26e33d6dd34336f8a337aa2996d71c841", size = 136284, upload-time = "2026-01-28T18:15:19.912Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e7/36/5ee6e05c9bd427237b11b3937ad82bb8ad2752d72c6969314590dd0c2f6e/psutil-7.2.2-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:ed0cace939114f62738d808fdcecd4c869222507e266e574799e9c0faa17d486", size = 129090, upload-time = "2026-01-28T18:15:22.168Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/80/c4/f5af4c1ca8c1eeb2e92ccca14ce8effdeec651d5ab6053c589b074eda6e1/psutil-7.2.2-cp36-abi3-macosx_11_0_arm64.whl", hash = "sha256:1a7b04c10f32cc88ab39cbf606e117fd74721c831c98a27dc04578deb0c16979", size = 129859, upload-time = "2026-01-28T18:15:23.795Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b5/70/5d8df3b09e25bce090399cf48e452d25c935ab72dad19406c77f4e828045/psutil-7.2.2-cp36-abi3-manylinux2010_x86_64.manylinux_2_12_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:076a2d2f923fd4821644f5ba89f059523da90dc9014e85f8e45a5774ca5bc6f9", size = 155560, upload-time = "2026-01-28T18:15:25.976Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/63/65/37648c0c158dc222aba51c089eb3bdfa238e621674dc42d48706e639204f/psutil-7.2.2-cp36-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b0726cecd84f9474419d67252add4ac0cd9811b04d61123054b9fb6f57df6e9e", size = 156997, upload-time = "2026-01-28T18:15:27.794Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8e/13/125093eadae863ce03c6ffdbae9929430d116a246ef69866dad94da3bfbc/psutil-7.2.2-cp36-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:fd04ef36b4a6d599bbdb225dd1d3f51e00105f6d48a28f006da7f9822f2606d8", size = 148972, upload-time = "2026-01-28T18:15:29.342Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/04/78/0acd37ca84ce3ddffaa92ef0f571e073faa6d8ff1f0559ab1272188ea2be/psutil-7.2.2-cp36-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:b58fabe35e80b264a4e3bb23e6b96f9e45a3df7fb7eed419ac0e5947c61e47cc", size = 148266, upload-time = "2026-01-28T18:15:31.597Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b4/90/e2159492b5426be0c1fef7acba807a03511f97c5f86b3caeda6ad92351a7/psutil-7.2.2-cp37-abi3-win_amd64.whl", hash = "sha256:eb7e81434c8d223ec4a219b5fc1c47d0417b12be7ea866e24fb5ad6e84b3d988", size = 137737, upload-time = "2026-01-28T18:15:33.849Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8c/c7/7bb2e321574b10df20cbde462a94e2b71d05f9bbda251ef27d104668306a/psutil-7.2.2-cp37-abi3-win_arm64.whl", hash = "sha256:8c233660f575a5a89e6d4cb65d9f938126312bca76d8fe087b947b3a1aaac9ee", size = 134617, upload-time = "2026-01-28T18:15:36.514Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pygments"
|
||||
version = "2.20.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pyrekordbox"
|
||||
version = "0.4.4"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "bidict" },
|
||||
{ name = "construct" },
|
||||
{ name = "numpy" },
|
||||
{ name = "packaging" },
|
||||
{ name = "psutil" },
|
||||
{ name = "python-dateutil" },
|
||||
{ name = "sqlalchemy" },
|
||||
{ name = "sqlcipher3-wheels" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/39/8b/8fb7f21f4eb676df35c53214129ad33624634e2b678069abbf6a88509c6b/pyrekordbox-0.4.4.tar.gz", hash = "sha256:8bdb59f7c45c743bd2021613a72604b7607092d386e25f20cce78adfe30bf9a4", size = 2310256, upload-time = "2025-08-17T20:09:18.773Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b1/a8/28d28f02683bee40e3cfa32caed7c46e47daffb51482798fea9af0ab23cb/pyrekordbox-0.4.4-py3-none-any.whl", hash = "sha256:2de933870126b9a0ef04bea372f1c5934efcfbb2501efb6b47d8ace0e17fb3d8", size = 76000, upload-time = "2025-08-17T20:09:16.985Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pytest"
|
||||
version = "9.0.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "colorama", marker = "sys_platform == 'win32'" },
|
||||
{ name = "iniconfig" },
|
||||
{ name = "packaging" },
|
||||
{ name = "pluggy" },
|
||||
{ name = "pygments" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/7d/0d/549bd94f1a0a402dc8cf64563a117c0f3765662e2e668477624baeec44d5/pytest-9.0.3.tar.gz", hash = "sha256:b86ada508af81d19edeb213c681b1d48246c1a91d304c6c81a427674c17eb91c", size = 1572165, upload-time = "2026-04-07T17:16:18.027Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d4/24/a372aaf5c9b7208e7112038812994107bc65a84cd00e0354a88c2c77a617/pytest-9.0.3-py3-none-any.whl", hash = "sha256:2c5efc453d45394fdd706ade797c0a81091eccd1d6e4bccfcd476e2b8e0ab5d9", size = 375249, upload-time = "2026-04-07T17:16:16.13Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "python-dateutil"
|
||||
version = "2.9.0.post0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "six" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", size = 342432, upload-time = "2024-03-01T18:36:20.211Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", size = 229892, upload-time = "2024-03-01T18:36:18.57Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "scipy"
|
||||
version = "1.18.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "numpy" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/a7/25/c2700dfaf6442b4effaa91af24ebce5dc9d31bb4a69706313aae70d72cd0/scipy-1.18.0.tar.gz", hash = "sha256:67b2ad2ad54c72ca6d04975a9b2df8c3638c34ddd5b28738e94fc2b57929d378", size = 30774447, upload-time = "2026-06-19T15:01:43.456Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/78/b5/915a19b3de2f7430062b509653563db1633ddbb6f021b06731521115d4e2/scipy-1.18.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:4c256ee70c0d1a8a2ace807e199ccd4e3f57037433842abb3fb36bc17eaa9578", size = 31036253, upload-time = "2026-06-19T15:00:43.216Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d7/88/b72def7262e150d16be13fca37a96481138d624e700340bc3362a7588929/scipy-1.18.0-cp314-cp314-macosx_12_0_arm64.whl", hash = "sha256:2ef3abc54a4ffc53765374b0d5728532dfdd2585ed23f6b11c206a1f0b1b9af8", size = 28673758, upload-time = "2026-06-19T15:00:46.663Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/91/02/2e636a61a525632c373cf6a9c24442a3ffb79e364d38e98b32042964ac32/scipy-1.18.0-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:f2a6af57bd9e4a75d70e4117e78a1bbee84f79ae3fbb6d0111005d6ebcc4cb8d", size = 20415514, upload-time = "2026-06-19T15:00:49.399Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c9/b6/2135974442f6aba159d9d39d774a1c8cb19947016725d69fecc685df45bf/scipy-1.18.0-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:3f1ac564d3bf6c03d861d2cd87a1bea0da2887136f7fb1bf519c05a8971452d6", size = 23034398, upload-time = "2026-06-19T15:00:51.941Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f6/e6/ba89ec5abf6ee9257c0d1ec985573f3ae32742c24bc03e016388a40b1b15/scipy-1.18.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:40395a5fcd1abee49a5c7aaa98c29db393eedc835138560a588c47ec16156690", size = 33998032, upload-time = "2026-06-19T15:00:54.838Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7f/c4/bc41eb19b0fd0db868f4132920879019318d80cc522ad8f2bca4611af808/scipy-1.18.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8ca01e8ae69f1b18e9a58d91afead31be3cef0dd905a10249dac559ee15460a0", size = 35283333, upload-time = "2026-06-19T15:00:58.152Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/53/a4/cbdeef6eb3830a8462a9d4ada814de5fc984345cc9ecf17cbec51a036f1e/scipy-1.18.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7a7f3b01647384dbc3a711e8c6778e0aabbe93959249fef5c7393396bcac0867", size = 35610216, upload-time = "2026-06-19T15:01:01.155Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/80/4d/b2b82502b65f661d1b789c1665dcdf315d5f12194e06fc0b37946294ebae/scipy-1.18.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6aa94e78ec192a30063a5e72e561c28af769dc311190b24fe91774eff1969709", size = 37418960, upload-time = "2026-06-19T15:01:04.155Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/93/3e/902d836831474b0ab5a37d16404f7bc5fafd9efba632890e271ba952635f/scipy-1.18.0-cp314-cp314-win_amd64.whl", hash = "sha256:2d8bbdc6c817f5b4006a54d799d4f5bab6f910193cbb9a1ff310833d4d270f61", size = 37288845, upload-time = "2026-06-19T15:01:07.822Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b6/43/8d73b337a3bdb14daa0314f0434210747c02d79d729ce1777574a817dcf6/scipy-1.18.0-cp314-cp314-win_arm64.whl", hash = "sha256:18e9575f1569b2c54174e6159d32942e03731177f63dce7975f0a0c88d102f5b", size = 24988971, upload-time = "2026-06-19T15:01:11.076Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b4/b4/f11918b0508a2787031a0499a03fbe3546f3bb5ca05d01038c45b278c09a/scipy-1.18.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:f351e0dd702687d12a402b867a1b4146a256923e1c38317cbc472f6372b94707", size = 31399325, upload-time = "2026-06-19T15:01:13.723Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7b/d1/1f287b57c0ff0ee5185dff3946d92c8017d39b0e431f0ae79a3ff1859512/scipy-1.18.0-cp314-cp314t-macosx_12_0_arm64.whl", hash = "sha256:7c7a51b33ce387193c97f228320cf8e87361daa1bba750638677729598b3e677", size = 29092110, upload-time = "2026-06-19T15:01:16.908Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ff/1a/7b74eb6c392fdcb27d414c0e7558a6d0231eb3b6d73571f479bb81ea8794/scipy-1.18.0-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:84031d7b052a54fae2f8632e0ec802073d385476eb9a63079bce6e23ef9283d4", size = 20833811, upload-time = "2026-06-19T15:01:20.488Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7c/ad/f3941716320a7b9cb4d68734a903b45fe16eff5fb7da7e16f2e619304979/scipy-1.18.0-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:56abf29a7c067dde59be8b9a22d606a4ea1b2f2a4b756d9d903c62818f5dacce", size = 23396644, upload-time = "2026-06-19T15:01:23.364Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/22/22/1446b62ffe07f9719b7d9b1b6a4e05a772833ae8f441fe4c22c34c9b250f/scipy-1.18.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1ad44305cfa24b1ba5803cbbebf033590ccbac1aa5d612d727b785325ab408b0", size = 34079318, upload-time = "2026-06-19T15:01:26.002Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/56/3b/b87da667098bb470fa30c7011b0ba351ee976dd395c78798c66e941665a3/scipy-1.18.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:945c1761b93f38d7f99ae81ae80c63e621471608c7eeead563f6df025585cd58", size = 35324320, upload-time = "2026-06-19T15:01:28.881Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f8/a1/c7932f91909759b0267f75fdea34e91309f96b895757534b76a90b6b4344/scipy-1.18.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:1a4441f15d620578772a49e5ab48c0ee1f7a0220e387110283062729136b2553", size = 35699541, upload-time = "2026-06-19T15:01:31.968Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f7/86/5185061a1fcc41d18c5dc2463969b3a3964b31d9ac67b2fb05d4c7ff7670/scipy-1.18.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:9aac6192fac56bf2ca534389d24623f07b39ff83317d58287285e7fbd622ff76", size = 37472480, upload-time = "2026-06-19T15:01:35.136Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/31/8e/f04c68e39919a010d34f2ee1367fd705b0a25a02f609d755f0bfbc0a15fc/scipy-1.18.0-cp314-cp314t-win_amd64.whl", hash = "sha256:e40baea28ae7f5475c779741e2d90b1247c78531207b49c7030e698ff81cee3f", size = 37365390, upload-time = "2026-06-19T15:01:38.091Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d5/19/969dc072906c84dd0a3b05dcf57ea750936087d7873549e408b35cfc3f97/scipy-1.18.0-cp314-cp314t-win_arm64.whl", hash = "sha256:368e0a705903c466aa5f08eefb39e6b1b6b2d659e7352a31fd9e2438365be0f8", size = 25279661, upload-time = "2026-06-19T15:01:40.817Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "six"
|
||||
version = "1.17.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", size = 34031, upload-time = "2024-12-04T17:35:28.174Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050, upload-time = "2024-12-04T17:35:26.475Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sqlalchemy"
|
||||
version = "2.0.50"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "greenlet", marker = "platform_machine == 'AMD64' or platform_machine == 'WIN32' or platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'ppc64le' or platform_machine == 'win32' or platform_machine == 'x86_64'" },
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/57/da/6fbf010c8ebb347679d0d100b22fe9ba5e13fd04046c5df7280d2f0bf706/sqlalchemy-2.0.50.tar.gz", hash = "sha256:af5607d11ef90fd6a5c0549fe0045dce1663d427426bcfb506dcb5346a85a3b9", size = 9907424, upload-time = "2026-05-24T19:20:04.018Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/df/32/10ac51b4be7cdecd7e93d069251c86dfbf70b7adbd7c67b48ccea6c49e1c/sqlalchemy-2.0.50-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c966932507a4d7d0a37314927dbfcd89720e3f37d2a1e3352e7ae7939fa8e8a0", size = 2158519, upload-time = "2026-05-24T19:27:56.472Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5a/76/e703d2f7681d7d66c4c891af3f07c7ccf4c76ad7f18351de035b5eda007a/sqlalchemy-2.0.50-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:faffef4bcc20a1892e65e155293d99d60855bbbc79250ab712819cfd56a8e6bb", size = 3282063, upload-time = "2026-05-24T20:09:38.57Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/31/26/ef168b184a25701f9995e8fb7e503fafd7a99c1c77cda1bc1a26ea2ed486/sqlalchemy-2.0.50-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6c206aec519a2e7bd08abbfb33436e325fd22c632d9c21a9047e376ce241646e", size = 3287069, upload-time = "2026-05-24T20:17:21.942Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c2/15/765acc2bc693bccc43ca4a95d5b69750da8aaf6db1b5c616536e087f8920/sqlalchemy-2.0.50-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:bef4ac756363227ef6402a75fee025a4bc690f92328e825868939b3b3a446a6d", size = 3230453, upload-time = "2026-05-24T20:09:40.398Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/63/61/08e03c3adbf5db0087a0b6816746fec8f3032fb2f7fc899a9bb9b2a48ce4/sqlalchemy-2.0.50-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:96fbee6b19c19cd1556c8bf9419447cf2ec149ffcab7ab64348c23e54ef8547f", size = 3252413, upload-time = "2026-05-24T20:17:24.067Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/03/0c/370a1f2db38436c615e10134c8a37de3688e74084792380695f3f5083860/sqlalchemy-2.0.50-cp314-cp314-win32.whl", hash = "sha256:8f00e3eb43ba30eb1b238ee03a8a62309486d1321eda3328bb611e0340033ad8", size = 2120063, upload-time = "2026-05-24T19:50:11.08Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7f/a0/fe92bb9817863bc13ba093bda931979a26cc2ca69f8e8f26d07add3d7c6f/sqlalchemy-2.0.50-cp314-cp314-win_amd64.whl", hash = "sha256:15708c613cd5005b7dffe1f66ee6a63ee8f5e46799f71c70ebad74178c676a39", size = 2145830, upload-time = "2026-05-24T19:50:12.452Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cc/ff/e5640a98a0b2f491eb8fde10fb6c773621a2e44340de231fafcc9370f4a9/sqlalchemy-2.0.50-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:3699dac4be410e97049a1658e9480da9cde956594aa0f3aebc60b88f21c5ba70", size = 2178435, upload-time = "2026-05-24T19:42:58.889Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b7/85/337116e186f1236375b5fb70c21cfac98e8e8ab0d3a47be838dc47a59e08/sqlalchemy-2.0.50-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f96233858e3df43932ac11589e22520da6e8aeb624b03fedfeebb0e8ea213086", size = 3566059, upload-time = "2026-05-24T20:01:20.848Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/96/34/bb0e190e161c3c2c24314a65add57218be14a4a9486886b7f5047c1ff7c8/sqlalchemy-2.0.50-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c4e70c46fad30c3bcc6a4708bc0130a3173e11a5b25f0ea4a9d8911b450f1f52", size = 3535366, upload-time = "2026-05-24T20:03:56.768Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/df/5a/a7f759f97e4fd499c5d4e4488c760d5a7fbecf3028b465a04274fcd52384/sqlalchemy-2.0.50-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:1918a3cf564d16d95bca7301005f41ab2ad50b07cd3b9da50d3ed986db148d6a", size = 3474879, upload-time = "2026-05-24T20:01:23.058Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9d/d9/2907ea38eb60687d297bf9c39e5ee58053c87b57fe8a9cae97090cecbf10/sqlalchemy-2.0.50-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:b00098cdbdbd38c7be3d568b0c9c3122b8c0ec62b911b57cd5e6e0254d60a76d", size = 3486117, upload-time = "2026-05-24T20:03:59.052Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f2/e3/5aa06f167559f8c0bdae487e297d23ba548150ab016a3418265d617a4985/sqlalchemy-2.0.50-cp314-cp314t-win32.whl", hash = "sha256:1fbd55a969d7ac44a98e3dec75016074f809fa08f871585ace58dde110d1bf3e", size = 2150823, upload-time = "2026-05-24T20:08:58.644Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/65/9b/112fb8f977582d7489d036e409e3723948bcf5320b3ac465f3c481bbe8f9/sqlalchemy-2.0.50-cp314-cp314t-win_amd64.whl", hash = "sha256:c5c3cdb753a9004183e1ccb634b41611654c989e61bc68617ce878e46d6f1e51", size = 2185794, upload-time = "2026-05-24T20:09:00.319Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d0/10/f7220e9b784d295d241c86ed99aeb537f92afcd469a64861f2717e9bb077/sqlalchemy-2.0.50-py3-none-any.whl", hash = "sha256:92064363517a3ff8212b5a93b8c62876579d8dfd1ca5b561335f30152d884fa9", size = 1943861, upload-time = "2026-05-24T19:59:01.119Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sqlcipher3-wheels"
|
||||
version = "0.5.7"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/61/ec/d79c2e5f5b9246471f217cdd5f703dc8d804bce557a047e1bfad4f33c376/sqlcipher3_wheels-0.5.7.tar.gz", hash = "sha256:5fd344abba62c487c4d857a6fb55cb8dff8a97d4c4fbb4f9394b473e8e315dee", size = 2669821, upload-time = "2026-01-07T09:53:25.48Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/1e/b5/8ca4fe7928d1e7151ecdf221ad757540e9a815ea10d854f047491859cd0c/sqlcipher3_wheels-0.5.7-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:6831fd121adf8374ecfb67b63d9bdfc3bc405e7103e35dd3152f7f2d2bbe87ae", size = 4946253, upload-time = "2026-01-07T09:51:46.013Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cd/39/5eca1cced6b3a78cb20201232103b58ded2d57e87c413212bfd2a30c7209/sqlcipher3_wheels-0.5.7-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:9d03491d9d4023cca969a4c87f01ea845368defcc52ea7cd71ce93deac3df286", size = 2898557, upload-time = "2026-01-07T09:51:47.261Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/34/93/8c34dd286ff8bea8785896eee401a5e15ab37901687ba09d2bf0df584712/sqlcipher3_wheels-0.5.7-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:1f3325fee4698fadbfa9b8245f213f7f3ecc3f0c86a761a2b26c787d54d3ecd1", size = 3163380, upload-time = "2026-01-07T09:51:48.384Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6d/2e/5b62f3c4181a64ab041589c6fdb509a70d69c2653f91d35b19c7550ad0d3/sqlcipher3_wheels-0.5.7-cp314-cp314-manylinux_2_28_aarch64.whl", hash = "sha256:f80b8c72df20f8e2696e3b23577145e94b3f5a10217fef02d5c78d0ef2e578ba", size = 7259742, upload-time = "2026-01-07T09:51:49.698Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/09/18/d78100264c66c96aec0723b2ddaf2b4edc72374e60c25b5ed8f3afb5e4f0/sqlcipher3_wheels-0.5.7-cp314-cp314-manylinux_2_28_i686.whl", hash = "sha256:9feb22952dd44a6e86bbb1e95a0a603bd7cc61332d0b1adad2484d2ccdc61704", size = 6875763, upload-time = "2026-01-07T09:51:51.009Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8b/e0/0529fc2d3cb0d58950557e2ac229d338506e9522a7d3eb03540acd4a6ee7/sqlcipher3_wheels-0.5.7-cp314-cp314-manylinux_2_28_ppc64le.whl", hash = "sha256:79fd014b64fefd85086cff6af85286cfb3b5761473f0a541f973a02a3c0e184b", size = 7413959, upload-time = "2026-01-07T09:51:52.344Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/06/81/290107b3bd5a84edcc7a542ce1e7fe42e401cd94f4a8b6ab22e546ba034d/sqlcipher3_wheels-0.5.7-cp314-cp314-manylinux_2_28_s390x.whl", hash = "sha256:92462368d7afc2c396466eeb4cfed85dccc12bef06a5f3a9a43a6598890ddd15", size = 7389084, upload-time = "2026-01-07T09:51:53.786Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/54/f1/9237d5d14807d1a6c7900330b55cad40e192f54dd4d5f732239a2271922d/sqlcipher3_wheels-0.5.7-cp314-cp314-manylinux_2_28_x86_64.whl", hash = "sha256:663f814ca785d064408ec96c7abdc6e940809efac34cdc1656fb3536cffb8ff5", size = 7039875, upload-time = "2026-01-07T09:51:55.236Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b9/46/ba150f6d1c38f0742055d5a693f5c3532a85c4aac3c8789d130ab8492dd9/sqlcipher3_wheels-0.5.7-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:47b5ae81361c0ca45fab27ce09f94ff795a7763767c9bcbbbce19923c82d2f65", size = 7252142, upload-time = "2026-01-07T09:51:56.581Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b0/d5/e493a4a19dedc2a9c25751bcc1a7eebc29a5cbc8859108b814ef20a8742e/sqlcipher3_wheels-0.5.7-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:869f77c3f8bfc94f05352b6ca42236adfb1b46f6c92082829a608cd561793a54", size = 6948368, upload-time = "2026-01-07T09:51:57.855Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6e/ac/e82f8e26d91859570c79e3c07d2de07c65001dde49b25cb60f058f694929/sqlcipher3_wheels-0.5.7-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:98f1b5f0f54316f105f553ec254961986c8dc617947f67daa11d49a19cbc9a58", size = 7385735, upload-time = "2026-01-07T09:51:59.874Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c2/2e/3152b89c2b5ce3f7c830b675451957349e4e0d2006d0d90354a50f5c8a01/sqlcipher3_wheels-0.5.7-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:e4ca2cd825be2c30e6530c20ba6112545aaa409ad77478fb66285f98f44aac78", size = 7027637, upload-time = "2026-01-07T09:52:03.298Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/76/8b/a1b3e8044128ab6a6343ef181a658298b598d0a5a2973d33e7f8fa5e45c0/sqlcipher3_wheels-0.5.7-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:a823f48efe2927a781d28ce31a11263be0e836016ea7ef26e53da18efba2569b", size = 7061293, upload-time = "2026-01-07T09:52:04.997Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cb/2f/350f8ab4e061c588ad90e12dec59f609a7e4b51c7dc921b9c62eee8c020c/sqlcipher3_wheels-0.5.7-cp314-cp314-win32.whl", hash = "sha256:0f3a11b144c46a33cb34235b253a46b1b255968be34430f9bd7f9ed710283d41", size = 2033959, upload-time = "2026-01-07T09:52:06.307Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a0/f4/ecb143774ccf0c6c8b7c718b97ac5a69ef59aa57d89c2422aa564d4a9b2f/sqlcipher3_wheels-0.5.7-cp314-cp314-win_amd64.whl", hash = "sha256:f210fdbbb794cc1b3739c7bff879b8bcc5949567f5c222e251882ec076e69b72", size = 2560254, upload-time = "2026-01-07T09:52:07.614Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2e/bc/82cd7b7ecc0ad647d9641c9115a875ea6fe8bbc04b83bfd583ebf954260f/sqlcipher3_wheels-0.5.7-cp314-cp314-win_arm64.whl", hash = "sha256:30f825607048436ab6134450ddcc49ad05f17479640a830c0ea8d78f1700e6e0", size = 2727306, upload-time = "2026-01-07T09:52:08.83Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8e/06/dcad55d438eeb9b9b46349434a75ee98f2c42196077a63d7e263969195eb/sqlcipher3_wheels-0.5.7-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:b822211ea0d50500bc8dc82863fa570543e0f224359ccb46b7675a24c89983f5", size = 4948699, upload-time = "2026-01-07T09:52:10.513Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/31/c2/66fcd2a7c3200323517e69c2ff49d48bb6aad15bc47d6f7d75710531a5c2/sqlcipher3_wheels-0.5.7-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:670f493bf5a9e70e86b0ae680a8bbf96aeaaf526c5018262186e6d7c52f021b1", size = 2900350, upload-time = "2026-01-07T09:52:11.766Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/7b/3a/94bebb8260624be45531af2692e9f5329ee1163237f027a3716ce3f2ae39/sqlcipher3_wheels-0.5.7-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:8c4a6dd38e9c4eff2304ea0751e8c5612394f277e63f007ae85118d9ce49b427", size = 3167268, upload-time = "2026-01-07T09:52:13.036Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a1/04/00d4d7e8905d9b53cc3cf490223b151df9a794e10138692fd4822962408b/sqlcipher3_wheels-0.5.7-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:7438567a502bfde64ca8732d1466ee6801deb97856315ac06d168a68456b2783", size = 7301506, upload-time = "2026-01-07T09:52:14.366Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/95/81/75e439c4f13cbf79b243de08394916450c812eb8e949b726a758089aaf3b/sqlcipher3_wheels-0.5.7-cp314-cp314t-manylinux_2_28_i686.whl", hash = "sha256:098112cdba65feefdbaba3eebbefe45b5481158882d21a7d133f475614a2fff0", size = 6915748, upload-time = "2026-01-07T09:52:16.166Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/48/3c/36e5aa4fbfe25884d7c930215cc2003b4ac7709866992086179982d04bc5/sqlcipher3_wheels-0.5.7-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:3f54e5284ebfeb2b3578ec35d3c4ad209ec1d4a3fd31b910608eadd95169cf83", size = 7453665, upload-time = "2026-01-07T09:52:18.415Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ec/62/9eddf7efdd523b3a109416f31185361608d1db60b550eb6e787a209765bc/sqlcipher3_wheels-0.5.7-cp314-cp314t-manylinux_2_28_s390x.whl", hash = "sha256:8d3bcfbb799abfa4e31e70e61c779e963b993748bf56bd0603007a47cb2a3c3f", size = 7431832, upload-time = "2026-01-07T09:52:22.041Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/61/86/3e1f57a9e65d42f67f6e2e07b6c1eee14ef12caa08eedcbeebdba4389ba4/sqlcipher3_wheels-0.5.7-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:97617b4c838ad82877a342df111aecbd8653dd6c4c7dc3dccd1ef6389247d614", size = 7077205, upload-time = "2026-01-07T09:52:23.85Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/98/0a/4c062770cb79860cf2cdced2b34887aa27e400fd261c553025b74ba0de74/sqlcipher3_wheels-0.5.7-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:b750c3d51134b82a4c82c9e5a6463de495ad75b7e073d03b1377083332b13ac8", size = 7296097, upload-time = "2026-01-07T09:52:25.622Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/58/c7/6a6cec55e666ddd998c2c15e37a40e4fcab3c0b67c9d03c3d12b5f30d56a/sqlcipher3_wheels-0.5.7-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:dbebc62280b9fd005511189d887c080c7595226bb7242c11f1298a0b25a4c123", size = 6988446, upload-time = "2026-01-07T09:52:26.983Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fd/f4/ce2198d1296f63d0917fde374a20c09c5c5e454d65681bf8c2ee4b59591b/sqlcipher3_wheels-0.5.7-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:a806f240ede8da4547f3e50dc3ad88c0d244712270fe361a9b7e3f5d52b4bc39", size = 7425807, upload-time = "2026-01-07T09:52:28.416Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/af/4b/d8735ffcf0b463bd54e23be7df46d77d40d43a672bc01a7a8da3e8f6daf3/sqlcipher3_wheels-0.5.7-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:7198ee5b4c51aae6a7dfab93da35fe82a3c8ed37b4bf6cf55d6f1a0eea387d72", size = 7067479, upload-time = "2026-01-07T09:52:29.867Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/97/54/663f4c349ae59a1a9b2180f2b06c9579ddc611649b50e2ad88c2e910582c/sqlcipher3_wheels-0.5.7-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:af83b12791a1221f1799d7ad6f79edb74a3be642326a9d780dd9f86a290f9c37", size = 7097843, upload-time = "2026-01-07T09:52:31.172Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f3/8e/f8737c593394f050a73742a81e47be2b46b6413f24e96fa0b10d01dc2555/sqlcipher3_wheels-0.5.7-cp314-cp314t-win32.whl", hash = "sha256:90994590f474975d6b480af96ae5ca3887e17303b1144bfea80a38c2e9fdc157", size = 2036431, upload-time = "2026-01-07T09:52:32.444Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/2a/05/573df32b1b1738779dc5326b71d0d611d28d386e7141526ed27043d476a1/sqlcipher3_wheels-0.5.7-cp314-cp314t-win_amd64.whl", hash = "sha256:85422846b4c86e63a2b495a2a60945f7a1a04641d1600859cb65fc68a1ce53ed", size = 2564275, upload-time = "2026-01-07T09:52:33.574Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/54/ee/40e79926f469d630bc1df03b131e8e84740b82dc7e202fe272d2da9110e3/sqlcipher3_wheels-0.5.7-cp314-cp314t-win_arm64.whl", hash = "sha256:aeeacd6da1fc293b97641a794d15d862d7928408ddad9ea5072fad99229970aa", size = 2728867, upload-time = "2026-01-07T09:52:34.718Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "typing-extensions"
|
||||
version = "4.15.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391, upload-time = "2025-08-25T13:49:26.313Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "win32-setctime"
|
||||
version = "1.2.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/b3/8f/705086c9d734d3b663af0e9bb3d4de6578d08f46b1b101c2442fd9aecaa2/win32_setctime-1.2.0.tar.gz", hash = "sha256:ae1fdf948f5640aae05c511ade119313fb6a30d7eabe25fef9764dca5873c4c0", size = 4867, upload-time = "2024-12-07T15:28:28.314Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/e1/07/c6fe3ad3e685340704d314d765b7912993bcb8dc198f0e7a89382d37974b/win32_setctime-1.2.0-py3-none-any.whl", hash = "sha256:95d644c4e708aba81dc3704a116d8cbc974d70b3bdb8be1d150e36be6e9d1390", size = 4083, upload-time = "2024-12-07T15:28:26.465Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "yarl"
|
||||
version = "1.24.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "idna" },
|
||||
{ name = "multidict" },
|
||||
{ name = "propcache" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/79/12/1e8f37460ea0f7eb59c221fdaf0ed75e7ac43e97f8093b9c6f411df50a78/yarl-1.24.2.tar.gz", hash = "sha256:9ac374123c6fd7abf64d1fec93962b0bd4ee2c19751755a762a72dd96c0378f8", size = 210798, upload-time = "2026-05-19T21:31:05.599Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/40/0e/e08087695fc12789263821c5dc0f8dc52b5b17efd0887cacf419f8a43ba3/yarl-1.24.2-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:f9312b3c02d9b3d23840f67952913c9c8721d7f1b7db305289faefa878f364c2", size = 129670, upload-time = "2026-05-19T21:29:56.631Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3a/98/ab4b5ed1b1b5cd973c8a3eb994c3a6aefb6ce6d399e21bb5f0316c33815c/yarl-1.24.2-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:a4f4d6cd615823bfc7fb7e9b5987c3f41666371d870d51058f77e2680fbe9630", size = 91916, upload-time = "2026-05-19T21:29:58.645Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ba/b1/5297bb6a7df4782f7605bffc43b31f5044070935fbbcaa6c705a07e6ac65/yarl-1.24.2-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:0c3063e5c0a8e8e62fae6c2596fa01da1561e4cd1da6fec5789f5cf99a8aefd8", size = 91625, upload-time = "2026-05-19T21:30:00.412Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/02/a7/45baabfff76829264e623b185cff0c340d7e11bf3e1cd9ea37e7d17934bd/yarl-1.24.2-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fecd17873a096036c1c87ab3486f1aef7f269ada7f23f7f856f93b1cc7744f14", size = 104574, upload-time = "2026-05-19T21:30:02.544Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f3/40/3a5ab144d3d650ca37d4f4b57e56169be8af3ca34c448793e064b30baaed/yarl-1.24.2-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:a46d1ab4ba4d32e6dc80daf8a28ce0bd83d08df52fbc32f3e288663427734535", size = 97534, upload-time = "2026-05-19T21:30:04.319Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9c/b5/5658fef3681fb5776b4513b052bec750009f47b3a592251c705d75375798/yarl-1.24.2-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:73e68edf6dfd5f73f9ca127d84e2a6f9213c65bdffb736bda19524c0564fcd14", size = 111481, upload-time = "2026-05-19T21:30:05.988Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/4c/06/fdcd7dde037f00866dce123ed4ba23dba94beb56fc4cf561668d27be37f2/yarl-1.24.2-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a296ca617f2d25fbceafb962b88750d627e5984e75732c712154d058ae8d79a3", size = 111529, upload-time = "2026-05-19T21:30:07.738Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c2/53/d81269aaafccea0d33396c03035de997b743f11e648e6e27a0df99c72980/yarl-1.24.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e51b2cf5ec89a8b8470177641ed62a3ba22d74e1e898e06ad53aa77972487208", size = 107338, upload-time = "2026-05-19T21:30:09.713Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ae/04/23049463f729bd899df203a7960505a75333edd499cda8aa1d5a82b64df5/yarl-1.24.2-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:310fc687f7b2044ec54e372c8cbe923bb88f5c37bded0d3079e5791c2fc3cf50", size = 106147, upload-time = "2026-05-19T21:30:11.365Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/14/18/04a4b5830b43ed5e4c5015b40e9f6241ad91487d71611061b4e111d6ac80/yarl-1.24.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:297a2fe352ecf858b30a98f87948746ec16f001d279f84aebdbd3bd965e2f1bd", size = 104272, upload-time = "2026-05-19T21:30:12.978Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5a/f7/8cffdf319aee7a7c1dbd07b61d91c3e3fda460c7a93b5f93e445f3806c4c/yarl-1.24.2-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:2a263e76b97bc42bdcd7c5f4953dec1f7cd62a1112fa7f869e57255229390d67", size = 99962, upload-time = "2026-05-19T21:30:15.001Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d7/39/b3cce3b7dbef64ac700ad4cea156a207d01bede0f507587616c364b5468e/yarl-1.24.2-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:822519b64cf0b474f1a0aaef1dc621438ea46bb77c94df97a5b4d213a7d8a8b1", size = 111063, upload-time = "2026-05-19T21:30:16.683Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a1/ea/100818505e7ebf165c7242ff17fdf7d9fee79e27234aeca871c1082920d7/yarl-1.24.2-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:b6067060d9dc594899ba83e6db6c48c68d1e494a6dab158156ed86977ca7bcb1", size = 105438, upload-time = "2026-05-19T21:30:18.769Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8f/d2/e075a0b32aa6625087de9e653087df0759fed5de4a435fef594181102a77/yarl-1.24.2-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:0063adad533e57171b79db3943b229d40dfafeeee579767f96541f106bac5f1b", size = 111458, upload-time = "2026-05-19T21:30:21.024Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e6/5c/ceea7ba98b65c8eb8d947fdc52f9bedfcd43c6a57c9e3c90c17be8f324a3/yarl-1.24.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ee8e3fb34513e8dc082b586ef4910c98335d43a6fab688cd44d4851bacfce3e8", size = 107589, upload-time = "2026-05-19T21:30:23.412Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fa/d9/5582d57e2b2db9b85eb6663a22efdd78e08805f3f5389566e9fcad254d1b/yarl-1.24.2-cp314-cp314-win_amd64.whl", hash = "sha256:afb00d7fd8e0f285ca29a44cc50df2d622ff2f7a6d933fa641577b5f9d5f3db0", size = 94424, upload-time = "2026-05-19T21:30:25.425Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/92/10/7dc07a0e22806a9280f42a57361395506e800c64e22737cd7b0886feab42/yarl-1.24.2-cp314-cp314-win_arm64.whl", hash = "sha256:68cf6eacd6028ef1142bc4b48376b81566385ca6f9e7dde3b0fa91be08ffcb57", size = 88690, upload-time = "2026-05-19T21:30:27.623Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9e/13/d5b8e2c8667db955bcb3de233f18798fefe7edf1d7429c2c9d4f9c401114/yarl-1.24.2-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:221ce1dd921ac4f603957f17d7c18c5cc0797fbb52f156941f92e04605d1d67b", size = 136248, upload-time = "2026-05-19T21:30:29.297Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/de/46/a4a97c05c9c9b8fd266bb2a0df12992c7fbd02391eb9640583411b6dab32/yarl-1.24.2-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:5f3224db28173a00d7afacdee07045cc4673dfab2b15492c7ae10deddbece761", size = 95084, upload-time = "2026-05-19T21:30:31.031Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/95/b2/845cf2074a015e6fe0d0808cf1a2d9e868386c4220d657ebd8302b199043/yarl-1.24.2-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c557165320d6244ebe3a02431b2a201a20080e02f41f0cfa0ccc47a183765da8", size = 95272, upload-time = "2026-05-19T21:30:33.062Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fe/16/e69d4aa244aef45235ddfebc0e04036a6829842bc5a6a795aedc6c998d23/yarl-1.24.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:904065e6e85b1fa54d0d87438bd58c14c0bad97aad654ad1077fd9d87e8478ed", size = 101497, upload-time = "2026-05-19T21:30:34.842Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/15/94/c07107715d621076863ee88b3ddf183fa5e9d4aba5769623c9979828410a/yarl-1.24.2-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:8cec2a38d70edc10e0e856ceda886af5327a017ccbde8e1de1bd44d300357543", size = 94002, upload-time = "2026-05-19T21:30:37.724Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a9/35/fc1bbdd895b5e4010b8fdd037f7ed3aa289d3863e08231b30231ca9a0815/yarl-1.24.2-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e7484b9361ed222ee1ca5b4337aa4cbdcc4618ce5aff57d9ef1582fd95893fc0", size = 106524, upload-time = "2026-05-19T21:30:40.196Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1f/f2/32b66d0a4ba47c296cf86d03e2c67bff58399fe6d6d84d5205c04c66cc6d/yarl-1.24.2-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:84f9670b89f34db07f81e53aee83e0b938a3412329d51c8f922488be7fcc4024", size = 106165, upload-time = "2026-05-19T21:30:41.888Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/95/47/37cb5ff50c5e825d4d38e81bb04d1b7e96bf960f7ab89f9850b162f3f114/yarl-1.24.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:abb2759733d63a28b4956500a5dd57140f26486c92b2caedfb964ab7d9b79dbf", size = 103010, upload-time = "2026-05-19T21:30:43.985Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6f/d2/4597912315096f7bb359e46e13bf8b60994fcbb2db29b804c0902ef4eff5/yarl-1.24.2-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:081c2bf54efe03774d0311172bc04fedf9ca01e644d4cd8c805688e527209bdc", size = 101128, upload-time = "2026-05-19T21:30:46.291Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b9/d5/c8e86e120521e646013d02a8e3b8884392e28494be8f392366e50d208efc/yarl-1.24.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:86746bef442aa479107fe28132e1277237f9c24c2f00b0b0cf22b3ee0904f2bb", size = 101382, upload-time = "2026-05-19T21:30:48.085Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fa/98/70b229236118f89dbeb739b76f10225bbf53b5497725502594c9a01d699a/yarl-1.24.2-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:2d07d21d0bc4b17558e8de0b02fbfdf1e347d3bb3699edd00bb92e7c57925420", size = 95964, upload-time = "2026-05-19T21:30:49.785Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/87/f8/56c386981e3c8648d279fdef2397ffec577e8320fd5649745e34d54faeb7/yarl-1.24.2-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:4fb1ac3fc5fecd8ae7453ea237e4d22b49befa70266dfe1629924245c21a0c7f", size = 106204, upload-time = "2026-05-19T21:30:51.862Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1a/1e/765afe97811ca35933e2a7de70ac57b1997ea2e4ee895719ee7a231fb7e5/yarl-1.24.2-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:4da31a5512ed1729ca8d8aacde3f7faeb8843cde3165d6bcf7f88f74f17bb8aa", size = 101510, upload-time = "2026-05-19T21:30:53.62Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ee/78/393913f4b9039e1edd09ae8a9bbb9d539be909a8abf6d8a2084585bed4b7/yarl-1.24.2-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:533ded4dceb5f1f3da7906244f4e82cf46cfd40d84c69a1faf5ac506aa65ecbe", size = 105584, upload-time = "2026-05-19T21:30:55.962Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/78/87/deb17b7049bbe74ea11a713b86f8f27800cc1c8648b0b797243ebb4830ba/yarl-1.24.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:7b3a85525f6e7eeabcfdd372862b21ee1915db1b498a04e8bf0e389b607ff0bd", size = 103410, upload-time = "2026-05-19T21:30:57.962Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/8f/be/f9f7594e23b5b93affff0318e4593c1920331bcaefda326cabcad94296a1/yarl-1.24.2-cp314-cp314t-win_amd64.whl", hash = "sha256:a7624b1ca46ca5d7b864ef0d2f8efe3091454085ee1855b4e992314529972215", size = 102980, upload-time = "2026-05-19T21:30:59.735Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/65/a4/ba80dccd3593ff1f01051a818694d07b58cb8232677ee9a22a5a1f93a9fc/yarl-1.24.2-cp314-cp314t-win_arm64.whl", hash = "sha256:e434a45ce2e7a947f951fc5a8944c8cc080b7e59f9c50ae80fd39107cf88126d", size = 91219, upload-time = "2026-05-19T21:31:01.934Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fd/4d/4b880086bd0d3e034d25647be1d830afc3e3f610e98c4ab3490af6b1b6d5/yarl-1.24.2-py3-none-any.whl", hash = "sha256:2783d9226db8797636cd6896e4de81feed252d1db72265686c9558d97a4d94b9", size = 53576, upload-time = "2026-05-19T21:31:03.909Z" },
|
||||
]
|
||||
@@ -1,244 +0,0 @@
|
||||
use rekordcrate::pdb::RowGroup;
|
||||
use rekordcrate::pdb::Artwork;
|
||||
use rekordcrate::pdb::ArtworkId;
|
||||
use rekordcrate::pdb::Page;
|
||||
use rekordcrate::pdb::PageType;
|
||||
use rekordcrate::pdb::PageIndex;
|
||||
use std::io::Write;
|
||||
use std::io::{Seek, SeekFrom};
|
||||
use binrw::BinWrite;
|
||||
use rekordcrate::pdb::LabelId;
|
||||
use rekordcrate::pdb::KeyId;
|
||||
use rekordcrate::pdb::GenreId;
|
||||
use rekordcrate::pdb::ArtistId;
|
||||
use rekordcrate::pdb::AlbumId;
|
||||
use rekordcrate::pdb::Album;
|
||||
use rekordcrate::pdb::Artist;
|
||||
use rekordcrate::pdb::Genre;
|
||||
use rekordcrate::pdb::Key;
|
||||
use rekordcrate::pdb::Label;
|
||||
use rekordcrate::pdb::Table;
|
||||
use std::fs::File;
|
||||
use std::path::Path;
|
||||
use binrw::BinRead;
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use binrw::Endian;
|
||||
use rekordcrate::pdb::{
|
||||
Header, Row,
|
||||
Track, TrackId,
|
||||
PlaylistTreeNode, PlaylistTreeNodeId,
|
||||
PlaylistEntry,
|
||||
};
|
||||
|
||||
#[derive(Debug)]
|
||||
pub enum DbError {
|
||||
UnknownPlaylist,
|
||||
UnknownTrack,
|
||||
PositionOutOfBounds,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
pub struct Database {
|
||||
pub header: Header,
|
||||
pub tracks: BTreeMap<TrackId, Track>,
|
||||
pub albums: BTreeMap<AlbumId, Album>,
|
||||
pub artists: BTreeMap<ArtistId, Artist>,
|
||||
pub artwork: BTreeMap<ArtworkId, Artwork>,
|
||||
pub genres: BTreeMap<GenreId, Genre>,
|
||||
pub keys: BTreeMap<KeyId, Key>,
|
||||
pub labels: BTreeMap<LabelId, Label>,
|
||||
pub nodes: BTreeMap<PlaylistTreeNodeId, PlaylistTreeNode>,
|
||||
// Keep playlist entries ordered by entry_index
|
||||
pub entries: BTreeMap<PlaylistTreeNodeId, Vec<PlaylistEntry>>,
|
||||
}
|
||||
|
||||
const PAGE_SIZE: u16 = 4096;
|
||||
|
||||
impl Database {
|
||||
pub fn export(self, path: &Path) -> Result<(), binrw::Error> {
|
||||
let pages = Table::from_rows(PageType::Tracks, 1, PAGE_SIZE, self.tracks.values().cloned().map(Row::Track))?;
|
||||
println!("{:?}", pages.len());
|
||||
println!("{:?}", pages.last());
|
||||
Ok(())
|
||||
}
|
||||
// pub fn export(self, path: &Path) -> std::io::Result<()> {
|
||||
// // 1) Reserve header page
|
||||
// let mut f = File::create(path)?;
|
||||
// f.write_all(&vec![0u8; PAGE_SIZE as usize])?;
|
||||
|
||||
// // 2) Emit tables linearly, collect table metadata
|
||||
// let mut next_page_index: u32 = 1;
|
||||
|
||||
// // ok now we're getting somewhere
|
||||
// // smash rows into row_groups of 16
|
||||
// // try to write those into a page until it is full
|
||||
// // yield pages until all the rows are in
|
||||
// // set the indices
|
||||
// // 420 blaze it
|
||||
|
||||
// let (tracks_first, tracks_last) =
|
||||
// Self::write_table(&mut f, PageType::Tracks, next_page_index, self.tracks.values().cloned().map(Row::Track))?;
|
||||
// next_page_index = tracks_last + 1;
|
||||
|
||||
// let (tree_first, tree_last) =
|
||||
// Self::write_table(&mut f, PageType::PlaylistTree, next_page_index, self.nodes.values().cloned().map(Row::PlaylistTreeNode))?;
|
||||
// next_page_index = tree_last + 1;
|
||||
|
||||
// let plist_entries = self.entries.iter().flat_map(|(_pid, vec)| {
|
||||
// // ensure contiguous entry_index
|
||||
// let mut v = vec.clone();
|
||||
// v.sort_by_key(|e| e.entry_index);
|
||||
// v.into_iter().map(Row::PlaylistEntry)
|
||||
// });
|
||||
// let (entries_first, entries_last) =
|
||||
// Self::write_table(&mut f, PageType::PlaylistEntries, next_page_index, plist_entries)?;
|
||||
// next_page_index = entries_last + 1;
|
||||
|
||||
// let (art_first, art_last) =
|
||||
// Self::write_table(&mut f, PageType::Artwork, next_page_index, self.artwork.values().cloned().map(Row::Artwork))?;
|
||||
// next_page_index = art_last + 1;
|
||||
|
||||
// // 3) Build header and back-patch at page 0
|
||||
// let header = Header {
|
||||
// // fill other required fields in your crate
|
||||
// sequence: 1, // or previous+1
|
||||
// next_unused_page: PageIndex(next_page_index),
|
||||
// tables: vec![
|
||||
// Table { page_type: PageType::Tracks, first_page: PageIndex(tracks_first), last_page: PageIndex(tracks_last), empty_candidate: 0, /* … */ },
|
||||
// Table { page_type: PageType::PlaylistTree, first_page: PageIndex(tree_first), last_page: PageIndex(tree_last), empty_candidate: 0, /* … */ },
|
||||
// Table { page_type: PageType::PlaylistEntries, first_page: PageIndex(entries_first), last_page: PageIndex(entries_last), empty_candidate: 0, /* … */ },
|
||||
// Table { page_type: PageType::Artwork, first_page: PageIndex(art_first), last_page: PageIndex(art_last), empty_candidate: 0, /* … */ },
|
||||
// ],
|
||||
// ..Default::default()
|
||||
// };
|
||||
|
||||
// // Write header at offset 0 and pad to one page
|
||||
// f.seek(SeekFrom::Start(0))?;
|
||||
// header.write(&mut f);
|
||||
// // pad residual of page 0 if BinWrite didn’t fill it:
|
||||
// let pos = f.stream_position()?;
|
||||
// if pos < PAGE_SIZE as u64 {
|
||||
// f.write_all(&vec![0u8; (PAGE_SIZE as u64 - pos) as usize])?;
|
||||
// }
|
||||
// Ok(())
|
||||
// }
|
||||
|
||||
// fn write_table<I>(
|
||||
// f: &mut File,
|
||||
// page_type: PageType,
|
||||
// start_index: u32,
|
||||
// rows: I,
|
||||
// ) -> std::io::Result<(u32, u32)>
|
||||
// where
|
||||
// I: IntoIterator<Item = Row>,
|
||||
// {
|
||||
// let mut page_index = start_index;
|
||||
// let mut first = 0;
|
||||
// let mut last = 0;
|
||||
|
||||
// // Pack rows into pages. Use your crate’s Page builder to handle row groups.
|
||||
// let mut packer = PagePacker::new(page_type, PAGE_SIZE);
|
||||
|
||||
// for row in rows {
|
||||
// if !packer.try_push(&row) {
|
||||
// // flush full page
|
||||
// let mut page = packer.finish(page_index, /*next_page=*/0);
|
||||
// // Temporarily set next_page=0; we will fix it after writing the next page.
|
||||
// Self::write_page(f, &mut page)?;
|
||||
// if first == 0 { first = page_index; }
|
||||
// last = page_index;
|
||||
// page_index += 1;
|
||||
// // start new page with the row that didn’t fit
|
||||
// packer = PagePacker::new(page_type, PAGE_SIZE);
|
||||
// assert!(packer.try_push(&row));
|
||||
// }
|
||||
// }
|
||||
|
||||
// // flush final page if it has content or if this is an empty table create one empty page
|
||||
// if packer.is_empty() && first == 0 {
|
||||
// let mut page = Page::empty(page_type, page_index, /*next_page=*/0, PAGE_SIZE);
|
||||
// Self::write_page(f, &mut page)?;
|
||||
// first = page_index;
|
||||
// last = page_index;
|
||||
// } else {
|
||||
// let mut page = packer.finish(page_index, /*next_page=*/0);
|
||||
// Self::write_page(f, &mut page)?;
|
||||
// if first == 0 { first = page_index; }
|
||||
// last = page_index;
|
||||
// }
|
||||
|
||||
// // Fix `next_page` pointers by rewriting page headers if your Page writer requires it.
|
||||
// // Simpler: during emission, remember the previous page file offset and seek back
|
||||
// // to patch its `next_page` to `page_index`.
|
||||
|
||||
// Ok((first, last))
|
||||
// }
|
||||
|
||||
|
||||
// fn write_page(f: &mut File, page: &mut Page) -> std::io::Result<()> {
|
||||
// page.write(f);
|
||||
// Ok(())
|
||||
// }
|
||||
|
||||
pub fn load_from_pdb(path: impl AsRef<Path>) -> rekordcrate::Result<Self> {
|
||||
let mut f = std::fs::File::open(path)?;
|
||||
let header = Header::read(&mut f)?; // uses the type’s endianness
|
||||
|
||||
let mut tracks: BTreeMap<TrackId, Track> = BTreeMap::new();
|
||||
let mut albums: BTreeMap<AlbumId, Album> = BTreeMap::new();
|
||||
let mut artists: BTreeMap<ArtistId, Artist> = BTreeMap::new();
|
||||
let mut artwork: BTreeMap<ArtworkId, Artwork> = BTreeMap::new();
|
||||
let mut genres: BTreeMap<GenreId, Genre> = BTreeMap::new();
|
||||
let mut keys: BTreeMap<KeyId, Key> = BTreeMap::new();
|
||||
let mut labels: BTreeMap<LabelId, Label> = BTreeMap::new();
|
||||
|
||||
let mut nodes: BTreeMap<PlaylistTreeNodeId, PlaylistTreeNode> = BTreeMap::new();
|
||||
let mut entries: BTreeMap<PlaylistTreeNodeId, Vec<PlaylistEntry>> = BTreeMap::new();
|
||||
|
||||
for table in &header.tables {
|
||||
let pages = header.read_pages(&mut f, Endian::NATIVE, (&table.first_page, &table.last_page))?;
|
||||
|
||||
for page in pages {
|
||||
for rg in page.row_groups {
|
||||
for row in rg.present_rows() {
|
||||
match row {
|
||||
Row::Track(t) => {
|
||||
tracks.insert(t.id, t.clone());
|
||||
}
|
||||
Row::Artist(a) => {
|
||||
artists.insert(a.id, a.clone());
|
||||
}
|
||||
Row::Album(a) => {
|
||||
albums.insert(a.id, a.clone());
|
||||
}
|
||||
Row::Genre(g) => {
|
||||
genres.insert(g.id, g.clone());
|
||||
}
|
||||
Row::Key(k) => {
|
||||
keys.insert(k.id, k.clone());
|
||||
}
|
||||
Row::Label(l) => {
|
||||
labels.insert(l.id, l.clone());
|
||||
}
|
||||
Row::PlaylistTreeNode(n) => {
|
||||
nodes.insert(n.id, n.clone());
|
||||
}
|
||||
Row::PlaylistEntry(e) => {
|
||||
entries.entry(e.playlist_id).or_default().push(e.clone());
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Ensure deterministic ordering within each playlist
|
||||
for v in entries.values_mut() {
|
||||
v.sort_by_key(|e| e.entry_index); // needs `pub entry_index`
|
||||
}
|
||||
|
||||
Ok(Self { header, tracks, albums, artists, artwork, genres, keys, labels, nodes, entries })
|
||||
}
|
||||
}
|
||||
+16
-23
@@ -1,31 +1,24 @@
|
||||
mod db;
|
||||
|
||||
extern crate walkdir;
|
||||
|
||||
use rekordcrate::pdb::Page;
|
||||
use rekordcrate::pdb::PageIndex;
|
||||
use rekordcrate::pdb::Row::PlaylistEntry;
|
||||
use rekordcrate::pdb::string::DeviceSQLString;
|
||||
use rekordcrate::pdb::RowGroup;
|
||||
use rekordcrate::pdb::TrackId;
|
||||
use rekordcrate::pdb::PageType;
|
||||
use rekordcrate::pdb::Table;
|
||||
use rekordcrate::pdb::{Header, PlaylistTreeNode, PlaylistTreeNodeId};
|
||||
use walkdir::WalkDir;
|
||||
use std::env;
|
||||
use std::fs::create_dir;
|
||||
use std::fs::create_dir_all;
|
||||
use std::{fs::exists, path::Path};
|
||||
use db::Database;
|
||||
use rekordcrate::util::RekordcrateError;
|
||||
use rekordcrate::pdb::Database;
|
||||
use std::fs::File;
|
||||
use std::path::Path;
|
||||
|
||||
const PAGE_SIZE: u32 = 4096;
|
||||
|
||||
// Example
|
||||
fn main() -> rekordcrate::Result<()> {
|
||||
let db = Database::load_from_pdb("ntr.pdb")?;
|
||||
println!("tracks: {}, playlists: {}", db.tracks.len(), db.nodes.len());
|
||||
println!("tables: {:?}", db.header);
|
||||
println!("tables: {:?}", db.header.tables);
|
||||
fn main() -> Result<(), RekordcrateError> {
|
||||
let db = Database::load_from_pdb("ntr.pdb").unwrap();
|
||||
|
||||
db.export(&Path::new("./data/output.pdb"))?;
|
||||
println!("{:?}", db.colors);
|
||||
|
||||
let output = Path::new("./data/output.pdb");
|
||||
let mut f = File::create(output)?;
|
||||
db.export(PAGE_SIZE, &mut f)?;
|
||||
|
||||
// let db = Database::load_from_pdb("./data/output.pdb").unwrap();
|
||||
// println!("{:?}", db);
|
||||
|
||||
// for n in db.nodes {
|
||||
// println!("{:?}", n);
|
||||
|
||||
+28
-10
@@ -4,23 +4,41 @@
|
||||
{
|
||||
"path": ".",
|
||||
},
|
||||
{
|
||||
"path": "h:ome\\ntr\\code\\rekordcrate",
|
||||
},
|
||||
{
|
||||
"path": "C:\\Users\\ntr\\code\\rekordcrate",
|
||||
},
|
||||
{
|
||||
"path": "/home/ntr/code/rekordcrate"
|
||||
},
|
||||
],
|
||||
"settings":
|
||||
{
|
||||
"LSP":
|
||||
{
|
||||
"LSP-pyright":
|
||||
{
|
||||
// Use maple-py/ as the python project root so pyrightconfig.json
|
||||
// there is picked up (extraPaths=["src"], venvPath=".venv",
|
||||
// pythonVersion="3.14"). The interpreter is the uv-managed venv.
|
||||
"settings":
|
||||
{
|
||||
"python.pythonPath": "$folder/maple-py/.venv/bin/python",
|
||||
"python.analysis.extraPaths":
|
||||
[
|
||||
"$folder/maple-py/src",
|
||||
],
|
||||
"python.analysis.typeCheckingMode": "basic",
|
||||
"python.analysis.diagnosticMode": "openFilesOnly",
|
||||
"python.analysis.autoImportCompletions": true,
|
||||
"python.analysis.indexing": true,
|
||||
},
|
||||
// Pin the LSP server's working directory to maple-py/ so file
|
||||
// lookups resolve relative to where pyrightconfig.json lives.
|
||||
"initializationOptions":
|
||||
{
|
||||
"pythonPath": "$folder/maple-py/.venv/bin/python",
|
||||
},
|
||||
},
|
||||
},
|
||||
"cargo_build":
|
||||
{
|
||||
"paths":
|
||||
{
|
||||
"C:\\Users\\ntr\\code\\maple\\maple-rs":
|
||||
"$folder/maple-rs":
|
||||
{
|
||||
"variants":
|
||||
{
|
||||
|
||||
Reference in new issue
Block a user