Play a nearby accessory's sound over direct BLE, from the device screen and the map - #139
Play a nearby accessory's sound over direct BLE, from the device screen and the map#139ubrt wants to merge 7 commits into
Conversation
parawanderer
added
enhancement
2bd1379 to
6637e02
Compare
GitHub Actions
Inactive
GitHub Actions
Inactive
|
Marking this as work in progress — please hold off merging. Testing with two real third-party Find My accessories surfaced a reliability gap: an accessory's stored key alignment can drift ahead of its true index, and once that happens the 12h search margin alone doesn't recover it (one of the two needed a multi-day margin to be found at all, which isn't a fix, just a wider blind spot). Root cause: Working on a fix that restricts alignment corrections (both the BLE feedback path here and the existing network-fetch path) to primary-key matches only, and allows correcting downward when one disagrees with the stored value. Will follow up once it's tested end to end — a real AirTag test is planned for next week, which should also cover ground this branch hasn't (only third-party accessories tested so far). |
|
Agreed, holding. Confirming the network half from the pinned source so you don't have to re-derive it — for i in sorted(key_to_ind[key], reverse=True):
accessory.update_alignment(report.timestamp, i)
Note I pushed to your branch while you were writing this (rebase onto @parawanderer has an AirTag and an owner iPad that can be powered off for the separated case, so the AirTag gap is coverable here too if that's useful alongside your test next week. (Reply drafted via Claude Code, reviewed by me.) |
|
Pushed the primary-key-only alignment fix mentioned above. Also been running a passive companion build locally on top of this branch's BLE work: live battery and signal strength for owned tags on the map and device info screen while in range. Mainly useful as a simple way to tell whether you're actually receiving anything from the tag at all - watching the reading update live, per advertisement, is what made it obvious something was wrong when a tag stopped being found despite sitting right next to the phone. Never pinned down what originally caused that drift, but the fix closes a real gap in how a match got trusted, and the tag has stayed found since. Not part of this PR, still iterating; may open it separately later. 
If you want, it would really help if you tested with your own AirTag too :) |
|
Quick note I wanted to drop here before I go all in on my own review as I am finishing up the other open items I still had for 1.1.0: so the app now optionally connects to iCloud. When it does fetch your tags from iCloud, if you enable the debug option in Settings, you'll see that on the "My Devices" device page of every AirTag it has the battery status as reported by iCloud on it. Now I think currently I only get the list from iCloud every 6 hours (I will probably tweak it because iPads update more frequently), but I'm curious if that matches the status you're seeing the tags report by bluetooth? 🤔 |
|
Good question. Two things worth knowing about what the Bluetooth side reports before comparing: The advertisement only carries a 2-bit battery field, so it is one of four coarse levels (full / medium / low / critically low), not a percentage. It's the same encoding FindMy.py's BATTERY_LEVEL map reads. So the comparison can only ever be "same bucket or not". On my two third-party tags there is nothing to compare against, which is itself the interesting result: both were imported via the iCloud account connection (not the zip import), and the iCloud record still reports battery level 0 ("not yet reported") for both. So for these tags the BLE reading is the only battery signal that exists at all. Presumably only Apple's own devices ever write that field for third-party accessories, or these vendors never report it. Your AirTags may well behave differently there, so your comparison would cover the case mine can't. More generally: for anyone without an Apple device around (this app's core audience, arguably), BLE is the only battery source there is, whatever iCloud's refresh interval. The design question that actually falls out of this is persistence: right now my build only shows the BLE value while it's live and lets it age out. Should a sighting's battery level be persisted and shown as "last known (from BLE, at time X)" until a fresher value arrives from whichever source? Curious what you think. The passive-watch build lives on my fork if you want to poke at it: https://github.com/ubrt/OpenTagViewer/tree/feature/nearby-tag-status (built on top of this PR's branch; not proposing it here, still iterating). |
046f3db to
ebffe4e
Compare
|
Coming off my sprint to implement the whole iCloud stuff here just to answer your question @ubrt (I will check your work tomorrow or as soon as I am able to with my devices). But basically I would say that: yes you should save that battery status information and probably the time it was recorded too. It's useful for the owner of the tag to have, imo. I would combine that storage with the persistence of the BLE scans as an alternative source of location history - on top of the apple FindMy network reports (to the extent that you fill the fields of the LocationReports with BLE info). Anyways, I highly recommend you rebase on |
Computes the accessory's current expected BLE MAC address(es) through the pinned FindMy.py fork's rolling-key derivation (main.py:currentMacAddresses, backed by the new RollingKeyPairSource.current_mac_addresses), scans for a match, and writes the DULT/FindMy/AirTag GATT play-sound characteristic - the same thing Find My itself does when a tag is close enough to reach, without going through Apple's network. New menu entry on DeviceInfoActivity. The GATT protocol details in ble/BleGattSoundTrigger.java - the service and characteristic UUIDs, the start opcodes, and the order the three protocols are tried in - are derived from AirGuard (Apache-2.0), verified against its AppleFindMy.kt and GoogleFindMyNetwork.kt. This repository is MIT, so the Apache-2.0 terms are recorded for the derived portion in a new NOTICE file rather than only in a Javadoc header. AirGuard ships no NOTICE of its own, so there is none to propagate. The three protocols are a fallback chain rather than belt-and-braces: not every accessory exposes the same characteristic, so relying on one alone misses devices. Cheap to keep - discoverServices() fetches the whole service table in one round trip and the three checks are local. Temporarily pins a personal FindMy.py fork (ubrt/FindMy.py) across all four places this repository pins it - see the comments at each - until the current_mac_addresses() addition has been offered upstream and lands in parawanderer's fork in turn. Verified: full JVM suite, the Python bridge suite, flake8, pyright, and on real hardware - see the branch's PR description for which accessories and in what state.
MapsActivity's "Ring" button was already there in the layout, wired to a no-op onClickRing - this fills it in rather than adding new UI. Toggling it starts AccessorySoundTrigger.playSoundContinuously (scan, trigger, pause, repeat via Rx repeatWhen) for that card's tag until tapped again or another tag's ring is started; the icon/label swap to a red stop glyph while running. BeaconInformation gained ownedBeaconAccessoryJson (mirroring the existing ownedBeaconPlistRaw), populated in BeaconDataParser, since MapsActivity's per-card data previously had no path to the accessory JSON the ble/ package needs - DeviceInfoActivity's one-shot trigger reads it from OwnedBeacon directly, but the map screen's BeaconInformation DTO didn't carry it. Updates TagCardLayoutTest, which had a test pinning the ring button as GONE from before this - "so that stops being true on purpose rather than by accident". That guard now flips: the button is shown at rest, plus new layout tests for the default label and for TagCardHelper's toggle/label behavior, run on the same aosp-atd managed device (no Maps involved). Requests BLE permission on tap, same as DeviceInfoActivity's one-shot trigger - missed on the first pass here, and invisible on a debug install that had already granted it via that other screen first. A fresh install (found by testing an actual release build) surfaced it: the ring button did nothing but log MISSING_PERMISSION on a loop, forever, with no dialog ever shown. Verified: full JVM suite (124 tests), instrumented layout suite (17 tests, including the new ring-button ones), installed and running - including the permission-request fix, verified end to end on a real release build after the bug surfaced there (two full scan/connect/trigger cycles, one of them after a retry).
AccessorySoundTrigger.playSound/playSoundContinuously now emit BleSoundTriggerUpdate items (SCANNING/CONNECTING/TRIGGERING, then one terminal DONE) instead of a single terminal result. Without this, both callers went silent for however long the scan and GATT handshake took, which reads as "nothing is happening" - especially the first time. Both screens now show the current phase (a replacing toast in DeviceInfoActivity, the ring button's own label on the map) instead of just a final result. The map's ring button also swaps its icon for a spinner for as long as an attempt is actually in flight (SCANNING/CONNECTING/TRIGGERING) - the label alone can sit on screen for several seconds with nothing else moving, which was mistaken for a stall rather than for work in progress. Also: BleGattSoundTrigger.trigger is retried up to 3 times (800ms apart) when it fails with a plain connection/write failure, not when no known sound service was found on the device - a retry cannot fix the latter, only the former is the kind of transient BLE flakiness a retry is for. Previously one dropped connection meant an immediate failure with no second attempt. BleAccessorySoundTrigger's three hardware-dependent seams (permission check, scanner, GATT trigger) are now constructor-injected instead of static calls to BlePermissions/NearbyAccessoryScanner/BleGattSoundTrigger, the same reasoning AppDependencies already uses for HardwareDescriber - those three need real Bluetooth to run, which a JVM test cannot arrange, but the orchestration logic around them (the permission gate, the retry count, what an empty candidate set or a scanner timeout maps to) does not and is now covered by BleAccessorySoundTriggerTest (11 tests, fakes only). The class is generic over the found-device type (<D>, fixed to BluetoothDevice in the real forRealBluetooth() factory) because the real Android class has no public constructor and this project has no Robolectric to fabricate one for tests - a plain String stands in instead. MapsActivity's handleContinuousPingUpdate now stops the loop outright on MISSING_PERMISSION or NO_CANDIDATE_MACS instead of looping on them forever - neither recovers by waiting and retrying, so continuing was pure battery burn with no chance of succeeding. Found next to the permission- request fix in the previous commit, but belongs here: this switch over phases is what this commit introduced. Verified: full JVM suite (135 tests, including the new suite), installed and running - two full scan/connect/trigger cycles on a real release build, one of them after a retry, plus the loading spinner confirmed visually on a subsequent release build.
The write itself is near-instant, so the label went scan -> connect -> Stop with no visible moment where it actually worked - which is what looked like a missing step. A successful DONE now shows ring_status_success for the whole pause before the next scan (roughly as long as an AirTag's chirp lasts); any other outcome still goes straight back to "Stop". Verified: full JVM suite, installed and running.
The BLE work needs RollingKeyPairSource.current_mac_addresses(), contributed by Ulrich Barrot (@ubrt) and merged as parawanderer/FindMy.py#1. This moves all four pin sites off the personal fork and onto 254a7624, which carries it. The same bump also brings the keychain-export work that landed on that branch meanwhile - including the fix for parawanderer#140, where a recovered peer was addressed by its escrow label instead of the bottle's id and every share came back unreadable. Two things the app was getting wrong against that API: A bare current_mac_addresses() searches with no margin, so an accessory whose true index has drifted below where alignment believes it is is simply absent from its own candidate set - no error, no match, "not found nearby" for a tag sitting on the desk. FindMy.py's own is_from takes 12 hours; so does this now. And the call discarded the indices the map exists to provide. A sighting is an observation of the same kind as a decrypted report, so it now feeds back through update_alignment and is persisted the same way, which collapses the next scan from a 12-hour range to three keys. Co-Authored-By: Ulrich Barrot <12350410+ubrt@users.noreply.github.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ches recordAccessorySeen fed back whatever index currentMacAddresses paired an address with, primary or secondary. A secondary key covers 96 consecutive primary indices, so its index is only the first one a search happened to reach, not the true one - and update_alignment only refuses a move backwards. Fed that index, it can ratchet alignment past the true index in the wrong direction, permanently. Measured on a real accessory that drifted 114 indices (28.5 hours) ahead this way and then needed a multi-day margin just to be found at all. The fix moves what crosses the bridge from the key index to the raw address: recordAccessorySeen now re-derives the key at that address itself and only accepts a match through its primary key, where the index is unambiguous. BleSoundTriggerResult.matchedKeyIndex becomes matchedMac throughout, since only Python can tell a primary key from a secondary one from an address - this side of the bridge never could, regardless of what shape crossed it. A primary match also corrects downward, which update_alignment itself cannot do (it only ever moves forward, correct for a fetch's own forward search but not for a BLE match that can legitimately land below the stored alignment - proof the alignment had already drifted too far ahead). recordAccessorySeen writes the corrected index straight into the serialized accessory instead of going through update_alignment for this.
Bypassing update_alignment for the downward index move also bypassed its backward-time guard. A device clock rolled back (manual change, bad carrier time) could persist a (past date, current index) pair, and once the clock corrected, the index extrapolated from that past date would overshoot the true one - eventually past the search margin, at which point the tag drops out of its own candidate set. The guard is now kept: a sighting older than the stored alignment date records nothing.
ebffe4e to
056d758
Compare
2 participants
Implements #17.
What it adds
DeviceInfoActivity: a "Play Sound Nearby" menu entry, one shot.MapsActivity: the existing (inert) Ring button on the tag card becomes a continuous ping toggle. Scan, trigger, pause, repeat until tapped again.Both show live progress: Scanning, Connecting, Sending, result.
Rationale for the FindMy.py / Java split is in my issue comment; you have already agreed with it.
Attribution
The protocol constants and fallback order in
ble/BleGattSoundTrigger.javaderive from AirGuard (Apache-2.0). Added aNOTICEnaming what is derived and from where. AirGuard ships no NOTICE of its own and its LICENSE has no copyright line filled in, so the file attributes the project rather than restating a notice its authors never wrote. README's stale🔴 BLOCKEDline left alone as asked.Testing
Pixel 10 Pro, Android 17, release build.
End to end: my own FineTrack Mini Smart Finder (60387A, third-party Find My), separated from its owner. Key derivation, scan, MAC match, GATT write, sound. Two cycles, one succeeding only after the retry fired. Path selected: generic FindMy
fd44.The fallback chain. Paths 2 and 3 were both observed being selected, not only reasoned about:
fd44on the accessory above, and the AirTag-specific7DFC9000on a different one. Path 1 (DULT) was never observed selected. The path 3 observation comes from TrackerHunter, which shares this GATT layer but not the key derivation above it, and that run was log only: path selection and an acknowledged write, not an audible chirp.Owner-nearby versus separated. Only the separated state works. With the owner present, a 15 second scan saw three separated accessories (
12 19beacon) and two in the owner-present short form (12 02, one at −49 dBm), and none matched any address derived from the accessory's key schedule. So it fails at detection, not at connect or write. Consistent with stek29's note that authorised playback is a separate, L2CAP based mechanism.In practice: it is not a continuous tone. From tapping the button to the accessory sounding takes roughly five seconds, the chirp itself runs for a few seconds, and then there is a gap before the next one, so while walking you get a repeating burst every several seconds rather than something you can home in on continuously. The gap between cycles is a single constant (
CONTINUOUS_PING_PAUSE_MS, currently 4 s) if you want it tighter. Scan times ranged from about 250 ms to twelve seconds depending on how recently the tag last advertised.Not tested: an Apple AirTag end to end through this app (only through the shared GATT layer), other third-party brands, Google Find My Device accessories, and Android versions other than 17.
Suites: 192 JVM, 573 instrumented (568 passed, 5 ignored, 0 failed) on
aosp-atd, 220 Python bridge, 275 strings across 10 locales, flake8 and pyright clean. New:BleAccessorySoundTriggerTest(11 tests, fakes standing in for the three hardware dependent seams) and ring button cases inTagCardLayoutTest, where the old test pinning that button as hidden is inverted rather than deleted.Rebased onto
2564cbc.