Skip to content

hci: move the protocol stack into a subpackage - #478

Open
deadprogram wants to merge 2 commits into
devfrom
hci-subpackage-move
Open

hci: move the protocol stack into a subpackage#478
deadprogram wants to merge 2 commits into
devfrom
hci-subpackage-move

Conversation

@deadprogram

@deadprogram deadprogram commented Sep 9, 2026

Copy link
Copy Markdown
Member

Moves the HCI protocol stack out of the root package and into
tinygo.org/x/bluetooth/hci, for the four reasons in the issue: separate the
hardware from the protocol, make unit tests possible, leave room for a more
complete protocol, and let a board add vendor specific extensions.

This is the second of three stacked pull requests. It builds on #477
ble: move the core value types into a leaf package, and the tests that the
move makes possible come in #476. Review #477 first.

What moved

hci.go, att_hci.go and l2cap_hci.go are now hci/hci.go, hci/att.go
and hci/l2cap.go, with no build tags, so the protocol code builds and
tests on the host. It carried hci || ninafw || cyw43439 || espradio before,
which is why it had no tests.

The four seams

  • Transport. A board supplies an hci.Transport and calls
    hci.NewStack. It is the only hardware dependent part, and each of the four
    backends now asserts that its transport satisfies it.
  • Vendor extensions. SetEventHandler and SetLEEventHandler receive
    anything the package does not handle, and SendVendorCommand sends the
    matching commands. hci_cyw43439.go used to reach into the hci struct from
    another file in the same package; it now goes through this route, which is
    the same one a board outside this repository would take.
  • The local GATT server. ATT held a *Characteristic. It now holds an
    hci.CharacteristicHandler, a set of functions. A struct of functions rather
    than an interface, because an interface call on this path costs about 5 kB of
    flash on TinyGo, measured on nano-rp2040 with examples/heartrate.
  • Events. gap_hci.go read h.advData and h.connectData as struct
    fields. It now goes through accessors.

Two commits, in order

  1. hci: decouple the protocol layers from the root package types, which keeps
    one package and the file layout unchanged, so the behaviour change is
    reviewable on its own.
  2. hci: move the protocol stack into a subpackage, plus the export renames.

Size

-size=short, against dev:

target example flash before after RAM before after
nano-rp2040 discover 41752 41632 6516 6516
nano-rp2040 heartrate 60340 61068 8436 8460
arduino-nano33 discover 39916 39812 5696 5696
circuitplay-express hci hci_uart advertisement 56212 56768 5820 5844
pico-w discover 671304 671072 37016 37016
xiao-esp32c3 discover 310884 307898 60208 58240

Verification

go vet and go test over ./ ./ble ./hci, both plain and with -race,
make smoketest-linux and make smoketest-windows, plus a build of all four
HCI transports and the two bledebug configurations.

Run on hardware. examples/discover, which is the central path, works on one
ninafw board, one pico-w and one esp32, against dev, against the move and
against the tests. examples/heartrate, which is the peripheral path, was also
checked on a ninafw board with a host as the central: the service and
characteristic discovery, the MTU and the read permissions are the same in all
three. The peripheral path is not yet checked on pico-w or esp32.

@deadprogram

Copy link
Copy Markdown
Member Author

This actually was run on xiao-esp32c3, pico-w and nano-rp2040 boards with success for both central and peripheral roles.

Base automatically changed from ble-package to dev September 9, 2026 19:25
Prepares the HCI, ATT and L2CAP code to move into a subpackage. The move
itself is a separate commit, so that this behaviour change can be tested
on hardware with the file layout unchanged.

ATT held a *Characteristic and called four of its methods. It now holds a
charHandler, a set of functions that gatts_hci.go fills in, so the public
API of Characteristic does not change. A struct of functions rather than
an interface, because an interface call on this path costs about 5 kB of
flash on TinyGo (measured on nano-rp2040 with examples/heartrate).

addLocalAttribute and addLocalCharacteristic take a uint8 rather than a
CharacteristicPermissions. ATT writes the value to the wire as a raw GATT
properties byte and never reads the permission bits, so the GATT concept
does not need to cross the boundary.

The advertising report and the connection events were read as struct
fields from gap_hci.go. They are now read through accessors, which is
what a subpackage needs. Two hooks, setEventHandler and setLeEventHandler,
let a controller consume an event or an LE subevent that this package does
not know. That is the extension point for controller specific events, and
it costs about 150 bytes.

Also moves SetRandomAddress down into the HCI layer, widens the ACL CID
to uint16 so that CIDs above 0xff are possible, and reads the reason out
of a Disconnection Complete event.

Flash on nano-rp2040 against dev: discover 41752 -> 41612, advertisement
56696 -> 57176, heartrate 60340 -> 60980.
The HCI, L2CAP and ATT code is now tinygo.org/x/bluetooth/hci. It has no
build tags, so it builds and unit tests on the host, and it no longer
carries the four board tags that kept it baremetal only.

A board supplies an hci.Transport and calls hci.NewStack. Transport is the
only hardware dependent part, so each of the four backends now asserts
that its transport satisfies it. Transport also loses ReadByte, which
nothing called.

The exported names drop the prefixes that were needed in one flat package.
ogfLECtrl becomes OGFLECtrl, attOpReadReq becomes OpReadReq, evtCmdComplete
becomes EventCmdComplete and leMetaEventConnComplete becomes
LEMetaConnComplete. ErrHCITimeout becomes hci.ErrTimeout. The root package
aliases every error under its old name.

The Read and Write methods on the packet types were byte serialisers with
the shape of io.Reader and io.Writer but not the meaning. They are now
unexported marshal and unmarshal.

AttributeProtocolError moves to the ble package, and ATT uses it instead
of a second copy of the same 17 codes. The ATT error response codes were
duplicated between errors.go and att_hci.go.

hci_cyw43439.go reached into the hci struct from another file in the same
package. It now uses SendVendorCommandWithoutResponse, which is the same
route a board outside this repository would take.

go vet found unreachable code after the infinite loop in waitUntilResponse,
which the build tags had hidden until now.
@deadprogram

Copy link
Copy Markdown
Member Author

Rebased and ready.

@deadprogram
deadprogram added this pull request to stack #479 September 9, 2026 19:35
Comment thread att_hci.go

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

att_hci and the new att file were not git mv so it is difficult to see if the att definition really changed that much

Comment thread hci.go

@acouvreur acouvreur Sep 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for hci and hci/hci, it was not git mv. Tracking and history is lost and comparison is difficult in this PR

@acouvreur

Copy link
Copy Markdown
Member

@deadprogram would you be so kind to git mv instead of having deleted and new files for both att and hci ? That would be perfect. Thank you

@deadprogram

Copy link
Copy Markdown
Member Author

The files were moved with git mv. Git keeps no rename record in a commit, so
git mv and a delete plus an add write the same tree. Git finds renames when
it makes the diff, from the similarity of the content, and the default limit is
50 percent. Two of the four files are above it and show as renames:

pair similarity shown as
errors.go -> ble/atterror.go 99% rename
l2cap_hci.go -> hci/l2cap.go 89% rename
hci.go -> hci/hci.go 48% delete plus add
att_hci.go -> hci/att.go 43% delete plus add

The two low ones are just below the limit. The cause is the export renames in
the same commit: ogfLinkCtl becomes OGFLinkCtl, ocfReadBDAddr becomes
OCFReadBDAddr, and so on, for 82 identifiers in hci.go and 52 in att.go.
Almost every line that names one of them changed.

To see all four as renames, lower the limit:

git diff -M40 dev...hci-subpackage-move --stat

git log -M40 --stat does the same for the commits. GitHub gives no control of
the limit in the web interface.

I can make the two files cross the limit if you want it. If the export renames
go into a commit of their own, the move alone measures 54 percent for hci.go
and 55 percent for att.go, and git then reports both as renames. One point to
note: the Files tab compares the base with the head over the whole pull
request, so a separate commit shows the renames only in the per commit view.
For the Files tab the export renames must go into a fourth pull request,
stacked after this one. Tell me if that is worth the extra step and I will do
it.

@acouvreur

Copy link
Copy Markdown
Member

The files were moved with git mv. Git keeps no rename record in a commit, so git mv and a delete plus an add write the same tree. Git finds renames when it makes the diff, from the similarity of the content, and the default limit is 50 percent. Two of the four files are above it and show as renames:
pair similarity shown as
errors.go -> ble/atterror.go 99% rename
l2cap_hci.go -> hci/l2cap.go 89% rename
hci.go -> hci/hci.go 48% delete plus add
att_hci.go -> hci/att.go 43% delete plus add

The two low ones are just below the limit. The cause is the export renames in the same commit: ogfLinkCtl becomes OGFLinkCtl, ocfReadBDAddr becomes OCFReadBDAddr, and so on, for 82 identifiers in hci.go and 52 in att.go. Almost every line that names one of them changed.

To see all four as renames, lower the limit:

git diff -M40 dev...hci-subpackage-move --stat

git log -M40 --stat does the same for the commits. GitHub gives no control of the limit in the web interface.

I can make the two files cross the limit if you want it. If the export renames go into a commit of their own, the move alone measures 54 percent for hci.go and 55 percent for att.go, and git then reports both as renames. One point to note: the Files tab compares the base with the head over the whole pull request, so a separate commit shows the renames only in the per commit view. For the Files tab the export renames must go into a fourth pull request, stacked after this one. Tell me if that is worth the extra step and I will do it.

I guess we learn something everyday :D

@deadprogram

Copy link
Copy Markdown
Member Author

A correction to my last comment. I measured the split, and the 54 and 55
percent that I gave you are not reachable. Those numbers assume that all 125
export renames can move to a later commit. They cannot: 27 of the names are
types and methods that the root package calls, such as Transport, HCI,
ATT, Connection, Poll, LECreateConn and the AttributeType constants.
The move does not compile without them, and those 27 alone use all of the
margin.

The measurements, with git as the judge:

shape hci.go att_hci.go renames shown
this pull request now 48% 43% no
the 98 optional renames deferred 49% 49% no
the decouple split into its own pull request 56% 44% hci.go only
both of the above together 56% 50% yes

So the split that I offered gives 49 and 49, and both files stay as a delete
plus an add.

Two changes together do work. The Files tab compares the base with the head
over the whole pull request, so the decouple commit counts against the
similarity of the move. If the decouple becomes a pull request of its own, and
the 98 optional renames go after the move, then both files cross the limit. The
cost is a five pull request stack, and att_hci.go lands at 50 percent
exactly, which is the limit with nothing to spare. A later edit to that file
would put it back to a delete plus an add.

My suggestion is to leave the history as it is and read the moves with a lower
limit:

git diff -M40 dev...hci-subpackage-move --stat

That reports all four files as renames. But if you prefer the five pull request
stack, say so and I will do it.

@deadprogram

Copy link
Copy Markdown
Member Author

The hardware check is done, and both pull request descriptions now record it.

examples/discover, the central path, runs correctly on one ninafw board, one
pico-w and one esp32, against dev, against this pull request and against the
tests on top of it. examples/heartrate, the peripheral path, was also checked
on a ninafw board with a host as the central. The service and characteristic
discovery, the MTU and the read permissions are the same in all three cases.

Two limits to note. The peripheral path is not yet checked on pico-w or esp32.
And dev completed the discovery too, so the baseline did not meet the Find
Information Response defect with these peers. The bounds checks in the tests
pull request still have only the unit tests behind them, which is what you
expect for malformed input from a peer. The hardware result shows no
regression, not proof of those three fixes.

That leaves only the rename question open on this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants