[🚧 WIP] SASL refactoring and new mechanisms

[nevans]

NOTE: This PR started as a big rough-draft for many of the other PRs listed below. Rather than close the PR and create a new tracking issue, I've been keeping the branch around as a set of experimental implementations for some of the TODO list items, while cherry-picking parts of it into their own PRs when they are ready.

• Issues and PRs
  • Closed
  • Open
• 🔒✨ Adds new mechanisms:
  • EXTERNAL
    • SASL: Add support for EXTERNAL mechanism #79
    • 🔒 Add SASL EXTERNAL mechanism #170
  • ANONYMOUS
    • SASL: Add support for ANONYMOUS mechanism #81
    • 🔒 Add SASL ANONYMOUS mechanism #169
  • OAUTHBEARER
    • SASL: Add support for OAUTHBEARER mechanism #80
    • 🔒 Add SASL OAUTHBEARER mechanism #171
  • SCRAM-SHA-1, SCRAM-SHA-256
    • SCRAM-SHA-1(-PLUS) + SCRAM-SHA-256(-PLUS) + SCRAM-SHA-512(-PLUS) + SCRAM-SHA3-512(-PLUS) supports #54
    • Add SASLprep. Code generated & tested with RFC3454 #64
    • 🔒 Add SASL SCRAM-SHA-* mechanisms #172
• 🔒 Better support for the core RFCs (3051, 4422, 4959, and 9051)
  • RFC4959 (2007): SASL-IR (included in IMAP4rev2) #34
    • ✨🔒 Add SASL-IR support #90
    • 🐛 Fix empty SASL-IR to send "=" #180
  • 🔒 Verify SASL authentication has completed #179
    • Needed for SCRAM-* and to support net-smtp, which already followed the RFC on this.
  • ✨ Minor updates to SASL::Authenticators API #184
  • Cancel SASL authentication after client-side exceptions
    • 🔒 Handle cancel for SASL AUTHENTICATE #324
• API improvements
  • Warn when using deprecated SASL mechanisms #62
  • ♻️ Make SASL.authenticator case insensitive #167
  • ♻️ Simplify lazy-loaded SASL::{Name}Authenticator registration #168
  • ♻️ Allow keyword args on all SASL authenticators #177
  • 🔒 SASL: Clarify usage of username vs authcid vs authzid #187
  • ✨ Add secret alias (for password, oauth2_token, etc) to relevant SASL mechanisms #195
  • Results should encapsulate the authenticator object, which may contain server-sent data.
    • Although there is room for improvement, this was mostly done by:
      • 🔒 Verify SASL authentication has completed #179
      • ✨ Minor updates to SASL::Authenticators API #184
    • AuthenticationSuccess (subclass of TaggedResponse)
    • AuthenticationFailed (subclass of NoResponseError)
    • AuthenticationIncomplete (includes response attr)
    • AuthenticationError
    • AuthenticationCanceled (subclass of BadResponseError)
  • SASL: Add support for lazy-loaded properties and event callbacks #82
    This PR originally had an implementation of this, but it was over-complicated and removed to simplify the API for the v0.4.0 release.
  • Consider supporting simplifications of the process state machine, e.g:
    • Maybe support specific conversation shapes with a single callback method per interaction. I think this is what mongo does.
    • Or, use blocks and yields to invert the API like Enumerable#each vs Enumerator#next. This is closer to the net-smtp API and could greatly simplify the mechanism implementation. It is a little bit trickier to adapt this style to multiple threads and Net::IMAP's receiver loop.
• Share the net-imap SASL implementation with other gems
  Should we create a shared net-sasl gem? #23.
  • Add a protocol adapter layer that answers all of the questions from https://www.rfc-editor.org/rfc/rfc4422.html#section-4 and can be used to simplify inclusion in other gems, e.g. net-smtp, net-pop, net-ldap, mongo, etc.
    • 🔒⚗️ Add experimental SASL::ClientAdapter #183
    • ♻️ Use SASL::ClientAdapter #194
  • Create PR for net-smtp: ruby/net-smtp@master...nevans:net-smtp:net-imap-sasl.
  • Create PR for net-pop
  • Create PRs for net-ldap, mongo, and possibly others...
• Improved documentation
  • SASL doc updates #166
  • 📚 Update SASL docs and add attr_readers #176
• Code re-organization
  • Extract authenticators to their own files #22
  • 🚚 Move and rename SASL authenticators #165

[nevans]

n.b this PR is currently based on #70, #71, #72, #73, #74, #75, and #76. I can rebase part of all of this on master, but I think those will be merged before this is.

Also, it's currently failing 2.6, because I used some numbered parameters in a few places. As discussed on #68, we can remove support for 2.6.

[Neustradamus]

@nevans: Good job!

[Neustradamus]

@nevans: Happy New Year!

Little question, have you a timeline?

[Neustradamus]

@nevans: I think that you can remove CRAM-MD5, DIGEST-MD5, LOGIN from all.

[Neustradamus]

@nevans: Thanks to work on this important part!

I hope that this part can be official soon ^^

[nevans]

@Neustradamus the majority of this was split off into other PRs and released with v0.4.0 (or later releases) a little over a year ago.

[Neustradamus]

@nevans: Yes yes but not all yet and net-sasl has not been updated yet too...

[Neustradamus]

@nevans: Good job but net-sasl has not been updated:

• https://github.com/nevans/net-sasl

Where is the code?

[nevans]

@Neustradamus This PR was for net-imap. Every significant piece of code that was originally in this PR has already been merged, and is available from Net::IMAP::SASL.

I'd like to sync net-sasl with the current net-imap sometime early next year. Currently, that gem is basically just a preliminary experiment, so I haven't prioritized it. But if you want to make an issue on that repo, I'll update it when it's done.

In the meantime, net-imap is bundled with ruby, and the Net::IMAP::SASL API is stable (with a few minor exceptions, which should be documented). If and when net-sasl is updated, there should be little to no functional difference between Net::SASL and Net::IMAP::SASL.