Implement auth_map and storage_map at endpoint level

This makes auth_map do what its name implies. Old auth_map in storage
module is deprecated and will be removed in the next release.

Author: foxcpp

docs/multiple-domains.md

@@ -1,69 +1,157 @@
 # Multiple domains configuration
 
-## Separate account namespaces
+By default, maddy uses email addresses as account identifiers for both
+authentication and storage purposes. Therefore, account named `user@example.org`
+is completely independent from `user@example.com`. They must be created
+separately, may have different credentials and have separate IMAP mailboxes.
 
-Given two domains, example.org and example.com. foo@example.org and
-foo@example.com are different and completely independent accounts.
+This makes it extremely easy to setup maddy to manage multiple otherwise
+independent domains.
 
-All changes needed to make it work is to make sure all domains are specified in
-the `$(local_domains)` macro in the main configuration file. Note that you need
-to pick one domain as a "primary" for use in auto-generated messages.
+Default configuration file contains two macros - `$(primary_domain)` and
+`$(local_domains)`. They are used to used in several places thorough the
+file to configure message routing, security checks, etc.
+
+In general, you should just add all domains you want maddy to manage to
+`$(local_domains)`, like this:
 ```
 $(primary_domain) = example.org
 $(local_domains) = $(primary_domain) example.com
 ```
+Note that you need to pick one domain as a "primary" for use in
+auto-generated messages.
 
-The base configuration is done. You can create accounts using
-both domains in the name, send and receive messages and so on.  Do not forget
-to configure corresponding SPF, DMARC and MTA-STS records as was
-recommended in the [introduction tutorial](tutorials/setting-up.md).
+With that done, you can create accounts using both domains in the name, send
+and receive messages and so on.  Do not forget to configure corresponding SPF,
+DMARC and MTA-STS records as was recommended in
+the [introduction tutorial](tutorials/setting-up.md).
 
-## Single account namespace
+Also note that you do not really need a separate TLS certificate for each
+managed domain. You can have one hostname e.g. mail.example.org set as an MX
+record for mulitple domains.
 
-You can configure maddy to only use local part of the email
-as an account identifier instead of the complete email.
+**If you want multiple domains to share username namespace**, you should change
+several more options.
 
-This needs two changes to default configuration:
+You can make "user@example.org" and "user@example.com" users share the same
+credentials of user "user" but have different IMAP mailboxes ("user@example.org"
+and "user@example.com" correspondingly). For that, it is enough to set `auth_map`
+globally to use `email_localpart` table:
 ```
-storage.imapsql local_mailboxes {
-    ...
-    delivery_map email_localpart
-    auth_normalize precis_casefold
-}
+auth_map email_localpart
+```
+This way, when user logs in as "user@example.org", "user" will be passed
+to the authentication provider, but "user@example.org" will be passed to the
+storage backend. You should create accounts like this:
+```
+maddy creds create user
+maddy imap-acct create user@example.org
+maddy imap-acct create user@example.com
 ```
 
-This way, when authenticating as `foxcpp`, it will be mapped to
-`foxcpp` storage account. E.g. you will need to run
-`maddy imap-accts create foxcpp`, without the domain part.
-
-If you have existing accounts, you will need to rename them.
-
-Change to `auth_normalize` is necessary so that normalization function
-will not attempt to parse authentication identity as a email.
+**If you want accounts to also share the same IMAP storage of account named
+"user"**, you can set `storage_map` in IMAP endpoint and `delivery_map` in
+storage backend to use `email_locapart`:
+```
+straoge.imapsql local_mailboxes {
+   ...
+   delivery_map email_localpart # deliver "user@*" to "user"
+}
+imap tls://0.0.0.0:993 {
+   ...
+   storage &local_mailboxes
+   ...
+   storage_map email_localpart # "user@*" accesses "user" mailbox
+}
+```
 
-When a email is received, `delivery_map email_localpart` will strip
-the domain part before looking up the account. That is,
-`foxcpp@example.org` will be become just `foxcpp`.
+You also might want to make it possible to log in without
+specifying a domain at all. In this case, use `email_localpart_optional` for
+both `auth_map` and `storage_map`.
 
 You also need to make `authorize_sender` check (used in `submission` endpoint)
 accept non-email usernames:
 ```
 authorize_sender {
   ...
-  auth_normalize precis_casefold
-  user_to_email regexp "(.*)" "$1@$(primary_domain)"
+  user_to_email chain {
+    step email_localpart_optional            # remove domain from username if present
+    step email_with_domains $(local_domains) # expand username with all allowed domains
+  }
 }
 ```
-Note that is would work only if clients use only one domain as sender (`$(primary_domain)`).
-If you want to allow sending from all domains, you need to remove `authorize_sender` check
-altogether since it is not currently supported.
 
-After that you can create accounts without specifying the domain part:
+## TL;DR
+
+Your options:
+
+**"user@example.org" and "user@example.com" have distinct credentials and
+distinct mailboxes.**
+
+```
+$(primary_domain) = example.org
+$(local_domains) = example.org example.com
+```
+
+Create accounts as:
+
+```shell
+maddy creds create user@example.org
+maddy imap-acct create user@example.org
+maddy creds create user@example.com
+maddy imap-acct create user@example.com
+```
+
+**"user@example.org" and "user@example.com" have same credentials but
+distinct mailboxes.**
+
+```
+$(primary_domain) = example.org
+$(local_domains) = example.org example.com
+auth_map email_localpart
+```
+
+Create accounts as:
+```shell
+maddy creds create user
+maddy imap-acct create user@example.org
+maddy imap-acct create user@example.com
+```
+
+**"user@example.org", "user@example.com", "user" have same credentials and same
+mailboxes.**
+
 ```
-maddy imap-acct create foxcpp
-maddy creds create foxcpp
+   $(primary_domain) = example.org
+   $(local_domains) = example.org example.com
+   auth_map email_localpart_optional # authenticating as "user@*" checks credentials for "user"
+
+   storage.imapsql local_mailboxes {
+      ...
+      delivery_map email_localpart_optional # deliver "user@*" to "user" mailbox
+   }
+
+   imap tls://0.0.0.0:993 {
+      ...
+      storage_map email_localpart_optional # authenticating as "user@*" accesses "user" mailboxes
+   }
+
+   submission tls://0.0.0.0:465 {
+      check {
+        authorize_sender {
+          ...
+          user_to_email chain {
+            step email_localpart_optional            # remove domain from username if present
+            step email_with_domains $(local_domains) # expand username with all allowed domains
+          }
+        }
+      }
+      ...
+   }
 ```
-And authenticate using "foxcpp" in email clients.
 
-Messages for any foxcpp@* address with a domain in `$(local_domains)`
-will be delivered to that mailbox.
+Create accounts as:
+```shell
+maddy creds create user
+maddy imap-acct create user
+```

docs/reference/checks/authorize_sender.md

@@ -16,8 +16,8 @@ check.authorize_sender {
     malformed_action reject
     err_action reject
 
-    auth_normalize precis_casefold_email
-    from_normalize precis_casefold_email
+    auth_normalize auto
+    from_normalize auto
 }
 ```
 ```
@@ -88,25 +88,26 @@ What to do if From or Sender header fields contain malformed values.
 What to do if error happens during prepare\_email or user\_to\_email lookup.
 
 **Syntax:** auth\_normalize _action_ <br>
-**Default:** precis\_casefold\_email
+**Default:** auto
 
 Normalization function to apply to authorization username before
 further processing.
 
 Available options:
-- precis\_casefold\_email   PRECIS UsernameCaseMapped profile + Unicode form for domain
-- precis\_casefold         PRECIS UsernameCaseMapped profile for the entire string
-- precis\_email            PRECIS UsernameCasePreserved profile + Unicode form for domain
-- precis                  PRECIS UsernameCasePreserved profile for the entire string
-- casefold                Convert to lower case
-- noop                    Nothing
+- `auto`                    `precis_casefold_email` for valid emails, `precise_casefold` otherwise.
+- `precis_casefold_email`   PRECIS UsernameCaseMapped profile + U-labels form for domain
+- `precis_casefold`         PRECIS UsernameCaseMapped profile for the entire string
+- `precis_email`            PRECIS UsernameCasePreserved profile + U-labels form for domain
+- `precis`                  PRECIS UsernameCasePreserved profile for the entire string
+- `casefold`                Convert to lower case
+- `noop`                    Nothing
 
 PRECIS profiles are defined by RFC 8265. In short, they make sure
 that Unicode strings that look the same will be compared as if they were
 the same. CaseMapped profiles also convert strings to lower case.
 
 **Syntax:** from\_normalize _action_ <br>
-**Default:** precis\_casefold\_email
+**Default:** auto
 
 Normalization function to apply to email addresses before
 further processing.

docs/reference/endpoints/imap.md

@@ -20,6 +20,10 @@ imap tcp://0.0.0.0:143 tls://0.0.0.0:993 {
     insecure_auth no
     auth pam
     storage &local_mailboxes
+    auth_map identity
+    auth_map_normalize auto
+    storage_map identity
+    storage_map_normalize auto
 }
 ```
 
@@ -62,4 +66,46 @@ Use the specified module for authentication.
 **Syntax**: storage _module\_reference\_
 
 Use the specified module for message storage.
-**Required.**
+**Required.**
+
+**Syntax**: storage\_map _module\_reference_ <br>
+**Default**: identity
+
+Use the specified table to map SASL usernames to storage account names.
+
+Before username is looked up, it is normalized using function defined by
+`storage_map_normalize`.
+
+This directive is useful if you want users user@example.org and user@example.com
+to share the same storage account named "user". In this case, use
+```
+    storage_map email_localpart
+```
+
+Note that `storage_map` does not affect the username passed to the
+authentication provider.
+
+It also does not affect how message delivery is handled, you should specify
+`delivery_map` in storage module to define how to map email addresses
+to storage accounts. E.g.
+```
+    storage.imapsql local_mailboxes {
+        ...
+        delivery_map email_localpart # deliver "user@*" to mailbox for "user"
+    }
+```
+
+**Syntax**: storage\_map_normalize _function_ <br>
+**Default**: auto
+
+Same as `auth_map_normalize` but for `storage_map`.
+
+**Syntax**: auth\_map_normalize _function_ <br>
+**Default**: auto
+
+Overrides global `auth_map_normalize` value for this endpoint.
+
+See [Global configuration](/reference/global-config) for details.
+
+
+

docs/reference/global-config.md

@@ -1,6 +1,6 @@
 # Global configuration directives
 
-These directives can be specified outside of any 
+These directives can be specified outside of any
 configuration blocks and they are applied to all modules.
 
 Some directives can be overridden on per-module basis (e.g. hostname).
@@ -23,6 +23,56 @@ objects. Should be writable.
 Internet hostname of this mail server. Typicall FQDN is used. It is recommended
 to make sure domain specified here resolved to the public IP of the server.
 
+**Syntax**: auth\_map _module\_reference_ <br>
+**Default**: identity
+
+Use the specified table to translate SASL usernames before passing it to the
+authentication provider.
+
+Before username is looked up, it is normalized using function defined by
+`auth_map_normalize`.
+
+Note that `auth_map` does not affect the storage account name used. You probably
+should also use `storage_map` in IMAP config block to handle this.
+
+This directive is useful if used authentication provider does not support
+using emails as usernames but you still want users to have separate mailboxes
+on separate domains. In this case, use it with `email_localpart` table:
+```
+    auth_map email_localpart
+```
+With this configuration, `user@example.org` and `user@example.com` will use
+`user` credentials when authenticating, but will access `user@example.org` and
+`user@example.com` mailboxes correspondingly. If you want to also accept
+`user` as a username, use `auth_map email_localpart_optional`.
+
+If you want `user@example.org` and `user@example.com` to have the same mailbox,
+also set `storage_map` in IMAP config block to use `email_localpart`
+(or `email_localpart_optional` if you want to also accept just "user"):
+```
+    storage_map email_localpart
+```
+In this case you will need to create storage accounts without domain part in
+the name:
+```
+maddy imap-acct create user # instead of user@example.org
+```
+
+**Syntax**: auth\_map_normalize _function_ <br>
+**Default**: auto
+
+Normalization function to apply to SASL usernames before mapping
+them to storage accounts.
+
+Available options:
+- `auto`                    `precis_casefold_email` for valid emails, `precise_casefold` otherwise.
+- `precis_casefold_email`   PRECIS UsernameCaseMapped profile + U-labels form for domain
+- `precis_casefold`         PRECIS UsernameCaseMapped profile for the entire string
+- `precis_email`            PRECIS UsernameCasePreserved profile + U-labels form for domain
+- `precis`                  PRECIS UsernameCasePreserved profile for the entire string
+- `casefold`                Convert to lower case
+- `noop`                    Nothing
+
 **Syntax**: autogenerated\_msg\_domain _domain_ <br>
 **Default**: not specified
 

docs/reference/storage/imapsql.md

@@ -156,6 +156,8 @@ See auth\_normalize.
 **Syntax**: auth\_map **table** <br>
 **Default**: identity
 
+**DEPRECATED:** Use `storage_map` in imap config instead.
+
 Use specified table module to map authentication
 usernames to mailbox names.
 
@@ -165,6 +167,8 @@ auth\_map.
 **Syntax**: auth\_normalize _name_ <br>
 **Default**: precis\_casefold\_email
 
+**DEPRECATED:** Use `storage_map_normalize` in imap config instead.
+
 Normalization function to apply to authentication usernames before mapping
 them to mailboxes.
 

docs/reference/table/email_with_domain.md …cs/reference/table/email_with_domains.mddocs/reference/table/email_with_domain.md renamed to docs/reference/table/email_with_domains.md docs/reference/table/email_with_domain.md renamed to docs/reference/table/email_with_domains.md

@@ -12,8 +12,8 @@ with `table.chain`. Example:
 ```
 modify {
     replace_rcpt chain {
-        email_local_part
-        email_with_domains example.org example.com
+        step email_local_part
+        step email_with_domains example.org example.com
     }
 }
 ```