automx2: Email client configuration made easy

Autoconfig is a proprietary method developed by the Mozilla foundation. It was designed to configure a mail account within Thunderbird, and other email suites like Evolution and KMail have adopted the mechanism.

When a user begins to create a new mail account she is asked to enter her realname and mail address, e.g. alice@example.com. Thunderbird will then extract the domainpart (example.com) from the mail address and build a list of URIs to search for a configuration web service in the following order:

A configuration service such as automx2 listening on one of the listed URIs will receive the request, process it and respond with a set of configuration instructions.

Thunderbird will use the instructions to automatically fill in the required fields in the account. The only remaining task for the user is to confirm the settings. After that she can immediately start to use her new mail account.

Autodiscover is a proprietary method developed by Microsoft. The protocol version supported by automx2 was designed to configure a mail account within Outlook 2016 ff. Service lookups use the URLs shown below and, as a fallback option, DNS lookups. Please note that Microsft uses a different autodiscover mechanism for Office 365, which is not yet supported by automx2 because information about the technical details are not available free of charge.

All HTTP(S) queries send a POST request and submit XML which contains information about the account that should be configured. The DNS queries search for a CNAME resource record first, which is supposed to redirect the mail client to a resource outside the mailbox owners domain, e.g. alice@example.com would be redirected to service.example-provider.com for configuration instructions. If the first DNS query fails the client may be redirected to a configuration service using a SRV RR like this:

The SRV RR used in the example above would send Alice’s client to service.example-provider.com and tell it to send the query to the configuration service on port 443.

Requests and responses use proprietary content types with an underlying property list format. automx2 will return unsigned data, which means that the service must be accessed via HTTPS only. Otherwise, your users have no way of knowing if they access the correct service and will be vulnerable to man-in-the-middle attacks.

There are packages available for Arch Linux, Gentoo Linux and NixOS. Note that these packages may not alway be up-to-date, because Linux distributions have their own release schedules. Please make sure to check the version numbers if you decide to go this route.

This is the method which should always provide you with the latest release. The path /srv/www/automx2 will be used as an example throughout this documentation. The BASH shell commands below should work with any modern Linux distribution.

Next, make sure to either login as the user created above, or change to this user via the 'su' command. This is important to ensure the correct file permissions. Now download the script that will install your automx2 service. Installation requires BASH plus either curl or wget to download additional data. The script will abort if neither of the latter two can be found.

Executing the setup script will create a Python virtual environment called .venv in the current directory. You may pass a custom directory path as an argument to install.sh, if necessary. To ensure a clean slate, installation will abort if the destination path already exists.

The script will automatically download the automx2 Python module and place it inside the newly created virtual environment. It will also create a launch script .venv/bin/flask.sh, which can run automx2 after you prepared the configuration.

If you use pre-built packages, please consult the documentation for your specific Linux distribution. The following applies only to pip based installations:

Change to the directory where automx2 has been installed previously. Activate the virtual environment as usual and use pip’s --upgrade option:

To make configuration more convenient, automx2 supports Mozilla-style placeholders. For example, the string %EMAILADDRESS% in database records will be replaced with the email address specified during the query. While based on a proprietary feature of Autoconfig, automx2 also applies placeholders to Autodiscover and Mobileconfig responses.

The configuration file defines automx2 runtime behaviour, and it specifies the backend automx2 should read mailbox account configuration data from.

During startup automx2 searches for runtime configuration instructions in the following locations. The first match will determine the configuration used.

To specify parameters and options automx2 uses an INI file syntax. The example configuration that ships with automx2 looks like this:

Place the content of the example configuration into one of the configuration locations automx2 looks for and adapt it to your needs. Then configure the database backend with data that suits your setup, as described below.

If you want to verify a vanilla installation of automx2 works, you can populate it with internal test data. Start automx2 as described in section Running automx2 and send the following request to populate your database:

This example assumes you are running automx2 on localhost listening on TCP port 4243, which is the suggested default port.

Once you have populated the database with sample data you can test if automx2 works. Use curl to send an account configuration request for user@example.com:

As shown in the example, make sure to quote the URL as necessary. Otherwise, your command shell might perform pattern matching for characters like the question mark ? (FISH does).

For years I have refused to support passwords in the generated configuration data, because of the risks involved. By the very nature of the process, cleartext passwords are required. This means that passwords can potentially be intercepted in transit, e.g. between a HTTPS proxy and automx2. Even worse, if the receiving user foolishly decides to save a configuration file locally, that file will contain the user’s password in plain text, visible for everybody with read access to the file. Also, depending on how automx2 is configured, cleartext passwords can potentially be written to log files.

My reluctance to implement password support has resulted in some code forks where people added the feature anyway. Recognising the desire for password support, I have decided on a compromise. automx2 version 2024.2 introduces password support, but for the time being only for Apple’s Mobileconfig. This is due to iOS devices downloading configuration profiles to a staging area by default, not offering the user an easy method to save the data elsewhere.

Note that the password support feature is deliberately disabled by default. To opt in, you need to set the environment variable PERMIT_CLEARTEXT_PASSWORDS=I_understand_the_risks. If you don’t know what that means, I suggest you hold off on this feature until you have done some research.

Once the feature is activated, you can send GET requests with additional parameters. Sending a password with support disabled will result in an error log message, and the supplied password will be ignored. URL encoding is very important here. I suggest that you create a HTML landing page for your users, with explanation about the risks inolved. Include a <form> to submit the necessary fields to automx2.

automx2 uses the SQLAlchemy toolkit to access databases. This allows a variety of databases, a.k.a. dialects, to be used, simply by defining the appropriate connection URL.

As mentioned in a previous section, you can use Alembic to create or upgrade your database. You need to start your first run using an empty database for this to work, because Alembic stores versioning information in said database. Database upgrades are based on this information. Follow the steps shown below, setting the RELEASE variable to the GitHub tag or release number of your choice.

Next, change the value for sqlalchemy.url in alembic.ini to match your automx2 configuration. Create an empty database unless you are using SQLite, in which case Alembic will create the database for you. The final steps are activating the automx2 virtual Python environment and invoke make.

You should see output similar to the following:

The following is an example for a OpenRC run script /etc/init.d/automx2 which I use for Gentoo Linux:

If you wish to override any of the settings, copy the following to /etc/conf.d/automx2 and uncomment/change variables according to your needs. This is purely optional.

If your system uses systemd you may want to deploy the following automx2.service unit file from the contrib section and place it in /etc/systemd/system/automx2.service:

Once you have installed the service you need to tell systemd to reload its list of available services:

It should now be able to tell you about a service named automx2:

Next enable and start automx2 using the following command:

You should see automx2 enabled and running:

You are now ready to start testing automx2, as described below.

While logged in as an unprivileged user, change into the installation directory and start the .venv/bin/flask.sh launch script:

Now that automx2 is up and running, you need to configure the web server proxy that will receive requests from the outside and forwards them to automx2.

The following example defines an HTTP server, which will listen for requests to both autoconfig.example.com and autodiscover.example.com. All requests will be forwarded to automx2, which listens on TCP port 4243 in this example. Requests to /initdb are restricted to clients connecting from the local host. The proxy_set_header directives will cause NGINX to pass relevant data about incoming requests' origins.

The following example shows an Apache configuration similar to the one above. ProxyPreserveHost directives will cause apache to pass relevant data about incoming requests' origins.