MailPump

A service that monitors a mailbox for messages and will automatically move them to another, usually on a different server.

Used to liberate your mail from proprietary providers that don’t provide automatic forward-and-delete functionality, i.e. most of them.

Download

Source Code

Usage

NAME:
   mailpump run - Run the pump

USAGE:
   mailpump run [command options] [arguments...]

OPTIONS:
   --source-url value                   source url [$MAILPUMP_SOURCE_URL]
   --source-auth-method value           source auth method (default: "normal") [$MAILPUMP_SOURCE_AUTH_METHOD]
   --source-username value              source imap username [$MAILPUMP_SOURCE_USERNAME]
   --source-password value              source imap password [$MAILPUMP_SOURCE_PASSWORD]
   --source-password-file value         source imap password file [$MAILPUMP_SOURCE_PASSWORD_FILE]
   --source-tls-skip-verify             skip source tls verification (default: false) [$MAILPUMP_SOURCE_TLS_SKIP_VERIFY]
   --source-transport value             source imap transport (persistent, standard) (default: "persistent") [$MAILPUMP_SOURCE_TRANSPORT]
   --source-debug value                 display source debug info (default: "persistent") [$MAILPUMP_SOURCE_DEBUG]
   --source-oauth2-provider value       source oauth2 provider (custom) (default: "custom") [$MAILPUMP_SOURCE_OAUTH2_PROVIDER]
   --source-oauth2-client-id value      source oauth2 client id [$MAILPUMP_SOURCE_OAUTH2_CLIENT_ID]
   --source-oauth2-client-secret value  source oauth2 client secret [$MAILPUMP_SOURCE_OAUTH2_CLIENT_SECRET]
   --source-oauth2-token-url value      source oauth2 token url [$MAILPUMP_SOURCE_OAUTH2_TOKEN_URL]
   --source-oauth2-scopes value         source oauth2 scopes [$MAILPUMP_SOURCE_OAUTH2_SCOPES]
   --dest-url value                     dest url [$MAILPUMP_DEST_URL]
   --dest-auth-method value             dest auth method (default: "normal") [$MAILPUMP_DEST_AUTH_METHOD]
   --dest-username value                dest imap username [$MAILPUMP_DEST_USERNAME]
   --dest-password value                dest imap password [$MAILPUMP_DEST_PASSWORD]
   --dest-password-file value           dest imap password file [$MAILPUMP_DEST_PASSWORD_FILE]
   --dest-tls-skip-verify               skip dest tls verification (default: false) [$MAILPUMP_DEST_TLS_SKIP_VERIFY]
   --dest-transport value               dest imap transport (persistent, standard) (default: "persistent") [$MAILPUMP_DEST_TRANSPORT]
   --dest-debug value                   display dest debug info (default: "persistent") [$MAILPUMP_DEST_DEBUG]
   --dest-oauth2-provider value         dest oauth2 provider (custom) (default: "custom") [$MAILPUMP_DEST_OAUTH2_PROVIDER]
   --dest-oauth2-client-id value        dest oauth2 client id [$MAILPUMP_DEST_OAUTH2_CLIENT_ID]
   --dest-oauth2-client-secret value    dest oauth2 client secret [$MAILPUMP_DEST_OAUTH2_CLIENT_SECRET]
   --dest-oauth2-token-url value        dest oauth2 token url [$MAILPUMP_DEST_OAUTH2_TOKEN_URL]
   --dest-oauth2-scopes value           dest oauth2 scopes [$MAILPUMP_DEST_OAUTH2_SCOPES]
   --log-level value                    log level (default: "info") [$MAILPUMP_LOG_LEVEL]
   --log-format value                   log format (text/json) (default: "text") [$MAILPUMP_LOG_FORMAT]
   --idle-fallback-interval value       fallback poll interval for servers that don't support IDLE (default: 1m0s) [$MAILPUMP_IDLE_FALLBACK_INTERVAL]
   --batch-size value                   deletion batch size (default: 15) [$MAILPUMP_BATCH_SIZE]
   --fetch-buffer-size value            fetch buffer size (default: 20) [$MAILPUMP_FETCH_BUFFER_SIZE]
   --fetch-max-interval value           maximum interval between fetches. can abort IDLE (default: 5m0s) [$MAILPUMP_FETCH_MAX_INTERVAL]
   --help, -h                           show help (default: false)

Authentication

MailPump supports three authentication methods: LOGIN, PLAIN, and OAUTHBEARER, which should be passed to the --source-auth-method and --dest-auth-method parameters:

LOGIN

The LOGIN authentication method corresponds to the IMAP LOGIN command1.

Example parameters:

ParameterValueRequired
*-auth-methodLOGINYes
*-usernamejoe.bloggsYes
*-passwordPassW0Rd1Yes

PLAIN

The PLAIN authentication method corresponds to the SASL PLAIN mechanism2.

Example parameters:

ParameterValueRequired
*-auth-methodPLAINYes
*-usernamejoe.bloggsYes
*-passwordPassW0Rd1Yes

OAUTHBEARER

The OAUTHBEARER authentication method corresponds to the SASL OAUTHBEARER mechanism3.

MailPump can be configured to use a custom OAuth2 provider. Built-in providers may be added in future releases.

ParameterExampleRequired
*-oauth2-providercustomYes
*-oauth2-client-idmailpumpIf *-oauth2-provider=custom
*-oauth2-client-secretd2baf3d2-5810-4dd1-afec-3a0101b28980If *-oauth2-provider=custom
*-oauth2-token-urlhttps://server.example.com/oauth2/tokenIf *-oauth2-provider=custom
*-oauth2-scopes4imapIf *-oauth2-provider=custom
*-usernamejoe.bloggsYes
*-password5b2F1dGgyLXJlZnJlc2gtdG9rZW4KYes

OAuth2 Login

The mailpump oauthlogin command will initiate an OAuth2 login and print a token suitable for passing to *-password.

NAME:
   mailpump oauthlogin - Generate an OAuth2 Token

USAGE:
   mailpump oauthlogin [command options] [arguments...]

OPTIONS:
   --provider value       provider (custom) (default: "custom") [$MAILPUMP_PROVIDER]
   --client-id value      client id [$MAILPUMP_CLIENT_ID]
   --client-secret value  client secret [$MAILPUMP_CLIENT_SECRET]
   --token-url value      token url [$MAILPUMP_TOKEN_URL]
   --scopes value         scopes [$MAILPUMP_SCOPES]
   --help, -h             show help (default: false)

Provider URL Examples

ProviderURL
Genericimap[s]://hostname[:port]/mailbox/path
Migaduimaps://imap.migadu.com/INBOX
Yahoo!imaps://imap.mail.yahoo.com/INBOX
Outlookimaps://outlook.office365.com/INBOX
GMailimaps://imap.gmail.com/INBOX

MailPump Multi

This describes MailPump’s multi mode. In this mode, multiple sources are “pumped” to a single destination, potentially relieving load on the target server.

Usage

NAME:
   mailpump run-multi - Run the experimental many-to-one pump

USAGE:
   mailpump run-multi [command options] [arguments...]

OPTIONS:
   --config value, -c value  path to configuration file, or '-' to read from stdin (default: "config.json")

Configuration Reference

Option (JSON Pointer)TypeDescription
/destinationConnection ConfigDestination server configuration.
/source/${name}Source ConfigSource server configuration.

Source Config

Option (JSON Pointer)TypeExampleDescription
/connectionConnection ConfigSource server configuration.
/target_mailboxstringINBOXName of the mailbox on the destination server.
/idle_fallback_intervalinteger, nanoseconds60000000000Fallback poll interval in the event that the server doesn’t support IDLE.
/batch_sizeinteger15No. messages to cache before ingesting.
/disable_deletionsboolfalseDebug flag, disables deletions from the source. Be VERY careful.
/fetch_buffer_sizeinteger20No. messages to fetch at a time.
/fetch_max_intervalinteger, nanoseconds30000000000Interval at which to poll for messages and flush cached messages, regardless of IDLE status.

Connection Config

Option (JSON Pointer)TypeExampleDescription
/urlstringimaps://imap.gmail.comIMAP Server URL
/usernamestringjoe.bloggsUsername
/auth_methodstringLOGINSee here.
/passwordstringPassW0Rd1See here.
/password_filestring/path/to/my-passwordSee here.
/systemd_credentialstringmy-credential-nameSee below.
/tls_skip_verifyboolfalseSkip TLS peer & hostname verification.
/transportstringpersistent, or standardIMAP transport implementation to use.
/debugboolfalseEnable IMAP session debug logging.
/oauth2Temporarily unsupported in multi-mode.

systemd Note

MailPump provides support for systemd’s LoadCredential=. If the systemd_credential configuration option is set, then the provided value will be searched for relative to $CREDENTIALS_DIRECTORY.

Example Configuration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
{
  "destination": {
    "systemd_credential": "destination",
    "tls_skip_verify": false,
    "transport": "persistent",
    "url": "imaps://imap.migadu.com",
    "username": "my-user@example.com"
  },
  "sources": {
    "au-com-yahoo-joebloggs": {
      "batch_size": 15,
      "connection": {
        "password": "my-insecure-password",
        "transport": "persistent",
        "url": "imaps://imap.mail.yahoo.com/INBOX",
        "username": "my-user@yahoo.com"
      },
      "fetch_buffer_size": 20,
      "fetch_max_interval": 300000000000,
      "idle_fallback_interval": 60000000000,
      "target_mailbox": "INBOX"
    },
    "au-com-yahoo-vs49688-junk": {
      "batch_size": 15,
      "connection": {
        "password_file": "/path/to/my-password",
        "tls_skip_verify": true,
        "transport": "persistent",
        "url": "imaps://imap.mail.yahoo.com/Bulk",
        "username": "my-user@yahoo.com"
      },
      "fetch_buffer_size": 20,
      "fetch_max_interval": 300000000000,
      "idle_fallback_interval": 60000000000,
      "target_mailbox": "Junk"
    }
  }
}

License

Copyright © 2022 Zane van Iperen

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License version 2, and only version 2 as published by the Free Software Foundation.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA


  1. https://datatracker.ietf.org/doc/html/rfc3501#section-6.2.3 ↩︎

  2. https://datatracker.ietf.org/doc/html/rfc4616 ↩︎

  3. https://datatracker.ietf.org/doc/html/rfc7628 ↩︎

  4. This may be specified multiple times to add multiple scopes. When configuring via environment variable, separate the values with commas. ↩︎

  5. This should be a base64-encoded OAuth2 Refresh Token. ↩︎