Spin up Your Own Mail Server inside a VPS

email inbox server vps

If you have a custom domain name of your own, it is likely that you are using an email using the same domain name. (Like [email protected]). In order to host the emails, you can either use a managed email host (like Google Suite or Zoho) or you can have your own mail server that lets you create and manage email addresses on your own. In this post, I am going to detail how you can run your own mail server inside a VPS (like DigitalOcean).

Why build a mail-server

Using managed email hosts, like Google Suite, is definitely painless. You don’t have to worry about maintaining related infrastructure — Google does that for you. But it costs you $5 per user per month. There is also Zoho which offers free email hosting on custom domains for up to 25 users. But Zoho’s deal-breaker is it does not offer POP/IMAP access for free users — you can browse email only from web-browsers, not using mobile or desktop email clients.

Building a mail-server is a good option for you if you want to save money, manage emails on your own servers and create as many email addresses as you want. Earlier, running mail-servers was a complicated affair. But, with Docker, it is now very easy to build a mail-server without breaking tons of other stuff.

What we are going to build

Here is what you’d have at the end of this tutorial:

  1. Hosting and mail-server running side by side inside a VPS.
  2. Ability to create email accounts with custom domain name.

What choices we have

There are a handful of robust email solutions available online which are open-source, secure and performs well under stress. Here are some of them:

  1. Mailcow — a docker-based email solution that is not only robust but also offers secure and reliable way to send and receive emails across multiple protocols. It has an integrated webmail browser as well.
  2. Mailu — an ‘insular’ mail-server that comes in the form of docker images. Mailu has an integrated email manager that helps you create and manage email accounts. It also supports aliases.
  3. Mailslurper — written in Go, it is a perfect solution if you are a small team or if you are building email enabled web-applications.
  4. Inbucket — it’s a disposable webmail server perfect for testing purposes. Inbucket lets you create email accounts without passwords. These email accounts can then be accessed via web-browser. It is built mostly for testing purposes.
  5. Docker-mailserver — built on top of docker, it offers extensive support for an array of email protocols. Docker-mailserver supports aliases, anti-spam, ClamAV antivirus, LetsEncrypt SSL certs among several other features.

Why choose Docker-mailserver

Docker based mail-servers are more secure than full-blown email solutions. This is because of separation of concerns that docker provides. Due to its modular nature, docker images are pre-configured and needs almost no tweaks while installing.

Among the five different options I listed above, I am going to use Docker-mailserver. Docker-mailserver does not use ports commonly used by web-servers. Mailcow, for example, uses port 443 which is commonly used for SSL connection. So, Mailcow and web-server do not go hand in hand.

A. Getting Started

1. DNS Configuration

For mail-server to have its own domain name, we first need to set up various DNS records. These DNS records not only sets up the sub-domain (or domain, for that matter) but also protects you against Sender Address Forgery.

Let us begin by creating these DNS records:

# Name              Type       Value
mail                IN A       your_server_ip
autodiscover        IN CNAME   mail
autoconfig          IN CNAME   mail

@                   IN MX 10   mail

(Here I assume you’re creating your mail-server on mail.yourdomain.tld. The autodiscover and autoconfig records let email clients discover email settings when you just enter mail.yourdomain.tld.

To protect against sender address spoofing, create this TXT record:

@                   IN TXT     "v=spf1 mx ~all"

Further, to report authentication failures, create this TXT record:

_dmarc              IN TXT     "v=DMARC1; p=reject; rua=mailto:[email protected]"

Here, [email protected] will receive notifications if authentication fails for a certain user. (Although you have not yet created any email address yet, set this to the email which you will create afterwards.)

Next, the following DNS records will set your mail-server up for various email protocols on different ports:

_imap._tcp          IN SRV     0 1 143   mail.yourdomain.tld.
_imaps._tcp         IN SRV     0 1 993   mail.yourdomain.tld.
_pop3._tcp          IN SRV     0 1 110   mail.yourdomain.tld.
_pop3s._tcp         IN SRV     0 1 995   mail.yourdomain.tld.
_submission._tcp    IN SRV     0 1 587   mail.yourdomain.tld.
_smtps._tcp         IN SRV     0 1 465   mail.yourdomain.tld.
_carddavs._tcp      IN TXT               "path=/SOGo/dav/"
_caldavs._tcp       IN SRV     0 1 443   mail.yourdomain.tld.
_caldavs._tcp       IN TXT               "path=/SOGo/dav/"

Once these DNS records are created, test them on MXToolbox.

2. Installing Docker

We’d need Docker to be installed inside the VPS to host the mail-server. You can follow this installation guide available on the official Docker site. You’d need Docker Compose as well. So install them now, if you have not yet.

3. Installing the mail-server

Once you have installed Docker and Docker Compose, it is fairly easy to install the mail-server. It is a good practice to use /opt for the mail-server files. Run these commands in a terminal:

cd /opt
mkdir mail-server # any other directory name you like
cd mail-server
docker pull tvial/docker-mailserver:latest
curl -o; chmod a+x ./
curl -o docker-compose.yml
curl -o .env

The curl commands downloads the tools necessary for running the mail-server.

4. Configuration

The last curl commands mentioned above downloads .env. Almost all configuration is saved inside this file. I have left most other options untouched, except for the following where you can set the sub-domain for mail-server.


B. Starting the mail-server

Just run this command from /opt/mail-server directory:

docker-compose up -d mail

And our mail-server is up and running.

C. Creating email accounts

In order to create an email account, use this:

./ email add [email protected] password

This sets up a new email account with the password you specify. There are several other things you can do with ./ Just run ./ for all the available commands.

D. Accessing emails

One of the trade-offs of Docker mail-server is that it does not have an integrated email client. That is, in order to read and send emails, you have to rely on email clients like Mozilla Thunderbird or the email apps on your mobile device. Because we have set up autodiscover and autoconfig DNS records, the email clients will easily find the necessary settings from our mail-server.

It is important to note here that while configuring you email client, always enter the full email address (which you have created earlier with ./ ) as username. Otherwise they may not work and will give a pesky username-password mismatch error.

E. Troubleshooting

Since we have web-server and mail-server inside the same VPS, sendmail function may not work properly. Other web-appllications, like WordPress may not be able to send out emails. You can read the logs in /var/log/mail.err. I found the following in my mail.err file:

MyVPS_Hostname sendmail[460]: My unqualified host name (MyVPS_Hostname) unknown; sleeping for retry
MyVPS_Hostname sendmail[460]: unable to qualify my own domain name (Main) -- using short name

To fix this, make sure your /etc/hosts file looks like this: localhost yourdomain.tld MyVPS_Hostname

Also, make sure these ports are not blocked by any firewall.


Docker mailserver‘s tagline reads “…Keep it simple and versioned. Easy to deploy and upgrade.” Indeed, using this tool, you don’t have to configure too many things — in our case, we just changed the host-name. I like how it offers enterprise level architecture without having complicated setup process. Also, it is actively developed and documentation is detailed.