Setup mail server sendiri memberikan kontrol penuh atas email infrastructure Anda. Artikel ini membahas instalasi dan konfigurasi Postfix (MTA) dan Dovecot (IMAP/POP3 server) untuk membangun mail server yang functional dan secure.

1. Persiapan dan Prasyarat

DNS Configuration

Sebelum setup mail server, pastikan DNS sudah dikonfigurasi dengan benar:

# A Record
mail.example.com     A     YOUR_SERVER_IP
MX Record
example.com          MX    10 mail.example.com
SPF Record
example.com          TXT   "v=spf1 mx a:mail.example.com ~all"
DKIM Record (akan digenerate nanti)
default._domainkey.example.com   TXT   "v=DKIM1; k=rsa; p=MIGfMA0GCSqG..."
DMARC Record
_dmarc.example.com   TXT   "v=DMARC1; p=quarantine; rua=mailto:[email protected]"
PTR Record (Reverse DNS) - Setup di provider
YOUR_SERVER_IP       PTR   mail.example.com

System Preparation

# Update system
sudo apt update && sudo apt upgrade -y
Set hostname
sudo hostnamectl set-hostname mail.example.com
Edit /etc/hosts
sudo nano /etc/hosts
Tambahkan:
YOUR_SERVER_IP mail.example.com mail
Install dependencies
sudo apt install -y \
postfix \
postfix-pcre \
dovecot-core \
dovecot-imapd \
dovecot-pop3d \
dovecot-lmtpd \
mailutils \
libsasl2-modules \
sasl2-bin \
opendkim \
opendkim-tools \
certbot \
python3-certbot-nginx

2. Konfigurasi Postfix

Setup Dasar Postfix

# Reconfigure postfix
sudo dpkg-reconfigure postfix
Pilih:
- Internet Site
- System mail name: example.com
- Root and postmaster mail recipient: admin
- Other destinations: example.com, mail.example.com, localhost.example.com, localhost
- Force synchronous updates on mail queue: No
- Local networks: 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
- Mailbox size limit: 0
- Local address extension character: +
- Internet protocols to use: all

Konfigurasi Main Postfix

Edit /etc/postfix/main.cf:

# Basic configuration
smtpd_banner = $myhostname ESMTP
biff = no
append_dot_mydomain = no
readme_directory = no
TLS parameters
smtpd_tls_cert_file=/etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_tls_security_level = may
smtp_tls_security_level = may
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
SASL authentication
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_local_domain =
smtpd_sasl_auth_enable = yes
broken_sasl_auth_clients = yes
smtpd_sasl_security_options = noanonymous
Restrictions
smtpd_helo_required = yes
smtpd_helo_restrictions = \
permit_mynetworks, \
reject_invalid_helo_hostname, \
reject_non_fqdn_helo_hostname
smtpd_sender_restrictions = \
permit_mynetworks, \
reject_non_fqdn_sender, \
reject_unknown_sender_domain
smtpd_recipient_restrictions = \
permit_mynetworks, \
permit_sasl_authenticated, \
reject_non_fqdn_recipient, \
reject_unauth_destination, \
reject_unknown_recipient_domain, \
check_policy_service unix:private/quota-status
Dovecot LDA
mailbox_command = /usr/lib/dovecot/deliver
Virtual mailbox domains
virtual_transport = lmtp:unix:private/dovecot-lmtp
virtual_mailbox_domains = /etc/postfix/virtual_domains
virtual_mailbox_maps = hash:/etc/postfix/virtual_mailbox
virtual_alias_maps = hash:/etc/postfix/virtual_alias
Size limits
mailbox_size_limit = 0
message_size_limit = 52428800  # 50MB
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
myhostname = mail.example.com
mydomain = example.com
myorigin = $mydomain
mydestination = $myhostname, localhost.$mydomain, localhost
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128

Master.cf Configuration

Edit /etc/postfix/master.cf:

#
# Postfix master process configuration.
#
smtp      inet  n       -       y       -       -       smtpd
submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth
  -o smtpd_sasl_security_options=noanonymous
  -o smtpd_sasl_local_domain=$myhostname
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o smtpd_sender_restrictions=reject_non_fqdn_sender,reject_unknown_sender_domain
  -o smtpd_recipient_restrictions=reject_non_fqdn_recipient,reject_unknown_recipient_domain,permit_sasl_authenticated,reject
smtps     inet  n       -       y       -       -       smtpd
-o syslog_name=postfix/smtps
-o smtpd_tls_wrappermode=yes
-o smtpd_sasl_auth_enable=yes
-o smtpd_sasl_type=dovecot
-o smtpd_sasl_path=private/auth
-o smtpd_sasl_security_options=noanonymous
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
-o smtpd_sender_restrictions=permit_sasl_authenticated,reject
-o smtpd_recipient_restrictions=permit_sasl_authenticated,reject

3. Konfigurasi Dovecot

Dovecot 10-mail.conf

sudo nano /etc/dovecot/conf.d/10-mail.conf

mail_location = maildir:~/Maildir
namespace inbox {
    inbox = yes
}
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
first_valid_uid = 1000
last_valid_uid = 0

Dovecot 10-auth.conf

sudo nano /etc/dovecot/conf.d/10-auth.conf

disable_plaintext_auth = yes
auth_mechanisms = plain login
Authentication dengan system users
passdb {
driver = passwd-file
args = scheme=SHA512-CRYPT /etc/dovecot/users
}
userdb {
driver = passwd-file
args = /etc/dovecot/users
}

Dovecot 10-master.conf

sudo nano /etc/dovecot/conf.d/10-master.conf

# Postfix smtp-auth
unix_listener /var/spool/postfix/private/auth {
    mode = 0666
    user = postfix
    group = postfix
}
Postfix lmtp
unix_listener /var/spool/postfix/private/dovecot-lmtp {
mode = 0600
user = postfix
group = postfix
}
Auth process
unix_listener auth-userdb {
mode = 0600
user = vmail
}

Dovecot 10-ssl.conf

sudo nano /etc/dovecot/conf.d/10-ssl.conf

ssl = required
ssl_cert = </etc/letsencrypt/live/mail.example.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.example.com/privkey.pem
SSL protocols
ssl_min_protocol = TLSv1.2
ssl_prefer_server_ciphers = yes
ssl_cipher_list = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384

4. User Management dan Testing

Create Mail Users

# Create mail user
sudo useradd -m -s /bin/bash user1
sudo passwd user1
Atau menggunakan Dovecot users file
sudo mkdir -p /etc/dovecot
sudo touch /etc/dovecot/users
Generate password hash
doveadm pw -s SHA512-CRYPT
Enter password twice, copy hash
Add user
sudo nano /etc/dovecot/users
Format: username:{SHA512-CRYPT}hash:UID:GID::/home/user/Maildir
user1:{SHA512-CRYPT}$6$rounds=5000$saltsalt$hash...:1000:1000::/home/user1/Maildir

Restart Services

# Test postfix configuration
sudo postfix check
Restart services
sudo systemctl restart postfix
sudo systemctl restart dovecot
Enable services
sudo systemctl enable postfix
sudo systemctl enable dovecot
Check status
sudo systemctl status postfix
sudo systemctl status dovecot

Testing

# Test local delivery
echo "Test email body" | mail -s "Test Subject" [email protected]
Check mail queue
sudo postqueue -p
Flush queue
sudo postfix flush
View logs
sudo tail -f /var/log/mail.log
Test SMTP authentication
telnet localhost 25
EHLO client.example.com
AUTH LOGIN
Enter base64 encoded username dan password
Test IMAP
telnet localhost 143
a login user1 password
b select inbox

Kesimpulan

Setup mail server sendiri memerlukan konfigurasi DNS yang tepat, SSL certificates, dan proper authentication mechanisms. Dengan Postfix dan Dovecot, Anda dapat membangun mail server yang robust dan scalable.

Checklist Setup:
– DNS records configured (A, MX, SPF, DKIM, DMARC)
– SSL certificates installed
– Postfix configured dengan TLS
– Dovecot configured dengan SSL
– User authentication working
– Testing completed
– Monitoring setup

Security Considerations:
– Always use TLS/SSL
– Implement SPF, DKIM, dan DMARC
– Monitor failed login attempts
– Keep software updated
– Use strong passwords
– Implement rate limiting

Monitoring:
– Check mail queues: postqueue -p
– Monitor logs: /var/log/mail.log
– Setup alerts untuk failed deliveries
– Monitor disk space untuk mail storage