Mail backups using newsyslog.conf for rotation

I first mentioned this idea yesterday. This morning I started implementing it.

UPDATE: this strategy is mentioned in Scenic BGP Route | TechSNAP 137 at about 0:47:30.

I back up my mail. On a regular and frequent basis. However, I wanted something more. What about the email which comes in between backups? How can I capture them and restore them after restoring my backup? Yesterday I had an idea: Keep a copy of each incoming email.

The key points are:

1. This is a solution for my personal email only.
2. Copies will be redirected to another server
3. The copies will be kept for a limited time.
4. The copies are not needed indefinitely because the main backups supersede them.

Let’s assume that you backup your email once a day (I backup every two hours). You then have a maximum window of 24 hours of lost mail. My proposed solution involves keeping a copy of incoming mail for a 24-hour period. That is not the same as: every email is kept for 24 hours.

The copied email will be stored in ~/$HOME/mail/mbox. An entry in /etc/newsyslog.conf will rotate that file every 24 hours. Thus, after three days, ~/$HOME/mail will contain:

$ ls -l ~/mail/
-rw-r-----  1 dan  dan  1941675 Oct 20 18:31 mbox
-rw-r-----  1 dan  dan   179020 Oct 20 00:00 mbox.0
-rw-r-----  1 dan  dan  2025035 Oct 19 00:00 mbox.1
-rw-r-----  1 dan  dan   208167 Oct 18 00:00 mbox.2

Should you wish to restore email for a given period, you have 24 hours worth in each file. This mail can then be fed back into procmail for distribution according to your rules.

Create the user for mailbox delivery

There are other ways to do this. You don’t have to create a user for the incoming mail. But I did.

$ sudo adduser
Password:
Username: mymailcopy
Full name: Mail archives for Dan Langille
Uid (Leave empty for default): 
Login group [mymailcopy]: 
Login group is mymailcopy. Invite mymailcopy into other groups? []: 
Login class [default]: 
Shell (sh csh tcsh bash rbash nologin) [sh]: 
Home directory [/home/mymailcopy]: /usr/home/mymailcopy 
Home directory permissions (Leave empty for default): 
Use password-based authentication? [yes]: no
Lock out the account after creation? [no]: yes
Username   : mymailcopy
Password   : <disabled>
Full Name  : Mail archives for Dan Langille
Uid        : 1002
Class      : 
Groups     : mymailcopy 
Home       : /usr/home/mymailcopy
Home Mode  : 
Shell      : /bin/sh
Locked     : yes
OK? (yes/no): yes
adduser: INFO: Successfully added (mymailcopy) to the user database.
adduser: INFO: Account (mymailcopy) is locked.
Add another user? (yes/no): no
Goodbye!

Install maildrop

I installed maildrop. I’m going to use that instead of procmail.

This can probably be done without a LDA (i.e. maildrop). But I didn’t.

Configuring maildrop for delivery to mbox

This is my ~/.mailfilter file:

# Dan Langille
#
# this directory is for mail archving for short term disaster recovery
#
TYPE="mbox"

logfile "$HOME/.mailfilter.log"

DEFAULT="$HOME/mail"

to $DEFAULT/mbox

This file is chmod 600, otherwise you’ll be told: maildrop[70958]: Cannot have world/group permissions on the filter file – for your own good.

newsyslog.conf

I added this entry to /etc/newsyslog.conf:

/usr/home/mymailcopy/mail/mbox        600 30     *    @T00  JB

Getting mail delivered

All the mail is destined for host dups.example.org. On that server, which is not the hostname of that server, I added these entries to /usr/local/etc/main.cf:

virtual_alias_domains = dups.example.org
virtual_alias_maps    = hash:/usr/local/etc/postfix-config/virtual_alias_maps

And /usr/local/etc/postfix-config/virtual_alias_maps contains:

mymailcopy@dups.example.org       mymailcopy

Sending out a copy of the incoming mail

I use procmail on my IMAP server. That’s where all of my personal mail is delivered. I added this line near the top of my ~/.procmailrc file:

:0c
! mymailcopy@dups.example.org

Oh, and during disaster recovery, you’ll want to turn off that copy. You don’t want copies of your copies when restoring the missing email into the restore backup.

Disaster Recovery

I have not tried this. It is all theory.

The plan: use procmail and formail to send the mail out again.

I think it is best if you do this from a third account: not the one you use for the copies, and not the one you use for receiving email. It is because the solution involves creating a .procmail recipe and feeding the files through there. In which case, it’s useful to create a new user for this purpose.

After you restore your main backup, you’ll want to replay your missing email. This is untested for this situation, but I have used this approach in the past.

Find the mbox files which represent the missing email. You might want to edit the files to reduce the duplicates. The goal is to reduce the mbox down to just the missing messages. I suggest copying the original files to a working directory, and using operating on those files.

This goes into .procmailrc on the third account:

:0
*
! foo@example.com

where foo@example.com the address to which you wish to send the mail. This is the email account which has just been restored.

Then you issue this command for each incoming mbox file which needs to be restored:

cat mbox.0 | formail -s procmail

I’ll have to test this soon.