I first mentioned this idea yesterday. This morning I started implementing it.
UPDATE: this strategy is mentioned in Scenic BGP Route | TechSNAP 137 at about 0:47:30.
I back up my mail. On a regular and frequent basis. However, I wanted something more. What about the email which comes in between backups? How can I capture them and restore them after restoring my backup? Yesterday I had an idea: Keep a copy of each incoming email.
The key points are:
- This is a solution for my personal email only.
- Copies will be redirected to another server
- The copies will be kept for a limited time.
- The copies are not needed indefinitely because the main backups supersede them.
Let’s assume that you backup your email once a day (I backup every two hours). You then have a maximum window of 24 hours of lost mail. My proposed solution involves keeping a copy of incoming mail for a 24-hour period. That is not the same as: every email is kept for 24 hours.
The copied email will be stored in ~/$HOME/mail/mbox. An entry in /etc/newsyslog.conf will rotate that file every 24 hours. Thus, after three days, ~/$HOME/mail will contain:
$ ls -l ~/mail/ -rw-r----- 1 dan dan 1941675 Oct 20 18:31 mbox -rw-r----- 1 dan dan 179020 Oct 20 00:00 mbox.0 -rw-r----- 1 dan dan 2025035 Oct 19 00:00 mbox.1 -rw-r----- 1 dan dan 208167 Oct 18 00:00 mbox.2
Should you wish to restore email for a given period, you have 24 hours worth in each file. This mail can then be fed back into procmail for distribution according to your rules.
Create the user for mailbox delivery
There are other ways to do this. You don’t have to create a user for the incoming mail. But I did.
$ sudo adduser Password: Username: mymailcopy Full name: Mail archives for Dan Langille Uid (Leave empty for default): Login group [mymailcopy]: Login group is mymailcopy. Invite mymailcopy into other groups? : Login class [default]: Shell (sh csh tcsh bash rbash nologin) [sh]: Home directory [/home/mymailcopy]: /usr/home/mymailcopy Home directory permissions (Leave empty for default): Use password-based authentication? [yes]: no Lock out the account after creation? [no]: yes Username : mymailcopy Password : <disabled> Full Name : Mail archives for Dan Langille Uid : 1002 Class : Groups : mymailcopy Home : /usr/home/mymailcopy Home Mode : Shell : /bin/sh Locked : yes OK? (yes/no): yes adduser: INFO: Successfully added (mymailcopy) to the user database. adduser: INFO: Account (mymailcopy) is locked. Add another user? (yes/no): no Goodbye!
I installed maildrop. I’m going to use that instead of procmail.
This can probably be done without a LDA (i.e. maildrop). But I didn’t.
Configuring maildrop for delivery to mbox
This is my ~/.mailfilter file:
# Dan Langille # # this directory is for mail archving for short term disaster recovery # TYPE="mbox" logfile "$HOME/.mailfilter.log" DEFAULT="$HOME/mail" to $DEFAULT/mbox
This file is chmod 600, otherwise you’ll be told: maildrop: Cannot have world/group permissions on the filter file – for your own good.
I added this entry to /etc/newsyslog.conf:
/usr/home/mymailcopy/mail/mbox 600 30 * @T00 JB
Getting mail delivered
All the mail is destined for host dups.example.org. On that server, which is not the hostname of that server, I added these entries to /usr/local/etc/main.cf:
virtual_alias_domains = dups.example.org virtual_alias_maps = hash:/usr/local/etc/postfix-config/virtual_alias_maps
And /usr/local/etc/postfix-config/virtual_alias_maps contains:
Sending out a copy of the incoming mail
I use procmail on my IMAP server. That’s where all of my personal mail is delivered. I added this line near the top of my ~/.procmailrc file:
:0c ! firstname.lastname@example.org
Oh, and during disaster recovery, you’ll want to turn off that copy. You don’t want copies of your copies when restoring the missing email into the restore backup.
I have not tried this. It is all theory.
The plan: use procmail and formail to send the mail out again.
I think it is best if you do this from a third account: not the one you use for the copies, and not the one you use for receiving email. It is because the solution involves creating a .procmail recipe and feeding the files through there. In which case, it’s useful to create a new user for this purpose.
After you restore your main backup, you’ll want to replay your missing email. This is untested for this situation, but I have used this approach in the past.
Find the mbox files which represent the missing email. You might want to edit the files to reduce the duplicates. The goal is to reduce the mbox down to just the missing messages. I suggest copying the original files to a working directory, and using operating on those files.
This goes into .procmailrc on the third account:
:0 * ! email@example.com
where firstname.lastname@example.org the address to which you wish to send the mail. This is the email account which has just been restored.
Then you issue this command for each incoming mbox file which needs to be restored:
cat mbox.0 | formail -s procmail
I’ll have to test this soon.
1 thought on “Mail backups using newsyslog.conf for rotation”
One issue which will arise during recovery: mail loop detection:
Somehow, that needs to be disabled during the recovery process.