dovecot / maildir migration from one server to another

D

dieselnutjob

Hi
I have an existing dovecot server running FreeBSD but the hosting provider is closing down, so I need to migrate to a new server.
The new server is also running FreeBSD.
I don't want to copy all the emails from the old mailserver to the new using a mail client, because currently I am on a mobile connection.
Ideally I would like to just stop the server, tar up everything, sftp it straight to the new one, update my dns and X509 certs and then start the new server.
However, I've never done this before.
My guess is that it isn't as simple as just copying over the maildir folders.
Presumably there's some stuff beyond the maildir folders that somehow logs the the sync state between clients and the server?
The only clients that we (two of us) use are Thunderbird and K-9 clients.
Thanks for any advice.
 
I've never done a file-based migration - out of similar uncertainty you mentioned. Back then, I chose to do it via mail/imapsync.
Unless someone chimes in on the file-level transfer - at least on that end I can confirm imapsync works reasonably comfortable as a risk-avoidance (but unfortunately network based) option.
 
using imapsync wouldn't be so bad, I guess, if I run it directly on one of the servers. I could log and set up a screen (in case my ssh connection over mobile flakes out) and set it going. The actual sync would be server to server so the mobile connection wouldn't be used for that.
 
SirDice said:
Why would the server log the state of the client?
Click to expand...

Ok. My question wasn't sensible.
Here is a more sensible one:-
Does dovecot have a "state" that is stored anywhere outside of the maildir folders?
or
Can I just copy over everyones' maildir folder, copy the the dovecot config, tweak it to the new server, and then start it up in the new place?
 
dieselnutjob said:
Can I just copy over everyones' maildir folder, copy the the dovecot config, tweak it to the new server, and then start it up in the new place?
Click to expand...
I see no reason why this wouldn't work. Dovecot really doesn't do much more than read files and service them over IMAP to clients. As long as the structure of the mail directories stays the same, some ownership, permissions, etc.
 
I guess it's reasonably safe because if something goes wrong the old server is still there with all the data still there and the daemon stopped. It wouldn't be too hard to back out the change.
 
cmoerz said:
I've never done a file-based migration - out of similar uncertainty you mentioned. Back then, I chose to do it via mail/imapsync.
Unless someone chimes in on the file-level transfer - at least on that end I can confirm imapsync works reasonably comfortable as a risk-avoidance (but unfortunately network based) option.
Click to expand...
+1

I did several migrations / forklift-upgrades via imapsync and it works nicely.
However - the last migration I did via imapsync was several years ago and also included the migration to ZFS. Since then I just zfs send|recvd the /var/mail dataset via incremental send|recv until the delta was small enough, then just shutdown the old server, do a last incremental send|recv and start up the new one.
 
If it's a VPS then just backup the entire filesystem and restore it on another location. Depending if it's zfs with send/recieve or if it's UFS with dump/restore over ssh. Then update the network configuration to reflect the new hosting and update your MTA configuration to reflect the new IP.

If you have new server up and running and want to migrate the current maildir you can use doveadm to sync/backup or move the maildir storage between the two running servers.
 
i simply use rsync on destination side

Code: 
rsync -av --delete backup-user@source-server.net:/var/maildirs/ /var/maildirs/

it is simple and it works

also make sure you have valid content in /usr/local/etc/dovecot/

if source and destination path differ, make appropriate changes

you may repeat this rsync command until there are minor updates,
then stop source mail service and update dns
and do your final rsync
start mail server in destination server and check logs (/var/log/maillog)

hope it helps
 
dieselnutjob said:
Ok. My question wasn't sensible.
Here is a more sensible one:-
Does dovecot have a "state" that is stored anywhere outside of the maildir folders?
Click to expand...
This is a legitimate concern, but AFAIK UIDVALIDITY is stored in the maildirs (at least by default) and UID values in the filenames. Provided you give the server the same hostname, and use the same usernames, clients should pickup where they left off.

If you use IMAPsync or the like, some clients will think all the mailboxes have been replaced by new ones and delete all their offline cache. Some downloaders, like getmail, may (depending on configuation) download a full set of duplicates.
 
I started the migration and tried a little experiment with my wife's inbox.
First I configured the new server and got it all running, but with no emails present.
Then I logged into my DNS provider and moved all the mx records from the old server to the new one.
Then I waited for an hour (dns ttl is 1hr).
Then I tested the new email server.
Then I logged into the old server and stopped all the deamons.
I did
cd /home/wife mv Maildir Maildir-migrated tar -czvf Maildir-wife.tgz Maildir-migrated
sftp the tgz file to the new server
On the new server
cd /home/wife tar -xzvf /home/me/Maildir-wife.tgz
Now on the new server I have two folders, Maildir with just a few recent emails, and Maildir-migrated with all the old ones.
So I did
mv Maildir Maidir-new && mv Maildir-migrated Maildir
This worked really well. I reconfigured her laptop and phone mail clients to the new server, by just changing the incoming/outgoing hostnames to the new server and all worked great.
But...
In Maildir-new/new there are four new emails.
I thought that I could just copy them:-
cd /home/wife cp Maildir-new/new/* Maildir/new/
but the problem is that those four emails don't show up in her email client.
Why is that?
 
It probably didn't work because all the dovecot.* files up one directory didn't copy over.
Don't constantly try and manually ship mail from one place to another. Shut the server down and move it all at once. Use another MX record at a lower priority (higher number) to force delivery to the new machine when you shut the other one down.
 
I found the problem. Those four emails belonged to root.
In fact having four emails in cur belonging to root seemed to totally mess up imap syncing.
Anyway chown wife * fixed it
 
concerning the dovecot.* files, there weren't any in Maildir-new
I get the feeling that they are created the first time that an imap client connects?
So provided that one doesn't connect an imap client to the new server before moving Maildir-migrated to Maildir then I think it's fine
 
You must log in or register to reply here.
Back
Top