Migrating from GMail to Google Apps

A quick guide to migrating contacts, calendar, email and email filters from GMail (user@gmail.com) to Google Apps (user@domain.com).

Calendar Entries

  1. On GMail calendar, goto Settings -> Calendars -> Export Calendars to download a ZIP of your calendars in ICAL format.
  2. On Google Apps calendar, goto Settings -> Calendars -> Import calendar and import the ICAL entries to your calendar.

Contacts

To preserve contact groups, each group needs to be migrated individually.

  1. On GMail contacts, use the Export option to export each group in Google’s CSV format. Then export one more CSV of “Everyone (All Contacts)”
  2. On Google Apps contacts:
    1. Use the Import option to import the CSV of Everyone, leaving the  “Add these imported contacts to: ” option unchecked.
    2. Then use the Import option to import each group in turn, checking the “Add these imported contacts to: New Group… ” option, and enter the name of the group.

Google Sync

At this point you might want to switch Google Sync over to your Google Apps account. Follow the steps here, but enter your Google Apps email address instead of your GMail address. You then need to go through the steps here and sign in with your Google Apps account to select which calendars to sync. Remember to click the “Google Apps User?” link rather than trying to go through the normal GMail login page – your Google Apps email will not work there!

Email Filters

If you have any rules defined to filter/label your incoming mail, you can export these using  “Filter import/export” in GMail labs:

  1. Go to Settings -> Labs and check “Enable” for the “Filter import/export” option on both your GMail and Google Apps account.
  2. In your GMail account, go to Settings -> Filters, click Select All and then Export, and save the XML document somewhere.
  3. In your Gogle Apps account, go to Settings -> Filters and then “Import filters”, select the file you saved in the previous step, and press “Open file”.

Email Delivery

You should now be ready to switch over your email. Hopefully your MX records already point at Google Apps and you have email forwarding set up from Google Apps to your GMail. If you don’t, you should set this up before you start trying to move your email.

Now is when you must commit to migrating by having all your new mail delivered to Google Apps instead of GMail!

  1. Login here, select “Manage this domain”, and ensure that in Users and Groups you have a user set up for the Google Apps email address you want to migrate your GMail to. If you created a group to forward the email address to GMail you will need to delete this group before you can create the user.
  2. Use the Inbox link at the top right to log in to the mailbox for the user (signing out if you are not currently logged in as that user).
  3. Send a test email to your Google Apps email address to ensure that you are receiving email correctly.
  4. Log in to your GMail account, go to Settings -> Forwarding and POP/IMAP and enable forwarding to your Google Apps address.

You should now be receiving all your email in your Google Apps mailbox. You may also need to update POP3/IMAP settings for any mail clients you are using.

Old Emails

The hardest part is now to migrate your old emails from your GMail mailbox to Google Apps. Google provide a number of officially supported options, however none of these are ideal – Mail Fetcher is only intended for new emails rather than migrating old emails, Email Uploader requires you to have all your email in Outlook, and IMAP Mail Migration requires you to upgrade to Google Apps Pro (but looks like a pretty good option other than that).

I decided to go for a free approach, using imapsync based upon these instructions. However I found that imapsync would frequently give up without copying the entire folder. I initially thought this was because the number of messages was too large, so I used the following script to split the mail into chunks by folder and also by date. I also tweaked the regexes to rewrite To/From/CC/BCC headers, and to be a little more tolerant of malformed headers:

#!/bin/bash

LOGFILE=imapsync.log
FOLDERS=( 'Folder1' 'Folder2' 'Folder3' 'Folder4' '...' 'FolderN' 'INBOX' 'All Mail' 'Bin' 'Drafts' 'Starred' 'Sent Mail' )
TIMES=( '--minage 1090' '--maxage 1091 --minage 999' '--maxage 1000 --minage 908' '--maxage 909 --minage 817' '--maxage 818 --minage 726' '--maxage 727 --minage 635' '--maxage 636 --minage 544' '--maxage 545 --minage 453' '--maxage 454 --minage 362' '--maxage 363 --minage 271' '--maxage 272 --minage 180' '--maxage 181 --minage 89' '--maxage 90' )

echo "Starting" > $LOGFILE

for FOLDER in "$@"
do
 for ((j=0;j<${#TIMES};j++))
 do
 TIME=${TIMES[${j}]}

 echo "" >> $LOGFILE
 echo "*** $FOLDER $TIME ***" >> $LOGFILE
 echo "" >> $LOGFILE

 while ! ~/imapsync-1.267/imapsync --host1 imap.gmail.com \
 --port1 993 --user1 user@gmail.com \
 --passfile1 ./passfile --ssl1 \
 --host2 imap.gmail.com \
 --port2 993 --user2 user@domain.com \
 --passfile2 ./passfile --ssl2 \
 --syncinternaldates --split1 100 --split2 100 \
 --authmech1 LOGIN --authmech2 LOGIN \
 --include "$FOLDER" \
 $TIME \
 --useheader "Message-ID" \
 --useheader "Date" --skipsize \
 --regexmess 's/Delivered-To: user\@gmail.com/Delivered-To: user\@domain.com/gi' \
 --regexmess 's/<user\@gmail.com>/<user\@domain.com>/gi' \
 --regexmess 's/^((To|From|Cc|Bcc):.*)user\@gmail.com(.*)$/$1user\@domain.com$3/gim' \
 --regexmess 's/Subject:(\s*)\n/Subject: (no--subject)$1\n/ig' \
 --regexmess 's/Subject: ([Rr][Ee]):(\s*)\n/Subject: $1: (no--subject)$2\n/gi' >> $LOGFILE 2>&1; do
 echo "" >> $LOGFILE
 echo "***** NOT COMPLETE - $FOLDER $TIME *****" >> $LOGFILE
 echo "" >> $LOGFILE
 tail -100 $LOGFILE | mail -s "Imapsync Restarting for $FOLDER $TIME" "user@domain.com"

 echo -n "Sleeping..." >> $LOGFILE
 sleep 1m
 echo "Done." >> $LOGFILE
 done

 echo "" >> $LOGFILE
 echo "***** COMPLETE - $FOLDER $TIME*****" >> $LOGFILE
 echo "" >> $LOGFILE
 tail -100 $LOGFILE | mail -s "Imapsync Complete for $FOLDER $TIME" "user@domain.com"
 done
done

echo "**** DONE ****" >> $LOGFILE
tail -100 $LOGFILE | mail -s "Imapsync Complete" -c "user@domain.com"

The script writes to a logfile, and emails the last 100 lines upon completion. I found this useful as some messages could not be copied. After some debugging I was able to locate these by searching for the Message-ID in my GMail to discover that they were all old messages imported from Outlook which had corrupted headers (newlines inside the header, so the Message-ID could not be found by imapsync). Manually deleting the corrupted messages from my GMail allowed the sync process to complete.

15 Comments

  1. Posted June 17, 2009 at 8:23 am | Permalink

    By far the easiest way to migrate emails is using the IMAP Mail Migration tool in the Premier and Education editions.

    You can get a free trial from Google, do the migration and revert to Standard Edition (free) within 30 days and pay nothing. I have done this many times for clients.

    Another option is to use the free program at gmail-backup.com

    Regards.

    Paul Myatt

  2. Gilles Lamiral
    Posted June 17, 2009 at 8:48 am | Permalink

    Hello James,

    Can I recopy your doc in the imapsync FAQ or future wiki (with the reference url) ?

    Thanks in advance.

  3. Posted June 17, 2009 at 9:15 am | Permalink

    Sure – feel free.

  4. Posted July 13, 2009 at 12:11 am | Permalink

    Easiest way by far is to use the POP feature:

    1. In your Gmail account (From where you want to import all your emails to your Google Apps statndard account) go to ‘Settings’
    2. Then click on ‘Forwarding POP/IMAP’ tab.
    3. Then select ‘Enable POP for all mail (even mail that’s already been downloaded)select allow POP for all’
    4. Now go to your Google Apps account.
    5. Go to ‘Settings’
    6. The click on ‘Accounts’ tab
    7. In the section ‘Get mail from other accounts:’ Click on the link ‘Add a mail account you own’
    8. A new window pops up, just follow the process of providing your Gmail Account information
    9. Probably 2nd page into it, it will ask for username, password, etc. and it will have a drop down menu next to Pop server: select the 1st ‘pop.gmail.com’ in the list
    10. do not check mark leave messages as gmail does not support it

    11. Check mark label and give it a good label like: ‘My Gmail’ or something.
    12. Then click Add Account and it will go through the verification. Once you verify the account. You are done.

    13. Now you can go back to the inbox and you should see your ‘My Gmail’ label. Click on it and it should download all your emails from your Gmail account. Please note that you will loose all the emails in your Gmail Account.

    14. Final step is to then go back to the Gmail Account and disable POP download and enable forwarding to your google apps email account.

    Hope this helps all the google apps users out there. Google Apps ROCKS….

    Please contact me at kimnkhan@gmail.com if you need google apps related support.

  5. Posted July 13, 2009 at 6:56 am | Permalink

    You can certainly do it this way, however I believe you would lose your labels and the starred/non-starred status of the messages which is why I chose to use imapsync. Also using imapsync allows you to transform the headers so messages sent to your GMail account will still say “To: me” rather than showing your old GMail address.

  6. Posted June 6, 2010 at 11:07 pm | Permalink

    We help small businesses migrate to Google Apps and we also ran into this problem recently. We have created a simple web interface that does all the migration in the background without installing any software. It can also be used to migrate from/to regular Gmail accounts.

    Feel free to give it a try here:

    https://apps.improffice.com/migrationbeta/

  7. Frankie Magic
    Posted July 9, 2010 at 6:14 am | Permalink

    Thanks for this post James.

    The easiest way to migrate or move all of your old email across from your gmail account (as you mentioned) is to use Google’s IMAP MIGRATION TOOL. As Paul mentioned, you can only get this tool by upgading to a premium account. You can do this for free by opting for a free 30 day trial. Unfortunately, you need a credit card to get the trial but if you cancel within 30 days, your card doesn’t get charged anything. Great to get your gmail emails across to your new Google Apps account and then go back to a standard (free) account.

    I see there are no instructions to using the imap migration tool so here they are;

    If you are in the email section of google apps click on “Manage This Domain” or just go to your control panel.

    Click on “Advanced Tools”

    1. Scroll down and click on “Set up mail (IMAP) migration”

    2. Enter this information:
    – “Name of this server connection:” – You can name it anything you want, eg. GmailToApps (spaces are not allowed)
    – “Server Software” – Other
    – “Host” – imap.googlemail.com
    – “Security” – SSL
    – “Port” – 993
    – “IMAP Path Prefix” – Leave this text box empty
    – “Allow up to “3”connections(3) to this server at any given time
    3. Press “Save Settings”
    4. Select “Specify a few user accounts”
    5. Follow the rest of the instructions.

    I hope this will help someone.

    Best wishes, Frankie Magic

  8. Jeremy
    Posted August 6, 2010 at 4:22 pm | Permalink

    My folder names had spaces, so for FOLDER in “$@” doesn’t do the trick. I messed around, maybe got it figured out, but it wouldn’t find the messages– so I decided to try Paul’s advice.

    After spending two hours getting the script to a point where I could run it, and then switching to the “just do the free upgrade” trick, I have to agree that “By far the easiest way to migrate emails is using the IMAP Mail Migration Tool in the Premier… edition.”

    Of course, don’t forget to cancel– I have 8 users here so I’d get hit with a $400 fee– but we really don’t need any of the other features with Premier. The Migration Tool makes it *simple* — and it properly creates all the needed folders. As far as I can tell, even the stars are being copied.

    Now, I don’t think it’s doing the “me” change that regexmess is facilitating- but my 8 little users can do without that!

  9. Travis Harris
    Posted November 20, 2010 at 6:52 pm | Permalink

    imapsync is no longer free. The most recent version I could find for download was 1.241 does anyone have a more recent version?
    Also, I had to change the mail commands and the “$@” on my system to get it to work.

  10. Travis Harris
    Posted November 20, 2010 at 9:15 pm | Permalink

    I found the binary for 1.315 here :
    http://mirrors.linux.edu.lv/ftp.freebsd.org/ports/i386/packages-6-stable/All/imapsync-1.315_1.tbz

  11. Travis Harris
    Posted November 21, 2010 at 2:20 pm | Permalink

    Sorry to keep posting over and over… but I have now uploaded the source to here : 1.350 (the binary would not work on my system)
    https://s3.amazonaws.com/imapsync/imapsync-1.350.tar.gz

  12. Alis
    Posted December 3, 2011 at 6:37 pm | Permalink

    Pretty good, I appreciate to all of the community to share their thoughts. I agree Trying Preimier Trial is way to go. But nothing was said about google doc, I have heaps their in different collections. Which I want to take over.

  13. Posted December 4, 2011 at 2:17 pm | Permalink

    Made a video how to easily migrate mail box with labels, should save you a lot of time.

    http://www.youtube.com/watch?v=SYzBbTz6xcs

  14. KelAt
    Posted April 21, 2012 at 9:50 pm | Permalink

    Lots of these posts have been claiming to show the easiest method. My personal nomination for the easiest and best method is http://www.gmail-backup.com. Super easy and works great.

  15. Posted September 18, 2012 at 1:37 pm | Permalink

    Hello, I think your site might be having browser compatibility issues.
    When I look at your blog in Ie, it looks fine but when opening in
    Internet Explorer, it has some overlapping. I just wanted to
    give you a quick heads up! Other then that, wonderful blog!

3 Trackbacks

  1. […] Gmail data (contacts, filters, e-mails, and all that jazz) to your Google Apps account. Instead, you need to go through all of this. So what I thought would take a max of 24 hours, mostly due to my host/Google recognizing MX […]

  2. […] Gmail data (contacts, filters, e-mails, and all that jazz) to your Google Apps account. Instead, you need to go through all of this. So what I thought would take a max of 24 hours, mostly due to my host/Google recognizing MX […]

  3. […] https://base6.com/2009/06/15/migrating-from-gmail-to-google-apps/ […]