Migrating from Google Photos to Apple Photos

In my ongoing effort to de-google my life, I recently moved my entire photos catalog from Google Photos to Apple Photos. The goal of this migration was also to remove duplicates from my library. Somewhen in 2016 I imported photos from my DSLR and Google's uploader mangled something up. I suddenly had duplicates for DNGs, PNGs and JPGs. Photos with wrongly attributed timestamps. A mess.

So here are the rough steps on how I went from this mess to a neatly organized photo library without any duplicates.

Download backup and bring order to chaos #

Getting a complete archive of all my photos and albums on Google Photos is straight forward. Go to takeout.google.com select "Google Photos" and hit the "Export" button.

After 1-2 days, you get an email with a link to download the archive. I've moved the archive to my 4TB Synology NAS and unzipped it there to save storage on my MacBook.

I first started to bring order to the chaos by grouping photos by year and month.
I've used Hazel.app for this. I setup a rule that watches the unzipped folder for images and sorts them into subfolders. The subfolders are named by the date the image was taken (using Date Taken (Hazel)-attribute) in the format YYYY-MM.

Next, I added a rule to apply the "Move to Subfolder"-rule on all folders that were generated by Google Takeout. This way, I don't have to manually move thousands of images into the root folder. I let Hazel do all the work.
Read more on how to process subfolders in the Hazel documentation.

Getting rid of duplicates #

Next on my list was getting rid of all the duplicates before adding the images to Apple Photos. I didn't check every single one of my +25'000 images by hand though. I've used Gemini II for this.

Gemini does a great job of detecing similar looking images and presenting you with a decision screen to select the images you want to keep.
To make review sessions shorter for me, I let Gemini only scan a single "month"-folder at a time. Otherwise I would spent like 1-2 hours reviewing photos from a single year.

So, drag & drop a folder onto Gemini, let it analyse the photos and then go through each detected duplicate image one by one.
A nice feature of Gemini is that after selecting 10-15 images, it detects a pattern and suggests applying it to the current scan.
For example, you always would like to keep the images with a higher resolution. Instead of manually selecting the high resolution photos, Gemini takes care of this step for you.

I let Gemini do its magic, but still manually check each detect duplicate. Just in case there were false positives and I would permanentely delete an image I wanted to keep.

If all images have been reviewed, I hit the "Remove" button and the duplicates are deleted.

Before discovering and sticking to Gemini, I also tried other software solutions. One I would like to highlight here is fdupes.

The CLI detects duplicates inside folders. The following is the bash script I first used to deduplicate my library.

#!/bin/bash

for i in {1..10}
do
    cd "/Volumes/photos/Takeout_Photos/Group$i";
    fdupes . -dN;
done

(My approach was different back then. I grouped photos into buckets of 1000 photos. The folders were numbered like Group10, Group11, etc.)

Another good software I discovered was dupeGuru.

Add photos to Apple Photos and upload library #

After spending a couple of hours sifting through duplicate desicion screens, it was finally time to import the entire library to Apple Photos. Seems easy, right? My entire library was roughly 240GB in size. My MacBook Pro only has 250GB internal storage though. So I've split the import into smaller steps.

I've imported each "year"-folder one by one and let the MacBook run plugged in for a while. While connected to power, the Photos.app uploaded the images quite fast. This worked well … for a while.

After importing ~50% of my library, I got stuck at a gnarly problem. The import stopped working with the error message "Not enough space". However, a quick check in Disk Utility.app showed that I still had 100GB of free storage on my hard drive. Why couldn't it import those 10 GB of images?

I already knew, that I couldn't tell Photos.app to "free up" space manually. Meaning I couldn't tell the app to get rid of the locally cached versions of images, which are already living in the cloud. Apparently the Algorithm™ takes care of this.
After many restarts and internet searches I found the solution: Disable automatic Time Machine backups. Apparently, Time Machine stores local snapshots on disk which can take up a lot of space. Somehow these snapshots don't show up in Disk Utility and are not substracted from free disk space.

Disabling Time Machine and a couple of restarts resolved the problem for me. Import and uploaded worked flawlessly afterwards.

Remove images from Google Photos #

This was the easiest task. I opened the Google Photos.app on my phone and mass selected thousands of photos at a time and hit the delete button.
In a matter of minutes the account was empty.

Was it worth it? #

Probably. I removed all duplicates, have access to a nice Photos.app widget on my homescreens and in general brought oder to my photo library.

I now have images from my infancy in the early 1990s to the current year on my phone. This regularly blows the minds of friends I meet. "What? You still have that photo from our childhood? And on your phone?!".

I just hope Apple doesn't mess up the Photos.app in future updates. 🤞

Next on my list is to create a new backup strategy using AWS Glacier. My plan is to do something similar what Pawel does: Create yearly, encrypted archive folders and put them into AWS Glacier Deep Archive.