Workflow

Find Duplicate Photos and Videos Safely (Without Deleting the Wrong Copy)

Duplicate finders are good at finding identical files and bad at knowing which copy matters. Here is how to tell true duplicates from files that only look alike, which camera files are never duplicates, and a quarantine-first cleanup with Apple Photos, Immich, fclones or Czkawka.

All My Cameras

— 4 min read
Share

Split the job in two. Exact duplicates, files that are identical byte for byte, can be found by checksum and cleaned up with confidence. Look-alikes, such as a resized copy, an edited version or a burst of near-identical frames, need a human to choose. Whatever tool you use, move duplicates to a quarantine folder rather than deleting them, and only empty it once you are sure.

Where duplicates come from

CauseExact or look-alike?
Google Takeout writes photos to year folders and again to each album folderExact
The same card copied twice, into two foldersExact
Phone photos backed up to a cloud and also copied to a drive, then mergedUsually exact, sometimes not if the cloud converted them
A photo downloaded from a cloud service or a chat appLook-alike: often recompressed or with metadata stripped
Edited copies (IMG_1234-edited.jpg from Takeout, exports from an editor)Look-alike, and usually one you want to keep
Bursts and quick repeat shotsLook-alike

What is not a duplicate

Several camera files look like copies and are not. Do not let a tool clean these up:

  • RAW and JPEG pairs. DSC00012.ARW and DSC00012.JPG are the same shot in two formats. The RAW holds far more data.
  • Proxy and preview files. DJI's .LRF, GoPro's .LRV, Insta360's .lrv and Sony's SUB proxies are small copies of a video. A "similar videos" tool may group them with the original. Keep the original, decide separately about proxies. See our guides to DJI, GoPro, Insta360 and Sony files.
  • Video chapters. GoPro's GX010123 and GX020123 are two halves of one take, not two copies.
  • Same name, different file. Two DJI_0001.MP4 from different flights. A name match proves nothing; only a checksum match does.

The safe routine

  1. Back up first. Clean-up tools act fast and in bulk. Make sure everything is on a second drive before you start.
  2. Decide which copy wins before you run anything. Usually: the one inside your organised folder structure, with its original name and metadata.
  3. Move, do not delete. Send duplicates to a quarantine folder on the same drive.
  4. Check the result: browse the cleaned folders, open some files.
  5. Empty the quarantine weeks later, not minutes.

Inside Apple Photos (Mac, iPhone, iPad)

Photos finds duplicates on its own. On a Mac, click Duplicates under Utilities in the sidebar. Select rows and click Merge. Apple says one photo or video remains where the duplicates were, and the removed copies go to Recently Deleted, where they can be recovered for 30 days.

This is the right tool if your library lives in Photos. It does not touch files in folders on a drive.

Inside Immich

Immich has a duplicates utility that uses machine learning to group visually similar items, so it catches look-alikes as well as exact copies. You review each group and choose what to keep. Immich's own documentation says that when you keep exactly one item, it merges albums, favourites, tags and descriptions from the trashed ones onto the one you keep. See moving Google Photos to Immich.

Folders on a drive: exact duplicates with fclones

fclones is a free, open-source command-line tool that finds groups of identical files, by content, quickly, including on hard drives. It also has a --dry-run option that shows what would change before anything does. On a Mac with Homebrew:

brew install fclones

Find duplicates and save the report:

fclones group "/Volumes/Archive/Footage" > dupes.txt

Read dupes.txt. Each group lists identical files. Then move the extra copies to a quarantine folder, protecting the copies you want to keep by path:

mkdir -p "/Volumes/Archive/_quarantine"
fclones move --dry-run "/Volumes/Archive/_quarantine" \
  --keep-path '/Volumes/Archive/Footage/2026/**' < dupes.txt

When the dry run lists the files you expect, run it again without --dry-run. Other useful options from the fclones documentation: --priority newest or oldest to decide which copy goes, --keep-name to protect patterns such as '*.ARW', and --name on group to look only at certain file types.

No install: list exact duplicates with built-in tools

On a Mac or Linux, this lists every file whose checksum matches another file's. It reads every byte, so it is slow on a large video archive, but it needs nothing extra and changes nothing:

cd "/Volumes/Archive/Footage"
find . -type f ! -name '.*' -print0 | xargs -0 shasum -a 256 > ~/hashes.txt
awk '{c[$1]++; l[NR]=$0; h[NR]=$1}
     END {for (i=1;i<=NR;i++) if (c[h[i]]>1) print l[i]}' ~/hashes.txt \
  | sort > ~/duplicates.txt

Lines with the same checksum at the start are identical files. Decide which to keep, and move the rest by hand.

Look-alikes: Czkawka and Krokiet

Czkawka is a free, open-source app for Windows, macOS and Linux. Its tools include exact duplicates by hash, similar images (different resolution or watermarks) and similar videos. Its newer interface is called Krokiet; the project says the older Czkawka GTK interface's version 12.0 is its last release and recommends Krokiet.

Use the similar-image tool on folders of downloads, chat saves and old exports, where recompressed copies pile up. Review every group yourself: the tool cannot tell an edited favourite from a throwaway. Leave camera originals out of a similar-images scan unless you want to review every burst.

Which copy to keep

  • The original over a download. Cloud downloads may be recompressed or have metadata changed.
  • The one with metadata. If one copy has the right date and location and another does not, keep the first. Takeout copies whose dates were fixed (see fixing Takeout dates) are no longer byte-identical to unfixed copies, so a checksum tool will not pair them.
  • The one in the right place. Keep the copy in your folder structure; remove the stray.
  • Edits are not duplicates. Keep the original and the edit unless you are sure you will never want the edit again.

Stop duplicates at the source

Most duplicate messes come from copying the same card or export more than once into different places. Copy each card once, into a new dated folder per camera, verify it, then format the card in the camera. Our verified copy guide covers the routine.

Read more