Finding and removing duplicate files with fdupes
Published on 2009-08-01.
If you have a lot of duplicate files "fdupes" can help you remove them very easily.
On Debian derived systems you can install fdupes using the command:
# apt-get install fdupes
The program is run from the terminal and the options are very basic. To search a directory for duplicate files, simply run the following command:
$ fdupes -r my_dir/
The command will recursively search the directory my_dir
for any duplicate files, and display the results.
You can also create a text file containing the names of you duplicate files running this command:
$ fdupes -r my_dir/ > my_duplicate_files.txt
What I needed was to remove all duplicate files except for one of each file. I also needed a list of the files that was kept and the ones that was removed. To do that I used this command:
$ fdupes -rdN my_dir/ > dubs.txt
The r
option makes fdupes search for files recursively. The d
option makes fdupes delete duplicates. The N
option, when used together with d
, preserve the first file in each set of duplicates and delete the others without prompting the user.
See man fdupes
for further instructions.