Your Windows 11 PC is swamped with duplicates, but the fix is ​​already integrated

If you are tired of downloading countless utility apps to manage storage on your PC, you will be happy to know that there is a way to identify duplicate files on your device using only the built-in tools. After all, incidents like the recent hijacking of the CPUID download site show that even trusted sources can carry risks.

Since Windows doesn’t have a dedicated GUI utility to find duplicate files, the trick uses a handy PowerShell script. No downloads or setup required. Since it doesn’t require any third-party tools, it’s the perfect solution for privacy-conscious users who prefer to avoid downloading additional tools.

How to make Powershell identify duplicate files

You just need to enter the correct script

PowerShell users have created various scripts to help identify duplicate files over the years. This specific script calculates the hash of files in a specific folder, then groups files that have the same hash (and therefore the same content). This is useful because it does not rely on file names, which you may have changed.

It then lists all the groups in which the same hash is detected more than once. You can see the full script below:

Get-ChildItem -Path "$env:USERPROFILEDownloads" -Recurse -File |
  Get-FileHash |
  Group-Object Hash |
  Where-Object Count -gt 1 |
  ForEach-Object {
    "`nDuplicate group:"
    $_.Group | Select-Object Path
  }

If you want to scan a specific folder, replace "$env:USERPROFILEDownloads" with the path of the desired folder. For example, I modified the script to also look at my Documents and Photos folders. If you want to view a folder in another directory, you can simply enter the full path of the file, for example “C:UsersMeganDocuments”.

Once the script runs, PowerShell will include a list of file paths under Doubles group. You can use these details to find the files using File Explorer (or in my case, a File Explorer alternative) and delete them.

It may take a while for the script to run if you have a large folder or a slow PC. So you may have to wait a few minutes to see the results in PowerShell. However, rather than trying to select an entire drive, I recommend using it for specific folders where duplicates are likely, such as your Downloads, Documents, and Photos folders.

Make your duplicate files easier to track

Get PowerShell to export results

It may take a while to clean up your files if you have created duplicates over a long period of time. I know my own PC is full of downloads that I have repeatedly saved from email attachments. So, what if you don’t want to re-run the script every time you want to do a little digital spring cleaning?

You can adjust the PowerShell script to export its results to a CSV file so that you can review its results repeatedly. This can be done with the Select an object And Export-Csv cmdlets added to the original script.

To export duplicate file paths to a CSV file that you can use with Excel or another spreadsheet, you can use the following script:

Get-ChildItem -Path "$env:USERPROFILEDownloads" -Recurse -File |
  Get-FileHash |
  Group-Object Hash |
  Where-Object Count -gt 1 |
  ForEach-Object {
    $_.Group | Select-Object Hash, Path
  } |
  Export-Csv -Path "$env:USERPROFILEDesktopduplicates.csv" -NoTypeInformation

This will save the CSV to your desktop in a file named duplicates.csv. You can adjust the script to change the location or name of the file. However, you should note that PowerShell takes some time to export the generated information and you may see the file icon before it is actually ready to display.

If you encounter an error stating that PowerShell could not find the directory for export, try a different folder. This could be due to a permissions issue.

Once the file is ready, you will have a document containing the paths of all your duplicate files.

When does using a GUI make more sense?

A dedicated app works better for certain use cases

Credit:

While it’s easy enough to just paste and tweak the PowerShell script to find your duplicate files, sometimes a dedicated storage management app works better. The most notable example is when you want to quickly scan entire drives. For this, you may want to use something like WizTree, which can identify duplicate files on your drives.

I’ve also found that the script doesn’t always work well for folders where you may have an excessive number of duplicates. Indeed, PowerShell only displays a limited number of things in its window. So you can scroll up to the maximum quantity without still seeing all the items listed. This happened when I scanned my Documents folder because it includes some folders from my self-hosted services.

I would mainly recommend this tip to people who are proficient in PowerShell and don’t want to use a third-party application to scan files on their PC.

The best Windows tricks are sometimes the least known

There are many tools buried in Windows that provide great functionality but aren’t always easy to find. PowerShell is one of the biggest examples of this, probably because of the learning curve involved.

Personally, I think it would be great if Windows improved its own storage analysis tools to help users get a better idea of ​​which files are duplicated and taking up space on their drives. For now, though, this tip can help you get your storage under control.