Detect Duplicate Files
Eliminate Duplicate Content. Improve Search Accuracy. Strengthen Governance.
Duplicate files create confusion, clutter search results, and make it harder for employees to trust the information they find. Detect Duplicate Files helps organizations identify duplicate and near-duplicate documents across SharePoint libraries, intelligently determine which version should be retained, and move redundant content into a designated review location.
The result is a cleaner, more trustworthy SharePoint environment that supports better content governance, improves findability, and reduces the risk of outdated information surfacing in Microsoft Search and Copilot experiences.
What This Skill Does
Detect Duplicate Files scans a SharePoint library, folder, or document collection to identify duplicate and near-duplicate content. Using file attributes and content analysis, the skill:
- Finds duplicate and version-related documents across folders and subfolders
- Identifies near-identical content, even when file names differ
- Determines the most authoritative version using consistent decision rules
- Separates duplicate content from active business content
- Generates a complete audit report for review and compliance purposes
- Moves confirmed duplicates to a controlled holding location for validation
Organizations can also run the skill in report-only mode when they want visibility into duplicate content without moving any files.
What You’ll Get
- An Excel workbook documenting duplicate file findings and actions
- Duplicate and near-duplicate document identification
- Recommendations for which versions should be retained
- Similarity scoring for reviewed files
- Detailed comparison results and review status
- Optional movement of confirmed duplicates into a designated review folder
- A clear audit trail of decisions and actions taken
How it works
- Determine the Scan Scope
The skill identifies the document library, folder, or selected content to review. - Analyze Files for Similarity
Documents are evaluated using file attributes and content analysis to identify duplicate and near-duplicate candidates. - Compare Content
Potential matches are reviewed and assigned similarity scores to determine confidence levels. - Identify Recommended Versions
The skill evaluates document quality, version indicators, metadata signals, and content completeness to determine which version should be retained. - Organize Duplicate Content
Confirmed duplicate files can be moved into a designated review location for validation and cleanup. - Generate the Report
A structured Excel workbook is produced, documenting findings, recommendations, and actions. - Return Results
A summary of the scan, duplicate findings, and report location is provided.
When to use this
- During SharePoint cleanup initiatives
- Before Microsoft 365 Copilot deployments
- When consolidating document libraries
- When preparing for migrations or intranet modernization projects
- When reducing content sprawl across SharePoint
- When improving search relevance and content quality
- As part of ongoing content governance reviews
Output
The skill generates a structured Excel workbook containing:
- Summary
A high-level overview of scan results, duplicate findings, and cleanup outcomes. - Reviewed Pairs
A detailed comparison log showing duplicate candidates, similarity scores, recommended actions, and file locations.
Upon completion, the skill returns a concise summary of findings along with the report location.
Why this matters
Duplicate content creates confusion, reduces trust in search results, and makes governance more difficult. As organizations adopt Microsoft 365 Copilot and AI-powered search experiences, content quality becomes increasingly important.
When multiple versions of the same information exist, users may struggle to identify the correct source, and AI systems may surface conflicting or outdated content.
The Detect Duplicate Files skill helps organizations establish a cleaner, more trustworthy knowledge environment by identifying redundant content and providing a structured approach to remediation. The result is improved findability, stronger governance, and greater confidence that employees and AI systems are working from the right information.
Prerequisites
- Access to the target SharePoint library, folder, or document collection
- Permission to analyze content and generate reports
- Permission to move files when duplicate remediation is enabled
