Why deduplicate Excel data
After merging exports or CRM downloads you often see duplicate emails, order IDs, or identical rows. Importing without dedup skews metrics, causes double shipments, or hits primary key errors.
Desktop Excel offers Data → Remove Duplicates, but it requires local Excel and is awkward for large .csv files. ComTools Remove Duplicates from Excel is a free online tool: upload, pick key columns, preview groups, download a clean file.
Steps
- Open Remove Duplicates.
- Upload
.xlsx,.xlsm, or.csv(row 1 = headers). - Pick the target sheet for multi-sheet workbooks.
- Check key columns for dedup (e.g. Email, OrderId). Leave all unchecked to compare entire rows.
- Choose keep first or keep last duplicate.
- Optional: enable trim whitespace or ignore case for text keys.
- Click Preview dedup to see duplicate counts, groups, and rows to remove.
- Click Download as XLSX or CSV.
Output modes
| Mode | Result |
|---|---|
| Deduped rows | Default—keep one row per key, drop the rest |
| Mark duplicates | Keep all rows; flag which are duplicates |
| Unique values only | Export distinct key values (useful for ID lists) |
Common scenarios
| Scenario | Settings |
|---|---|
| Duplicate emails / phones | Key = that column only; keep first |
| Remove only fully identical rows | No key columns selected (whole-row match) |
| See duplicates before deleting | Output mode = mark duplicates |
| Build a unique ID list for SQL | Unique values mode → copy to Text to SQL |
vs Excel built-in
| Feature | Excel built-in | ComTools online |
|---|---|---|
| Dedup by column | Yes | Yes, multiple columns |
| Preview rows to remove | No | Yes, with groups |
| CSV upload | Open in Excel first | Direct upload |
| Keep last duplicate | Sort first | One click |
| Mark duplicates | Conditional formatting | Export marked file |
Typical workflow
Merge files → Dedup → Format → Excel to SQL INSERT
Related steps:
- Merge multiple Excel files before dedup
- Excel to SQL INSERT after keys are unique
- CSV import tips for encoding and batch load
- SQL IN clause from ID list after exporting unique values
Topic guides
- Remove duplicates by one column
- Remove duplicates by multiple columns
- Keep first duplicate
- Keep last duplicate
Options explained
- Trim whitespace:
alice@x.comandalice@x.comtreated as the same key. - Ignore case:
ABC@mail.comvsabc@mail.commatch when enabled. - Keep first / last: Which row survives in each duplicate group.
FAQ
Row limit?
Large sheets are supported; preview may truncate very large duplicate groups for display—download reflects full processing.
Dedup before or after merge?
Usually merge first, then dedup the combined sheet.
Still getting duplicate key errors on INSERT?
Keys in Excel may differ by invisible spaces—enable trim; or DB has existing rows—use UPDATE instead.
Is my file stored?
Uploaded for server-side parse and export; do not upload sensitive production data to untrusted hosts.