How the dedupe works
Every line is reduced to a comparison key — the line itself, optionally trimmed of surrounding whitespace and lowercased — and checked against a set of keys already seen. The scan is a single linear pass using a hash set, which is why a 100,000-line list processes in well under a second rather than the minutes a naive line-vs-line comparison would take. Keep first preserves the original position of each line's first appearance; keep last keeps the final appearance instead — useful when later entries are updated versions of earlier ones.
Sort lines alphabetically or numerically
After deduping you can sort the result: A → Z and Z → A use locale-aware comparison so accented characters order sensibly, and numeric compares by numeric value — so 2 sorts before 10, which a plain alphabetical sort gets wrong. Sorting is applied to the output only; the duplicates panel keeps reporting against the original order. You can also use the page purely as a line sorter: paste a list with no duplicates and pick a sort.
Remove duplicates without spreadsheets
The usual advice for deduping a column is to paste it into a spreadsheet and hunt through menus for a remove-duplicates command or build a COUNTIF formula. For a flat list that's overkill: paste it here, and it's done before you could have opened the application — with the removed lines listed for verification, which spreadsheet dedupe doesn't show you either. When you're finished, copy the clean column back out. For genuinely tabular data with multiple columns where duplicates are defined by one key column, a spreadsheet remains the right tool; for one-line-per-item lists, this is faster.
Frequently asked questions
Is the comparison case-sensitive?
By default no — "Apple" and "apple" count as the same line, which matches what most people want for emails and keywords. Tick Case-sensitive to treat different capitalizations as distinct lines. Trimming is separate: with Trim before comparing on (the default), "apple" and "apple " are duplicates, but the kept line's original spacing is preserved in the output only if trimming is off.
Does the original order survive?
Yes, when sort is set to original order: unique lines appear exactly where they (first or last) appeared in your input. Choosing any sort option reorders the output, of course — switch back to original order at any time; nothing is destroyed.
How large a list can this handle?
Lists beyond 100,000 lines are processed in a single pass and remain responsive — the practical ceiling is your browser's textarea, not the algorithm. For very large lists the duplicates panel caps its rendering at 2,000 entries (with a note), while the counts and the output always reflect the full list.
Is my list uploaded anywhere?
No. The processing is JavaScript running in your tab; no server sees your data and the tool works with the network disconnected. That makes it safe for customer emails, license keys, and other lists you shouldn't paste into random websites that process server-side.