← All tools

Remove duplicate lines

Paste your list and each repeated line stays only once. Your text never leaves this computer.

Words
0
Characters
0
Characters (no spaces)
0
Sentences
0
Paragraphs
0
Lines
0
Reading time
0
Speaking time
0

Where duplicate lines come from

Email lists merged from several sources, customer records exported on different dates, keyword lists grown by copy and paste, and log files all end up containing the same lines many times. Duplicates mean the same person gets an email twice, counts come out wrong and lists grow for no reason.

How it works

The tool reads the lines from top to bottom and drops a line if it has already seen it. The first occurrence of each line stays, so the order of the list is kept. That matters for lists where order means something, such as priorities or dates.

In this cleanup lines must match exactly: "Ann Smith" and "ann smith" count as different. To remove duplicates while ignoring case, use "Remove duplicates" on the Sort tab, which also sorts the list.

Watch out for hidden differences

Two lines can look the same but differ because of a trailing space or an invisible character. To avoid that, apply "Trim spaces at the start and end of lines" on the same page first. For email lists, lowercasing every address also catches duplicates such as "Ann@example.com" and "ann@example.com".

Why not just use Excel?

Excel's Remove Duplicates works too, but you have to paste the data into a sheet, pick columns and copy the result back. For a simple list it is much quicker to paste the text and copy the result. Your text is processed in your browser and never sent anywhere, so it is safe for customer lists too.

How to use it

  1. Paste your list into the box, one item per line.
  2. On the Clean up tab the options to remove duplicate and empty lines are already on.
  3. Copy the result, or move it to the input and apply another action.

Frequently asked questions

Which line is kept?

The first occurrence of a repeated line stays, so the order is not changed.

Is upper and lower case counted as a difference?

In this cleanup lines must match exactly to count as duplicates. To ignore case, use Remove duplicates on the Sort tab, which also sorts.

Do lines that differ only by spaces count as the same?

Yes when trimming spaces at the start and end is on, so "apple" and "apple " become one line.

Related tools