DevKitHub

Programming

Sort, Deduplicate and Number Lines

Paste a list to sort, deduplicate, trim, reverse or number it. Natural sorting orders item2 before item10, which plain alphabetical order does not.

5 lines
5 lines
5 in · 5 out0 duplicates removed0 blanks removed

This tool runs entirely in your browser. Your input is never uploaded, stored or logged.

How it works

Two sort orders are offered and the difference matters more than it sounds. Plain alphabetical order compares character by character, so item10 comes before item2 — "1" really does sort before "2" — and that is correct and almost never what was wanted from a list of filenames, versions or identifiers. Natural order compares runs of digits as numbers, so item2, item10 and v1.9.0 before v1.10.0 all come out the way a person would write them. Sorting by length is also available, with ties broken by content so the result is reproducible rather than dependent on the engine.

The operations run in a fixed order, because the order changes the answer: trim, then remove empty lines, then deduplicate, then sort, then reverse, then number. Trimming before deduplicating means "a" and "a " collapse into one line; the other way round leaves both, which is almost never intended when both boxes are ticked. Removing blanks before sorting stops them piling up at the top of the result.

Deduplication keeps the first spelling it encountered, so with case-insensitive matching on, "Apple" survives and "apple" is dropped rather than the text being normalised to one case. Input is split on CRLF, CR and LF alike, so a file written on Windows behaves the same as one written anywhere else, and the output is joined with plain newlines.

Common problems

Every example below is run against this tool in our test suite, so what it says here is what the tool actually does.

The sorted list has item10 before item2.

Why:
Plain alphabetical order compares one character at a time, and "1" sorts before "2". Every character after that is irrelevant, so any number with fewer digits loses.
Fix:
Use natural order, which compares digit runs as numbers. It is the difference between v1.10.0 and v1.9.0 sorting the way you expect.

Duplicates that look identical were not removed.

Why:
They differ by trailing whitespace, or by a carriage return left over from a Windows file. Two lines that render the same are not the same string.
Fix:
Turn on trim as well as deduplicate. Trimming runs first, so the lines match before the comparison happens.

Case-insensitive deduplication changed the casing of my lines.

Why:
Some tools lowercase every line in order to compare them and then forget to put the original back, so the survivors come out normalised.
Fix:
Does not happen here — comparison is case-insensitive but the line kept is the first one as it was written.

Frequently asked questions

What is natural sorting?
It compares runs of digits as numbers rather than as characters, so item2 comes before item10 and v1.9.0 before v1.10.0. Plain alphabetical order puts the shorter number last, which is correct character by character and rarely what anyone wanted.
In what order do the options apply?
Trim, remove empty, deduplicate, sort, reverse, number. The order matters: trimming before deduplicating means "a" and "a " collapse into one, and removing blanks before sorting stops them piling up at the top.
Is my list uploaded anywhere?
No. Everything happens in your browser. That matters for this tool in particular, because the lists people sort and deduplicate are often exported identifiers, email addresses or log lines.

Related tools

Last updated