wc and uniq
In this page:
wc [-l | -w | -c] file_name
sort file_name | uniq [-c]Counting Lines, Words, and Characters
wc reports line, word, and byte counts. Passing -l, -w, or -c restricts the output to just that one count, which is easier to use in scripts than parsing the default three-column output.
< instead of passing it as an argument avoids the filename being printed alongside the count.Example: Counting Lines, Words, and Characters
#!/bin/bash
printf "one two\nthree\n" > sample.txt
echo "lines: $(wc -l < sample.txt)"
echo "words: $(wc -w < sample.txt)"
rm -f sample.txt
Login to try C/C++/Java/PHP code in the editor
Removing Adjacent Duplicates
uniq collapses runs of identical consecutive lines into one. It does NOT remove duplicates scattered non-adjacently through the file, so input is usually sorted first to bring duplicates together.
Example: Removing Adjacent Duplicates
#!/bin/bash
printf "a\na\nb\na\nb\nb\n" > raw.txt
echo "uniq alone (misses non-adjacent duplicates):"
uniq raw.txt
echo "sort then uniq (all duplicates collapsed):"
sort raw.txt | uniq
rm -f raw.txt
Login to try C/C++/Java/PHP code in the editor
Counting Occurrences
uniq -c prefixes each line of output with the number of consecutive times it appeared, turning a list of items into a frequency table.
Example: Counting Occurrences
#!/bin/bash
printf "apple\nbanana\napple\napple\nbanana\n" | sort | uniq -c
Login to try C/C++/Java/PHP code in the editor
Finding the Most Frequent Items
Chaining sort | uniq -c | sort -n sorts items, counts consecutive duplicates, then sorts by that count numerically, producing a ranked frequency list, a very common one-liner.
Example: Finding the Most Frequent Items
#!/bin/bash
printf "a\nb\na\nc\na\nb\n" | sort | uniq -c | sort -n
Login to try C/C++/Java/PHP code in the editor
- Forgetting
uniqonly removes ADJACENT duplicate lines, not all duplicates anywhere in the file; the input usually needs to be sorted first. - Using
wc -land forgetting it counts newline characters, so a file missing a trailing newline may report one line fewer than expected. - Not knowing
uniq -cprefixes each line with how many times it occurred, which is a very common combination withsort.
wc -l,wc -w,wc -ccount lines, words, and bytes respectively.uniqonly collapses consecutive duplicate lines, so input is typically piped throughsortfirst.uniq -cprefixes each output line with a count of how many times it repeated.sort | uniq -c | sort -nis a classic combo for frequency counting.
Chapter Quiz — Complete all 12 topics to unlock
0/12 topics done
Complete these topics first: