Huffman Coding
What is Huffman Coding?
Huffman coding builds a compact binary code for a set of symbols by giving frequently-used symbols shorter codes and rarely-used symbols longer codes, reducing the total number of bits needed to store or transmit the data — all without losing any information.
Example: What is Huffman Coding?
#include <iostream>
using namespace std;
int main() {
cout << "Frequent symbols get short codes, rare symbols get long codes -- reduces total bits needed";
return 0;
}
public class Main {
public static void main(String[] args) {
System.out.println("Frequent symbols get short codes, rare symbols get long codes -- reduces total bits needed");
}
}
print("Frequent symbols get short codes, rare symbols get long codes -- reduces total bits needed")
#include <stdio.h>
int main() {
printf("Frequent symbols get short codes, rare symbols get long codes -- reduces total bits needed");
return 0;
}
Login to try C/C++/Java code in the editor
Frequency Table
The algorithm begins by counting how often each symbol appears in the input; symbols that occur more often will end up closer to the root of the resulting tree and therefore get shorter codes.
Example: Frequency Table
#include <iostream>
#include <map>
using namespace std;
int main() {
string text = "abracadabra";
map<char,int> freq;
for (char c : text) freq[c]++;
for (auto& p : freq) cout << p.first << ":" << p.second << " ";
return 0;
}
import java.util.*;
public class Main {
public static void main(String[] args) {
String text = "abracadabra";
Map<Character,Integer> freq = new TreeMap<>();
for (char c : text.toCharArray()) freq.merge(c, 1, Integer::sum);
for (var e : freq.entrySet()) System.out.print(e.getKey() + ":" + e.getValue() + " ");
}
}
from collections import Counter
text = "abracadabra"
freq = Counter(text)
for ch, count in sorted(freq.items()):
print(f"{ch}:{count}", end=" ")
#include <stdio.h>
int main() {
char text[] = "abracadabra";
int freq[26] = {0};
for (int i = 0; text[i]; i++) freq[text[i]-'a']++;
for (int i = 0; i < 26; i++) if (freq[i]) printf("%c:%d ", 'a'+i, freq[i]);
return 0;
}
Login to try C/C++/Java code in the editor
Merge Two Smallest
Repeatedly taking the two symbols (or partial trees) with the smallest combined frequency and merging them into a new internal node builds the Huffman tree from the bottom up, always favoring the currently cheapest pair.
Example: Merge Two Smallest
#include <iostream>
#include <queue>
using namespace std;
int main() {
priority_queue<int, vector<int>, greater<int>> pq;
for (int f : {5, 9, 12, 13, 16, 45}) pq.push(f);
int a = pq.top(); pq.pop();
int b = pq.top(); pq.pop();
pq.push(a + b);
cout << "Merged " << a << "+" << b << " into new node freq " << a+b;
return 0;
}
import java.util.*;
public class Main {
public static void main(String[] args) {
PriorityQueue<Integer> pq = new PriorityQueue<>(Arrays.asList(5, 9, 12, 13, 16, 45));
int a = pq.poll(), b = pq.poll();
pq.add(a + b);
System.out.println("Merged " + a + "+" + b + " into new node freq " + (a+b));
}
}
import heapq
pq = [5, 9, 12, 13, 16, 45]
heapq.heapify(pq)
a = heapq.heappop(pq)
b = heapq.heappop(pq)
heapq.heappush(pq, a + b)
print(f"Merged {a}+{b} into new node freq {a+b}")
#include <stdio.h>
int main() {
int freqs[6] = {5, 9, 12, 13, 16, 45};
int minIdx = 0;
for (int i = 1; i < 6; i++) if (freqs[i] < freqs[minIdx]) minIdx = i;
int a = freqs[minIdx]; freqs[minIdx] = 1000000;
int min2Idx = 0;
for (int i = 1; i < 6; i++) if (freqs[i] < freqs[min2Idx]) min2Idx = i;
int b = freqs[min2Idx];
printf("Merged %d+%d into new node freq %d", a, b, a+b);
return 0;
}
Login to try C/C++/Java code in the editor
Prefix Codes
Because no code word is ever a prefix of another code word in this tree, a stream of Huffman-encoded bits can always be decoded unambiguously left to right, without needing separators between symbols.
Example: Prefix Codes
#include <iostream>
using namespace std;
int main() {
string codeA = "0", codeB = "10", codeC = "11";
cout << "None of " << codeA << "," << codeB << "," << codeC << " is a prefix of another -- unambiguous decoding";
return 0;
}
public class Main {
public static void main(String[] args) {
String codeA = "0", codeB = "10", codeC = "11";
System.out.println("None of " + codeA + "," + codeB + "," + codeC + " is a prefix of another -- unambiguous decoding");
}
}
code_a, code_b, code_c = "0", "10", "11"
print(f"None of {code_a},{code_b},{code_c} is a prefix of another -- unambiguous decoding")
#include <stdio.h>
int main() {
printf("None of 0,10,11 is a prefix of another -- unambiguous decoding");
return 0;
}
Login to try C/C++/Java code in the editor
Practice
Working through this by hand, you keep combining the two smallest frequencies into a new node whose frequency is their sum, then treat that combined node as a new candidate for the next merge, continuing until only one tree remains.
Example: Practice
#include <iostream>
using namespace std;
int main() {
cout << "Keep combining the two smallest frequencies into a new node, treat it as a candidate for the next merge";
return 0;
}
public class Main {
public static void main(String[] args) {
System.out.println("Keep combining the two smallest frequencies into a new node, treat it as a candidate for the next merge");
}
}
print("Keep combining the two smallest frequencies into a new node, treat it as a candidate for the next merge")
#include <stdio.h>
int main() {
printf("Keep combining the two smallest frequencies into a new node, treat it as a candidate for the next merge");
return 0;
}
Login to try C/C++/Java code in the editor
Chapter Quiz — Complete all 5 topics to unlock
0/5 topics done
Complete these topics first: