3 snippets across 3 stacks - Bash & Linux, Python, SQL
uniq [options] [input [output]]sort access.log | uniq -c | sort -rn | head -20 847 GET /api/users
623 GET /api/health
412 POST /api/loginNote uniq only removes adjacent duplicates, so you almost always need to sort first. -c prefixes each line with its count, -d shows only duplicates, -i ignores case.
{expression for item in iterable if condition}emails = ["[email protected]", "[email protected]", "[email protected]", "[email protected]"]
unique_lower = {e.lower() for e in emails}
print(unique_lower){'[email protected]', '[email protected]', '[email protected]'}Note Set comprehensions automatically deduplicate results. Useful for extracting unique transformed values from a collection.
SELECT DISTINCT column FROM table;SELECT DISTINCT city
FROM users
ORDER BY city;-- Returns each city only once, no duplicatesNote DISTINCT applies to the entire row when used with multiple columns. SELECT DISTINCT city, state treats (city, state) pairs as the unit of uniqueness.