Chapter 1 · Practice 3 of 4
Canonicalize input once, before you validate it
Why it matters
A check only works on the form of the data that will actually be used. If you validate one representation and use another, an attacker picks the gap. Common examples:
- URL encoding:
%2e%2e%2fis../after decoding. Decoding twice turns%252einto.as well. - Unicode: the same visible text can have several byte sequences. A username check that runs before normalization can be bypassed with a look-alike form.
- Paths:
uploads/./../configanduploads/../configare the same file. - Case:
Adminandadminmay be one account to the database and two to your code.
The rule is: decode and normalize exactly once, then validate, then use the same value. OWASP's Input Validation Cheat Sheet asks for a consistent Unicode normalization policy before validation, storage and comparison, and adds that normalization is not sanitization and does not replace output encoding.
In practice: let your framework decode the URL once, apply NFC (or NFKC for identifiers like usernames) to free text, lowercase identifiers you compare case-insensitively, and resolve paths to an absolute form before checking them (see the path traversal practice).
Example
import unicodedata
def canonical_username(raw: str) -> str:
# NFKC folds compatibility characters (full-width letters, ligatures)
value = unicodedata.normalize("NFKC", raw).casefold().strip()
if not value.isascii() or not value.replace("_", "").isalnum():
raise ValueError("letters, digits and underscores only")
if not 3 <= len(value) <= 30:
raise ValueError("3 to 30 characters")
return value # store and compare this exact value