Highlights
Denial of Service: 36 prior fixes. Scrutinize any change in this area.
pandas/_libs/src/parser/tokenizer.c: most-fixed (9 issues). Treat as high-risk during review.
14 high-severity fixes in this history; regressions here are high-impact.
Recurring patterns
The bug types that recur here, drawn from past fixes, not open vulnerabilities.
Denial of Service: The C-based tokenizer is highly prone to crashes, buffer overflows, and infinite loops when parsing malformed CSV files, particularly when handling extremely long exponents, missing fields, or empty rows. Crucial entry points must constrain maximum parsed digit sizes and validate stream buffer bounds.
Memory Corruption: Relocation and trimming of internal parser stream buffers can leave stale pointers, causing pointer arithmetic errors and memory corruption if the base pointers are not immediately updated and properly cast during reallocation.
Memory Corruption: Unsafe resizing of internal arrays via NumPy APIs (e.g. using refcheck=False) while active external views are retained allows out-of-bounds reads and writes, violating memory safety.