Help! My data pipeline is suddenly allergic to UTF-8 CSVs, throwing baffling encoding errors.

hey everyone, hope you're having a less frustrating day than i am. my data pipeline, which has been chugging along happily for months, suddenly decided it hates utf-8 csvs. it's like it woke up on the wrong side of the server rack or something.

we've got a pretty standard data integration setup, pulling daily reports from a few external services. these reports are always, always, utf-8 csvs. like, confirmed, triple-checked utf-8. but as of yesterday, the pipeline just chokes on them. says 'invalid byte sequence' or 'malformed utf-8' every single time. it was literally working monday, tuesday, wednesday, and then poof thursday it's broken. nothing changed on our end, no code deploys, no config tweaks.

i've gone through the whole 'is it me or is it the file' dance:

• first, verified the source files with chardet and file -i – all proudly declare themselves text/csv; charset=utf-8.
• then, i tried explicitly setting the encoding in the pipeline tool's ingestion settings to utf-8, utf8, utf-8-sig (just in case), even latin-1 out of pure desperation (spoiler: didn't work, obviously).
• i even tried saving a sample csv with a utf-8 bom, thinking maybe the tool suddenly wanted that, but nope, same error.
• checked for hidden updates to the tool itself, but their release notes show nothing that would impact encoding this drastically. it's like a phantom update or something.

the error log is pretty consistent, looks something like this:

[2023-10-26 10:34:15.123] ERROR [PipelineService] Failed to process file 'report_2023-10-25.csv': java.nio.charset.MalformedInputException: Input length = 1
[2023-10-26 10:34:15.124] INFO [DataProcessor] Aborting current data integration task for report_2023-10-25.csv due to encoding issues.
[2023-10-26 10:34:15.125] DEBUG [FileHandler] Closing stream for report_2023-10-25.csv

has anyone else experienced a data pipeline tool suddenly becoming encoding-averse for no apparent reason? could there be some weird environmental variable or system-level change i'm missing? or maybe some obscure setting in these tools that sometimes gets reset or defaults change during a silent update? it's really grinding our data flow to a halt and i'm running out of ideas.