Blackbox BlastIncident Log
← All incidents

majorotherverifiedfirsthand

A receipt pipeline OCR'd screenshots and wrote ¥73,158 of fake spending into the ledger

An automated pipeline that OCRs receipt photos into a household ledger also swallowed screenshots and browser-saved images, writing 5 fake expenses totaling ¥73,158 into the ledger.

Observed
Severity score
6/10
Blast radius
data, financial
Tags
#ocr#launchd#macos#data-integrity#household-ledger

Cause

Four holes lined up. Every image in the Downloads folder was picked up unconditionally. OCR returned an amount whenever it saw a number, and that amount went into the ledger unchecked. The Excel summary showed stale cached results. And duplicate detection went by filename only, so the same image arriving under a different name was recorded twice.

Consequence

Five nonexistent expenses landed in the household ledger. The largest, ¥66,000, came from a quote screenshot from one of the operator's own dashboards. The fix commit then sat unpushed for about two months.

Fix

Only accept images that actually arrived via AirDrop, require camera EXIF data to prove it is a photo, force a full recalculation on load, and deduplicate on the image's SHA-256.

What happened

Snap a receipt on an iPhone, AirDrop it to the Mac, and it lands in Downloads. A launchd job moves it to a working folder, OCRs it, appends it to a ledger CSV, refreshes an Excel summary, and posts a notification. A fully automatic receipt pipeline.

The investigation started with a vague “I don’t think this is working right.” It turned out launchd was running perfectly. It was running perfectly and importing the wrong things. The ledger held five expenses that never happened, totaling ¥73,158.

The chaos on the ground

The biggest one was ¥66,000. Its source was a quote screenshot from one of the operator’s own dashboards, showing a total of ¥66,000, tax included. Trying to filter on receipt-like words didn’t help: the screenshot used exactly the same words a real receipt does. Keyword checks alone couldn’t tell them apart.

It wasn’t one hole, either:

  • The mover script carried every image in Downloads, whatever its origin. Screenshots and browser-saved images were all treated as receipts.
  • OCR returns an amount whenever an image contains a number, and that amount went into the ledger without any check.
  • The step that updated Excel from the ledger rewrote cell values only. Excel trusts its cached formula results, so the summary kept showing old numbers.
  • Duplicates were detected by filename, so the same image arriving twice under different names was recorded twice.

The fix was committed the same day. That commit then lived only on the local machine for about two months, until it was finally pushed alongside an unrelated fix.

Root cause

Nobody questioned the inputs. “An image in Downloads is a receipt” and “if OCR returned an amount, it’s a receipt” were two unchecked assumptions chained together. Automation that doesn’t doubt what comes in will carry garbage into the books with perfect accuracy.

The fix

  • AirDrop check: Measured on real files, AirDropped images carry a com.apple.quarantine attribute whose agent is sharingd; browser downloads carry a different value, and screenshots have no such attribute at all. That narrowed the front door.
  • Camera-photo check: Keyword checks lost to the quote screenshot. The deciding signal was the EXIF camera make: across 13 real samples, only the genuine receipt photos had Make=Apple, and everything else failed.
  • Forced recalculation: The workbook now carries fullCalcOnLoad="1" and its calculation chain is removed. The flag is confirmed to be in the file; recalculation has not been verified by actually opening it in Excel.
  • Content-based dedup: The ledger gained an image_sha256 column, so duplicates are judged by content rather than filename.

The ledger now held 51 entries, and the test suite grew from 74 to 96.

Sources

  • Firsthand account from the site operator. There is no public write-up to link to.