Blackbox Blast
BB Preflight
← Incident log

CASE · Minor · monitoring-gap

VerifiedFirsthand record

We wrote to Japanese authors in English. And "few replies" meant we could only count one platform

The tool that drafts notices to the authors of articles we feature sometimes wrote English messages to authors of Japanese articles. Separately, replies looked low relative to the number of articles, but automatic reply detection existed for Zenn only, so a "0" on any other platform could mean no reply or simply not checked.

3 min read

OBSERVED
CATEGORY
monitoring-gap
BLAST RADIUS
reputation
DETECTED BY
manual-observation
TAGS
#outreach #language-detection #hardcoded-list #measurement #silent-failure #dogfooding

CH 1

Cause

The message language came from a fixed list: Japanese only if the platform was Zenn, Qiita, note or Hatena Blog. Any Japanese site not on the list (personal blogs, company sites) got English. Reaction detection was the same story: only Zenn's comment section was read automatically, while X and Qiita relied on the operator marking reactions by hand. No mark meant both "no reply" and "not looked at".

CH 2

Consequence

Several Japanese-site authors received an English notice (among sent English messages, at least 2 were to Japanese sites). One unsent message was rewritten in Japanese. The low reply count was not a real measurement: reactions on 22 X contacts and 3 Qiita contacts are still unverified, and only a human looking can check them. The damage was impression only; no data or money was lost.

CH 3

Fix

Language detection now also picks Japanese when the title contains kana or the domain ends in .jp, on top of the platform list (articles from English-speaking organizations stay English). Reaction numbers are read in two groups: platforms where detection works and platforms counted by hand. Before using "0 reactions" as a result, check that reactions can be detected on that platform at all.

What happened

Blackbox Blast contacts the author of each featured article: "we covered this; tell us if anything is wrong." The tool builds the message and link per article. It never posts to X by itself; a person checks the text and clicks, which opens the compose screen pre-filled (to respect X's rules).

On November 22, 2026, the operator noticed that a message to a Japanese author was in English, and asked why so few authors had replied relative to the number of articles. We looked into both.

Two causes

  1. Language came from a fixed list. Japanese if the platform was Zenn, Qiita, note or Hatena Blog, otherwise English. Every Japanese site off the list, personal blogs and company sites alike, got English.
  2. Only one platform's replies could be counted. Automatic detection read Zenn's comment section only. X and Qiita were marked by hand, and with no mark the screen showed nothing. "No reply came" and "nobody looked" both appeared as zero.

What the numbers really said

The impression of few replies came from how the number was read. On Zenn, where detection works, 9 of 32 had a reaction. For 22 X contacts and 3 Qiita contacts we cannot tell whether there was no reaction or no look.

Fixes

  • Language detection now also uses kana in the title and .jp domains. English-speaking organizations stay English.
  • One unsent message was rewritten in Japanese. Already-sent English messages can't be recalled, so they were left alone.
  • Reaction counts are read in two groups: auto-detected platforms and hand-counted ones.

Lesson

Code that branches on a list of names goes quietly down the other path the moment an unlisted name shows up. And a zero displays as zero even where nobody counted. Before reading a number, check how much of the world it actually looks at.

Sources

  • Firsthand account from the site operator. There is no public write-up to link to.
PREFLIGHTIs your own Mac prepared?BB Preflight is a free Mac tool that checks 21 things, from AI agent permissions to where your keys live, without changing anything.Run the check →