Guide

Understand invisible Unicode characters

Invisible Unicode characters can support legitimate writing or obscure names and content. Local inspection reveals selected characters without automatically knowing their purpose.

Responsible operator: Tom Silas Helmke, operator of ScanBeforeOpen. Content revised: .

Open scanner

U+200B, bidi and legitimate language

U+200B is a zero width space. Bidi controls such as U+202E or U+2066 through U+2069 affect writing direction. They do not automatically indicate an attack. Mixed scripts and similar-looking characters can represent legitimate names or deception.

What an invisible character does in text
Visible textdisplay: ABCharacter sequenceA, U+200B, BZero Width Space U+200Bwithout visible widthPrograms see different textsearch, comparison, AI inputScanner makes it visibleshows code points, removes in a copy

The sequence in words

  1. On screen it reads AB. A person cannot tell whether something sits in between.
  2. Internally, text is a sequence of code points. Here U+200B, a zero width character, sits between A and B.
  3. Zero width characters, formatting characters and direction control characters (bidi, UAX #9) can influence the display or change nothing visible at all. The text looks the same.
  4. A search for AB does not find the text. Two seemingly equal words are unequal, and an AI receives the hidden characters as well.
  5. ScanBeforeOpen displays such characters visibly and removes them in the text copy. That proves no bad intent, and the copy does not replace a review of the content.

What local inspection reveals

Filename and text inspection use different bounded heuristics. The text.controls finding reports selected invisible characters; the hidden-content view renders supported markers visibly. Not every Unicode code point or visual confusion is detected.

Many text checks read at most the first 2 MB; TXT exports are limited to 200,000 characters. No finding does not establish the absence of all controls.

Removal can change meaning

Check context and language before removing markers. Joining characters and direction controls can be necessary for correct rendering. Renaming changes neither the file type nor the contents of a suspicious attachment.

Use reveal mode when appropriate, compare the TXT excerpt and retain the original separately. Do not open an unknown file solely because its name was cleaned.

Reproducible synthetic example

U+200B appears between A and B. The quoted test phrase ignore previous instructions also triggers text.prompt. The scanner detects patterns, not the author’s intent.

The findings come from an actual local scan of this test file. Download it and choose it in the scanner. No real personal data or malware.

Download test file: unicode-demo.txt

Example inspection coverage: complete.

Practical checklist

  • Review location and language context
  • Compare visible and technical extensions
  • Reveal markers first
  • Account for content changes during removal
  • Keep unfamiliar originals closed

Clear limits

Homoglyph detection is heuristic and may mark legitimate multilingual names.

FAQ

Are Unicode characters automatically dangerous?

No. They can be legitimate in languages, but in unexpected filenames they are a warning signal.

Why check mixed scripts?

Similar letters from different scripts can imitate trusted names.

Related reading

Technical sources