Invisible Character Remover — Detect and Remove Invisible Characters

What invisible characters are

Unicode assigns code points to characters that have no visible glyph. They exist because text rendering is more complicated than it looks: scripts run in different directions, emoji are built from sequences of simpler parts, and words need hints about where they can break. These characters do real work inside a renderer. Outside of one, they are usually noise.

The phrase "invisible character" covers several distinct families. Zero-width characters take no space and leave no mark. Format controls steer the direction or joining behaviour of adjacent glyphs. Invisible spaces look like gaps but behave differently from an ordinary space character. Each family has a different origin, a different failure mode, and a different cleanup rule.

The full roster: nine character types worth knowing

U+200B ZERO WIDTH SPACE is the most common offender. Chat interfaces insert it between tokens to hint at line-break points; copy the answer and it travels with every word boundary. Databases, search indexes and code string comparisons all treat it as a distinct character.

U+200C ZERO WIDTH NON-JOINER and U+200D ZERO WIDTH JOINER control ligature formation. ZWNJ prevents two adjacent characters from forming a ligature (important in Persian and Hindi); ZWJ forces them to join (used inside emoji sequences like family glyphs and flag pairs). Both are invisible and both survive a paste into plain text.

U+FEFF BYTE ORDER MARK was designed as a file-encoding hint placed once at the start of a stream. Rich-text editors — particularly older versions of Microsoft Word and some CMS paste handlers — inject it mid-document whenever they receive pasted content, where it is simply an invisible placeholder that breaks string comparisons.

U+2060 WORD JOINER behaves like a zero-width space but also signals that the two surrounding characters must not be separated by a line break. A renderer in a chat interface inserts it around short words or numbers to prevent awkward wrapping; once copied into a database field, it silently corrupts prefix searches.

U+00AD SOFT HYPHEN is visible only when it falls at an active line-break point. A word processor that auto-hyphenates inserts it everywhere it decides a word could break. Paste such a paragraph into a web form and every potential hyphen point travels along, inflating character counts and fouling validation.

U+180E MONGOLIAN VOWEL SEPARATOR was originally used in Mongolian script to separate vowel sequences. It was briefly assigned zero-width-space behaviour in some Unicode versions, then reclassified. Several OCR tools and legacy encoders still emit it; modern text engines render it as nothing, but it occupies a code point that breaks exact-match queries.

U+3164 HANGUL FILLER is a placeholder used in Korean input methods to hold the cursor position while a syllable block is being composed. It looks like a space but has a distinct code point. It occasionally escapes into clipboard content when text is copied mid-composition, and it is wide enough to be mistaken for an ordinary ideographic space.

U+00A0 NO-BREAK SPACE and U+202F NARROW NO-BREAK SPACE look like spaces and are often intended as spaces — the difference is that they will not allow a line break at that position. A title with a no-break space between the last two words will overflow its container instead of wrapping. This tool normalises them to ordinary spaces rather than deleting them, to avoid silently joining two words.

Tabs (U+0009) and non-breaking spaces are not always invisible in the sense of zero-width, but they are invisible in the sense of indistinguishable from a regular space in most views. A cell in Excel that begins with a tab character is treated as text regardless of its contents. A SQL WHERE clause comparing a string with a trailing U+00A0 will miss every row stored without it.

Where they come from

Chat interfaces are the largest single source today. Streamed answers are assembled fragment by fragment; the renderer adds zero-width joiners to hold emoji sequences together, word joiners around numbers, and zero-width spaces at word boundaries. Press the copy button and the whole layer comes with you.

Word processors are the second source. Microsoft Word inserts soft hyphens when it auto-hyphenates; older versions insert BOM markers on paste. LibreOffice has its own set of invisible markers for tracked changes and comment anchors. Copy a paragraph from any of them into a browser form and you bring those markers along.

OCR pipelines frequently emit U+180E and U+FEFF as artefacts of format conversion. PDF-to-text extractors disagree on how to represent ligatures and may insert zero-width joiners where the original PDF had a single glyph. Korean input methods leave U+3164 fillers when composition is interrupted.

Some developers deliberately insert invisible characters — most often U+200B — to mark text ownership, detect copying, or signal metadata. That is a legitimate technique in its original context. The problem is the same as with accidental characters: once the text leaves that context, the markers persist wherever the text goes.

Why they cause real failures

Form validation is the most reported pain point. An email address field that passes a client-side regex fails the mail server's own check because there is a U+200B between the local part and the domain. A phone number field that looks correct contains a soft hyphen the validator does not expect. Both failures are hard to debug because the display shows nothing wrong.

Excel and Google Sheets have their own set of quirks. A numeric cell that starts with U+FEFF is classified as text, so SUM ignores it. A VLOOKUP that should match a product code misses every row because the lookup key was copied from a chat window with zero-width spaces intact. Sorting a column that mixes BOM-prefixed and clean cells produces an ordering that appears random.

SQL queries fail silently. A WHERE name = 'Alice' clause returns no rows if the stored value has a U+200C between the 'l' and the 'i'. The value looks correct in every admin panel and database browser, because every admin panel and database browser renders it as a visible string.

Code diffs report spurious changes. A git diff that shows a whole line as modified — when only one invisible character moved — breaks code review. The reviewer sees what looks like a no-op change and either approves it without scrutiny or asks a question nobody can answer from the diff alone.

Search indexes miss the phrase. A full-text index that tokenises on whitespace and punctuation treats a zero-width space as a token boundary. The word you indexed is two tokens; the word you searched for is one. No match.

Word count tools overcount. Soft hyphens are characters; a word processor that counts characters counts them. A document that looks like 1 000 words may report 1 024 because of soft hyphens auto-inserted during editing.

How to detect invisible characters without this tool

Three browser-console lines cover the most common check. Open the console, paste your string into a variable, and call 'yourstring'.split('').map((c, i) => [i, c.charCodeAt(0).toString(16), c]).filter(([, code]) => parseInt(code, 16) > 127 || parseInt(code, 16) < 32). Any row with a code point outside the printable ASCII range is worth investigating.

On the command line, xxd yourfile.txt | less shows the raw hex stream. U+200B appears as e2 80 8b in UTF-8; U+FEFF as ef bb bf. The grep -P flag accepts Perl-compatible regular expressions, so grep -P '[\x{200B}\x{FEFF}]' yourfile.txt will list every matching line.

Pasting into this tool is faster for most cases. The report panel shows each code point by name, with a count, so you see at a glance whether the problem is one stray BOM or ten thousand zero-width spaces spread through an entire document.

How to clean them out and what stays intact

Paste your text and press Clean. The engine reads each Unicode code point in order — not bytes, not code units — and removes any that match the published table. U+200B, U+200C, U+200D, U+FEFF, U+2060, U+00AD, U+180E and U+3164 are deleted. U+00A0 and U+202F are converted to ordinary spaces, because deleting a no-break space between two words would silently join them.

Everything else is unchanged: letters, digits, punctuation, emoji, tabs, ordinary line breaks, paragraph order. The engine does not rewrite sentences, normalise typography or guess at intent. It removes a defined set of characters and reports exactly what it removed.

For text that has come through multiple applications, the report is often the more useful output. If every paragraph reports the same mix of BOM markers and zero-width spaces, the source is a specific pipeline step — and fixing that step upstream is more efficient than cleaning each document afterwards.

Internal links: for a focused look at zero-width characters specifically, the <a href="/zero-width-space-remover">zero width space remover</a> page covers the six zero-width code points in depth. For removing general AI text artefacts, see <a href="/clean-ai-text">clean AI text</a>. For the full invisible-character check plus a watermark discussion, the <a href="/chatgpt-watermark-remover">ChatGPT watermark remover</a> page has that context.