Skip to content

encode() writes a lone UTF-16 surrogate as invalid UTF-8 in a string of 50 characters or fewer #303

Open

Description

@mark1russell7

Summary

encode() writes a JavaScript string with a lone UTF-16 surrogate in two different ways, and the length of the string selects the way:

  • In a string of 50 characters or fewer, utf8EncodeJs() writes the surrogate as its 3-byte form. For example, U+D800 becomes ED A0 80. These bytes are not valid UTF-8.
  • In a longer string, TextEncoder.encodeInto() writes U+FFFD (EF BF BD), as the WHATWG Encoding Standard tells.

The MessagePack spec says that the str format holds UTF-8. Thus a short string with a lone surrogate gives a str that a strict decoder in another language can refuse. The same value also gives different bytes when it has padding.

Version: 3.1.3. src/utils/utf8.ts on main (6d4b666) is the same.

Repro

npm install @msgpack/msgpack@3.1.3, then node repro.ts with Node.js 22.18 or later:

// F0137: @msgpack/msgpack 3.1.3 encodes a string of 50 characters or fewer with utf8EncodeJs, and a longer string with
// TextEncoder.encodeInto. utf8EncodeJs writes a lone UTF-16 surrogate as the 3-byte form of the surrogate (ED A0 80 for
// U+D800), which is not valid UTF-8. TextEncoder writes U+FFFD (EF BF BD). The MessagePack str type holds UTF-8.
import { encode } from "@msgpack/msgpack";

const hex = (bytes: Uint8Array): string => Buffer.from(bytes).toString("hex").replace(/(..)/g, "$1 ").trim();
const strPayload = (bytes: Uint8Array): Uint8Array => {
  const head = bytes[0]!;
  if (head >= 0xa0 && head <= 0xbf) return bytes.subarray(1);
  if (head === 0xd9) return bytes.subarray(2);
  throw new Error(`not a fixstr or a str 8: ${head.toString(16)}`);
};
const isUtf8 = (bytes: Uint8Array): boolean => {
  try {
    new TextDecoder("utf-8", { fatal: true }).decode(bytes);
    return true;
  } catch {
    return false;
  }
};
const textEncoder = new TextEncoder();
const padding = "x".repeat(60);
const cases: Array<[string, string]> = [
  ["lone high surrogate at the end", "a\ud800"],
  ["lone low surrogate", "a\udc00b"],
  ["low surrogate before high surrogate", "\udc00\ud800"],
];
let faults = 0;
for (const [label, text] of cases) {
  for (const [size, value] of [["short", text], ["long", padding + text]] as const) {
    const payload = strPayload(encode(value));
    const expected = textEncoder.encode(value);
    const same = hex(payload) === hex(expected);
    if (!same) faults++;
    const tail = (bytes: Uint8Array): string => hex(bytes.subarray(Math.max(0, bytes.length - 6)));
    console.log(
      `${same ? "ok   " : "FAULT"} ${label}, ${size} (${value.length} characters): last bytes ${tail(payload)}, valid UTF-8: ${isUtf8(payload)}, TextEncoder: ${tail(expected)}`,
    );
  }
}
if (faults > 0) {
  console.log(`BUG PRESENT: ${faults} of ${cases.length * 2} strings encode a lone surrogate as bytes that are not valid UTF-8, but only when the string has 50 characters or fewer.`);
  process.exit(1);
}
console.log("OK: each string encodes as TextEncoder encodes it.");

Output

FAULT lone high surrogate at the end, short (2 characters): last bytes 61 ed a0 80, valid UTF-8: false, TextEncoder: 61 ef bf bd
ok    lone high surrogate at the end, long (62 characters): last bytes 78 78 61 ef bf bd, valid UTF-8: true, TextEncoder: 78 78 61 ef bf bd
FAULT lone low surrogate, short (3 characters): last bytes 61 ed b0 80 62, valid UTF-8: false, TextEncoder: 61 ef bf bd 62
ok    lone low surrogate, long (63 characters): last bytes 78 61 ef bf bd 62, valid UTF-8: true, TextEncoder: 78 61 ef bf bd 62
FAULT low surrogate before high surrogate, short (2 characters): last bytes ed b0 80 ed a0 80, valid UTF-8: false, TextEncoder: ef bf bd ef bf bd
ok    low surrogate before high surrogate, long (62 characters): last bytes ef bf bd ef bf bd, valid UTF-8: true, TextEncoder: ef bf bd ef bf bd
BUG PRESENT: 3 of 6 strings encode a lone surrogate as bytes that are not valid UTF-8, but only when the string has 50 characters or fewer.

The CI run of the repro (Node.js on Ubuntu): https://github.com/mark1russell7/wirelab/actions/runs/44229138782

Expected

Each string encodes as TextEncoder encodes it: a lone surrogate becomes EF BF BD, at each string length.

Possible fix

In utf8EncodeJs(), after the check for a surrogate pair, a value that is still in the range 0xD800 to 0xDFFF is a lone surrogate. The function can write U+FFFD for it. U+FFFD also has 3 bytes, so utf8Count() stays correct. I can send a pull request with a test if you want one.

An AI coding assistant (Claude Code) found this fault, wrote the repro and wrote this text.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions