OpenAI Introduces Text Watermarking in Response to EU AI Act

OpenAI has announced a new text watermarking approach to comply with the EU AI Act, with a target false positive rate of 1% for watermark detection. The technology achieves an 80% detection rate for 200-token passages and 95% for 400-token passages. However, detection rates vary significantly by content type, with mathematics showing substantially lower performance compared to psychology. For example, using 10% synonyms in 400-token passages reduces detection from 92% to 66%, and with 25% synonyms, it drops to 17%.

The Astra model scores 49.57 points for unwatermarked text and 49.76 for watermarked text, while other models like AutomationBench and DeepSWE show similar trends with slight variations in scores. OpenAI plans to roll out text watermarking for eligible ChatGPT and Codex users in the EU in the coming weeks, while API customers globally can opt in today for select models. Additionally, approved researchers and expert organizations will gain access to a text watermark detector in the upcoming weeks. OpenAI acknowledges that text watermarking and detection are still early technologies with significant limitations, stating that detection improves with text length but is overall lower for rigid content types. They also emphasize that a watermark does not measure human contribution and expect to revisit their approach as technology and standards evolve.