Backdooring a summarizerbot to shape opinion

Model spinning maintains accuracy metrics, but changes the point of view.

Cory Doctorow
6 min readOct 21, 2022

--

An old fashioned hand-cranked meat-grinder; a fan of documents are being fed into its hopper; its output mouth has been replaced with the staring red eye of HAL9000 from 2001: A Space Odyssey; emitting from that mouth is a stream of pink slurry. Image: Cryteria (modified) https://commons.wikimedia.org/wiki/File:HAL9000.svg CC BY 3.0 https://creativecommons.org/licenses/by/3.0/deed.en PublicBenefit https://commons.wikimedia.org/wiki/File:Texture.png Jollymon001 https://commons.wikimedia.org/w

What’s worse than a tool that doesn’t work? One that does work, nearly perfectly, except when it fails in unpredictable and subtle ways. Such a tool is bound to become indispensable, and even if you know it might fail eventually, maintaining vigilance in the face of…

--

--