Undetectable, undefendable back-doors for machine learning

IMPNet will give your model hallucination and you can’t stop it.

Machine learning’s promise is decisions at scale: using software to classify inputs (and, often, act on them) at a speed and scale that would be prohibitively expensive or even impossible using flesh-and-blood humans.

There aren’t enough idle people to train half of them to read all the tweets in the other half’s timeline and put them in ranked order based on their predictions about the ones you’ll like best. ML promises to do a good-enough job that you won’t mind.

Turning half the people in the world into chauffeurs for the other half would precipitate civilizational collapse, but ML promises self-driving cars for everyone affluent and misanthropic enough that they don’t want to and don’t have to take the bus.

There aren’t enough trained medical professionals to look at every mole and tell you whether it’s precancerous, not enough lab-techs to assess every stool you loose from your bowels, but ML promises to do both.

All to say: ML’s most promising applications work only insofar as they do not include a “human in the loop” overseeing the ML system’s judgment, and even where there are humans in the loop, maintaining vigilance over a…