Como Is Infosec

Content moderation is a security problem.

Cory Doctorow
6 min readAug 7, 2022

--

Cryteria, CC BY 3.0 (modified)/Crosa, CC BY 2.0 (modified)

in·fo·sec (/ˈinfōˌsek/): information security

co·mo (/koh-moh/): content moderation

Content moderation is really, really hard.

Content moderators:

  • seek to set conversational norms, steering transgressors toward resources that help them better understand the local conversational rules;
  • respond to complaints from users about uncivil, illegal, deceptive or threatening posts;
  • flag or delete material that crosses some boundary (for example, deleting posts that dox other users, or flagging posts with content warning for adult material or other topics);
  • elevate or feature material that is exemplary of community values;
  • adjudicate disputes about impersonation, harassment and other misconduct.

This is by no means a complete list!

There are many things that make content moderation hard. For starters, there are a lot of instances in which there is no right answer. A user may sincerely not intend to harass another user, who might, in turn, sincerely feel themself to be harassed. Deciding which of these is the case is an art, not a science.

But there is an another side to como, one that’s far more adversarial: detecting, interdicting and undoing deliberate attempts to circumvent the limitations imposed by content moderation policies. Some examples:

Some of these tactics are social, others are technical, and many straddle the boundary…

--

--

Cory Doctorow

Writer, blogger, activist. Blog: https://pluralistic.net; Mailing list: https://pluralistic.net/plura-list; Mastodon: @pluralistic@mamot.fr