Thread · 3 tweets · 30 Jan 2019

Worse than that, it's hard to disentangle from context. We've been evaluating an algorithm to assess comment toxicity because at first brush it seemed to be quite racist: if you just changed names from statistically White to Black toxicity went up sharply.

But the fact is: racist comments, which are evidently highly toxic, are much, much more likely to contain words relating to being Black than White. The algorithm isn't so much being racist as correctly pointing out that a comment is more likely to be toxic if it mentions race.

Machine learning is currently not much ready to go beyond this sort of statistical association. Defining fairness under such limitations is at best difficult indeed.