John Oliver, AI, and Media Accuracy
Published on: Feb 28, 2023Filed under: Technology Media
“Knoll’s Law of Media Accuracy” [1] is a concept that asks why you find flaws in media coverage of for subjects you know really well, but then somehow accept stories for areas in which you’re not an expert.
The cynical takeaway is that you treat every story, regardless of whether you’re knowledgeable in the subject, with the same level of skepticism.
So, when I saw that Last Week Tonight was doing a story on AI , I got a little worried. I like the show. I enjoy their deep dives into subjects that I might not generally research. But AI? That’s sorta my jam. What if the show just gets it horribly wrong and I have to reevaluate my previously cribbed positions?
Spoiler - they did pretty good!
I didn’t pick up anything that was egregiously incorrect. There were plenty of oversimplifications, but those seemed to have been done in service of both run time and the needs of the audience. I specifically liked the cancer detector over-indexing on rulers when classifying growths as malignant. Another common example was a Dog or Wolf classifier that over-indexed on the presence of snow due to biased training data [2].
[Aside - the [2] link is from a 2016 paper advocating for explainable deep learning models, so suffice it to say, the conclusions raised in Last Week Tonight aren’t exactly new in the field. They’re just expensive.]
I also was very appreciative of the show calling out the difference between Narrow and General AI. And that LLMs like ChatGPT are confident bullshiters - something I mentioned in my breakdown [3].
All in all, I found the segment to provide a good high level understanding of the recent advances in AI.