Reading Log - Jan 23 - 29, 2023
Published on: Jan 29, 2023Filed under: Scratch
This past week proved unusually full, and didn’t leave a lot of time for reading or engaging with larger thoughts. What I did get through, however, is largely language related. Or language tangential.
Do Large Language Models learn world models or just surface statistics?
In my overview of ChatGPT, I noted that Large Language Models (LLM) fit nicely into the Chinese Room Thought Experiment. The above paper tests that thought experiment by focusing on the expertise side. That is, even though model lacks context - for instance what a tree actually is outside of a description and statistics - does the model actually have a working understanding of what is or isn’t true?
For the paper, a LLM was trained on a string of Othello moves, using the list of moves as sentences. A second model was generated based on world state (a represent of the board at the current moment).
Since LLMs work by predicting the next word given the previous words, the team wanted to see how the model would…react… given an interrupted new state. The second model was used as a comparison - a human-understandable expert so to say.
The results?
It did pretty good!
Would ChatGPT get a Wharton MBA [PDF]
Next up was the headline grabbing story that seemed everywhere - that ChatGPT earned an MBA!
It didn’t.
The professor used the model to pass a single test from a single MBA course with a B-. And it did so with mixed results due to, well, basically spitting out the probable answers and not really having a contextual and internalized representational understanding of the MBA material.
I guess Knoll’s Law [x] strikes again.
Orwell Was Right
A comic? Look, it was a busy week. And this one was good. It’s on assumptions and stereotyping, and it inspired me to buy a book of Orwell’s essay.
HD TVs Ruined Sitcoms
Finally, not language, but formatting, so I’m letting it in. This is a video essay on how the shift from standard to high definition (and 4:3 to wide screen) has changed television. As the author points out, when it comes to sitcoms, that change hasn’t always been for the better. One note, the author says that the HD restoration of The Wire is a good example of upscaling. Despite having The Wire on Blu Ray, I still haven’t watched it because the claustrophobia and uncertainty of the original aspect ratio help a ton with the storytelling.