GPT-4 creator Ilya Sutskever interview

Here’s a fascinating interview with the creator of GPT-4, Ilya Sutskever.

Some fascinating takeaways:

  • It sounds like Ilya believes that AI might be more than just a collection of statistical predictions, and that eventually, given enough time, data, scale, and further improvements, it might actually develop understanding.
  • we can’t know the limits of a model. We might assume with high confidence that there are certain limitations to a given model, but it’s very possible that our current assumptions will turn out false.
  • prediction = compression
  • predicting the next word or the next pixel well enough could lead to things we’re not even considering yet
  • hallucinations can be reduced through reinforcement learning from humans
  • interactions with humans can help reduce hallucinations and improve outputs
  • large language models learn compressed representations of the world world processes that produce data
  • it’s possible to learn more from less data






Leave a Reply

Your email address will not be published. Required fields are marked *