Parth Parikh has created a Pong game with a twist. It features integrated LLM-based commentary generation.

I’ve wanted to do this for the longest time — in fact, back in my undergrad days, I used to carry around a GitHub Gist to store such ideas, and an entry for this one dates back to Feb 16 2020. That’s five years! Anyway, for the longest time we never had the right technology to pull this off in a cost-effective, near-realtime manner.

However, late last month, with OpenAI’s gpt-4o-mini-tts, we finally have the resources to do it! I envision a future where gpt-4o-mini-tts-like software is integrated on the edge — that is, edge LLMs. When that happens, next-generation gaming consoles will leverage such technology, opening up a whole universe of possibilities in gaming, especially in sports-simulation titles.

As someone who played Pong on a home system, I can say that this makes the game much better. Can you imagine this with better games?