Next Token Prediction Might Make PyTorch-like Frameworks Redundant [Video]

LLMs are becoming the default solution for many of the problems that businesses and researchers face. Even when it comes to domains outside language and text, people have been experimenting with LLMs for guessing and predicting the next token, which sparks an interesting conversation around the need for other tools like PyTorch, as LLMs can just do the whole job in the future.

Interestingly, according to Andrej Karpathy, these models are more like tools designed to predict the next piece of a sequence, whether that’s words, images, or other types of information. This next token prediction framework can be a universal tool for solving a wide variety of problems, beyond text. 

“If that is the case, it’s also possible that deep learning frameworks (e.g. PyTorch and friends) are way too general for what most problems want to look like over time,” said Karpathy. 

It’s a bit sad and confusing that LLMs (“Large Language Models”) …