MS Dashboard (@Demo) - v133.3.1
How ChatGPT Works: The Model Behind The Bot
A brief introduction to the intuition and methodology behind the chat bot you can’t stop hearing about.
This gentle introduction to the machine learning models that power ChatGPT, will start at the introduction of Large Language Models, dive into the revolutionary self-attention mechanism that enabled GPT-3 to be trained, and then burrow into Reinforcement Learning From Human Feedback, the novel technique that made ChatGPT exceptional.
Large Language Models
ChatGPT is an extrapolation of a class of machine learning Natural Language Processing models known as Large Language Model (LLMs). LLMs digest huge quantities of text data and infer relationships between words within the text. These models have grown over the last few years as we’ve seen advancements in computational power. LLMs increase their capability as the size of their input datasets and parameter space increase.
The most basic training of language models involves predicting a word in a sequence of words. Most commonly, this is observed as either next-token-prediction and masked-language-modeling.
Read more:
https://medium.com/towards-data-science/how-chatgpt-works-the-models-behind-the-bot-1ce5fca96286
Recent messages from
