From the course: Introduction to Prompt Engineering for Generative AI

Large language models

From the course: Introduction to Prompt Engineering for Generative AI

Start my 1-month free trial

Large language models

“

- [Instructor] So what is a large language model or what is a language model to begin with? For that, let's head over to GPT. And yes, that's a model related to ChatGPT. I'm going to go ahead and input the sentence, "I try to learn something new." And then I'm going to hit Submit. Now, the next word was every and then day. And you can see that I click on every and I get this nice little distribution or sort of percentages. Now, what do these mean? Now, we can think of the model getting to the word new and that's a sort of junction or an intersection. And the model needs to choose the next word. What happens if we create a sort of pie chart with these statistics? So I'm going to go ahead and say that every is almost 50%, so 49. And then there is a 21% chance of \n. That's a new line starting. Next up, there's everyday as one word. And that's 17%. And next there is each at about 6%. And when I say other, I mean pretty much everything else. Now, if I divided this into a roulette and sort of spun this, it's likely that I will get every but it's not guaranteed. Now, this sounds very simple but coming up with this distribution is quite incredible. In order to do that, these large language models are trained on huge, huge amount of texts, almost the entire internet, enormous amounts of books. And then on top of that, sometimes they're fine-tuned for particular tasks. Now, here it has the prompt, "I try to learn something new," and it comes up with the completion, every and then day. So a large language model takes an input and gives us some sort of token output.

Contents