Very late at the party…. I know.
Hand spectacle if you have used a generative AI, like Cat,, Gemini Or In depth? Well, complete class (or almost).
Now keep your hand if you know how Genai works? Oh… 😅
If you had to place your hand, don’t feel bad! It is a complex subject. Your role may benefit from using The most recent * (I say the most recent .. we will see on this subject) Bring Tech, but “I am not a scientist of the data, I don’t need to learn how he does what he does !!”. To this public, I say … Where is your sense of curiosity and wonder? 😛
Ok, jokes aside. If you are interested in a quick (No Technical expertise required) Friend in the way Genai does what he does, join me in the coming minutes! Warning: If you rather seek to increase your use of these technologies (for example: how to write better prompts?), Or interested in mathematical details, this is not the right blog for this.
Three stages of knowledge:
- How “AI” works to start
- Which makes “generative ai” any other
- So – how Genai works then
In a great simplification: Predictive modeling (whose automatic learning infamous is a branch) is most often used as “supervised – You choose something to be expected and you try to do it using other related things to which you have access. The “something” is called a target (as it is the aim of your request), and the things you use are called features. (The two have other names, it is in addition to today).
Oh – and automatic learning is just a subset of AI (Although both are often wrongly interchangeably used), and it is also generator AI. Here is a nice blog explaining all this if you are interested, credit the author.
Imagine that you are stuck in a meeting, and you do not have windows immediately to answer this for you: how can you predict Time outside? Well .. usually look at it. Although “local” weather reports are not always exact in your location (if you live in England like me, you know how much everything changes). But there are other indicators: temperature, season, how people are dressed and – especially – what time did 2 hours ago, during your last speech to have a coffee? ☕
All these elements (and many others) will give you a good indication of what time is probably outside. Current time is your targetand all the indicators mentioned above your features. It will not always be perfect, but quite decent! In a word, it’s automatic learning:
The study of the update of algorithms is suitable for the deduction of future results based on observed data.
Absolutely not the definition, but it captures what we need. OK – SO: AI is used to predict, on the basis of related things. How is AI GenerativeAn AI subtype, different then?
Nothing. There are many false ideas surrounding the Genai, so I want to quickly attack two:
- How it works is so different from other AI!
No. Always predicts things. Ok, let’s be pedantic – when “things” are collected and stored, we call it data. He uses data to predict data. (We will enter the how Next section)
It’s much better than that. But new? It depends on how you quantify that I suppose. Never heard Eliza? Or maybe Google’s cat (and really funny)?
But why is it recently that we are all surprised by its quality? Two main reasons, one easier to grasp than the other. The first: Data. The more things you know (now we know, the data), the better you will have to predict the related problems. We have collected so much data over the years, it’s amazing.
The second: Context. Remember the days that Google Translate was fun, if not ridiculous sometimes? Disappeared for a long time. If you are more technically fashionable, the words that you are probably impatient to see me write are: Attention mechanism.
Imagine the weather earlier. It was sunny all day, the report you saw this morning showed a clear sky, and yet … you go out, and a rain spell from nowhere! You should have expected. After all, you are in London. Unpredictability is the name of the game here, and in addition, you have seen these unpleasant clouds there are hours…. ☔
Why is all this relevant? Due to the context. Genai uses a mechanism to apply the importance of the context in its surrounding text / image / whatever the generation called attention. In short:
The models will make respect for The most relevant / important aspects of features and surrounding context.
Thisis indeed new. You can know more about the Technical Deets HEREBut for our ends, remember – context and a lot of data.
Summary of what we know so far:
- Genai predicts a “target” using data.
- Genai uses a “attention mechanism” to enforce the context.
So the only question is really: what is the target, and what is the data he uses to predict? The answer, of course, differs according to what we generate (images? Text? Videos?). However, everything at this level of data science can be represented as vectors. Although we are not going to worry about what it is even, understand that – if you know how the “text” is generated, you can develop more (because they are all vectors!).
How is the text generated then?
By predicting two things:
I. The word missing in a sentence, based on other words and the global surrounding context.
> Principle: You deliberately hide part of a sentence and ask your model to predict it. What would make sense over there? Have I ever seen similar things?
This is known as Masked language modeling (Mlm))
In the example above, without more context, the “flowers” seem very likely. “Cars” is a cry far, but “broccoli”, although a little dark, is possible. This is a large part of the reason why the generation of text has a certain degree of arbitrariness – there is no “correct” choice for this MLM problem.
II The following sentence, based on those previous and contextual surrounding contexts.
> Principle: You feed the options of model 2 and have it choose the one that follows “logically” a given sentence. What would our normal human texts look like?
This is known as the following sentence prediction (Nsp))
In this case, there should be a clear winner.
And that’s it!
There are so many, so much more to learn about Genai, but you should now, hopefully, you feel more confident the next time you use your favorite LLM, you understand how it does its job.
I hope you enjoyed it next time. 👋
This blog was not written with the help of a generative AI. Although it would certainly have helped = D
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~bo ~~ ~~~~~~~~~~~~
I am open to answer any other question in private-send me a message on LinkedIn or by email 🙂
Find me on: Github || Liendin or by e-mail at [email protected]
