Understanding Google's GPT-Killer- The Revolutionary Pathways Architecture [Storytime Saturdays]
The reason why their model Bard will be much more than a language model
Google’s Bard has gotten a lot of attention recently. There has been a lot of discussion on whether Bard will be able to compete with ChatGPT. A lot of people have been speculating on how Google will stack up against ChatGPT and Microsoft.
This seemed like the perfect topic to cover in my second Substack Newsletter, AI Made Simple. In it, I covered some of the design details that gave Google’s AI Models a huge advantage over GPT. In this email, I’ll summarize some of the details here, and then link to that post for an in-depth explanation-
Google’s 3 Pillars to Beating ChatGPT
The Context- In late 2021, Google published an article on their architecture for training models, called Pathways. Pathways promised to do things differently. How you ask? They had 3 very notable elements in their model training that were very different from the norm. Let’s cover them all.
Decision 1, Multi-Task learning- Traditional Machine Learning models are trained to do one thing. Typically, you will use one model for translating languages, another one for recognizing images etc. Google is taking a huge departure from that. Google will ‘train a single model to do thousands or millions of things’ (this is a quote from their publication). This statement might not shock many of you, since we saw models like ChatGPT also do the same thing. However, that is not so for the next element we talk about.
Decision 2, Multiple Senses- Take ChatGPT. For all its abilities, it can only handle text information. Google’s architecture goes beyond this. Models using the Pathways architecture, are trained in tasks handling images, text, and much more. This gives them a big edge over the other models, which can only handle a single modality.
Decision 3, Sparse Activation- The models needed to achieve the above two capabilities would be huge. More Neurons—> More capabilities. However, this increases costs. Sparse Activation reduces costs significantly. For any given task, only a portion of the neural network is activated. This saves costs significantly and allows different parts of your network to specialize in different tasks. This is remarkably similar to how our brains work.
To those of you that want to learn more, you can find my detailed breakdown below. However, before you go, I want to tell you about 2 exciting developments.
Firstly, you can now download this newsletter as a free Ebook. I’ve compiled the posts from January into one book and put them on Google Drive. You can download it here. Feel free to download and share it with whomever you want.
Secondly, I’m doing a limited-time deal on this newsletter. You will be able to buy a premium subscription to this newsletter, for 60% off. Think about all the savings!! Whether you’re looking to buy this newsletter for yourself, or as a gift for someone else (it’s super easy), now is a great time to take the next step. As always, all purchases are protected by a 2-Month Satisfaction guarantee. If within 60 days, you decide that this newsletter is not for you, just message me. I will give you a full refund.
For those of that want to learn more, go right ahead and on.
Loved the post? Hate it? Want to talk to me about your crypto portfolio? Get my predictions on the UCL or MMA PPVs? Want an in-depth analysis of the best chocolate milk brands? Reach out to me by replying to this email, in the comments, or using the links below.
Stay Woke,
Go kill all,
Devansh <3
Reach out to me on:
Instagram: https://www.instagram.com/iseethings404/
Message me on Twitter: https://twitter.com/Machine01776819
My LinkedIn: https://www.linkedin.com/in/devansh-devansh-516004168/
My content:
Read my articles: https://rb.gy/zn1aiu
My YouTube: https://rb.gy/88iwdd