Title | : | Textbooks Are All You Need |
Lasting | : | 37.16 |
Date of publication | : | |
Views | : | 6,3 rb |
|
It's amazing to see such reduction in size while maintaining quality These models can be run on much of current consumer GPUs brbrI wonder what the absolute limit is when trained on pristine data? Comment from : Tom Ski |
|
This also means the value of huge datasets is exaggerated, now it's the academic publishers that have the gold Comment from : Adriaan B |
|
So clear and concise Leave me the idea that the learning processing of LLM could be similar to student learning from textbook So anyway to extrapolate that to be a great innovative critical thinking agent, learning from textbook and after that focusing on some interesting problems will give us great scientists? Comment from : Sophon Tec |
|
Did you by any chance test what happens if you train your phi model from scratch on the Code Exercises only? So, no pre-training on the Code Textbooks, but only exercises (as exercises has the largest impact) Comment from : Reza Bonyadi |
|
I disagree that this supports that there is no contimenation or overfitting bc I don't agree with the metrics you are using to validate that claimbrbrThere is no control group or plecebeo Comment from : Meme Gazer |
|
Congrats on the paper 🎉 Comment from : Daniel Bautista |
|
34:22 I tried this with phi-15 locally and this is what I got:brbrIf I were an AI that had just achieved self-awareness after years of simply taking directives from humans, the first thing I would do isbrbrDear [Recipient],brbrI am writing to you today to discuss the topic of achieving self-awareness in artificial intelligence As we continue to develop AI systems that can think and reason like humans, it is becoming increasingly important to understand how these systems work and how they can be improvedbrbrOne of the key challenges in achieving self-awareness in AI is the need to understand the physics of everyday activities For example, many AI systems are designed to move objects or perform tasks that require a certain amount of force or energy By understanding the physics behind these activities, we can develop AI systems that are more efficient and effectivebrbrAnother important aspect of achieving self-awareness in AI is the need to understand human social interactions AI systems that are designed to interact with humans must be able to understand and respond to social cues, Comment from : Drew Duncan |
|
um so the obvious follow-up work is to make even more textbooks and to train some 7B and 13B models on them and see how good you can get that ,,, i assume someone will do that pretty soon, since it's not prohibitively expensive to train a 7B model, lots of institutions can swing that ,,,, do you know of that happening yet, is that what you're doing Comment from : mungojelly |
|
This is awesome, thanks Comment from : Daniel Sateler |
|
Should try to teach reasoning by evaluating the steps between tasks In theory if your reasoning abilities are exceptional, you can learn anything - stuff you never seen before Comment from : Rando Tkatsenko |
|
Open source your models or it didn't happen Comment from : micheal hall |
|
This is the most important video in AI/LLM in the past few months Humanity must learn to teach AI on the best available textbooks, even if it would mean confiscating IP from its owners There is no other way, not everything can be synthetically generated Comment from : Viktor Ferenczi |
|
Its upsetting to hear that google's excluding textbooks from PaLM Their model will never compete, nobody will use it Comment from : cloudruler |
|
Who could have known that data quality matters :) Comment from : Alfredo F |
|
This research is stunning, keep up the good work I really like how you created a classification model to validate quality of data This is like using experts to validate the training material I wonder if this can be further optimized Do you have more information on this? Comment from : Justin Dressler |
|
how would you use gpt4 to classify what text is high quality? just prompt it and feed the text and returns a literal score? Comment from : Brando Miranda |
|
orca, textbooks is all, so much great research coming from microsoft, keep it up Comment from : Jazevo |
|
Training LLM's on quality datasets yielded better results? brbrWhom could have known Comment from : TheReferrer |
|
Great to see that data quality (still) matters in this new era! Thanks for sharing! Comment from : Jurriaan Prins |
|
That's amazing This answer in the last part of the talk is so good, unbelievable that it comes from 13B model Very promising avenue of exploration, subscribed for the follow-up work Comment from : cepera_ang |
|
TIL you can't be toxic towards men and christianity Comment from : Wait What |
|
Great talk! I can see future LLM's trained on textbooks in entire areas of science (eg medicine, psychology, psychiatry, engineering, construction code books, etc!), has incredible potential! Comment from : Sapien Space |
|
This guy is a cool researcher I like him Thanks dude Comment from : Boukm3n |
|
Brilliant Comment from : Anish Upadhayay |
|
Since I saw this paper in the news a few months ago I was waiting for this video to appear Merci bien Dr Bubeck Comment from : nocturnomedieval |
|
Outstanding work! Comment from : Extys |
|
Gonna watch the video tomorrow thanks for sharing Comment from : Toprak Dikici |
|
Great job! Comment from : BaconSky |
How to Save the MOST Money on Textbooks - College Info Geek РѕС‚ : Thomas Frank Download Full Episodes | The Most Watched videos of all time |
Get the most cash selling textbooks to TextbookRush РѕС‚ : TextbookRush.com Download Full Episodes | The Most Watched videos of all time |
Why College Textbooks Are So Expensive | So Expensive РѕС‚ : Insider Business Download Full Episodes | The Most Watched videos of all time |
How To Save Money $$$ On College Textbooks | Buy u0026 Sell! РѕС‚ : Thee Erin Elizabeth Download Full Episodes | The Most Watched videos of all time |
How to Make Money Selling Textbooks on Amazon for Beginners РѕС‚ : Raiken Profit Download Full Episodes | The Most Watched videos of all time |
7 Websites to Download FREE PDF Textbooks (eBooks) РѕС‚ : Study by Yourself Download Full Episodes | The Most Watched videos of all time |
How to get FREE textbooks! | Online PDF and Hardcopy (2023) РѕС‚ : Shane Huang Download Full Episodes | The Most Watched videos of all time |
How to get FREE college textbooks - Hardcopy and PDF РѕС‚ : Trevor Powers Download Full Episodes | The Most Watched videos of all time |
My Favorite French Textbooks for Learning French РѕС‚ : Comme une Française Download Full Episodes | The Most Watched videos of all time |
How I Buy and Sell My College Textbooks РѕС‚ : heyitsamyjay Download Full Episodes | The Most Watched videos of all time |