Następny
Attention in transformers, step-by-step | Deep Learning Chapter 6 27 min 
You Can Still be Prescribed Leeches | Fascinating Horror Shorts 1 min 
The Insane Engineering of the P-38 Lightning 27 min 
Defunctland: Walt Disney's City of the Future, E.P.C.O.T. 50 min 
Susan Kare: Designing Icons & Graphics For the Original Mac 52 min 
Ground zero responders recount their haunting 9/11 experiences | 60 Minutes Archive 16 min 
The Insane Engineering of the F-14 Tomcat 30 min 
I Returned To The World's Most Polluted Country ( And Instantly Regretted It ) 37 min 
Kingmakers - 12 Minute Gameplay Overview Deep Dive 13 min 
The most expensive software bug in history... 6 min 
World's First Touchscreen MacBook 1 min 
When did we start using passports? - Kristin Surak 7 min 
The Polar Cruisers: Winter Climb to the Hanging Cities | AI 4K — Retrofutura 8 min 
Я загрузил ИИ в Скайрим… и создал чудовище 😈 26 min 
Digging a SECRET GARAGE Part 5 Major Progress 39 min 
В Ход Идёт Всё 😅 (Crossout) 17 min 
$10,000 Power Washer! 1 min 
WWDC 2026 Impressions: Yeah, That's About Right 17 min 
Why your best ideas usually start as bad ones | Think Like A Musician 5 min 
The magical forest realm hiding at the bottom of the ocean - Luka Seamus Wright and Salomé Buglass 6 min 
I Travelled Around The Entire Earth With $0 72 min 
BETA SQUAD MAFIA GAME: YOUTUBER ALL STARS EDITION 68 min 
Why American Houses Are So Flimsy 15 min 
The Hidden Engineering Behind the Falkirk Wheel 16 min 
Neighbours reaction #secretgarage 2 min 
Law professor calls the new pardon system “anarchy” #shorts 1 min 
The Awful Origins of "Sweet FA" | Fascinating Horror Shorts 1 min 
The Sahara desert is growing! #shorts #science #SciShow #sahara #desert 1 min 
The Insane Engineering of the V-22 Osprey 24 min 
How Nike Lost Its Way 19 min 
My Favorite Twilight Knockoff 101 min 
Defunctland: The Bizarre Garfield Dark Ride 34 min 
The incredible engineering of Venice - Stephanie H. Smith 7 min 
The Physics Behind the Thumb Trick 18 min 
Plasma Speaker DIY (AM Demodulator) 11 min 
Together, we can save the planet #shorts #earthday 1 min 
The Pi Song 4.0 (Memorize 400 Digits Of π) 6 min 
Why is everyone suddenly neurodivergent? 8 min 
Do you ever feel isolated amongst your peers? #shorts 1 min 
What are tariffs, and how do they work? 6 min 
The Los Angeles Aqueduct is Wild 23 min 
Why isn't Hisako Koyama more famous? #shorts 1 min 
Scientists Can't Agree On The First Ancient City 26 min 
That Time We Put Spiders in VR 12 min 
The GIANT Strawberry! #PeppaPig #Shorts 1 min 
1955 vs 2025, who actually had it better? 25 min 
Chelsea Finn: This is the State of the Art in Robotics 59 min 
The 22 provinces of China explained 26 min 
Tales VS Toys: Valentine's Spy Mission 💗 #PeppaPig #Shorts #toys #toyplay 1 min 
I sent robot forgeries to a handwriting expert 24 min 
Why Don’t We Use Mothballs Anymore? 12 min Demystifying attention, the key mechanism inside transformers and LLMs. Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support Special thanks to these supporters: https://www.3blue1brown.com/lessons/attention#thanks An equally valuable form of support is to simply share the videos. Demystifying self-attention, multiple heads, and cross-attention. Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support The first pass for the translated subtitles here is machine-generated and, therefore, notably imperfect. To contribute edits or fixes, visit https://www.criblate.com Звуковая дорожка на русском языке: Влад Бурмистров. ------------------ Here are a few other relevant resources Build a GPT from scratch, by Andrej Karpathy https://youtu.be/kCc8FmEb1nY If you want a conceptual understanding of language models from the ground up, @vcubingx just started a short series of videos on the topic: https://youtu.be/1il-s4mgNdI?si=XaVxj6bsdy3VkgEX If you're interested in the herculean task of interpreting what these large networks might actually be doing, the Transformer Circuits posts by Anthropic are great. In particular, it was only after reading one of these that I started thinking of the combination of the value and output matrices as being a combined low-rank map from the embedding space to itself, which, at least in my mind, made things much clearer than other sources. https://transformer-circuits.pub/2021/framework/index.html Site with exercises related to ML programming and GPTs https://www.gptandchill.ai/codingproblems History of language models by Brit Cruise, @ArtOfTheProblem https://youtu.be/OFS90-FX6pg An early paper on how directions in embedding spaces have meaning: https://arxiv.org/pdf/1301.3781.pdf ------------------ Timestamps: 0:00 - Recap on embeddings 1:39 - Motivating examples 4:29 - The attention pattern 11:08 - Masking 12:42 - Context size 13:10 - Values 15:44 - Counting parameters 18:21 - Cross-attention 19:19 - Multiple heads 22:16 - The output matrix 23:19 - Going deeper 24:54 - Ending ------------------ These animations are largely made using a custom Python library, manim. See the FAQ comments here: https://3b1b.co/faq#manim https://github.com/3b1b/manim https://github.com/ManimCommunity/manim/ All code for specific videos is visible here: https://github.com/3b1b/videos/ The music is by Vincent Rubinetti. https://www.vincentrubinetti.com https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown https://open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u ------------------ 3blue1brown is a channel about animating math, in all senses of the word animate. If you're reading the bottom of a video description, I'm guessing you're more interested than the average viewer in lessons here. It would mean a lot to me if you chose to stay up to date on new ones, either by subscribing here on YouTube or otherwise following on whichever platform below you check most regularly. Mailing list: https://3blue1brown.substack.com Twitter: https://twitter.com/3blue1brown Instagram: https://www.instagram.com/3blue1brown Reddit: https://www.reddit.com/r/3blue1brown Facebook: https://www.facebook.com/3blue1brown Patreon: https://patreon.com/3blue1brown Website: https://www.3blue1brown.com
Packages published by the creator behind this video.
This creator has not set their rates yet. Send a request and we will pass it on — the price is agreed before anything is charged.
A recurring seat in my content. Subscribers enter a queue — or a random draw when there are more subscribers than slots — and appear in the video itself, in the credits, or in the description.
per month
Price on request
A guaranteed appearance of at least 10 seconds in my next video. Price depends on where in the runtime the insert lands.
Price on request
A guaranteed appearance of at least 10 seconds inside a Short cut from one of my existing videos.
one-off
Price on request