下一段
Attention in transformers, step-by-step | Deep Learning Chapter 6 27 分钟 
The Basement: Jeremy Corbell | Betrayal, Surveillance, and the Secret Higher Than Nuclear Weapons 240 分钟 
Defunctland: The History of Disneyland's Adventure Thru Inner Space 36 分钟 
Why the US is Struggling to Land on the Moon 20 分钟 
Disney's FastPass: A Complicated History 103 分钟 
The Basement: Diana Pasulka | Angels, Vatican Archives, and a Government Email 215 分钟 
Omega Red (2027) Dakoda Shepley & Ryan Reynolds 10 分钟 
Ramayana: Jai Jai Ram (Song) | Namit M, Nitesh T | Ranbir, Yash | AR Rahman, Shreya, Arijit S, Kumar 3 分钟 
The Unsolved Mystery of The Oakville Blobs | Fascinating Horror Shorts 1 分钟 
Brokers are offering felons a path to Trump pardons — at a steep price with no guarantees 14 分钟 
The Best Car I've Ever Driven: McLaren W1 17 分钟 
EMPTY GESTURE 2 分钟 
Live Service Games - Scott The Woz 29 分钟 
Don't Deep Fry Your Turkey | Fascinating Horror Shorts 1 分钟 
Budget Gaming - Scott The Woz 54 分钟 
LORN - UPSIDE DOWN COPS 4 分钟 
Framework 13 Pro: The Modular Laptop is Real! 13 分钟 
NOT TODAY 3 分钟 
ONE OF US 3 分钟 
This Cheese is Rotting, Maggot-Infested, Illegal and... Delicious? | Fascinating Horror Shorts 1 分钟 
My Thoughts on Resident Evil Movie 19 分钟 
Would You Ride an UNDERWATER Slide? | Fascinating Horror Shorts 1 分钟 
How a broken heart affects your body 6 分钟 ![A stroll around my hometown, Aldridge (and the stories that shaped my youth) [PART 1]](https://adm.6-9.space/storage/films/posters/yt_a-stroll-around-my-hometown-aldridge-and-the-stories-that-shaped-my-youth-part-1_1789704139.jpg)
A stroll around my hometown, Aldridge (and the stories that shaped my youth) [PART 1] 40 分钟 
BETA SQUAD MAFIA GAME: YOUTUBER ALL STARS EDITION 68 分钟 
Inside Maine’s Forgotten Company Town 57 分钟 
The Ohio State Penitentiary Fire | Fascinating Horror 12 分钟 
What Is the AI Security Trilemma? Smart, Fast, or Secure AI? 10 分钟 
The #1 Worst Drink for Your Gut 2 分钟 
S̶i̶l̶k̶s̶o̶n̶g̶ NakeyJakey is Outdated 42 分钟 
Your Next Budget PC Upgrade is...Custom Watercooling?? 25 分钟 
Crazy Larper Scammed $1.3 Million from Victims 23 分钟 
These Are Some Wild Bodycams 12 分钟 
Turns out building dry stone walls is difficult 25 分钟 
Edgar Allan Poe's story of the Red Death - Iseult Gillespie 8 分钟 
Have you HEARD this?? 2 分钟 
The Dark Side of the World’s Lightest Structural Metal 16 分钟 
The Story Of The Black Tom Explosion | Fascinating Horror 11 分钟 
What’s at the center of the earth?: Crash Course Geology #4 10 分钟 
What Was Pokemon Shock? | Fascinating Horror Shorts 1 分钟 
Happy Teacher Appreciation Week! 1 分钟 
Send this to a dad you know 1 分钟 
Debunking the bulls**t 10 分钟 
the prompting trick nobody teaches you 1 分钟 
Why your best ideas usually start as bad ones | Think Like A Musician 5 分钟 
Baby Learns to Be Brave | Courage Song | Good Values for Kids | Preschool Songs | BabyBus 21 分钟 
My Donkey Got a Boo-Boo | Caring & Doctor Song for Preschoolers | BabyBus 24 分钟 
How might LLMs store facts | Deep Learning Chapter 7 23 分钟 
The Dark Truth of the Panama Canal 41 分钟 
Our projects create A LOT of damage #Aiper #AiperIrriSense2 #SummerWithAiper #PrimeDay #2026PrimeDay 2 分钟 
Rate our gaming setup 🎮 #gta6 #gaming #desksetup 1 分钟 Demystifying attention, the key mechanism inside transformers and LLMs. Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support Special thanks to these supporters: https://www.3blue1brown.com/lessons/attention#thanks An equally valuable form of support is to simply share the videos. Demystifying self-attention, multiple heads, and cross-attention. Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support The first pass for the translated subtitles here is machine-generated and, therefore, notably imperfect. To contribute edits or fixes, visit https://www.criblate.com Звуковая дорожка на русском языке: Влад Бурмистров. ------------------ Here are a few other relevant resources Build a GPT from scratch, by Andrej Karpathy https://youtu.be/kCc8FmEb1nY If you want a conceptual understanding of language models from the ground up, @vcubingx just started a short series of videos on the topic: https://youtu.be/1il-s4mgNdI?si=XaVxj6bsdy3VkgEX If you're interested in the herculean task of interpreting what these large networks might actually be doing, the Transformer Circuits posts by Anthropic are great. In particular, it was only after reading one of these that I started thinking of the combination of the value and output matrices as being a combined low-rank map from the embedding space to itself, which, at least in my mind, made things much clearer than other sources. https://transformer-circuits.pub/2021/framework/index.html Site with exercises related to ML programming and GPTs https://www.gptandchill.ai/codingproblems History of language models by Brit Cruise, @ArtOfTheProblem https://youtu.be/OFS90-FX6pg An early paper on how directions in embedding spaces have meaning: https://arxiv.org/pdf/1301.3781.pdf ------------------ Timestamps: 0:00 - Recap on embeddings 1:39 - Motivating examples 4:29 - The attention pattern 11:08 - Masking 12:42 - Context size 13:10 - Values 15:44 - Counting parameters 18:21 - Cross-attention 19:19 - Multiple heads 22:16 - The output matrix 23:19 - Going deeper 24:54 - Ending ------------------ These animations are largely made using a custom Python library, manim. See the FAQ comments here: https://3b1b.co/faq#manim https://github.com/3b1b/manim https://github.com/ManimCommunity/manim/ All code for specific videos is visible here: https://github.com/3b1b/videos/ The music is by Vincent Rubinetti. https://www.vincentrubinetti.com https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown https://open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u ------------------ 3blue1brown is a channel about animating math, in all senses of the word animate. If you're reading the bottom of a video description, I'm guessing you're more interested than the average viewer in lessons here. It would mean a lot to me if you chose to stay up to date on new ones, either by subscribing here on YouTube or otherwise following on whichever platform below you check most regularly. Mailing list: https://3blue1brown.substack.com Twitter: https://twitter.com/3blue1brown Instagram: https://www.instagram.com/3blue1brown Reddit: https://www.reddit.com/r/3blue1brown Facebook: https://www.facebook.com/3blue1brown Patreon: https://patreon.com/3blue1brown Website: https://www.3blue1brown.com
Packages published by the creator behind this video.
This creator has not set their rates yet. Send a request and we will pass it on — the price is agreed before anything is charged.
A recurring seat in my content. Subscribers enter a queue — or a random draw when there are more subscribers than slots — and appear in the video itself, in the credits, or in the description.
per month
Price on request
A guaranteed appearance of at least 10 seconds in my next video. Price depends on where in the runtime the insert lands.
Price on request
A guaranteed appearance of at least 10 seconds inside a Short cut from one of my existing videos.
one-off
Price on request