Up next
Attention in transformers, step-by-step | Deep Learning Chapter 6 27 min 
China's Hidden Wilderness | Double Episode - Wild China | BBC Earth 
They Left The Tourist Trail: Kris Kremers and Lisanne Froon | Fascinating Horror 11 min 
How the Suez Canal changed history 3 min 
Wildlife Of Japan & China In 4K | BBC Earth 42 min 
5 life-changing Linux tips 1 min 
Night-time filming in the Okavango Delta 🐆 #BigCats247 #BigCats #Lions #Cheetahs #Leopards #Shorts 3 min 
The Bath Curse Tablets | Fascinating Horror Shorts 1 min 
DefunctTV: The History of Under the Umbrella Tree 26 min 
Godzilla x Kong: Supernova #shorts #godzilla #godzillaxkong #godzillaxkongsupernova 1 min 
The Great Gaming Gift Grift - Scott The Woz 22 min 
MSDOS 5 min 
STRAIGHT PIPES 2 min 
NO SAFETY ON A BODY 2 min 
iOS 27 Hands-On: Top 5 New Features! 16 min 
Spinning in a Dream 2 min 
Nothing Phone 4b: They Can't Say It! 11 min 
UNDEAD DOGS 1 min 
The Best Car I've Ever Driven: McLaren W1 17 min 
Sec. Hegseth’s initial reaction to the U.S. fighter jet being shot down: “S***” #shorts 1 min 
Can I help Harry find his estranged mother from this childhood cemetery photo? 28 min 
✨Living Alone Diaries | reorganize the kitchen and rooms, having Heytea, & set up the lab space 12 min 
How Many Countries Can I Visit in 7 Days? 43 min 
OVERGROWTH — Dead City | Post Apocalyptic Action Thriller | 2026 Full Movie | English | 4K 62 min 
McVay doesn’t “buy into those external narratives” about Rams’ season #shorts 1 min 
GOODBYE BALL 😂☄️ 1 min 
The Childhood Game Collection - Scott The Woz 138 min 
JUMANJI: Open World (2026) Will It Be Game Over? 11 min 
How to Use YouTube Studio in 2026! (Complete Guide) 19 min 
The Luna Ultra Just Got Better! (Big Firmware Update!) 9 min 
Best 4K Multicam Setup for Live Events! (Full Gear Breakdown) 12 min 
The vulnpocalypse might not be so bad after all 34 min 
10 years of AI reporting | 60 Minutes Marathon 214 min 
LAST TO LEAVE THE FOOTBALL PITCH 63 min 
The mist that keeps the Tyne Tunnels safe 22 min 
IBM’s cloud collab, Meta’s Muse Glimmer & OpenAI’s upcoming Astra model 37 min 
The Gameshow That Killed 73 People | Fascinating Horror 11 min 
Fixing Tom Brady’s YouTube Channel 1 min 
Why Love Island Might be the Most Valuable View on the Internet 1 min 
What Was Pokemon Shock? | Fascinating Horror Shorts 1 min 
Dude Perfect's $100M Business, Mapped 13 min 
❤️ Find Your Intersection with Content 1 min 
The Jagermeister Pool of Death | Fascinating Horror Shorts 1 min 
Dodo Learns to Make Right Decisions | Good Habits | Sheriff Labrador | Kids Cartoon | BabyBus 56 min 
He Built The Ultimate Spy Tool (Free and Open-Source) 22 min 
EXTREME Hide & Seek (Hunting my Employees 2024) 24 min 
Jensen Huang: The Mindset That Built NVIDIA 49 min 
wHeRE's tHe PowEr ArmOR? 😵💫🤖 #fallout #engineering #videogames 1 min 
Baby Evie Starts TEETHING! 🦷😩 #PeppaPig #Shorts 1 min 
Hole in the Wall! #PeppaPig #Shorts 1 min 
Peppa & Mandy Find CHEESE in the Clubhouse 🧀 Peppa Pig Full Episodes | 1 Hour of Kids Cartoons 32 min Demystifying attention, the key mechanism inside transformers and LLMs. Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support Special thanks to these supporters: https://www.3blue1brown.com/lessons/attention#thanks An equally valuable form of support is to simply share the videos. Demystifying self-attention, multiple heads, and cross-attention. Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support The first pass for the translated subtitles here is machine-generated and, therefore, notably imperfect. To contribute edits or fixes, visit https://www.criblate.com Звуковая дорожка на русском языке: Влад Бурмистров. ------------------ Here are a few other relevant resources Build a GPT from scratch, by Andrej Karpathy https://youtu.be/kCc8FmEb1nY If you want a conceptual understanding of language models from the ground up, @vcubingx just started a short series of videos on the topic: https://youtu.be/1il-s4mgNdI?si=XaVxj6bsdy3VkgEX If you're interested in the herculean task of interpreting what these large networks might actually be doing, the Transformer Circuits posts by Anthropic are great. In particular, it was only after reading one of these that I started thinking of the combination of the value and output matrices as being a combined low-rank map from the embedding space to itself, which, at least in my mind, made things much clearer than other sources. https://transformer-circuits.pub/2021/framework/index.html Site with exercises related to ML programming and GPTs https://www.gptandchill.ai/codingproblems History of language models by Brit Cruise, @ArtOfTheProblem https://youtu.be/OFS90-FX6pg An early paper on how directions in embedding spaces have meaning: https://arxiv.org/pdf/1301.3781.pdf ------------------ Timestamps: 0:00 - Recap on embeddings 1:39 - Motivating examples 4:29 - The attention pattern 11:08 - Masking 12:42 - Context size 13:10 - Values 15:44 - Counting parameters 18:21 - Cross-attention 19:19 - Multiple heads 22:16 - The output matrix 23:19 - Going deeper 24:54 - Ending ------------------ These animations are largely made using a custom Python library, manim. See the FAQ comments here: https://3b1b.co/faq#manim https://github.com/3b1b/manim https://github.com/ManimCommunity/manim/ All code for specific videos is visible here: https://github.com/3b1b/videos/ The music is by Vincent Rubinetti. https://www.vincentrubinetti.com https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown https://open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u ------------------ 3blue1brown is a channel about animating math, in all senses of the word animate. If you're reading the bottom of a video description, I'm guessing you're more interested than the average viewer in lessons here. It would mean a lot to me if you chose to stay up to date on new ones, either by subscribing here on YouTube or otherwise following on whichever platform below you check most regularly. Mailing list: https://3blue1brown.substack.com Twitter: https://twitter.com/3blue1brown Instagram: https://www.instagram.com/3blue1brown Reddit: https://www.reddit.com/r/3blue1brown Facebook: https://www.facebook.com/3blue1brown Patreon: https://patreon.com/3blue1brown Website: https://www.3blue1brown.com
Packages published by the creator behind this video.
This creator has not set their rates yet. Send a request and we will pass it on — the price is agreed before anything is charged.
A recurring seat in my content. Subscribers enter a queue — or a random draw when there are more subscribers than slots — and appear in the video itself, in the credits, or in the description.
per month
Price on request
A guaranteed appearance of at least 10 seconds in my next video. Price depends on where in the runtime the insert lands.
Price on request
A guaranteed appearance of at least 10 seconds inside a Short cut from one of my existing videos.
one-off
Price on request