⚪ NEOTLIX.SPACE — Every creator is a director. Every fan is a future star! ⚪ Anyone Can Star in Any Video ⚪ Rent Out Your Face ⚪ Monetize Your Videos ⚪ Turn Your Following into Royalties ⚪ See How It Works ⚪

Up next

But what is cross-entropy? | Compression is Intelligence Part 2 But what is cross-entropy? | Compression is Intelligence Part 2 34 min 《胶囊计划:英雄》第10话:醒【热血 | 科幻 | 武侠 | Made By Bilibili】 《胶囊计划:英雄》第10话:醒【热血 | 科幻 | 武侠 | Made By Bilibili】 16 min ✨️приключения в мире капитализма✨️ ✨️приключения в мире капитализма✨️ 1 min 太子继位的局势即将尘埃落定,秦牧却带着“先帝”杀回来了 #牧神记 #国漫 #shorts #donghua #madebybilibili 【加入会员专享最新集】 太子继位的局势即将尘埃落定,秦牧却带着“先帝”杀回来了 #牧神记 #国漫 #shorts #donghua #madebybilibili 【加入会员专享最新集】 1 min THE FLAMINGO DANCE | Official Surreal AI Music Video THE FLAMINGO DANCE | Official Surreal AI Music Video 5 min 《时光代理人 第三季》第5话:接风宴【治愈 | 奇幻 | 悬疑 | Made By Bilibili】 《时光代理人 第三季》第5话:接风宴【治愈 | 奇幻 | 悬疑 | Made By Bilibili】 23 min 《牧神记》第99话:医毒斗法 《牧神记》第99话:医毒斗法 21 min 《凡人修仙传》第190话花絮:谷双蒲亮明内奸身份,陆道友惨遭毒手?【加入会员专享最新集】 《凡人修仙传》第190话花絮:谷双蒲亮明内奸身份,陆道友惨遭毒手?【加入会员专享最新集】 5 min 夏斐暂住竟是早有安排?#时光代理人 #LinkClick #shorts #MadeByBilibili【加入会员专享最新集】 夏斐暂住竟是早有安排?#时光代理人 #LinkClick #shorts #MadeByBilibili【加入会员专享最新集】 1 min Любая историческая передача Любая историческая передача 2 min The Story Of The Black Tom Explosion | Fascinating Horror The Story Of The Black Tom Explosion | Fascinating Horror 11 min I tested every new Apple Product! I tested every new Apple Product! 15 min iPhone Duo Hands On! iPhone Duo Hands On! 2 min SEGA SUNSET SEGA SUNSET 4 min None an Island None an Island 4 min Insta360 X6: The Vlogging Camera I Didn't Expect! Insta360 X6: The Vlogging Camera I Didn't Expect! 16 min I tested Dream Tech with my Biggest Fans! I tested Dream Tech with my Biggest Fans! 15 min 2 LATE 2 LATE 2 min I Said Yes to Every Email for a Month! (Again) I Said Yes to Every Email for a Month! (Again) 31 min Pardon brokers defend their business #shorts Pardon brokers defend their business #shorts 1 min This Might Actually Work This Might Actually Work 1 min Disgusting Freak Should Be Arrested Disgusting Freak Should Be Arrested 12 min Mr Rizz IRL Mr Rizz IRL 1 min What a Wild Journey What a Wild Journey 23 min Why won’t AI agents just follow the rules? Why won’t AI agents just follow the rules? 36 min Milkoi - LAST TIME | Glitch Hop | NCS - Copyright Free Music Milkoi - LAST TIME | Glitch Hop | NCS - Copyright Free Music 3 min Mount Wilshire: The Volcano That Almost Destroyed LA | Fascinating Horror Mount Wilshire: The Volcano That Almost Destroyed LA | Fascinating Horror 11 min The vulnpocalypse might not be so bad after all The vulnpocalypse might not be so bad after all 34 min How to Use YouTube Studio in 2026! (Complete Guide) How to Use YouTube Studio in 2026! (Complete Guide) 19 min AI Agents vs Business Rules: Which Should Make Decisions? AI Agents vs Business Rules: Which Should Make Decisions? 11 min Shot Down Over Iran, The Pardon Economy, Battering Ram | 60 Minutes Full Episodes Shot Down Over Iran, The Pardon Economy, Battering Ram | 60 Minutes Full Episodes 44 min OpenAI talks GPT-6 Astra and Millenium Prize, researchers create WeWorm exploit & IBM’s US Open app OpenAI talks GPT-6 Astra and Millenium Prize, researchers create WeWorm exploit & IBM’s US Open app 34 min This mic sounds like it costs way more 🎙️ This mic sounds like it costs way more 🎙️ 1 min How to Start a Faceless YouTube Channel in 2026! How to Start a Faceless YouTube Channel in 2026! 18 min Barbarella (2027) Edgar Wright & Sydney Sweeney Barbarella (2027) Edgar Wright & Sydney Sweeney 9 min Claude Fable 5 is smarter💡 Claude Fable 5 is smarter💡 1 min The Largest Biological Attack in US History | Fascinating Horror The Largest Biological Attack in US History | Fascinating Horror 11 min "Nobody Leaves Without Paying" - The Ycuá Bolaños Fire | Fascinating Horror "Nobody Leaves Without Paying" - The Ycuá Bolaños Fire | Fascinating Horror 12 min IBM’s cloud collab, Meta’s Muse Glimmer & OpenAI’s upcoming Astra model IBM’s cloud collab, Meta’s Muse Glimmer & OpenAI’s upcoming Astra model 37 min The Luna Ultra Just Got Better! (Big Firmware Update!) The Luna Ultra Just Got Better! (Big Firmware Update!) 9 min This Mars rover's cool. But have you seen its wheels? This Mars rover's cool. But have you seen its wheels? 23 min Dude Perfect's $100M Business, Mapped Dude Perfect's $100M Business, Mapped 13 min 1000 feet of wireless range on a mic this small 🤯 1000 feet of wireless range on a mic this small 🤯 1 min Canada in 4K HDR – Lake Louise, Waterfalls & Real Nature Sounds | Ep. 17, Part 2 Canada in 4K HDR – Lake Louise, Waterfalls & Real Nature Sounds | Ep. 17, Part 2 56 min Solo in the Most Remote Corner of the East Coast Solo in the Most Remote Corner of the East Coast 76 min Jackass: Best and Last (2026) 4K - Escape Room From Hell (Part 2) (9/10) | Movieclips Jackass: Best and Last (2026) 4K - Escape Room From Hell (Part 2) (9/10) | Movieclips 4 min Old Florida - The Last Outlaws of the Everglades Old Florida - The Last Outlaws of the Everglades 79 min Better Call Saul - Gus Confronts Mike (S4E4) | Movieclips Better Call Saul - Gus Confronts Mike (S4E4) | Movieclips 2 min Steven Seagal’s Wild Voodoo Sword Fight | Belly of the Beast Steven Seagal’s Wild Voodoo Sword Fight | Belly of the Beast 9 min The Cave (2005) - Spooky Rock Climber Scene (4/12) | Movieclips The Cave (2005) - Spooky Rock Climber Scene (4/12) | Movieclips 3 min Superman 2 Man of Tomorrow #shorts #superman #superman2 #supermanmanoftomorrow Superman 2 Man of Tomorrow #shorts #superman #superman2 #supermanmanoftomorrow 1 min

But what is cross-entropy? | Compression is Intelligence Part 2

2026 34 min

Where the loss function for training LLMs comes from. Job opportunities aligned to this audience: https://3b1b.co/talent Early views and other perks for supporters: https://3b1b.co/support Home page: https://www.3blue1brown.com Part 1: https://youtu.be/l6DKRf-fAAM If you want to learn more about Lagrange Multipliers, I made multiple videos and articles for the topic back in my Khan Academy days: https://www.khanacademy.org/math/multivariable-calculus/applications-of-multivariable-derivatives/lagrange-multipliers-and-constrained-optimization/v/constrained-optimization-introduction Manim animations by Aaron Gostein and Grant Sanderson NanoGPT animation by Clayton Rabideau 3d black-box model by Paul Dancstep Chess game and Lagrange Multiplier scenes by Nishad Deulkar Music by Vince Rubinetti Timestamps 0:00 - Language trees and zipping 3:02 - Recap optimal codes 5:20 - Defining cross-entropy 8:26 - Intuition and examples 12:59 - Application to language trees 14:55 - Pre-training LLMs 20:38 - What makes this loss function best? 26:13 - Distillation 30:12 - 3b1b Talent 31:35 - KL Divergence ------------------ These animations are largely made using a custom Python library, manim. See the FAQ comments here: https://3b1b.co/faq#manim Music by Vincent Rubinetti. https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown https://open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u ------------------ 3blue1brown is a channel about animating math, in all senses of the word animate. If you're reading the bottom of a video description, I'm guessing you're more interested than the average viewer in lessons here. It would mean a lot to me if you chose to stay up to date on new ones, either by subscribing here on YouTube or otherwise following on whichever platform below you check most regularly. Mailing list: https://3blue1brown.substack.com Twitter: https://twitter.com/3blue1brown Bluesky: https://bsky.app/profile/3blue1brown.com Instagram: https://www.instagram.com/3blue1brown Reddit: https://www.reddit.com/r/3blue1brown Facebook: https://www.facebook.com/3blue1brown Patreon: https://patreon.com/3blue1brown Website: https://www.3blue1brown.com

Work with this creator

Packages published by the creator behind this video.

All packages

This creator has not set their rates yet. Send a request and we will pass it on — the price is agreed before anything is charged.

Monthly participation

per month

A recurring seat in my content. Subscribers enter a queue — or a random draw when there are more subscribers than slots — and appear in the video itself, in the credits, or in the description.

Price on request

Sign in to request

Guaranteed placement — new video

one-off

A guaranteed appearance of at least 10 seconds in my next video. Price depends on where in the runtime the insert lands.

Price on request

  • First 1–3 minutes
  • Middle of the video
  • End of the video
Sign in to request

Guaranteed placement — Short

one-off

A guaranteed appearance of at least 10 seconds inside a Short cut from one of my existing videos.

Price on request

Sign in to request

More like this

《胶囊计划:英雄》第10话:醒【热血 | 科幻 | 武侠 | Made By Bilibili】

《胶囊计划:英雄》第10话:醒【热血 | 科幻 | 武侠 | Made By Bilibili】

Made By Bilibili - Chinese Donghua Animation · Animation

✨️приключения в мире капитализма✨️

✨️приключения в мире капитализма✨️

Eva Morozova · Animation

太子继位的局势即将尘埃落定,秦牧却带着“先帝”杀回来了 #牧神记 #国漫 #shorts #donghua #madebybilibili 【加入会员专享最新集】

太子继位的局势即将尘埃落定,秦牧却带着“先帝”杀回来了 #牧神记 #国漫 #shorts #donghua #madebybilibili 【加入会员专享最新集】

Made By Bilibili - Chinese Donghua Animation · Animation

THE FLAMINGO DANCE | Official Surreal AI Music Video

THE FLAMINGO DANCE | Official Surreal AI Music Video

Vivad · Animation · Fantasy · Music · Surreal · Short Film · Cinematic · creative · Life · ambience · video · Fashion · scenes · stories

《时光代理人 第三季》第5话:接风宴【治愈 | 奇幻 | 悬疑 | Made By Bilibili】

《时光代理人 第三季》第5话:接风宴【治愈 | 奇幻 | 悬疑 | Made By Bilibili】

Made By Bilibili - Chinese Donghua Animation · Animation · Trailer

《牧神记》第99话:医毒斗法

《牧神记》第99话:医毒斗法

Made By Bilibili - Chinese Donghua Animation · Animation

《凡人修仙传》第190话花絮:谷双蒲亮明内奸身份,陆道友惨遭毒手?【加入会员专享最新集】

《凡人修仙传》第190话花絮:谷双蒲亮明内奸身份,陆道友惨遭毒手?【加入会员专享最新集】

Made By Bilibili - Chinese Donghua Animation · Animation

夏斐暂住竟是早有安排?#时光代理人 #LinkClick #shorts #MadeByBilibili【加入会员专享最新集】

夏斐暂住竟是早有安排?#时光代理人 #LinkClick #shorts #MadeByBilibili【加入会员专享最新集】

Made By Bilibili - Chinese Donghua Animation · Animation · Trailer

Любая историческая передача

Любая историческая передача

Eva Morozova · Animation

The Story Of The Black Tom Explosion | Fascinating Horror

The Story Of The Black Tom Explosion | Fascinating Horror

Fascinating Horror · Horror · Documentary · Mystery · War · History · Music · Other · education · true stories · stories

I tested every new Apple Product!

I tested every new Apple Product!

Mrwhosetheboss · Music · Tech · technology · science

iPhone Duo Hands On!

iPhone Duo Hands On!

Mrwhosetheboss · Music · Tech · technology · science

SEGA SUNSET

SEGA SUNSET

LORN · Music

None an Island

None an Island

LORN · Music

Insta360 X6: The Vlogging Camera I Didn't Expect!

Insta360 X6: The Vlogging Camera I Didn't Expect!

Think Media · Action · Music · Walk · kit · video · education · tests

I tested Dream Tech with my Biggest Fans!

I tested Dream Tech with my Biggest Fans!

Mrwhosetheboss · Music · Tech · video · technology · science · gadgets · kids

2 LATE

2 LATE

LORN · Music

I Said Yes to Every Email for a Month! (Again)

I Said Yes to Every Email for a Month! (Again)

Marques Brownlee · Music · Gaming · Life · daily life · daily · Tech · video · technology · science

Pardon brokers defend their business #shorts

Pardon brokers defend their business #shorts

60 Minutes · Other

This Might Actually Work

This Might Actually Work

penguinz0 · Other

Disgusting Freak Should Be Arrested

Disgusting Freak Should Be Arrested

penguinz0 · Other

Mr Rizz IRL

Mr Rizz IRL

MrBeast · Other

What a Wild Journey

What a Wild Journey

penguinz0 · Other

Why won’t AI agents just follow the rules?

Why won’t AI agents just follow the rules?

IBM Technology · Other · Updates · video · reports · tools · security · education

Milkoi - LAST TIME | Glitch Hop | NCS - Copyright Free Music

Milkoi - LAST TIME | Glitch Hop | NCS - Copyright Free Music

NoCopyrightSounds · Music · Other · Gaming · video · Songs

Mount Wilshire: The Volcano That Almost Destroyed LA | Fascinating Horror

Mount Wilshire: The Volcano That Almost Destroyed LA | Fascinating Horror

Fascinating Horror · Horror · Documentary · Mystery · History · Music · Other · education · true stories · stories

The vulnpocalypse might not be so bad after all

The vulnpocalypse might not be so bad after all

IBM Technology · Other · Updates · video · tools · security · education

How to Use YouTube Studio in 2026! (Complete Guide)

How to Use YouTube Studio in 2026! (Complete Guide)

Think Media · Music · Walk · testing · video · tools · education · tips

AI Agents vs Business Rules: Which Should Make Decisions?

AI Agents vs Business Rules: Which Should Make Decisions?

IBM Technology · Updates · video · education

Shot Down Over Iran, The Pardon Economy, Battering Ram | 60 Minutes Full Episodes

Shot Down Over Iran, The Pardon Economy, Battering Ram | 60 Minutes Full Episodes

60 Minutes · History · Interviews · video · reports

OpenAI talks GPT-6 Astra and Millenium Prize, researchers create WeWorm exploit & IBM’s US Open app

OpenAI talks GPT-6 Astra and Millenium Prize, researchers create WeWorm exploit & IBM’s US Open app

IBM Technology · Action · Other · Updates · video · tools · security · science · education

This mic sounds like it costs way more 🎙️

This mic sounds like it costs way more 🎙️

Think Media · video · tools · education · tips

How to Start a Faceless YouTube Channel in 2026!

How to Start a Faceless YouTube Channel in 2026!

Think Media · Music · video · tools · education · tips

Barbarella (2027) Edgar Wright & Sydney Sweeney

Barbarella (2027) Edgar Wright & Sydney Sweeney

Film Royalty · Animation · Action · Updates · Trailer · Movie News · filmmaking · video · science · driver · tips

Claude Fable 5 is smarter💡

Claude Fable 5 is smarter💡

IBM Technology · education

The Largest Biological Attack in US History | Fascinating Horror

The Largest Biological Attack in US History | Fascinating Horror

Fascinating Horror · Horror · Documentary · Mystery · History · Music · Other · city · Country · education · true stories · stories

"Nobody Leaves Without Paying" - The Ycuá Bolaños Fire | Fascinating Horror

"Nobody Leaves Without Paying" - The Ycuá Bolaños Fire | Fascinating Horror

Fascinating Horror · Horror · Documentary · Mystery · Crime · History · Music · Other · daily · education · true stories · stories

IBM’s cloud collab, Meta’s Muse Glimmer & OpenAI’s upcoming Astra model

IBM’s cloud collab, Meta’s Muse Glimmer & OpenAI’s upcoming Astra model

IBM Technology · Other · NVIDIA · Updates · video · tools · education

The Luna Ultra Just Got Better! (Big Firmware Update!)

The Luna Ultra Just Got Better! (Big Firmware Update!)

Think Media · Music · Other · pov · Life · video · tools · education · tips

This Mars rover's cool. But have you seen its wheels?

This Mars rover's cool. But have you seen its wheels?

Tom Scott · photography · drive · flying · education

Dude Perfect's $100M Business, Mapped

Dude Perfect's $100M Business, Mapped

Colin and Samir · Music · cooking · car · video · education

1000 feet of wireless range on a mic this small 🤯

1000 feet of wireless range on a mic this small 🤯

Think Media · moments · video · tools · education · tips

Canada in 4K HDR – Lake Louise, Waterfalls & Real Nature Sounds | Ep. 17, Part 2

Canada in 4K HDR – Lake Louise, Waterfalls & Real Nature Sounds | Ep. 17, Part 2

4K Relaxation Channel · Music · nature · relax · Traveling · video · Relaxation · Scenic · relaxing · Nature Relaxation · scenes

Solo in the Most Remote Corner of the East Coast

Solo in the Most Remote Corner of the East Coast

Peter Santenello · Adventure · Documentary · War · Family · Other · trip · Country · village · drive · video · food · India · scenes

Jackass: Best and Last (2026) 4K - Escape Room From Hell (Part 2) (9/10) | Movieclips

Jackass: Best and Last (2026) 4K - Escape Room From Hell (Part 2) (9/10) | Movieclips

Movieclips · Animation · Trailers · moments · MOVIECLIPS · movie clips · scenes · lines · favorite films

Old Florida - The Last Outlaws of the Everglades

Old Florida - The Last Outlaws of the Everglades

Peter Santenello · Adventure · War · Family · Other · Life · drive · video · India · scenes · stories

Better Call Saul - Gus Confronts Mike (S4E4) | Movieclips

Better Call Saul - Gus Confronts Mike (S4E4) | Movieclips

Movieclips · Animation · Drama · Trailers · moments · MOVIECLIPS · movie clips · scenes · lines · favorite films

Steven Seagal’s Wild Voodoo Sword Fight | Belly of the Beast

Steven Seagal’s Wild Voodoo Sword Fight | Belly of the Beast

FilmIsNow Epic Scenes · Animation · Horror · Action · moments · movie clips · scenes

The Cave (2005) - Spooky Rock Climber Scene (4/12) | Movieclips

The Cave (2005) - Spooky Rock Climber Scene (4/12) | Movieclips

Movieclips · Animation · Horror · Thriller · Trailers · moments · MOVIECLIPS · movie clips · scenes · lines · favorite films

Superman 2 Man of Tomorrow #shorts #superman #superman2 #supermanmanoftomorrow

Superman 2 Man of Tomorrow #shorts #superman #superman2 #supermanmanoftomorrow

Film Royalty · Animation · Action · Superhero · Cinematic · Trailer · moments · scenes · tips