0:08 聽到這些真的讓我感到很不舒服。 謝謝您的介紹，但我不喜歡聽到關於我自己的事情。
0:19 就像一場表演一樣，好吧，也許你們可以鼓掌。
0:26 你們當中有多少人知道輝達是誰？
0:33 你們當中有多少人知道GPU 是什麼？
0:41 好的，很好，我不需要改變我說話的方式。
0:46 女士、先生、羅森鮑姆校長、尊敬的各位教職員、
0:52 尊敬的賓客、自豪的家長們， 最重要的是，加州理工學院2024 屆的畢業生。
1:08 對你們來說這真是快樂的一天。 你們得顯得更興奮一點。 你們知道你們將從加州理工學院畢業。
1:14 這是一所偉大的理查費曼、萊納斯鮑林的學校，
1:21 還有對我和我們這個行業影響很大的人，卡弗·米德。
1:27 是的，這是一件大事。
1:33 今天是個令人無比自豪和喜悅的日子。 這是你們所有人的夢想成真，
1:39 but not just for you 但不只適合你們 因為你的父母和家人做出了無數的犧牲
1:46 to see you reach this milestone. 看到你們達到這個里程碑。 所以讓我們藉此機會祝賀他們，感謝他們，
1:53 and let them know you love them. 並讓他們知道你愛他們。
2:04 你不會想忘記這一點 因為你不知道你還要在家裡住多久。
2:14 你今天要感到非常感恩。
2:19 作為一個自豪的家長，我很高興我的孩子們沒有搬出去， 每天看到他們真是太好了，
2:27 但現在他們搬走了，這讓我很難過。 所以希望你們能花一些時間陪伴父母。 你在這裡的旅程證明了你的性格、
2:35 決心、願意做出犧牲 為你的夢想而努力，你應該感到自豪。
2:41 有能力做出犧牲、忍受痛苦和磨難，
2:46 你在生活中將需要這些品質。 你和我有一些共同點。
2:55 首先，輝達的兩位首席科學家都來自加州理工學院。
3:07 我今天發表這次演講的原因之一是因為我在招募。
3:16 我想告訴你們，輝達是一家非常棒的公司， I’m a very nice boss, 我是個很好的老闆，
3:24 universally loved, 深受人們喜愛， come work at NVIDIA. 來輝達工作吧。
3:30 你們和我都對科學和工程充滿熱情， 儘管我們相差了40歲，
3:40 我們都正處於事業的巔峰。
3:47 對於一直關注輝達和我本人的各位， 你知道我的意思。
3:52 只是就你的情況來說， 你還要攀登更多的山峰。
3:59 我只是希望今天不是我的巔峰。 還不是巔峰。
4:05 所以我會一如既往地努力工作 以確保我未來能攀登更多的高峰。
4:11 去年，我很榮幸在台灣大學發表畢業典禮演講，
4:16 我分享了幾個關於輝達歷程的故事 我們所學到的可能對畢業生有價值的經驗教訓。
4:23 我必須承認我不喜歡給建議， 尤其是對別人的孩子。
4:34 所以我今天的建議很大程度上會隱藏在一些故事裡
4:43 我很享受 以及一些我享受的生活經驗。
4:51 我相信，我是當今世界上任職時間最長的科技公司執行長。
5:02 31 年來，我努力維持生意不破產， 不會感到無聊，也不會被解僱。
5:09 因此我很榮幸能享受人生的許多經歷，
5:16 從創建輝達開始，從無到有，到今天。
5:21 我談到了創建CUDA 的漫長道路， 一個程式設計模型。 我們花了20 多年發明的程式設計模型，
5:30 這正在徹底改變當今的計算技術。 我談到了一個非常公開的、取消的
5:39 我們參與的世嘉遊戲機項目， 以及知識的誠實，
5:44 我知道理查費曼關心的是 非常深入地討論過，也常談論過，
5:50 正是知識的誠實和謙遜拯救了我們的公司。 撤退，策略性撤退，是我們最好的策略之一。
6:01 所有這些都是我在畢業典禮上談到的違反直覺的教訓。
6:09 但我鼓勵畢業生參與人工智慧，
6:15 我們這個時代最重要的科技。 我稍後會再談一下這一點，
6:21 但你們都知道人工智慧。 很難不沉浸其中、被它包圍，
6:26 並對此進行了大量討論。 當然，我希望大家都能使用它，玩它，
6:33 取得令人驚訝的結果和一些奇蹟 有些令人失望，有些令人感到驚訝。
6:41 但你必須享受它，你必須參與其中， 因為它進展得太快了。 這是我所知道的唯一技術
6:48 即同時以多個指數級方式推進。 所以技術變化非常非常快。
6:55 所以我建議台灣大學的學生們跑步不要走路，
7:03 並參與人工智慧革命。 然而，一年後，它發生的變化令人難以置信。
7:11 所以今天我想跟大家分享我的觀點，
7:17 從我的角度來看，一些正在發生的重要事情 你們即將畢業。
7:24 這些都是正在發生的非凡的事情 你們應該有一個直觀的理解，
7:31 因為這對你們來說很重要， 這對於整個產業來說都很重要。 希望你們能充分利用眼前的機會。
7:40 電腦產業正在從基礎開始轉型， 實際上是來自基柱。
7:46 從基柱開始，一切都在改變。 並且跨越每一個層面，很快，每個產業也都會改變。
7:54 原因很明顯， 因為今天的計算機 是最重要的知識工具。
8:00 它是每個行業和每個科學領域的基礎。 如果我們對計算機的改造如此深刻，
8:08 當然，它會對每個行業產生影響。 我稍後會談論這一點。
8:14 當你們進入工業界時， 了解正在發生的事情很重要。 現代計算可以追溯到IBM System 360。
8:24 那就是我學習的建築手冊。 這是一本你不需要學習的建築手冊。
8:29 許多更好的文檔 以及對計算機和構築的更好的描述
8:35 自那時起就被提出。 但System 360 在當時非常重要，
8:43 事實上，System 360 的基本思想和架構，
8:49 System 360 的主要思想、架構與策略
8:57 至今仍在統治著電腦產業。 它是在我出生一年後推出的。
9:04 80年代，我是第一代超大規模積體電路工程師之一
9:11 學會了設計晶片 來自米德和康威的里程碑式教科書。
9:17 我不確定這裡是否還教授這門課。 它應該是在超大規模積體電路系統的介紹中。
9:22 基於加州理工學院卡弗·米德的開創性工作 晶片設計方法和徹底改變積體電路設計的教科書。
9:31 它使我們這一代能夠設計出超級晶片，並最終設計出CPU。
9:38 CPU 導致運算能力呈指數級增長、 性能、令人驚嘆的技術進步，
9:48 這就是所謂的摩爾定律， 推動了資訊科技革命。 我們參與的工業革命，
9:56 我這一代人也參與其中， 實現了世界上前所未見的事物的大規模生產。
10:03 大規模生產那些看不見、容易複製的東西， 軟體的大規模生產。
10:09 並催生出了一個價值3兆美元的產業。 當我坐在你這個位置時，資訊科技產業還很小。
10:19 透過銷售軟體賺錢的想法只是幻想。
10:25 但今天它是最重要的商品之一， 最重要的技術
10:32 以及我們產業生產的產品創造。 然而，NARD 縮放、電晶體縮放的極限，
10:42 和指令級並行性降低了CPU 的效能。 CPU 效能提升正在減緩
10:50 此時，計算需求持續呈指數級增長。 這種不斷擴大的差距
10:59 計算需求與計算機能力之間，
11:06 如果不加以解決，計算能源消耗和成本、通貨膨脹，
11:12 最終將扼殺所有產業。 正如我們所言，我們看到了計算通膨的非常明顯的跡象。
11:20 And after two decades of advancing NVIDIA’s CUDA, 20多年的先進的輝達CUDA， 輝達的加速運算提供了一條前進的道路。
11:28 這就是我來這裡的原因。 因為最終，業界意識到 加速運算的驚人有效性
11:38 幾十年後，我們終於看到了計算通膨的跡象。
11:45 將耗時的演算法轉移到GPU 專注於並行處理，
11:52 我們通常可以實現10 倍、100 倍，有時甚至1000 倍的加速，
12:00 節省金錢、成本和能源。 我們現在加速電腦圖形等應用領域，
12:06 當然還有射線追蹤、基因定序、科學計算， 天文學、量子電路模擬、
12:14 SQL資料處理，甚至pandas，資料科學。 加速計算已到達臨界點。
12:21 這是我們對電腦產業的第一個偉大貢獻， 我們對社會的第一個偉大貢獻是加速運算。
12:31 它為我們提供了可持續計算的前進道路。 成本將繼續下降
12:37 因為計算需求不斷增加。 節省一百倍的時間、成本或能源
12:49 加速運算的開放必將引發
12:55 其他地方的新發展。 我們只是不知道那是什麼 直到深度學習進入我們的意識。
13:05 一個全新的計算世界出現了。 傑夫辛頓、亞歷克斯沙舍夫斯基和伊利亞蘇斯克弗
13:11 使用輝達CUDA GPU 訓練AlexNet 震驚了電腦視覺界
13:17 贏得2012 年ImageNet 挑戰賽。 這是重要時刻，
13:23 深度學習的大爆炸， 這是人工智慧革命開始的關鍵時刻。
13:32 改變公司後我們的決定
13:38 這是值得注意的。 我們在AlexNet 之後所做的決定改變了我們的公司以及其他一切。
13:47 我們看到了深度學習的潛力，並相信， 只是透過原理思考相信，
13:54 透過我們自己對深度學習可擴展性的分析，我們確信這一點。
13:59 我們相信這種方法可以學習其他有價值的功能。 深度學習可能是通用函數學習器
14:06 有多少問題很難 或無法用基本第一原理來表達。
14:13 當我們看到這個的時候， 當我們看到這個的時候，我們想 這是我們必須關注的技術
14:21 因為它的限制可能僅受模型和資料規模的限制。
14:29 然而，當時也存在一些挑戰。 這是2012年，2012年剛過不久。
14:35 我們如何探索深度學習的極限 無需建構這些龐大的GPU 叢集？
14:42 當時我們是一家小公司， 建構這些大規模GPU 叢集可能要花費 數億美元。
14:49 如果我們不這樣做， 如果我們擴大規模，沒有人能保證它一定會有效。
14:56 然而，沒有人知道深度學習的規模能達到多大。 如果我們不建造它，我們就永遠不會知道。
15:04 這是其中之一，如果你建造它，他們會來嗎？ 我們的邏輯是，如果我們不建造它，他們就來不了。
15:10 因此我們致力於 基於我們最初的原則信念和分析。
15:16 我們已經到了關鍵點 我們相信這會非常有效 當公司相信某件事時，我們就應該採取行動。
15:24 所以我們深入研究深度學習， 並在接下來的十年中系統地重新發明了一切。
15:31 我們重新發明了每個運算層，從GPU 本身開始。 現代GPU 的發明，
15:38 這與過去的GPU 有很大不同 我們首先發明的
15:44 我們繼續發明了計算的幾乎所有方面， 互連、系統、
15:50 網絡，當然還有軟體。 我們投資了數十億美元。
15:55 我們向未知領域投資了數十億美元。 數千名工程師十多年來
16:01 致力於深度學習以及深度學習的推展與擴展 我們並不清楚這項技術能走多遠。
16:11 我們投資了數十億美元。 我們設計並製造了超級計算機 探索深度學習和人工智慧的極限。
16:18 2016 年，我們發布了DGX-1，
16:23 我們的第一台人工智慧超級計算機， 我向舊金山的一家新創公司交付了第一台
16:31 無人知曉的創業公司， 我的一群朋友正在研究人工智慧，
16:40 一家名為OpenAI 的公司。 2022 年，AlexNet 誕生10 年後，
16:49 之後計算量增加了一百萬倍， 一百萬倍。
16:54 如果你能想像一下，如果你的筆記型電腦 能力強一百萬倍？
17:01 一百萬倍後，OpenAI 推出了ChatGPT， 人工智慧成為了主流。
17:08 在這十年中，輝達從圖形公司轉型為 你們很多人可能第一次知道我們是GPU 製造商，
17:17 到現在的一家建立海量資料中心的人工智慧公司 規模超級計算機。
17:24 我們徹底改變了我們的公司。 我們也徹底改變了計算方式。
17:30 當今計算的基本方式已經發生了根本性的改變。
17:35 計算堆疊現在使用GPU 處理大型語言模型
17:41 在超級電腦上進行訓練 而不是CPU
17:47 它們是程式設計師所寫的處理指令。
17:52 我們現在正在創建人類無法編寫的軟體。 我們現在正在創建能夠完成人類無法完成的事情的軟體
17:59 想像一下，即使只是10 年前。 計算機現在是意圖驅動而不是指令驅動。
18:08 告訴計算機你想要什麼，它就會找到答案。 和人類一樣，人工智慧應用將理解任務，
18:20 推理、計劃、 並組織一個大型語言模型團隊來執行任務。
18:28 未來的應用程式將與我們做事的方式非常相似，
18:33 組成專家團隊、使用工具、 推理和計劃，並執行我們的使命。
18:41 軟體以及軟體的功能已經徹底改變。 甚至我們的產業，隨著它的變化和轉型，
18:51 創造了另一個產業， 這是一個世界上前所未有的行業。
18:56 一個產業正在我們眼前形成。 AI的輸入和輸出都是詞元。
19:06 對在座的所有工程師來說，你們都知道我的意思。 這些是嵌入智慧的浮點數。
19:14 公司現在正在建設 一種以前不存在的新型資料中心，專門用於
19:20 生產智能詞元。 本質上就是人工智慧工廠。
19:29 就像尼古拉·特斯拉發明的交流發電機
19:36 過去的工業革命， 我們現在有了人工智慧詞元生成器，
19:43 它們將成為新工業革命的工廠。 這裡有大型工業生產能源和電力。
19:51 我們現在有一個龐大的產業，生產一種看不見的東西，叫做軟體。 在未來，在不久的將來，
19:58 我們將有生產、製造的產業 智慧詞元、人工智慧生成器。
20:07 一種新的計算模型已經出現， 一個新的行業已經出現，
20:13 因為我們從第一原理推理， 形成了我們對未來的信念，並根據這些信念採取行動。
20:22 下一波人工智慧浪潮是機器人技術， 除了語言模型之外，人工智慧還
20:30 還有物理世界模型。 我們與數百家機器人製造公司合作，
20:38 機器人車輛、拾放臂、人形機器人、 甚至整個巨大的倉庫都由機器人控制。
20:46 但與我們的人工智慧工廠策略和經驗不同， 這是透過推理和深思熟慮的行為形成的，
20:55 我們的機器人之旅源自於一連串的挫折。
21:01 眾所周知，輝達發明了GPU。 這是在我們發明人工智慧工廠之前。
21:07 我們對電腦產業的第一個偉大貢獻是 透過可程式著色器重塑電腦圖形。
21:15 我們在2000 年發明了GPU 和可程式著色。 我們希望將GPU 整合到每台電腦中，
21:22 所以我們開始將GPU 與主機板晶片結合起來， 我們推出了一款出色的整合式顯示卡
21:30 當時適用於AMD CPU。 我們的晶片組業務立即獲得了成功。
21:37 我認為它幾乎是在一夜之間就從零增長為十億美元。 但突然之間，AMD 想要
21:44 控制電腦中的所有技術， 我們想保持獨立，
21:50 所以他們收購了ATI，不再需要我們。 我們向英特爾求助。
22:01 這可能不是一個好主意， 但我們轉向了英特爾
22:06 並協商了連接英特爾CPU 的授權。
22:11 蘋果對我們正在開發的產品感到興奮
22:16 並要求我們和他們一起開發一台新電腦， 這就是第一款MacBook Air。
22:23 英特爾看到了發生的事情並決定 他們不想讓我們再這樣做了，
22:29 所以他們終止了我們的協議。 好吧，我們又一次改變了主意，這次我們獲得了ARM 的許可，
22:39 我們打造了一款低功耗SoC、一款行動SoC， 世界上第一個本質上是電腦的SoC，
22:48 一台功能齊全的計算機，這真是令人難以置信。 我們的晶片讓Google興奮不已，
22:55 他們要求我們開發一款新設備， 原來是Android 行動裝置。
23:02 高通決定不讓我們這麼做， 所以他們不想讓我們連接他們的數據機，
23:09 如果不連接調變解調器就很難建構行動裝置。 而且當時也沒有其他LTE 數據機公司，
23:17 所以我們不得不退出行動裝置市場。 嗯，這幾乎是在一年裡發生的，
23:26 我們會建造一些東西， 它將獲得巨大的成功， 產生巨大的興奮感，
23:32 一年後， 我們被趕出了這些市場。 嗯，沒有更多的市場可以求助，
23:40 我們決定建造一些東西 我們確定沒有顧客的地方，
23:50 因為有一件事你絕對可以保證
23:55 其中沒有顧客的地方， 也沒有競爭對手， and nobody cares about you. 沒有人在乎你。
24:02 所以我們選擇了一個沒有客戶的市場，一個價值0 億美元的市場，
24:08 它是機器人技術。 我們製造了世界上第一台機器人計算機
24:14 處理無人理解的演算法 當時被稱為深度學習。
24:19 這已經是10多年前的事了。 十年後，我非常開心
24:25 憑藉我們已建立的 以及創造下一波人工智慧浪潮的機會。 更重要的是，
24:31 我們發展了敏捷性和韌性的文化。
24:36 一次又一次的挫折， 我們擺脫了它並滑向下一個機會。
24:42 每次，我們都獲得了技能並增強了品格。 我們加強了我們的企業品格。
24:50 我們的公司很難分散注意力，也很難氣餒， 而且沒有被挫折擊退
24:56 如今，我們面臨的情況看起來不像是個機會。
25:02 諷刺的是，我們今天建造的機器人計算機 甚至不需要圖形，
25:08 這就是我們開始旅程的原因。 那麼我們今天的情況
25:15 告訴我們一些事情並教我們一些東西。 正如理查費曼所說，世界是不確定的，
25:21 世界可能會不公平，並會給你嚴厲的懲罰。 迅速地，抖掉它。
25:47 顯然，你們對你的書太過關注了。
25:53 迅速地，抖掉它。 拜託，這很聰明。 我逗得自己笑了。
26:04 還有其他機會，或創造一個機會。
26:09 讓我再給你講一個故事。 我曾經在我們的一個國際站點工作過
26:16 每年夏天一個月。 當我們的孩子十幾歲的時候，我們在日本度過了一個夏天。
26:22 週末我們參觀了京都和銀廟。 如果你們還沒機會去，那一定要去。
26:29 它以其精美的苔蘚花園而聞名。 我們去的那天是典型的京都夏日，
26:37 悶熱潮濕，黏糊糊的。 熱量從地面散發出來。
26:45 空氣依然濃重，一動也不動。 我們和其他遊客一起漫步在精心修剪的苔蘚花園。
26:54 我注意到了那位孤獨的園丁。 現在，記住，苔蘚花園，
27:00 這是銀廟， 苔蘚花園非常大。 大概跟這個草地一樣大。
27:05 它擁有最大的收藏 幾乎涵蓋世界上所有種類的苔蘚。
27:12 而且它保養得非常精緻。 我注意到孤獨的園丁蹲著，
27:20 用竹鑷子小心地摘取苔蘚
27:26 放進竹箩里。 你必須用竹鑷子，
27:37 你知道，就這一個園丁。
27:43 籃子看起來是空的。 嗯，有那麼一刻，我以為他在採摘想像中的苔蘚
27:52 變成一堆想像中的死苔。 於是我走向他
27:59 我說：你在幹嘛？ 他用英語說：“我正在撿死苔蘚。”
28:06 我正在照料我的花園。 我說，但是你的花園這麼大。
28:14 他回答說：“我已經照顧我的花園25年了。”
28:20 我有充足的時間。 嗯，這是我一生中最深刻的教訓之一。
28:30 它確實教會了我一些東西。 這位園丁致力於他的手藝並從事其一生的事業。
28:40 當你這樣做的時候，你就有充足的時間。 每天早晨，
28:46 我每天早上都會做同樣的事情， 我每天早晨開始 先完成我最優先的工作。
28:53 我有一個非常明確的優先事項清單 我先從優先順序最高的工作開始。 在我上班之前，我的一天就已經很成功了。
29:04 我已經完成了我最重要的工作 並能用我的每一天去幫助別人。
29:11 當人們因為打擾我而道歉時， 我總是說我有充足的時間，事實也確實如此。
29:20 2024年畢業生們， 我幾乎想像不出還有誰比你們對未來有更充分的準備。
29:31 你們獻出了自己， 你們努力工作， 你們獲得了世界一流的教育
29:37 來自世界上最負盛名的學校之一。 當你進入下一個階段時，
29:45 吸取我學的東西 希望它們能夠對你有幫助。
29:51 我希望你相信一些事情， 一些不尋常的事情， 一些尚未探索的東西，
29:57 但要讓它知情並讓它有理由。 然後全心全意地去實現它。
30:05 你可能會發現你的GPU， 你可能會發現你的CUDA， 你可能會發現你的生成人工智慧，
30:11 你可能會找到你的輝達。 我希望你們將挫折視為新的機會。
30:18 你的痛苦和磨難會增強你的性格、 你的韌性和敏捷性，
30:24 它們就是終極超級強國。 我最看重自己的能力的是
30:33 智力並不是排在首位的。 我忍受痛苦和折磨的能力，
30:39 我能夠長時間地專注於某件事，
30:44 我應付挫折和發現眼前機會的能力 我認為我的超能力是
30:51 我希望它們也是你的。 我希望你能找到一門手藝。
31:05 我希望你能找到一門手藝。 第一天的決定並不重要，
31:11 短期內做決定並不重要， 但我希望你能找到一門手藝，
31:16 你要用一生去完善， 磨練技能，並讓它成為你一生的事業。
31:27 最後，優先考慮好你的生活。 有很多事情正在發生，
31:32 有很多事情要做， 但要優先考慮你的生活 你將有充足的時間去做重要的事情。
31:39 恭喜2024 屆的畢業生，加油。
0:08 It really makes me cringe listening to all that. Thank you for that kind introduction, but I hate hearing about myself.
0:19 And just as a show, well, maybe if you could just applaud.
0:26 How many of you know who NVIDIA is?
0:33 And how many of you know what a GPU is?
0:41 Okay, good, I don’t have to change my speech.
0:46 Ladies and gentlemen, President Rosenbaum, esteemed faculty members,
0:52 distinguished guests, proud parents, and above all, the 2024 graduating class of Caltech.
1:08 This is a really happy day for you guys. You got to look more excited. You know you’re graduating from Caltech.
1:14 This is the school of the great Richard Feynman, Linus Pauling,
1:21 and someone who’s very influential to me and our industry, Carver Mead.
1:27 Yeah, this is a very big deal.
1:33 Today is a day of immense pride and joy. It is a dream come true for all of you,
1:39 but not just for you because your parents and families have made countless sacrifices
1:46 to see you reach this milestone. So let’s take this moment and congratulate them, thank them,
1:53 and let them know you love them.
2:04 You don’t want to forget that because you don’t know how long you’re going to be living at home.
2:14 You want to be super grateful today.
2:19 As a proud parent, I really loved it when my kids didn’t move out, and it was great to see them every day,
2:27 but now they’ve moved out, it makes me sad. So hopefully you guys get to spend some time with your parents. Your journey here is a testament of your character,
2:35 determination, willingness to make sacrifices for your dreams, and you should be proud.
2:41 The ability to make sacrifices, endure pain and suffering,
2:46 you will need these qualities in life. You and I share some things in common.
2:55 First, both chief scientists of NVIDIA were from Caltech.
3:07 And one of the reasons why I’m giving this speech today is because I’m recruiting.
3:16 And so I want to tell you that NVIDIA is a really great company, I’m a very nice boss,
3:24 universally loved, come work at NVIDIA.
3:30 You and I share a passion for science and engineering, and although we’re separated by about 40 years,
3:40 we are both at the peaks of our career.
3:47 For all of you who have been paying attention to NVIDIA and myself, you know what I mean.
3:52 It’s just that in your case, you’ll have many, many more peaks to go.
3:59 I just hope that today is not my peak. Not the peak.
4:05 And so I’m working as hard as ever to make sure that I have many, many more peaks ahead.
4:11 Last year, I was honored to give the commencement address at Taiwan University,
4:16 and I shared several stories about NVIDIA’s journey and the lessons that we learned that might be valuable for graduates.
4:23 I have to admit that I don’t love giving advice, especially to other people’s children.
4:34 And so my advice today will largely be disguised in some stories
4:43 that I’ve enjoyed, and some life experiences that I’ve enjoyed.
4:51 I’m the longest running tech CEO in the world today, I believe.
5:02 Over the course of 31 years, I managed not to go out of business, not get bored, and not get fired.
5:09 And so I have the great privilege of enjoying a lot of life’s experiences,
5:16 starting from creating NVIDIA, from nothing to what it is today.
5:21 And so I spoke about the long road of creating CUDA, a programming model. The programming model that we dedicated over 20 years to invent,
5:30 and that is revolutionizing computing today. I spoke about a very, quite public, canceled
5:39 Sega game console project we worked on, and where intellectual honesty,
5:44 something that I know Richard Feynman cares very deeply about and spoke quite often about,
5:50 where intellectual honesty and humility saved our company. And how a retreat, a strategic retreat, was one of our best strategies.
6:01 All of these are counterintuitive lessons that I spoke about at the commencement.
6:09 But I encouraged the graduates to engage with AI,
6:15 the most consequential technology of our time. And I’ll speak a little bit more about that later,
6:21 but all of you know about AI. It’s hard not to be immersed in it and surrounded by it,
6:26 and a great deal of discussion about it. And of course, I hope that all of you are using it and playing with it,
6:33 with surprising results and some magical, some disappointing, and some surprising.
6:41 But you have to enjoy it, you have to engage it, because it’s advancing so quickly. It is the only technology that I’ve known
6:48 that is advancing on multiple exponentials at the same time. And so the technology is changing very, very quickly.
6:55 So I advised the students at the Taiwan University to run, don’t walk,
7:03 and engage the AI revolution. And yet, one year later, it’s incredible how much it’s changed.
7:11 And so today, what I wanted to do is share with you my perspective,
7:17 from my vantage point, of some of the important things that are happening that you’re graduating into.
7:24 And these are extraordinary things that are happening that you should have an intuitive understanding for,
7:31 because it’s going to matter to you, it’s going to matter to the industry. And hopefully, you take advantage of the opportunity ahead of you.
7:40 The computer industry is transforming from its foundations, literally from studs.
7:46 Everything is changing from studs on up. And across every layer, and soon, every industry will also be transformed.
7:54 And the reason for that is quite obvious, because computers today are the single most important instrument of knowledge.
8:00 And it’s foundational to every single industry and every field of science. If we are transforming the computer so profoundly,
8:08 it will, of course, have implications in every industry. And I’ll talk about that in just a little bit.
8:14 And as you enter industry, it’s important you know what’s happening. Modern computing traces back to the IBM System 360.
8:24 That was the architecture manual that I learned from. It’s an architecture manual that you don’t need to learn from.
8:29 A lot better documentation and better descriptions of computers and architecture
8:35 has been presented since. But the System 360 was incredibly important at its time,
8:43 and in fact, the basic ideas of the System 360, the architecture of it,
8:49 the principal ideas and architecture and strategy of the System 360
8:57 are still governing the computer industry today. And it was introduced a year after my birth.
9:04 In the 80s, I was among the first generation of VLSI engineers
9:11 who learned to design chips from Mead and Conway’s landmark textbook.
9:17 And I’m not sure if it’s still being taught here. It should be in the introduction of VLSI systems.
9:22 Based on Carver Mead’s pioneering work here at Caltech on chip design methodologies and textbook that revolutionized IC design.
9:31 And it enabled our generation to design supergiant chips and ultimately the CPU.
9:38 The CPU led to exponential growth in computing. The performance, the incredible technology advances,
9:48 that’s called Moore’s Law, fueled the information technology revolution. The industrial revolution that we are part of,
9:56 that my generation was part of, saw the mass production of something the world had never seen before.
10:03 The mass production of something that was invisible, easy to copy, the mass production of software.
10:09 And it led to a $3 trillion industry. When I sat where you sat, the IT industry was minuscule.
10:19 And the concept that you could make money selling software was a fantasy.
10:25 And yet today, it’s one of the most important commodities, most important technologies
10:32 and product creations that our industry produces. However, the limits of the NARD scaling, of transistor scaling,
10:42 and instruction level parallelism have slowed CPU performance. And the slowed CPU performance gains is happening
10:50 at a time when computing demand continues to grow exponentially. This exponentially growing gap
10:59 between demand of computing and the capabilities of computers,
11:06 if not addressed, computing energy consumption and cost, inflation,
11:12 would eventually stifle every industry. We see very clear signs of computing inflation as we speak.
11:20 And after two decades of advancing NVIDIA’s CUDA, NVIDIA’s accelerated computing offers a path forward.
11:28 That’s the reason why I’m here. Because finally, the industry realized of the incredible effectiveness of accelerated computing at precisely the time
11:38 that we’re witnessing computing inflation after several decades.
11:45 By offloading time-consuming algorithms to a GPU that specializes in parallel processing,
11:52 we routinely achieve 10, 100, sometimes 1,000-fold speedups,
12:00 saving money, cost, and energy. We now accelerate application domains from computer graphics,
12:06 ray tracing, of course, to gene sequencing, scientific computing, astronomy, quantum circuit simulations,
12:14 SQL data processing, and even pandas, data science. Accelerated computing has reached a tipping point.
12:21 That is our first great contribution to the computer industry, our first great contribution to society, accelerated computing.
12:31 It now gives us a path forward for sustainable computing where cost will continue to decline
12:37 as computing requirement continues to grow. A hundred-fold, a hundred-fold of anything in time or cost or energy savings
12:49 that accelerated computing opened surely would trigger
12:55 a new development somewhere else. We just didn’t know what it was until deep learning came to our consciousness.
13:05 A whole new world of computing emerged. Jeff Hinton, Alex Krzyzewski, and Ilya Sutskever
13:11 used NVIDIA CUDA GPUs to train AlexNet and shocked the computer vision community
13:17 by winning the 2012 ImageNet Challenge. This was the big moment,
13:23 the big bang of deep learning, a pivotal moment that marked the beginning of the AI revolution.
13:32 Our decisions after AlexNet transformed our company
13:38 is something that’s worth taking note of. Our decisions after AlexNet transformed our company and likely everything else.
13:47 We saw the potential of deep learning and believed, just believed through principle thinking,
13:54 believed through our own analysis of the scalability of deep learning.
13:59 We believed the approach could learn other valuable functions. That maybe deep learning is a universal function learner
14:06 and how many problems are difficult or impossible to express using fundamental first principles.
14:13 And so when we saw this, when we saw this, we thought this is a technology we really have to pay attention to
14:21 because its limits are potentially only limited by model and data scale.
14:29 However, there were challenges at the time. This is 2012, shortly after 2012.
14:35 How could we explore the limits of deep learning without having to build these massive GPU clusters?
14:42 At the time we were a rather small company, building these massive GPU clusters could cost hundreds and hundreds of millions of dollars.
14:49 And if we didn’t though, there was no assurance that it would be effective if we scaled.
14:56 However, no one knew how far deep learning could scale. And if we didn’t build it, we’d never know.
15:04 This is one of those, if you build it, will they come? Our logic is if we don’t build it, they can’t come.
15:10 And so we dedicated ourselves based on our first principled beliefs and our analysis.
15:16 And we got ourselves to the point where we believed this was going to be so effective and when the company believes something, we should go act on it.
15:24 So we dove deep into deep learning, and over the next decade, systematically reinvented everything.
15:31 We reinvented every computing layer, starting with the GPU itself. The invention of the modern GPU,
15:38 which is very different than the GPU of the past that we invented in the first place,
15:44 and we went on to invent just about every other aspect of computing, the interconnects, the systems,
15:50 the networking, and of course, software. We invested billions.
15:55 We invested billions into the unknown. Thousands of engineers for a decade
16:01 worked on deep learning and advancing and scaling deep learning without really knowing how far we could really take the technology.
16:11 We invested billions. And we designed and built supercomputers to explore the limits of deep learning and AI.
16:18 And in 2016, we announced DGX-1,
16:23 our first AI supercomputer, and I delivered the first one to a startup in San Francisco,
16:31 a startup nobody knew anything about, a group of friends of mine who were working on artificial intelligence,
16:40 a company called OpenAI. In 2022, 10 years after AlexNet,
16:49 and about a million-fold increase in computing later, a million-fold.
16:54 If you could just imagine, what would it be like if your laptop was a million-fold more capable?
17:01 A million-fold later, OpenAI launched ChatGPT, and AI went mainstream.
17:08 During this decade, NVIDIA transformed ourselves from a graphics company that many of you probably first knew us as, that builds GPUs,
17:17 to now an AI company that builds massive data center scale supercomputers.
17:24 We transformed our company completely. We also transformed computing completely.
17:30 The fundamental way of doing computing today has been radically changed.
17:35 The computing stack now uses GPU to process large language models
17:41 that are trained on supercomputers rather than CPUs
17:47 that are processing instructions written by programmers.
17:52 We are now creating software that no humans can write. We are now creating software that does things that no humans can
17:59 imagine, even just 10 years ago. Computers are now intention-driven rather than instruction-driven.
18:08 Tell a computer what you want, and it will figure out how. And like humans, AI applications will understand the mission,
18:20 reason, plan, and orchestrate a team of large language models to perform tasks.
18:28 Future applications will do and perform very similar to the way we do things,
18:33 assemble teams of experts, use tools, reason and plan, and execute our mission.
18:41 Software and what software can do has been completely changed. Even our industry, as it’s being changed and transformed,
18:51 created yet another industry, an industry the world’s never seen before.
18:56 An industry is forming right in front of our eyes. AI’s input and output are tokens.
19:06 For all the engineers in the room, you know what I mean. These are floating-point numbers that embed intelligence.
19:14 Companies are now building a new type of data center that didn’t exist before that specialize
19:20 in producing intelligence tokens. Essentially, AI factories.
19:29 Like AC generators that Nikola Tesla invented
19:36 of the past industrial revolution, we now have AI token generators,
19:43 and they will be the factories of a new industrial revolution. There’s large industries producing energy, electricity.
19:51 We now have a large industry producing something invisible called software. In the future, in the very near future,
19:58 we’ll have industries that are producing, manufacturing intelligence tokens, AI generators.
20:07 A new computing model has emerged, and a new industry has emerged,
20:13 all because we reasoned from first principles, formed our belief about the future, and we acted on them.
20:22 The next wave of AI is robotics, where AI, in addition to a language model,
20:30 also has a physical world model. We work with hundreds of companies building robots,
20:38 robotic vehicles, pick-and-place arms, humanoid robots, and even entire gigantic warehouses that are robotic.
20:46 But unlike our AI factory strategy and our experience there, which was really formed through reasoning and deliberate action,
20:55 our robotics journey resulted from a series of setbacks.
21:01 As you know, NVIDIA invented the GPU. This was before we invented AI factories.
21:07 Our first great contribution to the computer industry was reinventing computer graphics through programmable shaders.
21:15 We invented the GPU and programmable shading in 2000. We wanted to integrate GPUs into every computer,
21:22 and so we started to combine our GPUs with motherboard chips, and we launched a fabulous integrated graphics chip
21:30 at the time for AMD CPUs. Our chipset business was an instant success.
21:37 I think it went from zero to a billion dollars practically overnight. But then all of a sudden, AMD wanted to
21:44 control all of the technology in the PC, and we wanted to stay independent,
21:50 so they purchased ATI and no longer needed us. We turned to Intel.
22:01 That probably wasn’t a great idea, but we turned to Intel
22:06 and negotiated a license to connect to Intel CPUs.
22:11 Apple was excited by what we were building
22:16 and asked us to work on a new computer with them, which became the first MacBook Air.
22:23 Well, Intel saw what happened and decided they didn’t want us to do that anymore,
22:29 and so they terminated our agreement. Well, we pivoted again, and this time we went and licensed ARM,
22:39 and we built a low-power SoC, a mobile SoC, the world’s first SoC that was essentially a computer,
22:48 a full operating computer, and it was incredible. Our chip excited Google,
22:55 and they asked us to work on a new device, which turned out to have been the Android mobile device.
23:02 Well, Qualcomm decided they didn’t want us to do that, and so they didn’t want us to connect to their modems,
23:09 and it’s hard to build a mobile device without being connected to a modem. And there were no other LTE modem companies,
23:17 so we had to exit the mobile device market. Well, this happened practically on a year rhythm,
23:26 and we would build something, it would be incredibly successful, generate enormous amounts of excitement,
23:32 and then one year later, we were kicked out of those markets. Well, with no more markets to turn to,
23:40 we decided to build something where we are sure there are no customers,
23:50 because one of the things you can definitely guarantee
23:55 is where there are no customers, there are also no competitors, and nobody cares about you.
24:02 And so we chose a market with no customers, a $0 billion market,
24:08 and it was robotics. We built the world’s first robotics computer
24:14 processing an algorithm nobody understood at the time called deep learning.
24:19 This is over 10 years ago now. Ten years later, I can’t be happier
24:25 with what we’ve built and the opportunity to create the next wave of AI. More importantly,
24:31 we developed agility and a culture of resilience.
24:36 One setback after another, we shook it off and skated to the next opportunity.
24:42 Each time, we gained skills and strengthened our character. We strengthened our corporate character.
24:50 Our company is really hard to distract and really hard to discourage, and no setback
24:56 that comes our way doesn’t look like an opportunity these days.
25:02 Ironically, the robotics computer that we built today doesn’t even need graphics,
25:08 which is why our journey started in the first place. So where we are today
25:15 tells us something and teaches us something. The world is uncertain, as Richard Feynman would say,
25:21 and the world can be unfair and deal you with tough cards. Swiftly, shake it off.
25:47 You’ve apparently been paying too much attention to your books.
25:53 Swiftly, shake it off. Come on, that’s pretty clever. I made myself laugh.
26:04 There’s another opportunity out there, or create one.
26:09 Let me tell you one more story. I used to work from one of our international sites
26:16 for one month each summer. When our kids were in their teens, we spent a summer in Japan.
26:22 For a weekend, we visited Kyoto and the Silver Temple. If you haven’t had a chance to go, you must.
26:29 It’s renowned for its exquisite moss garden. The day we visited was quintessential Kyoto summer day,
26:37 suffocatingly hot and humid, sticky. Heat is radiating from the ground.
26:45 The air was thick, still. Along with the other tourists, we wandered through the meticulously groomed moss garden.
26:54 And I noticed the lone gardener. Now, remember, the moss garden,
27:00 this is the Silver Temple, the moss garden is gigantic. It’s about the size of this courtyard.
27:05 And it has the collection, the largest collection of just about apparently every species of moss in the world.
27:12 And it’s just exquisitely maintained. I noticed the lone gardener squatting,
27:20 carefully picking at the moss with a bamboo tweezer
27:26 and putting it in the bamboo basket. And you have to, it’s a bamboo tweezer,
27:37 you know, and it’s just this one gardener.
27:43 And the basket looked empty. Well, for a moment there, I thought he was picking imaginary moss
27:52 into a pile of imaginary dead moss. And so I walked up to him
27:59 and I said, what are you doing? And in his English, he said, I’m picking dead moss.
28:06 I’m taking care of my garden. And I said, but your garden is so big.
28:14 And he responded, I have cared for my garden for 25 years.
28:20 I have plenty of time. Well, that was one of the most profound learnings in my life.
28:30 And it really taught me something. This gardener has dedicated himself to his craft and doing his life’s work.
28:40 And when you do that, you have plenty of time. I begin each morning,
28:46 I do every single morning exactly the same way, I begin each morning by doing my highest priority work first.
28:53 I have a very clear priority list and I start from the highest priority work first. Before I even get to work, my day is already a success.
29:04 I’ve already completed my most important work and can dedicate my day to helping others.
29:11 And when people apologize for interrupting me, I always say I have plenty of time and I do.
29:20 Graduates of the class of 2024, I can hardly imagine anyone more prepared for the future than you.
29:31 You dedicated yourself, you worked hard, you earned a world-class education
29:37 from one of the most prestigious schools in the world. And as you commence into the next stage,
29:45 take my learnings and hopefully they’ll help you along the way.
29:51 I hope you believe in something, something unconventional, something unexplored,
29:57 but let it be informed and let it be reasoned. Then dedicate yourself to making it happen.
30:05 You may find your GPU, you may find your CUDA, you may find your generative AI,
30:11 you may find your NVIDIA. I hope you will see setbacks as new opportunities.
30:18 Your pain and suffering will strengthen your character, your resilience and agility,
30:24 and they are the ultimate superpowers. Of all of the things that I value most about my abilities,
30:33 intelligence is not top of that list. My ability to endure pain and suffering,
30:39 my ability to work on something for a very, very long period of time,
30:44 my ability to handle setbacks and see the opportunity just around the corner I consider to be my superpowers,
30:51 and I hope they’re yours. And I hope you find a craft.
31:05 I hope you find a craft. It’s not important to decide on day one,
31:11 it’s not even important to decide any time soon, but I hope you do find a craft,
31:16 that you want to dedicate your lifetime to perfecting, to hone the skills of, and let it be your life’s work.
31:27 And then lastly, prioritize your life. There’s so many things going on,
31:32 there’s so many things to do, but prioritize your life and you will have plenty of time to do the important things.
31:39 Congratulations, class of 2024, go get them.