Reading List
Papers, books, videos, and engineering blogs that have shaped how I think about systems.
🎥
Videoshelf
Iconic lectures, tutorials, and talks that every tech nerd should watch.
Must-Watch Series
Neural Networks: Zero to Hero
Andrej Karpathy
Neural Networks Visualized
3Blue1Brown
Stanford CS231n
Fei-Fei Li & Karpathy
Practical Deep Learning
fast.ai / Jeremy Howard
Legendary Talks & Lectures
Let's Build GPT from Scratch
Andrej Karpathy
Attention Is All You Need
Yannic Kilcher
Turing Award Lecture: Deep Learning
Yann LeCun, Yoshua Bengio, Geoffrey Hinton
Geoffrey Hinton on AI Risks
The Godfather of AI
Demis Hassabis: DeepMind
Lex Fridman Podcast
Sam Altman: OpenAI CEO
Lex Fridman Podcast
AlphaGo & AlphaZero
David Silver / DeepMind
John Carmack: Doom, Quake, VR, AGI
Lex Fridman Podcast
The Character of Physical Law
Richard Feynman
Jim Keller: Moore's Law & Future of Computing
Lex Fridman Podcast
The Computer Revolution Hasn't Happened Yet
Alan Kay / OOPSLA 1997
Dario Amodei: Anthropic CEO on AI Safety
Lex Fridman Podcast
George Hotz: Hacking, AI, Self-Driving
Lex Fridman Podcast
How We Teach Computers to Understand Pictures
Fei-Fei Li / TED
Historic Moments in Tech
The Mother of All Demos
Doug Engelbart, 1968
Steve Jobs Introduces Macintosh
Apple, 1984
Windows 95 Launch
Bill Gates & Jay Leno, 1995
Steve Jobs Introduces iPhone
Apple, 2007
First Android Demo
Google, 2007
AlphaGo - The Movie
DeepMind, 2016
First Falcon 9 Landing
SpaceX, 2015
GPT-4 Developer Livestream
OpenAI, 2023
Google Duplex AI Assistant
Google I/O, 2018
📚
Bookshelf
Technical
Fundamentals & Algorithms
📖
Structure and Interpretation of Computer Programs
Abelson & Sussman
📖
The Art of Computer Programming
Donald Knuth
📖
Introduction to Algorithms (CLRS)
Cormen, Leiserson, Rivest, Stein
📖
The Algorithm Design Manual
Steven Skiena
📖
Programming Pearls
Jon Bentley
📖
Compilers: Principles, Techniques, and Tools
Aho, Lam, Sethi, Ullman
Systems & Infrastructure
📖
Computer Systems: A Programmer's Perspective
Bryant & O'Hallaron
📖
Operating Systems: Three Easy Pieces
Arpaci-Dusseau
📖
Database Internals
Alex Petrov
📖
Database Systems
Hector Garcia-Molina
📖
Designing Data-Intensive Applications
Martin Kleppmann
📖
Site Reliability Engineering
Google SRE Team
Distributed Systems
📖
Distributed Systems
Maarten van Steen
📖
Understanding Distributed Systems
Roberto Vitillo
📖
Building Microservices
Sam Newman
📖
Release It!
Michael Nygard
Architecture & Design
📖
The Pragmatic Programmer
Andrew Hunt & David Thomas
📖
Clean Code
Robert C. Martin
📖
Refactoring
Martin Fowler
📖
Domain-Driven Design
Eric Evans
📖
Patterns of Enterprise Application Architecture
Martin Fowler
📖
System Design Interview
Alex Xu
Programming Languages
📖
The Go Programming Language
Alan A. A. Donovan
📖
Concurrency in Go
Katherine Cox-Buday
📖
Java Concurrency in Practice
Brian Goetz
📖
Introduction To Information Retrieval
Christopher Manning
Engineering Career
📖
The Mythical Man-Month
Fred Brooks
📖
Staff Engineer
Will Larson
📖
An Elegant Puzzle
Will Larson
Nerd Classics
Computing History
📖
The Soul of a New Machine
Tracy Kidder
📖
The Dream Machine
M. Mitchell Waldrop
📖
Where Wizards Stay Up Late
Katie Hafner
📖
Masters of Doom
David Kushner
📖
The Cuckoo's Egg
Clifford Stoll
Hacker Culture & Essays
📖
Hackers & Painters
Paul Graham
📖
Coders at Work
Peter Seibel
📖
The Phoenix Project
Gene Kim
📖
The Code Book: The Science of Secrecy
Simon Singh
Science & Mathematics
📖
Gödel, Escher, Bach: An Eternal Golden Braid
Douglas Hofstadter
📖
Surely You're Joking, Mr. Feynman!
Richard Feynman
📖
The Elegant Universe
Brian Greene
📖
Chaos: Making a New Science
James Gleick
📖
The Information: A History, A Theory, A Flood
James Gleick
📖
What If? Serious Scientific Answers to Absurd Questions
Randall Munroe
AI, ML & The Future
Practical AI/ML Engineering
📖
AI Engineering
Chip Huyen
📖
Designing Machine Learning Systems
Chip Huyen
📖
Build a Large Language Model (From Scratch)
Sebastian Raschka
📖
Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow
Aurélien Géron
📖
Natural Language Processing with Transformers
Tunstall, von Werra, Wolf
📖
Machine Learning Design Patterns
Lakshmanan, Robinson, Munn
📖
The Hundred-Page Machine Learning Book
Andriy Burkov
Theory & Foundations
📖
Deep Learning
Goodfellow, Bengio, Courville
📖
Pattern Recognition and Machine Learning
Christopher Bishop
📖
Reinforcement Learning: An Introduction
Sutton & Barto
📖
The Book of Why
Judea Pearl
📖
A Thousand Brains
Jeff Hawkins
AI Safety, Ethics & Philosophy
📖
Co-Intelligence: Living and Working with AI
Ethan Mollick
📖
The Alignment Problem
Brian Christian
📖
Human Compatible
Stuart Russell
📖
Superintelligence
Nick Bostrom
📖
Life 3.0
Max Tegmark
Business & Leadership
Startups & Entrepreneurship
📖
Zero to One
Peter Thiel
📖
The Lean Startup
Eric Ries
📖
Blitzscaling
Reid Hoffman
📖
The Innovator's Dilemma
Clayton Christensen
Management & Leadership
📖
High Output Management
Andy Grove
📖
The Hard Thing About Hard Things
Ben Horowitz
📖
Good to Great
Jim Collins
📖
Measure What Matters
John Doerr
📖
The Diary of a CEO
Steven Bartlett
Self-Improvement & Philosophy
Productivity & Habits
📖
Atomic Habits
James Clear
📖
Deep Work
Cal Newport
📖
Think Again
Adam Grant
Philosophy & Psychology
📖
Meditations
Marcus Aurelius
📖
Man's Search for Meaning
Viktor Frankl
📖
The Courage to Be Disliked
Ichiro Kishimi
📖
Thinking, Fast and Slow
Daniel Kahneman
📖
The Psychology of Money
Morgan Housel
Sci-Fi & Fiction
Classic AI & Consciousness
📖
I, Robot
Isaac Asimov
📖
2001: A Space Odyssey
Arthur C. Clarke
📖
Do Androids Dream of Electric Sheep?
Philip K. Dick
📖
Neuromancer
William Gibson
Simulation & Digital Minds
📖
Permutation City
Greg Egan
📖
Diaspora
Greg Egan
📖
Blindsight
Peter Watts
📖
Sum: Forty Tales from the Afterlives
David Eagleman
Singularity & Superintelligence
📖
A Fire Upon the Deep
Vernor Vinge
📖
Rainbows End
Vernor Vinge
📖
Accelerando
Charles Stross
AI Ethics & Development
📖
The Lifecycle of Software Objects
Ted Chiang
📖
Exhalation
Ted Chiang
📖
Stories of Your Life and Others
Ted Chiang
Post-Scarcity & AI Societies
📖
The Culture Series (Player of Games)
Iain M. Banks
📖
The Diamond Age
Neal Stephenson
📖
The Moon is a Harsh Mistress
Robert A. Heinlein
📖
Rendezvous with Rama
Arthur C. Clarke
Cyberpunk & Near-Future
📖
Snow Crash
Neal Stephenson
📖
Cryptonomicon
Neal Stephenson
📖
The Three-Body Problem
Cixin Liu
📖
Project Hail Mary
Andy Weir
Dystopian & Philosophical
📖
1984
George Orwell
📖
Brave New World
Aldous Huxley
📖
Contact
Carl Sagan
Epic Space Opera
📖
Dune
Frank Herbert
📖
Foundation
Isaac Asimov
📖
Ender's Game
Orson Scott Card
📖
The Hitchhiker's Guide to the Galaxy
Douglas Adams
📜
Papershelf
Research papers that shaped modern computing. Links to freely available versions where possible.
Machine Learning & AI
●
Attention Is All You Need
●
Language Models are Few-Shot Learners (GPT-3)
●
Improving Language Understanding by Generative Pre-Training (GPT)
●
BERT: Pre-training of Deep Bidirectional Transformers
●
PaLM: Scaling Language Modeling with Pathways
●
LLaMA: Open and Efficient Foundation Language Models
●
Deep Residual Learning for Image Recognition (ResNet)
●
ImageNet Classification with Deep CNNs (AlexNet)
●
Generative Adversarial Networks
●
Neural Machine Translation by Jointly Learning to Align and Translate
●
Word2Vec: Distributed Representations of Words
●
Retrieval-Augmented Generation (RAG)
●
Chain-of-Thought Prompting
●
Constitutional AI
●
Faster sorting algorithms discovered using deep RL (AlphaDev)
Google Infrastructure (The Classics)
●
The Google File System
●
MapReduce: Simplified Data Processing on Large Clusters
●
Bigtable: A Distributed Storage System for Structured Data
●
Spanner: Google's Globally-Distributed Database
●
F1: A Distributed SQL Database That Scales
●
The Chubby Lock Service
●
Borg: Large-scale Cluster Management at Google
●
Omega: Flexible, Scalable Schedulers for Large Compute Clusters
●
Dremel: Interactive Analysis of Web-Scale Datasets
●
Megastore: Scalable Highly Available Storage
●
Mesa: Geo-Replicated Data Warehousing
●
Pregel: Large-Scale Graph Processing
●
FlumeJava: Easy, Efficient Data-Parallel Pipelines
●
MillWheel: Fault-Tolerant Stream Processing
●
Dapper: Large-Scale Distributed Systems Tracing
●
Zanzibar: Google's Consistent, Global Authorization System
●
Monarch: Google's Planet-Scale Monitoring Infrastructure
●
Slicer: Auto-Sharding for Datacenter Applications
●
B4: Experience with a Globally-Deployed Software Defined WAN
●
Jupiter Rising: A Decade of Clos Topologies in Google's Datacenter
●
Maglev: A Fast and Reliable Software Network Load Balancer
●
TensorFlow: A System for Large-Scale Machine Learning
●
Pathways: Asynchronous Distributed Dataflow for ML
Distributed Systems & Databases
●
Dynamo: Amazon's Highly Available Key-value Store
●
Amazon Aurora: Design Considerations
●
Amazon DynamoDB: A Scalable NoSQL Database
●
Firecracker: Lightweight Virtualization
●
A Relational Model of Data for Large Shared Data Banks
●
Raft Consensus Algorithm
●
Paxos Made Simple
●
Time, Clocks, and Ordering of Events
●
Kafka: A Distributed Messaging System
●
The Log: What every software engineer should know
●
CockroachDB: The Resilient Geo-Distributed SQL Database
●
FoundationDB: A Distributed Unbundled Transactional Key Value Store
Data Structures & Algorithms
●
The LSM-Tree (Log-Structured Merge-Tree)
●
Space/Time Trade-offs in Hash Coding (Bloom Filters)
●
HyperLogLog: Cardinality Estimation
●
Skip Lists: A Probabilistic Alternative to Balanced Trees
●
The Bw-Tree: A B-tree for New Hardware
●
Consistent Hashing and Random Trees
●
Cuckoo Filter: Better Than Bloom
●
B-Trees and Database Systems
●
Gorilla: Fast Time Series Database
●
SIEVE: Efficient Cache Eviction
Classics & Foundational
●
Go To Statement Considered Harmful
●
No Silver Bullet - Essence and Accident in Software
●
Out of the Tar Pit
●
The UNIX Time-Sharing System
●
Bitcoin: A Peer-to-Peer Electronic Cash System
●
CAP Theorem: Twelve Years Later
●
How Complex Systems Fail
●
End-to-End Arguments in System Design
●
A Note on Distributed Computing
●
Reflections on Trusting Trust
🌐