Tensor parallelism
Split every weight matrix across GPUs: column and row splits with real numbers, a TP transformer block, and what it costs in communication.
Projects and perspectives from AI Next Labs
What we build. What we learn. What’s next.
An idea, taken apart
Start with one document. There’s more than one idea inside.
From inside our team
Projects, experiments, and ideas worth sharing.
This is a window into the work at AI Next Labs. We share the things we build, the ideas we’re exploring, and what we learn along the way.
Read our perspectives01Explore learning02Explore blog03Explore showcase04Browse resources
Learning
Explore concepts our team works with, explained through interactive guides.
Blog
Our perspectives on AI, the topics we’re discussing, and lessons from our work.
Showcase
Projects from the team, with a closer look at how they were made.
Resources
Take an idea further. Guides and resources from the things we share.
Inside the labFeatured Learning
Learning
How RAG finds an answer
Turn a document into searchable chunks. Find the right facts. Give them to the AI to help it answer.
Latest from the team
Projects, learning, and perspectivesThe Coyote That Never Fell
In 2016, the godfather of AI told the world to stop training radiologists. A decade later the opposite had happened, and the shape of his mistake is the most useful map we have for what to become in the age of AI.
Turn study notes into a PDF you can check
Keep an editable copy, save a PDF from Google Docs, and check page breaks, tables and equations before sharing.
Data parallelism and ZeRO
Copy the model, split the batch: ring all-reduce, overlapping communication, the batch-size recipe and ZeRO stages 1 to 3.
Foundations: where memory and compute go in LLM training
Transformer shapes, the 16 bytes per parameter, activation memory, FLOPs, gradient accumulation and recomputation, with step-through diagrams.
Control what ChatGPT remembers about you
In about 5 minutes, review remembered details, switch memory off, remove one saved memory, and start a Temporary Chat.