Shivani Virdi

I wasted 100+ hours trying to understand AI from academic papers.
It's a terrible way to start.

Dense math. Vague diagrams. "Attention Is All You Need" is brilliant, but it's not a starting point.

Then I found this Stanford CS229 guest lecture on Building LLMs.

Here’s why it’s different:
→ Explains the 8-step data filtering pipeline (it's not just "train on the internet").

→ Shows why post-training (SFT, DPO) is what turns a "model" into a useful "assistant".

→ Breaks down Scaling Laws in simple terms (what actually makes models better).

→ Actually covers how to evaluate models (like Chatbot Arena) when there's no single "right" answer.

The kicker?

This 1-hour lecture gives more practical intuition than a week of trying to decode dense academic papers.

Topics that actually matter for builders:
• Pre-training vs. Post-training (what they are and why you need both)
• Tokenization (the part everyone skips but shouldn't)
• Scaling Laws (how compute, data, and parameters relate)
• The Ops (SFT, DPO, and Evals)

The uncomfortable truth:
While everyone's reading paper summaries,
this lecture's wisdom on data pipelines and evals
is what actually builds real intuition.

Full lecture here: https://lnkd.in/gUzsiN_e

♻️ Repost to help someone escape "paper summary" hell.

If I had to rebuild every AI system I’ve ever worked on,
I’d start with one thing

Data.

Everything else can change: models, frameworks, even the architecture.

But if your data isn’t right, the whole system collapses under its own weight.

𝗗𝗮𝘁𝗮 𝗶𝘀𝗻’𝘁 𝗷𝘂𝘀𝘁 𝗶𝗺𝗽𝗼𝗿𝘁𝗮𝗻𝘁.
𝗜𝘁’𝘀 𝘁𝗵𝗲 𝗲𝗻𝗴𝗶𝗻𝗲 𝗼𝗳 𝗲𝘃𝗲𝗿𝘆 𝗔𝗜 𝗮𝗽𝗽.

Here’s why 👇

𝗙𝗼𝘂𝗻𝗱𝗮𝘁𝗶𝗼𝗻 𝗼𝗳 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲
↳ The diversity and quality of your dataset define the model’s reasoning ceiling.
↳ Garbage in, hallucination out.

𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗶𝘀 𝗞𝗶𝗻𝗴 (𝗳𝗼𝗿 𝗥𝗔𝗚)
↳ Retrieval-based systems live and die by the precision of what they fetch.
↳ More data ≠ better data, relevant context wins.

𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴 𝗳𝗼𝗿 𝗥𝗲𝗹𝗲𝘃𝗮𝗻𝗰𝗲
↳ Domain datasets turn a general model into an industry expert.
↳ Without them, “specialization” is just prompt dressing.

𝗕𝗶𝗮𝘀 & 𝗙𝗮𝗶𝗿𝗻𝗲𝘀𝘀
↳ Every dataset carries human fingerprints.
↳ Cleaning, balancing, and annotating correctly isn’t optional; it’s ethical engineering.

Once you understand that, the next question is obvious:

𝘏𝘰𝘸 𝘥𝘰 𝘺𝘰𝘶 𝘱𝘳𝘦𝘱, 𝘤𝘭𝘦𝘢𝘯, 𝘢𝘯𝘥 𝘧𝘦𝘦𝘥 𝘥𝘢𝘵𝘢 𝘸𝘪𝘵𝘩𝘰𝘶𝘵 𝘭𝘰𝘴𝘪𝘯𝘨 𝘺𝘰𝘶𝘳 𝘮𝘪𝘯𝘥?

Here are the tools I wish I’d known earlier 👇

1. 𝗙𝗶𝗿𝗲𝗰𝗿𝗮𝘄𝗹 → Crawl & convert entire sites into clean Markdown.
   
2. 𝗧𝗮𝘃𝗶𝗹𝘆 → Web-search built for LLMs; retrieves, synthesizes, and formats for RAG.
   
3. 𝗠𝗮𝗿𝗸𝗜𝘁𝗗𝗼𝘄𝗻 (𝗠𝗶𝗰𝗿𝗼𝘀𝗼𝗳𝘁) → Converts PDFs, DOCX, videos → knowledge-base-ready text.
   
4. 𝗚𝗶𝘁𝗜𝗻𝗴𝗲𝘀𝘁 → Turns Git repos into contextual knowledge graphs (code + docs + commits).
   
5. 𝗟𝗮𝗻𝗴𝗘𝘅𝘁𝗿𝗮𝗰𝘁 (𝗚𝗼𝗼𝗴𝗹𝗲) → Extracts entities + metadata from raw text, with full traceability.
   
6. 𝗗𝗼𝗰𝗹𝗶𝗻𝗴 → Layout-aware parser that keeps structure intact for PDFs and tables.

If you’re building anything beyond a toy demo, start here.

Because the best AI systems don't just need 𝘮𝘰𝘳𝘦 data, they need 𝘣𝘦𝘵𝘵𝘦𝘳 data.

Save it.
Share it.
Build smarter.

♻️ Reposting this helps every AI engineer fix their foundation.

Stop comparing RAG and CAG. I wish I knew how each contributes to context before spending hours trying to get one do the job of other.

Most teams are still trying to squeeze costs out of their RAG pipeline.

But the smartest teams aren't just optimising,
they're re-architecting their context.

They know it’s not about RAG vs. CAG.
It’s about knowing how to leverage each, intelligently.

It's about Context Engineering.

𝗧𝗵𝗲 "𝗣𝗮𝘆-𝗣𝗲𝗿-𝗤𝘂𝗲𝗿𝘆" 𝗣𝗿𝗼𝗯𝗹𝗲𝗺:
Retrieval-Augmented Generation (RAG)
RAG is powerful, giving LLMs access to dynamic data.

But from a cost perspective, it’s a “pay-per-drink” model.

Every single query has a cost attached:
• 𝗖𝗼𝗺𝗽𝘂𝘁𝗲 𝗖𝗼𝘀𝘁: API calls to an embedding model.
• 𝗜𝗻𝗳𝗿𝗮𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲 𝗖𝗼𝘀𝘁: Hosting a vector database and a retriever.
• 𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗖𝗼𝘀𝘁: Latency and irrelevant results degrade user experience, which costs you users.
  
Optimising RAG helps, but you're still paying for every single lookup.

𝗧𝗵𝗲 "𝗣𝗮𝘆-𝗢𝗻𝗰𝗲, 𝗨𝘀𝗲-𝗠𝗮𝗻𝘆" 𝗦𝗼𝗹𝘂𝘁𝗶𝗼𝗻:
Cache-Augmented Generation (CAG)
CAG flips the cost model on its head.

It’s built for efficiency with scoped knowledge.

Instead of fetching data every time, you:
→ Preload a static knowledge base into the model's context.
→ Compute and store its KV cache just once.
→ Reuse this cache across thousands of subsequent queries.

The result is a massive drop in per-query costs.
• 𝗕𝗹𝗮𝘇𝗶𝗻𝗴 𝗳𝗮𝘀𝘁: No real-time retrieval latency.
• 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗮𝗹𝗹𝘆 𝘀𝗶𝗺𝗽𝗹𝗲: Fewer moving parts to manage and pay for.
• 𝗜𝗻𝗳𝗿𝗮-𝗹𝗶𝗴𝗵𝘁: The most expensive work (caching) is done upfront, not on every call.

It’s Not RAG vs. CAG. It’s RAG + CAG.

The most cost-effective AI systems don't choose one.
They use a hybrid approach, like the teams at 𝗠𝗮𝗻𝘂𝘀 𝗔𝗜.

The goal is to match the data's nature to the right architecture.

This is 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴: strategically deciding what knowledge is cached and what is retrieved.

✅ Use CAG for your static foundation:
This is for knowledge that doesn't change often but is frequently accessed. Pay the upfront cost to cache it once and enjoy near-zero marginal cost for every query after.

✅ Use RAG for your dynamic layer:
This is for information that is volatile, real-time, or user-specific. You only pay the retrieval cost when you absolutely need the freshest data.

The Bottom Line
Stop thinking in terms of "RAG vs. CAG."
Start thinking like a 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿.

By building a static foundation with CAG and using RAG for dynamic lookups, you create a system that is not only powerful and fast but also dramatically more cost-effective at scale.

RAG isn't dead, and CAG isn't a silver bullet. They are two essential tools in your cost-optimisation toolkit.

If you're building an AI stack that's both smart and sustainable, this is for you.

♻️ Repost to share this strategy.
➕ Follow Shivani Virdi for more.