Mar 17, 2026 · 58m · startup-ideas
Building AI Agents that actually work (Full Course)
0:00 / 0:00
▶ Watch on YouTube →
gold bands on the timeline = statements, start to end. Hover to read, click to jump. CC turns on captions
In this masterclass, Greg Isenberg and Remy Gaskill demystify autonomous AI agents, demonstrating how to build an AI-driven operating system using local markdown files, Model Context Protocol (MCP) integrations, and reusable Standard Operating Procedure skills.
How this conversation actually went
Every chapter scored 0–10 on four independent dynamics. Hover any point for the reasoning behind the score. Greg holds 12.3% of the talking time here. How this is scored →
Greg as informed peer 3.0 Guest teaching 5.3 Guest disagreement 0.0 Greg pushing back 0.5
speaking balance: gold is Greg, purple is the guest (3 minute bins)
Sharpest disagreement ▶ 17:32 Remy Reframing Agent Memory as Intentional Isolation
In a very collaborative tutorial episode, Remy delivers his clearest contrarian reframe by asserting that agents lacking automatic cross-session cloud memory is a crucial feature rather than a limitation.
Hardest push from Greg ▶ 36:07 Greg Challenges Value of Inbox SummarizationGreg directly challenges the practical business utility of basic inbox summaries, prompting Remy to pivot into demonstrating complex multi-step workflow automation.
Biggest teaching moment ▶ 1:35 Remy Defines Question-to-Answer vs Goal-to-ResultRemy lays out the foundational shift from chat models to agents, clearly educating the audience and host on why planning and multi-step execution separate true agents from standard LLM chats.
Greg holds their own ▶ 52:00 Greg Details Live Scheduled Car Scraping AgentGreg demonstrates his own hands-on automation competence by detailing a multi-site scraping cron job he built to monitor specific vehicle inventory listings.
the scores for every segment, with the reasoning behind each
| Chapter | Topic | Greg as informed peer | Guest teaching | Guest disagreement | Greg pushing back | Why |
|---|---|---|---|---|---|---|
| Structuring an AI-Driven Company | 3 | 5 | 0 | 0 | Greg sets the beginner-friendly premise of the episode and contributes the ping-pong analogy for chat models. Remy establishes the core framework differentiating chat (question to answer) from agents (goal to result). | |
| The Agent Loop and Core Components | 0 | 6 | 0 | 0 | Remy delivers an uninterrupted technical walkthrough of the observe-think-act agent loop and its four underlying components. Greg listens without interjecting. | |
| Live Multi-Harness Setup: Building a Portfolio Site | 1 | 4 | 0 | 0 | Remy executes live parallel prompts across Claude Code, Codex, and Antigravity. Greg interjects briefly with a quick clarifying check about the desktop application interface. | |
| Security Considerations and the Driving Analogy | 4 | 5 | 0 | 1 | Greg raises security questions and inspects output errors in the generated portfolio site, proposing a cold email agency loop. Remy explains permissions scoping and uses a car driving analogy. | |
| Onboarding an Executive Assistant Agent | 2 | 6 | 0 | 0 | Greg identifies the voice transcription tool Monologue by Every. Remy explains why agent memory intentionally differs from default cloud chat memory. | |
| Context Engineering with Agents.md Files | 3 | 6 | 0 | 0 | Remy demonstrates setting up Agents.md context files and explains the industry shift toward context engineering. Greg crystallizes the concept as a persistent reminder file. | |
| Building Self-Improving Agents with Memory.md | 4 | 5 | 0 | 1 | Greg actively probes on practical failure points, asking whether memory.md files become bloated and counterproductive over time. Remy outlines line limits and manual pruning best practices. | |
| Integrating Tools via Model Context Protocol (MCP) | 3 | 5 | 0 | 0 | Remy explains Model Context Protocol (MCP) using a universal language translator metaphor. Greg validates Anthropic's role in developing MCP and recalls past coverage on the show. | |
| End-to-End Executive Assistant Workflow in Action | 4 | 6 | 0 | 3 | Greg challenges whether simple inbox summarization is high-value. Remy accepts the point and demonstrates deep multi-tool chaining across Granola meeting notes, Stripe links, Notion, and Gmail drafts. | |
| Understanding AI Skills as Standard Operating Procedures | 4 | 6 | 0 | 1 | Greg pushes for precise conceptual clarity between memory.md files and .skill SOP packages. Remy demonstrates the skill creator meta-skill and viral hook reference packaging. | |
| Creating Custom Skills and Workflow Chaining | 3 | 5 | 0 | 0 | Remy builds a live referral skill and showcases his comprehensive meta ads analysis workflow. Greg applies the classic jobs-to-be-done framework to identify business processes worth automating. | |
| Scheduled Autonomous Tasks and OpenClaw Walkthrough | 5 | 5 | 0 | 0 | Greg shares his personal recurring agent setup for scraping car marketplace listings, demonstrating concrete practical expertise before guiding Remy to provide beginner harness recommendations. |
Statements from this episode (20)
Gaskill: Founders using AI agents are 10 to 20 times more productive
“The founders and employees that are utilizing agents are like, no word of a lie, 10 to 20 times more productive in their day.”
Gaskill: Chat models are question-to-answer, AI agents are goal-to-result
“The way I think of it is a chat model is question to answer, but then an agent is goal to result. So moving from just like, ah, you asking AI replies, then you do the work, to you giving the agent a task, it planning out the task, and then executing, and then …”
Gaskill: Popular AI Agent Platforms Are Merely Agent Harnesses
“And all of the popular AI agent platforms on the market that you'd be familiar with are just agent harnesses. They're just applications where this loop is facilitated.”
Gaskill: Claude Code, Codex, and Antigravity are secure by default
“So, by default, Antigravity, Cloud Code, and Codex, they're very, very secure because they're built by these massive companies that have a lot on the line to protect.”
Gaskill: Core agent concepts translate seamlessly across all agent harnesses
“Once you know how to drive, you can kind of jump in any car, whether it's like an old Toyota, a Range Rover, and you inherently sort of know what to do. And that just comes down to understanding all these key concepts that we're going to go through today. And …”
Gaskill: Claude Code displays agent loops best among major harnesses
“I think Claude code does the best job of actually displaying that loop and allowing you to see what it's thought about compared to anti-gravity and codex, but it's all just going through the same sort of loop process that I described earlier.”
Gaskill: AI agents must be onboarded like human employees
“And the way I like to think about building agents is onboarding them like a real employee. So if you took on a real executive assistant, you couldn't expect just for them to come into the office and you need to give them a task without explaining your business…”
Gaskill: Manual agent memory setup is superior to automatic chat memory
“And with agents, you have to set up memory and control exactly what you give it. And I think that's actually a benefit, not a limitation, because what happens is if you're using ChatGPT and it's got the auto memory, you're having conversations about three diff…”
Gaskill: AI optimization shifted from prompt engineering to context engineering
“Prompt engineering used to be the big thing. It was like, here's the ultimate prompt for going viral on social media, or use this prompt for this. And now it's all about context engineering. It's about how well can you load up your agent with all the informati…”
Gaskill: Keep Claude.md Agent Context Files Under 200 Lines
“A best practice for those Claude.md files is to keep it around like no more than 200 lines.”
Gaskill: Agent productivity gains come from tool integration, not web search
“By default, most of these agent harnesses, they just have web search baked in, but if you want to actually start linking it up to your tools like Gmail, Calendar, and everything else, which is where the real productivity gains are made, you need to do so, ah, …”
Gaskill: Anthropic built MCP as a universal tool translator
“Anthropic built MCP to basically sit as this translator in between your tools, so that Claude can still just speak English, and your tools can just speak their languages, and this MCP speaks every language, and then just translates your calls from your agent t…”
Gaskill: Future-proof AI stacks rely on local markdown files
“The real future-proof AI stack is just having those markdown files on your computer.”
Gaskill: Markdown is easier for LLMs to digest than PDFs or Docs
“Markdown files is because it's just the easiest sort of format for your LLM, for your agent to actually digest and understand, compared to if you were to give it your files as like a docs or a PDF file.”
Gaskill: People will stop using SaaS apps directly in favor of AI operating systems
“I think that everyone's gonna have, like, an AI operating system they work in, and everyone will just have personal agents and agents to manage each department of their company, and people won't actually use these apps anymore.”
Gaskill uses Claude Code as a unified interface to replace SaaS apps
“Like, I've connected up Gmail Google Drive, Calendar, Granola for my meeting notes, Stripe for payments, Notion for project management, and I don't even enter these tools anymore. I just sit and call code as one central place,”
Gaskill: AI Skills Act as Standard Operating Procedures for Agents
“So the easiest way to think about skills is SOPs for AI. So standing operated standing operated, oh my god, standard operating procedures for AI. It's a mouthful. It means once you explain something once, you never have to explain it ever again.”
Gaskill built an AI agent to scrape and analyze 220 Oodie ads
“And I did an example yesterday with the UDI, which is a super large e-com brand, and it ran through and basically scraped all of, it took screenshots of all the landing pages, it went and scraped all of the ads that are running, all like, 220. It then did a fu…”
Gaskill: OpenClaw is among the hardest agent harnesses to learn; Cowork is easiest
“I would say that OpenClaw is probably, like, one of the hardest to learn and set up of these harnesses. I would say Cowork is probably the easiest.”
Gaskill: Build AI agent context files via interview-style LLM prompts
“To work out what roles you want to start to build out an agent for, go into Claw or your favorite chat model and get it to help you build out those context files through an interview style process. Just say, ask me questions to build this out. I would connect …”