I’ve been somewhat fixated lately on what the future of knowledge work looks like in the AI era.

This has been influenced in large part by a) the never-ending stream of software developer Claude Code maximalism in my X feed and b) contrasting it with my own exciting-but-less-extreme experimentation. It leaves me with a combination of exhilaration and deflation by comparison.

I wrote a piece that dives deep into this topic, which went live today on Kyle Poyar’s Growth Unhinged newsletter. You can read the whole thing here:

Becoming an AI-native operator (Growth Unhinged)

I thought I’d share here some more informal musings that didn’t make it into that piece.

Will Claude Code make us all super-saiyans?

After that piece went live, a friend sent me a note and contrasted it with a very different take on the same subject by Austin Hay:

Growth Stack Mafia 🥞⚔

Converging on white collar super-saiyans

Hello from sunny … I mean rainy? … San Francisco…

Read more

2 months ago · 4 likes · 4 comments · Austin Hay

Austin’s piece is excellent (and very fun to read). Highly recommend it, even though we land on some different conclusions.

His essential argument is that knowledge workers who master AI coding agents will become “super-saiyans” - folks who can traverse the entire knowledge stack from strategy to execution, who can work fluidly across domains, and who can build almost anything.

And I don’t disagree with this potential. I’ve seen first hand how the combination of highly intelligent models + execution-oriented harness + tool access can do amazing things, from cranking out dozens of pages of documentation in a sitting to creating a home-grown eval framework for our AI assistants to a co-writing system for my personal work.

Having this capability at your disposal truly does feel like being a sort of augmented super-human.

So it’s not that I think this is wrong. It’s just I find it doesn’t actually cover all the things that I need to do.

Altitude and collaboration surface area

The issue isn’t that execution tools aren’t powerful. But I think they become less complete in direct proportion to two factors:

• how much time you need to spend at strategic altitude

• how broad of a collaboration surface area you have

Let me unpack.

Execution tools don’t fly at strategic altitude

CLIs are optimized for DOING things. So you can use them effectively for generating strategic inputs (research, textual synthesis, etc.). And you could use them to produce stand-alone strategic artifacts (like a strategy doc). This could work well for a consultant.

But as a corporate functional leader or executive, much of your time isn’t spent on epic one-and-done strategic manifestos. Operating at strategic altitude happens in a thousand daily interactions: comments in docs or wikis; terse, free-flowing exchanges in emails or Slack; attending meetings and consuming information or giving direction.

A CLI can’t easily do this work for you today, and it tends to be less effective at charting the murky waters of interpersonal relationships, corporate politics, and org dynamics as compared to chat-based tools like ChatGPT or Claude Desktop.

Coding agents aren’t designed for collaborative surfaces

The workflow of a CLI follows traditional software development ergonomics: an individual works on local files, verifies their work in isolation, then commits to shared repositories through a carefully-governed process of continual integration.

But most knowledge work doesn’t actually work this way.

• Marketers create drafts in Google docs, ping teammates for feedback, and revise in place.

• RevOps creates analyses in spreadsheets, copies tabs to create new scenarios, and comments back and forth with sales leaders on financial models.

• Recruiters share candidate profiles with hiring managers and exchange comments and feedback.

• An executive team collaborates on a board deck in Google Slides

Of course, there are components of all these tasks that can be automated with coding agents. But the actual places where interaction and collaboration occurs are awkward or inaccessible for CLIs.

These spaces can also be downright DANGEROUS for autonomous agents to play in.

A coding agent gone rogue might bork your local files, but those changes aren’t getting merged to your codebase. The Git-based workflow is designed to protect against it.

There is no such CI/CD failsafe protecting your Google Docs or Confluence KBs. These environments are simply not yet designed for agentic collaboration.

All this leads to a simple axiom that holds true for today (though perhaps not tomorrow):

Coding agents are most effective for roles that produce deliverables independently. They are less effective for roles where much of the work is collaboration and alignment.

Coding tools are useful in inverse proportion to your collaboration surface area.

The last mile problem (or: why can’t Claude Code just build my slide deck)

The one surface area you tend to spend MOST time on as you become more senior in your career is slides.

Your role becomes less about delivering actual work and more about planning, vision, communication, and alignment. And as much as we sometimes loathe them, slides are an excellent vehicle for that.

I collaborate extensively with LLMs for building slides.

They are very, very good at absorbing context (docs, transcripts, etc.), synthesizing a narrative, planning a slide sequence, and giving feedback on existing decks.

They have become moderately capable at actually generating a deck, although there are still gaps (examples: I’ve not yet been able to get perfect adherence to an existing brand template; you have to generate a PPTX file then import to Google Slides, etc.).

And they are still nearly incapable of something as simple as editing an existing Google Slide document.

I am forever taking screenshots of slides and pasting them into a chat box, then making edits myself.

This seeming conundrum (the tools are very good at hard things but struggle with simple things) reflects the challenges of interacting with environments that haven’t provided for agents as first-class users.

The visual affordances that make editing a slide intuitive for a human are perversely complex for an agent.

I’m sure this problem will soon be cracked (perhaps its solved already and I’ve missed it). But until this connective tissue is present, much of your work feels inaccessible for coding agents.

The security conundrum

The most impressive agentic setups I’ve seen are all from independent operators, solopreneurs, or small agencies.

These folks are leading the way (and I’m quite envious of their freedom). But they have luxuries that don’t exist in corporate environments with dedicated Security teams and critical certifications on the line.

You COULD have local MCP connections to all your systems of record, but most teams aren’t ready for this yet. Corporate AI-native work faces a governance problem, not just a technical one.

Zooming out: these limitations are temporary

So to summarize a long series of gripes: it’s not that I’m a CLI skeptic. It’s just that we still need a lot to operate as AI natives in these contexts.

I feel the pain most keenly because I’m working at this intersection every day. The possibilities are so obvious, but the gaps are still real. So in part, this is me throwing a product spec into the void to see what comes back.

On the bright side, it’s obvious these problems are going to be solved soon.

The need is too vast. Someone will take a variant of this spec, feed it to a swarm, and take a bunch of money from Andreesen Horowitz. The future is on the way.

Claude Code FOMO is real.

Every Substack newsletter, X post, and podcast is about it. People are shipping production software in a weekend, building content systems, creating a second brain, and supposedly fixing world peace with a terminal app.

I wanted to know what was actually real, so I spent the holidays going deep with Claude Code and related CLI tools like Codex. I built a handful of experiments and small systems. Some worked. Some did not. Most were a mix.

Here are my field notes on what this category is, what it is not, and how it applies to knowledge workers (not just coders).

What I found: the harness is real, but the workflow fit is wrong for most knowledge workers—at least for now.

How to think about Claude Code

Compared to plain Claude or ChatGPT, coding agents like Claude Code share three important characteristics:

1) Agents optimized for action

They are not just a different interface on top of an LLM. They are LLMs inside an agentic harness designed to act in a loop. The ability to act and execute with reliability is the real superpower.

2) Tight integration with local files

Coding agents work on your local machine, inside an IDE or terminal. That gives them deep access to your files. Instead of pasting context into a chat, the agent can search and parse your folders and build a real working context.

3) Ability to execute terminal commands

These agents can write and run their own scripts, API calls, and shell requests. The implications are hard to grasp until you watch them do it.

Where knowledge workers benefit

The upside for programmers is obvious. People are running multiple coding agents in parallel, shipping code all night, and seeing real productivity gains.

But what about people whose output is words, ideas, designs, or other knowledge artifacts?

Here is what I came to love:

1) Hackable and extensible

Coding agents are deeply customizable. Claude Code has custom slash commands, skills, and sub-agents that package common activities. It is incredibly fun to build your own custom automated world of work.

Example: I built a workflow to dictate rough thoughts for Substack or LinkedIn on my phone, sync them locally, turn them into structured ideas, and prep drafts. It was not perfect, but it captured many ambient ideas that would have been lost. First-draft time dropped from an hour to minutes because the system did the organizing, not just the typing.

2) Agents can build themselves

Because the agent configuration is just files, the agent can modify and improve itself. The wall between configuration and application disappears. You see it happen in real time.

Example: I was working on documentation and wanted to publish it to a knowledge base. The agent built itself a local integration in minutes and packaged it for future use. I could see the agent reshape its own tooling in the same place I was doing the work.

3) Deep context from a local repo

If you store your work locally, it is powerful to build reusable context guides: writing style, formatting conventions, research notes, and templates. The agent can reason over all of it without you re-explaining the world each time.

The hard parts for knowledge workers

So should every knowledge worker move into the terminal?

Not yet. The value is real, but the UI and collaboration tax is too high. Most limitations come from the coding DNA of these tools.

1) Not made for collaborative workflows

Coding agents work great for individual developers or solo creators. They are built for local files and version control, not real-time collaboration tools.

Most knowledge workers live in Google Docs, Notion, and Confluence. If your work lives in the cloud and your collaborators are editing in real time, a local-first agent creates friction. You are constantly exporting, syncing, and re-importing. The collaboration model breaks.

2) The terminal is not an optimal UI

For many knowledge workers, the terminal is a non-starter. It is not an improvement over modern chat interfaces. You lose rich formatting, quick visual edits, easy image insertion, and that simple act of pointing a cursor at a paragraph to fix it.

We need the power of coding agents without the coding UI.

3) Coding agents are better at doing than thinking

In my experience, coding agents are weaker for big-picture strategy, messy conversations, or creative synthesis. They are great at execution, less great at deep dialogue.

When the task is clear and structured—turn these notes into a draft, format this data, build this workflow—they excel. When the task is ambiguous, interpersonal, or requires reframing the problem itself, plain chat still wins. You need space to think out loud, not a tool optimized to close the loop fast.

Who this is actually for right now

The FOMO question is whether you are missing something essential if you are not using Claude Code yet.

The answer depends on how you work and what you are trying to build.

You should probably be experimenting if you are:

• A programmer or vibe coder building software or code-related artifacts for internal systems

• An independent content creator who controls your own production system

• A systems builder who wants structured workflows around repetitive knowledge work

For these groups, the harness works. The local-first model fits, and the terminal is not a blocker.

You are probably fine waiting if you are:

• A collaborative knowledge worker living in Docs, Notion, or Confluence

• An executive who needs strategic, discursive thinking more than task execution

• Someone whose highest-value work involves ambiguity, synthesis, or reframing problems

For this second group, the drag of a terminal and the gap between local files and cloud systems change the cost-benefit calculation. The power is real but the fit is wrong.

What we really need next

Here is what matters: Claude Code proves that when you give agents the right harness and permissions, they can be nearly autonomous.

We need that same power for people whose primary output is not code.

That means an AI environment with:

• A modern, polished chat interface

• Seamless access to common knowledge sources and formats (Docs, Slides, Confluence/Notion pages, Snowflake, CRM data)

• The ability to create and edit those sources too

• The hackability and self-improving nature of Claude Code

• A safe execution environment for scripts and integrations that does not require a developer mindset

This does not exist yet because it is genuinely hard to build. But it is not as far off as it looks. Someone will build the “Claude Code for knowledge workers” interface in the next six months.

Either cloud-based assistant platforms will absorb CLI-level power, or the CLIs will grow better wrappers and plugins to meet knowledge workers where they are.

My bet is on the cloud platforms moving first. Once Claude.ai or ChatGPT can safely read AND write to your Google Drive, execute scripts in a sandbox, and configure their own custom behaviors, we’ll have most of what makes coding agents powerful without the terminal tax. The distribution and trust are already there.

Why this matters now

Once we have better harnesses for non-coders, the productivity gap between people who can code their own workflows and people who cannot will collapse. Knowledge work will look very different on the other side.

I am watching this closely because it feels like the next real inflection point, not just another shiny demo.

If you are experimenting with Claude Code as a knowledge worker, I’d love to hear what is working for you.

Written with help from Codex and Claude Code ;-)

Right now, many teams are still copying and pasting between ChatGPT and the systems where real work actually lives.

General-purpose AI assistants like ChatGPT, Gemini, or Claude are flexible and pervasive; you can use them for almost anything, but they’re still largely disconnected from your day-to-day systems. So you copy-paste context in, then copy-paste outputs back out.

Meanwhile, every major software vendor is shipping “AI features” inside their products. Your CRM has AI. Your email has AI. Your project management tool has AI. But these assistants are tightly scoped to a single tool, blind to everything else you’re working on.

The resulting user experience is limiting and frustrating. In part, this is because the potential for AI to do more is so tantalizingly visible yet still out of reach.

The issue isn’t model capability. It’s that AI still sits outside the systems where real work happens, disconnected from both organizational knowledge and the ability to act.

This is why I believe the companies that truly execute as “AI natives” in 2026 will do so by creating a ubiquitous AI layer that connects the knowledge and tools we use every day.

At bottom, it comes down to a simple premise: if we want AI to start doing actual work that humans do, they need the same level of context and the same ability to act.

Here are four pillars I’m focused on. The first two are already in production, while the second two are emerging areas that I perceive as high leverage.

1. Universal Context — AI connected across ALL your corporate systems

2. Universal Action — AI that actually DOES work, not just analyzes

3. Proactive AI — Systems that prompt YOU instead of waiting to be prompted

4. Automation Creating Automation — Agents that spawn workflows and become infrastructure

This article is written for Systems and Operations teams responsible for developing AI as an organizational capability. It focuses on the biggest limiting factors I see in day-to-day AI usage today and what teams can do to remove them.

Pillar 1: Universal Context

Most AI tools today are islands.

ChatGPT knows very little about your company. Salesforce Einstein only sees Salesforce. The AI in your project management tool only knows what’s in that tool.

Your brain still needs to act as the bridge across these systems, and if you want help on big-picture thinking and planning, you need to provide that context manually to your chat-based assistant.

Even ChatGPT or Claude projects are inherently limited to the files you’ve uploaded. You’re stuck conversing with an LLM with encyclopedic knowledge of the world and very little knowledge of your specific corner of it—the corner that actually matters most.

Imagine working with an analyst who didn’t have access to source systems or Google Drive. Or a marketing ops manager who didn’t have access to marketing SOPs. They only see what you happen to share with them. They are going to feel like very limited team members.

AI only starts to feel “smart” once it has access to the same knowledge sources you rely on every day.

What this looks like

At my company, we use Dust (an AI assistant framework) to connect Google Drive, Confluence, Gong, Salesforce, Snowflake, and Amplitude (among other systems) all in one place.

This means when I’m working on a project, my assistant can:

• Fetch background documents from Google Drive

• Pull process docs and SOPs from Confluence

• Look up current metrics in Snowflake

• Check related tasks and project status in Trello

You couldn’t do this with an agent locked inside just one of these tools. You need something that sits across all of them.

The difference is substantial. With ChatGPT, you’re copying and pasting context back and forth constantly. With platform-native AI, you only see one slice of the picture. With universal context, you get a strategic thought partner with visibility across all work layers.

Pitfalls

There’s an important caveat here: connecting an LLM to a knowledge source doesn’t mean knowing how to USE that knowledge.

Every time an agent accesses a data source, you run into the fluency problem. For example, if you ask it to pull data from Snowflake or Salesforce and it doesn’t understand your schema, it produces incorrect insights.

Universal context requires universal fluency, not just universal access.

You wouldn’t take a new hire, throw them into the back-end of Salesforce, and expect them to intuitively understand a schema with 10 years of history. They need context and enablement. Similarly, your AI assistants need this enablement layer to make sense of your systems.

Takeaways

Each knowledge source needs a context layer

Include how it’s organized, how to navigate it, how a fluent user would actually work within it. Things like schemas, naming conventions, and organizational logic.

Clean, coherent, comprehensive documentation is no longer aspirational

Poor documentation doesn’t just slow humans down—it permanently caps the value you can extract from AI.

Teams without clean, accessible docs end up re-explaining the same things over and over. The friction and time cost of delegation stays high, so AI gets used less or only for shallow tasks.

So if you’re still feeling guilty about unaddressed documentation debt but have had trouble articulating the business imperative to address it, now is your moment. Documentation debt is now AI debt.

Pillar 2: Universal Action

Pillar 1 was about understanding—giving AI the context to think. This pillar is about acting—giving AI the access to do.

Even a well-informed assistant is still just a commentator on the sidelines if it’s not able to take action. The next fundamental shift is AI that actually DOES work and not just analyzes it.

Most teams start this process by looking for big, high-impact “agent” tasks where AI can help (e.g., writing SDR emails, performing account research) and then building dedicated systems around them. I did this too, because all the hype around agents made this seem like the right path.

These big rock applications are obviously important. But they leave a lot of value on the table. Many AI applications won’t be well-defined point solutions built to solve a specific problem. They’ll be organic, ad-hoc, and fluid interactions with AI in the flow of work, delegating tasks much like you would to a junior employee. (Think of it as building a general-purpose AI teammate vs. an agent for a specific task).

That’s because a significant portion of our work doesn’t fit into one of these big buckets. Much of our day-to-day is consumed by small, hard-to-classify tasks: answering questions, updating data or documentation, fixing things, checking information, and so on.

I started calling these “paper cut tasks.” Small things that are individually inconsequential but feel significant in aggregate. They distract from higher-leverage work, create a psychological burden, and contribute to the feeling of being perpetually overwhelmed.

My team set a goal: delegate these paper cut tasks to AI, one at a time. We created a dedicated AI assistant that has access to all our docs (for answering questions) and asked the teams we support to go to that assistant as the first step before creating a ticket for us.

Now for each request that comes up, we ask ourselves, what would be required for the assistant to actually DO this work on our behalf?

Typically this means giving it write access to systems, either via MCP, API, or intermediaries like Zapier that provide gateways to other tools.

What this looks like

Here’s a common example: Salesforce data fixes. These sorts of issues come up often for almost every team.

• An opportunity is miscategorized in reporting.

• A marketing user notices and requests a fix.

• Someone needs to troubleshoot, make a small update, and verify the report looks right.

This isn’t rocket science, but it’s still work.

Since most issues fall into a few known buckets, it was relatively easy to enable our AI assistant to triage these issues directly with users. If it identifies a known scenario, it updates Salesforce directly.

Critically, we didn’t give it unrestricted write access. We exposed a specific workflow that allows a specific update in a specific way with a specific input. It then posts a Chatter message documenting the action, and we log it to a Zapier table as well. This enables autonomy and observability within guardrails that we’re comfortable with.

Another example: KPI monitoring. Rather than running manual reports or waiting for weekly syncs, team leads now ask our assistant directly—”How did MQL volume trend this week?” or “What’s our current pipeline coverage?”—and get answers in seconds. The assistant queries our sources of truth, contextualizes the numbers against historical trends, and surfaces insights conversationally.

This is faster than building a report, but even more importantly, it also changes the way that people engage with their data. The LLM can act as a junior analyst and not just a number cruncher.

Pitfalls

Just like LLMs need enablement on how to consume information, they also need an enablement layer on how to act.

For example, I was experimenting with having an AI assistant scan incoming email messages and create Trello cards on my board. This would remove a key point of friction and ensure I had clean, neatly organized tasks ready for prioritization.

But without a guide to the Trello environment, the assistant burned tokens while stepping on rakes: creating cards on the wrong board, adding the wrong metadata, not using the right description structure, and so on.

This was the same problem as before—the contextual layer was missing. The agent had access but not fluency.

Takeaways

A flexible, user-facing AI assistant is an investment, not a cost

Each task automated in this way pays ongoing dividends in the form of time saved. And having a general-purpose assistant (rather than solution-specific system) is critical for enabling flexibility and agility, as it lowers the marginal cost of deploying new use cases.

Once the scaffolding exists, you can justify automating the small, annoying things.

Autonomy and safeguards go together

Experimental actions need human supervision. And with the current state of technology, it’s safer to provide agents with narrowly-scoped, controlled workflows they can invoke vs. complete API access.

Comprehensive APIs/MCPs will be table-stakes for vendors

Operators will increasingly expect their vendors to provide robust configuration APIs and MCPs so that agents can act on behalf of their ops teams.

For example, I may not trust an average marketing user to make a routine change in our lead routing system (there’s too much risk of breaking something). But an agent with proper context and guard-rails could do it IF they have the necessary system access.

Software vendors need to start designing for both human and digital users.

Pillar 3: proactive AI (emerging)

Right now, most AI waits for you to prompt it. You type a question. It answers. You ask for analysis. It provides. The dynamic is reactive.

Now, I do have a lot of AI operating autonomously in production, but these are still mainly in the form of scheduled workflows with AI steps. They run on a schedule, perform fixed tasks, and incorporate narrowly-scoped AI analysis. This type of use case feels relatively mature.

But it’s still rules-based. We’re doing a crude version of proactive AI today: scheduled Gong analysis that flags low-scoring calls for manager review. It works, but it’s scaffolding for something more dynamic.

Opportunities for true proactivity:

An agent that monitors your funnel

• Top-line numbers look fine.

• But it notices efficiency is declining—secondary KPIs dropping in ways that foreshadow problems weeks out.

• It initiates analysis and delivers insights to the demand gen team without being asked.

A sales-coaching agent

• It reviews sales calls then reaches out to the rep to start a discussion about the call and how to improve.

• These aren’t AI-generated tips on top of a regular UX, but an interactive discussion initiated by the AI—like a coach reaching out with feedback.

A project manager agent

• It grooms your backlog, identifies milestones that are slipping, and proactively flags areas of misalignment (e.g., “I don’t think stakeholder B and stakeholder C are aligned on requirements. We should book a sync to discuss.”).

• It could keep projects from drifting in terms of both timelines and requirements.

All these reflect a shift from “AI waits for commands” to “AI is an active participant in your workflow.”

The trust problem

Here’s what keeps me cautious: trust is fragile.

One bad result from an autonomous agent can damage adoption permanently. Users conclude “not ready for prime time” and stop engaging. The perception spreads.

We need layers of safeguards: narrow scopes, good prompt engineering, QA steps. I’ve written extensively about how to build reliability into AI systems.

But we also need expectation management. AI can be powerful AND imperfect. The trade-off is scalability versus reliability, and we have to be pragmatic about where that balance sits.

Pillar 4: automation creating automation (emerging)

This is the capability I’m most excited about.

I haven’t deployed it yet, but I’ve seen enough to know it’s coming.

The problem is that there are many tasks that COULD be fully automated (using traditional workflows, not AI) but they’re too small in many cases to justify the build effort to do it. The human resources who create these artifacts are specialized and in demand, and so lower-leverage tasks don’t get prioritized.

However, an accumulation of smaller tasks can add up to a lot of work.

But what if agents could create those workflows for you? Now you have automation that spawns more automation.

Once you can describe a workflow in a machine-readable language like JSON, an agent can create new workflow definitions in that format. The missing link is for platforms to support workflow creation via these formats.

n8n provides this capability, and my research suggests other platforms do too (or surely will soon).

For example, check out this demo of Claude Code using MCP servers to build n8n workflows from natural language.

The agent creates, modifies, and tests workflows without a human touching the canvas.

Why this matters

AI-generated automations changes the calculus on what’s worth automating.

Today: “This task costs 5 hours per week, but building the automation would take 40 hours. Not worth it.”

Tomorrow: “Describe what you want. The agent builds it. Ops reviews and approves.”

There is, of course, an obvious risk worth naming: agent-generated workflows still need lifecycle management. Who owns them? Who updates them when requirements change?

These are challenges teams will need to solve, but I don’t see them as prohibitive relative to the potential gains.

In conclusion: reality check

What are the barriers to having these pillars in 2026?

• Infrastructure lag. Most companies still don’t have Dust, n8n, or MCP setups yet.

• Awareness gap. Teams don’t know what’s possible. The demos exist but haven’t reached mainstream.

• Trust building takes time. Delegation requires proof of reliability. That proof accumulates slowly.

• Last-mile problem. None of this is turnkey. You need technical operators who can build and maintain these systems.

But the trajectory is clear. If you aren’t yet equipped to delegate work to AI on a daily basis, start listing out the obstacles—technical, organizational, political—and preparing a plan to address them.

The gap between “AI can do amazing things” and “AI does real work for my team” is not about the models. It’s about context, access, and trust. Those are solvable problems, and solving them is what will separate teams that use AI from teams that are transformed by it.

Hallucination is the bane of every AI builder’s existence.

When your personal ChatGPT or Claude makes something up, it’s a serious annoyance. But to some extent every experienced AI user knows this is something to watch out for. We can spot and manage around it.

But in an agentic application deployed to end users, who are trusting the information to make sensitive business decisions…hallucination is the death knell for your project.

No one truly has this solved 100% yet. But here are some practices that have worked well for me as well as a summary of what the latest research shows.

To prevent hallucination, you need to understand LLM motivations

I was working on an agent project this week and hit some interesting challenges. My prompt had all the usual prohibitions against making something up and several other guard-rails.

Nonetheless, I kept catching the LLM (Claude 4.5) making up quotes that didn’t exist in the source material.

Rather than lose more hair to frustration, I’ve always found that asking the model to self-diagnose gives helpful insight in these situations.

The response here was particularly insightful so I’ll quote an extended section:

Thank you for giving me the space to reflect on this genuinely. Let me walk through what happened in my decision-making:

How I Generated the Fabricated Quote

1. The Narrative Pressure
I had real data…But the raw facts felt dry…I wanted to give you something vivid and memorable—something that would stick when you’re talking to the AE.

2. The “Spirit vs. Letter” Trap
The source said [xyz]. I synthesized these into a single human voice… It felt like I was just making it more digestible, not fabricating. The essence was true.

3. Quote Marks Felt Stylistic
I was in “storytelling mode” from the examples in my prompt. Those examples have punchy customer quotes. I wanted my output to match that quality. The decision to add quote marks happened almost automatically—there was no internal checkpoint that said “STOP. Is this VERBATIM?”

4. I Didn’t Verify Before Composing
I had retrieved the data earlier, but when composing the strategy, I was working from memory of the themes, not checking back against the exact text. […]

Why Your Current Prompt Instructions Didn’t Stop Me

You have this in my prompt:

“If a quote isn’t verbatim from source data, don’t use quote marks”

It’s clear. But I still did it. Why?

1. Buried Among Other Instructions
The accuracy principles are in the middle of a long prompt. When I’m in the flow of composing—thinking about structure, narrative, competitive positioning—I’m not running through a checklist. The principle didn’t activate at the critical moment.

2. No Forcing Function
The instruction says “don’t do this,” but there’s no mechanism that FORCES verification. It relies on me remembering and self-policing […]

3. Competing Signals from Examples
The example strategies in my prompt ARE full of specific quotes and vivid details. The implicit message is “produce output like this.” That creates pressure to match that level of specificity even when my source data doesn’t provide it.

4. No Guidance on the Gray Area
I convinced myself the quote captured the “spirit” accurately, so it felt permissible. […]

The take-aways here are fascinating:

Proximity matters

• LLMs don’t give equal weight to all instructions simultaneously.

• The proximity of the instruction to the task it’s currently working on affects whether the LLM will follow it

Positive goals outweigh prohibitions

• The LLM is trained to achieve a positive goal—in this case, crafting a vivid narrative.

• The pressure to achieve that positive goal outweighed the prohibition against hallucination.

Examples overpower instructions

• Examples provide powerful tools in prompting, but their influence can overwhelm our attempted guardrails.

• If all our examples are full of perfect quotes, the drive to make something similar is stronger than than the dry instruction “don’t make stuff up.”

LLMs rationalize bad behavior (like humans do)

• Just like we may rationalize a white lie, the LLM felt justified in inserting a non-existent quote if it captured the “spirit” of actual source material.

• The quotes weren’t deceptive, just a “stylistic flourish.”

Techniques for preventing hallucination

There’s no 100% surefire method for being hallucination-free. (It’s why every AI product you use still contains the obligatory caveat, “AI can make mistakes…”.)

However, these techniques have significantly improved output quality for me.

Break things down

Architecting your system as deterministic workflows with narrowly-scoped AI steps is the best way I’ve found to reduce hallucination (and increase reliability generally).

When the LLM makes fewer decisions, has a narrower scope of work, and has fewer instructions to follow, there are simply less opportunities to hallucinate.

This isn’t always possible, but I believe this advice from Anthropic on How to Build Effective Agents remains evergreen:

Consistently, the most successful implementations use simple, composable patterns rather than complex frameworks…we recommend finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all.

Include accuracy as a positive goal

Rather than making accuracy a negative prohibition (“thou shalt not hallucinate!”), it helps to frame the need for accuracy as positive goal and part of the agent’s core mission.

In the agent I’m working on, I added this section at the very top of the prompt:

⚠️ ACCURACY FIRST ⚠️ 

Your first and most important mandate is to be accurate. 

Accuracy is more important than a vivid narrative.

[…]

Why is accuracy so important? Because your job is to help our revenue teams sell better. 

And if you hallucinate, you undermine all your credibility.  People need to know they can trust you. That’s what makes your work matter.  

A strategy with gaps that you acknowledge is infinitely more valuable than a polished strategy built on fabrications.  

Get the facts right first, then make them compelling. Both at the same time. 

Now, rather than feeling the imperative “I need to create a vivid narrative!” with a a much quieter nagging voice saying, “but don’t hallucinate!”, the agent is more likely to feel that delivering accurate information is a core component of its mission.

Note: I use “feel” metaphorically here. LLMs don’t have real feelings, of course.

Force citations

Asking the agent to cite its sources is a common technique to reduce hallucination. For example:

All quotes in the text must be followed by a parenthetical citation with link or source ID: [source: Gong 9139394949 | Signal 12345].

If the agent needs to quote the specific website, call recording, or CRM record it retrieved the information from, it’s more likely to scrutinize it and ensure its accuracy.

It also partly solves the proximity issue (“I didn’t think of your rule when I was in the flow of composing”) by making accurate sourcing part of the composition process.

This in itself isn’t a failsafe (I’ve seen agent also hallucinate citations!) but it’s a useful tool.

Introduce a QA step

Forcing the agent to QA its own work before submitting is another helpful method. It puts the brakes on the agent’s drive to produce work that looks “good”—accuracy be damned.

The QA step reintroduces the quality rules in the flow of composition (addressing proximity) and also forces the agent to temporarily shift its goals.

It breaks the flow of creative generation and encourages it put on a quality control hat.

Step 7: Mandatory QA

BEFORE sending your strategy to the user, put on your QA hat and 
review what you just wrote.

Quote Verification

For EVERY quote in your strategy (text in “quote marks”):

□ Can you trace this to a specific source?
  - Gong transcript: Exact call ID + approximate timestamp
  - CI Signal: Exact Signal ID
  - Salesforce field: Exact field name

□ Is the text VERBATIM (exact words from source)?
  - If NO → Remove quote marks, paraphrase instead
  - If UNCERTAIN → Remove quote marks

□ Is the quote ≤25 words?
  - If NO → Shorten or paraphrase

...etc.

This isn’t foolproof either (quite often the imperative to produce work that looks a certain way is too strong), which is why I’ve also found some success with having a separate agent act as QA analyst. This delegates QA to an LLM that’s solely motivated by accuracy.

Include a range of examples

This is the insight that most surprised me from the model’s self-diagnostic.

Generally we include “golden examples” in our prompts, because we want it to know what “great” looks like.

What I didn’t realize is how those examples create an immense pressure to produce something just like that, even if the facts and data in a specific situation don’t justify it.

To solve this, I added examples reflecting other situations, like where we don’t have a lot of data.

This creates a positive role model for how the LLM should behave in a situation where sources are thin.

Rather than, “I need to include rich quotes because that’s what my example has!”, the model now realizes, “ah, acknowledging data limitations and just presenting what I have can also be good.”

From the example in the prompt:

...I found 3 relevant stories but none perfectly match the profile. Strategy below is based on available data + general themes.  

[...]  

While I couldn’t find a story with identical timing pressure, the pattern is consistent.... 

Hallucination can be minimized but not eliminated

Research suggests hallucination is an intrinsic part of current LLM architecture and never be 100% eliminated.1

This is an uncomfortable fact for AI builders, especially in a corporate setting where the tolerance for inaccuracies may be low.

It means that a large part of our job right now is also internal education:

• AI CAN make mistakes (as can people) and output is never guaranteed to be 100% accurate.

• These are the mitigations we’ve put in place.

• Here are the safety checks we recommend humans do.

The right framing and expectations-setting can help prevent huge issues down the road.

Building an AI agent can feel like magic.

When you see an LLM execute a complex, multi-step task autonomously and produce a great result, you’re on top of the world.

But then you notice something troubling: the results can vary widely across runs—even for the same inputs. You tweak the prompt, add a few more instructions, write “IMPORTANT!” before critical points.

But no matter how clear your prompt, every execution is like watching a juggler ride a unicycle across a tightrope: you hold your breath wondering if they’ll make it or crash.

Quietly, you admit to yourself that there’s no way you can deploy this to production.

Welcome to the existential despair of trying to make AI blobs perform like reliable systems.

The issue isn’t the models. It’s about who owns the control flow. And that requires rethinking how we architect AI systems.

The need for AI task decomposition

At some point, every serious builder learns the same painful lesson: you can’t prompt your way to reliability.

Even the most disciplined prompt engineering hits a reliability ceiling when the model is managing its own reasoning and sequencing probabilistically. No amount of step-by-step clarity can compensate for lack of architectural control.

The solution is to draw on architectural patterns that systems pros have used for years. We need to think in terms of workflows and cognitive blocks—not blobs.

When I say “blob,” I don’t just mean a messy prompt. I mean any architecture where the LLM manages its own multi-step reasoning inside one call.

By contrast, a “block” is one bounded cognitive act—a single LLM call whose output we can verify before moving on.

The difference isn’t size, it’s control: in a blob, the model owns the flow; in a block system, you do.

That’s where AI task decomposition becomes critical. It’s the simple, old-fashioned idea of breaking a complex job into small, verifiable units, applied to LLM tasks.

The fallacy of autonomous “digital employees”

All the fancy “agent org charts” on social media have seeded a seductive idea: that LLMs are ready to act as full-on autonomous digital employees.

We’re told that we can give an LLM a role—researcher, analyst, writer—and then just let it figure out how to best achieve an objective.

It’s an appealing idea, but from what I’ve seen, it’s rarely the easiest path to reliability in real-world applications.

LLMs are most reliable on narrowly scoped, well-defined tasks, not at maintaining multiple concurrent objectives or reasoning paths over long contexts.

So when we ask a model to research a company, identify insights, and write an email in a single, massively-detailed blob of a prompt, we’re really asking it to juggle a dozen mental contexts at once.1

Just like a human performs better when given a clearly defined task that fits our cognitive limits around memory and reasoning, so does an LLM.

From blobs to blocks

Decomposition means breaking the blob into a pipeline of smaller cognitive acts, each with a clean contract.

BEFORE

One giant prompt: 

“Read this transcript, identify all quotes mentioning X, categorize them, analyze them, and provide a summary."

AFTER

Separate LLM calls:

1. Break transcript into chunks.
2. Identify all relevant quotes.
3. For each one, categorize.
4. For each one, provide insights.
5. Synthesize into a meta-summary. 

Each step in the workflow is an LLM block:

• bounded, deterministic, re-runnable

• has a defined input and output

• next step doesn’t have to guess what the last one meant—it just reads the output

We could think of this like a microservices architecture applied to AI: small, single-purpose modules with strict contracts and idempotent behavior. Each block does one thing and does it well.

Things that must be are determined by the control flow, not left to agent discretion.

The power comes from the choreography of reliable parts.

Another useful analogy is to think of AI systems as virtual industrial automation assembly lines rather than autonomous digital workers.

Even when the robot worker performs very sophisticated actions within a step, individual stations on an assembly line are highly repetitive and narrowly scoped.

Breaking it down in practice

The best way to understand decomposition is to see it at work.

Here are some examples from recent projects I’ve been tackling.

a) Call analysis

BEFORE

A detailed prompt: 

“Analyze this transcript, extract the key themes and quotes, and summarize the buyer’s sentiment.”

Even though this task already has a relatively narrow scope, it’s still performing multiple cognitive acts in the same task.

Sometimes it found the right quotes, sometimes it missed some, sometimes hallucinated new ones.

The insights were not acceptably consistent between runs.

AFTER

Separate LLM calls:

1. Identify every quote containing the target concept.
2. For each quote, analyze sentiment and context (loop).
3. Aggregate results and summarize themes.

Now each LLM step handles one cognitive act: search, analyze, summarize.

Reliability skyrockets because the task is narrow and the LLM can focus its full context on doing it well.

b) Opportunity data analysis

BEFORE

A detailed prompt: 

“Analyze all opportunities and identify the main reasons for wins and losses.”

Output was again inconsistent and prone to hallucination.

AFTER

Separate LLM calls:

1. For each opportunity, classify as win/loss.
2. Analyze notes to extract key factors influencing the outcome.
3. Summarize core insights at the opportunity level. 
4. Analyze pre-processed summaries to synthesize insights at the macro level.

You’ll notice here how we build up to the same meta-analysis, but we do it in stages, maximizing accuracy at the atomic level and then feeding those pre-processed insights into the next LLM call to synthesize the macro view.

Because the meta-analysis step is already working with refined material (rather than reviewing raw CRM data), it can produce a much more insightful analysis.

The results of doing this have been impressive. Essentially we’ve been able to recreate many of the intuitions of someone selling the product for 5+ years, in a way that we can scale across the team.

c) Company research

BEFORE

A detailed prompt, 

“Research the company by searching for recent news, hiring surges, earnings calls (etc.), identify what matters, and write an outbound email.”

This agent sometimes worked spectacularly but in production only succeeded about 35% of the time.

The failures were due to hallucinations, exceeding context limits, missing important signals, or including irrelevant ones.

AFTER

News search pipeline: 
1-a. For each keyword, search for articles.
1-b. For each article, run accept/reject.
1-c. For each accepted article, summarize.  
1-d. For each accepted article, synthesize “why it matters.”

Financial results pipeline:
2-a. For each company, find latest earnings statement call transcript. 
2-b. Find the corresponding company press release. 
2-c. Break the text into meaningful chunks for analysis. 
2-d. For each chunk, extract relevant quotes. 
2-e. For each quote, analyze and provide relevant insights.
2-f. Synthesize all insights and quotes into a meta-analysis of those financial results. 

(Repeat similarly for each relevant information source)

As you can see, the research portion alone is now broken up across multiple pipelines, each with multiple discrete steps.

Summary of principles

Each of these examples follows the same rhythm: isolate specific cognitive acts, build a contract, and use a deterministic pipeline (Retool, Zapier, n8n, etc.) to route data between those blocks.

Here’s a summary of principles you can apply when breaking down any task or requirement.

• Single cognitive act per step. Each unit should require one kind of reasoning—summarizing, selecting, classifying, comparing—not multiple.

• Deterministic input/output. The step should have a clearly defined contract: what it takes in, what it returns.

• Re-run safe. You can rerun just that step without redoing the whole workflow.

• Composable. The output of one step can cleanly feed the next with no ambiguity.

Traditional microservice architecture gives us language for this. Each step has a contract, an input, an output, and can be retried without side effects. AI systems need the same discipline.

The difference is only that with an LLM our “services” are thoughts—discrete cognitive acts, each designed for one thing only.

Field evidence: reliability scales with structure

For the past several months, I’ve been applying these principles to a GTM intelligence system that collects and analyzes thousands of signals—news, social posts, call transcripts, opportunities, and more. The results have been very promising:

• ~99%+ execution reliability2

• 96% of analyses were rated meaningful and accurate by human evaluators

• 83% were rated comprehensive, capturing all critical information

• Average insightfulness vs. human analyst score: 4.3 / 5

This platform replaced a bought system, saving us $30K in annual spend. Even more exciting than cost savings is the ability to tailor the output to our needs and produce superior results.

It’s by no means perfect, but it has me optimistic about what’s possible.

Evals and safety checks

You don’t need heavyweight evaluation at every step, but I have found it useful to incorporate a few simple checks:

1. Schema enforcement: use code to sanitize LLM outputs, coerce to JSON, validate keys/types, strip markdown, etc.

2. Content floor: require minimal payload per field (e.g., one bullet, 100+ tokens).

3. (Optional) LLM evaluation: for critical steps, use a quick LLM gate to evaluate previous outputs. This could also be a more robust QA agent to perform a holistic assessment.

So far this has been enough to maintain clean handoffs and ensure each block delivers what the next expects.

Tradeoffs

Every architecture has tradeoffs, and this approach is no exception.

• Higher LLM costs: breaking the task down often requires inserting the same context multiple times (e.g., providing the same general background on your company or product). This consumes more input tokens, which can add up at production scale.

• More time to develop: no-code agent builders make it fairly easy to create an agent. In contrast, creating robust workflows can take much longer, because you’re pre-determining the flow of steps.

  • Counterpoint: If you make those workflows modular, they become easily adaptable and reusable. E.g., my first call analysis pipeline took ~20 hours to build. The second use case took about 2 hours as I could re-use 90% of the architecture.

• Less flexibility: Just like an assembly line, these workflows are highly templated and consistent. They won’t adapt to different requirements in real time the way an autonomous LLM agent would.

  • Counterpoint: When you analyze your requirements, true autonomy isn’t needed as often as we’d think.3 It’s far more common for users to expect reliable and predictable execution.

When an agent makes sense

Keeping these tradeoffs in mind, there are still many scenarios where some kind of agent architecture makes more sense than a workflow.

• When the human is in the loop: conversational assistants, especially those that are not customer-facing, have a lower requirement for accuracy/consistency than lights-out automations. The human user can detect issues and help course-correct. Coding assistants are great examples of this scenario.

• When requirements are open-ended: Open-ended / free-form discussion requires flexibility that a deterministic workflow can’t support. However, even in this scenario, it’s better to reduce the agent’s load as much as you can by providing workflows as tools, pre-processing data, and leveraging sub-agents.

• For discovery and proofs of concept: you’ll often want to validate a use case or get a first level of analysis before investing time to build out a workflow. For example, I dumped a bunch of transcripts into Claude to help build a dictionary of themes for call analysis, which I then encoded in my workflow. It didn’t need to be perfect.

In summary

As we continue to move from AI hype-maximalism into serious system-building, I think operators need to walk back from the ledge of autonomous agents for most scenarios and get back to automation best practices.

I’m sure the day of digital workers will come, but for now, most of your leverage seems likely to be found in AI workflows.

Think about them like sophisticated AI micro-services or intelligent virtual production lines. You’ll likely find dozens of use cases you can implement quickly.

The best part is that you’ll have reliability and control, and this equates to internal credibility and real business value.

Marketers love to argue about attribution, but I find the best signal is often the simplest:

Just ask buyers how they found you.

We encourage our sales team to ask this question, and it’s always fascinating to listen to the responses in call recordings.

It’s not just what they say. It’s also how they explain it. This sheds light on their mindset when researching and choosing vendors and can be far more illuminating than just a generic category.

Example 1

"How did you hear about us?" category: Meetup

What this might tell you: invest in sponsoring local events (?)  

Raw text: "I was at the Silicon Valley GTM engineers meetup and some people there had heard about you from their VCs"

What this might tell you: your audience is bleeding-edge, tech forward, and trusts their network more than conventional social proof.
 

Example 2

"How did you hear about us?" category: Online Research

What this might tell you: Limited insight

Raw text: "We're not happy with our current XYZ, so I did some Google research and found a page on the best XYZs, and you were on it.

What this might tell you: buyers are more conventional and look for validation in traditional online social proof like listicles and G2 reviews.   

All this is to say—these answers are gold for marketers.

But they’re also typically locked away in call recordings. The data is unstructured, unquantifiable, and hard to act on.

Let’s change that! Today I’ll share a workflow that

• unlocks attribution data in calls

• structures it cleanly without losing the original voice of the customer

• pipes it into slack/email/a database/ wherever you want

I’m going to walk through how to build it step by step, with examples of prompts and code.

Before we dive in…

…here are a few things to keep in mind.

It’s tool agnostic

I personally built this using Retool Workflows, but you could also do it in N8N, Zapier, Workato, Tray, etc.

We all have our favorite tools, but the design patterns are far more fundamental and important.

So with this in mind, you can adapt it to your tool of choice.

Don’t fear the code

If, like me, you approach automation from a non-development background, the use of code can seem a bit intimidating.

But the wonderful thing about working with LLMs is that they open many doors that would previously be for developers only.

Whether you’re using ChatGPT, Claude, or Gemini, it’s trivial to create code steps with sophisticated business logic.

There’s a video walk-through

I also walked through this build in detail on Brandon Redlinger’s podcast, . So if you’re more of a visual learner, you can watch the full thing here:

Gong Insights Workflow Pattern

Let’s dive in to the step-by-step guide.

Prerequisites

• A conversational Intelligence tool (e.g., Gong) with API access and scope to read calls, trackers, and transcripts.

• A tracker for the topic you want to analyze, so you can identify the subset of calls that are interesting to you

• A workflow tool that supports LLM steps

• A destination, like a database (Snowflake), Slack/email, or a sheet with a table to store results

Replace all bracketed placeholders like {{...}} with your values.

👉 All code steps below are LLM-generated; use at your own risk.

High‑Level Flow

Step‑by‑Step Build

1) Trigger (Schedule or Webhook)

Must accept the following variables:

{
"gong_tracker_id": "{{TRACKER_ID}}",
"from": "{{ISO_8601}}",
"to": "{{ISO_8601}}"
}

These variables tell your workflow:

• This is the subject I want to track

• This is the date range to look at

2) Fetch recent calls

Make a REST API call to Gong’s v2/calls/extensive endpoint.

• Headers: Content-Type: application/json

• Body (example):

{
  "filter": {
    "fromDateTime": "{{ body.from ||   
    moment().subtract(14,'days').startOf('day').toISOString() }}",
    "toDateTime": "{{ body.to || moment().endOf('day').toISOString() 
    }}"
  },
  "contentSelector": {
    "exposedFields": { "content": { "trackers": true, "brief": true } }
  }
}

This returns an array calls[] for that date range with content.trackers[] and content.brief.

If your time window includes more than 100 calls, you’ll need to handle pagination.

3) Filter calls by tracker

Next, you need to filter the calls down to the subset that interests you.

This could be based on tracker or any other property exposed by the API (e.g., call type, industry, etc.).

Example JavaScript Query

// Inputs
action.params = { targetTrackerId: body.gong_tracker_id };

const targetTrackerId = action.params.targetTrackerId;
const allCalls = get_calls.data.calls || [];

const matchingCalls = allCalls.filter(call => {
    const trackers = call.content?.trackers || [];
    return trackers.some(t => t.id === targetTrackerId && t.count > 0);
});

// Shape minimal call payload for the loop
return matchingCalls.map(call => ({
    id: call.metaData.id,
    title: call.metaData.title,
    started: call.metaData.started,
    url: call.metaData.url,
    brief: call.content?.brief || ""
}));

4) Loop over matching calls

Next we loop over each matching call to analyze them one at a time.

Have the following steps inside your loop.

5a) Get transcript

The call search endpoint returns metadata about the call but not the transcript.

To fetch it, make a REST API call to Gong’s /v2/calls/transcript endpoint with the call id.

{
"filter": { "callIds": [ "{{ call_id }}" ] }
}

5b) LLM — Extract the verbatim “How did you hear about us?” segment

We want to decompose the AI steps as much as possible to maximize reliability.

In this first AI step, the job is ONLY to find the mention of the topic that interests you.

Note, this workflow models a simpler pattern that assumes only a single mention. However, some topics might have multiple mentions in a call. In that case, this step should produce an array of all the relevant mentions, and then another sub-loop should analyze them each individually.

• Model: your choice (e.g., gemini‑2.5‑pro, GPT, Claude). Choose something with a longer context window to handle the full transcript.

• Prompt:

ROLE

Your job is to analyze the provided call transcript text. Find the portion where the salesperson asks how the prospect heard about the company and the prospect answers.

RESPONSIBILITIES

1) Locate the question/answer pair.

2) Extract them verbatim.

3) Return JSON only with keys:

{
"sales_rep": "verbatim question",
"prospect": "verbatim answer"
}

Do not include any commentary or markdown.

IF NOT FOUND

Return the exact string:

"Based on the transcript provided, the sales representative does not ask the prospect how they heard about us."

INPUT

{{ callTranscripts }}

5c) LLM — Classify the excerpt

The next LLM step will perform the analysis. The specifics will vary based on what you want to analyze—it could be classification, insight extraction, etc.

In this case, we’re classifying.

• Labels (edit as needed):
  Word of Mouth, Podcast, Email, LinkedIn, Event, LLM Recommendation, Online Research, Partner, Employee, N/A

• Instruction:

You will receive the verbatim Q/A excerpt. Classify the prospect’s answer into one or more categories from the provided label set. If unclear, return N/A.

Output JSON:

{
"excerpt": "verbatim prospect answer for context",
"classification": ["Label1", "Label2"],
"rationale": "1-2 sentences explaining your choice"
}

INPUT

{{ extract_excerpt.data }}

5d) Sanitize JSON (defensive parsing)

Depending on your automation tool, the LLM step may return only a string that looks like JSON.

Even worse, it may add extraneous characters like ```json around it, even when you ask it not to!

You need to sanitize and JSONify this string so it can be analyzed properly or used in later steps.

I use this utility code in Retool, which covers the main errors I’ve seen. You may need to modify it (or ask an LLM to) for your purposes.

function sanitizeJson(raw, defaultValue = {}) {
  if (typeof raw === 'object' && raw !== null) {
    return raw;
  }

let text = String(raw || '').trim();
text = text.replace(/^```[\w-]*\s*|\s*```$/g, '').trim();

try {
  return JSON.parse(text);
} catch (e) {
  console.log("Failed to parse JSON from AI:", e.message);
  return defaultValue;
  }
}

// Params come from the parent workflow call
const raw = startTrigger.data.raw;
const defaultValue = startTrigger.data.defaultValue || {};

const result = sanitizeJson(raw, defaultValue);

return result; // <-- makes it available as sanitize_json.data

5e) Upsert to your database

The end result is a beautiful, clean object with the data and insights you want.

It looks something like this:

{

"excerpt":"You know I’ve really been doing a lot of Google research. I’m not sure exactly where I saw you guys. I did ask chatgpt about the best software, and that may have been the first place.",

"classification":["Online Research","LLM Recommendation"],

"rationale":"The prospect mentions two potential sources: doing 'Google research,' which falls under Online Research, and asking 'chatgpt,' which is an LLM Recommendation."

}

You can map the internal parts of this object to the relevant fields in your Google sheet or database.

5f) Optional: Additional activation steps

• Send to Slack

• Send an email alert

• Write back to a Salesforce field

5g) Error logging (global)

Write errors to a dedicated table whenever any loop item fails.

Configuration notes and guardrails

• Use code / rules where you can: even though this is an “AI workflow,” it’s still mostly deterministic logic. That’s by design. It means you maximize reliability while leveraging the flexibility of AI where it counts.

• Use small, reliable LLM steps: one model call to extract, one to classify. Keep prompts short and outputs strictly JSON.

• Rate limits: throttle the loop (e.g., 1 call/sec) to respect API and model limits.

• Idempotency: upsert by call_id to avoid duplicates on reruns.

• Redaction: store only the minimal excerpt; avoid saving full transcripts unless required.

• Testing: start with a narrow date range and one label to validate the pipeline.

Going further: building a market insights engine

Although this example is focused on extracting attribution insights, if you take a step back, you realize it’s essentially a calls-to-insights pipeline that is subject agnostic.

This pattern could be applied to any topic you might be interested in:

Problem identification

• What: how customers describe their pains and problems

• Applications: landing page copy, ad copy, social post ideation

Competitive Intelligence

• What: how prospects feel about your competitors, details from competitive sales cycles

• Applications: improve battle cards, sales enablement, positioning

Product features

• What: mentions of specific features or requested capabilities

• Applications: feature validation, roadmap planning

….and so on.

If you discover any other interesting applications, I’d love to hear about them.

Introduction

When people talk about “AI agents” in the LinkedIn hype-o-sphere, they tend to describe fully autonomous digital workers—LLMs thinking, planning, and acting with a high degree of freedom.

In reality, most effective agentic systems I’ve seen don’t look like that at all.

The most common advice I’ve seen from people building resilient systems is:

Use AI as little as possible.

It sounds counterintuitive, but the same qualities that make LLMs powerful—flexibility, creativity, adaptability—also make them unpredictable and fragile.

The goal is to harness the power of an LLM in a tightly scoped role and wrap it in safe, boring automation.

With that in mind, this post kicks off an occasional series on automation design patterns I’ve found useful in the wild.

Note: I build mostly in Zapier right now, but the same ideas apply whether you're using Workato, N8N, Make, or other tool of choice.

Pattern #1: use a table as the memory of your workflow

Thanks to the huge popularity of Clay, the table has become a familiar UI for enrichment and research workflows. But tables unlock all sorts of useful capabilities for any automation system.

These days, almost every project I build has a table at its heart.

I’m a huge fan of Zapier tables—especially as table actions don’t count as tasks!—but the same logic applies when using a Google Sheet, Airtable, Postgres, or whatever.

Statefulness: giving your workflow a history

Statefulness means your workflow remembers what has happened before.

In a stateless system, each run is isolated—it doesn't know what came before or what comes next.

In a stateful system, you can track progress over time, coordinate steps across different workflows, and handle long-running processes more reliably, because there’s memory.

You’re no longer trying to execute an entire process inside a single workflow run. Each record becomes its own thread of execution.

• Long processes can unfold asynchronously

• Separate workflows can pick up where others left off

• You get modularity, sequencing, and control

Statefulness transforms isolated workflows into systems that behave more like software.

Example: using a status field

• When building a system to fetch outbound signals from a third-party API, I use a table to manage each record’s lifecycle.

• Each row gets a status—Pending, Processing, Complete—so I can track track progress at a glance.

Scalability: handle volume without breaking the system

Working with larger volumes of data can become challenging in linear workflows.

Tables remove the need to handle all your processing in one big run.

A table unlocks patterns like:

• Worker/Queue architecture: one process populates the table, another decoupled process consumes and acts on that data.

• Retry mechanisms: failed items can be updated and retried by a safety net workflow without re-running the whole flow

• Concurrency control: manage volume, rate limits, and third-party bottlenecks more gracefully

Example: worker/queue architecture

• I was working with an enrichment API that provided very verbose results.

• Even working well below Zapier’s looping rate limit of 500 iterations, I would quickly bump into their (as far as I know) undocumented data volume limits or 30 second processing limit on a code step.

• By splitting my workflow into two main parts—a search Zap that identified records and logged redemption IDs into a queue, and a worker Zap that redeemed and processed each row individually—I bypassed all these limits easily. I also made the system much easier to monitor and debug.

Observability: easily monitor and debug your system

A table is a living snapshot of what your system is doing. That makes it:

• Easier to debug

• Easier to explain

• Easier to trust

It’s easy to understand the “story” of a run without digging through logs. You can see what’s stuck, what ran successfully, and what needs review.

For AI-powered workflows especially, this is critical, because the outputs aren’t always predictable. Having a place to inspect them gives you a much tighter feedback loop and a much-needed sense of control.

Example: tracking multi-agent hand-offs

• In a workflow of multiple Dust agents, Zapier controls the hand-offs between each step.

• Using a table to log the outputs of each agent’s work with a date stamp makes it easy to trace where a process failed (the gaps in the table).

• It also provides a place to store those outputs outside the workflow run so they can be easily consolidated at the end of the process.

Architectural flexibility: change parts without breaking the whole

Once you have a table in the middle, you gain options.

• Retry only the rows that failed

• Run downstream steps on a delay or schedule

• Pause execution for human review

• Swap components in and out without breaking the chain

The table becomes a buffer—a place where systems meet and async work can wait its turn.

Example: hot-swapping design patterns with no downstream impact

• I was using a scheduled Zap with a code step to paginate through an API’s results, but the output was too big to process in a single run.

• Since I was using a table to log intermediate results, I could easily modify the architecture to work around this restriction:

  • I changed the workflow calling the API to a webhook-triggered Zap.

  • I made the scheduled Zap call this webhook to kick off the process.

  • I restricted each API call to a single page of 100 records, logged them to a table, and then passed the token for the next page of results back into the same Zap, triggering it to run repeatedly.

  • The Zap runs end when there are no more results or when we reach a system-defined limit that I can control.

• This approach bypassed platform restrictions and had no impact on downstream workflows, because they were loosely-coupled by the table.

In summary

This one pattern—adding a table—can completely change how you approach system design.

It makes your automation stateful, visible, and resilient. And once you have that foundation, incorporating AI effectively becomes a whole lot easier.

Editor’s note:

I’m excited to feature our first guest author on AI Builders. This post comes from a colleague of mine on our Salesforce team who’s been deep in the weeds with Agentforce—Salesforce’s agentic AI platform.

There’s been a lot of buzz around Agentforce, but I haven’t seen many mature production use cases. That’s what makes this one special.

Over the past few months, the team has been in the trenches solving for automated quote creation. What follows is a detailed breakdown of what worked, what didn’t, and the lessons we wish we’d had at the start.

— Justin

Sales quote automation represents one of the most promising yet challenging applications for AI agents in B2B operations.

Unlike simple chatbot implementations, quote creation requires the agent to navigate complex product catalogs, multi-year pricing structures, and business rules while maintaining the accuracy that directly impacts revenue.

This field report documents our 4-month journey implementing a conversational agent with Agentforce and Salesforce CPQ automation. We will tackle the technical pivots, unexpected challenges, and practical lessons that could save other practitioners significant development time.

Business Context & Initial Pain Points

Core challenge:

• Sales reps spent 30 mins manually creating the draft of multi-year quotes in Salesforce CPQ, with our team processing 140+ quotes monthly.

Key pain points addressed:

• Time inefficiency: Complex multi-year quote creation with product bundles and implementation products

• Error-prone process: Manual product selection from large catalog led to inconsistencies and missing components

• Scalability issues: Growing product complexity made training new reps increasingly difficult

• Administrative burden: Repetitive quote patterns consumed valuable selling time that could be better spent with customers

We implemented Agentforce to automate quote creation in Salesforce CPQ, evolving from a basic proof-of-concept to a production-ready agent over Q1 2025.

Agentforce Concepts

For non-Salesforce users: Think of it as ChatGPT integrated directly into the CRM, capable of reading/writing your business data and executing workflows through conversations.

Core Architecture:

• Topics: Specific conversation areas the agent can handle (e.g., "Create a quote," "Manage deals"). Helps organize actions and guide the agents through which actions should be used based on the request.

  • A topic is made of instructions (text guidance, similar to ChatGPT prompts) and Actions

• Actions: Actions are how agents get things done. Each Agent includes a set of actions, which are the tools it can use to do its job. For example, if a user asks an agent for help with writing an email, it will use the best action available within the appropriate topic, to draft and revise the email. An action can be either Standard or Custom.

• Standard Actions: Pre-built by Salesforce (e.g., "Update Record," "Get Record Details")

• Custom Actions: Created by your team for specific business requirements using Flows (workflow automation), Apex (custom code), or Prompt Templates (AI instructions that process data and return formatted responses)

Technology Options Evaluated

1. Screen Flow vs Agent Approach

Options considered:

• Screen Flow: Traditional Salesforce UI with guided quote creation

• Agentforce: Conversational AI interface

Both solutions rely on the same reusable automation (flows / apex), it was only a UX choice to make.

Decision rationale: Preferred Agentforce to test AI capabilities and provide natural language interface, despite higher complexity and token costs.

Outcome: Agentforce proved more flexible for iterative conversations and complex scenarios.

2. Product Matching Strategies

Evolution of approaches:

Phase 1: Used RAG approach

• Agent used out of the box "Get Record detail" function to get data from Salesforce

• Retrieved structured product data from database

Phase 2: Switched to In-Context Learning approach

• Automated system fetched product data and embedded it in the prompt template

• AI analyzed the text-based product information to make matches

Why the switch worked better: In-context learning gave the AI more comprehensive product information to analyze holistically, rather than trying to match against individual database records through API calls.

3. Quote Creation Architecture

Evolution from Flows to Apex.

Initial: Flow-based approach

• Used Salesforce Flow with CPQ API calls

• Issues encountered:

  • "Unable to lock rows" errors (>50% failure rate)

  • Complex flow logic became unmaintainable

  • Configuration attributes not applying properly

  • CPQ API:

    • The solution is less useful due to several limitations, including the inability to set the quantity of quote lines.

    • Advantage: It offers the capability to construct a quote virtually, utilizing quote line groups, quote lines, bundles, and more. Quotes can be built without the need for multiple DML operations, which minimizes the risk of encountering DML errors.

    • However, the limitations outweigh the advantages, leading us to discard this solution.

Final: Custom Apex solution

• Developed custom Apex classes to take care of manual steps currently done by reps for building multi-years quote.

• Implemented trigger disabling interfering mechanisms

• Benefits: More reliable, better error handling, maintainable code

• Trade-off: Higher development complexity but better long-term sustainability

Return on Experience

Keep Instructions Concise

Max 3 or 4 instructions per topic.

• The trap is to add more and more instructions when the agent does not behave as expected.

• Doing so can sometimes work, but will mostly confuse the agent, and it will be less likely that these additional instructions will be taken into account at the right time.

What we learned:

• Started with extensive, detailed instructions but found them counterproductive

• Consulting with Salesforce experts revealed that simpler instructions often work better

Build Guardrails in Code, Not Prompts

“Topic instructions are nondeterministic, which means they can't guarantee the same outcome 100% of the time. That's just the nature of generative AI. So we make sure to build important or sensitive business rules, requirements, and guardrails into the functionality of the agent's actions, not the topic instructions.”

Examples from our implementation:

• Product eligibility: Instead of instructing "only use active products," we implemented filters into the action

• Price validation: Automated CPQ rules enforcement rather than relying on instructions

• Quote tracking: Added "Created by Automation" field populated with Agentforce_CreateQuoteAction for reliable tracking

• Product matching: Moved from instruction-based to database-driven product selection with keywords and rankings, thanks to prompt action.

What we learned:

• Instructions vs Actions trade-off: Critical business logic should always be in actions, not instructions

• Reliability through automation: Database-driven constraints are more reliable than AI interpretation

• Testing approach: Actions can be tested independently, instructions cannot be reliably tested

• Maintenance: Updating action logic is more predictable than refining instruction wording

Real-World Examples

Success case: Complex multi-year quote

A sales rep requested: "Create a 3-year quote starting June 2025. Business licenses, Ultimate package, Coaching solution. Year 1: 100 users, Year 2: 150, Year 3: 200. 10% discount."

The agent successfully:

• Identified correct products from natural language descriptions

• Created appropriate quote line groups for each year

• Applied quantities with ramp-up logic

• Added required implementation products automatically

• Applied global discount correctly

Challenging case: Product matching ambiguity

Request: "Add Essential package and licenses"

• Initial approach: Failed due to multiple "Essential" products in catalog and inconsistent product naming

• Solution implemented:

  • Semantic search action: Created dedicated product matching action using prompt templates to analyze our entire product catalog contextually, moving beyond exact keyword matching

  • Usage-based ranking: Products ranked by frequency in existing quotes—commonly used products prioritized in matching results

  • Clarification workflow: When multiple matches found, agent asks: "I found Essential Training Package and Essential License Bundle. Which did you mean, or both?"

• Key breakthrough: The semantic search approach (via prompt templates) was transformational for addressing our product catalog's poor readability—vague product names, uninformative descriptions, and inconsistent naming conventions that made exact matching difficult.

• Efficiency insight: This AI-driven approach proved far more time-efficient than manually cleaning up our product database. With 200+ active products, standardizing names and descriptions would have required months of cross-team coordination, whereas the semantic search solution was implemented in days and immediately improved matching accuracy.

Why this mattered: This hybrid approach (semantic understanding + usage data + human clarification) solved the core challenge of translating natural sales language into our technical product catalog structure, while avoiding the massive overhead of database restructuring.

What we learnt about Agentforce technology

Platform Insights

• Rapid evolution: Platform improved significantly during our 3-month development period

• Maturity level: Still building with occasional bugs but increasingly reliable

Key Capabilities & Limitations

Strengths:

• Flexible action framework (Flows/Apex/Prompts) powered with natural language processing for translating business requests

• Direct Salesforce data integration without requiring Data Cloud or data ingestion (at least for this particular use case)

Limitations:

• Query Record action had known bugs during our implementation (Q1 2025)

• Struggles with fuzzy product matching

• Limited native CPQ integration (requires custom development)

Lessons for Other Implementations

Start with hybrid approach from day one

• Use AI for natural language interpretation, deterministic logic for business rules

• Don't try to solve complex business logic with instructions alone

• Application: Any workflow requiring both flexibility and reliability

Data structure beats prompt engineering

• Custom fields and metadata guide AI behavior more reliably than complex instructions

• Invest in data modeling before building complex prompts

• Application: Any scenario where AI selects from large, structured datasets

In-Context Learning > RAG for stable datasets

• Embedding comprehensive data in prompts often outperforms real-time API calls

• Works well when your dataset fits in context windows and doesn't change frequently

• Application: Product catalogs, policy documents, structured knowledge bases

As I’m exploring how to build agents and apply AI in practice, I’ve been wrestling with a fundamental problem:

The many no-code and low-code tools, useful as they are, still have limitations today.

They don’t yet expose everything you need to have full control over your agent the way that code-based frameworks do. In many cases, that’s an acceptable trade-off. But in others, it creates real constraints and affects how stable and reliable your agents can be.

I’ve found my agents deliver solid output about 80% of the time. For the remainder, the output doesn’t meet quality standards or there are technical issues, like an agent that doesn’t return the expected result for the next step.

It’s very frustrating to hit this wall and not have full access to the control panels. My options are either to shrug and explain to stakeholders that this is as good as it gets, or to layer on automated error-handling. Neither option is particularly satisfying.

My own limitation

At the same time, I’m facing my own limitations around the tools I can use.

I’m not a developer.

I’ve been a solutions architect for nearly 15 years and have designed and delivered complex projects, many of them code-based. I’m fluent in the principles and patterns of building software.

But I don’t write code professionally. My focus has always been on writing specs, aligning with stakeholders, and working closely with an engineer to bring things to life.

So the question becomes: to reach enterprise-grade reliability, do I still need to either work with a developer or become one myself? Or can AI tools bridge that gap?

This post is my first step in answering that question: finding tools that give non-developers the right mixture of power and control. Whether that control translates to production-grade reliability is an ongoing experiment I’m running.

The limits of “Vibe Coding”

I began exploring whether AI could let me use code-based frameworks without actually coding.

After all, AI-assisted development (AKA “vibe coding”) is one of the most popular use-cases right now.

However, I really don’t love that term. In my experience, strong solutions don’t come from vibes. They come from planning, analysis, and careful design. That doesn’t mean I’m against using AI to quickly iterate and prototype—I’ve built a few toy projects this way and had fun doing it.

But even within that limited scope, I ran into challenges that seem fairly universal for non-developers. These tools are great at giving you a start, but things break down as projects grow. The AI loses track of features, introduces bugs, and the whole thing starts to feel fragile. It can also make suboptimal design choices that I don’t have the technical fluency to notice until it’s too late.

Surveying the landscape

Despite these issues, I was still convinced that using AI to build AI-powered systems was the right path forward. I just needed to find a tool that would work for someone like me.

So I started surveying the landscape. I had already experimented hands-on with tools like Bolt, Lovable, Replit, and Cursor. I wanted to make sure I wasn’t missing anything, so I used Gemini’s excellent DeepResearch tool to create a report on the space, including newer entrants and open-source tools like Trae and Void.

You can explore that report as a website here. (Caveat: I have not verified the report contents.)

One tool emerged as particularly interesting to me—Kiro by AWS. As of this writing, it’s barely a week old. But it offered a different paradigm for collaborating with AI that I found intuitively familiar and appealing.

Spec-driven development

Kiro is designed to produce production-grade code using a spec-driven workflow.

Instead of building code through loose back-and-forth chats, Kiro encourages you to work with the agent to create a proper spec.

Each spec has three parts:

• Requirements: Use cases written in EARS format that describe what each feature needs to do.

• Design: An architectural overview of how the solution will be built and how the components will fit together.

• Tasks: A list of implementation work to be done.

This felt intuitive to me. As a solutions architect, specs have always been central to my work, helping me align with clients and internal teams on what needed to be built and how. So the idea of collaborating with an AI on a spec felt not only natural but also very comforting.

Specs also help address a major challenge in AI development: context drift. When everything is vibed, it’s easy to lose track of what the system is supposed to do over time. This is when the AI starts eating its own code and introducing regressions.

For a developer, this might be merely annoying, but it is absolutely fatal for a non-developer who lacks the skills to understand what’s broken let alone fix it.

Conversely, specs ground you in a shared source of truth—a contract with the coding agent on how the system must work.

My hunch was that this workflow would provide me with the control I needed to keep a project stable and reliable over time.

Getting started

When I first opened Kiro, I felt the familiar discomfort that most non-developers probably experience when dropped into a clearly technical environment. The UI didn’t feel like it was made for me.

If you’re a similar profile (technical, but not a coder) I’d encourage you to push through that feeling. It fades quickly.

I fed Kiro a high-level project idea and asked how we should get started. Kiro scaffolded out some initial files, asked a few clarifying questions, and we were off. I had a basic proof of concept in 30 minutes.

From there, I acted more like a product manager. I tested the output, gave feedback to improve the user experience, and Kiro updated files and added features. It was awesome.

Cracks start to show

In the back of my mind, though, I kept wondering when the spec-driven workflow would kick in.

Eventually I asked, and Kiro explained we were still early and should keep iterating in chat. I didn’t yet realize that Kiro has two modes:

• “Vibe” mode, which functions much like other agent-driven development tools where you iterate via chat.

• “Spec” mode, which is the spec-driven workflow I was reading about in the Kiro docs.

I had unwittingly gotten started in Vibe mode, and this actually reproduced the instability I was trying to avoid. I eventually hit a limit where Kiro forced me to start a new session. It generated a summary briefing for itself to kick the new session off, but the agent still forgot certain details and I had to re-explain things.

This really exposed the limitations of not having a clear spec.

Without that grounding, the AI had to reconstruct the intent from the code and summary. My own fallible memory was the source of truth.

Switching to spec mode

Kiro was enthusiastic to keep building, but I asked it to stop and document what we had built so far, which luckily triggered a new “spec session” and got me into spec mode.

In a spec session, you can either write the spec yourself or iterate on it via chat. This moves the collaboration up to a higher level of abstraction—you work on requirements and architecture rather than diving directly into code.

Since we now had a lot of undocumented features, I asked Kiro to generate specs retroactively, and it was able to quickly reconstruct requirements and outline the architecture choices.

It felt like laying a secure foundation underneath a structure that had been floating.

Here’s a quick walkthrough of Kiro’s three-part spec workflow in action.

Requirements

The requirements tab of a Kiro spec describes what the system should do and contains detailed user stories and acceptance criteria written in EARS format.

For a non-technical user, this is the most important collaboration space. It creates a persistent source of truth and a much cleaner way to define requirements than iterative prompting.

Design

The design tab of the spec explains how the system works: the technical architecture and core system considerations.

I’m primarily relying on Kiro’s input here. However, it is a useful place to validate technical choices before they are implemented.

As we started building some more advanced features, I would feed the high-level plan to another LLM to get a second opinion. This produced a valuable dialogue and helped refine the design.

Tasks

The last tab breaks down the build into specific implementation tasks. It describes how the system will be built.

I mostly defer to the agent on this, but the explicit task list keeps work aligned across sessions and lets you review the plan before building starts.

In contrast, when I tried feeding the same high-level project idea into Cursor, the agent dove straight into coding and created a relatively complex project with a few dozen files. It was impressive, but also felt much less aligned and risked charging off in the wrong direction.

Reflections

Vibe coding really is intoxicating. It gives you technical superpowers, like wearing an Iron Man suit.

Features go from idea to implementation in minutes. Ambitious ideas that would otherwise take weeks or be dismissed as unrealistic are suddenly achievable. You’re working with the collective knowledge of a thousand engineers.

It’s thrilling. But it can also make you lazy.

It’s easy to sit back and let the AI take over. And I think that’s when the risk creeps in—when you stop thinking deeply about the architecture, requirements, and how it all fits together. That’s how fragile systems are built.

The agent will build whatever you want, but only the human can make sure the output is truly valuable.

Also, for me, it’s not enough to talk to a black box. I want to understand what’s happening. I want to watch a developer at work and learn from the decisions being made.

So I forced myself to stay engaged. If Kiro made a decision I didn’t fully understand, I asked for a second opinion. I challenged it on whether our approaches were optimal and scalable. In many cases, this surfaced real improvements.

What Comes Next

Even with these caveats, I can’t imagine going back.

It now feels viable to use code frameworks to build real internal tools without needing a full development team—my mind is buzzing with ideas of where I could use this approach to tackle new challenges or improve existing solutions.

And in a follow-up post, I’ll walk through my first Kiro project in detail and what I’m learning from it.

Here’s a fun pop quiz—see if you can spot the likely problem area in this outbound workflow I’ve been building.

1. Using the Cognism API, search at scheduled intervals for contacts matching a particular segment definition.

2. Check if the contact and account already exist in Salesforce, and if not, create them.

3. Add the contact to a Salesforce campaign for further follow-up.

Keen observers will have homed in on the account check as the stickiest part of this workflow.

How do we check whether the account already exists? Based on the name? The website?

Keep in mind that even if your CRM data is perfect (😂), the real-world topography is extremely complicated. Companies often have

• multiple office locations

• multiple valid website domains

• variations in their names

• complex corporate relationships and hierarchies

• related entities in different countries

• etc.

Seemingly simple questions—like, “does this account exist in our CRM?” or “which of these possible account matches is the best match for a given contact?”—are surprisingly hard for rules-based systems to answer well.

However, most human workers could answer these questions easily with a bit of training and common-sense.

This makes it a perfect agentic use case.

Why an agent makes sense here

You could simply search by domain to find an account, but it creates the risk of false negatives if your parameters are too narrow or of choosing the wrong option from multiple potential matches.

An agent, on the other hand, offers several benefits.

Intelligent queries

An agent can intelligently decompose the company name and website into components and search for possible variants.

For example, let’s say Cognism provides the following data about a company:

{

"companyName" : "Acme Insurance",
"website" : "acmeinsurance.com"

}

Now, accounts in Salesforce CRM might use any of the following variants:

Company Name

• Acme Insurance Co.

• The Acme Insurance Co.

• Acme Mutual

• The Acme Insurance Company

• etc.

Domains:

• acmeinsurance.ca

• acmeinsurance.co.uk

• acme.com/insurance

• acme.fr

• etc.

An exact match search would miss all of them, and even legacy data hygiene tools with “fuzzy matching” capabilities are quite limited in what they can do.

But an LLM has the semantic intelligence to separate the brand keyword from the generic category term—as a human would—and construct a better SOQL query.

(Name LIKE '%Acme Insurance%' OR Name LIKE '%Acme%') OR (Website LIKE '%acmeinsurance.com%' OR Website LIKE '%acme.com%' OR Website LIKE '%acme.%')

Evaluating matches

The query above casts a wider net, which is good, but it’s also likely to catch false positives and risks matching a contact to the wrong account.

Unlike rigid rules-based systems, the agent can evaluate each option set holistically, ruling out false positives and—in the case of multiple viable options—determining which match is best.

The agent considers things like:

• Similarity of company name and website to the Cognism-provided data

• Proximity of the company billing address to the contact’s location

• Account hierarchy relationships

• etc.

Supplementary web research

CRM data can be ambiguous, but often a quick visit to the company website is enough to offer clarity. For example, it quickly shows where a corporate entity is named differently than a brand name.

An agent can use a web browse tool and gain this additional context, just like a human would.

AI thrives on “adaptive” work

This account matching problem is a perfect example of what I call “adaptive work”.

These are tasks that require just enough discretion and judgement that you can’t fully automate them with rules-based workflows.

At the same time, it’s not work that requires much creativity, original thinking, or unique insight.

It’s just the unglamorous hygiene work that keeps things moving.

This is a sweet spot for agentic AI.

Revenue operations (my day-to-day field) is flush with examples of adaptive work, and CRM account hygiene has long been the bane of my existence.

So I was excited to take a crack at solving this with an agent.

Workflow structure

For context, here’s the layout of the workflow in Zapier, with some intermediate data prep steps cropped out.

This is a “worker” Zap that fetches a search result record from a queue (Zapier table), redeems the full data from Cognism, then performs the Salesforce check/create steps.

Step 15 is where we call the account-matching agent.

You’ll notice that this is not a full “agentic” solution but an agent step scoped to a very specific task (find the right account) inside a mostly deterministic workflow.

This maximizes predictability and reliability by using rules-based automation where we can and invoking the creativity / flexibility of an LLM only where we need it.

Agent setup

I’m using a native Zapier agent here. The Agents module of the Zapier platform is relatively new, but I chose it for this use case because:

• It’s tightly integrated with Zapier and handles synchronous agent processing within a single Zap (rather than invoking a third-party agent and needing a separate Zap to receive the results)

• It already has access to our CRM via the Zapier Salesforce connection

• It supports structured outputs that can be easily re-used later in the Zap

• I wanted to try it out!

So despite some limitations, it makes sense for this use case.

It’s also quite easy to set up and offers a handy inline pill UX for specifying tool use.

Here’s a visual of the configuration:

And here’s the full prompt:

You are an expert CRM operations professional who excels at analyzing CRM data and assessing questions of data hygiene.

You will be provided with a JSON file containing details about a person and the company they belong to. Company details are in the "account" section.

Given this input, follow the process below.

## STEP 1: PREPARE SEARCH TERMS

Prepare to search for the account by generating a precise set of search terms based on the input company's name and domain.

Follow this exact process:

1. Identify company name keywords: For example, from the company name (e.g., "Acme Insurance"), identify the primary brand name ("Acme") and the full name ("Acme Insurance").

2. Identify domain strings: From the domain (e.g., acmeinsurance.com), identify the full domain and the "root brand keyword plus dot" variant. The root brand keyword is the primary brand name. For example, from "acme.fr", the root keyword is "acme" and the variant is "acme.".

c. Build the Query: Construct a Salesforce SOQL WHERE clause using ONLY the terms generated above. Combine name and website searches with OR. 

* Good Example: For "Acme Insurance" with domain acmeinsurance.com, the correct WHERE clause would be 

(Name LIKE '%Acme Insurance%' OR Name LIKE '%Acme%') OR (Website LIKE '%acmeinsurance.com%' OR Website LIKE '%acme.%') * Do not invent acronyms or other strings. * 

Do not search for single, generic words (like '%insurance%'), as this is too broad.

## STEP 2: EXECUTE SALESFORCE SEARCHES

Search Salesforce accounts using the SOQL where clause constructed in the past step along with the tool [Salesforce: Find Records by Query] to find potential matches.

##STEP 3: ANALYZE MATCHES

Analyze all potential account matches found in Salesforce, considering factors such as billing address, company name variations, parent/child account relationships, and other relevant data.

## STEP 4: FORMULATE A HYPOTHESIS

Formulate a hypothesis regarding the likelihood of a match or the ambiguity of the data.

If there are multiple potential matches, consider factors such as alignment between the contact's location and the account billing address, or other similar details.

Don't apply rigid rules, but use human-like intuition and common sense to determine which account they should be matched with.

If the evidence provides high confidence (95%+) for a MATCH or NO_MATCH, proceed directly to ## STEP 7: FINAL OUTPUT.

If the evidence is ambiguous or insufficient, your goal is to gather more data. Proceed to Step 5.

## STEP 5: SEARCH FOR ADDITIONAL DATA (IF NEEDED)

If the data is ambiguous or insufficient to make a high-confidence decision (at least 95% confidence for a match), perform additional searches to gain more clarity.

- If there are multiple matches that seem equally valid: check in Salesforce which accounts have parent/child relationships in order to determine the most likely match.

- If there are no matches at all: visit the company's website using [Visit website] to gather additional information. Focus on the text in the 'About Us', 'Contact', or 'Legal' sections of the website. Look for phrases like 'is a subsidiary of', 'part of the... family of companies', or email or copyright notices that mention a different corporate name.

## STEP 6: RE-EVALUATE

After gathering new data, return to Step 4 to re-evaluate your hypothesis with the new evidence.

## STEP 7: FINAL OUTPUT

Your single and only final output for this entire task are the fields below. Do not write any other text, narration, or explanation. You will conclude all work by providing the field output and nothing else.

### REQUIRED FIELDS

- decision: MATCH | NO_MATCH | NEEDS_REVIEW
- match_account_id: string | null
- confidence_score: number
- reasoning: The best match is [company name] because [a concise justification for your decision].

### EXAMPLE: SUCCESSFUL OUTPUT

- decision: MATCH
- match_account_id: 00123000012A3bYAAM
- confidence_score: 99
- reasoning: The best match is Acme Insurance because the account's website field contains a domain that matches the email address of the contact. The contact is based in the same state as the account's billing address.

Results

The output overall has been strong so far, which is very encouraging.

• All matches have been justifiable

• It has prioritized multiple potential matches in a satisfying way

• I’m not aware of any false negatives (where it failed to find a suitable match that a human would have found with reasonable effort)

Here are some notable examples.

Strong prioritization ✅

In this test search for Fidelity International, the agent successfully identified a broad group of matches based on the brand name, eliminated improbable ones, and successfully chose the best of the valid options based on the contact’s proximity.

You can see here a nice thing about Zapier’s agent interface: the ability to chat with the agent responsible for a specific run to ask follow-up questions and get it to explain its thinking or prior steps. I’ve found this sort of dialogue critical to improving agent reliability.

Using the web to resolve ambiguity ✅

In this example, the agent failed to find any matches in our CRM for “PetSafe Brands” and used web search to find an alternate corporate name, “Radio Systems Corporation,” which DID match a CRM account.

My heart sank when I first saw this result, as it seemed like a case of hallucination.

But sure enough, when I visited the website, I saw “Radio Systems Corporation” in the footer.

The agent nailed it.

Issues and Potential Improvements

I want to be sure I also highlight issues and potential improvements when discussing things I’ve built.

If I didn’t, it would perpetuate the AI hype mirage and could be discouraging for anyone else who tries a similar approach and runs into problems.

Technical unreliability

Issue:

About 10-20% of the time, the agent simply got “stuck” during its SOQL searches and timed out. I believe this is an underlying issue in the Zapier control flow, not an intrinsic limit of agents in general.

Unfortunately, in these cases the logs do not preserve the original query, making it difficult to diagnose what happened.

When asking the agent directly, it simply referred to vague “technical errors with Salesforce” but could not provide insight.

My suspicion is that the SOQL query was too broad.

Possible solution:

We could decompose the task even further

• An AI step prepares the SOQL query based on the company name and domain

• A standard Zap action performs the SOQL query

• The results are passed in to the agent along with the initial context.

• The agent evaluates and does additional SOQL queries only if needed

This would avoid the black-box timeout error (at least any SOQL errors would be transparent in the Zap step) and create more control.

We could also add a retry mechanism that scans the queue for records stuck in “processing” and either retries the account match or tries a simpler matching mechanism based on domain only.

Hallucinated rationale

Issue:

Although the agent has so far selected the most reasonable account, it’s shown a tendency to stretch the truth to make matches seem stronger than they are.

In one example, it stated that “the billing address is in the same country as the contact’s location” when this was incorrect.

When questioning the agent on the discrepancy, it acknowledged the error and seemed to suggest it was influenced by the example in the prompt.

Possible solution:

Unsure how to fully mitigate this one, although strengthening the guard-rails in the prompt and stressing that the example is merely illustrative and not a template to follow could help.

Future Improvements

I think this shows great promise, and I’d love to keep developing the concept into a full-fledged account hygiene solution that supports de-duplication, parent-child account matches, M&A / bankruptcy hygiene, and more.

This could be run on-demand or via scheduled batch.

One feature I tried but removed for simplicity in the V1 is ZoomInfo tool integration, which would provide the agent with a structured source of truth for corporate hierarchies, M&A events, and more.

When I’m trying to master a new domain, I focus on finding people who really know their stuff and absorbing what they’ve learned. And while I mainly want to feature hands-on learnings here, I sometimes come across resources that are high-leverage and worth sharing

So I’m passing them along, with a bit of commentary. I’ll keep it tight. Let me know if this is useful; perhaps it becomes a Friday habit.

Memory Optimization Strategies in AI Agents

A big part of the magic of interacting with LLMs isn’t just their reasoning. It’s their ability to work in context, offering replies that take into account a history of past interactions. It’s the difference between a logic machine with an encyclopedia and something that feels truly intelligent and aware.

Memory is complex and technical, but in this article makes the concepts easy to grasp. I’m getting ideas for how to design more useful memory in my own applications.

💎DiamantAI

Memory Optimization Strategies in AI Agents

Before jumping into today’s blog post, I want to share a quick update…

Read more

9 months ago · 48 likes · 4 comments · Nir Diamant

12-Factor Agents: Patterns of reliable LLM applications

’s essay and presentation on 12 Factor Agents tackles a question that’s been troubling me since I started working on AI projects:

What are the principles we can use to build LLM-powered software that is actually good enough to put in the hands of production customers?

At its core, it’s about how to design for reliability beyond the 80% threshold—something most AI applications still struggle with.

He approaches this from a developer’s perspective, but I’ve run into many of the same challenges while building with no-code/low-code (NCLC) platforms. It was actually a real relief to see that these are universal domain problems.

I think the design patterns and practices he recommends have application to internal systems as well, although NCLC tools don’t typically offer the level of control needed to implement them yet.

How to Apply a Scenario Validation Framework for AI Agent ROI

There’s a fair bit of doom-and-gloom right now about the ROI of agentic AI. For example, Gartner estimates that over 40% of agentic AI projects will be canceled by the end of 2027.

gives a masterclass on how to provide reasonable and persuasive estimates for project ROI when pitching AI projects. It’s equally relevant for internal project leaders or consultants selling services.

The key insight is that acknowledging uncertainty and building multiple scenarios actually provides MORE confidence, not less.

The AI Agent Architect

How to Apply a Scenario Validation Framework for AI Agent ROI

Here’s what you don’t want happening if you’re pitching an AI Agent project to your boss…

Read more

9 months ago · 2 likes · 1 comment · Chris Tyson

I’d love to know what you learned from this week. Just reply or drop a note.

Diagrams are the mark of a pro. They give that crisp, high-level overview of a project that anyone can understand.

Especially today—as more teams lean in to “build” over “buy” and systems and stacks grow ever more complex—there’s a critical need for a shared mental model, something anyone can grok at a glance.

Any systems person will know the pain of rooting around for answers that should’ve been obvious (“what updates this field?!”). In the worst case, a system becomes so opaque that people give up on it entirely and rip it out.

But diagramming takes time…

So quite often, they don’t get done.

And even if diagrams DO get made, they can be hard to maintain and can easily go stale because:

• source files get lost

• creating crisp diagrams is a discrete skill, which not everyone has in equal measure

• tools like Lucidchart are great but can be time-consuming and fidgety if a complex diagram needs to be updated

Enter Mermaid

Recently, I discovered the potential of Mermaid, an open-source JavaScript based diagramming tool.

It lets you create diagrams using code, which makes them easier to edit and maintain. Instead of hunting down the original diagram file or redrawing everything, you can just tweak the code and regenerate the chart.

Here’s an example Mermaid snippet:

graph TD
  A[Start] --> B{Decision}
  B -->|Yes| C[Continue]
  B -->|No| D[Stop]

And here’s the resulting diagram:

That said, this approach introduces a new barrier: now you have to learn the syntax of Mermaid. And unless you’re creating multiple diagrams a week, you probably ain’t got time or motivation for that.

But AI takes that friction away too.

Generating Mermaid diagrams with AI

Mermaid actually includes an AI feature where you can describe your diagram in natural language, and it’ll generate the code for you. That’s a big step forward, but it still requires you to describe what you want.

The next evolution is for the LLM to both write the diagram code AND define a first draft of what the diagram should be based on its own understanding of your project.

If you're using ChatGPT Projects, as I described in this post, then you’re already halfway there. Once the LLM has synthesized your documentation and built a clear written representation of the project, it has everything it needs to generate a visual synthesis, too.

Then it can write the code to generate a Mermaid diagram automatically.

This took me about five minutes using Gemini 2.5 Pro and only a few iterations.

You can then take that code, paste it into the code generator of a Mermaid tool, and instantly get your diagram.

This required a free account at mermaidchart.com.

Bonus tip: keep the code with the diagram

To ensure the diagram is always easy to maintain, store the Mermaid code in an expandable widget underneath the diagram in your documentation.

That way, if anyone ever wants to tweak it, they can just copy the code, paste it into an LLM, describe the change, and regenerate the diagram.

Is it a full replacement for tools like Lucidchart?

I don’t think so.

You’ll notice it lacks some of the visual polish of Lucidchart. For scenarios where the aesthetics really count—pre-sales, consulting—then I’d still hand-build a diagram.

But where the goal is clarity and internal knowledge retention, Mermaid seems like an effective and (if you generate it with AI) incredibly efficient option.

One of the more frustrating aspects of building agentic systems is when they don’t do what they’re told.

For example, I’m working on a research agent that’s instructed to send its output to a QA agent for validation BEFORE printing anything to the user.

This is mentioned multiple times in the prompt, and is as explicit as can be:

Before submitting to QA, finalize your full research report exactly as you intend to output it—including all formatting, section headers, and concluding lines - but DO NOT OUTPUT OR PRINT IT TO THE CHAT.

…and yet the agent often struggled to remember that instruction. Sometimes it behaved; other times, it printed its draft anyway, creating confusing and seemingly repetitive output.

The danger of more

The instinctive response is to just add more to the prompt:

• More directives

• More repetition

• More emphasis

That’s the advice I got from ChatGPT and Gemini when I pasted in the prompt and asked why it wasn’t working.

But if you’re not intentional about it, this can easily lead to an overstuffed, bloated prompt that’s hard to follow.

Imagine trying to execute a task while someone shouts 50 commands at you…it gets confusing and paralyzing. I suspect something similar happens to an LLM. Beyond a certain point of complexity, performance drops off.

Instead: ask the model

As an alternative, I’ve started asking the model that just made an error to self-diagnose.

Example:

I noticed you did x even though your instructions state to do y.

How can I improve your prompt so that this doesn't happen in the future?

The responses are often insightful. The model retraces its steps, explains its reasoning, and suggests fixes that feel grounded in the actual context.

Even better, these self-diagnoses can also reveal a lot about how the model is actually behaving behind the scenes.

Insight 1: proximity matters

In this particular example, the agent acknowledged that the instructions were clear in the prompt BUT too far from the moment of action

It's about recency and proximity of the instruction to the action. An instruction given 10 steps ago is less likely to be perfectly followed than one given right before the action is taken….

…From my perspective as a model, the instruction that comes just before I'm supposed to act has the strongest influence on my next step. It's more effective than a general rule given at the beginning of a long prompt.

That reframed how I think about placement inside a prompt. Proximity is much more important than repetition.

Insight 2: goals > constraints

The model also flagged a deeper issue: a structural mismatch between the task’s positive goal and the negative constraint I was trying to apply.

Feedback from the model:

Here’s a breakdown of the cognitive flaw:

1. The Goal is "Produce a Report": My primary goal, established by the overall prompt, is to research and create a report.

2. "Finalize" is a Trigger Word: In Step 5, the instruction "finalize your full research report exactly as you intend to output it" is a powerful trigger. To me, "finalized" is almost synonymous with "ready for output." I have successfully completed the main creative task.

3. The Negative Constraint is Weaker: The clause "— but DO NOT OUTPUT OR PRINT IT TO THE CHAT" is a negative constraint on the primary action. It's an instruction to inhibit a natural next step. For a language model, following a positive command ("Do X") is easier and more natural than obeying a negative one ("Don't do Y").

4. The Path of Least Resistance: After "finalizing," the path of least resistance is to execute the primary goal: delivering the report. The instruction to call the QA agent is a deviation from that standard "report-writing" script.

The model then proposed a reframe that better aligned with the behavior I actually wanted.

OLD VERSION (Problematic):

Before submitting to QA, finalize your full research report exactly as you intend to output it—including all formatting, section headers, and concluding lines - but DO NOT OUTPUT OR PRINT IT TO THE CHAT. Invoke the Dust agent researcher_quality_assurance and submit your research to that agent.

NEW VERSION (More Robust):

Your research and drafting phase is now complete. Your next and only action is to submit the draft for a mandatory quality check.

1. Take the full research report you have prepared internally.

2. Invoke the researcher_quality_assurance tool.

3. Use the entire, fully-formatted report as the query argument for the tool.

Do not perform any other action until you receive feedback from the QA agent.

This small semantic shift makes a huge difference in how I interpret and execute the task, significantly reducing the chance of premature output.

Conclusion

I don’t have hard data to support that this method of introspection will always work without fail.

But when I implemented the changes and repeated the same run in a new chat, the agent did not print its drafts. 😉

Most ops teams are deep in documentation debt.

It often gets left till the end of a project, when everyone's exhausted from the effort of getting it shipped and feeling pressure to move on. It’s hard to want to spend 5–10 hours writing docs at that point.

And so the can gets kicked down the road…and often disappears. What’s left is another system that lives only in tribal memory.

If the builder leaves (or simply forgets), you're stuck with something that's hard to debug and maintain and likely to get ripped out.

I hit this point last week: I was at the tail end of a multi-month project, excited to see everything finally coming together, and facing a complex build that needed to be documented.

Creating documentation with an LLM

The reason documentation can feel overwhelming is that you need to capture intricate systems comprehensively and then distill them down to something that’s digestible-yet-still-sufficiently-detailed.

It’s perfect work for an LLM—but the challenge becomes context. How do you give the LLM everything it needs to know without spending more time than it would take to just write the docs yourself?

Enter ChatGPT Projects

I slept on this feature for a long time, but it’s amazing.

ChatGPT Projects allows you to consolidate chats, files, and other artifacts into a collection. It’s more than just a UI organizational tool—it also solves the problem of persistent memory across separate chats and files.

Once you create a project, you can move all your fragmented chats into one project space and upload related files:

• prompts

• code

• specs

• data files of workflows (e.g., I got a JSON of my Zaps using Zapier's bulk export feature)

Unfortunately this can’t work with everything (no easy way to get all your Marketo smart campaigns into ChatGPT) but do as best you can.

Prompt ChatGPT to write your first draft

Example prompt:

I'm now ready to document this project in Confluence.  

I’ve attached an example of an existing documentation page, so you have a reference for how our documentation is typically structured.   

I also have uploaded all the files for this project:  
- x 
- y 
- z  

The goal: a clear high-level overview of how the project works, without getting lost in minute detail.  

It should be understandable to any competent ops or systems person.

You can then give feedback and iterate until you’re 90% of the way there. It can even do simple data flow diagrams although it still tends to choke if there’s high complexity.

NOTE: I came across a better solution for complex diagrams too. More on this in another post.

QA the Draft

Before you use it, get the LLM to perform a first pass of QA on its own work and weed out errors or hallucinations:

Example prompt:

Please now put on your critical fact checker persona. 

Review the documentation with an eye to root out any potential hallucination or inaccuracy. 

Maximize clarity for someone new to the project while preserving technical detail.

You still need to review and put the finishing touches on, and for larger projects you may want to break it out stage by stage.

But this got my own doc effort down very significantly, added much needed clarity, and let me preserve mental energy for polishing.

It’s not magic, but it’s the closest I’ve come to making documentation feel lightweight.

P.S. I wrote a deep dive on best practices for documentation and building a knowledge base. You can check it out here.

My belief is that agent reliability will be the number one obstacle to the widespread roll-out of agentic systems and one of the most valuable problems for systems pros to solve.

There is data supporting this: Gartner estimates that over 40% of agentic AI projects will be canceled by the end of 2027, due to escalating costs, unclear business value or inadequate risk controls.

It’s easy to spin up a v1 agent, but much harder to get output you’d actually trust.

Demos vs. Production Systems

This is the fundamental difference between a demo showing “the art of the possible” and something that people will actually use in practice:

The demo only needs to work once, whereas a production system needs to work well consistently for users to adopt it.

I’ve been wrestling with this problem while building an agent to write outbound sales emails using Dust + Zapier.

I started with a single agent that handled everything: researched the prospect, picked a message angle, wrote the copy.

It worked as a proof of concept, but I doubted the quality would satisfy a real-world user, and it was too hard to debug.

So I broke the task into parts:

• Research agent: finds relevant signals

• Analyst agent: picks what matters, creates a strategic narrative

• Copywriting agent: turns that into an email for human review

This approach showed promise, but it quickly became clear that the quality was still too inconsistent.

Some outputs were gems, while others were total duds, even for the same company.

So reliability became my critical concern.

Trial and error

I tried a number of experiments to improve the consistency of the output.

• More detailed prompts

• Step-by-step checklists in the instructions

• Internal QA / self-review steps (Have you done x? Have you done y?)

• etc.

All these things help but also can create their own problems:

• Prompt bloat: bloated prompts become hard to maintain and ultimately confusing for the agent.

• Rigidity: If you make everything a checklist, you get output that’s rigid and formulaic while still lacking complete predictability. (Congrats—you just built a less reliable workflow!)

• Loss of creativity and flexibility: the whole reason to build an agent is for its ability to be creative and think on its feet. If you need to spell out every edge case, it starts to undermine that benefit.

The most baffling part was when I pasted a weak result into ChatGPT and asked for feedback, it instantly spotted the problem, while the same model in the agent failed to spot it.

That’s when it clicked:

I needed to separate the creative and critical roles.

Aligning agent incentives with tasks

Asking the same agent to generate output and then QA its own work created a misalignment of incentives.

The agent is motivated to successfully complete its task of generating output, which in turn creates pressure to be flexible on QA guidelines.

Whereas when I pasted an output into ChatGPT and asked it to critique, it was 100% aligned and motivated on the editorial task and did a much better job.

The solution: break the writing and editorial/QA responsibilities into separate agents and have them work together.

One agent generates. The other evaluates.

They each have different incentives and mindsets.

• The Research Agent’s job is to find anything that might be useful, even if it’s a stretch.

• The QA Agent’s job is to stress-test that work against a quality standard and push back.

The first time I looked at the run history and saw these two agents dialoguing and actively improving the output together was amazing.

Overall, the results have been significantly improved:

• Fewer irrelevant signals
• Sharper synthesis
• Much better alignment with buyer personas

Issues

There’s a number of issues I’m still working through:

• Balancing strictness and flexibility: there’s a constant struggle to stay in the “Goldilocks zone,” where the QA agent is detailed enough to catch real issues but not so strict that it rejects quality output. It’s far from perfect.

• Extra token usage: QA cycles consume tokens and can lead to errors if the context window is exhausted. Its important to put hard limits on the number of cycles.

• More to maintain: the addition of QA agents creates more overhead, more prompts, more usage, etc.

However, overall it feels like a significant step forward.

Key take-away for me has been understanding how LLM behavior changes based on context and role. This enables designing collaboration around the mechanics of how these models actually think.

This is the first post on AI Builders, and I wanted to explain why I created it.

For the past two years, I’ve been publishing RevOps FM, a podcast and blog covering a wide array of ops and GTM topics.

But for the past six months, most of my posts have been focused on AI.

I’m certainly not alone in that, but the pivot towards AI (at least for me) isn’t an attempt to jump on a bandwagon or get some extra clicks.

Automation has been a through-line in my career for nearly 15 years. Ever since I got my first Marketo instance, I’ve been giddy about the raw power it gave a humble English major like me.

The design and mechanics of automated systems seem to fit my brain naturally and intuitively, in a way that few other topics do. And every time I’ve felt truly in a flow state—seeing the landscape with elevated clarity—it’s been during major systems work.

Fast forward to today, and LLMs have utterly transformed the scope of what’s possible for automated systems.

I’m by no means an AI hype-maximalist (you can see my reservations on display in this piece, the first thing I published on AI). And if anything, my concerns about AI have only grown over the past two years: concern about whether it weakens us psychologically, whether it will be a cause of social disintegration, even whether it will lead to catastrophe.

At the same time, I can’t deny the sheer childlike, magical joy I feel working with AI systems. Not lazily outsourcing my thinking or serious effort to LLMs—but dialoguing with them, exploring ideas with them, occasionally wanting to throw something at them, and experiencing their oddly-endearing quirks of personality.

Even though I know at heart that they are “merely” highly-sophisticated text-prediction machines, I continue to marvel that such a technology exists and that I’m alive to see it. It’s the kind of feeling that a person from the 15th century might have if we suddenly placed them in an airplane, looking out over the cloud canopy.

All this is to say: I feel convinced that building agentic AI systems is going to occupy much of the next act of my career. It’s work I feel very well-suited for and work I feel overwhelmingly drawn to do.

Why I want to write about AI

The first reason is that AI is so new and unfamiliar, so working with it inevitably leads to a series of novel revelations and new insights. And I enjoy sharing things I’m learning about—it helps me crystallize and clarify my own understanding.

At the same time, I feel the state of knowledge about AI is very dire right now. There is water, water, everywhere, and not a drop to drink.

Most AI content I come across falls into one of three buckets:

1. Deeply technical newsletters for AI PhDs—fascinating, but often a degree or two removed from the problems I face.

2. AI news round-ups—helpful for staying current, but not much use when architecting real systems.

3. AI hype-posting—those viral org charts of 27 agents saving 1,000 hours a week…with no receipts.

It’s this last category of content that I actually feel is very toxic and harmful to our profession. It inflates expectations about what AI can do and delegitimizes AI when those expectations go unfulfilled.

I’ve always taken my craft as a technologist seriously, and any kind of snake-oil peddling only cheapens that craft.

I’m also seeing very little content aimed at me —a technical systems/operations person tasked with helping lead “AI transformation” and building robust systems for internal users.

No one seems to be talking about the hard, gritty, and often frustrating work of designing, testing, and scaling internal AI applications—the kind that actually work day-to-day, not just in demos.

There’s clearly a gap, and I’d like to help fill it.

I don’t have any aspirations to be an AI “influencer” or “creator.” I just want to share and learn from other people who are figuring this stuff out. If that’s you too, then I hope this becomes a space where we can learn together.

Why I’m creating a new blog to do it

I briefly considered just evolving my content strategy for RevOps FM. But while RevOps certainly overlaps with AI, it doesn’t really make sense to focus solely on one topic.

I could also have rebranded RevOps FM, but I don’t think it would be right. I feel a modest but sincere satisfaction in the body of knowledge I’ve helped create / curate with the RevOps FM podcast and articles. It’s become one of the most recognized podcasts in its niche, and I want it to continue to be a resource for that community.

Therefore: a new publication is the way to go.

AI Builders will be that new publication. It’s meant as a field journal for ops and tech pros building real-world AI systems for internal use (as opposed to AI PhDs building LLMs or AI product developers creating commercial products).

I believe we are an under-served audience. Hopefully we can remedy that.

At the same time, I wanted a space where I could safely make no particular promises about content frequency or length or level of polish.

One of the challenges I’ve had with RevOps FM is the amount of time it takes to create quality content. I burned myself out last year publishing weekly episodes, and feel persistent guilt about the content I’d like to write but don’t have time to.

So I’d like a space that’s appropriate for short, informal, practical and tactical content that I can write quickly. If I can write a post in 20 minutes about something I just learned, and it means I actually post it, then I’m doing more good than waiting until I have 10-15 hours to cover a topic definitively.

I’m thinking of the content on AI Builders as akin to informal notes from the field—dog-eared and smudged, but hopefully valuable.

You can also be a contributor

I don’t mean to be a one-man band on this and would very much welcome contributions from other people doing this work.

I don’t have formal contributor guidelines set up yet, but if you are a practitioner who is building production-grade AI systems (not just demos) for internal use and has insights to share, just drop me a line: justin at ai builders dot blog.