Documenting with AI (part 1)

Using ChatGPT projects to create comprehensive, contextually-aware system overview docs

Most ops teams are deep in documentation debt.

It often gets left till the end of a project, when everyone's exhausted from the effort of getting it shipped and feeling pressure to move on. It’s hard to want to spend 5–10 hours writing docs at that point.

And so the can gets kicked down the road…and often disappears. What’s left is another system that lives only in tribal memory.

If the builder leaves (or simply forgets), you're stuck with something that's hard to debug and maintain and likely to get ripped out.

I hit this point last week: I was at the tail end of a multi-month project, excited to see everything finally coming together, and facing a complex build that needed to be documented.

Creating documentation with an LLM

The reason documentation can feel overwhelming is that you need to capture intricate systems comprehensively and then distill them down to something that’s digestible-yet-still-sufficiently-detailed.

It’s perfect work for an LLM—but the challenge becomes context. How do you give the LLM everything it needs to know without spending more time than it would take to just write the docs yourself?

Enter ChatGPT Projects

I slept on this feature for a long time, but it’s amazing.

ChatGPT Projects allows you to consolidate chats, files, and other artifacts into a collection. It’s more than just a UI organizational tool—it also solves the problem of persistent memory across separate chats and files.

Once you create a project, you can move all your fragmented chats into one project space and upload related files:

• prompts

• code

• specs

• data files of workflows (e.g., I got a JSON of my Zaps using Zapier's bulk export feature)

Example project for something I’ve been working on

Example files attached to the project

Unfortunately this can’t work with everything (no easy way to get all your Marketo smart campaigns into ChatGPT) but do as best you can.

Prompt ChatGPT to write your first draft

Example prompt:

I'm now ready to document this project in Confluence.  

I’ve attached an example of an existing documentation page, so you have a reference for how our documentation is typically structured.   

I also have uploaded all the files for this project:  
- x 
- y 
- z  

The goal: a clear high-level overview of how the project works, without getting lost in minute detail.  

It should be understandable to any competent ops or systems person.

You can then give feedback and iterate until you’re 90% of the way there. It can even do simple data flow diagrams although it still tends to choke if there’s high complexity.

NOTE: I came across a better solution for complex diagrams too. More on this in another post.

QA the Draft

Before you use it, get the LLM to perform a first pass of QA on its own work and weed out errors or hallucinations:

Example prompt:

Please now put on your critical fact checker persona. 

Review the documentation with an eye to root out any potential hallucination or inaccuracy. 

Maximize clarity for someone new to the project while preserving technical detail.

You still need to review and put the finishing touches on, and for larger projects you may want to break it out stage by stage.

But this got my own doc effort down very significantly, added much needed clarity, and let me preserve mental energy for polishing.

It’s not magic, but it’s the closest I’ve come to making documentation feel lightweight.

---

P.S. I wrote a deep dive on best practices for documentation and building a knowledge base. You can check it out here.