So You Want to Send Me AI-Generated Work
I've been sending lots of individual responses to LLM-generated emails and analyses, and thought it might make sense to publicly share my policy for this type of correspondence. This is a living document and may be updated from time to time.
I uncovered something really wild which I think you should write about! Can I share my AI-generated analysis?
Probably not.
I have noticed that these requests typically come with voluminous documents with more data than I can reasonably review, let alone assess for accuracy. I am lost in the sauce, as they say.
Sifting truth from non-truth takes real time. It requires me to read through and analyze dozens of pages of text, many of which are riddled with errors and incorrect claims that I have to catch one at a time.
When I find errors, it makes me doubt the validity of all the other claims. If you send me 10 conclusions you co-created with Codex and nine of them end up being suspect, this is a very different experience for me than if you send me one real conclusion that I can fact-check and verify. When I start finding untruths, it's really hard not to lose interest. Beyond being annoyed at the tremendous waste of time, I start to wonder why I should believe anything you have to say.
But I've reviewed the output and vouch for everything!
Except many times you haven't, at least not as thoroughly as you think you have. This is why I keep finding glaring errors and plausible explanations for anomalies you think you've found.
I'm disturbed by how dismissive people are when I come across these. "Oopsie" will not rebuild shattered trust. You used AI, and it warned you that "AI can make mistakes!" We all know that AI makes mistakes and that someone is supposed to check important information. That requires more than a cursory glance.
Please check carefully and thoroughly before sending me anything. What I want to see is a footnote with evidence for each of your conclusions, written in a way I can easily parse and understand. Yes, this can take hours. (I'm estimating 8–20 hours for each Codex writeup.)
Lastly, I've found that somewhere in the process of co-creating an analysis with Claude or Codex or whatever, people become increasingly attached to their conclusions and reject other plausible explanations. Output tailored to your unique set of prompts is engaging and agreeable. The same material will look very different when applying more rigor, whether that's using a model or not. I need to know that you can approach the topic objectively, like a scientist. That means some level of detachment from your hypothesis, and the willingness to abandon the conclusion you came in with if the facts don't add up.
Will you respond to my LLM-written pitch for a service?
Probably not. It's usually written (or generated) in a very odd way, and it's almost always pitching a service I don't need because you didn't even do any research.
Can I share LLM output on something we're discussing?
I do not take as hard of a line on this as, say, Cory Doctorow. I appreciate that you have spent months or years discussing the topic with an LLM and will politely skim the output. Having said that, I can absolutely guarantee you that I'm more interested in what you have to say.
Will you answer career or industry questions I generated using an LLM?
I'm happy to explore your questions, or those of your chatbot, in a paid consulting session. I can send my rates over if you're interested. (I do love to chat with students or folks new to the industry when I have time, and I don't charge for that, but please do me the courtesy of writing your questions yourself.)
I vibe-coded a BADBOOL app. Will you give me feedback and promote it?
I created BADBOOL (the Big-Ass Data Broker Opt-Out List) to make it easier for people to manually remove their information from data broker sites and then check to see whether it had actually been removed. It is for people who are choosing to do this on hard mode because it's more effective. They want to be as thorough as possible, even if it takes more time, and they don't want to put the rest of their data at risk.
Sometimes people don't want to spend the time. Another option discussed in the list itself is to pay for a service, such as EasyOptOuts (only $20/year), to do it for you. They could also, theoretically, vibe-code something themselves.
Unfortunately, I simply don't have the time to beta test the dozens of variations of these apps I get emails about each week. (Sadly I don't always even have the time to beta test or give feedback to apps I'm incredibly impressed with that solve a problem I have.) I'm also unable to take on unpaid consulting with my suggestions for improvements, and I'm unlikely to recommend an app someone threw together in an afternoon. Broadly, I would recommend running extensive security testing and also efficacy checks and user experience testing, none of which require my feedback or support.