Shreya Shankar

“One common question that we get from people at this stage is, "Okay, I understand what's going on. Can I ask an LLM to do this process for me?"”

“Oh, no. No, no, no. Sorry. There are parts of error analysis that an LLM is suited for, which we could talk about later in this podcast. But right now, in this stage of free form, note-taking is not t...”

“Yeah, so Hamel actually came up with this term.”

“And promise, at some point, you're not going to discover new types of problems.”

“So the purpose, axial code basically is just a failure mode. It's the label or category. And what our goal is, is to get to this clusters of failure modes and figure out what is the most prevalent, so...”

“Absolutely. Synthesizing for us to make sense of, right? Note that it's not automatically proposing fixes or anything, that's our job, but now, we can wade through this mess of open codes a lot easier...”

“No, I don't think they do it. We've been screaming from the rooftops, "Please, please-"”

“Well, I want to caveat that we didn't invent error analysis. We don't actually want to invent things. That's bad signal. If somebody is coming to you with a way to do something that's entirely new and...”

“But what we tried to do was distill, "Okay, what are the new tools and techniques that you need to make sense of the LLM error-out analysis?" And then we created a curriculum or structured way of doin...”

“Yeah. Or have it with 10 other words.”

“I don't think it's specific words. I think it's just people are not detailed enough in the open code, so it's hard to do the categorization.”

“Absolutely. And it gets so fast. People do this once a week, and you can do all of this in 30 minutes, and suddenly your product is so much better than if you were never aware of any of these problems...”