AI-Driven Software Has Changed How We Buy Software

AI-Driven Software Has Changed How We Make Technology Buying Decisions

Talking with business leaders always creates one of those unexpected aha moments I appreciate. Erin shared her recent experience evaluating a new AI product. It promised to deliver what her company needed, but she was hesitant to commit. Even asking careful questions like the ones I mentioned in last week’s email didn’t give her the reassurance she needed. 

Erin's pause made sense, but the reason wasn't obvious to her. AI-driven software has changed the way we make our buying decisions. The questions we’ve always asked still matter. We just need a second layer of scrutiny that has never existed before.

What's Changed Isn't Obvious

These new AI products and services are built differently than the software we’ve been using for the last few decades.

That difference is invisible to business leaders, and that’s structurally okay. Leaders don’t need to become technical experts to take advantage of AI-driven products. But understanding how this technology significantly impacts our business does mean asking different questions.

Difference 1: We’re not buying software. We're buying someone’s judgment.

Traditional software is programmed by people to do specific things. The rules are clearly defined, and the software is built to deliver the same reliable outcome every time. 

AI-driven products, on the other hand, are based on what they’ve learned to do.

People who built these large language models decide what to teach it. They select the data, they define the training rules, and they decide what guardrails (if any) are built into the model. We can’t see what these decisions are or how they’re made, but our results are determined by them.

What this means for us: We’re not evaluating how something works. We’re trusting unknown people to decide how the system works for us.

One thing you can ask: Will you walk me through a situation where your product got it wrong and what happened next?

Listen for: An honest answer, not an evasive AI tools don’t guarantee accuracy every time answer. We already know that.

Difference 2: Most AI products rely on tools and technologies we didn’t choose.

Traditional software consists of technical layers that the company owns, understands, maintains, and controls.

AI-driven solutions are very different. They are a collection of third-party tools and services built on top of models owned by a few large companies like OpenAI, Anthropic, and Google. 

When these companies change their models — which happens often —, their pricing, their terms of use, the company you’re doing business with reacts to these changes. Your costs fluctuate. Your results shift because the underlying learning isn’t the same. 

One thing you can ask: If your primary AI provider changed significantly in the next 6 months, what would that mean for us?

Listen for: Companies can’t predict what changes will happen, but this unknown must be baked into their overall product design and strategy. The answer should be realistic and straightforward. 

Difference 3: The people behind the product creation matter more than they do with traditional software.

AI has made rapid product building possible. You’ve probably read articles and posts about vibe coding. This is the practice of creating products with AI assistance. People who never built a piece of software, who lack technical discipline, now feel empowered to build and launch.

But creation speed does not equal technical expertise, competence, and scalability. Building something that works right now isn’t the same as designing a technically solid product that will stand up to day-to-day use. 

Traditional software took longer to build for a reason. There was discipline and structure in the process. The final product was carefully tested in a real world setting. Ongoing support and future enhancements were planned for in the early pre-development stage. Reliability was valued over speed. 

One thing you can ask: Who in your company designed the core product architecture and are they still there?  Who is responsible for supporting and maintaining this product? This is the difference between a shiny demo and a product that won't collapse under the weight of daily use.

Listen for: Signs that this is a real company with competent staff, not a couple of people who vibe coded their way through a weekend.

The Bottom Line

None of this means you shouldn’t use AI tools and products. They aren’t inherently bad or wrong. They’re simply different than what we’ve long understood, and the way we make decisions about their use has to adapt. 

We've just begun to settle down, brush aside the endless hype, and discover where the AI value lives for our companies. Along with this shift (that we didn't ask for) comes the time for fresh thinking.

You don’t have to become an AI expert. It does mean coming to the conversation with different questions.  In next week’s email, I’ll share some thoughts on how these new systems fail differently and why that matters.