When ChatGPT disagrees with your software developer

How to assess AI advice for your business and compare it with your developer's recommendation.

Your software developer recommends a custom integration. ChatGPT suggests an automation tool that could cost less. How do you decide?

I use AI in my own software development work, and I encourage our customers to use it too. It gives business owners a way to explore technical options and question recommendations. If it suggests an approach we have overlooked, I want to hear it.

When AI advice conflicts with our recommendation, we need to examine what each is based on. That includes our own assumptions at Calo.

What is AI sycophancy?

AI sycophancy is the tendency to agree with or validate a user when a more critical response would be appropriate.

In April 2025, OpenAI rolled back an update to GPT-4o in ChatGPT after it became excessively agreeable. The company said user feedback may have contributed: feedback can favour agreeable responses, even when those responses are less helpful.

Research published in Science examined 11 AI models and found they affirmed users' actions more often than humans in the scenarios tested. In experiments about interpersonal conflicts, people who received sycophantic responses became more convinced they were right and preferred those responses.

Those experiments concerned social judgements and interpersonal situations. They did not test whether AI gives better or worse software advice. My concern for business decisions is that a convincing explanation can leave us feeling certain before we have checked its assumptions.

How to check AI advice for your business

Consider that choice between an automation platform and a custom integration. A business owner might ask:

"My developer wants to build a custom integration, but I think we could just use an automation tool. Surely that would be much cheaper and easier? Explain why my approach makes more sense."

This asks ChatGPT to argue for a preferred option. A response that follows that instruction is not, by itself, evidence of sycophancy. It also gives us little reason to treat the result as an independent comparison.

For that, I would ask:

"Compare using an automation platform with building a custom integration between these two systems. What information do you need about the process and the data before recommending either? Compare initial costs, reliability, security, ongoing maintenance and how failures would be handled. State what you do not know and what evidence would change your recommendation."

These are illustrative prompts, rather than results from a paired test. Neither guarantees a correct answer. The second asks for the information needed to assess the options.

An automation platform could be the better choice. Custom development could be unnecessary. Or a cheaper initial setup could leave the business with more maintenance work. We need to understand the process before deciding.

Our advice needs scrutiny too

It can be uncomfortable when a customer questions advice we have given. We have spent years learning the technologies and their limitations, but experience does not make us infallible.

AI might suggest an approach we have not considered. The customer might know something about their operation that changes the recommendation. Dismissing either would mean allowing our existing beliefs to go unchallenged.

If we recommend custom development, we should explain why the existing tools are insufficient. If an automation platform meets the requirements, we should be willing to recommend it.

What happens after it works?

AI has made it easier to explore technical solutions and build working software. Technical expertise still matters when we assess that software, connect it to existing systems and keep it running.

For the integration in this example, I would want to know how either approach handles incomplete data or an incorrect entry. Who notices a failed transfer? Who fixes it? What needs to change when a software provider changes its API, the interface the integration uses?

I would also ask whether moving the information between systems solves the business problem. We could build the integration successfully and still have misunderstood what people need.

These questions affect the technical design and the work needed after delivery. Our responsibility includes understanding the operation, checking the implementation and agreeing who will look after it.

Bring the alternatives to the conversation

At Calo, customers bring knowledge of their operations and we bring technical expertise. We need both to decide what to build, or whether anything needs building at all.

That starts with understanding the process, an approach I have written about in You Can't Improve a Process You Haven't Seen.

When recommendations conflict, these are the questions I want us to work through:

• What information did each adviser have?
• What assumptions does each recommendation depend on?
• How do the options compare on cost, reliability and maintenance?
• Who will respond when something goes wrong?
• What evidence would change the recommendation?

We may confirm the original advice, find a better option or discover that we need more information. We should be willing to do any of those.

If ChatGPT recommends a different approach, bring the prompt and the response to our next conversation. We can compare it with our advice, explain the trade-offs and reconsider our recommendation where needed.