Enterprise Intelligence Group

What you should be asking your SaaS provider for (it isn't a chatbot)

Before buying an AI add-on, test how it handles your business context, permissions and exceptions. Use these eight questions to structure a vendor evaluation.

Before buying an AI add-on, establish whether it supports the decisions and workflows you need. If work spans several systems, ask the vendor to demonstrate the required integrations, permissions and failure handling. An embedded assistant and a connected agent architecture should each be evaluated against those requirements.

The rest of this article is about the second part of that evaluation: where the business's judgment ends up, and how to find out before the purchase order moves.

When the vendors you already pay start selling AI

Look at the last twelve months of release notes from the software that runs your business: the planning suite, the ERP, the category-management tool, the pricing engine, the workforce system, the CRM. Where a vendor has added an assistant, or has put the word "agent" on a roadmap slide, check what that feature can reach beyond the vendor's own module.

If each vendor's assistant is adopted separately, a retailer can end up with several assistants, each fluent in one module, each holding a different sliver of the company's judgment, none able to see the others. Whether that happens depends on how the assistants are connected and governed, which is something to establish before buying rather than after. Nine is used below as an illustration, not a forecast.

Run separately and ungoverned, that is not one AI strategy; it is several vendors' AI strategies in parallel.

The problem isn't the vendors. It's where the judgment ends up.

What a vendor's assistant can use is the information and the tools available to it under the permissions you configure: its own module's data and workflows, plus whatever context you supply. Ask how your decision rules, your exceptions and information from other systems reach it, and test that it applies them correctly. Examples of context that has to arrive from outside the module: why your planner overrode the forecast in week 40, which allocation rule your category manager quietly suspends for a promotion, whose lead times slip before a holiday, which substitutions your shoppers accept and which only your buyer would, and which supplier's deal terms make the "wrong" order the right one.

The exceptions, the trade-offs, the history of what was tried and reversed: that knowledge is the intelligence layer of a retail business. It is the thing that makes a decision commercially sound rather than merely consistent. Record that knowledge explicitly rather than assuming an assistant already has it. If each vendor's assistant captures its own slice of it, that layer can become fragmented across products you do not own. Whether you can get it back out depends on what each vendor demonstrates and contracts for: a usable export, clear ownership and change control.

If you plan a shared intelligence layer, test whether its architecture, contracts and export formats provide the ownership and change control you require. Compare that evidence with the controls available across your existing products.

What "open access" actually buys you

The alternative is not "build everything yourself". It is a different shape of buying.

General-purpose reasoning models from the major AI platform providers can, in principle, be pointed at your business through an agent layer you own; whether a given model and a given vendor integration actually support that is something to verify in evaluation, not assume. Confirm what would need to change in your systems of record, integrations and permissions before selecting that architecture.

Four things can follow from that shape. Each is a capability to verify with the vendor, not a guarantee:

One intelligence layer, not nine. The decision rules, exceptions and guardrails your operators hold get captured once, in a form you own, and applied wherever a decision is made: merchandising, planning, supply chain, store operations. Cross-functional coordination can then be designed into the system rather than left to meetings, where the agents and integrations involved genuinely support it.

Strategy that actually reaches the decisions. When the executive team sets a direction, say protect margin in this category or prioritise availability in that region, a company-owned layer gives you one place to express it and apply it to every agent that honours that layer. Check how each participating assistant receives those rules, whether it can apply them, and how conflicts are handled.

Model choice. When a better reasoning model becomes available, adopting it at the layer you own is possible only if the architecture supports model replacement, and only after representative-case testing against your own decisions and guardrails.

Less walks out with the vendor. Captured judgment, decision logs, the audit trail of what the agent decided and why: these should be yours, exportable and inspectable, and the contract and the export format are where you verify that. If you change a planning tool later, the intelligence you built should not leave with it.

The eight questions

Which brings me to the title. The next time a vendor you already pay demos their AI, these are the questions I would want answered before a purchase order moves. None of them is hostile.

• Can my agent call your system? Not "do you have an assistant". Do you expose an API, a CLI, an MCP connection, with read and write, under permissions I control?
• Does your AI know my business, or only your module? Can it take my rules, my exceptions and my other systems as context from outside your product, or is its world your database?
• Where does the judgment live? If I have captured how my best category manager decides, can I bring that logic to your agent? Or is the reasoning locked inside your model where I cannot see or change it?
• Who owns the decision log? Every recommendation, every action, every override: is it mine, exportable, complete enough to explain a decision to a regulator or a new starter six months later?
• What leaves with you if I leave? Prompts, captured knowledge, tuned behaviour, history. Say it in writing.
• Which model, and can I choose? Whose reasoning engine sits underneath, how quickly do you adopt a better one, and can I bring my own?
• How does your agent coordinate with the agents from my other vendors? If you use multiple agents, whose guardrails win when two agents disagree about the same shelf?
• What is the handoff rule? When does your agent stop and hand back to a person? Can I set that line, or have you set it for me?

Treat incomplete or unclear answers as unresolved requirements. Decide whether to proceed only after the relevant capabilities, limits, costs and controls have been demonstrated on your own data and on a representative case.

The eight questions are also available as a one-page guide, 8 questions for your SaaS provider before you buy their AI, in the EIG Library.

Monday's move

You do not need a platform decision to start. You need one decision.

Pick a single consequential, repeating call in your business: a replenishment rule, a markdown cadence, a promotional allocation. Sit with the person who makes it and write down how they actually decide: the evidence they use, the exceptions they honour, the things they would never do regardless of what the number says. That page is a concrete evaluation input: it states what any assistant or agent, embedded or connected, would need to know and be allowed to do for that decision. It is not proof that one architecture is superior.

Then take that page and the eight questions into your next vendor conversation, and make it specific and testable: ask the vendor to show how their AI would handle that decision, its exceptions and its permissions.

A focused starting point

Define what your business needs before comparing software. EIG supports business requirements analysis and documentation, RFI/RFP preparation and vendor evaluation for retail, CPG and distribution teams. Scope, participants, permitted sources and deliverables are agreed before the work begins.

Explore vendor and technology selection.

Justyn Windsor is the founder of Enterprise Intelligence Group, which provides retail, CPG and distribution consulting across business decisions, technology selection and delivery. AI transformation is a specialist practice, not a requirement for engaging EIG.

The EIG Library  ·  Insights  ·  AI Maturity Assessment  ·  Contact