The five questions: what's live today on someone else's freight, what it does when it can't finish, how much of your team it consumes, how the automation number is defined, and what record it leaves behind.
1. What's live today, on someone else's freight?
Ask for a production load count, a customer count and the date each was measured. Then ask the vendor to open a live production load on the call. A vendor who can only demo on test data has never had a driver send a POD from the wrong load.
Separate "shipped" from "shipped long enough to trust." At Chain, tracking has five years in production and the booking agent launched in June 2026. Both are live. Those are different claims, and a good vendor will make the distinction for you.
2. What does it do when it can't finish?
The most under-asked question in the category. Failure behavior is the product.
- Does it escalate, or does it guess?
- Who gets told, and can that person act at 11 PM?
- Does it stop messaging the driver once it escalates?
- Is the escalation written into the load's record?
- Can it decide to say nothing at all?
A system that always produces an answer will eventually produce a wrong one to a carrier. Ask to see a message the agent deliberately didn't send.
3. How much of your team does it consume?
How many hours of your people's time between signature and the first automated load? Who does the field mapping? Are there implementation or integration fees? When does billing start? If you want a new rule in month four, is that a setting or a ticket?
Pilots are fine; we run them. What's worth skepticism is a months-long bespoke build described as a partnership. A forward-deployed engineer is a cost, not a feature.
4. How is the automation number defined?
Ask what counts as automated, what's in the denominator, what the spread looks like across customers, and what reporting you'll actually get. A percentage without a denominator is decoration. Here's what a good answer looks like:
| Level | Requires | Denominator |
|---|---|---|
| L1 | All carrier communication handled by AI, no broker messages | Chat-eligible loads |
| L2 | L1 plus tracking evidence across key milestones | Every load in the cohort |
| L3 | L2 on every stop of a multi-stop load | Every load in the cohort |
5. What's left behind when it's done?
After the load delivers, where does the conversation live? One thread, or four systems? Does the arrival time land in your TMS on its own? Do documents land where your billing desk expects them? Claims surface months later, and reconstructing communications from four systems is the expensive part.
The checklist
| Ask them | A good answer | Walk away if |
|---|---|---|
| Production loads, with a date | A specific number and when it was measured | "Millions," no date |
| Show me a live production load | They open one on the call | Only recorded demos |
| What happens when it can't finish | A named escalation path, and it stops talking | "It handles everything" |
| Hours of my team's time to go live | A number, and who does the mapping | "We'll scope it together" |
| Define your automation rate | Definition, denominator and range | One number, no denominator |
| Where the record lives | One readable thread per load | "It's all in the platform" |
Ask us all five on the call. We'll answer them in order, including the ones where the answer is "not yet."