If you’re piloting in 30–90 days… you’re likely balancing a live network, stretched teams, and a short clock to prove value without breaking service. The messy parts missing fields, angry emails, carrier exceptions—show up fast, and it looks like work, not failure.

Decision criteria

1) integration depth

What it means in real ops: The tool can connect to your TMS/WMS/ERP, carrier portals, EDI/API feeds, and email/portal workflows in a way that supports daily execution, not just one-time imports. You should be able to point to specific objects (shipment, stop, charge, doc, appointment) and see how they sync.

How it fails: “We integrate” means a CSV upload and a fragile daily job that breaks the moment a field changes.

2) exception handling

What it means in real ops: When the network deviates—late pickup, appointment missed, capacity short, doc mismatch—the system routes the exception to the right queue with context and next actions. It should support triage, escalation, and closure codes.

How it fails: Everything becomes a generic “ticket” with no operational path to resolution.

3) auditability

What it means in real ops: You can reconstruct who did what, when, and why for a shipment, charge, or compliance decision, including overrides and deleted/edited data. Logs should be exportable and retained per your policy.

How it fails: You can’t prove why a decision was made or who approved it when Finance or a customer asks.

4) controls

What it means in real ops: Role-based access, approvals, threshold rules, and separation of duties are built into workflows (e.g., who can release a tender, approve an accessorial, or change consignee info). Controls should align to your SOPs.

How it fails: Anyone can override anything, and you learn after a costly mistake.

5) ownership

What it means in real ops: Named business owners exist for configuration, exception queues, master data, and KPI definitions, and they can make changes without waiting weeks. Ownership is clear across Ops, Finance, Customer Service, and IT.

How it fails: No one owns the workflow, so issues bounce around and the pilot stalls.

6) data lineage

What it means in real ops: For any output (rating result, invoice flag, ETA update), you can trace back to the source inputs (EDI segment, portal event, email field, manual entry) and see transformations. This is essential for dispute handling.

How it fails: Outputs look “smart” but can’t be explained or reconciled to source data.

7) SLA support and escalation paths

What it means in real ops: The vendor can commit to response times for critical outages and operational blockers, with an escalation ladder that reaches engineers when needed. You should know what “P1” means and how it is handled.

How it fails: Support is email-only with unclear priorities, and outages linger during peak windows.

8) evidence capture

What it means in real ops: The system attaches and organizes documents and artifacts (BOL, POD, lumper receipt, appointment confirmations, rate con, photos) at the shipment/stop/charge level with searchable metadata.

How it fails: Evidence stays in inboxes and shared drives, so disputes become rework.

9) role-based approvals

What it means in real ops: You can route approvals by lane, customer, dollar threshold, commodity, or accessorial type, with delegation for PTO and after-hours rules. Approvals should be fast, visible, and reversible.

How it fails: Approvals happen in chat/email, leaving no consistent record.

10) configurability without breaking change

What it means in real ops: You can adjust rules, fields, and workflows with versioning and rollback so you can test changes before pushing to live operations. Changes should have effective dates.

How it fails: Every change requires a vendor ticket, and releases unexpectedly alter production behavior.

What to ask vendors

Security & compliance

1) Show me where user permissions are configured and how role-based access is enforced for tendering, approvals, and financial adjustments.

2) What data is encrypted at rest and in transit, and where are keys managed?

3) Where is customer data stored (regions), and how do you handle data residency requirements?

4) What happens when an employee leaves—how fast can access be revoked, and is there an audit log of access changes?

5) Show me your incident response workflow: who is notified, within what timeframe, and what artifacts are provided to customers.

6) How do you support retention and deletion policies for shipment documents and logs, including legal holds?

Workflow ownership & change management

1) Who on my team can change a rule or workflow without vendor involvement, and what permissions are required?

2) Show me versioning: where do I see what changed, who changed it, and how to roll back.

3) What’s your release process—how often do you deploy, and how do you prevent breaking production workflows?

4) How do you support a pilot-to-scale transition: what configurations are one-off versus reusable templates?

5) What’s the minimum set of owners you expect on our side (Ops, Finance, IT), and what decisions do they each own?

6) If we need to pause a workflow during a peak window, who can do it and how quickly?

Exception handling

1) Show me an exception queue: how are items prioritized, assigned, and escalated when SLAs are at risk?

2) What happens when required data is missing—where is it flagged, and what are the allowed next actions?

3) Who can override an automated recommendation, and how is the reason captured?

4) How do you prevent duplicate work across teams when the same exception appears in multiple systems?

5) Show me how you handle conflicting inputs (EDI says delivered, POD not received) and how the system prompts resolution.

6) What happens after hours: how do alerts route, and what is the fallback if no one responds?

Reporting & audit logs

1) Show me where every approval, override, and field edit is logged for a single shipment, including timestamps and user IDs.

2) Can I export audit logs and event histories on demand, and in what format?

3) How do you report on exception aging, rework volume, and root-cause categories by customer/carrier/lane?

4) Show me how you reconcile financial outcomes: accessorials approved vs denied, disputes opened vs closed, and reasons.

5) What happens when a report number changes due to late data—how is the change recorded and explained?

6) Who can edit KPI definitions, and how do you prevent “metric drift” over a quarter?

Demo script (text-only)

Run these 10 scenarios live

1) missing info

2) conflicting docs

3) after-hours quote

4) accessorial approval

5) tender rejection recovery

6) Duplicate shipment creation: two tenders issued for the same load—show detection, prevention, and cleanup steps.

7) Appointment reschedule: consignee moves the window—show how updates propagate, who is notified, and what gets logged.

8) POD late: delivery event posted but POD not received—show follow-up workflow, evidence capture, and aging metrics.

9) Carrier no-show: pickup missed—show escalation, alternate carrier tendering, and how service impact is recorded.

10) Invoice dispute evidence: billed detention without proof—show how evidence is requested, attached, approved/denied, and audited.

Red flags (AI-washing)

• vague answers
• no audit trail
• “works best with perfect data”
• Can’t run scenarios live during the demo; insists on slides only
• Black box decisions with no explainability or evidence capture
• No role-based controls for approvals and overrides
• No rollback or environment separation (test vs production)
• Can’t name an owner for integrations, exception queues, or KPI definitions
• Over-reliance on manual exports to “make it work”
• Support model unclear: no defined SLAs, no escalation path to engineering
• Data model is inflexible; every new field is a paid professional services effort
• Dismisses edge cases as “rare” instead of showing handling paths

Simple scoring rubric

Use a 1–5 scale for each decision criterion above.

1 = Not proven: the vendor describes it, but cannot show it live or it depends on custom work with unclear timelines.

3 = Partially proven: the vendor can demonstrate core flows, but gaps exist (limited logging, manual steps, narrow integrations, or weak exception routing).

5 = Operationally proven: the vendor shows it live end-to-end, including logs, permissions, overrides, and exportable reporting that matches real execution.

Instruction: score each criterion, then total.

Weighting guidance (adjustable): if you’re moving money (rating, accessorials, invoicing), consider giving extra weight to auditability, controls, and evidence capture; if you’re protecting service, consider extra weight on exception handling, escalation paths, and integration depth.

If a vendor can’t pass scenarios, they’ll fail in week 2.
Book Demo to know more