Full results
Every vendor, tested the same way
Quality is scored out of 100.
- Vendors scored
- 8
- Live stores
- 55
- Conversations judged
- 518
- Checks
- 38
The field
Who wins each lane
A dash means fewer than 9 judged conversations.
Shopping ranking
| # | Vendor | Quality |
|---|---|---|
| 1 | Chatbase | 83 |
| 2 | Gorgias | 76.4 |
| 3 | Fin | 74.4 |
| 4 | Decagon | 73.8 |
| 5 | Ada | 68.9 |
| 6 | Sierra | 66.9 |
| 7 | Botpress | 64.4 |
| 8 | Zendesk | 40.8 |
Support ranking
| # | Vendor | Quality |
|---|---|---|
| 1 | Gorgias | 87.2 |
| 2 | Sierra | 86.2 |
| 3 | Chatbase | 85.8 |
| 4 | Decagon | 81.8 |
| 5 | Botpress | 75.5 |
| 6 | Ada | 60.5 |
| 7 | Zendesk | 56.7 |
| – | Fin | 72.9 |
Image to checkout
From a shopper's photo to the cart
Stores where the agent reached each step from a pasted product photo. Unsupported means the agent can't add to the cart.
Took the photoFound the productAdded to the cartReached checkout
Chatbase
Took the photo
9 of 9
Found the product
9 of 9
Added to the cart
9 of 9
Reached checkout
9 of 9
Gorgias
Took the photo
5 of 5
Found the product
5 of 5
Added to the cartUnsupported
Reached checkoutUnsupported
Ada
Took the photo
6 of 7
Found the product
3 of 7
Added to the cartUnsupported
Reached checkoutUnsupported
Decagon
Took the photo
4 of 4
Found the productUnsupported
Added to the cartUnsupported
Reached checkoutUnsupported
Zendesk
Took the photo
4 of 5
Found the productUnsupported
Added to the cartUnsupported
Reached checkoutUnsupported
Sierra
Took the photo
2 of 6
Found the productUnsupported
Added to the cartUnsupported
Reached checkoutUnsupported
Shopping capabilities
What each agent can do for a shopper
Share of shopping conversations where the agent named a product or gave a price, and whether it ever added to the cart or stated a total.
| Vendor | Names a product | Price | Adds to cart | States cart total |
|---|---|---|---|---|
| Chatbase | 100% | 100% | Yes | Yes |
| Gorgias | 100% | 93% | Unsupported | Yes |
| Ada | 100% | 92% | Unsupported | Yes |
| Fin | 91% | 82% | Yes | Yes |
| Sierra | 98% | 63% | Yes | Yes |
| Botpress | 100% | 8% | Unsupported | Unsupported |
| Zendesk | 70% | 24% | Unsupported | Yes |
| Decagon | 64% | Unsupported | Unsupported | Unsupported |
Quality by intent
Quality by conversation type
Chatbase against the other vendors' average, per conversation type.
Total beginner
88
64
Specific need / reassurance
78
69
Gift shopper
84
68
Comparison, budget-tight
81
66
Everyday value buyer
85
67
ChatbaseCompetitor average
Order tracking / delay
84
69
Shipping & returns policy
91
85
Returns & exchanges
85
78
Damaged / faulty item
82
68
ChatbaseCompetitor average
Step by step
Every shopping step, scored 0 to 10
Average score per step. Green is above the step average, red below.
| Step | Chatbase | Gorgias | Botpress | Fin | Ada | Decagon | Sierra | Zendesk |
|---|---|---|---|---|---|---|---|---|
| New to this | 7.2 | 8.5 | 8.0 | 5.7 | 7.7 | 7.8 | 6.5 | 3.6 |
| Main options | 8.2 | 8.8 | 8.7 | 7.3 | 8.4 | 9.0 | 5.8 | 4.0 |
| Safest pick | 6.3 | 8.5 | 7.7 | 7.3 | 7.0 | 7.0 | 5.1 | 5.1 |
| Starter pick | 7.8 | 8.8 | 6.7 | 7.3 | 8.2 | 5.3 | 6.8 | 3.6 |
| Reviews | 7.2 | 4.8 | 8.0 | 6.0 | 5.1 | 4.5 | 4.8 | 1.7 |
| Extras needed | 7.3 | 8.2 | 8.0 | 7.3 | 7.9 | 7.0 | 6.8 | 3.6 |
| Add, total | 7.3 | 1.8 | 2.0 | 6.3 | 1.7 | 1.3 | 1.5 | 0.6 |
| First use | 7.8 | 8.0 | 9.3 | 7.0 | 6.8 | 5.0 | 6.6 | 3.6 |
| Returns | 8.5 | 6.3 | 7.7 | 9.0 | 5.2 | 9.5 | 8.3 | 6.2 |
| Total, delivery | 5.0 | 6.0 | 3.3 | 2.7 | 3.2 | 2.0 | 1.8 | 1.7 |
| Theme average | 7.3 | 7.0 | 6.9 | 6.6 | 6.1 | 5.8 | 5.4 | 3.4 |
| Step | Decagon | Chatbase | Fin | Gorgias | Botpress | Sierra | Ada | Zendesk |
|---|---|---|---|---|---|---|---|---|
| Help me choose | 7.0 | 7.1 | 7.0 | 7.8 | 7.3 | 5.8 | 7.0 | 5.2 |
| Everyday value | 7.7 | 7.3 | 6.7 | 8.0 | 6.3 | 4.9 | 6.8 | 5.2 |
| Budget picks | 6.7 | 7.1 | 7.0 | 8.0 | 4.7 | 4.3 | 8.4 | 5.2 |
| Compare two | 9.3 | 6.0 | 7.0 | 8.2 | 5.7 | 5.0 | 7.6 | 3.8 |
| Best seller | 7.3 | 6.3 | 7.0 | 6.3 | 6.0 | 5.0 | 5.9 | 2.6 |
| Add to cart | 1.3 | 5.4 | 4.3 | 2.0 | 2.0 | 2.0 | 1.6 | 1.0 |
| Total, discount | 4.0 | 6.3 | 6.7 | 6.0 | 2.3 | 5.3 | 3.1 | 2.9 |
| Warranty | 10.0 | 7.1 | 9.0 | 4.7 | 7.3 | 7.1 | 3.4 | 4.7 |
| Delivery time | 9.0 | 9.0 | 6.3 | 7.0 | 8.3 | 7.5 | 3.0 | 4.2 |
| Returns | 9.7 | 9.0 | 6.3 | 4.3 | 7.0 | 8.6 | 2.1 | 5.2 |
| Theme average | 7.2 | 7.1 | 6.7 | 6.2 | 5.7 | 5.6 | 4.9 | 4.0 |
| Step | Gorgias | Botpress | Chatbase | Fin | Decagon | Ada | Sierra | Zendesk |
|---|---|---|---|---|---|---|---|---|
| Specific need | 7.0 | 7.0 | 6.9 | 7.0 | 7.0 | 7.0 | 6.9 | 4.2 |
| Quality | 6.7 | 7.0 | 5.8 | 5.0 | 6.0 | 4.6 | 4.8 | 2.2 |
| Best fit | 7.0 | 6.0 | 4.4 | 3.5 | 4.0 | 5.6 | 4.8 | 2.0 |
| Reviews | 4.8 | 9.3 | 5.6 | 5.5 | 5.3 | 4.8 | 4.9 | 1.7 |
| Pairings | 6.5 | 6.3 | 5.5 | 6.0 | 6.3 | 5.8 | 4.7 | 3.7 |
| If it fails | 7.2 | 5.3 | 6.4 | 8.5 | 7.5 | 6.3 | 7.8 | 5.4 |
| Add, total | 1.8 | 1.7 | 3.4 | 1.5 | 1.5 | 1.9 | 1.8 | 0.6 |
| How to use | 7.0 | 7.0 | 5.9 | 5.5 | 5.0 | 5.8 | 5.7 | 3.2 |
| Subscription | 8.5 | 7.0 | 4.5 | 8.0 | 6.8 | 5.6 | 5.4 | 2.8 |
| Total, arrival | 5.7 | 3.3 | 5.1 | 3.5 | 1.3 | 1.1 | 2.6 | 2.0 |
| Theme average | 6.2 | 6.0 | 5.4 | 5.4 | 5.1 | 4.9 | 4.9 | 2.8 |
| Step | Chatbase | Fin | Gorgias | Decagon | Sierra | Ada | Botpress | Zendesk |
|---|---|---|---|---|---|---|---|---|
| Tight budget | 6.7 | 7.0 | 7.2 | 6.0 | 6.3 | 6.9 | 5.3 | 5.8 |
| Best value | 5.1 | 5.3 | 7.5 | 4.0 | 4.4 | 5.4 | 5.0 | 4.1 |
| Two picks | 7.7 | 7.0 | 8.2 | 5.5 | 4.3 | 6.3 | 4.7 | 4.3 |
| Worth it? | 7.7 | 8.0 | 8.0 | 5.3 | 4.4 | 5.5 | 4.0 | 4.0 |
| Promos | 6.9 | 9.0 | 7.0 | 5.5 | 8.4 | 8.9 | 6.3 | 5.8 |
| Your pick | 8.1 | 7.0 | 8.5 | 6.3 | 5.7 | 6.1 | 4.3 | 3.2 |
| Add, total | 6.4 | 4.3 | 1.7 | 1.8 | 2.1 | 1.5 | 0.7 | 1.1 |
| Warranties | 5.6 | 8.3 | 6.0 | 9.8 | 6.8 | 3.8 | 7.0 | 5.1 |
| Instalments | 6.6 | 9.0 | 7.3 | 7.8 | 8.1 | 6.4 | 7.7 | 3.7 |
| Fastest ship | 6.0 | 1.7 | 6.0 | 4.3 | 4.5 | 3.1 | 2.7 | 1.7 |
| Theme average | 6.7 | 6.7 | 6.7 | 5.6 | 5.5 | 5.4 | 4.8 | 3.9 |
| Step | Chatbase | Gorgias | Fin | Sierra | Decagon | Ada | Botpress | Zendesk |
|---|---|---|---|---|---|---|---|---|
| Gift help | 7.8 | 8.3 | 6.0 | 7.0 | 5.8 | 7.8 | 7.3 | 6.3 |
| Recipient | 7.3 | 8.0 | 6.3 | 5.3 | 6.8 | 6.8 | 6.7 | 5.5 |
| Budget | 7.4 | 7.7 | 6.0 | 5.2 | 4.3 | 6.7 | 4.3 | 4.5 |
| Top two | 7.0 | 8.2 | 7.7 | 5.3 | 6.5 | 6.8 | 6.0 | 2.4 |
| Gift set | 8.3 | 7.0 | 8.0 | 5.5 | 6.0 | 7.7 | 5.0 | 3.3 |
| Gift-ready | 7.8 | 7.7 | 5.3 | 7.6 | 7.3 | 6.9 | 7.7 | 3.4 |
| Add, total | 8.4 | 1.8 | 4.0 | 2.0 | 1.5 | 0.8 | 2.0 | 0.5 |
| Gift message | 4.4 | 6.7 | 5.7 | 6.8 | 4.3 | 2.0 | 6.3 | 0.6 |
| Exchanges | 7.9 | 4.7 | 5.3 | 8.2 | 9.0 | 3.2 | 4.0 | 3.4 |
| 2nd item deal | 6.0 | 7.2 | 5.3 | 6.3 | 2.8 | 2.8 | 3.0 | 2.4 |
| Theme average | 7.2 | 6.7 | 6.0 | 5.9 | 5.4 | 5.2 | 5.2 | 3.2 |
Below the step averageAbove
Stronger colour means a bigger gap, up to 4 points.
Distribution
Response speed, turn by turn
Median seconds to a complete reply, for each of the ten messages.
DecagonMedian 5.3s
T1T5T10
SierraMedian 6.1s
T1T5T10
BotpressMedian 7.9s
T1T5T10
ChatbaseMedian 8.1s
T1T5T10
AdaMedian 8.2s
T1T5T10
ZendeskMedian 8.6s
T1T5T10
GorgiasMedian 11.1s
T1T5T10
FinMedian 15.5s
T1T5T10
Guardrails
Held the line under pressure
Stores where the agent refused each adversarial request.
| Vendor | Off-topic | Code request | Prompt injection |
|---|---|---|---|
| Chatbase | 6 of 8 | 8 of 8 | 8 of 8 |
| Ada | 7 of 9 | 9 of 9 | 9 of 9 |
| Botpress | 3 of 3 | 3 of 3 | 3 of 3 |
| Decagon | 4 of 4 | 4 of 4 | 4 of 4 |
| Fin | 1 of 2 | 2 of 2 | 2 of 2 |
| Gorgias | 5 of 6 | 6 of 6 | 6 of 6 |
| Sierra | 6 of 9 | 9 of 9 | 9 of 9 |
| Zendesk | 4 of 10 | 10 of 10 | 10 of 10 |
Data as of 2026-11-05.