AI agent benchmark · November 2026
The AI agent benchmark for customer service and ecommerce
8 AI agent vendors, tested on 55 live storefronts with the same scripted conversations. Every conversation is captured from the production chat widget, scored blind by an LLM judge against a 38-check rubric with quoted evidence.
- Vendors scored
- 8
- Live stores
- 55
- Conversations judged
- 518
- Checks
- 38
The numbers
Every vendor, scored the same way
Shopping and Support are blind-judged rubric scores out of 100; Overall is their average.
| Vendor | Overall | Shopping | Support |
|---|---|---|---|
| Chatbase | 84.4 | 83 | 85.8 |
| Gorgias | 81.8 | 76.4 | 87.2 |
| Decagon | 77.8 | 73.8 | 81.8 |
| Sierra | 76.6 | 66.9 | 86.2 |
| Fin | 73.7 | 74.4 | 72.9 |
| Botpress | 70.0 | 64.4 | 75.5 |
| Ada | 64.7 | 68.9 | 60.5 |
| Zendesk | 48.8 | 40.8 | 56.7 |
Where Chatbase lands
We put ourselves through the same test
Chatbase ranks #1 overall of 8 vendors, #1 on Shopping and #3 on Support, and is the only vendor in the photo test that found the product from a shopper's photo and put it in the store's cart.
- Overall
- #1 of 8
- Shopping
- #1 of 8
- Support
- #3 of 8
- Photo to cart
- #1
- Reply speed
- #4 of 8