AI Spend Classification: How Accurate Is It Really?

The Gap Between Vendor Claims and Real-World Accuracy

When procurement leaders evaluate spend analytics platforms, they encounter vendor claims like "92% accuracy" or "fastest time to classification." But what do those numbers actually mean? How are they measured? And most importantly, what accuracy should you expect in your actual implementation?

This article bridges that gap by examining real-world spend classification accuracy benchmarks, comparing vendor claims to independent validation data, and helping you understand the true cost of misclassification.

What Counts as Accuracy? Vendor Definitions Matter

Vendors measure accuracy in different ways, making direct comparison difficult:

• Segment-level accuracy: Classifying broad categories like "IT Services" or "Facilities" (2-digit UNSPSC). This is easiest and typically 92-96% accuracy.
• Family-level accuracy: More granular categories like "Software Licensing" or "Janitorial Services" (4-digit UNSPSC). Typical: 88-93%.
• Class/commodity-level accuracy: Detailed classification like "Cloud Computing Services" or "Office Cleaning" (6-8 digit UNSPSC). Typical: 80-88%.

When a vendor claims "92% accuracy," they often mean segment-level only. Commodity-level accuracy is what matters for cost analysis, but vendors highlight the higher number in marketing materials.

Real-World Accuracy Benchmarks by Platform

Based on independent implementation data from 30+ organisations across 2024-2026:

Why the Data Shows Lower Accuracy Than Vendor Claims

Vendors test on curated, clean datasets. Real implementations involve:

• Supplier names that vary wildly across the organisation
• Incomplete or missing supplier descriptions
• Ambiguous line items that could fit multiple categories ("Software" could be SaaS, licensing, or hosting)
• Tail categories with very few historical examples for training
• Multi-language supplier data in global organisations

Real-world accuracy is typically 5-10 percentage points lower than vendor benchmarks.

Accuracy Varies Dramatically by Category Type

Spend classification accuracy is not uniform. High-volume, standardised categories are easier; complex, low-volume categories are harder:

High-Accuracy Categories (92-98%)

• Standard office supplies (pens, paper, furniture)
• IT hardware (laptops, monitors, peripherals)
• Utilities (electricity, water, waste)
• Subscription software (named SaaS products)
• Commodity parts and materials

Medium-Accuracy Categories (82-91%)

• Indirect labour (staffing, contractors)
• Professional services (consulting, legal, accounting)
• Maintenance and repair services
• Marketing and advertising
• Travel and transportation

Low-Accuracy Categories (75-85%)

• Outsourced operations (cleaning, security, catering)
• Custom software development
• Construction and capital projects
• Procurement of goods with unclear supplier intent
• Cross-category bundled spend

The Cost of Misclassification

Accuracy numbers are abstract; misclassification costs are real. Example: if you have £50M annual indirect spend with 85% accuracy at commodity level:

• 7.5M in misclassified spend
• If misclassified items average 15% price variance from correct category benchmark, that's £1.125M in hidden costs
• If 30% of misclassified spend is off-contract, add another £2.25M in compliance premium

For strategic categories (indirect labour, outsourcing, professional services), misclassification costs 2-5% of category spend. For commodity categories, costs are lower: 0.5-1%.

Why Taxonomy Depth Matters More Than Overall Accuracy

A platform may achieve 90% accuracy overall, but that masks what matters: accuracy in the categories that drive your savings opportunities.

If your organisation has £50M professional services spend, 85% accuracy in that category costs you £2.25M in hidden savings (conservatively). But if your office supplies category (£2M) is classified at 98% accuracy, that precision doesn't help much.

Before evaluating platforms, rank your spend categories by size and strategic importance. Then ask vendors: what accuracy do you achieve specifically in those categories? Request case studies or references in your industry.

The Role of Training Data in Accuracy

AI classification accuracy scales with training data. Platforms perform better when they have:

• Volume: 2+ years of historical transactions per category (1000+ transactions for reliable model training)
• Consistency: Stable supplier naming conventions and category definitions
• Cleanliness: Manually coded historical data with high quality, not auto-generated or inherited classifications
• Diversity: Spend across different suppliers, cost centres, and regions to avoid overfitting to specific patterns

If you have only 6-12 months of data, or historical data with inconsistent codes, accuracy will be lower. Expect to dedicate 2-4 weeks upfront to manual validation of a representative sample to establish a quality baseline.

Vendor Claims vs Independent Validation

The disconnect between vendor claims and real-world performance stems from how testing is conducted:

Vendor Testing (Often Shows 92-96% Accuracy)

• Uses curated datasets: suppliers with complete master data, consistent naming, clear descriptions
• Tests on balanced categories: equal representation of high-volume and tail spend
• Often reports segment-level accuracy (broader categories easier to classify)
• May exclude ambiguous or boundary-case transactions

Real-World Implementation (Often Shows 80-88% Accuracy)

• Uses actual enterprise data: inconsistent naming, missing descriptions, legacy data
• Tests on the actual distribution: often heavily skewed toward low-volume, heterogeneous tail spend
• Includes ambiguous transactions that could fit multiple categories
• Measures at commodity level (the level that matters for decisions)

Recommendation: Request to run a pilot on your actual data (100K-200K transactions) before full commitment. Budget 2-3 weeks and expect 5-10% sample validation.

Can Accuracy Be Improved Over Time?

Yes. Model accuracy improves through continuous learning:

• Year 1: Initial accuracy at 85-88%, declining to 80-82% for edge cases as validation continues
• Year 2: Model retraining on accumulated corrections lifts accuracy to 90-92% for common categories
• Year 3+: Mature models stabilise at 92-95% segment, 88-92% family, 85-88% commodity for high-spend categories

But accuracy plateaus for tail spend—those categories with only 10-20 transactions annually never generate enough training data for ML to improve significantly. Expect tail spend to remain 75-80% accurate permanently, requiring periodic manual review and reclassification.

Conclusion: Accuracy Expectations and Action Items

Real-world AI spend classification accuracy is 85-92% at commodity level, varying by category complexity. Vendor claims of 92-95% accuracy typically reflect segment-level performance only.

Misclassification costs 2-5% of spend in strategic categories, making accuracy in those categories critical. Before evaluating platforms, identify your top 10-15 spend categories, benchmark their current accuracy, and ask vendors for category-specific accuracy data—not overall statistics.