71% of advertisers now consider incrementality the most important KPI for retail media investments, according to a January 2024 survey by the Association of National Advertisers (ANA). With US retail media ad spending projected to exceed $62 billion in 2025, marketers are demanding proof that their campaigns drive net new sales, not just conversions that would have happened anyway. This shift from traditional ROAS to incremental measurement represents a fundamental change in how brands evaluate retail media network (RMN) performance.

The Retail Media Measurement Problem

Retail media grew 20.4% in 2024 to reach $52.3 billion in the US alone, making it the fastest-growing ad channel. But this explosive growth brought a critical challenge: how do you know which sales actually resulted from your advertising?

Traditional return on ad spend (ROAS) tells you the total revenue generated during a campaign. What it doesn't tell you is what would have happened without the ads.

But most RMN ads appear close to the point of purchase, so advertisers have reservations about how much of the attributed sales would have happened regardless. If someone was already planning to buy your product and saw your ad minutes before purchasing, should that count as a successful campaign outcome?

This is why incrementality has emerged as the new standard.

What Is Incrementality in Retail Media?

Incrementality measures the additional sales or conversions that occurred specifically because of your advertising, nothing more, nothing less.

The basic formula:

Incrementality = (Test Group Sales - Control Group Sales) / Control Group Sales


In practice, this means comparing two groups:

• Test group: Shoppers exposed to your ads
• Control group: Similar shoppers who weren't shown your ads

The difference represents your true incremental lift.

Why Incrementality Matters More Than ROAS

Consider this scenario:

• You spend $100,000 on Amazon Ads and generate $500,000 in attributed sales (5.0 ROAS). Impressive, right?
• But what if $400,000 of those sales would have happened organically, customers searching for your brand, repeat buyers, people who saw your product recommended, or shoppers already planning to purchase?

Your incremental ROAS (iROAS) is actually much lower:

• True incremental sales: $100,000
• Incremental ROAS: 1.0 (not 5.0)

This distinction is critical because the control group contains no impressions or clicks from your brand; counting only attributed conversions would dramatically understate true lift.

How Brands Measure Incrementality

There are several methodologies for measuring incremental lift, each with different complexity levels and resource requirements:

1. Geo-Based Holdout Testing

Divide your target markets geographically. Run campaigns in 50% of markets while holding back the other 50% as a control.

Example: A CPG brand runs Walmart Connect ads in the Northeast and Southeast regions while pausing ads in the Midwest and West. After 4 weeks, they compare sales lift between test and control regions.

Pros: Relatively straightforward to implement
Cons: Requires sufficient scale across multiple markets

2. Audience-Based Control Groups

Create matched cohorts of users, exposing one group to campaigns while suppressing ads for the control group.

This approach, often called "ghost bidding," is becoming more common among sophisticated RMN advertisers. Recent experiments show that iROAS performance ranged from 253% to 1,609% across advertisers, clear evidence that some programs create substantial value, while others have room to optimize.

3. Media Mix Modeling (MMM)

According to research highlighted by eMarketer, 49% of marketers worldwide are now using MMM, which uses historical data and statistical models to quantify the impact of each marketing channel over time.

Pros: Can measure long-term effects across all channels
Cons: Requires significant data history and statistical expertise

4. Platform-Specific Tools

Amazon Marketing Cloud provides relatively straightforward incrementality measurement within Amazon's ecosystem, offering closed-loop attribution for campaigns running on the platform.

Clean rooms allow for granular analysis at the SKU, category, or audience level, helping brands understand not only if a campaign drove incremental value, but where and with whom it performed best.

The Data Consolidation Challenge

Here's where most brands hit a wall: measuring incrementality requires unified data across multiple platforms.

Amazon and Walmart accounted for 84.2% of retail media digital ad spending in 2024, but most brands run campaigns across 4-10 different RMNs. The percentage of CPG brands spending on four or more RMNs rose to 85% in 2024, up nearly 20 points year-over-year.

Each platform reports metrics differently:

• Amazon calls it "Total Sales"
• Walmart reports "Gross Merchandise Value"
• Target uses custom attribution windows
• Instacart measures "Orders Influenced"

Without standardized data, you can't:

• Run meaningful A/B tests across platforms
• Identify which RMN drives truly incremental sales
• Detect cannibalization between channels
• Establish consistent control groups

Building Your Data Foundation

Before investing in expensive incrementality tools, you need consolidated reporting across all RMNs.

‍

Step 1: Centralize Campaign Data

Pull performance metrics from every platform into a unified dashboard. Key metrics to track:

• Impressions and reach
• Clicks and click-through rate
• Conversions and sales
• Spend and cost per acquisition
• New-to-brand (NTB) sales (a useful proxy metric)
• ROAS (as a baseline for comparison)

‍

Step 2: Standardize Naming Conventions

Create consistent campaign naming across platforms so you can compare apples to apples. Without this, analyzing cross-platform performance becomes nearly impossible.

‍

Step 3: Establish Historical Baselines

To avoid noise and bias in incrementality measurements, you need to carefully define which users to include in your conversion population and analysis. This requires at least 8-12 weeks of historical data showing:

• Organic sales trends by product/category
• Seasonal patterns
• Regional variations
• Customer acquisition rates

For brands running campaigns across multiple platforms like Amazon Ads, Google Ads, Meta Ads, and retail media networks, automated data connectors can save 15-20 hours per week previously spent on manual reporting. Tools like Dataslayer connect 50+ advertising platforms to destinations like Google Sheets, Looker Studio, Power BI, or data warehouses (BigQuery, Snowflake, Amazon Redshift).

Proxy Metrics While Building Incrementality Capabilities

Not every brand has the resources to run controlled experiments immediately. Here are interim metrics that correlate with incremental impact:

New-to-Brand (NTB) Sales

New-to-brand is a favorite indicator of reach and incremental sales because it eliminates doubt on customer acquisition.

However, NTB metrics have limitations, they're only measuring the first time a customer shopped that brand within that marketplace, not reflective of a true NTB customer across all channels.

Better approach: If you can incorporate your own customer data and compare it to customers who purchased within an RMN, you can get a truer sense of which customers are actually new to the brand.

Market Share Lift

Track your share of category sales before, during, and after campaigns. If your market share increases during campaign periods and holds afterward, that suggests incremental impact.

Conversion Rate by Touchpoint

Compare conversion rates for shoppers at different stages:

• Zero previous brand interactions
• One previous touchpoint
• Multiple touchpoints

Lower conversion rates on first exposure suggest your ads are reaching new audiences (more incremental).

The Role of Natural Language Queries in Data Analysis

One emerging trend: using AI to explore retail media data through conversational queries.

New tools allow marketers to ask questions in plain English rather than building complex SQL queries or pivot tables:

• "Which RMN delivered the highest new-to-brand rate last quarter?"
• "Show me conversion trends by platform for the past 90 days"
• "Compare Amazon Ads performance to Criteo Retail Media by product category"

This approach makes data exploration faster, especially when investigating patterns that might indicate incrementality issues (like sales dropping despite increased ad spend, or one platform cannibalizing another).

For teams working with consolidated data in platforms like Claude, ChatGPT, or other AI assistants, Model Context Protocol (MCP) integrations can enable these natural language queries directly against your marketing data sources.

Real-World Applications

Case Study 1: Reducing Wasted Spend

Mondelez used Walmart Connect's Search Incrementality feature to optimize ad frequency and rotate creative assets seasonally, helping dramatically reduce ad fatigue while increasing engagement and conversions 53% year-over-year and incremental ROI 29%.

Case Study 2: Cross-Platform Optimization

A beauty brand running campaigns on Amazon, Ulta, and Sephora used geo-based holdout testing to measure true incremental lift. Results showed:

• Amazon: 2.8x iROAS (strong incremental performance)
• Ulta: 1.2x iROAS (moderate incrementality)
• Sephora: 0.7x iROAS (largely capturing existing demand)

Based on these insights, they shifted 30% of Sephora's budget to Amazon, resulting in 22% more incremental sales overall.

What's Next for Incrementality Measurement

With retail media spend projected to reach $60.81 billion in 2025, adding roughly $29.2 billion in new ad dollars, more growth than Meta and Alphabet will see combined, the pressure to prove incrementality will only intensify.

Between Q1 2024 and Q3 2024, the number of RMNs that offered access to Media Mix Modeling rose 50%, showing that platforms are responding to advertiser demands.

Three Trends to Watch

1. AI-Powered Measurement
AI-driven tools can automate the creation of test-and-control experiments, making it easier for retailers of all sizes to evaluate campaign effectiveness. Machine learning models can identify patterns in massive datasets and recommend optimal control groups.

‍

2. Clean Room Adoption
Clean room technologies provide secure environments for aggregating and analyzing both exposure and purchase data from multiple sources without exposing personally identifiable information (PII).

‍

3. Standardization Push
Industry leaders are working to establish standards around terminology and measurement automation. Only 6% of advertisers fully trust retailers' reported media metrics, according to Bain & Company, driving the push for independent verification.

Your Incrementality Roadmap

Here's how to start measuring true campaign impact:

‍

Months 1-2: Foundation

• Consolidate data from all RMNs into unified reporting
• Establish baseline metrics (organic sales, conversion rates, market share)
• Standardize campaign naming and tracking
• Identify which platforms provide native incrementality tools

‍

Months 3-4: Proxy Metrics

• Track new-to-brand sales by platform
• Monitor market share trends during campaign periods
• Analyze ROAS variations across similar campaigns
• Look for cannibalization patterns between channels

‍

Months 5-6: Controlled Testing

• Start with simple geo-holdout tests on your largest RMN
• Use platform-specific tools (Amazon Marketing Cloud, Walmart incrementality features)
• Document learnings and refine methodology

‍

Months 7+: Advanced Measurement

• Implement media mix modeling across all channels
• Partner with third-party measurement providers for validation
• Build incrementality testing into quarterly planning
• Use insights to optimize budget allocation

Frequently Asked Questions

Q: Is incrementality testing expensive?

It depends on your approach. Only 26% of in-house marketers currently conduct incrementality testing internally, but costs vary widely:

• Low cost: Geo-holdout tests using existing campaigns ($0 additional cost)
• Medium cost: Platform-specific tools like Amazon Marketing Cloud (included in platform spend)
• High cost: Full media mix modeling with external partners ($50,000-$500,000 annually)

Start with free or low-cost methods before investing in sophisticated tools.

Q: How is incrementality different from attribution?

Attribution tells you which touchpoint gets credit for a conversion. Incrementality tells you whether the conversion would have happened anyway.

For example, last-click attribution might credit your Amazon ad for a sale. But if that customer was already searching for your brand, incrementality analysis would show minimal lift from the ad.

Q: Can I measure incrementality for small campaigns?

Yes, but you need sufficient scale for statistical significance. As a rule of thumb:

• Minimum 1,000 conversions per test group
• At least 2-4 weeks of campaign duration
• Enough budget to create meaningful test vs. control split

If your campaigns are smaller, start with proxy metrics like new-to-brand sales or market share lift.

Q: What's a "good" incremental ROAS?

Recent experiments show that iROAS performance ranged from 253% to 1,609% across advertisers. However, benchmarks vary by:

• Industry and product category
• Campaign objective (acquisition vs. retention)
• Platform maturity
• Competitive intensity

Generally, iROAS above 2.0 (200%) indicates strong incremental performance, while anything below 1.0 means you're largely capturing demand that would have converted organically.

Q: How do I convince leadership to invest in incrementality measurement?

Present the business case in terms of wasted spend:

If you're spending $1M annually on retail media with a 4.0 ROAS ($4M in attributed sales), but 60% of those sales are non-incremental, you're only generating $1.6M in true lift (1.6x iROAS).

Knowing which campaigns drive real incrementality lets you reallocate budget to maximize actual business impact rather than vanity metrics.

Q: Should I stop using ROAS entirely?

No. Incremental ROAS is ideal for post-campaign analysis, while total ROAS is more suited for in-flight optimization and audience penetration tracking.

Use both:

• ROAS for day-to-day campaign management and efficiency tracking
• iROAS for strategic budget allocation and quarterly planning

Conclusion

Incrementality has moved from academic concept to business imperative in retail media. With 71% of advertisers now ranking it as their most important KPI, brands that can't demonstrate true incremental lift will struggle to justify increasing RMN budgets.

The good news: you don't need to implement sophisticated testing immediately. Start by consolidating your data, establishing baselines, and tracking proxy metrics like new-to-brand sales. As your measurement capabilities mature, layer in controlled experiments and advanced modeling.

The brands winning in retail media aren't just driving conversions, they're proving which conversions wouldn't have happened without their advertising. That's the difference between optimizing for vanity metrics and driving actual business growth.

Looking to consolidate retail media data across platforms? Try Dataslayer free for 15 days to connect Amazon Ads, Criteo Retail Media, Google Ads, and 50+ platforms to Google Sheets, Looker Studio, BigQuery, or Power BI.