Incrementality: The Rigorous Way to Prove to Your CFO Ads Actually Worked

Firearms marketing's actual impact can be hard to measure because mainstream platforms abandoned the industry. The robust digital marketing and measurement tools available to Petsmart.com are not available to Guns.com, for example. We're hard at work at Armanet fixing that with a product that shows the incremental value of your ad campaigns.

Incrementality is basically measuring "how many people would have bought your product anyway" in order to compare that to "how many people did buy." The difference between those two groups of people — and the associated revenue — is the actual lift of your ad campaign, and is ultimately how a CFO or CEO at any firearms marketing company is going to be convinced to increase your ad budget.

But any conversation between a marketing leader and a CEO or CFO is going to come down to clarity on what an incrementality measurement tool is actually measuring. Because not all measurements are created equal and design matters a lot.

The Ultimate Marketing Question

At some point, the person who writes the marketing checks asks the same question: did advertising actually cause more sales? Inherent to the question is that a certain amount of sales would have happened anyway.

Simple question, not-so-simple answer. The vast majority of ad platforms don't answer that question — or even try to. They report on ROAS and call it a day, but ROAS alone is an incomplete metric.

ROAS tells you how much revenue happened after ads ran — that's attribution. Incrementality is the rest of the story. It isolates what changed because advertising happened.

To figure out what changed because advertising happened means measuring what happens when you don't advertise in a clean, representative, unbiased, and statistically significant manner. Easier said than done.

ROAS tells you what happened. iROAS tells you what ads caused.

If you show ads to people who are already in-market, many of them will buy. Some of those purchases will be because of the ads. Others will be accelerated by them. Some would have happened anyway — the person was already going to buy your firearm regardless of the ad. This is especially true if you're a 2A advertiser primarily advertising on 2A native properties.

The nature of sites in Armanet's network — Guns.com, AR15.com, Armslist — is that the userbase is inherently always some level of in-market for a gun-related product. Otherwise, why would they be there? ROAS alone doesn't answer the question of how many of those people would become your customers anyway.

This leads to conversations around attribution windows, last click versus first click, view-through credit, and so on. Those conversations are ultimately symptoms of the same underlying issue: the system cannot separate baseline demand from incremental demand. You're left with a number that looks precise but doesn't tell you what you actually need to know, and it doesn't help you make the case for expanding your ad budget internally.

"Did this campaign create sales that wouldn't have happened otherwise?" is the ultimate question.

What Incrementality Actually Means

The concept is simple. You need to compare two scenarios: what happened when people saw your ads, and what would have happened if they hadn't. The difference between those two outcomes is the incremental lift — sales that are additive on top of what would have occurred naturally.

The challenge is that you can't actually observe both scenarios for the same person. You can't show someone an ad and also not show them an ad at the same time.

So one has to construct a valid comparison that stands up to statistical scrutiny. How an incrementality test is designed matters a lot. We take a simple and transparent approach grounded in observed behavior, and we encourage our advertisers to scrutinize our process and help us improve if they feel the design is wrong.

How Armaview Measures Incrementality

Armaview measures incrementality by comparing outcomes between two real groups of users inside the same media environment.

Group one: users who saw your ads.

Group two (holdout): users who did not see your ads, but are matched on the factors that matter for purchase behavior in our ecosystem — same publishers, same ad units, same geographies, same timeframe. Importantly, the holdout group is not ad-free. They still see ads. They still participate in the same auctions. They simply do not see ads from the advertiser being measured.

That distinction is critical. We're not comparing "people who saw ads" versus "people who saw no ads." We're comparing "people who saw your ads" versus "people who saw everything else in the same environment." This isolates the causal impact of your advertising without removing users from the real market conditions in which your media actually runs.

For this measurement to be statistically significant, spend needs to be meaningful — $15k/month or more for a minimum of three months.

What the Lift Represents

The difference in purchase behavior between these two groups is the incremental lift — real, statistically significant, observed lift rather than theoretical. It represents additional orders and revenue that appear in the exposed group but not in the matched holdout group, where lift equals the difference in conversion rate between users who saw your ads and those who saw ads for everything but your product.

So when we report incremental lift, we're reporting the portion of performance that is additive on top of the baseline demand you'd expect without any advertising. Some of your attributed revenue was going to happen anyway. Some of it was created by your ads. Incrementality tells you which is which — and then allows you to calculate incremental ROAS, or iROAS.

How to Interpret Incremental ROAS

Incremental ROAS is incremental revenue — the dollar value of the lift measured by the incrementality study — divided by spend. It answers a very specific question: if I spend another dollar under similar conditions, how much additional revenue does that dollar create?

That makes incremental ROAS one of the most important metrics for business planning. It tells you the marginal return of your spend. If your total ROAS is 10x but your incremental ROAS is 2x, that's actually great. Most CPG retail brands view a 2x iROAS as scale-worthy because it means they have enough margin for the spend to be truly profitable.

Why This Matters in Practice

Most advertising fails because of a lack of confidence in the numbers. When you can't trust your measurement, you can't make good decisions. You end up cutting spend that was working or doubling down on spend that wasn't. You argue about attribution windows instead of optimizing campaigns, and you treat marketing as a cost center instead of a growth lever.

Incrementality restores confidence by anchoring measurement to a clear counterfactual: without this campaign, here is what would not have happened.

The Reality of Channel Mix in the 2A Industry

Every retailer in this category is overestimating the incrementality of at least one channel and underestimating another. The most common pattern in firearms e-commerce is that branded search, retargeting, and bottom-funnel affiliate are over-credited, while CTV, podcast, content affiliate, and upper-funnel display are under-credited. Last-click and even position-based attribution will produce that exact distortion every time.

Incrementality testing is not a research project — it is a budget reallocation tool. The point is not perfect attribution; the point is directionally correct allocation. A retailer who runs four reasonably designed holdout tests over the course of a year will end up with a meaningfully better channel mix than a retailer who keeps trying to model their way to truth with attribution software.

The cheapest incrementality test most retailers can run is a retargeting holdout. Pick a 10 percent holdout of the retargeting audience, run it for 30 to 45 days, and compare conversion rates between the served and held-out groups. Most retailers find the lift is two to four times smaller than what their attribution platform reports. That alone will reshape how the channel is funded going forward.

Affiliate incrementality is harder to test because of the partner relationships involved, but the pattern in most categories is consistent. Loyalty and cashback affiliates drive minimal incremental revenue because they activate at the moment of conversion intent. Content and review affiliates drive meaningful incremental revenue because they generate awareness and consideration earlier in the funnel. Most programs in this category cannot distinguish between the two in their reporting, which means partners providing genuine value and partners simply clipping the conversion are getting paid the same way.

What Armaview Is

Armaview is our incrementality measurement product. It runs alongside your campaigns and produces a clear read on incremental lift — not simulated or modeled from assumptions, but deterministically observed from real user behavior in the real media environment where your ads run.

The output is simple: here's how much additional revenue your campaign created, and here's your incremental ROAS. The goal is not to replace ROAS and attribution but to complement them with the kind of measurement that builds genuine proof your ads actually worked.

The Bottom Line

It all comes back to the same question: did advertising cause more sales?

Armaview answers that with a valid comparison — users who saw your ads versus matched users who didn't — and gives you a number you can actually use: incremental revenue, incremental ROAS, and the confidence that comes from knowing what your spend actually produced.

Talk to Us About Running an Incrementality Study

Related Articles

All posts
Top