Incrementality Analysis: Measure the true impact of ASO and paid UA

If you’re responsible for app store marketing—whether optimizing organic growth, improving user engagement, or scaling paid user acquisition—one of the toughest challenges is proving what’s truly driving results.

Most analytics tools can show you that installs or revenue have increased, but they don’t tell you why. Was it your latest metadata update, or just a seasonal surge? Did your rebrand actually attract new users, or were they users who would have installed anyway?

When reporting on impact, app marketing teams often face the same questions: Are we sure this isn’t just seasonality? How can we tell this growth wouldn’t have happened anyway? Without a way to separate real impact from external influences, decisions often rely on guesswork.

That’s where AppTweak’s Incrementality Analysis in Reporting Studio comes in—the only solution to measure your organic and paid app marketing impact with statistical precision.

What do we mean by incrementality?

Incrementality refers to the true, measurable impact of a marketing effort, beyond external influences like seasonality, market trends, or organic growth. Instead of assuming a correlation between a specific event and a KPI change, incrementality isolates the actual cause-and-effect relationship.

👉 Learn more about what incrementality is and its importance for ASO

By using predictive modeling, we establish a baseline scenario—what would have happened without the event—and then compare it to actual performance. This allows us to quantify the real impact of ASO efforts, paid campaigns, and external marketing initiatives with high statistical confidence.

Understanding the value of incrementality measurement

Measuring incrementality isn’t just about seeing whether performance changed after an event, it’s about proving with confidence that your initiative was the reason for that change.

Incrementality measurement is essential for understanding the impact of key app marketing efforts, including:

• Metadata updates: Prove the impact of new keywords in metadata on search visibility.
• ASO creative updates: Understand how new icons, screenshots, and videos affect installs.
• App store featurings: Measure how being promoted by the App Store or Google Play impacts downloads.
• Promotional content: Understand the short- and long-term impact of running in-app events.
• Apple Search Ads campaigns: Analyze the incremental uplift of paid campaigns beyond expected trends.
• And much more.

To see Incrementality Analysis in practice, it’s important to understand our two distinct models for analyzing different types of events:

Extrapolation model: Analyzing long-term impact

Our default extrapolation model uses only data from before the event took place to predict expected performance.

Extrapolation is the best approach for most use cases, as it provides a clear, unbiased prediction of what would have happened without an event. It works well for measuring the long-term impact of metadata updates, store creative changes, major campaigns, or local market shifts.

For example, during the 2024 U.S. presidential election, Bitcoin.com Wallet saw a surge in downloads. Using the extrapolation model, we analyze three years of historical data to establish a baseline forecast of expected downloads during and after the event.

This incrementality analysis revealed a statistically significant uplift (+79%) in Bitcoin.com’s downloads tied to election week, as well as a 160% incremental lift in the 31 days following:

The extrapolation model confirmed that downloads stayed elevated for weeks, proving the impact was not just a temporary spike but a true market shift beyond the baseline forecast (downloads expected if the election had not taken place, like typical end-of-year trends).

👉 Deep dive into AppTweak’s download estimates, explained by our data scientists

To take this analysis further, we measured the incremental impact of the presidential inauguration (January 20, 2025) on the keyword “crypto” in the United States. With AppTweak, we saw a significant lift in maximum reach (estimated impressions) for “crypto” directly tied to the event.

But not all campaigns result in an incremental lift. Some initiatives may even negatively impact performance, revealing valuable insights into market preferences.

For example, we analyzed the impact of Twitter’s rebrand to X on the US App Store (during the period of the app’s name, description, screenshots, and icon change):

An incrementality analysis revealed a significant 27% drop in downloads in the weeks following the rebrand, compared to the baseline forecast.

Interpolation model: short-term event analysis

On the other hand, the interpolation model is designed to measure short-term events that don’t have a lasting impact by comparing performance before and after an event.

Whereas extrapolation only looks at data up until the start of the event, interpolation also looks at data after the event ends to try and predict what happened between the last day before the event and the first day after.

As a result, interpolation can be used instead of extrapolation to analyze on/off tests for paid UA campaigns. If turning off a campaign causes a drop in total downloads (organic + paid) larger than the installs attributed to the campaign by your mobile measurement partner (MMP), this suggests the campaign was driving additional organic installs beyond just paid traffic.

In other words, it would show the paid UA effort had a measurable incremental impact on organic growth, something you can now track by connecting your MMP or Apple Search Ads console in AppTweak for a unified view of marketing impact.

How AppTweak isolates incremental impact with data science

To better understand the methodology behind Incrementality Analysis, we asked our Data Scientist, Lucas Weinberg, to explain the approach in more detail:

AppTweak’s Incrementality Analysis applies predictive modeling to quantify whether a specific event—such as a metadata update or a UA campaign—had a measurable effect on your key performance indicators.

We achieve this using NeuralProphet, a modern forecasting framework that enhances traditional time series analysis with deep learning techniques. Unlike static data analysis, time series forecasting captures complex trends, seasonality, and event-driven fluctuations, ensuring uplift is correctly attributed to a marketing action rather than external noise.

Key components of our incrementality model:

• Trends — Identifies the overall direction of data using changepoints for flexible trend modeling.
• Seasonality — Captures recurring weekly or yearly patterns that influence app performance.
• Holidays & events — Accounts for major date-specific spikes (e.g., New Year’s Eve, Christmas Day) that may affect downloads in a specific country.

Ensuring statistical confidence: How we validate results

Once a forecasted baseline is established, AppTweak applies statistical validation to confirm whether an event truly influenced performance:

• 95% confidence interval — Defines the range within which expected performance should fall, ensuring deviations beyond this range are truly incremental effects.
• P-values for statistical significance — A low p-value (< 0.05) confirms that the measured impact is unlikely to be due to chance, meaning the event genuinely influenced app performance.
• Pre-event & post-event impact analysis — Measures impact across two key periods: event range (when the event directly influences the KPI) and post-event range (the period following the event to capture any lingering effects).

By combining forecasting with rigorous statistical validation, we ensure that incrementality measurement is not just precise—but actionable.

Conclusion

Incrementality Analysis is the perfect solution for teams looking to isolate, quantify, and fully understand the impact of their app store marketing efforts.

With AppTweak’s Incrementality Analysis, you can:

• Make better budget decisions by identifying which efforts drive real growth.
• Prove the ROI of ASO and paid campaigns with statistically validated insights.
• Optimize both short-term and long-term strategies based on quantifiable impact.

Request a demo from our team to discover incrementality analysis for your ASO and paid UA impact: