App Store product page optimization: what you can A/B test, and how to read it

Product page optimization is Apple’s built-in A/B test for your App Store product page. In App Store Connect, you create up to three treatments, versions of your page with a different app icon, different screenshots or different app previews, and the App Store shows them to a share of randomly chosen people on iOS 15 and iPadOS 15 or later, for up to 90 days. App Analytics then reports each treatment’s conversion rate, its lift over your original page and a confidence level; at 90% confidence, a treatment can be labelled Performing Better or Performing Worse. It cannot test your app name, subtitle or keywords, and it measures how many viewers download, not where your app ranks in search.

This guide follows Apple’s documentation step by step: what a test can change, how traffic is split, the 90-day limit and the review, how to read the results, where Apple’s own pages disagree, and what to test first.

What can you test with product page optimization?

Three elements. App Store Connect Help describes a treatment this way:

By default, test treatments are copies of your app’s original App Store product page. You can edit the app icon, screenshots, and previews of each treatment.

Whatever you leave unchanged in a treatment comes from your original page. Here is every part of a product page, with what a test can do with it.

Element	In a test?	How it changes otherwise
App icon	Yes, if the icon is already in your app’s binary	A new app version
Screenshots	Yes	A new version, or by applying a treatment
App previews (videos)	Yes	A new version, or by applying a treatment
App name	No	A new version
Subtitle	No	A new version
Keyword field	No	A new version
Description	No (see below)	A new version
Promotional text	No	Any time, without a new version

None of the fields that App Store search matches as text, the name, the subtitle and the keyword field, can be tested. A change to them goes to everyone with a new version, and the only way to judge it is to measure ranks before and after, as the guide to judging a listing change explains.

Where Apple’s pages disagree. One page, in App Store Connect Analytics Help, says a test can cover “screenshots, app previews, descriptions, and app icons”. The step-by-step page for setting up a treatment lists only the icon, the screenshots and the previews, and so does the treatment resource of the App Store Connect API: an icon, then screenshot sets and preview sets per localization, and no field for a description. Until App Store Connect shows you a description field in a treatment, plan your tests around the three visual elements.

How does a product page optimization test work?

You set up a test in App Store Connect, in your app’s Product Page Optimization section, with the Account Holder, Admin, App Manager or Marketing role. The app must be live: “Your app must be in the Ready for Distribution state to test its product page.” A test has four settings.

1. A reference name, only for you. Apple’s limit: “Name your test in up to 64 characters.”
2. Up to three treatments. Apple warns that “the more treatments you include in a test, the longer it may take to reach conclusive results.”
3. A traffic proportion: the share of people who see a treatment instead of your original page, divided equally between the treatments. Apple’s example: with 40% of traffic and two treatments, each treatment gets 20%, and your original page keeps 60%.
4. The localizations. All of them are selected by default. Apple adds: “If the localization shown to a user is excluded from your test, the user won’t be included in the test.”

People are chosen at random, and they keep the version they were given:

These will be shown to a percentage of randomly selected people on the App Store in iOS and iPadOS 15 or later, who will see the same treatment for the duration of your test.

Two consequences are easy to miss. A treatment is more than a product page: Apple says its screenshots, previews or icon “may appear in search results, as well as on the Today, Games, and Apps tabs.” And an alternate icon follows the people who download from that treatment: it “will display on the App Store as well as on their device”.

Some pages cannot be tested at all: “Product page optimization tests aren’t available for custom product pages or for product pages on the App Store for Apple Watch or iMessage.”

How long does a test run?

Up to 90 days. Apple’s product page optimization page says it in two sentences: “You can create one test at a time which will run for up to 90 days. Keep in mind that you cannot change a test once it’s started.”

Before you start, App Store Connect can estimate the duration. You choose the improvement in conversion rate you hope for, and it uses your app’s daily impressions and new downloads to estimate the time and the impressions needed. The estimate is only a guide, and Apple is frank about it: “it may not be possible to reach your desired improvement in conversion rate within 90 days.”

Once the test runs:

• Results take a little while. “Test results appear in App Analytics after there are five first-time downloads associated with the test.”
• Stopping is final. “Once you stop a test, it can’t be restarted.” You would have to create the same test again.
• A new app version can blur it. Apple’s overview doesn’t say that a new version ends a running test. It says that releasing one during a test “may impact the results if your version contains assets or metadata that are currently being tested.”

Apple’s pages disagree a second time here. The product page optimization page says one test at a time. A troubleshooting answer in App Store Connect Analytics Help says “You can test different product page elements (screenshots, description, preview video, etc.) in separate tests at the same time”, and adds to “avoid testing overlapping elements simultaneously”. Whatever App Store Connect lets you do, one test at a time is the setup whose results you can read without doubt.

Does a test go through App Review?

Yes, for anything new in it.

Before you can publish your test, any new metadata that you’d like to include in your treatments must be approved by App Review. If you plan to include any alternate app icons, you’ll need to include all variants of your icon in the binary of your published app. Product page optimization tests that don’t include any alternate app icons can be submitted for review independent of a new app version.

In practice:

• New screenshots or previews go to review with the test, without a new version. When you start the test, App Store Connect asks you to add it to a submission, and “Items in the submission will be reviewed together with your latest iOS version.”
• An alternate icon must already be in the binary of the version on the App Store, “which should be built with Xcode 13 or later and support alternate icons in asset catalogs through an appropriate SDK”. So an icon test starts one release earlier, when you ship the icon variants.
• Reordering screenshots or previews already on the App Store, or only changing the icon, needs no new review: in that case, Apple says, “your metadata is already approved and you don’t need to resubmit”.

How does Apple report the results?

In App Store Connect, open the Analytics tab, then Acquisition, then Product Pages. For each variant, your original page and each treatment, App Analytics shows:

• Unique impressions: “the number of unique devices that viewed each variant”.
• Conversion rate: an estimated percentage of people who downloaded or pre-ordered your app from that variant. Apple says the estimate “incorporates the data observed during the test, as well as existing data.”
• Improvement: the estimated relative lift compared with the baseline. The baseline is your original page unless you pick another variant.
• Confidence: “The probability that the data gathered in a test suggests that two variants are performing differently.”
• A status: Collecting Data, Performing Better, Performing Worse, or Likely to be Inconclusive.

When a treatment reaches 90% confidence, it may be labeled as Performing Better or Performing Worse compared to the baseline.

The method is Bayesian, with techniques “designed specifically for App Store product page data”, and each estimate comes with a credible interval: “This represents a 90% interval, meaning there’s a 90% probability that the conversion rate or lift falls within this range.” Likely to be Inconclusive means that, at the current pace, there probably won’t be enough data after 90 days. Among Apple’s suggestions for that case is “testing a more dramatic change that may show larger lift”.

Three practical readings follow from Apple’s own words:

• Wait for a label. Apple recommends waiting to apply a treatment or stop a test “until at least one treatment has been declared to be performing better or worse than the baseline with at least 90% confidence.”
• Read the numbers as estimates. The conversion rate mixes test data with existing data, and a 90% label still leaves, by Apple’s own definition of confidence, a 10% probability that the difference is not real.
• Applying is partial. Look at what the button does before you press it:

You can apply one treatment per test, and the action can’t be undone. Keep in mind that only the app previews and screenshots from the treatment will be applied. To apply changes to the app icon, set it as the default icon in your next app version.

Applying a treatment while the test still runs also stops the test.

Product page optimization does not measure your search rank

This is the difference most guides skip. A test answers one question: of the people who see your page, or your screenshots in search results, which version gets more of them to download? Measuring your ranking answers another: for one keyword, in one country, where does your app appear in the list? The two stay separate for three reasons.

1. Different fields. App Store search matches your name, subtitle, keyword field and primary category as text. A test can change none of them.
2. Different numbers. A test reports a conversion rate, a lift and a confidence for each variant. It reports no keyword and no rank.
3. An unknown link. Apple lists downloads among the user-behaviour signals of search, “downloads, ratings and reviews, and more”. A page that converts better may bring more downloads. Apple does not say whether, or how much, that moves your rank.

If you want to know whether a winning treatment changed your rank, measure your ranks before and after you apply it, keyword by keyword and country by country, with several searches a day; the guide to measuring rankings shows how, and ASO Gauge’s method does it three times a day. The App Store search guide lists the signals Apple documents.

Two neighbouring tools are often confused with product page optimization:

• Custom product pages are alternate pages that you send people to with a link, or assign to keywords so that they appear in search results for those keywords. They are not tests, and product page optimization cannot test them. See the custom product pages guide.
• Google Play’s store listing experiments can also test text, such as your descriptions. The Google Play guide explains what they cover.

What should you test first?

Apple does not rank what to test. Its documentation points to a few strong first candidates.

1. The first screenshots. Apple says “the first one to three images will appear in search results when no app preview is available”. People see them before they open your page, and a treatment’s screenshots appear there too.
2. The opening of your app preview. Previews “autoplay with muted audio”, so Apple asks for visually compelling first seconds. A video is a hypothesis, not a rule: a page with one can convert worse than a page without, and only a test tells you which.
3. The icon, once you have shipped its variants in a version. It appears everywhere, including on the device of the people who download from the treatment.
4. One bold difference at a time. Apple says larger improvements need fewer weeks of data, and asks you to consider how many elements you change “so you can more easily determine which one led to a particular result.” A small app learns more from one clear difference than from three subtle ones.
5. Your largest localization first. “Each treatment can be localized in all of the languages that your app supports, or just those you choose”, and Apple warns that the localizations you select can make a test longer.

Common pitfalls

• Stopping early. A stopped test cannot be restarted, and Apple recommends waiting for a 90% label.
• Changing several things in one treatment. You will know that it won, not why.
• Shipping the assets under test in a new version. Apple says it may impact the results.
• Excluding a localization by accident. People who see an excluded localization are not in the test.
• Forgetting the icon after applying. Only screenshots and previews are applied; the icon waits for your next version.
• Reading a winner as a ranking win. A higher conversion rate is not a higher rank. Measure the rank on its own.
• Expecting a quick answer on a small app. Results appear after five first-time downloads, and a test that cannot conclude in 90 days is marked Likely to be Inconclusive.

Sources

• Apple, Product page optimization: what can be tested, random assignment, traffic, 90 days, review, monitoring, applying.
• Apple, App Store Connect Help, Overview of product page optimization: requirements, iOS 15, new versions during a test, pages that cannot be tested.
• Apple, App Store Connect Help, Create a test, Configure test treatments, Run a test and Apply a test treatment to your product page: settings, icons, review, stopping, applying.
• Apple, App Store Connect Analytics Help, Product Page Optimization: statuses, conversion rate, lift, confidence, Bayesian method, definitions, troubleshooting.
• Apple, App Store Connect API, AppStoreVersionExperimentTreatment and AppStoreVersionExperimentTreatmentLocalization: what a treatment holds.
• Apple, Creating your product page: screenshots in search results, app previews, which fields change with a new version.
• Apple, App Store search: the signals behind search results.

Checked on 26 October 2026.