Unlock the benefits of streamlined A/B testing on Shopify Hydrogen
Pack's A/B testing feature is far from a minimum viable product for Shopify marketers. We combined Shopify Hydrogen's powerful server-side content rendering with Pack's drag-and-drop editor to deliver a blazing-fast, integrated experience that other Shopify split testing tools simply can't match.
Let's take a closer look at the benefits of Pack's split testing feature.
Pre-rendered content maintains site speeds during testing
Why do A/B tests on a Shopify theme often slow down page load times—which impacts conversions and muddies test results?
Most A/B testing tools for Shopify themes load the testing experience when content starts loading on your shoppers' device (client-side). With Shopify Hydrogen, your content and test data load on the server, before data is even sent to your shopper's device. This significantly speeds up load times — resulting in higher conversions and cleaner test results.
Server-side testing can reduce the risk of bugs
Injecting an A/B testing script onto your theme can be prone to bugs that can be difficult to resolve and negatively impact the customer experience—especially when they conflict with other apps. To resolve these bugs, a developer generally gets brought in, reducing their bandwidth for other vital tasks like page building and optimization. With Pack's server-side A/B testing, in-depth testing and optimizations don't are less likely to impact your site functionality.
Vast testing opportunities improve the customer experience
With Pack, you can test every element or feature on your site from copy, images, fonts, navigation — even your free shipping meter. If it's on your site, Pack can help your team test it without friction. Our visual editor and native tracking tools make it easy to create page variants and track events without developer assistance.
Reduce reliance on third-party apps
Shopify apps provide immense value, but using too many slows down your site. The more apps you add, the higher your monthly costs and the more time your team spends acclimating to different tools. A/B testing is native to Pack, so it reduces your need for third-party apps. What's more, even the most non-technical team members can use it efficiently.
How Pack's native A/B testing feature works
A/B testing on Pack + Hydrogen storefronts takes just five simple steps:
1. Set up your test
In Pack's admin, you can create a new test, specify a title, handle, description, and target your test to a specific audience—using a URL, query, path, or UTM parameter.
You can add an unlimited number of test variants, and set a goal for your test. Finally, you can determine how much site traffic you want to send to each variant. To ensure the test is running smoothly, we recommend starting small — directing 1% to 10% of users to your new variant(s). If no red flags arise, gradually increase the amount of traffic until you reach an even 50/50 split.
Note: Although you can add unlimited test variants, we recommend using only one variant at a time. This will give you the cleanest test data possible and reduce the time spent uncovering actionable results.
2. Edit page variants
With Pack's visual Customizer, you can toggle on testing mode, flip between your test variants, and make edits to your test visually. Whether you're testing copy, imagery, or toggling features on and off, Pack allows you to quickly test anything that you can edit within the Customizer.
3. Preview & run your test
After previewing your test variants, you can turn on a test with a click, and pause it at any time.
4. Analyze your results
With your testing infrastructure in place, it's time to watch the data flow into Pack's admin, and/or your preferred analytics platform. You'll want to continue running the test until you've reached a statistically significant data pool (around 1,000 site visitors at minimum with a 90% interaction rate). Then, you can identify the variants with the highest conversion rate without worrying about the validity of the results.
5. Ship your winning page variants
Once you know which variants were most successful, it's time to conclude the test, promote your winning variant, and watch your metrics soar.
Split testing on a Shopify theme vs. a Pack + Hydrogen storefront
If your team is considering the switch from a Shopify theme to a Pack + Hydrogen storefront, you're probably wondering how the testing experiences differ. So, let's review the basic mechanics, pros, and cons of testing on a Shopify theme vs. a Pack + Hydrogen storefront.
Split testing on a Shopify theme
Shopify themes are built on Liquid, a development framework that's great for static content and doesn't rely heavily on user interaction.
Pros of testing on a Shopify theme
- Ease of use: Many A/B testing apps can be installed on a Shopify theme with a single click.
Cons of testing on a Shopify theme
- Resource-heavy: A/B testing tools can range from a few hundred dollars per month to a few thousand. Additionally, your team will have to continuously manage the app to keep up with product changes.
- Not built for dynamic content: While it's possible to build dynamic elements into your Shopify theme, you won't have much control over when and how content loads. Testing is inherently dynamic, so A/B testing tools may negatively impact the customer experience on a theme. For example, page flashes often happen during loading because of how quickly the page needs to refresh to load dynamic content.
- Bug-prone: The majority of A/B testing tools and Shopify apps rely on scripts that are injected into your theme. Those scripts frequently conflict with other scripts on your site, creating bugs that harm site functionality.
