SEO A/B Testing: A Scientific Approach To SEO

Generally, SEO is about following the best tried and tested advice out there and adhering (for the most part) to Google’s Webmaster Guidelines.

Following a (good) strategy is still the best way to grow in organic revenue and to minimize wasted effort. A/B testing for organic search is the next step in validating your tactical improvements / changes to maximize the overall output of your efforts.

SEO A/B testing isn’t a brand-new concept, yet few companies have invested in it. I want to outline the general idea behind it and how to go about it the right way. Keep in mind that A/B testing for organic search will only provide actionable results for larger websites with a large pool of pages / categories.

Why do we want to A/B test?

There’s 2 main reasons here:

1.) We can justify spend in certain areas.

As SEO’s, we have had to often provide an ROI estimation for certain changes / improvements, especially if the product owner has no buy-in.

Giving an estimation of increased traffic / sales when a canonical tag is placed on certain pages is pretty much impossible, but testing 500 pages with self referencing canonical tags, title tags optimized for targeted keywords, schema markup and optimized images against 500 control group pages that fall under the same category, that is something we can make a much more accurate ROI estimation for.

2.) Avoid bloopers.

You have this amazing idea that’s going to increase organic search revenue. It involves schema markup on most of your landing pages to increase CTR and get these pages in featured snippets.

You go ahead, the feature gets released and the next day you walk into the office and all your landing pages are on page 100 of the SERP’s. You managed to get a Spammy Structured Markup Penalty.


We could have prevented this with a test.

The Setup

Our main objective is to identify a large group or category of pages and then split those pages up into two groups (group A and group B).

An example would be Travelstart, they have a large group of airline pages that consists of landing pages, each revolving around one airline.

Lets say, for arguments sake, there are 500 of these airline pages, and we want to test how changing the <h1> tag to include the primary keyword would affect rankings of said pages.

Our steps would look like the following:

1.) Pull the ranking data for the primary keyword only for all 500 pages so we have a clean ratio of 1:1, keyword : landing page.

The next step can be tricky, but with some tinkering, we can usually get a fairly close match, keep in mind that this does not have to be 100% accurate.

2.) Pop your keywords into Google Adwords Keyword Planner to get the exact search volume and then divide the keywords up so you have a roughly equal search volume in both groups (A & B) and a roughly equal amount of keywords in both groups.

Now we should have something that looks like this:

Keep in mind these are not real search volumes

Our total search volume for both groups fairly the same.

3.) Now that we have our two groups we can create 2 extra columns to match the keyword with the landing page on each row.

4.) We need to now add these keywords into a rank tracker such as SEO Monitor or Ahrefs.

Make sure to create a separate group for each group of keywords in your ranking tracker so we can see the difference between the two groups as a whole. If your ranking tracker does not support this functionality, you will need to export your keywords + ranking data at set intervals, place the keywords back into their respective groups and then calculate the ranking movements for the groups as a whole.

5.) Now we can make our changes to our landing pages to test our hypothesis. We will use Group A as our control-group and make the changes we believe will impact rankings to Group B.

Before this is done, make sure that we have existing ranking data of both groups of keywords.

6.) After we have made our changes, we can start monitoring the average rank of our two groups.

SEO Monitor has a built in feature for this:

We can now see what percentage of visibility score (basically just ranking position) goes up or down for each group.

We can also do this on a daily, weekly or monthly schedule in Excel or Google Sheets by exporting the ranking data and getting the average rank of all keywords in the group.

We can then store each interval (daily, weekly or monthly, whichever you deem fit for your test) average amount to see how the rankings change (up or down).
Obviously in this case, the lower the average, the better the keywords are ranking in Google.

We can now continue to monitor the average and make a decision to either roll back the changes or keep the changes and roll it out on larger groups of pages and have solid data to motivate the extra resources.




Leave a Comment

Your email address will not be published.