Most SEO changes are judged by looking at a graph afterwards and seeing what you hoped to see. Rankings and traffic move for many reasons: seasonality, updates, competitors. A simple experiment with a comparison group tells you whether your change made the difference.
Short on time? Comppound does this work for SaaS companies. AI-native, and a person reviews every page.
Have us do itTest on groups of pages
SEO tests work best on templates with many similar pages, such as integration or template pages. Split them into a test group and a control group with similar traffic. Change only the test group.
Change one thing
Test one change at a time: a title pattern, a new section, a different opening. Changing several things at once leaves you unable to say which one mattered.
Comppound can do this for you
Want the reporting done for you?
Comppound sets up Search Console, GA4 and AI visibility tracking, ties organic traffic to sign-ups and sends a one-page report each month with what shipped and what comes next.
- Tracking set up and tested
- Organic and AI traffic tied to sign-ups
- A monthly report that leads to decisions
Compare against the control
After a fixed period, compare the change in clicks for the test group with the change for the control group over the same dates. If both moved together, the change probably did nothing. Decide the period and the measure before you start.
For single pages, keep a log
Many SaaS sites lack enough similar pages for a proper test. For single pages, keep a change log with dates and compare like-for-like periods, accepting that the evidence is weaker.
Worked example · an illustration, not a client result
Tallyroom's title test
Tallyroom split its integration pages into two similar groups and added “sync hours to invoices” to the titles of one group. It compared clicks for both groups over the following weeks.
| Group | Change | Comparison |
|---|---|---|
| Test | New title pattern | Change in clicks over the test period |
| Control | No change | Change in clicks over the same period |
| Decision | Roll out if test clearly beats control | Otherwise revert |
Do it yourself
Design one test
- 1Pick a template with at least ten similar pages.
- 2Split them into two groups with similar traffic.
- 3Choose one change and the measure, usually clicks.
- 4Fix the test period in advance.
- 5Compare the groups and decide: roll out or revert.
Free tool for this: SERP snippet preview, Conversion rate calculator
Common mistakes
No control group
Without one, seasonality and updates look like results.
Stopping early
Ending a test the moment it looks good overstates the effect.
Testing tiny groups
Very small groups give noisy results. Use templates with enough pages.
Rather not do it yourself?
Want the reporting done for you?
Comppound sets up Search Console, GA4 and AI visibility tracking, ties organic traffic to sign-ups and sends a one-page report each month with what shipped and what comes next.
- Tracking set up and tested
- Organic and AI traffic tied to sign-ups
- A monthly report that leads to decisions
Questions
No. Conversion tests split visitors; SEO tests split pages, because search engines see one version of each page.
Not for simple tests. Search Console data and a spreadsheet are enough to compare two groups.
Key terms
- A/B TestingA/B testing compares two versions of a page or element by randomly splitting visitors between them and measuring which performs better on a chosen goal.
- Split URL TestingSplit URL testing sends visitors to two different URLs, each hosting a full page version, to compare entirely different designs or flows.
- Statistical significanceStatistical significance indicates whether a difference in data is unlikely to be due to random chance alone, given a chosen threshold.
- Search SeasonalitySearch seasonality is the regular rise and fall in searches for a topic across the year. Knowing it helps you time content and read traffic changes correctly.
- Year-over-year comparisonA year-over-year comparison measures a metric against the same period one year earlier, which removes most seasonal swings from the reading.
- Sample SizeSample size is the number of visitors or conversions each test variant needs before a result can reliably detect the effect you care about.
