Our PPC team conducted a comprehensive experiment to determine whether AI-generated Google Search Ads can outperform their human-written counterparts. Our aim was to compare the performance of ChatGPT's AI-written ads against the ads crafted by our ESC PPC team.
We carefully designed the experiment across three distinct ad groups, each with two ads, to ensure reliable results and valuable insights. We focused on Dynamic Search Ads, one written by our PPC team (ESC Copy) and one generated by ChatGPT (AI Copy) tailored to each ad group theme.
We set the ad rotation in each ad group to "Do not optimise: Rotate ads indefinitely." By rotating the ads without any optimisation bias, we aimed to give every ad variant an equal opportunity to showcase its potential.
The criteria for success were straightforward - we considered an ad to outperform the other if it generated more conversions, achieved a higher conversion value, and delivered a better ROAS while staying within budget.
Excitingly, we're here to share the intriguing findings, which might reshape the way we approach advertising strategies.
Will the AI-generated ads prove to be more effective, or will the human-written ads retain their edge?
Shopify SEO audit for your online store TODAY ⬇Experiment Results
Let's explore the findings and see which ad variant emerged victorious after running this experiment and the ads for a total of 72 days.
Ad Group 1
The results of ad group 1 provided the most interesting insights in our experiment. The results showed that the ESC ads, despite spending 33% more, received 43% fewer impressions and 29% fewer clicks compared to the AI ads. This led to a 88% higher CPC. However, the ESC ads resulted in a 23% higher CTR. Most notably, the ESC ads outperformed the AI ads in terms of conversions, achieving an impressive 298% more conversions, leading to a remarkable 462% higher conversion rate and a significantly lower cost per conversion (66% lower). Moreover, the ESC ads generated an astounding 478% more conversion value and achieved a 334% higher ROAS.
Ad Group 1
| Metrics | AI Ad Copy | ESC Ad Copy | Difference to AI Copy |
|---|---|---|---|
| Cost | £145.92 | £194.40 | 33% |
| Impr | 3,702 | 2123 | -43% |
| Clicks | 253 | 179 | -29% |
| CPC | £0.58 | £1.09 | 88% |
| CTR | 6.83% | 8.43% | 23% |
| Conv | 5 | 22 | 298% |
| Conv rate | 2.16% | 12.15% | 462% |
| Cost/conv | £26.68 | £8.94 | -66% |
| Conv val | £172 | £994 | 478% |
| ROAS | 118% | 511% | 334% |
Ad Group 2
Ad Group 2 revealed some intriguing results. The ESC ads, compared to the AI ads, showed a significant difference in spending, allocating 263% more budget. As a result, they received 222% more impressions and 285% more clicks. Interestingly, the ESC ads achieved a 6% lower cost per click (CPC) and a 19% higher click-through rate (CTR).
The ESC ad achieved 165% more conversions than the AI ad. Despite this, the conversion rate was 31% lower for the ESC ads. Moreover, the cost per conversion was 37% higher for the ESC ads, which is worth noting.
The ESC ads managed to generate a substantial 360% more conversion value, contributing to their 27% higher return on ad spend (ROAS).
Ad Group 2
| Metrics | AI Ad Copy | ESC Ad Copy | Difference to AI Copy |
|---|---|---|---|
| Cost | £63.88 | £231.91 | 263% |
| Impr | 1,422 | 4,585 | 222% |
| Clicks | 87 | 335 | 285% |
| CPC | £0.73 | £0.69 | -6% |
| CTR | 6.12% | 7.31% | 19% |
| Conv | 9 | 23 | 165% |
| Conv rate | 9.99% | 6.87% | -31% |
| Cost/conv | £7.35 | £10.08 | 37% |
| Conv val | £294 | £1,356 | 360% |
| ROAS | 461% | 585% | 27% |
To sum it up, in Ad Group 2, the ESC ads clearly outperformed the AI ads in terms of conversions, conversion value, and ROAS, despite some differences in click-related metrics and cost per conversion.
Ad Group 3
In Ad Group 3, the ESC ad spent 56% more, resulting in 43% more impressions and 49% more clicks compared to the AI ad. While its cost per click (CPC) was 5% higher, it managed to secure a 4% higher click-through rate (CTR).
The true standout performance of the ESC ad in Ad Group 3 was evident in its conversions. It achieved an impressive 208% more conversions than the AI ad, leading to a remarkable 107% higher conversion rate. Moreover, the ESC ad significantly lowered their cost per conversion by 49%, which is quite noteworthy.
As a result of their higher number of conversions, the ESC ads generated 167% more conversion value, contributing to their 71% higher return on ad spend (ROAS).
Ad Group 3
| Metrics | AI Ad Copy | ESC Ad Copy | Difference to AI Copy |
|---|---|---|---|
| Cost | £94.76 | £148.05 | 56% |
| Impr | 4,022 | 5,752 | 43% |
| Clicks | 171 | 254 | 49% |
| CPC | £0.55 | £0.58 | 5% |
| CTR | 4.25% | 4.42% | 4% |
| Conv | 5 | 15 | 208% |
| Conv rate | 2.92% | 6.06% | 107% |
| Cost/conv | £18.95 | £9.63 | -49% |
| Conv val | £105 | £281 | 167% |
| ROAS | 111% | 190% | 71% |
To sum it up, in Ad Group 3, the ESC ads clearly outperformed the AI ads in terms of conversions, conversion value, and ROAS. Their increased budget investment paid off with higher impressions and clicks, further solidifying their superior performance. Ad Groups Combined
Taking all ad groups into account, the ESC ads showed some impressive results. They spent 89% more and, as a result, received 36% more impressions and 50% more clicks compared to the AI ads. Although their cost per click (CPC) was 25% higher, they managed to achieve a 10% higher click-through rate (CTR).
Now, let's talk about conversions – the ESC ads really nailed it! They secured a remarkable 214% more conversions than the AI ads, leading to an outstanding 109% higher conversion rate. Additionally, the ESC ads had significantly lower cost per conversion (40% lower compared to the AI ads), making a substantial impact.
When it comes to generating value, the ESC ads didn't disappoint. They scored a whopping 360% more conversion value, contributing to their impressive 144% higher return on ad spend (ROAS).
Ad Groups Combined
| Metrics | AI Ad Copy | ESC Ad Copy | Difference to AI Copy |
|---|---|---|---|
| Cost | £305 | £574 | 89% |
| Impr | 9,146 | 12,460 | 36% |
| Clicks | 511 | 768 | 50% |
| CPC | £0.60 | £0.75 | 25% |
| CTR | 5.59% | £6.16% | 10% |
| Conv | 19 | 60 | 214% |
| Conv rate | 3.75% | 7.83% | 109% |
| Cost/conv | £15.89 | £9.55 | -40% |
| Conv val | £572 | £2,631 | 360% |
| ROAS | 188% | 458% | 144% |
In summary, across all ad groups, our ESC ads clearly outperformed the AI ads in terms of conversions, conversion value, and ROAS. Their increased spending was justified with higher impressions and clicks, demonstrating their overall better performance.
Here are the metrics we watched most closely visualised:
Overall, the ESC ads generated 60 conversions and a conversion value of £2,631, while the AI ads generated 19 conversions and a revenue of £572.
The ESC ads achieved a combined ROAS of 458%, while the AI ads achieved a disappointing combined ROAS of 188%.
Conclusion
In this experiment, our human-written ads emerged as the champions, outperforming AI-generated ads. Their artful creativity led to stronger engagement, higher conversions, and cost-efficiency, showcasing the unique strengths of human-crafted content in advertising. The human touch remains an irreplaceable force in driving impactful campaigns and forging genuine connections with the audience.
As we continue to explore the vast potential of AI and its integration into advertising strategies, we must acknowledge and cherish the extraordinary capabilities that human-written ads bring to the table. In this dynamic landscape, the partnership of AI-driven insights and human creativity holds the key to crafting ads that truly leave a lasting impact on consumers and propel brands to new heights of success.
