Email Testing Using The Taguchi Method
Andrew Kordek | June 19, 2012
Does this sound familiar?
Q: What time of the day is best to send an email? A: Test it Q: Does lifestyle imagery perform better than other imagery? A: Dunno…test it. Q: How does the placement of buttons affect my click thru and subsequent conversion rate? A: We should test that.Almost every email seminar or discussion I have attended or had has always veered to having to test things in email. In fact, almost every single email blogger and speaker alive has uttered the words “test it”. There is no escape from the testing conundrum and you are either in a culture of test, learn and adapt or not. As an ex-brand side email marketer, I can safely say that testing is hard. It takes commitment, passion, patience, resources and most of all a willingness for things to change. Some ESP’s make testing so hard that it’s almost not worth the effort and we are only talking about the simple A/B stuff. Mix in that word called “multivariate” and now you are asking us brand side folks to saw off our limbs in the name of testing. A/B and Multivariate testing both have advantages and disadvantages to each methodology and we have done both for our clients successfully. However one methodology of testing that is not often discussed in the email circles is that of Taguchi testing. I suspect that the reason for this is that there are not a lot of agencies that can perform Taguchi based email testing as it does require some special considerations and skill. With that in mind, lets talk about what Taguchi testing is and how it can be applied to your email program.
How the Taguchi Method WorksAs you know, there are dozens of factors that can impact an email campaign – everything from copy, design, time of day, subject lines to signatures, buttons, imagery and so much more. Taguchi is a testing methodology that allows for accelerated testing without having to build and validate every possible combination of test factors. By systematically choosing certain combinations of test factors, it is possible to isolate their individual contributions to the email’s success. The result is identifying which combination of factors is critical to increasing conversions and ROI. Here are 4 indicators that it might be time to conduct Taguchi testing on your email program:
- Traditional testing methods such as A/B tests aren’t improving results
- The need and desire to test lots of factors simultaneously
- The organization is wanting the best “recipe” delivered
- You are in need of fast results of which A/B and Multivariate testing simply cannot deliver on