Email Testing Using The Taguchi Method
Andrew Kordek | June 19, 2012

Does this sound familiar?

Q: What time of the day is best to send an email? A: Test it Q: Does lifestyle imagery perform better than other imagery? A:  Dunno…test it. Q: How does the placement of buttons affect my click thru and subsequent conversion rate? A: We should test that.
Almost every email seminar or discussion I have attended or had has always veered to having to test things in email.  In fact, almost every single email blogger and speaker alive has uttered the words “test it”.  There is no escape from the testing conundrum and you are either in a culture of test, learn and adapt or not. As an ex-brand side email marketer, I can safely say that testing is hard.  It takes commitment, passion, patience, resources and most of all a willingness for things to change.  Some ESP’s make testing so hard that it’s almost not worth the effort and we are only talking about the simple A/B stuff.  Mix in that word called “multivariate” and now you are asking us brand side folks to saw off our limbs in the name of testing. A/B and Multivariate testing both have advantages and disadvantages to each methodology and we have done both for our clients successfully.  However one methodology of testing that is not often discussed in the email circles is that of Taguchi testing.  I suspect that the reason for this is that there are not a lot of agencies that can perform Taguchi based email testing as it does require some special considerations and skill.  With that in mind, lets talk about what Taguchi testing is and how it can be applied to your email program.

How the Taguchi Method Works

As you know, there are dozens of factors that can impact an email campaign – everything from copy, design, time of day, subject lines to signatures, buttons, imagery and so much more.  Taguchi is a testing methodology that allows for accelerated testing without having to build and validate every possible combination of test factors. By systematically choosing certain combinations of test factors, it is possible to isolate their individual contributions to the email’s success.  The result is identifying which combination of factors is critical to increasing conversions and ROI. Here are 4  indicators that it might be time to conduct Taguchi testing on your email program:
  1. Traditional testing methods such as A/B tests aren’t improving results
  2. The need and desire to test lots of factors simultaneously
  3. The organization is wanting the best “recipe” delivered
  4. You are in need of fast results of which A/B and Multivariate testing simply cannot deliver on
Trendline has a 9 step Taguchi test design process for all of our clients which typically takes 4-6 weeks depending on the organization’s frequency of email delivery.  As with any sort of testing methodology, having the ability to back it up with results is the key to knowing whether or not you are doing it right.  After executing a Taguchi test, one of our clients increased their clicks by 480% for a program that had a razor thin target and had seen response rates decline in the preceding year. If you are ready to take your testing and program to a new level of sophistication which is results driven, we would be happy to discuss our methodology and case studies with you anytime. Drop us a line.
Andrew Kordek
Andrew Kordek
Chief Strategist & Co-Founder
More posts by Andrew →