UPDATE: Email Testing Using The Taguchi Method

Andrew Kordek


Q: What is the best time of  day  to send an email?

A: Test it

Q: Does lifestyle imagery in emails perform better than other imagery?

A: Dunno…test it

Q: How does the placement of buttons affect my email click through and subsequent conversion rate?

A: We should test that


Almost every email seminar or discussion I have been a part of  has veered to having to test a number of things within a given email marketing program.  In fact, almost every recognized email blogger and speaker alive has uttered the words “test it.” There is no escape from the email testing conundrum and you are either in a culture of “test, learn adapt,” or you are not.

As a former client-side / brand email marketer, I can  confidently say that email testing is hard. It takes commitment, passion, patience, resources and most of all an openness and willingness for things to change.  Some Email Service Providers (ESPs) make testing so difficult that it’s almost not worth the effort and that’s just  talking about the simple A/B stuff.  Mix in that word called “Multivariate” and now you are asking client-side or brand folks to saw off their limbs – all in the name of testing.

There are advantages and disadvantages to A/B and Multivariate email testing, and Trendline has successfully implemented both methodologies on behalf it’s clients. However one methodology not often discussed in email circles is that of Taguchi testing.  I suspect the reason for this is there are not a lot of agencies that can perform Taguchi based email testing as it requires some special considerations and research expertise .  With that in mind, let’s talk about what Taguchi testing is and how it can be applied to your email program.


As you know, there are dozens of factors that can impact email campaign performance; everything from copy, design, time of day send, subject lines to signatures, buttons, imagery and so much more.  Taguchi is a testing methodology that allows for accelerated email testing without having to build and validate every possible combination of test factors. By systematically choosing certain combinations of test factors, it is possible to isolate their individual contributions to an email’s success.  The result is identifying which combination of factors is critical to increasing conversions and ROI.

4 indicators that it might be time to conduct Taguchi testing on your email program:

  1. Traditional testing methods such as A/B tests aren’t improving results
  2. The need and desire to test lots of factors simultaneously and at scale
  3. The organization wants to deliver a  best “recipe” email
  4. You need fast results, which  A/B and Multivariate testing simply cannot deliver

Trendline has a 9 step Taguchi email test design process  which typically takes 4-6 weeks to implement, depending on the organization’s frequency of email delivery. As with any sort of email testing methodology, having the ability to back it up with results is the key to knowing whether or not you are doing it right. After executing a Taguchi test, one of our clients increased their clicks by 480% for a program that had a razor thin target and had seen response rates decline in the preceding year. That result has remained steady for over five years.


Are you ready to take your  email program to a new level of testing sophistication which is results driven? Trendline would be happy to discuss our testing methodologies and case studies with you . Drop us a line.


About the Author(s)

Andrew Kordek

Andrew Kordek is a Co-Founder of Trendline Interactive.

Let's Take This to the Inbox

Sign up for our news, resources and updates. The inbox is our favorite place after all. We’ll make sure it’s worth it. (You can unsubscribe at any time, but you probably already knew that.)