Testing

Testing Your Marketing Online is Paramount

Online Marketing Testing is one major area where most agencies and businesses have a hard time understanding the scientific research methodologies. They feel that they must basically trust a group of really smart people more than their own “common sense. The fact is those really smart people use mathematics and statistics to back up the validity of their findings. This hold true in A/B Testing as well as Multivariate Testing.

Strength in Numbers
Experimental scientists / Data Analysts who have done this for years know how dangerous it is to draw general conclusions using only an eyeball method of determining the outcome of results. Numbers can superficially lie and give you the sense of a particular result, but when looking at them deeper and put the results in context you may find something completely different, but what you will find for sure is the truth.

What data scientists try to do is study a large enough data sample size to ensure with a high probability that the knowledge they are gaining from their results truly reflects the most likely outcome in all real world instances.

This is an attempt to fight against random chance. When you only have a small amount of data, there is a much greater probability that the outcome is not valid. Valid testing results are a staple in determining the success of your campaigns, which are supporting your overall Digital Strategy.

Marketing lesson
Data scientists use a set of specialized principles, consisting of procedures, math and logic, then apply those to the marketing medium that is being measured. These procedures make sure that there is enough data and that the data is good.  The math makes sure that the correct answer is gathered.  The logic makes sure that the answers are turned into the right actions or decisions.

It’s not enough just to “see” which tests of landing pages or emails perform best in a single test, it must be ensured that statistical significance has been acheived. Make sure the test is statistically valid before basing a decision on the outcomes!

That takes more than just looking for a set of points on a testing platform due to these possible validity threats:

• History Effects: The effect on a test variable by an extraneous variable associated with the passage of time.
• Instrumentation Effects: The effect on the test variable caused by a variable external to the experiment, which is associated with a change in the instrument of measurement.
• Selection Effects: The effect on a test variable by an extraneous variable associated with different types of subjects not being evenly distributed between experimental treatments.
• Sampling Distortion Effects: The effect on the test outcome caused by failing to collect a sufficient number of observations.