16 Feb 2007 1012H

Multivariate analysis in user interface testing

I’ve learned recently that we now have the capability at Brulant to conduct not only A/B split testing, but multivariate analysis on user interfaces as well. This is a huge capability, and pretty interesting stuff. To begin with, let’s step back into algebra a bit. Everyone’s familiar that in science, we’re trying to observe the effect of an independent variable on a dependent variable, very roughly, cause and effect. This relationship can be described by a line on a two-dimensional plane by the equation

y = mx + b

where the y coordinate is determined by m, the slope of the line (rise over run) times the x coordinate plus a constant b (where the line crosses zero on the x-axis).

Now, assuming the relationship between these two variables is linear, let’s say you do a bunch of observations between two variables to gauge the size of the effect, one of which is dependent on the other for its value. You can draw a line of “best fit” through these observations if they all fall in a certain pattern. But if you have multiple variables that have a linear relationship, how do you gauge the size of the effects of each? As it so happens, that equation looks like

y = m1x + m2x + m3x + … + mnx + b

and each of these m’s, the predictors, will give you an indication of what effect that variable has on the response.

The practical upshot for user interface testing is that each of these m’s could be say, a user interface widget or an ad or other web page component and we can use web metrics data to calculate the magnitude of effect of each widget on a website and their relationship to conversion.

Anyway, there’s a lot of assumptions that go into multiple linear regression analysis, but this is a start; I don’t want to even think about what happens when the relationship is nonlinear, which in fact, most relationships are in this world, like the Zipf distributed, long tail graphs in web metrics data.

As an example of what I picked up from grad school, this is something I originally learned from economist Wei-Yin Hu when he taught the Asian Americans and the US Economy course at UCLA, but, as a non-math guy, it took me a long time, and I think it finally came together when I took a finite maths course a few years ago, before I understood the ramifications of what he was saying.

Permanent link to Multivariate analysis in user interface testing

Filed under Geekery, User Experience, Web

Be the first to respond to "Multivariate analysis in user interface testing"

RSS feed for comments on this post. TrackBack URL

No responses yet.

And now it's your turn.

Fire your weapon, soldier. Just be careful of friendly fire. NAME & EMAIL required.

Your response