Improve Campaign Success by Leveraging User Experience: Part 1

This is a post I wrote available HERE. I am posting on this site to make it more widely available.

—————————————————————————

CEM and Web Analytics Overlaps

Recently I was working with a large web based company with Tealeaf CEM tools and happened on an issue/opportunity that would save the client double-digit-millions of dollars. Having worked with Omniture as a consultant and HP as a web analyst, I had to think back if I would have discovered this same issue with the other web analytics toolsets.

As I thought about it, the resounding response was, “Yes, yes I could have found that issue with a web analytics package.” The difference is the process, and how the process fits in with the client’s/company’s processes.

—Now I don’t want to make this into another “my tool is better than your tool” post. I promise not to do that, I just wanted to point out the difference in processes that could be used to find the same issue.—

I’m not going to be the guy that pretends there are hard lines between CEM tools and Web Analytics tools. Those lines are crossing every day. It’s just a big Venn diagram that keeps pushing in towards the center. I think most of us who use both tools realize that. The differences are the angles and the processes. At this point I highly respect the companies that use both a web analytics toolset and Tealeaf products. You can find different things with each tool, sometimes it is hard justifying both toolsets to the execs, but they both have their unique value propositions (which also happen to overlap more and more as years go on).

There are 2 types of web analytics issues/opportunities that can be found on a web site, your low hanging fruit and your high hanging fruit. When I was a consultant at Omniture, the head of consulting espoused finding the low hanging fruit: 1- because it was easy to do and 2- many times there is just as much value in the low hanging fruit than in the high hanging fruit. The problem I had with that, I was always handed the high hanging fruit and I had the wrong tools to get at them. Often the anomalies were handed my way because I was the guy that knew how the system worked. I either found the heart of the issue (through a lot of hard work) or failed because I just couldn’t get high enough up the tree or broke a couple of branches in the process. It was a high risk position with very little reward. I simply lacked the right tools. That is why it was so refreshing for me to discover Tealeaf. Tealeaf is the ladder that I can place against the fruit tree to get at that high hanging fruit that no one is touching in the web analytics world. The web analytics world can definitely see some juicy fruit high up there, but often just can’t reach it…

The same can be said about Tealeaf getting at the low hanging fruit in the web analytics world, it can be done, but you have to try it from the top of the ladder. Thus the Venn diagram analogy…

So here is the process that I went through to find the problem. I want to compare it to web analytics processes I would expect to see from two different types of companies:

1. A large company that has strict release dates and heavy control on client side scripting.

2. A mid-size company with virtually no restrictions to update the implementation.

First I discovered that a particular browser had lower conversion rates than other browsers. OK, this one is easy to find in both a web analytics tool and Tealeaf. So we know there is a problem.

1. Large Company: Easy to find

2. Mid-Size Company: Easy to find

Now I need to know if this is related to a specific checkout process. Easy to do in Tealeaf, just add each checkout process as its own event (takes minutes) and let the data chug.

1. Large Company: Hopefully separating out varying checkout processes was thought through. I’ll assume it was, so easy to do.

2. Mid-Size Company: Even if it wasn’t thought out it should be easy to have an engineer add in the tracking for each process. May take an hour, may take a day or two. Let the data chug.

It is related to a single checkout process. Replaying a few browser sessions I see a common occurrence, a message telling users to update the security in their browser. This is where the split often happens between CEM and web analytics.

1. Large Company: To find this issue there is a lot of digging that needs to happen. You can pull up the browser in question and walk through the process hoping you have the same issue, but often, if QA didn’t see it you won’t see it.

2. Mid-Size Company: Same as a large company.

Now I want to see how prevalent the security message is for that browser in the process. Maybe there is a common occurrence between these sessions that will help pinpoint the problem. I add an event to the security message (minutes to do) and let the data chug.

1. Large Company: If the security message was discovered, but there was no way to find that it happened in the web analytics tools, then need to update the implementation. If it requires server side coding you could be looking out 3 months for the next release date. If there is less concern around client side scripting AND you can identify that the message was displayed by looking in the DOM, you could get at it a little quicker.

2. Mid-Size Company: If the security message was discovered, and no way to see it in the analytics tools. Just implement further tracking. May take an hour, may take a day or two. Let the data chug.

I was able to determine that the message appeared for N% of users on that browser. And the conversion rate for those that saw the message was rather low. Now replaying those specific sessions, I see a series of clicks and page views that lead up to the message. Now let me create a sequence event to track how often those series of events occur. Now let the data chug.

1. Large Company: Sequence events are nearly nonexistent in an out of the box web analytics tool. May be able to get at this with some advanced segmentation or data warehousing.

2. Mid-Size Company: Same

Using the sequence event, I was able to determine that 99% of the time it was this sequence that created the security message. “Bag it and tag it”! Time to pass on the data AND the replayable sessions to QA, Product Management and Engineering. It is then added to the list of bugs to fix.

1. Large Company: Finally able to determine the cause of the low conversion. Now, convincing Product Management and Engineering is a whole other ball of wax.

2. Mid-Size Company: Finally. Now get in a room with everyone and talk it through. They’ll see the issue easy enough. Added to the list of bugs to fix.

The difference here, with CEM tools I was able to pull out the problem and pinpoint in less than a day. By providing real evidence to the engineering group, the issue was taken seriously and the fix was added to the list.

With web analytics tools we may eventually get there, but it will take days to months to completely flesh out the problem. Convincing engineering will take some more time if you are in a large company.

Once again, this is not a “MY TOOL IS BETTER THAN YOUR TOOL” post. There are different processes that get you to the same solution. I just feel like I’m climbing a ladder with Tealeaf rather than struggling up branches to get to those high hanging fruits in the web analytics world.

The Myth of the “Universal Tag” and the upcoming Tag Management Systems (TMS)

So I just listened to the webinar from Peterson and Ensighten on Tag Management Systems. This has always been a hot topic in my career. At Omniture I was part of the original team to implement and identify directions for the “Universal Tag”. I use quotes because, as was pointed out in the webinar, it really wasn’t a “Universal Tag” it was more of a helper tag to push out data to partners. It also came with unreasonable costs (at least in my opinion). Why would we charge for work that the browser was doing? Yes the data that was already being collected through the Omniture implementation could be leveraged toward partners, but the cost was unreasonable and further entrenched the customer into the Omniture tagging architecture. I complained up the channels at Omniture, but the opportunity at leveraging the tags for further revenue streams was more appealing than building out an open architecture free for everyone to use.

Fast forward and I left Omniture to be an analyst at HP. Managing tags was a HUGE issue and we looked into Tealium for help. What really made sense to us at the time was an open architecture that enterprise online tools could turn to for help in easily collecting data on customers. A central source where industry specific data was collected and then passed to any partner that wanted data to run their online tools. I had some contacts from the Omniture partner program and got some feedback on what would really work. We decided to build out our own architecture and make it open source so anyone could access it and partners could build out new functionality. I worked with Matt Wright (now the CTO at Keystone Solutions) to build out the architecture for an open source tag. Well, we had built the tag and were implementing it when I had an offer to make a lot of money and travel the world. So I left the #measure world for a year. During that time, Matt left HP for Keystone, he open-sourced our tag management architecture and has since inked a deal with webanalyticsdemystified (nice work guys). I know that Keystone has been having some success with the open sourced tag management system and that is why I was surprised to hear Eric Peterson say that online managers should run away from open source tag management.

For me keeping tag management open makes more sense than building a new industry around it. And the reason I say that is because of the power that comes from being the center of data collection. Many online tools are vying to be the center of data collection for the web. It is an extremely strategic position to be in. Everything begins at the center of data collection and distribution. That is the one reason I think that keeping the architecture open makes sense. The one question I had during the webinar was how Ensighten planned on creating checks and balances so their position of power was not abused. And also, I was curious how they would plan on working with online tools to implement new feature sets. Some kind of open architecture to develop on and then get reviewed by Ensighten developers and analysts would be ideal. Maybe if Ensighten was a non-profit entity that would give me less worry on where they might end up.

But as some of you know, I joined Tealeaf because of their data collection setup (easily collect data without bugging developers) and potentials for extreme analyses of data (they collect everything). Just because Tealeaf has a different way to collect data does not mean I think that a TMS system is moot. There will always be a need to access and distribute data directly from the browser (unless the request-response internet model ceases to exist). In fact there is code that Tealeaf uses that would be nice to add to a TMS system so data collection can be flipped on rather than reviewed, implemented and tested by clients (ideally). So, yes I am on board for an architecture that can more easily implement all these tags that online managers need to run their website. My only concern is the strategic position that the de facto TMS system may find itself in. Let’s make sure no abuse comes of it. My vote will always be on Open Source or non-profit entity because of that strategic position.

Built to fail: Innovation strategies for the web

Recently there was a post on the web analytics demystified users group about having a mistake in your analysis. This got me thinking about some of the classes I took recently around innovation and strategy (I recently finished up my MBA). The classes always pointed to the same thing, successful companies are built to fail. And the model of being built for failure is becoming somewhat of a disrupter because it nurtures innovation. Constantly staying ahead requires that you try different things and measure the success.

My last post was about Omniture and some of the concerns I had about the change from innovative culture to more organized and clanish. I thought about how this happened or how it was allowed to happen and it seems like this is a pretty common occurrence to any successful organization. Think about it, Omniture made some ground by being more innovative than Webtrends and other web analytics companies (well maybe not visual sciences, but visual didn’t get the marketing/sales piece). Having some success the executive team started looking around for experienced management that had success. They looked for the people that really did well at making money from existing products. When you move those people into another company with a successful product, they will also be successful. Why? Because they organize and market things very well.

Well, put them in a company where there is great uncertainty and that is where they have a tough time. Using the same set of tools that they have developed doesn’t work because it stifles innovation. You start to see huge planning cycles for products that may or may not have a market. A lot more money is spent to go after a market that may or may not want the product or the market may not even exist. The cool thing about being innovative is that you can try many things at a low cost and see which ones find a market. I was sad to see Omniture lose some of the agility it once had, but maybe Omniture has found its market, and they are ready to make the switch to being more organized. But if there is still uncertainty about where things are going with web analytics and web optimization they should focus on management schooled in innovation.

So, back to planning to fail. Planning to fail means that you try things that you think will be successful or at least have a chance at success and measure the success of the changes. The culture that creates innovation is the one that says it is “OK to fail”. But with that failure they build in a way to fail safely. That is the needed piece, designing an architecture that allows for safely failing. This way many things can be tried and a fraction will end up being a “bulls eye”.

I probably watched the sixty minutes video on IDEO like a dozen time in different classes. There probably isn’t a better company to get the point across about innovating better than IDEO. They basically get a bunch of people together from different backgrounds and allow them to brainstorm out a product for a market.

IDEO Innovation Techniques:

Different individuals’ backgrounds create more ideas.

• They don’t hire the “you are like me” people. They look for diversity to engender different ideas.

There is no bad idea.

• An idea should never be shot down. In fact the craziest ideas should be explored because there may be something there that has the seeds of innovation.

Rapid prototyping.

• Once they have a few ideas they quickly create the prototype to see something tangible and get feedback from customers. They have a machine shop on premise to rapidly create a prototype for their ideas.

Get out in the real world.

• They do a lot of customer research by leaving the building and interacting with potential customers. This includes showing different prototypes.

Even though they take these steps to be successful, still only a fraction of their ideas are successful, but the point is, they have successful ideas. With most companies they see a need and plain and simple get lucky. If they are looking at a market in fluctuation or at a market with an uncertain future, these types of concepts should probably be used to keep innovative and stay ahead of any potential competition, especially with those that compete on price.

But, back to my original thought on web analytics and web site optimization. We use web analytics to create actions that optimize the site, meaning we ‘help’ our potential customers better find our site and ‘help’ them convert once they click through. Doing any analysis without taking action is just silly, but sometimes that analysis is just going to be dead wrong. But as a web analyst we can’t spend our time second guessing if we truly want to find those gold nuggets that really kick the site into overdrive. That is why changes to the site need to be made, but measured and measured quickly. And, sometimes you really don’t know the affects of what your changes may have made. So, beyond using a good web analytics tool, a tool like RUM or Tealeaf would be great to quickly understand how the users are reacting to the changes individually rather than in aggregate. It is like observing your customer’s actions at a store when they pick up the new product. Also, survey any users who may have been affected by the change to get the attitudinal data. Combining the attitudinal and the behavioral should give the picture of the affects from the change that was made.

If the change is wreaking havoc on the site, get it back to how it was before and analyze what happened. You may learn something about your customers you did not know before and you may have just come closer to the gold vein you have been mining for.

And just to show that this should apply to the web here is part of a manifesto from Avinash Kaushik, a guru in web analytics.

“I believe that God created the Internet so we could fail faster. In the offline world it is very expensive to experiment and test, the cost of failure is very high. As a result we don’t take risks. We keep doing what we think ‘works’, until the day we go bankrupt. The web changes that. You can take dramatic risks, at very low costs and learn big. Your website is nothing but a machine built to make you smart by taking lots of risks. Why should you tolerate ideas getting killed on conference room tables or by your HiPPO’s? Why accept opinions when you can convert them into hypothesis and get them validated for cheap and quickly? Why not let your customers actively be a part of helping you create customer experiences that deliver value to them AND to you? The cost of taking risk on the web is low. You can try an idea. As soon as it is live data starts following it. If the idea is a total loser then kill it fast, does not have to cost you a ton of money. What is more likely is that you will find winners that you had never imagined. Give it a try. Fail faster.”

-Ryan Ekins

6 Months Since I Left Omniture

I started at Omniture in 2000 when it was named myComputer.com. Josh and John were younger, I remember seeing them enjoy a ride in John’s first Porsche. They built an engineering friendly culture and for the most part everyone enjoyed working there. Then came the bubble burst and I luckily found a more steady job in Utah Valley. MyComputer.com went down from 130+ employees to 30+ employees in less than a year. In 2003 the company, now rebranded Omniture, won a dark-horse race for the HP contract and I was hired on (kind of funny, now I work for HP). I was employee number 40 of the regrowth. I loved the Omniture Culture, I had come from a family-run business and was a bit tired of the nepotism and family politics. Omniture was highly innovative and innovation was explicitly expected from everyone that worked there.

Rather than working as an engineer, I came back to work as an implementation consultant. That is when I discovered my love for digital marketing and web-site optimization. And the next few years were a whirlwind of growth, new employees, and demanding clients. When the whirlwind started to settle I found myself working on the first Genesis integrations and passing those integrations on to engineering and consulting.

Over the years it soon became apparent the the engineering-driven culture was beginning to cease. The highly innovative culture became more organized and clanish. Organized because Omniture brought in more ”seasoned” management and more clanish because the “good-ol-boys” mentality seemed to increase as everyones’ pockets grew bigger and bigger. Not to mention that the CEOs balance from John Pestana was gone once Pestana left (think Jobs and Wozniak).

So, I was sad to see the culture change from highly innovative to organized-clanish. But a big thing that affected my decision to leave was the unsure future of web analytics. I hope Omniture wins the analytics game as I still have many friends that work there and it would be an overall benefit for Utah Valley if they are successful. As we start to see industry conversion in analytics I’m very interested to see who will come out on top.  If anything Omniture will definitely end up a successful consulting/agency group because of the knowledge they have gleaned over the years. And of all the vendors out there they have the best shot.

6 months later I must say that I do miss innovating at Omniture. There is still a lot of hope that Omniture will emerge victorious in this Petabyte Age, but only time will tell. If anything, they definitely have a shot. I feel they should probably fine tune their innovation rather than focusing on organization to increase their chances. All the best to my friends at Omniture.

-Ryan Ekins