Improve Campaign Success by Leveraging User Experience: Part 2

This is a post I wrote available HERE. I am posting on this site to make it more widely available.

—————————————————————————

In my earlier post, I shared two tips on how to perform campaign tracking beyond what a typical web analytics solution can provide. The goal is to avoid providing a negative user experience that would ruin and otherwise well run campaign. The first tip was to set up Tealeaf with performance metrics in order to measure your campaign’s user-experience. The second tip was to add campaign IDs to a group list, allowing you to quickly identify campaigns that may be having an issue. In this post, I’ll give you two more tips on this topic.

Tip #3: Measure Conversion
Don’t forget your KPIs! If you’re a retailer, make sure you track your orders. If you’re a B2B company, make sure you keep track of your leads, etc. Look at your success counts over campaign click-through ratios. Use the dimensional analysis capabilities in Tealeaf to hone-in on differences that merit replay of a few sessions in order to understand the user experience. Keep track of what campaign groups are converting and which ones are not. Replay sessions that convert well and sessions that don’t and look for stark differences.

Tip #4: Non-Converting Metrics
There’s no avoiding it—some campaigns are going to be more successful than others. But don’t leave it to pure conversion rates to understand the campaign success and the user experience. Some campaigns do well at conversion, some are good for branding, others may have unexpected outcomes.

1. Registration: Did the user register? If so, he may be open to further marketing, and that’s a win in itself.
2. Abandoned Revenue: Did the user add products to the cart and then abandon? If he went into the checkout process, chances are you have a way to contact him again. Look at the campaigns that generate large amounts of abandoned revenue to find prospects that are open to more marketing. That means additional opportunity.
3. Information Pages: Did the users spend a lot of time on information pages? Chances are you just successfully placed your brand in the mind of the user. A branding success.
4. Don’t forget REPLAY: before you kill a campaign make sure there are no unexpected outcomes. Walk through the customer experience by replaying 5-10 sessions in Tealeaf. You may be surprised by what you find.

Although there may be some overlaps with the metrics you are tracking in your web analytics tool, adding campaign tracking to Tealeaf gives a holistic view of what your prospects experience when they click through from a campaign. Keep your eyes open for anomalies and stark differences. Then understand what’s going on by replaying web sessions. It’s a great way to be further informed about the campaigns you have running at your company.

How do you track your campaigns in terms of how well they are performing from a user’s point of view?

Improve Campaign Success by Leveraging User Experience: Part 1

This is a post I wrote available HERE. I am posting on this site to make it more widely available.

—————————————————————————

Learn from Measurement to Accelerate Innovation

At a client I was surprised by one of the concerns they have with measuring web traffic in general. Their concern is not with technology, manpower or budget, the concern is with culture. Their culture is highly innovative and creative and there are hints of resistance to web measurement. This has created concerns that web measurement will not be fully embraced. I was actually a bit surprised by this. I see measurement and innovation, done well, as the next innovation focused disruptor. One of my favorite subjects during my MBA was innovation; culture was always stressed as important for enabling innovation and implementing strategy. Of course, changing culture is akin to turning a large cruise liner. It is a large effort that takes a lot of time. The more I thought about this client, the more I could see the reasons for the resistance. Organization and innovation are polar opposites. The dark side of innovation is free movement, but utter chaos. The dark side of organization is complete organization with no movement. These two sides need each other to operate properly, but leaning to one side or the other depends on the state of the market. Anything with the web, mobile, cloud, etc. as a market needs to lean heavily to the innovative side. Otherwise, as we continue to see in this ever changing world, companies focused on organization bite the dust. My hope is that this client can stop seeing web measurement as another form of measurement and accountability, but as a tool for learning.

We’ve all heard the mantras, “You don’t know what you can’t measure”, “If you can’t measure it, you can’t improve it”, etc. These are valid statements that are more on the organization side (needed to take advantage of innovation). They are like the brakes on a car. If you drive a car without brakes how fast are you really going to drive? But any innovative company should be concerned, if these brakes are misused; they freeze up, the car stops moving and the competition passes by. So, yes, there is a dark side to measurement. Measurement is organization, plain and simple. If measurement is used as a way to just show reports and ensure some incremental improvement to the status quo, there is reason for concern. If reports are used in this way the company is merely policing the status quo. The big question should always be, “Am I Learning Something?”. If there is no learning there is no way to challenge the status quo which is necessary for small to big innovations. If measurement is used as a learning tool, it can empower innovation and further accelerate innovation. If used as a learning tool the incremental and LARGE improvements will come because you know your market and your customers. That is what I love about Tealeaf’s set of tools. Yes, you can create some great reports and measure incremental improvements, but the most powerful piece is understanding the customer experience. This puts a real story behind the numbers and empowers innovation. Being able to drill in to individual sessions based on abandonments, voice-of-customer, time-to-complete, customer-struggle, etc. moves it from numbers on a report to a learning experience. My hope here is that eventually this company I am working with will see Tealeaf as an accelerator to innovation and not just another reporting tool. In that way, turning that cruise-liner of a culture doesn’t need to happen. Innovation can move forward accelerated with customer experience learning.

CEM and Web Analytics Overlaps

Recently I was working with a large web based company with Tealeaf CEM tools and happened on an issue/opportunity that would save the client double-digit-millions of dollars. Having worked with Omniture as a consultant and HP as a web analyst, I had to think back if I would have discovered this same issue with the other web analytics toolsets.

As I thought about it, the resounding response was, “Yes, yes I could have found that issue with a web analytics package.” The difference is the process, and how the process fits in with the client’s/company’s processes.

—Now I don’t want to make this into another “my tool is better than your tool” post. I promise not to do that, I just wanted to point out the difference in processes that could be used to find the same issue.—

I’m not going to be the guy that pretends there are hard lines between CEM tools and Web Analytics tools. Those lines are crossing every day. It’s just a big Venn diagram that keeps pushing in towards the center. I think most of us who use both tools realize that. The differences are the angles and the processes. At this point I highly respect the companies that use both a web analytics toolset and Tealeaf products. You can find different things with each tool, sometimes it is hard justifying both toolsets to the execs, but they both have their unique value propositions (which also happen to overlap more and more as years go on).

There are 2 types of web analytics issues/opportunities that can be found on a web site, your low hanging fruit and your high hanging fruit. When I was a consultant at Omniture, the head of consulting espoused finding the low hanging fruit: 1- because it was easy to do and 2- many times there is just as much value in the low hanging fruit than in the high hanging fruit. The problem I had with that, I was always handed the high hanging fruit and I had the wrong tools to get at them. Often the anomalies were handed my way because I was the guy that knew how the system worked. I either found the heart of the issue (through a lot of hard work) or failed because I just couldn’t get high enough up the tree or broke a couple of branches in the process. It was a high risk position with very little reward. I simply lacked the right tools. That is why it was so refreshing for me to discover Tealeaf. Tealeaf is the ladder that I can place against the fruit tree to get at that high hanging fruit that no one is touching in the web analytics world. The web analytics world can definitely see some juicy fruit high up there, but often just can’t reach it…

The same can be said about Tealeaf getting at the low hanging fruit in the web analytics world, it can be done, but you have to try it from the top of the ladder. Thus the Venn diagram analogy…

So here is the process that I went through to find the problem. I want to compare it to web analytics processes I would expect to see from two different types of companies:

1. A large company that has strict release dates and heavy control on client side scripting.

2. A mid-size company with virtually no restrictions to update the implementation.

First I discovered that a particular browser had lower conversion rates than other browsers. OK, this one is easy to find in both a web analytics tool and Tealeaf. So we know there is a problem.

1. Large Company: Easy to find

2. Mid-Size Company: Easy to find

Now I need to know if this is related to a specific checkout process. Easy to do in Tealeaf, just add each checkout process as its own event (takes minutes) and let the data chug.

1. Large Company: Hopefully separating out varying checkout processes was thought through. I’ll assume it was, so easy to do.

2. Mid-Size Company: Even if it wasn’t thought out it should be easy to have an engineer add in the tracking for each process. May take an hour, may take a day or two. Let the data chug.

It is related to a single checkout process. Replaying a few browser sessions I see a common occurrence, a message telling users to update the security in their browser. This is where the split often happens between CEM and web analytics.

1. Large Company: To find this issue there is a lot of digging that needs to happen. You can pull up the browser in question and walk through the process hoping you have the same issue, but often, if QA didn’t see it you won’t see it.

2. Mid-Size Company: Same as a large company.

Now I want to see how prevalent the security message is for that browser in the process. Maybe there is a common occurrence between these sessions that will help pinpoint the problem. I add an event to the security message (minutes to do) and let the data chug.

1. Large Company: If the security message was discovered, but there was no way to find that it happened in the web analytics tools, then need to update the implementation. If it requires server side coding you could be looking out 3 months for the next release date. If there is less concern around client side scripting AND you can identify that the message was displayed by looking in the DOM, you could get at it a little quicker.

2. Mid-Size Company: If the security message was discovered, and no way to see it in the analytics tools. Just implement further tracking. May take an hour, may take a day or two. Let the data chug.

I was able to determine that the message appeared for N% of users on that browser. And the conversion rate for those that saw the message was rather low. Now replaying those specific sessions, I see a series of clicks and page views that lead up to the message. Now let me create a sequence event to track how often those series of events occur. Now let the data chug.

1. Large Company: Sequence events are nearly nonexistent in an out of the box web analytics tool. May be able to get at this with some advanced segmentation or data warehousing.

2. Mid-Size Company: Same

Using the sequence event, I was able to determine that 99% of the time it was this sequence that created the security message. “Bag it and tag it”! Time to pass on the data AND the replayable sessions to QA, Product Management and Engineering. It is then added to the list of bugs to fix.

1. Large Company: Finally able to determine the cause of the low conversion. Now, convincing Product Management and Engineering is a whole other ball of wax.

2. Mid-Size Company: Finally. Now get in a room with everyone and talk it through. They’ll see the issue easy enough. Added to the list of bugs to fix.

The difference here, with CEM tools I was able to pull out the problem and pinpoint in less than a day. By providing real evidence to the engineering group, the issue was taken seriously and the fix was added to the list.

With web analytics tools we may eventually get there, but it will take days to months to completely flesh out the problem. Convincing engineering will take some more time if you are in a large company.

Once again, this is not a “MY TOOL IS BETTER THAN YOUR TOOL” post. There are different processes that get you to the same solution. I just feel like I’m climbing a ladder with Tealeaf rather than struggling up branches to get to those high hanging fruits in the web analytics world.