5 Steps to Picking a Cross Device Partner
The ad tech arms race aside, better cross-device mapping of your users can do two things for you:
- Give you better insights of how your customers navigate through your digital assets
- Increase performance of your online campaigns
When auditing a cross-device partner, we highly recommend that you look at these two goals separately. But before we jump into discussion of how to best score different cross-device partners, we recommend you freshen up on some of the language commonly used in this space in our blog on that topic.
Step 1: Try before you buy
After you form your list of potential cross-device suppliers, ask to test their cross-device graph. The marginal cost of this service for the suppliers is near-zero, and all of these companies are in growth stages, so take advantage of free tests on offer.
You will need to run your tests for about a month. This will give their matching algorithms enough time to sync with your data so that you can start to get reliable information that you can audit.
If you are using a probabilistic partner, make sure that you are sending over at least a million unique cookies a week, otherwise your data might be too small to make anything meaningful out of it. For a deterministic partner, make sure to check your data blind spots. For example, if Google is mapping your devices, you probably shouldn’t rely too much on them to do this in China, where it is blocked and only has a very small market share through VPN usage.
Step 2: Create your own device graph to test against
Your cross-device partners will give you accuracy measures that are quite high (90-95% within their training data). To understand the value of a device graph, you should be instead looking at recall and precision metrics, not accuracy. If you have your own cookie data that’s linked to CRM data through an order or transaction ID, then you should be doing your own audit and calculate your own recall and precision. Use log level data on your own sites traffic, from tag containers such as Tealium, Adobe, Ensighten, or tracking tools like Omniture, Google Data Transfer files or Footprint (our own in house solution.
So what is this device graph that I am building?
Well, you take your disparate cookie IDs and match these up to their global identifier. This is a field or element that can be linked to a single user like a CRM id or a hashed email.
For example, customer ID 123 purchased three things over the last month. Each purchase was associated with a different cookie id - A, B or C, where A and B are desktop cookies and C is a mobile cookie. You can then build the following graph of pairs belonging to customer 123:
B – A
B – C
C – A
This way, if one of those cookies shows up in log files, you can match it to customer 123. Scale this approach, and you can create your true device graph for all your known customers with CRM details. This can then be compared against the probabilistic and even deterministic output from any partner you are auditing. You will now be able to calculate both recall and a precision metrics, which you can use to compare device partners.
How do you use these outputs to determine which cross device supplier is best for your business?
Step 3: Score suppliers against your customer insight goals
Let’s start with a specific example: you want to know about the cross device behaviour of your customers. How many use multiple devices? At what times of day? How can you catch them when they are likely to be engaged with your brand?
The gold standard is a cross-device platform that gives you an unbiased view of your customers. Therefore, you are going to need very high recall and precision, as well as high sync and match rates. Any time when you are not reaching 40-60% in match rates or 90% in sync rates, you will have to prove to yourself that the information you are missing is just white noise (e.g. isn’t a systemic tracking failure which excludes part of your core audience), and doesn’t impact the true averages of your potential customer population. Furthermore, your precision and recall should be above 80 to 85%.
Of course, with customer insight, you’ll have to also consider your opportunity cost. Let’s say that it costs you about £100k to receive cross device mapping of your users, per year. What could you do, or have already done within your company and how does that match up with this sum? Are you considering or running usage and attitude studies? These can be very accurate on simple behavioural questions such as device ownership.
While having a great cross device supplier can be novel, if you are trying to capture a complete view of your customers and visitors, the risk of moderate to low recall and precision can obfuscate the value of our investment.
Step 4: Score suppliers against a performance ROI
Performance is different to insight when it comes to value. If you can use cross device mapping to find a way to squeeze out a 5% increase in revenue at reasonable cost, then extreme levels of precision and recall really mean very little in the end. Here, we will run through two examples of this, at extreme ends of the spectrum:
Low precision and high recall: Let’s say that for only a specific segment of your customers (because of low precision), you have very high recall, meaning you are mostly right about a device a visitor has when you make a prediction about that visitor. Therefore, for certain paid activity that allows you to target your segment to where recall is high, you can substantially increase performance. An example of this would be Google’s remarketing lists for search ads. If you feed in hashed emails and target by previous exposure to generic campaigns , a device supply graph that is highly accurate will now be able to better measure previous exposure to your specific generic campaigns across all devices. This will not only increase your retargeting audience, but also increase click through rates as remarketing ad groups can be better customised to their actual audience.
High precision and low recall: Low recall isn’t particularly bad if your existing channel activity might already have a lot of error associated with it. This might be the case in display retargeting activity that does not track cookies across devices. In this case, you are inadvertently retargeting users at a rate higher than your ideal frequency cap. Therefore, if you are flexible with your ad serving and can feed your own cookie ids to your campaign activity, you can now cap on a new global id built through a device graph.
For example, let’s say your recall is 20% in your device graph. Since you were already over-serving ads for same visitor, for the 80% of your predictions where you mismatch devices, you won’t see a large negative impact. However, for the remaining predictions where you are now correctly identifying journeys across devices for majority of your multiple device visitors (the high precision bit), you will see a substantial benefit by reducing overexposure to your ads and more effectively deploying your display budget.
Step 5: Score yourself in your ability to technically deal with a partner
As you have read through the last few steps, you might have raised a few internal flags about your technical capacity to both audit and implement the device graph. So here’s a list of your requirements to complete steps one to four. If you don’t have the capacity, you can work with an agency or data analytics partner to build these systems:
1. An FTP download of up to 2 gb of device graph data.
2. A distributive computing cluster in the cloud, which can query the above file on a daily level. I would suggest 4-8 machines, with at least 64 gb of memory each on an on demand contract (Amazon’s AWS or Google’s Compute platforms).
3. Analysts with knowledge of big data computing and SQL/HIVE/PRESTO.
4. Ad ops or developers that can deploy cross device pixels on your creative and build systems that track sessions across your site.
Is it worth it to buy a cross device solution? There isn’t one answer for everyone. It really depends not only on your goals, but the capabilities and the specific campaign activity you might be running, or are planning to run. Because this can be a daunting process many organisations are reflecting on this investment based on cultural aspirations : “we see ourselves as tech cutting edge, and therefore to continue this self association, we must pay for the latest. Then, once these tools are available to our employees, they will be able to make the most of it.” This is not a viewpoint I have considered here, and Forward3D generally stress the importance of finding incremental value in any tech for our systems, and that of our clients.
Peter Ouzounov - Senior Data Scientist