Big Data, Big Analytics
I had the honor to be interviewed for Michael Minelli, Michele Chambers and Ambiga Dhiraj‘s book “Big Data, Big Analytics: Emerging Business Intelligence and Analytic Trends for Today’s Businesses” [amazon link].
Here is an excerpt (pp. 31-34):
Empowering Marketing with Social Intelligence
We also spoke with Niv Singer, Chief Technology Officer at Tracx, a social media intelligence software provider. Niv had quite a bit to say about the big data challenges faced in the social media realm and how it’s impacting the way business is done today – and in the future.
As a result of the growing popularity and use of social media around the world and across nearly every demographic, the amount of user-generated content – or “big data” – created is immense, and continues growing exponentially. Millions of status updates, blog posts, photographs, and videos are shared every second. Successful organizations will not only need to identify the information relevant to their company and products – but also be able to dissect it, make sense of it, and respond to it – in real time and on a continuous basis, drawing business intelligence – or insights – that help predict likely future customer behavior. And if that sounds like a tall and complex order, that’s because it is. Singer explains how this can be a challenging:
It can sometimes be a real challenge to unify social profiles for a single user who may be using different names or handles on each of their social networks, so we’ve built an algorithm that combs through key factors including content of posts, and location, among others, to provide a very robust identity unification.
This brings us to the topic of influence and the age old debate of “who is an influencer?” To some brands, influence is measured purely by reach and to others, true influence is more of a function of quality and thoughtfulness of posts showing a real understanding of a given topic, and yet others gauge influence via social engagement or conversations. Because influence is so subjective, Singer believes the client should have the flexibility to sort influencers by any of these characteristics:
Very intelligent software is required to parse all that social data to define things like the sentiment of a post. We believe using a system that’s also able to learn over time what that sentiment means to a specific client or brand and then represent that data with increased levels of accuracy provides clients a way to “train” a social platform to measure sentiment more closely to the way they would be doing it manually themselves. We also know it’s important for brands to be able to understand the demographic information of the individual driving social discussions around their brand such as gender, age, and geography so they can better understand their customers and better target campaigns and programs based on that knowledge.
In terms of geography, Singer explained that they are combining social check-in data from Facebook, Foursquare, and similar social sites and applications over maps to show brands at the country, state/region, state, and down to the street level where conversations are happening about their brand, products, or competitors. This capability enables marketers with better service or push coupons in real time, right when someone states a need, offering value, within steps from where they already are, which has immense potential to drive sales and brand loyalty.
These challenges are in the forefront of technology, but also require very creative people and solutions. Every component in the system must be able to be distributed across multiple servers that don’t rely on each other. No single point of failure is allowed – the data must therefore be replicated and stored on different machines, but should still be consistent. The data is later accessed in unpredictable ways. Singer likes to use an analogy to a book in a library:
Finding a book by title or ISBN number is easy, even in a very big library. Finding, or counting, all the books written by specific authors is also relatively easy. It gets a little more complicated when we try to locate all the books written in a certain year, since we usually keep the books on shelves and sort them according to the author. If we need to count the number of books that contain the word “data” in their title written every year, it gets even more complicated… and when we need to locate all the books that contain the phrase “big data” in them, well, you can imagine.
Fundamentally, Singer doesn’t view social data as a silo and, instead, believes that the real power comes in mining social data for business intelligence, not only for marketing, but also for customer support and sales. As a result, they’ve created a system from the ground up that was architected to be open. It’s designed to be a data management system that just happens to be focused on managing unstructured social data, but we can easily integrate with other kinds of data sets. It was built with the expectation that social data would not live in an island, but would be pushed out to other applications to provide added business value and insights and that they would be pulling external data in.
This open approach like Singer is suggesting is extremely important because it enables businesses to take action with the data! Examples include integration with CRM systems like Salesforce.com and Microsoft Dynamics to enable companies to get a more holistic view of what’s going with their clients by supplementing existing data sets that can be more static in nature with the social data set, which is more dynamic and real-time. Another example is integration with popular analytics platforms like Google Analytics and Omniture, so marketers can see a direct correlation and payoff of social campaigns through improved social sentiment or an increase in social conversations around their brand or product.
Where does Singer think this is all headed next? To the next big holy grail: an ability to take all this unstructured data and identify a customer’s intent to buy:
Customer intent is the big data challenge we’re focused on solving. By applying intelligent algorithms and complex logic with very deep, real-time text analysis, we’re able to group customers into buckets such as awareness, opinion, consideration, preference and purchase. That ability lets marketers create unique messages and offers for people along each phase of the purchase process and lets sales more quickly identify qualified sales prospects.
One of Tracx customers is Attention, a heavily data-driven social media marketing agency also based in NYC. The Attention team uses the platform as the backbone of their social market research. Attention’s CEO and Founder, Curtis Hougland, had this to say about Big Data’s impact on marketing:
Social media is the world’s largest and purest focus group. Marketers now have the opportunity to mine social conversations for purchase intent and brand lift through Big Data. So, marketers can communicate with consumers when they are emotionally engaged, regardless of the channel. Since this data is captured in real-time, Big Data is coercing marketing organizations into moving more quickly to optimize media mix and message as a result of these insights. Since this data sheds light on all aspects of consumer behavior, companies are eliminating silos within the organization to align data to insight to prescription across channels, across media, and across the path to purchase. The days of Don Draper are over, replaced by a union of creative and quant.
The capability to understand a customer’s intent that Hougland and Singer are referring to is not only good for the corporations; it’s also a helpful capability for the client too. Think about it, most people communicate socially because they are looking to share, complain, or find something they need. Wouldn’t it be great if someone was listening and knew your intent so that they can provide customer assistance or get you what you need for the best price?
Innerlogics 