I have written about ways to rise above the noise and get attention for your content (see my post Winning the Attention Game in Content Marketing). An important part of this involves tuning into the voice of your customer. After all, it is hard to get them to care if your info is not relevant to their concerns and needs. That’s where the concept of trend spotting comes in.
The ultimate in relevancy is to be able to connect with consumers based on their interests at the moment. While it is difficult to target down to the individual user, you can take some cues from online behavior such as the information they provide, posting activity and the paths visitors take across your website.
Looking more broadly out across your target audience, it would seem that the world of social media should give marketers the ability to zero in on hot topics of customers at any given time: what are they buzzing about? Who is driving the conversation? This would be valuable information, as it could allow us to tune content and campaigns accordingly.
There are many ways to find the trending topics across broad audiences – but try to do the same for laundry detergent, for example, and it might not be so easy. This article explores that question, namely: how does one find the trending topics and rising stories in specific areas related to your market and customer concerns?
The Unstructured Big Data Analytics Challenge to Trend Spotting
It would seem to be a simple question but it is not, as I came to realize, following some research that included many conversations with a range of experts.
Being in tech, a fast moving field with many sub specialties and tribes, I wanted to find a way to zero in on the hot conversations, rising topics and news in noisy areas like big data, for example. This grew out of our desire to leverage agile content marketing, something I discussed in my post: Lies My Content Marketing Expert Told Me.
The functionality did not jump out when I took a closer look at the searching and reporting capabilities of the major social media platforms and monitoring tools. So I issued questions on Twitter, Quora, and sent emails to a range of contacts, outlining my questions:
How do you spot rising topics and conversations within a given area of specialty? E.g. let’s say I have a database product that is great for big number crunching and want to find what’s trending in big data – how would I do this, and group together the conversations that relate to the same topic? Bonus points if you can show who has greatest influence and share of the conversation. I want to be able to see what the buzz is about, where it started and who or what is driving it.
As it turns out, these questions are not trivial. Jure Leskovec, a Stanford University researcher, and expert on how info spreads online (see my Flack’s Revenge post: Research Reveals Roles of MSM, Twitter and Blogs in Online Info) did not have quick or easy answers when I spoke with him recently; he advised me to take a closer look at the field topic modeling to learn more about the state-of-the-art.
I also had very nice conversations with Jen Zingsheim of eOutreach, and Warren Sukernek of SDL Social; both explained the challenges. I won’t bore you with all of the gory techie details, but at a high level, to solve this you would need to be able to:
- Access or create an index, i.e. database, of social media interactions ideally across channels, i.e. spanning the Web, Twitter, Facebook, etc.; it has to be fresh and updated in real-time or near real-time.
- Search across the index and group similar topics together, even though the precise words may vary.
- You would also need vast storage and processing capabilities.
Also, it is one type of challenge to search a term such as big data, that you know is hot, and another altogether for a system to independently identify rising topics in a given area. Most premium solutions are keyword driven, and let you train the system by specifying search phrases.
The underlying technology for accomplishing the desired trend spotting identification relates to the fields of semantic search and artificial intelligence, both evolving, but we are not quite there yet.
So What is Hot in Big Data today?
It is the age old (well, decade old, anyway) social media question: what is hot today? There, historically, have been many ways to answer this question. You have your classic trending topics and hash tags. You have your news aggregators and social news sites (some can be helpful because they organize the rising stories by category). You have Google Trends and Insights. You have your most popular and shared news topics. You have your social media searching and monitoring solutions.
None of these exactly address the stated need. No doubt there is tech somewhere, perhaps coming out of research labs of major IT vendors and in use at the biggest companies that accomplishes this – but what does the average social media manager, toiling in the trenches, with a limited budget, do to get a more granular view of what is hot in specific areas?
The good news is that there are some free services that take you in the right direction, and some paid ones that get even closer. Many offer APIs, so their functionality can be extended.
I list a number of them below (apologies for any good ones that I might have left out; please feel free to add your favorites in the comments section, below. I kicked the tires of a much longer list by plugging in “big data” – if some of the obvious choices were left out, it could be because I tried them and found them lacking for IT and tech trends; however they could be useful for other areas).
Some try to solve the problem algorithmically, and others rely on human editors / curators; one (WhattheTrend) even uses crowdsourcing. The focus is on low cost or free services; many of the free ones offer subscriptions for a higher level of functionality.
BuzzFeed says it is a “leading social news organization… Our technology powers the social distribution of content, detects what is trending on the web, and connects people in realtime with the hottest content of the moment.” It features highlighted stories across a number of categories, and lets you search across its site or the “viral Web”.
Social Mention searches across social and online channels for a given term, and brings back fresh data (you can specify ranges from one hour to one month). It also identifies top users, keywords and sources, as well as identifies sentiment.
Trends Map has a beautiful interface. It shows trend spotting on a world map, and opens a dynamically updating side bar with real time tweets based on the search term.
TrendSpottr is a “real-time viral search and predictive analytics service that identifies the most timely and trending information for any topic or keyword.” I found it to be very effective; the site returned some of the hottest stories in a news magazine layout, organized via a trend spotting score (I could not find an explanation of how they calculate this), and also showed popular related hash tags, searches and sources. It does this across social media channels, or individual ones.
TweetCharts was launched by social media guru Dan Zarrella. It provides a wealth of fresh data for any user name, hash tag, word or phrase that you enter, and returns a number of parameters, such as top hash tags, links, and nice charts.
TweetDeck might seem to fall in a different category as it is more of a dashboard; however I am a big user and found this nice hack: start a new column that brings in tweets with the desired hash tag; click on the button at the bottom labeled “Show what is popular in this column”; it will show a cloud of the most popular related hash tags, users and links.
TweetMeme features the “hottest links on Twitter” across a number of channels, including Technology.
WhattheTrend “provides a constantly updated list of trending topics from Twitter and allows you and thousands of other people across the world to explain… why this item is interesting or important to people right now.”
Some of the premium sites that can help include: Trackur, Sysomos, Sprout Social, Radian6, and SDL Tridion and Tracx (this last one just introduced some new functionality, including a special Trends Report, which sounds very promising, see screen shot below).
In writing this column, I was immeasurably aided by Scott Scanlon’s Ultimate List of Trending, Real Time Search and Social Search Sites. Also, the team at Social2B helped out, and Lee Traupel suggested some solutions on Quora. Thanks for your help, all.
What options have you found to be helpful when it comes to trend spotting in the social media wild?