A presentation from the NewMR ‘Listening is the New Asking’ Text Analytics Event - March 8 2011 ‘Online Text Analytics – Top Ten Tips: Welcome to the oldest new game in town’ Anna Tomkowicz Freshminds Research, UK
The sponsor of the ‘Listening is the New Asking’ event is Zinc Research For more information about NewMR events visit newmr.org The copyright for this material is jointly owned by The Future Place and the presenter. Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Online Text Analytics – Top Ten Tips Welcome to the oldest new game in town 8th March 2011
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
There are a lot of conversations going on out there...
> In 2010, Twitter users sent 25 billion Tweets and more than 100 million new accounts were registered. > More than 3.5 billion pieces of content are shared each week on Facebook. > Purpose built Facebook pages have created more than 5.3 billion fans. > In 2009, there were 126 million blogs on the internet. Source: e-consultancy, Twitter Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
... and lots of automated tools to analyse them
Powered by
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
It’s all really confusing
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
I will take you through Top Ten Tips for using automated tools for text analytics
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Different tools lead to different results 7000 6000 5000 4000 Twitter
3000
Forum
2000
Blog
1000 0
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
1
Check which areas of the internet your tool is actually monitoring
• You are probably missing out on data on closed communities. Most tools are unable to trawl private areas on Facebook. • You can use youropenbook.org to source quotes from open profiles, but there is no tool enabling robust analysis. • There are serious ethical implications in sourcing quotes from closed communities.
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
2
Check how your tool recognises location or nationality
• Most tools use the domain name to determine location (com vs. co.uk), which may lead to bias. • The language used on a forum isn’t necessarily indicative of the user nationality.
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
3
Check how recent your data is
• Some tools provide you with fresh data every day, with mentions from the most popular websites coming in first. • Other tools may lag up to a week. • Sometimes, speedy delivery of the data happens at the expense of accuracy, so a quick data set may not be a very good one. • Make a decision – do you need insight from great quality data, or the ability to quickly react at the first indication of a crisis? Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
4
Automated sentiment analysis is difficult
• Tools use complicated algorithms to detect sentiment. 20%
• There are certain limitations, e.g. challenges around irony. • Most tools are between 60%-85% accurate, but if you look at the usual sentiment metrics observed in consumer goods, you will see that if you mark all mentions as ‘neutral’ you will be on average 70% accurate.
70%
10%
• Accuracy goes down once you compare automated coding with manual coding on a one by one basis as opposed to on aggregate. Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
5 Topic
Emerging topics are sensitive to the search terms used Alterian
Biz360
Brandwatch
Nielsen
Radian6
Scoutlabs
Baristas
Yes
Burger King
Yes
Cannabis
Yes
Caribou
Yes
Celebrity
Yes
Costa Credit Cards Fair Trade Falling in love
Yes
Flat White
Yes
Freebies Gold Card
Yes
• Imposing a search term can have an impact on all your results and terms you are detecting
Goo Guns
Sysomos
Yes Yes Yes Yes Yes Yes
Yes Yes
Yes
Yes
Keep going
Yes
Proximity
Yes
Yes
Rewards
Yes
Yes
Yes
Yes
Service
Yes
Sundays
Yes
Survey
Yes
Wifi
Yes
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Which leads us to our over-arching tips
6
• Make sure you have clear business objectives before you commit to online text analytics.
7
• Don’t try to distill the analysis to a single metric.
8
• Use multiple data sources and robust sampling techniques.
9
• Tools change every week, so stay abreast of the developments and changes in methodology.
10
• Challenge the findings!
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Q&A
Anna Tomkowicz FreshMinds
Ray Poynter The Future Place
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
FreshMinds Research FreshMinds Research is an award-winning research-led consultancy specialising in qualitative & quantitative market research and business analysis. We bring together the brightest minds* and the freshest thinking to create effective, commercial solutions to every client challenge. Clients include HSBC, The Cabinet Office, innocent and eBay; from tech start-ups to global FMCG companies.
For more information, please contact: FreshMinds 229-231 High Holborn London WC1V 7DA www.freshminds.co.uk T: 020 7692 4300 F: 0870 460 1596 E:
[email protected]
Anna Tomkowicz Lead Research Manager
[email protected] 020 7692 4554 Phil Burgess Head of Consumer Research
[email protected] 020 7692 4549
And no two projects are the same. Our approach is always fresh and energetic, our focus always commercial. We aim to inspire, offer new perspectives and help you move your business forward. * (ours, yours, consumers’ and experts’)
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
