Social Media Research


Social Media Research - Rackcdn.comb219de03d4fa586d9adb-0e04f025bcde650ce6d504251ebbc914.r60.cf3.rackcdn.com/J...

3 downloads 117 Views 4MB Size

A  Presenta*on  from  the  NewMR  “Social  Media   Research”  Event   9  October,  2012  

Enriching  Social  Data  for  Market  Research     Jasper  Snyder,  Converseon  

All  copyright  owned  by  The  Future  Place  and  the  presenters  of  the  material   For  more  informa:on  about  NewMR  events  visit  newmr.org  

#NewMR

Enriching Social Data for Market Research Jasper Snyder 9 October 2012

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

Questions for Today’s Session n ions ca t s e u q f sorts o research and t a h W •  edia social m answer? is analys millions m o r f e v you mo n- and semio d w •  Ho billions of u ents to and docum research d e r u t c stru arket m e l b a action ? s insight d ia-base to d e m l ia an soc ch be tailored c w o H •  sear e r t cases? e k e r s u ma t n t clie differen g learnin r for e n i h c matte e s ma  • How d, oand why does it work research? market

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

3

Social-media research can support both traditional market research goals and PR functions. Traditional Market Research through Social Media Listening

Communications Functions through Social Media Monitoring

•  Thoughts and opinions about products and brands

•  Consumer complaints and product malfunctions

•  Market awareness of products or brands

•  Crisis monitoring and response

•  Purchase triggers

•  Reputation management

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

4

These two use cases – market research and communications – closely align with two services.

Social Listening

Social Media Monitoring

When what matters most is understanding a consumer segment or market.

When what matters most is delivering customer service, navigating a crisis situation or detecting reputation threats.

Goal is to understand a population “out there” in the world.

Goal is comprehensive, real time coverage.

Higher tolerance for missing content.

Higher tolerance for irrelevant content.

Lower tolerance for irrelevant content.

Lower tolerance for missing content.

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

5

The Social Media Research Process: From Raw Data to Insights

1. Data Collection

2. Data Enrichment

3. Analysis & Insight Generation

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

6

Stage 1: Social Data Collection Primary Goal: 1. Data Collection

2. Data Enrichment

Identify and acquire the data that can answer your business questions.

Primary Challenges:

3. Analysis & Insight Generation

1.  Pull in relevant data and metadata 2.  Coverage of appropriate social media channels 3.  Eliminate spam and irrelevant content.

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

Stage 2: Data Enrichment Primary Goal: 2. Data Enrichment

1. Data Collection

Implement document- and subdocument-level enrichments like topic, consumer segment, emotion and sentiment. Primary Challenges:

3. Analysis & Insight Generation

1.  Data normalization 2.  Classification 3.  Scalability

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

8

Stage 3: Analysis & Insight Generation Primary Goal: 2. Data Enrichment

1. Data Collection

Connect the dots between a suite of metrics and data points in order to reach sound strategic conclusions. Primary Challenges:

3. Analysis & Insight Generation

1.  Reliability 2.  Strategic Value

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

9

Social media represent a massive compendium of documents…

Approx. Monthly Volume

Furthermore…

Blogs

30 million new posts

On-site comments and social cues and sharing

Facebook

1.8 billion status updates

Social cues (e.g., “likes”) and comments

Twitter

4 billion tweets

Social cues like favoriting and flagging other users

400 million social actions

240 years of video content uploaded each month

Social Media Channel

YouTube

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

10

…and these documents are all of different types.

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

11

Harvesting Data and Metadata from Social Media Documents: A Tweet Dissected

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

12

Harvesting Data and Metadata from Social Media Documents: A Tweet Dissected Datapoints: •  Author Name •  Text •  Publication Date •  Some hashtags

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

13

Harvesting Data and Metadata from Social Media Documents: A Tweet Dissected Metadata: •  Person or tweet that a tweet is in reply to •  Follower count of author •  Times retweeted •  Times favorited •  Author description

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

14

Sorting Social Metadata

A B

Tweets that contain #Ford in the text.

C

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

15

Relevancy as a Sorting Task… Irrelevant Documents

All Social Media Documents

•  Spam •  Documents not in target language (e.g., not English)

All Documents Containing Your Boolean Query

•  Contain keyword but not relevant to client question Relevant Documents

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

16

Data Enrichment: What Should We Measure?

Metric

Explanation

Sentiment

Does the author make a negative or positive point about a product or brand?

Topics

What topic is the author talking about the product or brand in relation to?

Purchase Stage

Has the author of a document already purchased the product when writing about it online?

Consumer Segmentation

What segment is the document’s author from?

Emotions

What emotions do authors express toward the target brand or product?

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

17

Data Enrichment: What Should We Measure?

Metric

Sorting Categories

Sentiment

Positive, negative, neutral

Topics

Pre-selected topic and unexpected topics

Purchase Stage

Before making a purchase or after.

Consumer Segmentation

Young male, middle-aged woman, etc.

Emotions

Joy, anticipation, surprise, fear, etc.

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

18

How can we implement the sorting tasks we’ve discussed so far?

Machine Sorters

Human Sorters

Sorting Tasks Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

19

Q: How do you know when a computer is correct?

A: The same way you know that a human is correct:

“I know it when I see it…”

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

20

Establishing A Basis for How Well Humans Agree With One Another Example 1: Inter-Coder Agreement on Sentiment

Example 2: Inter-Coder Agreement on Emotion

Item

Coder 1

Coder 2

Tweet

Coder 1

Coder 2

1

Positive

Positive

I do not like the cats with thumbs “advert”

Disgust

Anger

2

Positive

Neutral

Trust

3

Neutral

Neutral

I say that video is real, definitely.

No Emotion Expressed

4

Negative

Positive

etc.





Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

21

Using Human Parallel Coding to Establish Gold Standards

Confusion Matrix: Human as Gold Standard

POSITIVE

NEGATIVE

NEUTRAL

TOTAL

POSITIVE

365

24

159

548

NEGATIVE

57

81

65

203

NEUTRAL

274

60

415

749

TOTAL

696

165

639

1500

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

Raw Accuracy: 61.5%

22

Using A Credit Matrix to Create Improved Measurement Credit Matrix

POSITIVE

NEGATIVE

NEUTRAL

POSITIVE

100%

0%

50%

NEGATIVE

0%

100%

50%

NEUTRAL

50%

50%

100% Partial Credit Figure of Merit: 82.3%

Confusion Matrix: Human 1 as Gold Standard

POSITIVE

NEGATIVE

NEUTRAL

POSITIVE

365

24

159

NEGATIVE

57

81

65

NEUTRAL

274

60

415

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

23

But how does the machine learn?

1.  Collection of Human Annotated Data

2.  Machine ingests coded data and finds patterns in each category classification

3.  Machine applies model from step two on raw data. Results are compared to human coding of same material.

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

24

In conclusion…. e case s u r u o about y ch or c i f i c e p •  Get sit market resear – is ring? monito to your k a e p s at trics th s e m t c •  Seleiness question bus e e.g., th ( a t a d f so ntage o e say), but al a v d a •  Takeof what peopl e additional th text a (e.g., most social t a d a t t me tion tha nts contain). a m r o f in e docum a i d e m ll ines wi s, h c a m n at tand th rfect” decisio s r e d n •  U ly make “pe ations and c rare classifi herently e s u a bec them. p are in l e s t h n n e judgm e, but we ca tiv subjec

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

25

Q&A

Ray Poynter VCU, Vision Critical

Jasper Snyder Converseon

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

Thank you Jasper Snyder, VP, Converseon [email protected]

Speaker: Jasper Snyder, Converseon NewMR Social Media Research Event, 9 October 2012, Session 2

© 2011 Converseon Inc. Proprietary and Confidential

27

A  Presenta*on  from  the  NewMR  “Social  Media   Research”  Event   9  October,  2012  

Enriching  Social  Data  for  Market  Research     Jasper  Snyder,  Converseon  

All  copyright  owned  by  The  Future  Place  and  the  presenters  of  the  material   For  more  informa:on  about  NewMR  events  visit  newmr.org