Survey Data - free text fields

[ngulliver]

I have been looking at some Survey/Questionnaires which contain a number of fields which appear to be free text fields (i.e. the field can be populated with an open ended response).

Obviously, it is difficult to build statistical information from such fields. Is there a good way of getting and displaying any meaningful results with this kind of field ?

Regards,

Neil

[swuehl · Accepted solution]

Maybe match your data to a dictionary of keywords (assuming you have some a priori knowledge of some keywords)?

Or create a word  / tag cloud (there are some samples here in the forum, using QV expressions only or an extension)?

Or use an external tool for more complex text mining (e.g. sentiment analysis)?

[Anonymous]

Neil

How about splitting the free text fields into their constituent words, adding a count field [or use count distinct, see the recent blog, A Myth About Count(distinct …) ].

Whack it into a pivot table and display words used ordered number of times used.

SurveyData :

LOAD * Inline [

  DataString

  The brown cat is the best one

  The white cat is clever

  I like both cats

  They both chase mice

  I love white cat.  White Cat!!  White Cat!!  White Cat!!  White Cat!!  White Cat!!

]

;

DataWords :

LOAD

1 as [DataWordCount] ,

lower ( SubField(DataString, ' ') ) as DataWord

resident SurveyData

;

Best Regards,     Bill

[Jason_Michaelides]

Difficult. In the absence of a clever reg expression or algorithm to parse the string, can you use WildMatch() or similar to at least look for key words or phrases and categorise the response this way?

[ngulliver]

I suspect that with the increase use of sentiment analysis from Social Media it will become more common to deal with issues such as this.

Many thanks for all your contributions.

Neil