I meant to blog at greater length about the ICWSM, which was very enjoyable and instructive. One often finds oneself in that “shall I write about this, or just do it?” dilemma. So I just did it … got talking to a number of clever people about the problems that they and we are experiencing deploying software to do these complicated things we do, and about their solutions and ours.

Things I learnt:

UMBC in Maryland under Tim Finin have a kick-ass team of graduate students. Much respect to Pranam and Akshay …
Twitter is great!
There are some cool betas doing sentimenting for blogs and news (check out textmap)
No one has cracked sentimenting satisfactorily for message boards
As message boards represent c. 85% of the volume of commentary of most of the brands we work for, this remains an issue …

We are a lot closer to solving a number of knotty problems as a result of the conference, though. Congratulations to Nicolas Nicolov of Umbria, Natalie Hurst of Nielsen BuzzMetrics and Matthew Hurst of Microsoft Live Labs inter alia for organising it.

A lot of the presentations at the ICWSM in Boulder are about extracting opinions from the web.

Opinions from respected sources have a lot of value commercially. They are likely to figure largely in the natural search results for a product category, as anyone who has searched (e.g.) for “best digital camera” will note. So market researchers and camera-buyers alike have a keen interest in what opinions have been expressed. Blogs are of particular interest here, because the format (a personal diary) seems to encourage the expression of opinion.

What is that characterises an opinion? Can one look for tell-tale words like “believe”, or “think” or “hate”? Is it sufficient to look for content which is self-selectingly opinionated, such as reviews? And when one has found an opinion, are there ways of training a program to analyse it for sentiment? How would a search interface communicate what constituted a reliable source? A number of impressive University-based teams are looking at capturing the low-hanging fruit of opinion online - searching sites which are dedicated to opinion (e.g. review sites) and looking for clear value judgements about a product or topic. Gilad Mishne of the University of Amsterdam talked about the utility of having an operator “opinion:” which could be used in the context of search.

But what about hidden opinions, opinions which are expressed by a selective reporting of facts, or by the way that facts are highlighted within a document? This kind of opinion-tracking would require that the program had a template of the “facts” in relation to a topic or product, and could score an article based on which of the “facts” the writer highlighted or omitted.

Mark Rogers will be attending the International Conference on Weblogs and Social Media in Boulder, Colorado between 25th-28th March.  And blogging about it if a broken arm permits!  His GSM phone is +44 7866 369181 if you would like to catch up.