
I attended a live webinar yesterday with Stephen Wolfram, who is behind the development of the Wolfram Alpha knowledge engine.
Wolfram Research will launch the tool in just a few weeks time. It's hard to describe, but unlike Google it does not trawl the web for weblinks in response to a query. Instead it tries to turn your question into a computation, and then applies that to the knowledge it has available to it.
For example a query "fish production france/poland" would assume you wanted to compare the production of the two countries. WA would access statistics it has on fish production and then create a graphical analysis over time for you.
The key difference to traditional search is that the content is being created for you in the few seconds after you press ENTER. It's not finding a graph in an existing document or website and linking you to it. So it could be the very first time anyone has ever tried to make that particular computation.
Wolfram Alpha is not going to work where questions don't have absolute answers - you need to be able to represent a question using computational terms, and obviously many questions won't fit that model.
Wolfram Alpha attempts to make up for this by using a side bar which will present results from other sources - such as Wikipedia if it knows nothing about the subject.
The system uses your IP address to make assumptions about the data and type of computation you may require. For example a query for "Richmond" in the United Kingdom would assume you wanted a statistical profile for the London Borough of Richmond, but in Virginia, it would assume you probably wanted a different area entirely. The results however give you options to clarify your query, so if I am in London but actually want to know about Richmond Virgina, I can easily select that option and the results change accordingly.
Stephen described the "four pillars" of Wolfram Alpha:
1. Data Curation - Wolfram has developed a methodology for curating data. They have a pipeline of data which comes to human experts, who correlate sources to produce reliable, computable data. 200 people are now working on the project. The topics available continue to grow on a daily basis.
2. Computation - The team has attempted to make accessible all known algorithms, formulae in the system. It now comprises 5 to 6 million lines of mathematica code.
3.Natural Language Processing - Wolfram Alpha carries out free form linguistic analysis to convert a question into a computation. It needs to be able to cope with different ways of expressing the same query, different spellings, short utterances, and ambiguity. For example does a query "11/7" mean a date or a division? The aim is that people can use it without having to learn analytical techniques.
4. Presentation of Results - The most important part is how the data is presented back to the user, so Wolfram Alpha prioritises elements of the data and decides which are the most appropriate graphics to render in relation to the query.
Stephen concluded that we have learned to compute lots of things in this world, but in the past carrying out such computation required expert analytical input. Now Wolfram Alpha will make it possible for the everyday web user to be an effective reference librarian/analytical scientist, performing analysis immediately and intuitively online.
There will be a free site, a professional subscription based site, where you could upload your own data to the system and make use of more advanced features. There is also the potential for corporations to buy services to apply the engine to their own corporate knowledge.
There will be a collection of APIs available which will allow users to use Wolfram Alpha or embed "pods" of information on their own sites, or even request data from the system to use in another website.
This could have enormous implications for the way in which both public and organisational data can be turned into knowledge by everyday users. It's clearly a very powerful tool, but I suspect its full potential won't be realised until the world is let loose on it in May.
Photo credit: Hybernaut