Posted by:Shyam Deval July 20th, 2016

You read that right – it’s not Sherlock saying it this time! IBM Watson’s cognitive computing capabilities and superhuman intelligence stands it in good stead to switch roles with the legendary fictitious detective Sherlock Holmes who could crack the most mind-boggling of mysteries, find answers to the unanswered questions everyone had and draw insights from information in a way no one could even fathom!

For those of you who aren’t yet familiar with IBM Watson, it is a technology platform that uses natural language processing and machine learning to reveal insights from large amounts of unstructured data. It has refined cognitive abilities to quickly understand context, learn from experience and draw inferences and insights from a sea of information that would otherwise be humanly impossible to wade through. This is way beyond mere data-matching or search engine functions – Watson uses advanced ways of inferring context, meaning, implications and fallouts of the voluminous content it indexes in its operations.

Given such abilities, the possibilities of what IBM Watson can do in every industry, are limitless! In today’s post, we take a look at the high-level process used by IBM Watson as it understands complex questions and finds answers for them and helps draw insights from massive volumes of data. The key here is that all this data that we are talking about could be highly unstructured (think random social media posts, news bytes, research journals, uploaded files of any format and so on) but Watson succeeds in making sense of it all and that is what makes it such a futuristic and promising technology!

  • Send Watson to school! #CorpusCreation
  • Well, not literally but you would need to help Watson educate itself! If you want to set up any application that uses Watson as a master of a certain subject or field of knowledge, you first need to help Watson educate itself – and mind you, this is no easy education. In Watson lingo, it means digesting humongous amounts of information on the possibly wide scope of the field. Information could be fed to Watson in n number of formats – web pages, documents, PDFs etc. This is referred to as providing the ‘Corpus’ of data and is a first step to training Watson.

  • Oversee Watson’s study! #ContentCuration
  • As Watson educates itself and makes itself familiar with the field, it is basically building up an expert level of background or contextual knowledge on the subject. Human experts must help in this process by ensuring that irrelevant, outdated or poorly written information is not fed into the corpus. This sort of content curation is extremely important to make sure Watson learns all that is relevant and useful – just like how we set the syllabus for a student at school!

  • Getting a grip on things #Pre-processing #Ingestion
  • Watson uses its god-gifted (or in this case IBM-gifted) abilities to make sense of all the volumes of data we feed it with. It uses artificial intelligence to ‘understand’ what is being said in all the data dumps that we hurl at it and equips itself with a reasonable understanding of the field or subject. It is now familiar with all the jargon used in the field, what those phrases mean and how they relate to each other. Based on this, it tries to ‘connect the dots’ by means of links and indices, and creates self-assist tools such as knowledge graphs to aid in providing answers to questions.

  • How do you train Watson’s brain #MachineLearning #NaturalLanguageProcessing
  • Now that Watson has a solid grounding of knowledge in the field, we need to further train it to help it interpret information, spot patterns and trends and attain a superior ability in providing responses to questions with high ‘levels of confidence’. A ‘level of confidence’ is how correct Watson thinks an answer is based on the scientific evidence backing it has found it to have. So how do we train Watson to give ‘confident’ answers?

    Training data is uploaded to Watson in the form of Questions and Answers. Human experts identify pairs of Q&A that can be fed into Watson to help increase its awareness on topics in the field. Not all possible questions need to be covered in this – Watson just uses the provided sample to get better at answering any question you may throw to it, with a high level of confidence. By studying these Q&A, Watsons learns to deduce linguistic patterns in the meaning of the phrases/terms, as used in the domain. It uses this knowledge along with its own scoring algorithms to arrive at the most likely accurate responses to any question it is faced with. How good it gets will obviously depend on the quality of Q&A you feed it with!

  • Solving complex problems in a real-world #WatsonApplications
  • Watson is now ready to give you solutions to complex real-world problems/situations and answers to baffling questions with a ‘Didn’t you know that? It’s quite elementary!’ air about itself! Basically, now Watson is ready to take on any question in the field and return you a set of responses with varying confidence levels and in many cases you could get 100pc accurate answers! It is also ready to start running analytics and drawing insights into all the terabytes of unstructured data that you have been struggling to make sense of. So it is a powerful search engine, intelligent answer-finder, curated content warehouse and a heavy-lifting analytics tool – all rolled into one!

  • Getting better and learning from experience #StayingUptoDate #ExperientialLearning
  • Just like a human SME would focus on keeping herself up-to-date with progress in the field and improves her logical and analytical thinking as she gains more experience and knowledge, Watson also has ways to keep getting better with what it does. You can help Watson with this by frequently updating the corpus with the latest and most relevant sources of information in the field. Watson also continues to learn with ongoing interactions with users, its success rates and understanding of user behavior. Interactions are also reviewed by humans and fed back to Watson to help it learn faster and mold itself to the often-evolving demands of the business.

    So that was a peek into what you can do with your huge warehouses of information and Watson. A lot of businesses can undergo transformation and present new-age conveniences to their consumers by leveraging the abilities of IBM Watson. Features such as secure cloud storage and several pre-defined Watson APIs (that deal with Language, Vision, Speech and Data – will talk about these in an upcoming blog) only make the whole process of using IBM Watson that much easier.

    Bring us a real-world problem you are trying to solve and we can help you design and build an app that makes good use of the Watson platform. Leveraging Watson can help your application deliver results in a few seconds or minutes as compared to the months it would take you to do something similar manually! But what is even more impressive is that it actually uses a high degree of human-like intelligence to do this – and that makes it the cognitive computing tool to look out for to deliver path-breaking and industry-disrupting applications.

    Like they say, what will YOU build with Watson?!

    ibm_watson_avatar_pos

    Here is how Internet of things is transforming healthcare.Download the infographic to know more!

    Leave a Reply

    Your email address will not be published. Required fields are marked *