XAI – A Data Scientist’s Mouthpiece

We outline the usefulness of Explainable AI, which allows you to explain the results of a multidimensional model - including having a multimodal decision boundary - to a business user.

By Adurthi Ashwin Swarup, Digite Inc

Data science is not just about solving business problems mathematically, but it is also about telling a story to stakeholders. It is a joy when one can draw out the “OOHs” and “AAHs” as mental bulbs warmly glow into existence as the results of an analysis are understood. More often than not, such storytelling is not possible when one bakes algorithmic outputs into products.

Data Scientist Mouthpiece

Having worked on “data- science products” throughout my career, I have often found that the sale of such products goes through when the product can guide the user through the story of the analyses it presents. But how does one get the personified product to explain the results of a multidimensional model, having a multimodal decision boundary, to a business user? Enter XAI ( eXplainable Artificial Intelligence).

Touted as the third wave of AI, it was ‘jargoned’ into existence to reduce the possibility of  justifying consequential decisions using “the algorithm made me do it”. The gravity of the problem comes to the fore as in the case of Microsoft’s twitter bot - Tay which learned to spout racist tweets) .

(Why the third wave of AI ? - refer to this interesting video by John Launchbury under the aegis of DARPA : https://www.youtube.com/watch?v=-O01G3tSYpU )

Algorithms are becoming ubiquitous and their outputs are being used by the society at large and without understanding their inner workings . How does one prevent bias from creeping into algorithms? If it does creep in, how does one find out and nip the problem in the bud without waiting for the algorithm to start actively discriminating or giving unintended results?  The folks over at FAT-ML (Fairness and Transparency in Learning) have taken up this noble cause.

While my reason of “increasing data science-product sales” may not be as noble, the use of XAI to explain algorithmic results to the ‘humans-in-the loop’ is completely in alignment with the aforementioned group .

Where does the eXplainability come from ?

The explainability in  XAI comes from a set of techniques that add contextual adaptability and interpretability to machine learning models. I would combine the two terms and call it “contextual reasoning”.

Peace sign

There is no formal mathematical definition for this term, but one may say that it is the ability to come to a conclusion from  concepts (like a human) rather than by just using probability.

For example, if one were asked to assign a reason why two fingers ( like in the picture alongside) have been raised - A human may assign a reason looking at the burger joint in the background or the black and white photos of soldiers grinning on the battlefield .

An algorithm on the other hand would calculate the probability distribution of what the two raised fingers mean from a large collection of training photographs  of people holding up the “v”-sign from the internet and then associate a reason based on this distribution.

So over a large number of cases , the algorithm may prove correct, but there would come a case of a soldier holding up the same sign at a burger joint that could cause problems because the algorithm would get confused because it associates soldiers holding up signs to victory.

If a human were to understand that the algorithm was biased by the large number of world war II photographs on which the model was trained, then he would make effort to circumvent such conclusions .

In fact, this would  also help  the data-scientist who trained the model, pointing him towards the need to de-bias the training photographs. The story does not end here however. Hoping that my readers have understood contextual reasoning, I bifurcate context into two subtypes - local context and global context .

Here is another example that illustrates the difference between the two. The highest temperatures ever recorded may not be worry some in the middle of the Sahara desert ( local context) but is definitely concerning if its so across the globe ( global context ….yes i did that ) .

The set of techniques under XAI can be grouped by the kind of contextual information they provide. By no means are these techniques new. Some of them like “global feature importance have been around for quite a while, but it is certainly new that data scientists  can combine these techniques to enable an algorithm explain itself in their absence .

In the next part of this blog series, I will delve into the kind of techniques covered under XAI



Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models Wojciech SamekThomas WiegandKlaus-Robert Müller


The author has a decade of experience leading data science teams in startups and larger organizations. His educational background is in Robotics, Physics and Finance.  He is currently working as the Director of Data Science at Digite Inc. implementing an XAI pipeline for their product RISHI – XAI for Enterprise Project Intelligence. He is a regular blogger and has spoken at various conferences.

Bio: Adurthi Ashwin Swarup is the Director of Data Science at Digité Inc. He brings with him a decade of experience in leading data science teams. He has worked with both startups and larger organizations to solve data science use-cases across industry verticals. His educational background revolves around physics, robotics, and finance. He blogs, speaks at conferences and often works on distributed algorithms for GPUs.