Data Science and Ethics – Why Companies Need a new CEO (Chief Ethics Officer)

We explain why data science companies need to have a Chief Ethics Officer and discuss their importance in tackling algorithm bias.

By Kate Strachnyi, Story By Data

New CEO Figure 1We live in a time, where individuals and organizations can store and analyze massive amounts of information; with over 2.5 ­quintillion records of data created every day.  People are being defined by what they search for on the internet, what they eat, where they travel to, where they hold membership accounts, etc. Organizations and individuals can leverage this data to uncover powerful findings. Data scientists/analysts often get lost in the techniques and methods of the trade. In doing so, they can forget to ask important questions such as: Who will be affected by the work? How we are ensuring that, by doing ‘good’ for one group, we are not inadvertently harming another? Who actually owns the data that we are working with? What is the protocol for using that data to make business decisions?

That’s not my job

Ethics is something that isn’t talked about enough in the data science realm; though ignoring it may have implications on several people (e.g. bias in algorithms). At a conference that I attended a few weeks ago, we had a roundtable discussion about ethics in data science/ artificial intelligence. One of the members at my table, the head of marketing for a large corporation, was asked about her views on data ethics. Her response was along the lines of “well, that’s not really my responsibility; I hope someone else is thinking about it”. Another member of the roundtable, from technology, also said “that’s not really my job to think of ethical implications of the actions of the company”. That struck a chord. I strongly believe it’s everyone’s responsibility to consider the ethics of their actions; especially as the actions may impact others around them. One such example of where a lack of attention given to ethics can lead to negative results lies within algorithm bias.

Algorithm bias

People assume that a computer can’t possibly be biased, prejudice or harbor stereotypes…right?

WRONG! While the techniques may be non-biased, once we feed in the data, assumptions, etc., there’s a high chance of bias appearing in the model; because these are conditions set by humans, who may project their biases into the results, even if they weren’t intending to do so. Cathy O’Neill provides some really great examples of different types of bias that impact our society today. As stated in her book, Weapons of Math Destruction,

“when we blithely train algorithms on historical data, to a large extent we are setting ourselves up to merely repeat the past. If we want to get beyond that, beyond automating the status quo, we'll need to do more, which means examining the bias embedded in the data. The data is, after all, simply a reflection of our imperfect culture.”

New CEO Figure 2

A key example that O’Neill writes about is how recidivism (the tendency of a convicted criminal to re-offend) risk algorithms are inherently discriminatory. She states that when you have a dataset that has a “favored” group of people and a “discriminated” group of people, and you’re deciding on an outcome that has historically been awarded to the favored group more often – in this case, it would be a low recidivism risk rating – then you cannot expect to maximize accuracy and keep the discrimination down to zero at the same time. This is likely not an intentional outcome but does highlight the need for more awareness around ethical implications of models/ algorithms.

Need for a Chief Ethics Officer

Inappropriate use of data science methods can lead to erroneous conclusions. Those that work with data have a lot of power; and with that power, comes great responsibility! There is a need for a new CEO – a Chief Ethics Officer who won’t have the option to say “it’s not my job to think about ethics”. The role might already exist in some companies in the form of a Chief Compliance Officer, Chief Privacy Officer, Chief Risk Officer, etc. If that’s the case, then the responsibilities of these roles need to be officially defined to include the monitoring of ethical implications.

Bio: Kate Strachnyi is the author of The Disruptors: Data Science Leaders and Journey to Data Scientist and currently working on a few new books: “Mothers of Data Science”, “Data Literacy for Kids” and Vol II of The Disruptors: Data Sciencce Leaders.