Interview: Josh Hemann, Activision on Why the Tolerance for Ambiguity is Vital
We discuss handling bias in data, other data quality concerns, advice, desired qualities, and more.
Josh Hemann is the Director of Analytic Services at Activision where his team builds data tools to support video game studios and embed analytics within the games they create.
Prior to this his industry experience spanned diverse settings such as air pollution research, aerospace, retail loyalty programs and recommendation systems for grocers.
Josh has an MS in Applied Mathematics from the University of Colorado at Boulder.
First part of the interview
Here is second and last part of my interview with him:
Anmol Rajpurohit: Q7. One of the most common issues is with data is bias. How do you identify and mitigate bias?
Josh Hemann:
I am reminded of the quote by John Tukey, "The best thing about being a statistician is that you get to play in everyone else's backyard." Identifying bias requires knowing something about the backyard you are in, being connected with the domain experts you are working with.
So you have to understand the data generating process, whether it’s by a machine or by nature, but you also have to understand how data is actually measured and persisted. In my work, it is the measurement and persisting steps that often introduce the most insidious biases.
AR: Q8. Besides bias, what are the other data quality issues you observe very often?
AR: Q9. What do you mean by "never show a picture without a statistic, never show a statistic without a picture"?
JH: This is advice I got years ago from Dr. Jeffrey Luftig while taking a statistics course from him. It means that it is easy to be misled by a single statistic or a pretty plot. We need data visualizations to validate the inferences we draw from statistics and likewise, we need statistics to validate the natural inferences we make when looking at data visualizations. I recently wrote about this topic here.
AR: Q10. What is the best advice you have got in your career?
JH: No one has told me this advice directly, but I have learned it by watching how
There is a lot of building of trust and quite frankly, selling, that is necessary to do analytical work. The results never simply speak for themselves, as I once thought they did.
AR: Q11. What key qualities do you look for when interviewing candidates for positions on your team?
JH:
Beyond being skilled in a particular area, I try to assess people’s tolerance for ambiguity and willingness to tackle poorly defined problems.
At least in our current world that tolerance needs to be high because these games are so complex, take hundreds of people multiple years to develop, and operate at huge scale. Mapping a business goal to statistical code that has to be executed in game operations can be horribly frustrating at times and you have to have a personality that can be OK with that and continue to progress.
AR: Q12. What are your favorite books and other resources on visualization? What do you like about them?
There is an old book titled Fundamentals of Interactive Computer Graphics by J.D. Foley and A. Van Dam that I don’t turn to much lately but was important when I was first doing this type of work. It covers lots of, well, fundamentals about coordinate transformations, view clipping, rendering 3D objects, etc. I find it helpful that I have some familiarity with these topics when I run across odd behavior in some software tool or an obtuse API in a graphics library, because I have an intuition for what must be going on behind the scenes regardless of how much the particular tool abstracts away the complexities.
AR: Q13. On a personal note, what keeps you busy when you are away from work?
Related: