Interview: Antonio Magnaghi, TicketMaster on Why Honesty is Key for Analytics Success

Antonio Magnaghi is Vice President, Data Sciences, at Ticketmaster, one of the World's largest eCommerce sites.

At Ticketmaster Antonio is leading key machine learning initiatives on recommendations, predictive user modeling, forecasting and large-scale distributed systems for real time optimization problems. Antonio worked on algorithmic content production and marketplace design at Demand Media. He acquired an extensive background in on-line advertising at Yahoo!/Yahoo! Research and Fox Audience Network/The Rubicon Project. Antonio conducted research on IP networks and network data mining while at Microsoft and Fujitsu Laboratories of America.

He holds four patents and a Ph.D. from the University of Tokyo, Japan.

First part of interview

Here is second part of my interview with him:

Anmol Rajpurohit: Q7. What were the key lessons from your experience of designing and implementing Lambda Architecture?

Antonio Magnaghi: In my experience, being able to implement and deploy our recommendation platform in an incremental manner has been crucial. This has been a key feature that we have exploited about lambda architectures. This allowed us, firstly, to deliver the batch portion of it. This powers email marketing channels and provides personalized recommendations. Once the batch portion of the platform was in place, we decided to move forward and deploy the (near) real-time component of such an architecture to customize the user experience on the site. The ability to serve these two channels (email and web-site) that are so different in terms of SLA's and overall requirements is an example of the versatility and power of lambda architectures. Because this design enabled us to deliver incrementally, data mining and learning exploration benefited as we could start to gather data early on in the process.

AR: Q8. Personalization and Recommendation systems have existed for decades. How have they been impacted in the last few years by the Big Data revolution?

AM: Recommender systems are one solution to a very common problem, that, in more general terms, can be cast into matching vertices in a bipartite graph in a manner optimal w.r.t. certain objective measures. Over the past decade, we have observed a greater appreciation for these types of algorithms and their efficacy. Several years ago, technologies required to build a full-fledged recommender system were not as mainstream as they are now. Currently, we have reached a tipping point and more and more companies understand how strategic it is to have full control on those technologies that have such a large impact on user experience, user retention and product discovery. I would expect this trend to become even more pronounced in the near term.

AR: Q9. What is the best advice you have got in your career?

AM: I have been extremely fortunate to work with and be mentored by some exceptional individuals in the field of ML and computational modeling; individuals whom I consider pioneers in this field and whom I am lucky to call friends now.

One of the most valuable lessons has been to always be completely honest and transparent about data and what emerges from it. This requires commitment to communicate with integrity and evangelize what ML can do and, equally importantly, cannot do. This requires us to constantly strive to remove our own biases and expectations. In my view, this is critical in order to foster learning and discovery.

AR: Q10. What trends do you expect to dominate the field of Personalization and Recommendation over the next 2-3 years?

AM: Recently, deep learning has been enjoying a lot of excitement due to fundamental theoretical breakthroughs matched by improved practical results. An increasing number of domains have benefited from this. I would expect this would be the case for recommendation systems as well. Similarly, I am quite excited to see recent work published about structured learning and contextual bandits and would expect ramifications in the field of personalization.

AR: Q11. What key qualities do you look for when interviewing for Data Science related positions on your team?

AM: The drive to "get dirty" with the data, the desire to handle data sets from the inception of the process and to learn from the data. In my experience, there is a really a lot of value in directly handling data yourself. It makes you aware of how, sometimes, this task can be challenging. This may inspire you on how to develop a better experimental setup where access to high quality data is more direct. If you have the desire to understand real-world data sets, other important, but somewhat ancillary, aspects take care of themselves. You may not have a lot of experience with Hadoop, Storm or other similar frameworks, however, if "you are on a mission" to understand and use data, you will have the right motivation to learn such technologies and be successful.

AR: Q12. Which book (or article) did you read recently and liked? What keeps you busy when you are away from work?

AM: I had bought Boosting by R. Schapire and Y. Freund a while back but did not have a chance to read it until recently. It is a great book from authors who have made seminal contributions to this important learning framework. The book is a very complete presentation. In my free time, I still enjoy coding and working on fun projects.

Related:

• Sastry Malladi, StubHub on Designing Big Data Architecture for the Unknown Future
• Ted Dunning, MapR on The Real Meaning of Real-Time in Big Data
• Mark Weiner, Temple University Health System on Addressing Healthcare Data Gaps through Advanced Simulation