Gaming Analytics Summit 2014: Day 1 Highlights
Highlights from the presentations by Gaming Analytics experts from Activision, Valve, Microsoft and Broken Bulb Studios on Day 1 of Gaming Analytics Summit 2014.
Across many industries, large and small organizations are using analytics and data science to offer greater insight and customer service. The gaming industry is almost well placed in that - particularly with online and social gaming - the companies already keep a vast amount of data on gamers. The challenge remains to make use of this data in a way that offers true value for money whilst enhancing the user experience.
I had recently attended the Gaming Analytics Summit 2014 (May 1-2, 2014) at San Francisco, CA, organized by the Innovation Enterprise. The summit brought together acclaimed speakers and attendees for deep insight into how the gaming industry uses analytics and data science. For speakers, it had a line-up consisting of 20+ leading executives working in Analytics, Data Science & Business Intelligence in gaming. Through real-life business case studies and deep-dive discussions, the summit offered solutions and insight from the leaders in the Gaming space.
To help its readers succeed in their Analytics pursuits, KDnuggets provides concise summaries from selected talks at the summit. These concise, takeaway-oriented summaries are designed for both – people who attended the summit but would like to re-visit the key talks for a deeper understanding and people who could not attend the summit. As you go through it, for any session that you find interesting, check KDnuggets as we would soon publish exclusive interviews with some of these speakers.
Here are highlights from selected talks on day 1 (Thu, May 1):
Arthur Von Eschen, Senior Director, Game Analytics at Activision talked about “Boosting Detection in Call of Duty”. For non-gamers, Boosting is defined as the act of two or more players collaborating through dishonest methods in ranked multiplayer lobbies to gain in-game rewards, such as camouflages. He gave examples of detecting unwanted behavior such as cheating, boosting, etc.. Boosting is most frequent of the unwanted behaviors and is most diverse in terms of patterns.
The problem is that it:
His solution approach included designing around the problems (such as constraints design, too smart players) and building an analytic service to detect boosting. Talking about analytics application, he mentioned it should employ many analytics models, be decision centric and capable of running at scale. Boosting detection service is similar to fraud detection systems. It is basically a classification problem which can be solved through semi-supervised learning on training data and anomaly detection. The major objective of modeling is to reduce false-positives.
After trying a lot of approaches, the gradient boosting machine (GBM) aka boosted trees was found to be the most optimal choice. Modeling wasn’t the most difficult part; it was Scaling, involving database optimization, query queue with query weights and jobs running in parallel even for same model. The results of boosting detection service provide data to studios to facilitate decision-making.
Mike Ambinder, Experimental Psychologist at Valve Software discussed about Valve’s approach to the acquisition, collection, and interpretation of data across its products and services.
Talking about decision-making at Valve, he mentioned the following key aspects:
He also noted that the decision making process is explicit, data-driven, theory-driven, iterative and based on measurable outcomes. He mentioned that it is important to define the questions first and thereafter think of designing the data schema. Next, he described Operational Game Stats (OGS), a platform used for recording gameplay metrics such as kills, deaths, hero selection, in-game purchases, bullet trajectories, etc. Organizational schemas are defined for each game and data is sent at relevant intervals. He concluded his talk with case studies where the insights obtained from an iterative loop of hypothesis and feedback were used to change game design, delivering significantly better user experience.
Grant Hogan, Senior Program Manager – Data Intelligence at Xbox Live Service Delivery, Microsoft talked about aggregating data to effectively run a healthy live service in real time. Discussing about operationalizing data to promote service health, he mentioned that Xbox LIVE service hosts 30,000,000+ unique players and targets availability of 99.950 service uptime availability for those customers. That’s only 22 minutes of “acceptable” unavailability each month. In order to achieve such a level of service availability, service must be monitored and cared round the clock. The service is spread across many thousands of servers most of which have very specific roles within the service.
Next, he discussed intelligence in scaling-capacity planning and elastic utilization of services. Discussing about elastic services and fluctuating capacity, he mentioned that uneven service utilization during holidays, title releases, and special events needs to be predicted and planned in advance. Data is also used to protect the service from bad actors. One of the challenges was to deal with 2.5 million gamertag complaints received in past 12 months. A “trust score” was defined for “good” gamers (based on age, past activity, level achieved, frequency, social factor, etc.), in order to automate the correct identification of when the “offensive” tag for any gamer is legitimate.
He emphasized that “an engaged community is an asset” - community ambassadors utilize the service more frequently, more broadly and understand the entire service offering more completely than other customers.
Robert Nelson, CEO, Broken Bulb Studios shared very interesting and insightful lessons from building games on Facebook and mobile. The rise of freemium social games happened first on Facebook. At that time, the best way to get players to spread the word about your game was to bombard them with pop ups, share prompts, and requests only loosely tied to your game. As the Facebook platform matured and freemium reached mobile devices, players began to disconnect from these more forced mechanics and become more responsive to visible but more passive approaches.
With over 2 million monthly players, Broken Bulb has earned considerable success without any funding, simply through high-quality games and strong metrics. He outlined the following suggestions:
He concluded the talk saying “Social is for Engagement not Acquisition” i.e. people come from word of mouth and not from mere social posting.
Highlights from selected talks on day 2 (Fri, May 2)
Related:
I had recently attended the Gaming Analytics Summit 2014 (May 1-2, 2014) at San Francisco, CA, organized by the Innovation Enterprise. The summit brought together acclaimed speakers and attendees for deep insight into how the gaming industry uses analytics and data science. For speakers, it had a line-up consisting of 20+ leading executives working in Analytics, Data Science & Business Intelligence in gaming. Through real-life business case studies and deep-dive discussions, the summit offered solutions and insight from the leaders in the Gaming space.
To help its readers succeed in their Analytics pursuits, KDnuggets provides concise summaries from selected talks at the summit. These concise, takeaway-oriented summaries are designed for both – people who attended the summit but would like to re-visit the key talks for a deeper understanding and people who could not attend the summit. As you go through it, for any session that you find interesting, check KDnuggets as we would soon publish exclusive interviews with some of these speakers.
Here are highlights from selected talks on day 1 (Thu, May 1):
Arthur Von Eschen, Senior Director, Game Analytics at Activision talked about “Boosting Detection in Call of Duty”. For non-gamers, Boosting is defined as the act of two or more players collaborating through dishonest methods in ranked multiplayer lobbies to gain in-game rewards, such as camouflages. He gave examples of detecting unwanted behavior such as cheating, boosting, etc.. Boosting is most frequent of the unwanted behaviors and is most diverse in terms of patterns.The problem is that it:
- Ruins integrity of leaderboards and competitions
- One of the top reasons players cite for why they quit playing multi-player
- Over 40% more likely to drop out of match early when boosters present
His solution approach included designing around the problems (such as constraints design, too smart players) and building an analytic service to detect boosting. Talking about analytics application, he mentioned it should employ many analytics models, be decision centric and capable of running at scale. Boosting detection service is similar to fraud detection systems. It is basically a classification problem which can be solved through semi-supervised learning on training data and anomaly detection. The major objective of modeling is to reduce false-positives.
After trying a lot of approaches, the gradient boosting machine (GBM) aka boosted trees was found to be the most optimal choice. Modeling wasn’t the most difficult part; it was Scaling, involving database optimization, query queue with query weights and jobs running in parallel even for same model. The results of boosting detection service provide data to studios to facilitate decision-making.
Mike Ambinder, Experimental Psychologist at Valve Software discussed about Valve’s approach to the acquisition, collection, and interpretation of data across its products and services. Talking about decision-making at Valve, he mentioned the following key aspects:
- No formal management structure
- Decision-making is a meritocracy
- All data is available to every employee
- We just want to make the best decisions possible
- We don’t want to rely on ‘instinct’ -> it is fallible
He also noted that the decision making process is explicit, data-driven, theory-driven, iterative and based on measurable outcomes. He mentioned that it is important to define the questions first and thereafter think of designing the data schema. Next, he described Operational Game Stats (OGS), a platform used for recording gameplay metrics such as kills, deaths, hero selection, in-game purchases, bullet trajectories, etc. Organizational schemas are defined for each game and data is sent at relevant intervals. He concluded his talk with case studies where the insights obtained from an iterative loop of hypothesis and feedback were used to change game design, delivering significantly better user experience.
Grant Hogan, Senior Program Manager – Data Intelligence at Xbox Live Service Delivery, Microsoft talked about aggregating data to effectively run a healthy live service in real time. Discussing about operationalizing data to promote service health, he mentioned that Xbox LIVE service hosts 30,000,000+ unique players and targets availability of 99.950 service uptime availability for those customers. That’s only 22 minutes of “acceptable” unavailability each month. In order to achieve such a level of service availability, service must be monitored and cared round the clock. The service is spread across many thousands of servers most of which have very specific roles within the service.
Next, he discussed intelligence in scaling-capacity planning and elastic utilization of services. Discussing about elastic services and fluctuating capacity, he mentioned that uneven service utilization during holidays, title releases, and special events needs to be predicted and planned in advance. Data is also used to protect the service from bad actors. One of the challenges was to deal with 2.5 million gamertag complaints received in past 12 months. A “trust score” was defined for “good” gamers (based on age, past activity, level achieved, frequency, social factor, etc.), in order to automate the correct identification of when the “offensive” tag for any gamer is legitimate.
He emphasized that “an engaged community is an asset” - community ambassadors utilize the service more frequently, more broadly and understand the entire service offering more completely than other customers.
Robert Nelson, CEO, Broken Bulb Studios shared very interesting and insightful lessons from building games on Facebook and mobile. The rise of freemium social games happened first on Facebook. At that time, the best way to get players to spread the word about your game was to bombard them with pop ups, share prompts, and requests only loosely tied to your game. As the Facebook platform matured and freemium reached mobile devices, players began to disconnect from these more forced mechanics and become more responsive to visible but more passive approaches.
With over 2 million monthly players, Broken Bulb has earned considerable success without any funding, simply through high-quality games and strong metrics. He outlined the following suggestions:
- Be less desperate (if your game has quality, you will get users eventually)
- Opt-in works a lot better than opt-out (a powerful choice)
- Deliver enhanced experience through social connections (while assuring them of no bad activities – for example: “we will never post on your behalf”)
- Everyone’s a closet bragger (people want to brag only to those they love)
He concluded the talk saying “Social is for Engagement not Acquisition” i.e. people come from word of mouth and not from mere social posting.
Highlights from selected talks on day 2 (Fri, May 2)
Related:
Learn Deep Learning with this Free Course from Yann LeCun