King: Playing to win with Google Cloud

In the world of mobile and social gaming, it takes more than a good idea for a games company to stand out from the competition. King , the makers of Candy Crush Saga and hundreds of other games, has achieved success in its market space with a focus on bite-sized entertainment, and a blend of art and science. “We have great game designers of course, but we’re also a very data-driven company,” says Åsa Bredin, FVP (First Vice President) Technology at King. “We try to validate our design with data to see how engaging a game is, and whether it’s at the appropriate difficulty.”

With 270 million players a month (as of Q2 2018), King operates at a scale that few other companies can ever hope to achieve, with its data archives in double-digit petabytes. Recently, the company had started to ask themselves whether a monolithic on-premises Hadoop environment set them up to tackle future challenges. Concerns centered around a future in which public cloud is a big part of the IT landscape, and when much of the innovation in data science and data engineering is now taking place in public cloud. When King began looking for alternative solutions, it knew it needed a cloud-based data platform that could maintain stability at scale and offer the latest analytics technology. After assessing a number of alternatives, King turned Google Cloud .

“Our infrastructure needs to support hundreds of thousands of concurrent connections per second, as well as our data warehouse, and we saw that Google has the capability to handle our needs,” says Jacques Erasmus, CIO at King. “At the same time, we were very excited by its focus on machine learning and artificial intelligence.”

BigQuery for scalability, reliability, flexibility

For years, King maintained one of the largest on-premises Hadoop clusters in Europe, but with an open source query engine suffering from stability issues, infrastructure management became a priority. “Maintaining a petabyte-scale data infrastructure on-premises costs a lot of time, manpower and, most importantly, organizational focus,” says Kenneth MacArthur, Senior Technical Project Manager at King. “One of the key drivers was to offload our infrastructure operations so that we could focus on what adds value to our business.”

In addition to the resource costs of an on-premises solution, King also had to split its data across different platforms to maximize efficiency, which could lead to data science teams waiting for data to be migrated before they could start work. Building a new data platform provided the opportunity to unify all its data in one place.

After taking some time to evaluate its options, King decided that Google Cloud had the best combination of scalability and analytics capability for its needs, and began work on building two new solutions: the company’s data warehousing infrastructure and a stand-alone platform for machine learning. For the former, King began migrating data from the on-premises cluster to BigQuery at the start of 2018, expecting to be finished by the end of the year.

BigQuery formed the core of the new data warehouse, with its facility for scaling and easy-to-use functionality. Its innovative features also meant that the warehousing team could experiment with new data structures to improve performance.

“Nested fields in BigQuery let us efficiently query our data without needing to join big tables,” says Tom Starling, Principal Data Warehouse Engineer at King. “This has been very useful in allowing our business units to quickly drill down from top level numbers to very granular data.”

In addition, with Cloud Storage , King could hold its massive data archive more securely. “It took us out of the capacity planning game,” says Kenneth. Meanwhile, Cloud Dataflow proved a cost-effective way for the data warehouse team to ingest data without the complications of its on-premises precursor.

New solutions with Cloud Machine Learning Engine

Alongside the data warehousing platform, King’s technology team began exploring Google Cloud Machine Learning Engine tools to look for solutions for the company’s data scientists. The Game Platform Technology group, headed by Åsa Bredin, works with multiple game teams, acting as a force multiplier where solving the problems of one team can often lead to new solutions for the whole company.

A challenging problem King is trying to solve with machine learning is determining the appropriate level of difficulty for a game without having to manually playtest it over a long period of time. The team used Cloud Deployment Manager to deploy Google Kubernetes Engine workloads to create hundreds of virtual players, trained with machine learning models. The resulting data is then piped back to King’s data analytics modules with Cloud Pub/Sub . This creates a powerful feedback loop that lets King very quickly optimize the design of its games based on a foundation of solid data.

“When we ran on-premises, we didn’t have a clear way to deploy all our applications simply, so Cloud Deployment Manager has been really useful for us,” says Alex Nodet, AI Engineer at King. “If I have to deploy our pipeline tomorrow, I can do it very quickly.”

"Our scaling capabilities were not dynamic enough, Google Kubernetes Engine solved that issue. It's also a great complement to Google Cloud Machine Learning Engine when developing AI applications. We have been able to turn our prototype into a production-scale support tool that people want to use,” adds Alex.

Promoting agility, reducing dependency with Google Cloud

With Google Cloud, King has built a reliable, scalable data warehousing and analytics platform that will reduce overhead management and offer exciting opportunities with cutting-edge machine learning technology. Freed from the burden of building and maintaining servers, King’s engineers can stay focused on adding value to the business says Åsa: “Now we can spend our manpower on making the best games we can instead of managing clusters.”

According to CIO Jacques Erasmus, King’s data scientists have already seen notable improvements to the efficiency of their workflows with the new data platform.

“With the old cluster, when our analysts wanted to work on a project, they’d spend perhaps a day building out the environment, importing the data, and so on,” says Jacques. “Today, with Google Cloud, they can set up their data and environment with just a few button clicks.”

With a stable, easy-to-use data platform built with BigQuery, King has reduced dependencies and empowered its teams of analysts and data scientists.

“Speed of delivery is crucial to us and with the new setup, our teams can solve their own problems with fewer dependencies,” says Åsa. “It makes us so much more agile when teams can find their own solutions.”

Cutting-edge tools for a brave new world

With the migration nearly complete, King is already looking to see what new opportunities it can work on with Google. Expanding its use of machine learning across all aspects of the business is a major focus. Meanwhile, the company is currently working to develop its analytics platform into a service for external customers, and Google Cloud provides an easy, more secure way of segregating data for new users without exposing King’s own proprietary information. This multi-tenancy model, with segregated data, is already being used in recruitment where candidates can be evaluated not just for their general data skills but also their facility with the technology that King uses. Google Cloud itself has proved to be an effective recruiting tool for King, providing the company with a solid foundation for the future in terms of talent as well as technology.

“Google Cloud has been helping us attract the kind of people that you need to manage operations of this scale and complexity,” says Jacques. “Top talent likes to work with cutting-edge tools and technology, and our engineers are very happy with our decision to move to Google.”