PlayFab Technical Overview Last updated November 18, 2015

Contents Features Future roadmap Tech stack & architecture Performance data

PlayFab Features

Game Manager PlayFab provides a unified web interface that gives an operations team a single destination for viewing and managing a live game. Specific features include: Permissions. A robust permission model so that different users can be given permission to access different features. For example, customer service reps can be given permission to edit player properties but not change item prices, and product managers can be given permission to create or edit items, but not stop or start game servers. Dashboards and Reporting. A set of basic dashboards and reports for viewing performance of the game, including DAU, MAU, ARPU, and retention. Additional reports will be added in the future, including real­time view of game stats (optionally by segment). Audit log. A list of all configuration or data changes made via the Game Manager to track down issues or monitor support changes. Customer support. Support reps can investigate issues and provide service recovery by granting or revoking virtual items or virtual currency, temporarily or permanently banning abusive players, looking up login history, editing player properties, reviewing purchase history, and more.

Player Accounts Player accounts are at the heart of PlayFab; every user gets a player account. Player accounts can be linked to and authenticated by multiple third party systems, including:

iOS / Android device ID (for anonymous authentication)

PlayFab Confidential ­­ Page 1

Facebook Steam Apple Game Center Google Play Amazon Kongregate PlayStation Network Xbox Live Username / password (game­specific or publisher­wide namespace) Custom authentication providers (such as an existing internal account system)

By linking multiple authentication providers to a single player account, PlayFab supports cross­platform game play. The Game Manager can be used to view and manage account linkages so customer support reps can deal with authentication issues.

Data Storage PlayFab provides an easy way to store custom player and game data. All data is stored as key/value pairs, where values can be plaintext, JSON, or binary blobs. Data can be stored at the following levels: Game title data. Title data which can be accessed by all players and game clients. This data is typically used for game configuration information, such as game difficulty setting, or level descriptions. Files. PlayFab provides support for uploading files, and then delivering via an integrated CDN. Today we use Amazon cloudfront for delivery; we may replace with alternate CDN’s in the future. Catalog data. Every item in the catalog (see section on in­game commerce) can have custom properties associated with it, such as rate­of­fire for a weapon, or mana points for a collectible card. Group data. Data shared among a specific group of players, such as a guild or lobby. Examples include guild name, or group message of the day. Players can be added and removed from a shared group; all members in the group can read or write shared group data Player data. Data stored per player account. Player information is broken down as follows: Scope:

PlayFab Confidential ­­ Page 2

Publisher­level ­­ data for a given player shared across all titles from that publisher, such as cross­game achievements, or gamer score.

Game­level ­­ player data for a specific game title, such as controller settings, or avatar image.

Character­level ­­ each player can have multiple characters (or vehicles) in a single game, such as tank configuration, or health remaining.

Permissions:

Private ­­ can only be accessed by the player. Public ­­ can be accessed by other players (e.g., description of a town)

Client access rights:

Read/write ­­ the game client can read or write the data. Read­only ­­ the game client can read the data, but only a game server can write it. Internal ­­ the data is invisible to the client; only the server can read or write it.

Stats: Stats are a special sub­case of player data. They must be integers, and have special properties:

Leaderboards. Any stat can be used to define a leaderboard (see social features). Match­making. Stats can be used as part of the match­making logic (e.g., find a match

based on player’s level or weapon rating) Segmentation. Stats can be used to define player segmentation rules.

In­game commerce PlayFab provides an in­game commerce system for managing and selling in­game entitlements with real or virtual currency. These can range from virtual items (e.g., a sword, a new tank), to content unlocks (e.g., unlock level 5, unlock an additional character slot). Specific features include:

Virtual currencies. Each game can have multiple virtual currencies, with optional initial balances, recharge rates, and maximums. Each player and character can have separate balances for each virtual currency, and virtual currency can be exchanged between characters and players.

Catalog. The game catalog defines the master list of all items in the game, with optional usage counts, expiration times, custom properties, stacking behavior, price in one or more virtual currencies, price in one or more real­world currencies. Bundles allow multiple items or currencies to be purchased together. Containers are like bundles, but must be unlocked, with an optional key, before their contents are granted.

Drop tables. Drop tables are weighted lists of items that can be used with bundles or containers to randomly grant items according to the desired probabilities. Examples include card­packs, treasure chests, or lucky draws.

PlayFab Confidential ­­ Page 3

Stores. A store is a subset of a catalog, with optional price overrides. Stores allow items to be discontinued from sale, yet still exist in the catalog.

Inventory. Each player and character have an inventory of items that is maintained on the server as items are purchased, granted, consumed, or expire. Items in the inventory can have custom per­instance properties.

Trading. Items can be traded between players. Features include escrow, for safe trading, and are transactional.

Real­money transactions. PlayFab supports a number of payment mechanisms for real­money transactions, including third party systems such as PayPal, Amazon, and Facebook. PlayFab also supports server­to­server receipt validation with protection from replay attacks for mobile platforms such as Google and Apple. PlayFab also provides platform­specific payments for Steam, PlayStation, and Xbox

Server­side game logic PlayFab provides several mechanisms for hosting custom game logic on the server. Cloud Script. Server­hosted Javascript which define custom functions that can be called from the game client and executed within the security context of the current player. Cloud script code can call the privileged server API which cannot ordinarily be called from the client. Cloud script can also be triggered by Photon room events, and soon by PlayStream events or PlayStream segment changes. Game server hosting. Developers can upload a custom game server executable for multiplayer games, and define regions around the world where the game server should be deployed. PlayFab automatically deploys and scales game servers based on load to maintain sufficient capacity for hosting new game matches. Built­in matchmaking supports assigning players to matches based on parameters such as min/max players, region, game mode, player statistics, and friend list. Developers may also provide custom match­making logic. Match history is stored automatically for each player. Games servers can write local files, including logs, replays, or crash dumps. When a game session is over, local files are archived and indexed for later analysis or replay. Game servers authenticate connecting players using a ticket system, and can call any of the privileged game server API’s to grant items or set custom properties. Today we support Windows server executables, with support for Linux game servers coming soon. Photon. PlayFab’s partnership with Exit Games makes it especially easy for games to provision multiplayer game rooms using Photon Cloud. Rooms can automatically call CloudScript for events such as EnterRoom or ExitRoom, and user authentication is taken care of automatically.

PlayFab Confidential ­­ Page 4

Marketing PlayFab has several features to help games market to their existing players. Push notifications. Currently, games can send custom mobile push notifications directly to a player through the server API (e.g., your base is being attacked). Soon, developers will be able to automatically send notifications based on segment changes, or in bulk to an entire segment. Articles. Developers can publish articles using the Game Manager to be displayed in the game. Examples include launcher message­of­the­day, interstitial notices, in­game notices, etc. Coupon codes. PlayFab can generate one­time coupon codes which may be redeemed in­game for any item in the catalog. Cross­game promotions. Publishers can cross­promote titles on PlayFab by enabling players in one game to unlock or be granted items in other games.

Social PlayFab has several social features to help promote higher engagement and retention through player­to­player interaction. Friends list. PlayFab maintains a friends list for each player, and can automatically add players to that list by matching against existing Facebook or Steam friends. Games can also add or remove players to the list directly. The friends list can also be used by leaderboards and matchmaking. Leaderboards. Any stat can be used to generate a leaderboard. Leaderboards can be:

Global or social, showing only members who are friends of a given player. Durable, or resettable, so for any given stat, the leaderboard can reset on an hourly,

daily, weekly or monthly schedule. When a statistic resets, all associated leaderboards and matchmaking rankings reset as well. Archives of leaderboards and player values for previous versions of the statistic are still retrievable after the reset.

Chat. PlayFab supports player chat rooms via partnership with Exit Games.

Developer Tools PlayFab provide a rich set of tools to help developers easily integrate PlayFab’s services into their game. These include: Multiplatform SDK’s. PlayFab provides SDK’s for the following devices or platforms:

PlayFab Confidential ­­ Page 5

Unity3D Unreal Blueprint Unreal C++ C# Windows (C++) ActionScript Javascript Cocos 2D NodeJS Java Xamarin

All of the SDK’s are auto­generated directly from the API code, making it easy to add new SDK’s as needed, and ensuring that the SDK stays consistent with any changes to the API. We have open­sourced our API generator via GitHub so developers can build their own SDK’s if needed. API. All PlayFab services are exposed via a JSON­based web API for fully cross­platform support. All API documentation is auto­generated from the code ensuring it is always up­to­date with any changes.

Future Roadmap PlayFab is investing in several significant feature areas that will be rolled out in coming weeks and months.

Marketplace PlayFab is building a marketplace of pre­integrated third party services, many of which can be activated entirely without any additional SDK needed on the client. Expected categories including advertising, attribution tracking, community, advanced analytics, payment providers, eSports tournaments, streaming, fraud detection, and more. Marketplace partner dashboards will be available via the Game Manager without requiring separate login. In many cases, partner data can be rolled up and exposed in the PlayFab dashboard without requiring any sort of click­through or deep­links.

PlayFab Confidential ­­ Page 6

Marketplace partners will be able to generate events, which will flow into PlayStream, and also expose actions which may be triggered via PlayStream Actions. Partners will also be able to access player profile data, providing a single consistent view of the player across the entire system of services.

PlayStream PlayStream is a real­time feed of all player activity in a game. Player events are fed into PlayStream from a variety of sources, including PlayFab core services, third party vendors, and the game itself.

PlayStream routes its events to a variety of destinations, including:

Event archive and index. A durable history of a player’s activity, with powerful search tools.

Data warehouse. Provides reporting on game performance. Real­time segmentation engine. Players are segmented into one or more categories

based on rules that are applied in real­time as events flow through the pipeline. Actions triggers. Actions can be triggered based on specific events or segment changes,

such as granting an item, sending a push notification, or calling a cloud script function. Webhooks. Events can be posted to any external web endpoint, including existing game

backend systems, or third party marketplace vendors. Event debugger. A filterable real­time display of player events as they occur, along with

detailed status information.

PlayFab Confidential ­­ Page 7

Game Operations Enhancements New features being added to the Game Manager include:

Batch segment actions. The ability to execute a given action for all players in a given segment, on an ad­hoc or scheduled basis.

New reports. Enhanced reports for measuring game performance. Real­time dashboards. View game metrics, by segment, in real­time. Messaging. A reliable message queue, enabling games to reliably send messages to

players within their games. Promotions. The ability to define and support special offers and promotions. Localization tools. String tables, hosted on the server, with support for arbitrary

languages. Includes integration with localization vendors who can provide automated localization workflow to avoid manually passing strings in spreadsheets.

New look­and­feel. We will be overhauling the game manager with a completely new UI.

PlayFab Technology Stack Our architecture consists of a variety server roles running on a mix of Linux and Windows. On Linux, we run several open source products such as Consul (service discovery / distributed key­value store), NGINX (proxy server), Jenkins (build / test / deployment) and soon, custom game server hosts. On Windows, we run web services written in C# using the ASP.Net MVC framework.

PlayFab Confidential ­­ Page 8

Why Windows Some developers are surprised when they learn that most of our servers run Windows. For us, the reason is simple: Windows is currently the highest performance environment for running C# code built with the .Net framework. We treat Windows VM instances as interchangeable, headless hosts for executing our C# code, and we don’t make use of any Windows­specific technologies such as Active Directory. Furthermore, our server provisioning, code deployments, logging and monitoring are all 100% scripted and do not require any sort of GUI interaction. Other than the OS, our entire stack is open source:

JSON.Net AWS SDK for .Net ASP.NET MVC .Net Core Framework C# Roslyn compiler

Given this fact, it’s only a matter of time until Linux becomes a first class option for hosting all of our services written in C#. In fact, it’s already possible, but we’re waiting for the project to mature before migrating any production systems to it.

Why C# and .Net We find C# and .Net to be an excellent language and framework for developing online services. C# combines high performance with high developer productivity, and its broad adoption gives us access to large pools of local developers with expertise in backend development (Microsoft) and in game development (Unity) in the Seattle area.

Some of the things we love about C# and .Net: High performance garbage collection:

Provides a huge boost in developer productivity with very little performance penalty in this use case of frequently instantiated short­lived objects.

Real generics (contravariance and covariance support): Supports highly reusable library code.

Attributes and reflection: Allows for rich metaprogramming which we use to do things like auto­generate our API documentation and to map our player data objects to DynamoDB tables.

Functional programming constructs: Lambdas, expression trees and LINQ make it incredibly fast and easy to write complex queries that are concise and correct.

Task Parallel Library: Enables high performance thread­safe concurrent processing.

PlayFab Confidential ­­ Page 9

Given the power of the language and the recent open source support from Microsoft, we predict that C# adoption by the programming community will grow significantly over the next few years, which is reflected in the latest GitHub language popularity trends.

Performance Limits According to the imperfect but useful Web Framework Benchmarks results, a single Windows Server machine running a C# web service with the MVC framework we use was able to process over 100,000 requests per second. Obviously, a service performing a more significant, real­world workload would not handle as high a request volume, but it proves that our fundamental tech stack is not imposing an unreasonably low limit on throughput.

PlayFab Architecture Operating online games at scale is a challenge. Load is unpredictable and can ramp up very quickly. Data access patterns are very demanding and write heavy. Our approach to scalability is to build components that are 100% elastic with respect to compute and storage.

Elastic Every layer of our core architecture is capable of adjusting compute and storage resource capacity based on usage level, in most cases automatically without human intervention. Our primary storage for player data is DynamoDB, which supports quickly adjustable read and write throughput capacity. Our services run on groups of auto­scaling EC2 instances.

PlayFab Confidential ­­ Page 10

Redundant We have no “special snowflake” servers. A machine terminating is never a cause for alarm. All server roles, including core web services, matchmaking, event stream processing and report generation have instances running in multiple data centers. The storage services we use are all replicated across multiple data centers as well.

Load Testing We frequently load test our API to ensure that the service scales predictably. In addition to routine test runs to verify the performance of new features, we run tests ahead of title launches that we expect to stress our service in new ways. In these runs, we simulate player sessions that match the usage patterns of the particular title. Our largest such test was recently run for a large multiplayer game company who wanted to know that our service could handle up to 1 million concurrent players. Some details:

Simulated 1,000,000 concurrent players Provisioned 1 million unique player accounts, each with 600 items in the inventory Simulated session length of 1,100 seconds, including the following calls per session:

1 login (w/ Android Device ID) 1 catalog request (6,000 items, delivered via CDN) 1 store request 1 inventory request (600 items) 1 title data request (7 keys) 2 battles per session, of 480 seconds each 3 AddCurrency calls per battle 3 ConsumeItem calls per battle 3 PurchaseItem calls per battle

Login ratio of new account creates vs. old accounts: 1:4 Load generators were sitting in same AWS region as API servers Load generated with 4 c4.8xlarge instances

Results:

Used 32 c4.4xlarge server instances, with 65% average CPU usage, running Win2012R2

Results in 21K requests per second

API call Median response time 95% response time

LoginWithAndroidID 63ms 185ms

PlayFab Confidential ­­ Page 11

GetTitleData 6ms 15ms

GetCatalogItems 1ms 5ms

GetUserInventory 388ms 577ms

GetStoreItems 1ms 7ms

AddUserVirtualCurrency 24ms 113ms

ConsumeItem 40ms 99ms

ValidateGooglePlayReceipt 119ms 282ms

PlayFab Confidential ­­ Page 12

Adventure Capitalist Launch The following chart of player logins depicts how our service performed in practice during the launch. Our elastic architecture automatically scaled to handle the sudden jump in load without any issues, and without any sort of manual intervention.

PlayFab Confidential ­­ Page 13

API Performance for AdCap Average request execution time measured at load balancer: 70 ms Average end­to­end request round­trip latency observed in North America: 110 ms API endpoint uptime for last 12 months (measured by Pingdom): >99.99%

CCU Performance The highest CCU numbers that we have seen in production was the launch of the PC multiplayer game LoadOut. That game hit a peak of 30K CCU at launch, and reached the top 10 on Steam. The following graph shows the performance of that game at launch (the Y­axis is DAU). As the following graph highlights, at peak we had 200 VM’s running on Amazon handling the game load. There was a glitch at launch caused by the custom matchmaker that Edge of Reality had built. Their matchmaker used MongoDB, and it couldn’t keep up with the number of writes, causing players to have problems finding matches. Once they replaced their box with one using SSD’s, they were able to keep up and player issues went away. The PlayFab services, however, were stable and did not impact performance.

PlayFab Confidential ­­ Page 14

PlayFab Confidential ­­ Page 15