How to choose/write a backend server for your game
The products which are marketed as backend servers or services for games usually try to provide features or building blocks of creating the features which are a part of the metagame of your product. Things like leaderboards, achievements, challenges and quests. They also provide basic features like authentication, cloud data storage and real-time messaging between players which you usually need to build your game. The main game logic of your product on the server is usually coded either in your game engine or a specially written software for the exact purpose. Some products like Playfab simply allow you to run whatever you want (if it is an executable file) on their servers for your authoritative game server logic. Some of them try to create a scripting system which allows you to write authoritative logic which means a more light-weight server but usually it means you don’t have many game engine features available at your fingertips and also you have to choose their language of choice and debugging is not as easy as it should be either. Here we intend to mostly talk about the first set of features and only a bit at the end about the game logic part.
There are 3 factors which are very important when choosing your backend. First of all the public services are usually very cheap at the beginning and they will only charge you more when your game really takes off which is ideal for many companies and types of games. Custom solutions like ours or the open source equivalents are usually cheaper in the long run but at the beginning might have a cost associated with them, especially because you need to run them on your own servers and pay for the servers yourself. Even if it is one machine only at the beginning, still it is some cost. Also when using custom solutions made by yourself or others, you need to manage the servers yourself which again is more costs.
Now you might question , why you should want to write your own or buy a custom backend solution, the answer is flexibility. The public services have to provide their features in a way which means they can run their service in a cost effective manner and also the features they provide might differ a lot from what you need.
They themselves differ very widely with each other. GameSparks for example provide more complex querying capabilities and now it doesn’t use MongoDB as database anymore so will not have the scaling issues while Playfab uses a different data model (it always has been more scalable due to database choice and more interesting architecture IMHO). Let’s say you want to build a game which runs tournaments on a weekly basis and provide prizes. Both of the popular public platforms have features which assist you in doing it but as soon as you have a special requirement like small prizes between you and your friends then you have to do a lot to make the system work with your design.
Also as said some of their features are simply less powerful because implementing them in a more powerful/correct manner would mean their system will be open to bad usage from developers. For example, unidirectional friend lists are there because transactions are expensive and if they allow you to run them then you might mess up with their service which is very important when they are providing a lot for free. Ability to run free form queries is tricky for the same reason and lots of special kinds of timers is discouraged because of this as well. Now when you make your own custom solution, you still need to be careful with transacitons, queries, your data sizes and many other things but you can choose where to spend your budget specificly and where you want to spend your resources or money if needed.
None of these mean public services are bad or a situation like what I painted will arise. They have limited resources as well and are providing a lot for free and as human beings have to choose some options with different tradeoffs. You just need to take a look at your game design and balance your specific needs with the server, management, administration and software costs when deciding on the tech stack for your game. I personally am very happy with what Playfab provides for many projects and used it and am sure this is true about GameSparks in many projects as well. They allow you to generate operational forms and do some other things Playfab does not provide. I really love the fact that Playfab allows me to upload and manage custom game servers in their systems and am not into JavaScript much and that is the only reason I prefer Playfab to GameSparks but definitely it can change for projects with different requirements from those which I usually work on.
How to write your own custom solution
If you decide to create your own custom solution or buy/use an open source or commersially available backend other than the public service providers then you have a few broad choices to make which affects you a lot. One choice is what technology to use and the other is how to scale, monitor and maintain the system.
One approach which was popular back then for some medium size games was building the bakcned part in your game engine and run multiple instances of it to scale it and use a powerful database behind it. When I was working at Mu
chDifferent, our products were made for such a system. We had uLobby which could be used to create lobby and metagame features of a game inside Unity and we had uGameDB which was a client for the powerful NoSQL database Riak. We made medium size games using this approach successfully and many our customers did the same in relatively big games as well. Not that the approach does not have limitations.
The advantages of the approach are the fact that you are inside your game engine and have lots of technical know-how and the communication between your game servers and metagame servers is very easy using the same library and RPC mechanisms. Back then before existence of gRPC and similar technologies it was a challenge for a relatively small team to develop their own RPC mechanism which works between different platforms and is flexible and high performance enough for multiplayer online games. Also this approach probably is more interesting if you are making a multiplayer authoritative server based game and are already using your game engine on the server.
The disadvantages are the fact that you don’t have any of the usual features which you need to implement for bigger games. Things like fault tolerance, gossip protocols (for clustering and finding other nodes) and distributed computing mechanisms usually don’t exist in game engines and you have to develop them yourself. If you need to run two instances of your lobby and friends of a player on lobby A are in lobby B and they try to send messages to each other, you are responsible for understanding the fact and relaying the messages.
We have developed a proof of concept system for a friend team and cleaned it up a bit and released it in github as an open source project. This is called MidNight and is available on my personal github page here. It is called midnight because it tries to handle the chaos of the players and many systems at work in a lobby. It contains basic authentication features, friend lists and game server instance management. There is a sample and minimal documentation but we are available for help if needed, just create an issue and we’ll jump in.
The other approach is to use frameworks which are created for the purpose of making distributed scalable server software. These days most cloud providers have PaaS offerings like managed Kubernetes setups on all major providers and Service Fabric on Azure. Even if you don’t want to use these for good reason or not, frameworks like Akka, Microsoft Orleans or the Erlang programming language makes it a lot easier to work on these kinds of software. I previously wrote about how we use them.
Using PaaS offerings might lock you in and also using Kubernetes is not necessarily beneficial if you don’t have a big team and are not developing things in too many languages. Micro service architectures have their own challenges when it comes to handling long running processes, transactions or cross domain business logic and they are much harder to debug. I personally see an actor as a sort of micro service (the concept is from a much smarter person) and am very happy with running a cluster of servers using an actor framework without any micro-service in the common sense whatsoever. You still develop each small part in a different actor while you avoid many of the complexities of running a micro service based system.
Our backend uses Orleans and has 3 processes other than Couchbase which is our on-premise database choice. One process is the Orleans silo which hosts the actors and can form a cluster and is stateful. Another is a front-end for games to connect to which is a stateless socket server which can scale independently of the Orleans silo and the third is a rest server front-end which can be hosted on IIS or using OWIN to serve requests of our web-based panel which is an angular based app.
We had many specific needs when we were developing Apadana and the current product reflects that. Ease of deployment on small clusters, ability to use C# as the scripting language on the server in a safe and secure manner and small binary messages for everything were a few of those. None of these were available on the public providers and we were burning with the desire to make them possible, so we did. Not that games cannot be made without these features but these are the things that could mean a much smoother gameplay experience in developing countries (small binary messages instead of huge JSON ones for every message), C# scripting so our developers can locally debug the scripts in visual studio using mock server APIs and don’t have to use unfamiliar languages (for C# and C++ game developers) like JavaScript which are not intended for large codebases which game server logics become. Ease of deployment of small clusters also mean that even for a fully available system which is tolerant of the fault of a machine we could use 2 Orleans silos and two front-ends and a Couchbase cluster based on what we need. If some availability could be sacrificed like for development and maybe soft-launch, we could run only one instance of each process and run all of them on a single machine.
To make management a bit easier we developed everything as console applications which make them easy to debug but used Topshelf to make them windows services which automatically start after crashes and also automatically start if the OS restarts after an update or some other event. Orleans took care of connecting failed and consequently restarted processes to each other for continuing execution. Now with Orleans availability on .NET core we can even runthe system on Linux and benefit rom Orleans transactions and also all perf improvements in .NET core and that is another reason to choose te tech you use carefully so you can rely on it to improve over time.
When you do this and write your own backend or use one with source code available, you can choose and you can choose a lot. You can choose to have quests for each user based on his/her friend’s units, reward them based on number of friends which completed something in a day and run custom match making logic which looks at weekly leaderboards so players close to each other are matched which makes their games more intense.
These are just some examples and some of them might be easier to do in one or another of the public providers. Playfab allows you to run custom match makers on your own servers for example which we actually did but this is something between writing your own stuff and using a public provider because it is essentially a mix of both and that is a good power to have. GameSparks also allows you to run js code in many many places and based on timers and it results in lots of flexibility on many fronts but overall, I guess you get the point. Especially when it comes to more fundamental stuff like what protocol to use, and other foundational things, having your own solution can make a huge difference.
What about game logic
Well if you decide to write your own dedicated/authoritative server for your game then the main choice is to use the game engine for the server as well or use some compact simulation software on the server. It is a big topic but in general if you need lots of the features of the game engine and/or your time to market is very important, then you should definitely go forward with the game engine but if you are making something which has to be as cost-effective as possible, then if you have the right people, you should try to code it on the server using a lightweight framework which can share its core with the client part of your game if the technologies partially match.
Final decision
The decision of what to do regarding all of these is not easy at all in many cases. Attempting this at the wrong time can cause your project to fail and your team to endure hard times but almost always for teams bigger than 5 developers or time frames larger than 8-10 months, it is an option which should not be dismissed lightly. Be aware that it would mean at least one of your developers should exclusively work on this but the good thing is that for next titles, lots of it can be reused unless you decide to build a game which is very different in many dimensions. At the end I just would like to say that we do contract work and consulting on Unity and distributed server systems both. We would be happy to talk to you if you are into an interesting project and like quality.