The game had a very ambitious scope. Towards the end, we had to rush a lot of things resulting in quite a buggy game (not even talking about the physics-based centipede I spend so much time on and had to replace with a much simpler version). Nevertheless, the team did great work from the graphics to the sound design to the level design and the programming. There just wasn’t enough time for polish and bug fixing. The Wildfuchs people are really nice and talented, I wish them all the success in the world and hope to jam with them again.

You can download our game here.

Some afterthoughts:

1.) When we did the brainstorming to come up with the game idea I was very uninspired. But Sunday morning I got an idea I would like to explore further: What if you would live in a home that’s constantly rotating? Where you sometimes would walk on the walls and the ceiling? How would you arrange things to be easy to use in multiple directions? If someone knows of a game with this trope please write me in the comments.

2.) When I was thinking about games that were having the concept of home as a central theme, I was remembering the game Cargo Commander. It’s such a great feeling when you went hunting for loot in abandoned cargo containers and can finally go back to your base where this heartwarming song is playing and you are finally safe. Also the whole story with being separated from your wife and your child and working hard to finally get home. It’s a good game, go play it!

3.) I forgot to make a blog post about it, but I also participated in Global Game Jam 2018 and we made a game inspired by my current job: It’s called Solar Mission and you can download it here.

]]>

You can obtain a good first guess by looking at some usage statistics (like those for the Battle Spot Single format). This information is called a prior in mathematical terms. Of course, you should always update your guesses, once you obtain new information. So if you see a team with Charizard and Hippowdon in the team preview you can be a little bit more certain that the Charizard will be a Charizard Y because Hippo is often paired with Charizard Y to reset its weather.

By using Baye’s theorem you can calculate, how much you should adapt your predictions to new evidence:

Let’s go through an example of how to use this formula. Please imagine the following situation:

You have your Ferrothorn out against the opponent’s Greninja. Greninja can only beat your Pokemon if it carries the move Hidden Power Fire. If it has this move you should switch out, otherwise, you should stay in and attack. Luckily your Ferrothorn carries the move Protect so you can scout for the opponents move by protecting your Pokemon for one turn. You do this and the Greninja attacks with Dark Pulse. But the opponent could be trying to hide his Hidde Power Fire move. How sure can you be now that he really does not have it? Let’s plug the numbers into the formula:

P(A|B) is the probability that the Greninja has Hidden Power Fire in spite of observing that it attacked with Dark Pulse. We want to calculate this.

P(A) is the probability of Greninja having the Hidden Power Fire move (given no evidence indicating otherwise). We can look that up: It’s ~20%.

P(B) is the probability of Greninja attacking with Dark Pulse in this situation.

P(B|A) is the probability of Greninja attacking with Dark Pulse given that it also has Hidden Power Fire.

Often it’s easier to just reason about the quotient P(B|A) / P(B). How much more likely/unlikely is it that Greninjna attacks with Dark Pulse if it also has Hidden Power Fire? This ratio is very hard to calculate precisely, as it depends on the play style of your opponent. But it’s possible to make pretty accurate guesses if you witnessed situations like that in the past and were able to learn from them. So if you think that the ratio in our example is 1/2 then you would arrive at 1/2 * 20% = 10% likelihood of Greninja having Hidden Power Fire.

Not calculating P(A|B) directly but thinking of P(A|B) as Prior * Evidence might help you avoid some mistakes:

**1.) Avoid base rate neglect**: Try not overreacting to evidence. If you see a Garchomp use Toxic on your Heatran that’s evidence that it does not have Earthquake, but it’s still very unlikely as 98.9% of Garchomps carry this move.

**2.) Always try to imagine alternative explanations for your opponent’s behaviour**. He switched in Volcarona on your Landorus’ Earthquake? Sure, that’s pretty good evidence that this Volcaron has Hidden Power Ice and maybe even a Choice Scarf. But it could also be a trick to scare you out and set up with Quiver Dance. Or maybe your opponent predicted that you would klick U-Turn instead of Earthquake last turn?

This post took me a long time to write as I struggled a little bit to make using Bayes’ Theorem for Pokemon sound useful. My next post about game tree pruning will be up much sooner and hopefully also a lot easier to apply!

]]>Cute game with co-op and physics based puzzles.

I think the two secret ingredients for a good video game are:

1.) Everything has some kind of weird physics

2.) All the game play elements can be combined with each other

Anyball tries to be “All sports, all at once” and I’m sure it will be a good video game.

I played it, it’s good.

The Internet was better back in the days, right?

A game where you pretend that you know what you are doing.

I told you that weird physics make everything better!

See the description of the trailer above.

Seems fun

Finally a game simulation of my job! Unfortunately you first have to buy a GPU cluster to play it.

Don’t know if it’s any fun, but certainly a nice idea.

]]>Wow, my opponent is such a ****head, why did he klick Earthquake in front of my Zapdos. He’s such a bad player! And now I lose.

Or, if he is not quite as easy to lose his temper:

Ok, I don’t agree with this play at all. He should really have protected that turn.

What’s going on here? Why do they think their opponent made a bad play if it resulted in a favorable outcome for them?

Often the answer to that is that the opponent made a very risky play that could have put him at a huge disadvantage. Let’s look at an example:

You lead your timid Nihilego against the opponents Charizard. Nihilego is guaranteed to outspeed and 0HKO with Power Gem, so you put huge pressure on the Charizard to switch out. You predict the switchout and set Stealth Rocks. But you made the wrong prediction: You can set up the Stealth Rocks but Nihilego gets 0HKOed with Outrage and you lose your only Charizard check.

Was it the wrong decision to klick on Stealth Rocks? Of course, in hindsight it was, but let’s look at it from a game-theoretic standpoint and make a payoff-matrix:

```
+--------------+------------+---------+
| | Switch Out | Outrage |
+--------------+------------+---------+
| Power Gem | 55% | 70% |
+--------------+------------+---------+
| Stealth Rock | 60% | 20% |
+--------------+------------+---------+
```

The numbers in the table are your estimated chances of winning the game after you and your opponent made their move choices and the turn ended. You now see that setting up Stealth Rocks was a rather risky choice. If you attacked with Power Gem you could have hurt the switch in or even 0HKOed the Charizard and got a Beast Boost from it. Setting up Stealth Rock would have been a little bit better than attacking if the opponent switched out, but a terrible decision if he decided to stay in.

So should you always attack in this situation? Should your opponent always switch out? But if your opponent always switches out, shouldn’t you always set up Stealth Rock?

Fortunately, there is a solution to this problem. You can adopt what’s called a mixed strategy. Most of the time you just attack with Power Gem, but sometimes you surprise your opponent with setting up Stealth Rock. Your opponent should do the same: Switch out most of the time, but sometimes attack with Outrage.

But what does sometimes mean? How often should you attack exactly? There are two things here that should influence your thought process. The first one is figuring out the psychology of your opponent and reacting accordingly. A player very low on the ladder might for example not know that Nihilego can attack with Power Gem and always stay in. Or you might know the player from previous matches for his very risky playstyle.

The other one is that you should try to hit what’s called a Nash equilibrium. Let’s look at the informal definition from Wikipedia:

Informally, a strategy profile is a Nash equilibrium if no player can do better by unilaterally changing his or her strategy

So if your opponent knows your strategy (attacking x% of the time) but can not exploit this knowledge to increase his odds of winning, then that’s a Nash equilibrium.

We can calculate the percentage of times you should attack by taking the payoff matrix into account. Let *q* be the percentage of time that your opponent chooses to switch out and E[x] the expected value of x.

* E[Chance of winning by attacking with Power Gem] = (55%)q + (70%)(1−q) = 70% – 15%q*

* E[Chance of winning by setting up Stealth Rocks] = (60%)q + 20%(1−q) = 20% + 40%q*

* E[Chance of winning by attacking with Power Gem] = E[Chance of winning by setting up Stealth Rocks]*

*⇒ 70% – 15%q = 20% + 40%q*

*⇒ q = ~91%*

So the opponent should switch out ~91% of the time and you should set up Stealth Rocks only ~9% of the time and attack ~91% of the time.

While this is certainly the mathematically optimal way to solve this problem, it’s rarely very practical to whip out your calculator in the heat of a Pokemon battle and then use a random number generator that tells you what you should do. I certainly haven’t seen anyone attempt this. And you will probably be just as good if you just follow your hunches in such situations.

However, it would be really nice if this knowledge would spread in the competitive community and we could replace:

Wow, my opponent is such a ****head

with

Wow, my opponent made a risky play

or even:

I think my opponent should only attack about 10% of the time in such a situation. This was really bad luck for me.

By the way, if you are an avid Pokemon player you might have noticed that I simplified the battle scenario a little bit. Charizard has two different mega evolutions Charizard X, and Charizard Y and only Charizard X will kill Nihilego with Outrage.

In the next blog post, I will show you, how you can predict what mega evolution your opponent has by using Baye’s theorem. The hype is real!

]]>One solution to this problem is called Value Function Approximation. The idea behind it is to not calculate the value function exactly, but to approximate it using supervised machine learning algorithms (e.g. Neural Networks). In some ways, this resembles how most people play Pokemon. They don’t think through all the possible ways the game could go, but just imagine a few possibilities and think about how good the situation would be if they chose one of them.

Good proxies for evaluating the chance to win might, for example, be the total health of your Pokemon compared to those of your opponent. Or if you have a boosted Pokemon on the field that can sweep. Or, more generally, that you were in a similar situation before and won the game at that time.

To get a good understanding of how Value Function Approximation works and how one would go about building a reinforcement algorithm, that can play Pokemon, I recommend this two resources:

- Reinforcement Learning: An Introduction (2nd Edition) by Richard Sutton and Andrew Barto. The second edition is currently a work in progress and freely available from one of the Author’s website
- Reinforcement Learning course by David Silver. David Silver is a researcher at DeepMind and one of the creators behind Alpha Go. In 2015 he held this excellent course at UCL.

Up until now, nobody has built an algorithm that can play Pokemon on a high level.

So is there a way to leverage knowledge about Reinforcement Learning to improve Pokemon battle skill? Maybe! A few things come to mind:

- While Reinforcement Learning algorithms can learn from self-play, they are able to learn much faster, if you provide them with training data from expert human level play. One of the reasons is, that it’s harder to learn what good plays are, if you only make bad plays in the beginning. So you can try to boost your own skill by watching videos from good Pokemon players (like Chemcoop for the Battle Spot Singles format)
- One of the challenges when training Machine Learning algorithms is always that of overfitting. A phenomenon humans are also very susceptible to. It means learning something not because it’s true, but because it was part of your anecdotal experience (=training data). So if you see a Pokemon with an unusual move three times in a row, don’t just assume that it’s now very common, but trust in statistical resources like this website instead.
- The more I learned about Reinforcement Learning, the more I realized that my brain must be doing all sorts of different things that I’m not even aware of. So sometimes a good strategy might not even be that important and we should just let our brains do their things and follow our hunches. Maybe regular sleep, a healthy diet, and a calm mindset will help you improve more than reading articles about the math behind competitive Pokemon.

The next post will be about Nash equilibria, the very heart of competitive multiplayer games, so don’t miss it!

]]>

The more interesting aspect, however, is figuring out how to play optimally, knowing all of these intricate details already. There is also advice for this out there, but often it’s mainly things like: “Just practice a lot!” or “Try to learn from your mistakes!”

In this series of blog posts, I will link competitive Pokemon play to the well-studied fields of game theory and reinforcement learning. That’s quite a theoretically way of looking at it, so I will also try to include some practical battling advice.

Pokemon is a game of hidden information, involves lots of randomness and does not have a Nash Equilibrium in pure strategies. I will talk about all of this in later blog posts. For now, let us just assume that you have perfect information about your opponents’ team, there is no randomness involved and both players use a pure strategy.

Under this assumptions it is quite easy to describe how to play optimal, but very difficult to actually do it, if you have no access to nearly infinite computing power. Since the problem of solving Pokemon battling has optimal substructure, you actually just need this little formula, called Bellman equation:

In regard to Pokemon battles the variables and functions in this equation can be described in the following way:

**x** is the current **state** of the battle. What Pokemon are currently on the field? What Pokemon have already fainted? Is Stealth Rock set?

**V is the value function. V(x)** describes how valuable the current state is. So in Pokemon V(x) would be how likely you will win the current battle given that it’s in the state x and both players will play optimally the rest of the game

**a** means **action**. In Pokemon actions are choosing the lead Pokemon, choosing an attack or switching out the current Pokemon.

**Γ(x)** is the set of all the possible actions you can take right now. So this set may be smaller if you are trapped by Shadow Tag or locked into Outrage.

**F is called the reward function. F(x,a)** is the reward you get for taking action a in the state x. A good way for defining the reward function F in Pokemon is to give a reward of 1 if you win the game at the end of the turn and a reward of 0 otherwise.

**T(x,a)** is the state you get into if you choose action a in state x.

**β** is a discount factor and represents the patience of the player. A low β means that the player prefers fast wins to safer wins that will take longer.

So with this formula, you can now calculate how good a certain situation in a battle is. It is then very easy to play optimally: Just choose the action where the resulting state is as valuable as possible.

Using Bellmann equations to evaluate game state is a recursive operation which involves simulating all possible ways the game could go. Since Pokemon battles can take many turns and the state space expands exponentially, this is computationally completely infeasible.

However, imagining different ways the game could go will help you in many situations, especially at the end of the battle, when there are not many options left.

A more practical way to go about being the very best Pokemon trainer is called **Value Function Approximation** and will be discussed in the next blog post.

]]>

Luckily there were also some trailers that really got me and presented game ideas I never encountered before. Here is a list with the 10 games I found the most fascinating:

It’s about a cat with a guitar. It reminds me of the music system for Cell Tune. I just have a soft spot for procedurally generated music

Don’t know about the gameplay but this seems quite charming.

I quote: “Disco Bear is an interactive narrative that follows a polar bear through tragedy, depression, and his journey back to the dance floor”

I think: Games with poorly photoshopped in pictures of animals is a totally underexplored genre that merits our fullest attention.

The combination of high tech server farms and an abandoned palace is really unsettling. And the idea that the enemy AI is trying to mimick your playstyle is certainly intriguing.

Some stealth games try to build deep systems that are proposed to create emergent situations with infinite replayability. This game however seems to feature dense, handcrafted environments and enemy behaviour. And that’s pretty cool.

A game that uses YOUR WHOLE KEYBOARD. And also in a very intuitive way. Never seen that before.

Insane artstyle, insane gameplay, insanely fun?

A game you play by surfing the internet, writing emails to the imaginary characters and stalking them on social media. Plot twist: The evil guy you are trying to find is real and you wake up with a horse head in your bed. This would truly take gaming to its next level.

One of my all time favourite games and the only one in this list I actually played. The creatures in this game are the best and most frightening I’ve ever seen in a video game.

While there are thousands of games that use rotation of the phone as an input, I’ve never seen one that takes it to this extremes. Probably for good reasons..

]]>In Ippatsu (working title) you are a lumberjack with a big flamethrower and need to kill as many other players as fast as possible. The gameplay is quite frantic: Every hit is an instant kill and you respawn after only 0,5 seconds. It’s basically Super Hexagon + multiplayer action. We only spend 28 hours making it, but the game is already fun as hell. Have a look at this video:

I can’t wait to learn more about Unity (the game engine we used) and develop more arenas, characters and nasty traps.

]]>