I Created a Politically Corrupt AI

This is our first blogpost on our new website.  Previously we blogged here.  In this post, Marnielle writes about creating a new AI for Political Animals:

I've been using Genetic Algorithm as an aide for game design and development. It fills me with excitement that I can simulate natural selection to help me look for the best solutions to problems. Now I'll tell you the story of how I used it to improve our AI.

My GA knowledge is still somewhat limited. I learned how to write code for it using this website (Go read it. It's awesome! So unlike smarty academic papers.) To give you an idea in casual speak, GA is basically simulating the theory of evolution to look for the most “fit” solution. In this simulation, the species or “individuals” are the solutions to the problem. At the start, a certain number of individuals are spawned with random configurations. As such, most of them are dumb solutions at the beginning. Each of them are then assessed and given a fitness score. The ones with higher score means they are closer to the solution. Based from this initial population, we spawn the next generation. To do that, we either mutate or let them breed (Yes, solutions can mate) with a rule that those with higher fitness score has a higher chance of being selected for mutation or breeding. With this new generation, we repeat the process of assessment and spawning the new generation until we can find that one individual that solves our problem.

When coding a GA, you need some important pieces. These are the individual representation, the fitness assessment function, mutation function, and crossover function. If you have these four, you can run a GA. Individual representation is a bit tricky. How do you represent a solution that can also be mutated and bred? One of the most common is a list of bits. This can be represented by a list of booleans or just integers and use bit manipulation. Mutation is just then flipping a random number of bits. Breeding or crossover is simply exchanging a certain number of bits from two individuals.

Representation by bits is the only representation I know of. It's what AI Junkie taught me and I sticked with it. That's until I've read a book called “Essentials of Metaheuristics”, a highly recommended book. The contents are written in an informal way, not in an academic bullshit way. It's a primer on different algorithms in the field of metaheuristics. Most of it, though, is GA. From there, I learned that you can represent an individual with anything. It can be lists, trees, graphs, your own data structure. Mutation and crossover can be any made up alteration of your representation. It can be adding a child, removing a child, changing a value, swapping nodes and edges. Anything! I realized how dumb I was for not going to that thought.

That gave me an aha moment. What if I automate the creation of our AI using GA. Our AI configuration is very simple. At the same time, AI is also the most neglected part of our game. We haven't touched it for a long time. We have a working AI that I configured by hand. But then, our mechanics have already changed too much that we don't know if it's still competitive. Configuring a new AI would take time.

My team gave me a week to work on this, May 2-8, 2016. I'm not so sure if it would work. What if looking for a better AI takes time, like it would take days running the simulation. I certainly thought so because the assessment function is to let two AI players pit against each other. The one who wins has a bigger fitness score. Now, a single playthrough takes time, even if I speed it up. The point is, making the GA could be a waste of time.

The first thing I did is I made a fast mode of our game. No animations, movement becomes teleportation, removed the standby/wait times, etc. It wasn't easy. I don't have time to write another version of the game solely for GA. Instead, I'm using what we have now and provided a mode that it can be played extremely fast. Finally, I have a mode where AI vs AI takes around 1 minute to complete 15 turns. Still not fast enough, but quite good already.

Then I made something called a “multi frame” GA. Basically, it's GA that is spread in multiple frames. Note that the assessment function is for AI to play the game. So the GA must wait for such game to end before it can move to the rest of the algorithm. In fact, if there are 10 indiduals in a generation, 10 games must be played before moving on to spawn the next generation.

Our AI configuration is all code. We use a utility based AI, by the way. It is represented as a set of “task scorers”. Each scorer has an arbitrary set of “considerations”. These considerations are classes that help compute the score for the task. The AI generates all the possible tasks, scores them using the scorers, and picks the one with the highest value.

My plan is to use GA to generate different combinations of these scorers and considerations until we get the one that beats the current best configuration. Before anything else, I needed my configuration to be saved in a file. Every time the GA finds a better AI, it should save the configuration in a file. So I turned the AI configuration into XML. I used the class names and variables of consideration classes in this file. I load them back using reflection. It looks like this now:

<ScorerSet id="IndividualZero" fitness="1" timestamp="">

<Scorer taskId="Bribe">

<Consideration name="BlockadedDistrictConsideration" />

<Consideration name="TerrorizedConsideration" />

<Consideration name="MinimumBudgetConsideration">

<Variable type="NamedInt" name="minimumBudget" value="1000" />

</Consideration>

<Consideration name="ReachableConsideration">

<Variable type="NamedFloat" name="multiplierIfUnreachable" value="0" />

</Consideration>

<Consideration name="MustHaveMatchingPlatformConsideration" />

<Consideration name="ReachReputationConsideration">

<Variable type="NamedFloat" name="populationPercentage" value="0.85" />

</Consideration>

<Consideration name="BonusTuningConsideration">

<Variable type="NamedFloat" name="bonusToSet" value="1.0" />

</Consideration>

<Consideration name="CommandPointsConsideration" />

<Consideration name="NeighborCountConsideration">

<Variable type="NamedInt" name="desirableCount" value="4" />

</Consideration>

<Consideration name="OpponentStaffCountConsideration" />

<Consideration name="BribeBetterThanCampaignConsideration">

<Variable type="NamedInt" name="minReputationGainDifference" value="1000" />

<Variable type="NamedInt" name="rankIfMet" value="17" />

<Variable type="NamedInt" name="bonusIfMet" value="10" />

<Variable type="NamedFloat" name="multiplierIfMet" value="5" />

</Consideration>

<Consideration name="ScandalCountReachedConsideration">

<Variable type="NamedInt" name="scandalCount" value="4" />

<Variable type="NamedFloat" name="multiplierIfMet" value="0" />

</Consideration>

</Scorer>

<Scorer taskId="RaiseFunds">

<Consideration name="BlockadedDistrictConsideration" />

<Consideration name="TerrorizedConsideration" />

<Consideration name="ReachableConsideration">

<Variable type="NamedFloat" name="multiplierIfUnreachable" value="0" />

</Consideration>

<Consideration name="HasSignificantFundsConsideration">

<Variable type="NamedInt" name="preferredAmountToRaise" value="1000" />

</Consideration>

<Consideration name="BonusTuningConsideration">

<Variable type="NamedFloat" name="bonusToSet" value="1.0" />

</Consideration>

<Consideration name="LowFundsConsideration">

<Variable type="NamedFloat" name="fundsThreshold" value="0.3" />

</Consideration>

<Consideration name="CommandPointsConsideration" />

<Consideration name="HigherReputationConsideration" />

<Consideration name="NeighborCountConsideration">

<Variable type="NamedInt" name="desirableCount" value="4" />

</Consideration>

<Consideration name="NearBailiwickConsideration" />

</Scorer>

<!-- ... Rest is ommitted -->

</ScorerSet>

The mutation function then is just a matter of:

• Add random consideration

• Remove random consideration

• Change some random variables

Crossover between two individuals is simply swapping a random number of considerations.

For the initial population, I used our existing AI as a starting point. I called it “Individual Zero”. The first population are individuals that are mutated versions of him. Now with all the pieces together, I have a GA that looks for a better AI.

First Runs

When I was about to finish, I was too excited to run it. I know it will take time so I plan to run the simulation overnight when I'm asleep. It was Saturday night. I'm sleepy and tired. I fixed some errors that would potentially halt the simulation. I set the settings for the fast mode. Every game lasts only 15 turns. The population size is set to only 10. Then finally I let the baby run. I watched the first few AI matches. After a while, nothing goes wrong, I locked the laptop and went to bed.

When I woke up, I checked it immediately. The simulation had hung. I looked at the Task Manager. 90% RAM usage. So I closed it. But lo and behold, it generated 10 AI configurations, each one better than the last one before it. So I was like “that went well”. I pushed my work and restarted my laptop. Then my laptop won't start. It only said “Diagnosing Your PC”. But I wasn't worried. I know I didn't do anything stupid. The simulation just probably messed up the memory. An hour later, my laptop was alive again. I immediately fixed the memory leak. Lesson learned when writing a resource intensive GA like this one.

After coffee, I decided to play a game against the new AI and see if it's really indeed better. Maybe it's just better against other AI but not humans. So I used my usual strategy. Bribed supporters on the first few turns. I acquired districts as soon as I can. I was winning. Every 5 turns, the game shows this graph about how many supporters each candidate has. I beat his numbers every time. I even have more acquired districts than him. Am I really playing against a better AI? On the last turn, I've made my actions. I knew I was gonna win. I controlled more districts than him. That dread that I probably just wasted time returned again. Election came... the AI won an unbelievable landslide. Jaw dropped. Turns out he didn't care about the voters and was befriending more patrons than me. Well, I guess I created a monster.

Thanks for reading. If you'd like to be updated on the latest Political Animals, please sign up for our mailing list!

How to Playtest Your Game

In this post, Tristan talks about our playtesting method.

The game has finally reached a point where we're ready to show it to the world! Well not yet exactly, but we did show it to a few friends. As any self respecting game designer would know, play-testing a game is one of the more important parts of the game design process. In fact, if there are any doubts about a game mechanic, play-testing it is the way to go. In this article, I talk about how we conducted preliminary play tests for the game, and some of the interesting things we learned during the play-tests.

The Play-testers

First let me talk about the play-testers. Our very first play-testers are a few friends from the game industry who we lured with our charms, err, the promise of free food, and those unlucky enough to sit by our table during the recent Manila Game Jam held at the Ateneo De Manila University(ADMU). Seriously guys we really appreciate you taking the time to sit through the game and answer our questions. Speaking of questions, what play-test would be complete without a play-test questionnaire?

The Play-test Questionnaire

For our first play tests we decided to go for a more general questionnaire, since our primary goal in doing these play-tests is to gauge where the game is in terms of fun, and to see if any problems will arise. As we iterate on the game based on preliminary feedback,  we'll also iterate on the questions, refining them until we're ready to test the game with more people.

Our questionnaire consists of two parts: Pre-game, and Post-game:

Pre Game Questions

The first part of the questionnaire deals mainly with knowing who the play tester is. This is important because answers to the questions will mostly be opinions of the player towards the game. Knowing the play tester will help us later on when deciding how much weight to put on his suggestions/feedback later on. Here are some of our Pre-game questions:

1) Do you consider yourself as a strategy gamer?

2) Please list down at least three strategy games that you have played.

3) Rate your skill as a strategy gamer.(1 lowest,5 highest)

4) Rate your interest in a game about politics. (1 lowest, 5 highest)

5) Rate how clean a campaign you will run.(1 dirty, 5 clean)

Questions 1,2, and 3 allows us to know what the play testers game preferences are which may shed some light on some of his/her Post Game answers later on.  Question 5 is of interest because it tells us how the player plans to play the game ( good or evil) at the start, and later on we compare it to his actual play style.

Tester Plays the Game

After answering the Pre Game questions, the play tester is given a short introduction to the game, the goals of the player, and basic mechanics by yours truly. Afterwards, the player is let loose in the districts of Summer Island to test his political mettle against the opposing candidate. During the course of the game, the play-tester is allowed to comment and ask questions about the game, while we take notes.

Post Game Questions

When winner of the Summer Island elections have been revealed, it's now time for the play-tester to answer the Post Game questions. The Post Game questions deal mainly about the play-testers feelings towards the game. Some of the questions:

1)  How fun was the game? (1 lowest, 5 highest)

2) Which part of the game did you enjoy the most?

3) Which part of the game did you find the most difficult?

4) How difficult is the game? ( 1 lowest, 5 highest)

5) How corrupt were you in the game? ( 1 lowest, 5 highest)

Remember during the Pre Game questions we asked how clean a campaign the player could run? In the Post game questions we ask how corrupt the player's candidate was during the campaign. It was quite surprising and fun( insert evil laugh here) to see that most players ended up being more corrupt than they thought they would be.  But the best feedback was one tester that insisted they would be super corrupt but ended up only being moderately corrupt (wouldn't it be wonderful if more of ouir politicians were like that?)

Play-test Results

First a disclaimer before we present our results. Since we just play-tested with a very small pool of players, results from the play-test are not accurate at all, and should be used merely to present a different perspective on the game. Also, a majority of the play-testers are game industry professionals who have insight into the game development process which may or may not have coloured their reaction towards the game.

The play-testers had an average of 2.78 (skill as a strategy gamer), 3.33 (interest in politics), 2.89 (running a clean campaign), 3.22( corruption), 3.5( had fun), 3.44( game difficulty), 4.17( accurate to political theme).

Aside from the values above, we also received qualitative feedback from the play tests.

Things Players Did Not Enjoy

Too Many Stats to Track

These were game feedback which had keywords like info, and stats  attached to words like Too much or Too many. These feedback seems to deal with the playing having a hard time processing game information hindering them from making decisions during the game.

AI Turn is too Fast

These were game feedback which mentioned keywords like AI, Fast and Quick. These feedback seems to deal with the player having a hard time knowing what the opponent is doing.

Things the Player Enjoyed

Being Corrupt

These were game feedback which mentioned the keywords dirty, bribery, and scandals. The feedback seems to show that the player enjoyed doing bad things in the game.

Dominating Districts

These were game feedback which mentioned the keywords domination, and winning.  These feedback seems to show that the player enjoyed seeing his territory expand visually in the game through the borders of districts he has captured.

Looking Forward To More Play-tests

If you're looking to do a similar process during your own play-tests here's a few things to keep in mind:

• Quantitative data is not useful with a very small pool of play-testers because results won't be reliable. Try using open ended questions in your questionnaire.

• Ask follow up questions. The play-tester rated fun as 4.5? Ask him what kept him from giving the game the full 5 points.

• Always clarify if the play-tester's answer is vague. The play-tester might not mean what you think he means. (at the same team be careful not lead the tester to conclusions)

• Observe which questions the player isn't asking. If there's a mechanic important enough to the game and the player is not asking questions about it, don't assume that the mechanic is clear to the player.

Thanks for reading.  Here's a copy of our playtest feedback questions for your reference.  If you'd like to be updated on the latest Party Animals, please sign up for our mailing list!

Customizing Candidates in Political Animals

I'm a little sad that we broke our streak of consecutive monthly devblog updates last month, but it was quite the hectic month for us as we tried to get work done in between dodging family gatherings for Christmas (A perennial nightmare for Filipino introverts).  We did get quite a bit of work done though, which I'm excited to share with you all today.

Procedural Party Animals

In the process of making Party Animals it was always in the back of our heads to make some aspects of the game procedural in order to increase the replay value of the game.  The way the game is made makes full procedural impossible in the same way a roguelike might be, but there are definitely a few aspects that when randomized will change the way the game plays for you.

First off is the candidates.  We guessed that being able to select your choice of animal candidate would be one of the things that players will like about the game, so we're hoping to making quite a few custom candidates to accommodate everyone's favorite animals (I've had at least one request for a giraffe, which should be interesting).  The candidate's name is also customizable, as well as their traits.  We haven't filled in the candidate's histories yet, but each trait will have an accompanying description alongside it.  Previously our candidates had pretty set histories and traits, but we figured there was more to gain by allowing players to customize their own candidate.

Aside from choosing a candidate's history, you can also rename them, choose their home district and platform as well as selecting the staff you want to hire for your campaign.  We currently only have three staff types right now but that should increase to at at least six by the time we're done.

New Maps

Some of you might have noticed that we also have a selection where you can choose the map you want to play in. New maps can definitely change the way the game is played and at the same time add some visual variety to the game.  The maps will take much more effort than the candidates so we're rolling them out little by little and not announcing anything yet until we've finalized them.  Previously we discussed that the Summer Island map is based on Samar Island in the Philippines.  With the new maps we're hoping to interest players from around the world so they will be very loosely based on countries or continents from around the globe. That said, I'd like to introduce you to Autumn Island!  It's still very rough and has none of the new buildings, but it should be fairly obvious where we're drawing inspiration from.

Thanks for reading.  If you'd like to be updated on the latest Party Animals news including our closed Alpha, please sign up for our mailing list!

The Machinations of Politics Part 2

In a previous blog post I mentioned that we will be using Machinations in designing the game mechanics for Political Animals. Just to summarize, everything in the game, from the districts, staffs,  and events can be considered as a resource, and that the game facilitates the interaction of these resources with each other. I also described a basic node: Resource Pools, and two different types of connections: Resource connections and State connections. If you would like to learn this in detail check out the book Mechanics: Advanced Game Design by Ernest W. Adamsand Joris Dormans

In this post I will be going over the core game-play of Party Animals using Machinations while introducing new nodes: converters and end conditions. Also, this is not a step by step tutorial on how to use Machinations, because the authors already did an excellent job right here. What this post hopes to achieve is to show you how to think about game design using Machinations by using Party Animals as an example. 

One thing important to know,  Machinations is not only a tool for modelling game mechanics, but also a framework for describing  game design on different levels( high to low level). I'll start by showing the game from a high level point of view, then  describe mechanics in detail in succeeding blog posts. 

The Whole is the Sum of its Parts

Party Animals is a 'territory acquisition' game at its core. The main objective of the player in the game is to get more reputation than his opponent before the election, and he does this through various means like campaigning, bribing, and participating in game events.  Look at the diagram below:

More often than not, actions that give the player reputation also require that the player spend campaign resources. There are two important campaign resources in the game: gold and command points. I will not explain the specifics here but for the purpose of showing mechanics I reduced both resources into the resource Funds(see B in the diagram).  

Another thing of note is that the player can also spend funds on various actions like rallying(raise voter concern on district issues), and gifting patrons(improve patron relationship) which improves the efficiency of his campaign.  By efficiency I mean getting more reputation for the same(or less) amount of funds. 

Now let's take a look at the different elements shown in the diagram:

A.  Fund Production - The triangle with an asterisk(A) is a source node. The source node produces funds and the resource connection(arrow connecting  node A to node B) stores the funds into the players Fund pool(node B). 

B. Fund Pool - The circles in the diagram are called pools and they store resources, in this case funds.

C. 'Get Reputation' Converter - The sideways triangle is called a converter.  Converters take in resources as input and transform them into other resources. If you take a look at the diagram you can see that there are two resource connections leading to C , 1) [B toC] , and 2) [D to C] . What this means is that in order to get Candidate Reputation(node E), the player needs to convert funds and District Reputation(node D). Without funds or available District Reputation then the player can't 'mine' reputation.

The converter converts District Reputation and Funds into Candidate Reputation

E. Candidate Reputation - Also a pool node, getting as much candidate reputation by converting district reputation and funds is the goal of the player in the game.

F,G. Campaign Investments - As stated above, the player can spend funds to improve the efficiency of his campaign so that he gets more reputation by expending less resources. Node F is a converter node which represents all actions in the game which can help the player gain reputation. 

As an example, a candidate can give gifts(spend gold) to a patron to get bonus reputation when campaigning in the patron's district.  

Nodes F and G represent actions in the game which improve the efficiency of the candidate's campaign

When the candidate 'invests in his campaign' by spendingfunds, an investment resource is produced from node F and stored in node G. As shown in the diagram, node G is connected to resource connections [C to E] and [D to C] by state connections( dashes). This means that whenever an 'investment resource' is stored in node G, [C to E] and [D to C ]increases by 1 which in turn means that the player can mine an additional reputation resource.

H. End Game Condition - This node determines how the game ends. The state connection [D to H] shows that the game will end when District Concern( node D) is depleted (less than or equal to 0).  Note that this end condition is used only for the purpose of this article.

As stated at the beginning of the post, the diagram shows the game-play from a high level point of view, thus a lot of the game mechanics which will make the game more interesting are missing. But it does show the following:

1) The player has funds to spend.

2) The player spends funds to acquire reputation.

3) The player can invest in his campaign in order to maximize the use of his funds. Which is what the player will be doing throughout most of the game. Now it's only a matter of breakingthese three down or elaborating  on them to make the game more interesting.

Key Concepts to Understand

Some of the skills I think is important when designing games using Machinations are:

1) Abstraction

In the diagram, I've grouped together several game mechanics and reduced them to a single node. For example, the Campaign Investment node groups together mechanics in the game which the player can spend funds on to improve the amount of reputation he can 'mine'.  Learning how to 'see' similarities in mechanics and how they can be grouped will be useful when trying to model game mechanics in machinations.

2) Learn to think in terms of relationships

Game mechanics in machinations are represented by nodes,  resource and state connections. It's important to understand how mechanics in your games are related to each other. When designing games using machinations think about how the different mechanics in your game work together.

3) Learn to see and use abstract resources

In Mechanics: Advanced Game Design, the authors distinguished between concrete and abstract resources. Basically concrete resources are the ones you normally see in the game, and abstract resources are the ones that you don't but are there. Some example of abstract resources that I regularly use are strategic advantage and player skill. An example of advantage as a resource is information in a game with hidden information as a mechanic.  Certainly getting information that your opponents do not have increases your likelihood of winning the game.

In this article I've shown you how we use the machinations tool in modelling our game mechanics, as well as given a few tips which will help you learn the machinations tool. In the next blog post I'll be talking about abstracting and elaborating mechanics as well as show some of the Party Animals game mechanics I mentioned earlier.

Thanks for reading.  If you'd like to tell us about political candidates from your side of the word, you can email us at heypartyanimals@gmail.com. To be one of the first people to try our closed alpha, please sign up for our mailing list!