A genetic algorithm using neural networks in Unity
Genectic algorithms are used to solve optimization/search problems, by mimicking biological processes like natural selection and mutation. They pair very well with neural networks, which for me, is a facinating combination. This is why I decided to learn more about it by implementing such an algorithm by myself. The goal was to train an AI that is able to drive cars down a racing track in Unity.
Car controller
For the car controller I wanted something simple. At the same time, driving the car should be challenging enough, so that training an optimal neural network would not be trivial. I therefore made the amount of acceleration and deceleration dependant on how much a trigger is pushed down. This makes it necessary to occasionally let go of the trigger, to take turns without touching the track barriers, which would slow the car down considerably. Additionally, the car controller is able to poll the exact progress the car has made within the track, which is important to evaluate its fitness. Fitness is used to compare neural networks with one another, to make sure that better genes are more likely to make it into the next generation.
Neural network
I implemented my own NeuralNet class, which not only stores the weights and biases of a neural network, but also provides necessary operators on neural networks. A mutation function randomly alters weights and biases depending on a given mutation rate and strength, to secure genetic diversity. The mating function produces an offspring from two neural networks, which on average consists of a 50/50 mix of its parents weights/biases. In my setup, each neural network has five inputs:
- track barrier distance, 45 degrees to the right of the car
- same as first, but to the left
- 1 if barrier to the right is the inner track barrier, else 0
- 1 if barrier to the left is the outer track barrier, else 0
- current car speed
These inputs can be propagated through the network, to produce two output values. One of them is normalized to a value between 0 and 1 and is used as the car’s acceleration input. The other output is kept between -1 and 1, used for steering left and right.
Evolution Controller
All that was left to do is to create and manage infinite generations of cars, controlled by ever improving neural networks. The evolution controller initializes a population of PopulationSize cars and corresponding neural networks. The cars then try to drive along the track to the best of their individual abilities for 25 seconds. After that, the controller will sort the list of neural networks by their performance and draw mating partners. Networks with better performance have proportionally better odds of being drawn. Mating ends when PopulationSize offsprings are produced. The offsprings then mutate and serve as the next generation. Parameter tuning is very important to achieve a good solution for a specific problem. One of the best sets of parameters I found was the following:
- PopulationSize: 50
- hidden layers: 1
- neurons per layer: 15
- mutation rate: 1
- mutation strength: 0.01
Using these parameters I achieved convergence around generation 44. As a human, I could still beat the maximum recorded fitness level, but it was challenging and needed some practice.
Conclusion
I find these kind of algorithms can be quite useful for video game development. Especially when artificial intelligence for a particularly complex task is needed, genetic algorithms could save development time and even produce a better result.