Skip to main content

Knowledge, knowledge, knowledge

· 3 min read

*This post is about a Starcraft bot I am developing that uses machine learning. The project is being developed as part of the "Daj Się Poznać 2017" contest.

The next few weeks will be spent blogging and creating a project for "Daj Się Poznać 2017." contest. The project requires things I don't know how to do very well yet. That's why I decided that this post will be such an attempt to encompass everything I should focus on in the near future. I really have a lot of bookmarks, especially articles from Arxiv, it is necessary to embrace this mess, make a selection and finally start reading and processing it :)

Books and courses​

I will definitely have to learn reinforcement learning eventually, starting with these basic and standard methods and ending with the current "state of art".

  1. Reinforcement Learning: An Introduction - for starters, I'm going to try to learn from this supposedly classic position on the subject of reinforcement learning by Richard S. Sutton and Andrew G. Barto.

  2. Kurs Reinforcement Learning na Udacity - this will have to be started and finished again from the beginning.

  3. UC Berkeley CS188 Intro to AI - someone from the contest's Slack recommended this to me, I haven't had time to review the whole thing, but it looks interesting.

I guess that's it for the basics. Next will be more interesting, I collected some links from r/machinelearning, the number of them in my bookmarks is overwhelming, so I chose the ones I think are the most relevant:

Arxiv​

  1. Playing Atari with Deep Reinforcement Learning - probably already a classic work about learning a neural network based on pixels. I intend to create one of the first models of my bot based on it.

  2. Deep Recurrent Q-Learning for Partially Observable MDPs - Starcraft is POMDP, so this article should come in handy to some extent.

  3. Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning - something that might be useful to me, I will definitely need some form of curriculum learning.

  4. Asynchronous Methods for Deep Reinforcement Learning - to quote the abstract: "The best performing method, an asynchronous variant of actor-critic, surpasses the current state-of-the-art on the Atari domain while training for half the time on a single multi-core CPU instead of a GPU". This will definitely come in handy for me as I don't have a GPU.

  5. PathNet: Evolution Channels Gradient Descent in Super Neural Networks - fresh work, training multiple neural networks with knowledge transfer and selection using a genetic algorithm.

Plus, of course, the list from awesome-deep-reinforcement-learning.

Summary​

That's it for now, I've chosen what I think are the most important things. I will spend this month learning about theory and creating simple models in OpenAI Gym. As for the project, this week I will be playing around with replay analysis using sc2reader and s2protocol.