Thoughts From Your Humble Curators
Woohoo! As deeplearning.ai launched last week, we started to see more reviews of the class. We will look at one by Arvind Naragaj. We will also zero in on one of the optional series within the class, called "Heroes of Deep Learning". This week, we will look at the Prof. Hinton interview by Prof Ng.
Oh, how about the OpenAI DotA-2 bot? Has it conquered the world of DotA-2 yet? From what we gather so far it doesn't seem to be the case..... So let's take a look in our Fact-checking section.
Other than deeplearning.ai and DotA-2, Stanford also just released the latest videos from cs231n 2017. So check out our Open Source Section!
As always, if you like our newsletter, please subscribe/forward to your colleagues!
OpenAI's Dota 2 Bot In Perspective
One of the biggest news last week is perhaps an OpenAI bot was able to beat pro Dota 2 player Dendi) (See Footnote 1). Public outlets rush to report the news and many of them reminded us how dangerous AI can become. And as you might also know, Elon Musk who launch OpenAI, says,
OpenAI first ever to defeat world's best players in competitive eSports. Vastly more complex than traditional board games like chess & Go.
And remember in Issue 24, we looked at DeepMind/Blizaard Starcraft II environment, and we said [on using reinforcement learning on SCII],
So far, DeepMind researchers are still perplexed by the problem - and all RL algorithms so far cannot beat the built in AI agents.
So had we perceived the status of technology incorrectly? We are certainly not the only group who felt surprised. AI researcher, Danny Britz also feels the same. So is SalesForce researcher, Stephen Merity
Since this issue is already discussed quite well by Britz' blog post and Merity's tweets' discussion. The Verge piece is pretty good if you want a less technical piece. OpenAI researchers also wrote two messages on the task. (Part I and Part II)
So we will just extract several important take-aways here:
- A multiplayer online battle arena (MOBA) is not an real-time strategy (RTS) Game. e.g. League of Legends or Dota 2 are MOBA, whereas Starcraft I,II are RTS. They look the same, but computationally they can be very different. A MOBA has significantly fewer actions to choose from because you only control one single character. Whereas RTS require you to control not only the Heroes, and it requires you to control all the buildings.
- OpenAI bot is a 1v1 bot. And A MOBA 1v1 game is very different from a MOBA 5v5. Most tournament game in DotA-2 is actually 5v5, So the machine has to deal with a team of 5 coordinated players. So even OpenAI researchers opine in their post: "1v1 is complicated, but 5v5 is an ocean of complexity." Also playing a MOBA 1v1 game usually mean you and your opponent will use the same lane, so reflex of the player will be the key. Of course, in this case machines have an huge edge.
- Then you should observe that DotA-2 API actually provide a lot of vital information which give advantage to the bot. For example, as the Verge piece points out - distance information can be easily access and give advantages to machines, which human doesn't have such advantage.
- Consider all these, many also observe that Open-AI engine has many human element involved. The e.g. Heroes was chosen manually out of the 110+ choices. Shadow Fiend was chosen. So the character picking is not done by machine. Then there is a key technique of creep blocking, which allows creep to reinforce a defense. Turns out it is trained separately.
So all of these 4 points should make you convince that we seem to be quite far away from beating general RTS games, not to say making Skynet in general.
So how do we see it? Our opinion is very similar to Britz - while that we believe popular outlets and Musk's comment are over the top. By its own, OpenAI DotA-2Bot is still an impressive engineering project. Their status is currently comparable to say DeepBlue before it met Kasparov. There are some known issues, such as many players rumored that they can beat the bot by a technique called creep-control. But it may be a bug in the engine, and expect OpenAI researchers would fix it one day.
On the other hand, you should still notice that beating DotA-2 in 5v5 and a general RTS game are still faraway from us. Hopefully saying so would stop your nightmares (and frankly, curb your enthusiasm) of an AI-powered doomsday machine.
- If this is the first time you heard of DotA-2, check out this video for the gameplay?