This is a review of the year of DeepMind in 2017. The brilliant aspect of DeepMind, of course includes AlphaGo, AlphaZero. But it also includes dimmer aspect such as privacy concern whether DeepMind intrude patient's privacy....
This is another review article by StatsBot, which has wider perspective on what deep learning technology was important. It focuses more on GAN with 10 subsections.
This last review is from TryoLabs. Of course, it is more focused on NLP.
Long time readers of Weekly know that we are big fan of Tim Dettmers on blog, in particular his advice on deep learning hardware. But now he surprises us by saying that we are at a "Deep Learning Hardware Limbo". What does he mean?
The main thesis of his article is that Titan V is overpriced at ~$3000. In that price tag, it will push a lot of consumer-level of deep learning developers to choose between AMD which has a lower price tag and proved to have similar performance. Dettmers also believes that Nervana's Neural Network Processor (NNP) could also compete with Nvidia if its price tag is under $2000.
So how do we see it? Nvidia has built their business on a complete solution of GPU processing, both in the hardware and software platforms. Last year or so, she also expands to a cloud solution. So any hardware competitors need to break such moat will need to come up with corresponding software solutions. More importantly they need to work with the deep learning community to make sure enough support is there for various toolkits such as Tensorflow, PyTorch.
AMD is perhaps the closest when it comes to such competitions. But Dettmers also added
AMD is in limbo itself – in software limbo. It seems they want to abandon OpenCL for HIP but currently they officially still push and support the OpenCL path. If they push through with HIP, and if they put some good deep learning software on the market (not only libraries for convolution and matrix multiplication but full deep learning frameworks, say, HIP support for PyTorch) in the next 9 months then their release of their new GPU in 2018Q3 has the potential to demolish all competitors.
All-in-all, we like Dettmers' view. Of course, more competition is usually good for consumers so we will see what happens in 2018.