A number of people recommended this post to me, and it is indeed good and worth reading. I say that only partly because it provides evidence that aligns with the preconceptions I already had :P
Specifically, here is what I wrote in this post:
I was thinking about this stuff after I was arguing about deep learning the other day and claimed that the success of CNNs on visual tasks was a special case rather than a generalizable AI triumph, because CNNs were based on the architecture of an unusually feed-forward and well-understood part of the brain – so we’d just copied an unusually copy-able part of nature and gotten natural behavior out of the result, an approach that won’t scale
The gist of Sarah’s post is that in image recognition and speech recognition, deep learning has produced a “discontinuous” advance relative to existing improvement trends (i.e., roughly, the trends we get from using better hardware and more data but not better algorithms) – but in other domains this has not happened. This is what I would expect if deep learning’s real benefits come mostly from imitating the way the brain does sensory processing, something we understand relatively well compared to “how the brain does X” for other X.
In particular, it’s not clear that AlphaGo has benefitted from any “discontinuous improvement due to deep learning,” above and beyond what one would expect from the amount of hardware it uses (etc.) If it hasn’t, then a lot of people have been misled by AlphaGo’s successes, coming as they do at a time when deep learning successes in sensory tasks are also being celebrated.
Sarah says that deep learning AI for computer games seems to be learning how to perform well but not learning concepts in the way we do:
The learned agent [playing Pong] performs much better than the hard-coded agent, but moves more jerkily and “randomly” and doesn’t know the law of reflection. Similarly, the reports of AlphaGo producing “unusual” Go moves are consistent with an agent that can do pattern-recognition over a broader space than humans can, but which doesn’t find the “laws” or “regularities” that humans do.
Perhaps, contrary to the stereotype that contrasts “mechanical” with “outside-the-box” thinking, reinforcement learners can “think outside the box” but can’t find the box?
This is reminiscent of something I said here:
My broad, intuitive sense of these things is that human learning looks a lot like this gradient descent machine learning for relatively “low-level” or “sensorimotor” tasks, but not for abstract concepts. That is, when I’m playing a game like one of those Atari games, I will indeed improve very slowly over many many tries as I simply pick up the “motor skills” associated with the game, even if I understand the mechanics perfectly; in Breakout, say, I’d instantly see that I’m supposed to get my paddle under the ball when it comes down, but I would only gradually learn to make that happen.
The learning of higher-level “game mechanics,” however, is much more sudden: if there’s a mechanic that doesn’t require dexterity to exploit, I’ll instantly start exploiting it a whole lot the moment I notice it, even within a single round of a game. (I’m thinking about things like “realizing you can open treasure chests by pressing a certain button in front of them”; after opening my first chest, I don’t need to follow some gradual gradient-descent trajectory to immediately start seeking out and opening all other chests. Likewise, the abstract mechanics of Breakout are almost instantly clear to me, and my quick learning of the mechanical structure is merely obscured by the fact that I have to learn new motor skills to exploit it.)
It is a bit frustrating to me that current AI research is not very transparent about how much “realizing you can open treasure chests”-type learning is going on. If we have vast hardware and data resources, and we only care about performance at the end of training, we can afford to train a slow learner that can’t make generalizations like that, but (say) eventually picks up every special case of the general rule. I’ve tried to look into the topic of AI research on concept formation, and there is a lot out there about it, but a lot of it is old (like, 1990s or older) and it doesn’t seem to the focus of intensive current research.
It’s possible to put a very pessimistic spin on the success of deep learning, given the historically abysmal performance of AI relative to expectations and hopes. The pessimistic story would go as follows. With CNNs, we really did find “the right way” to perform a task that human (and some animal) brains can perform. We did this by designing algorithms to imitate key features of the actual brain architecture, and we were able to do that because the relevant architecture is unusually easy to study and understand – in large part because it is relatively well described by a set of successive “stages” with relatively little feedback.
In the general case, however, feedback is a major difference between human engineering designs and biological system “design.” Biological systems tend to be full of feedback (not just in the architecture of the nervous system – also in e.g. biochemical pathways). Human engineers do make use of feedback, but generally it is much easier for humans to think about a process if it looks like a sequence of composed functions: “A inputs to B, which inputs to C and D, which both input to E, etc.” We find it very helpful to be able to think about what one “part” does in (near-)isolation, where in a very interconnected system this may not even be a well-defined notion.
Historically, human-engineered AI has rarely been able to match human/biological performance. With CNNs, we have a special case in which the design of the biological system is unusually close to something humans might engineer; hence we could reverse engineer it and get atypically good AI performance out of the result.
But (I think; citation needed!) the parts of the brain responsible for “higher” intelligence functions like concept formation are much more full of feedback and much harder to reverse engineer. And current AI is not any good at them. If there are ways to do these things without emulating biology, many decades of AI research has not found them; but (citation needed again) we are no closer to knowing how to emulate biology here than we were decades ago.
That might be for the best. In order to hold the economy together (for human workers) and ensure human safety, we need AI to develop slowly enough that its arc of development can be directed towards human goals.