You're right about reference [2], which can alter things by ~1 order of magnitude (words are usually ~3-10 tokens).
Additionally as others have pointed out, we don't live entirely in the text world. So, we have the nice benefit of understanding objects from visual and proprioceptive inputs, which is huge.
The paucity of data argument made well-known by Noam Chomsky et al is certainly worth discussing in academia; however, I am not as moved by these arguments of the stark differences in input required between humans and ML as I once was.
In image processing for example, sending 10k images in rapid succession with no other proprioceptive inputs, time dependencies, or agent-driven exploration of spaces puts these systems at an enormous disadvantage to learn certain phenomenon (classes of objects or otherwise).
Of course there are differences between the systems, but I'm beginning to be more skeptical that saying that the newer ML system can't learn as much as biological systems given the same input (obviously this is where a lot is hidden).
Thank you for the tokens-to-words factor! Much appreciated.
I'm definitely in agreement that multi-task models represent an ability to learn more than any one specialized model, but I think it's a bit of an open question whether multi-task learning alone can fully close the digital-biological gap. Of course I'd be very happy to be proven wrong on this though by empirical evidence in my lifetime :)
Of course there are differences between the systems, but I'm beginning to be more skeptical that saying that the newer ML system can't learn as much as biological systems given the same input (obviously this is where a lot is hidden).