Understand before you Process
J. R. Firth said “You shall know a word by the company it keeps.” The state-of-the-art has taken this literally as can be seen from the current approach. A large set of corpora is fed into the neural nets or similar digital contraptions named transformers, every word is analyzed in “context” of every other nearby word and this underlying approach drives innovation in NLP. And probably it’s aptly named as Natural Language ‘Processing’, because after all it is huge amount of data ‘processing’.
Elon Musk says that thinking from first principles is the way to do things if you want to do something different. There is a need to use this strategy now because munching on more and more data to find correlations is not going to yield us what we are looking for. Geoffrey Hinton said, “The future depends on some graduate student who is deeply suspicious of everything I have said.” Which is quite similar to what Elon Musk means. You need to keep an open eye and you should not be blinded by something just because everyone is pursuing that.
Let’s think for a moment what Firth actually meant was to ‘understand’ every word in ‘context’ of the other words based on their meanings, like a human does, to fully understand and grasp what is it that the words are trying to say. Shouldn’t that be called Natural Language ‘Understanding’? Context not just in proximity of occurrence in a corpus but also in context of meaning of other words.
If we think that we have solved or are very close to solving NLU, we would be delaying the change that is needed. John Searle’s argument exposes the truth. I am not saying that Deep Learning is useless. But relying solely on speed and not thinking about the engine cannot help you win the race (if it is one).
“Design is not just what it looks like and feels like. Design is how it works.” — Steve Jobs. This also happens to be a philosophy that nature exudes. A marvelous design which is reflected outside as well as in the intricacies too. Humankind has always drawn inspiration from nature to create. Be it art or science, nature has always been a motivational factor. And the processing of language to understand it is outright unnatural.
I think we have it backwards. We are trying to understand language by processing. What we actually need to do is to first understand language and then process it to build lasting and robust solutions. Now, to put a ding in the Universe, we need to put some round pegs in square holes, we need to leave the Navy and become Pirates, become misfits that don’t adhere to the tried and tested but instead incite a revolution by thinking different. And this is the time to do it before AI winter strikes again.
