Professor Huang Guangbin, who recently earned his full professorship title, agreeably shares some background about himself and his experiences in NTU EEE thus far:
“The past fifteen years of research life and teaching in NTU have been most enjoyable, and I treasure the close friendships I have formed with colleagues, and their continued encouragement and support these past years. We actually formed many ad-hoc research teams amongst ourselves; it is our mutual research interests that unify and bring us together. I admire the fact that every one of our colleagues and collaborators, in their thirst for knowledge and discovery, embody the very spirit of research.
I also deeply appreciate the freedom that the managements of NTU and NTU EEE have allowed us in terms of research topics and interests. This fluidity is essential for the success of research. Great research ideas are never born of pressure and rushed deadlines; they come about from true interest.
The realms of artificial intelligence and machine learning (especially artificial neural networks) have become very popular and successful recently. However, a mere 15 years ago, many people actually doubted whether neural networks were useful. Some researchers started to run away from neural network research areas! It seemed as if the whole world was spending their efforts on tuning the parameters of neural networks, resulting in a lack of progress research-wise in machine learning.
I strongly felt that some challenging research issues may have caused the neural networks research community to become stuck somehow. This fuelled my drive to conduct research in these areas as I believed that I may be able to make some contributions to the research community with my interdisciplinary research and engineering background. And thus, I decided to leave manufacturing industries and joined NTU with a strong mission and commitment to help the machine learning community – especially neural network community – get out of the local minimal in their research.
It has always been enjoyable having discussions with colleagues and collaborators on varying research topics on different occasions. Those discussions I have had will always remain fresh in my memory. I have never forgotten those moments when we gathered, holed up in some nook and cranny in NTU EEE or canteens in NTU, and in the heat of our discussions sparked embers of ideas. Together, we worked to act on those ideas and gave them a place in reality, which ended up attracting thousands of researchers to adopt in the past years. Their influence may burn very brightly – indeed as one of the key machine learning techniques in the future.”
Prof Huang highlights memorable moments in his research journey early on:
“After I joined NTU in 2001, I spent most of my time pondering on two questions: 1) Why were most learning algorithms so inefficient in learning and 2) Why are brains more intelligent and efficient than computers.
There were no clear answers at first, so I started to feel like research in this direction was a little hopeless. A brief but vigorous immersion in literature ensued in a bid to revive my slumping spirits. I looked to the classics and read “Romance of the Three Kingdoms” – a famous classical Chinese storybook – cover to cover several times.
It was a peculiar thing. I was reading this novel for the seventh time one midnight, in 2002. With my brain full of the characters and each of their unique, vibrant personas, I suddenly came to the realisation that over millions of years, there have been trillions of humans and animals, each with a different brain, and there was no way to ‘install’ different learning algorithms in so many different brains for so many different applications and tasks. I thought then that there might, in fact, be some common learning algorithms in those trillions of different brains which are data and application independent.
I called my student to the lab immediately and we tested our data-independent algorithm on that very day in the wee hours of the morning. We went on to name our resulting algorithm the “Extreme Learning Machines (ELM)”. At that time, we realised that we might have found some secret of the biological learning mechanism, that is, biological learning could be implemented without tuning neurons.
Random neurons and random `wiring’ may be two specific implementations of such `learning without tuning hidden neurons’ learning mechanisms. Neurons could be inherited from their ancestors as well. Overall speaking, entire living brains are structured and ordered, but they may be seemingly `random’ and `unstructured’ in a particular layer or neuron slice of brains.
We were surprised that brains, one of the most sophisticated things in the universe, may be very similar to physical worlds which are overall structured but locally full of stochastic Brownian motions – that there was such a similarity between living brains and the physical world! Biological learning is just perfect and beautiful due to the co-existence of its universally structured architectures and locally generated randomness!
Neurons can be independent of particular training data. This has been later validated in rats and monkeys’ brains in 2013 and 2015, respectively. This finding may overturn the conventional biological learning and cognition tenet that the tuning of neurons may be critical to learning.
This new learning philosophy was also completely different from traditional neural network learning tenets. We tried to discuss these new ideas of ours with some pioneers in the field of neural networks but almost none of them believed us in the early days! It was also hard to prove it in theory in the beginning.
Research life is sometimes very lonely to researchers. But in the end, my research team and I felt vindicated when in 2005, we proved the correctness of our hypothesis in theory (after many rounds of mistakes in proofs). Eventually, its theoretical proof was published in 2006. I feel immensely proud of how I spent my first 5 years after joining NTU, which laid the foundation for my team’s future research works. The resultant learning techniques, which are up to tens of thousands times faster than conventional learning techniques, shed light on real-time learning, cognition and reasoning.
It is surprising that although few machine learning pioneers believed the ELM theories and techniques in earlier days, I found that most biologists and neuroscientists could be very receptive to ELM theories and techniques when I discussed with them.
Machine learning researchers usually follow traditional machine learning and neural networks theories and believe that learning can only be implemented based on delicate tuning. Many machine learning pioneers who did not believe in the validity of ELM in the earlier days are now moving to ELM research areas as well!
My researchers and I have had so many interesting discussions with biologists, neuroscientists and artificial neural network experts since.
A great deal of the credit for the continued development and successes of ELM should go to my PhD students. Informal discussions with them unearthed a plethora of important ideas! I used to cook meals for them at home, and we would discuss critical research topics over card games and food on the weekends.
Once, we discovered that the popular machine learning technique, Support Vector Machines, actually provides suboptimal solutions. My researchers and I spent the next six months writing a paper on it, which went to become the most highly cited paper out of hundreds of thousands of IEEE publications since it was published.
Although we always embark on research hoping for fruitful outcomes, reality is not often so kind. We believed that ELM theories could help explain some aspects of Darwinism, so in 2014-2015, we decided to investigate down this branch of thinking. I believe that the inheritance of neurons from their ancestors, and some randomness, such as random neurons and random ‘wiring’, in each new generation could aid evolution and the natural selection of systems (even those of biological organisms). Yet, after two years of hard work, my researchers and I temporarily paused work on this front due to a lack of validatory big data. We will take it up again, sometime in the future.”
Prof Huang also gives his remarks on the current EEE-related projects he is involved in:
“I used to work in manufacturing industries, and liked the real-life hands-on experiences I had accumulated. Having said that, my research experiences in NTU EEE have been just as fulfilling. During my time at NTU EEE, I have also been mainly leading industrial R&D projects sponsored by BMW, Rolls-Royce, Delta Electronics and ST Engineering.”
On the things Prof Huang is eagerly looking forward to in his new full professorship position:
“I’m looking forward to close collaborations with our colleagues in the university and also with collaborators from leading universities and companies in the world. Together, I am confident that we all can discover efficient solutions to overcoming some challenging bottlenecks in machine learning applications.
It will be exciting to see the convergence of machine learning and biological learning with the long-term point of view. Extreme Learning Machines (ELM) may be one of the fundamental `learning particles’ filling the gaps between machine learning and biological learning. I wish we could make some important contributions to fill this gap with the synergy of interdisciplinary researches in machine learning, brain science, smart materials and smart sensors.
Machine learning is still laborious and costly in many implementations. My NTU EEE team’s goal is to help the research community and industrial sectors get out of this local minimum in machine learning research. In the near future, we wish to have some interesting machine learning solutions which can: 1) Deal with complicated applications with a small size of data and 2) Enable pervasive learning and pervasive intelligence. It will be interesting to help move machine learning from engineering to science. I believe that 'machine learning science' will play important roles in the end.
I’m also looking forward to having our spinoff company set up and move forward steadily. I believe that this company will not only help transfer technologies to products, it will also benefit NTU and its family members as well as Singapore.”
Favourite places of Prof Huang in NTU EEE:
“My favourite places include the staff lounge, as well as other cosy corners with comfortable sofa seats around campus where I can brainstorm with colleagues and students on research ideas, and are free to talk about our dreams for research and future plans.”
Interests and hobbies in his free time:
“I am an avid watcher of movies, whether it be at the cinema or at home. Here in Singapore, the natural beauty of the Chinese Gardens is a powerful incentive to exercise, especially when surrounded by all that lovely scenery.
When I get the chance, I also like to go mountain climbing, often with other researchers. It is always enjoyable and productive when we brainstorm during our climbing expeditions! You’d be surprised by how creative one can get when filled to the brim with fresh mountain air!”
Some advice to pass on to students and other faculty members:
“To all my dear NTU EEE faculty members: I have found that, in addition to teaching and research, it is the friendships that make our work as educators more harmonious, creative, productive and enjoyable. The NTU EEE family consist of a warm group of people.
I only wish to say that I treasure the friendships that I have formed, and look forward to closer collaborations with all of you.
To my dear NTU EEE students: There are many golden opportunities awaiting you, especially in this era of machine learning and big data!”
Published on: 30-Sep-2016