The Impact of Machine Learning on Computational Software
An interview with Rob van Nieuwpoort from Leiden University
If you want to know what role machine learning will play in developing computational software, then the group of professor Rob van Nieuwpoort at LIACS in Leiden is a good starting point. There, the new possibilities this technology offers are being explored in various ways.
VORtech aspires to be a bridge between academic research in the field of computational science and its application in industry and government. That is why we actively follow what research is being done. This led to an interesting conversation with Rob van Nieuwpoort, professor at the group Efficient Computing and eScience at Leiden University.
Rob van Nieuwpoort has a solid track record in the field of computational science. He was, among other things, director of the eScience center in Amsterdam, which is committed to raising the level of scientific software development in the Netherlands to a higher level. He was also a part-time professor in Amsterdam. For the past year and a half, he has had his own research group in Leiden, where he can fully pursue the research that fascinates him.
Generating efficient code for special hardware
The project “SuperCode” is such a research project. Rob talks about it enthusiastically: “With SuperCode, we want to use Large Language Models (LLMs) such as ChatGPT to generate code for emerging hardware. By that, we mean, for example, the special hardware that is being developed for AI applications. You can also use this hardware for other types of calculations. But if you don’t program them very carefully, you will never get the performance out of it. The idea is to use LLMs to help with that.”
“LLMs are already being used to help with programming. Our students use it a lot. We are going a step further in our research,” says Rob. “We take such an LLM and fine-tune it with examples of code that is optimized for a certain hardware. We also feed the model with information about that hardware. Then we try and see if the model can generate code for a hardware architecture that it is not trained for. If we succeed in having LLMs help with that, then you have both faster software and software that uses less energy. The latter is also becoming an important theme.”
The SuperCode project is open to additional industrial parties who want to take a look and possibly contribute case studies. Rob: “it would be very useful if we could also test these techniques more broadly”.
Foundation models for technical applications
Another project that Rob van Nieuwpoort’s group will be working on, the FIND project, explores a completely different application of machine learning for calculation software. Rob explains: “The idea here is to create so-called foundation models for technical-scientific applications. A foundation model is a machine learning model that has been trained for a broad category of applications. Training such a foundation model is a lot of work, but once you have it, you can then fine-tune it relatively easily to all kinds of specific applications. This is already being done a lot for AI applications, for example to create a chatbot that can advise the user on clothing choices. The foundation model then already has all the knowledge to communicate smoothly with people and with fine-tuning you bring in the knowledge about fashion.
What we want to do is to create these kinds of foundation models for medical diagnostics, for example. You can then fine-tune them to applications for MRI or radio diagnostics. That should make it much easier to quickly create AI applications for these kinds of applications.” The FIND project has already been approved and will probably start soon. In addition to this project and Supercode, Rob van Nieuwpoort’s group has also been awarded other projects. It is clear that their work is also seen as relevant by research funders. Rob van Nieuwpoort: “What I especially want is for our research to have an impact.” That should certainly be possible with these kinds of projects.