Part 2: Pioneering New Modelling

In part one of this article, we highlighted an emerging, principled approach to creating neural network models. Going beyond the general transformer-based approaches that have become synonymous with AI, this approach captures the behaviour of physical systems with learned differential equations. In doing so, it has the potential to produce models that better generalise to new environments, creating surrogates that capture the inner workings of a system – not just statistical relationships between inputs and outputs. This paradigm has become fundamental to the work we’re doing at Mach42, underpinning all of the modeling work we do. 

Our early work on physical systems, especially within fusion, convinced us of the power of this approach. Across a range of scenarios, from standard plasma physics simulations to fusion target modeling, we found again and again that conventional neural network approaches were – with a lot of manual configuration – able to predict simple physical parameters reasonably well. When it came to predicting continuous time behaviour, though – particularly under new physical conditions – conventional approaches fell short. Predictions were noisy, irregular and – worst of all – inaccurate. Dynamical-equation-based networks provided an immediate step change in our results: lower errors, higher stability, and models that had the potential to control systems in real time.

It’s within our primary focus area of electronic design automation – EDA – that we’ve seen the most compelling progress, though. Verification – the process of evaluating circuit designs and determining their fitness for purpose – is far-and-away the most time-consuming (and compute-resource intensive) aspect of circuit design. Creating fast circuit models suitable for verification has the potential to cut semiconductor design costs by billions of dollars across the industry. But it’s an extremely challenging task – standard modeling approaches, including those using neural networks, have repeatedly fallen short in a domain where accuracy and speed are critical.

By incorporating dynamical equations into our circuit models, though, we’ve not only been able to dramatically improve prediction fidelity – we’ve been able to develop circuit models with entirely new and more powerful capabilities. Most importantly, as a consequence of modeling in continuous time, we’ve been able to create accurate neural network models that can sit within existing, industry-standard simulators and behave near-identically to real designs – only orders of magnitude faster.

There’s certainly progress being made on the LLM front. Transformer-based models are improving every day, and, as our understanding of their capabilities grows, we continue to find new applications for them. When it comes to real-world physical systems, though, neural networks that capture dynamical equations are at the forefront of a new AI revolution – and, at Mach42, we’re excited to see where this leads us.