Rust for Machine Learning in Production

This is one for our technical readership …

Over the last 14 months Sent-ai-ance has been developing our practice to focus on Machine Learning in Production (MLP). Part of our growth strategy has been to bring in Dr Gareth Digby – a specialist in cloud computing and Systems Engineering. Gareth previously taught on the MSc in Software Engineering at Oxford University. Gareth comes to us from Amazon Web Services. Gareth will lead our cloud infrastructure practice, as well as developing and delivering new courses with us.

In parallel, I have been leading a transformation away from our roots in Python-based code, towards our in-house standard, Rust. Rust is rapidly becoming part of our core capability and moving forward we plan to use Rust for most new projects – particularly web-native projects.

The eco-system for Machine Learning using Rust is in a good place, and continually developing.

• Burn

• Linfa

• Mistral.rs

• Candle

• Peroxide

… and many, many more

Many technical readers of this blog will understand our switch towards Rust. The language both enforces and encourages a level of rigour that is unparalleled by many other programming languages. This is not just around the core syntax and semantics of the language. Rust is hugely supported by an ecosystem of tools that promote security and safety.

But I wanted to say something at a personal and experiential level. The experience of learning and using Rust has been a dichotomy: both frustrating and delightful. Frustrating, because the language enforces so many things that just have to be right. In learning the language I have spent way too much time just trying to get code to compile. Error after error is signalled by my IDE or by the build system. The language really does demand precision in thought and deed. No sloppiness is tolerated here!

Often I have had to resort to re-reading 'The Rust book' and in the worst-case scenarios resorting to Large Language Models for a helping hand. (And it may surprise you to know that I resist that option as far as I am able, before my will is broken. I don't want to sink into that unthinking bliss of over-reliance on an AI tool to do the one thing that I have always been reasonably good at … programming. I want to stay sharp – and that means, for me, doing it for real).

Now I am reasonably tolerant of this learning process. I have made it my business over the years to learn new computer languages whether I needed them or not. This has been part of my personal, professional discipline. So, my target has been to learn a new language every 2-3 years since I left University. And mostly I have succeeded.

I am going to be honest and say that Rust has been one of the toughest to learn. Firstly, it has a large and rich syntax. There are a lot of symbols to learn! The language inherits ideas from a broad class of other languages. I might be accused of over-stating the case to say that it is multi-paradigm .. but that is the way it feels sometimes. Also, some of the restrictions at first feel awkward and uncomfortable.

But the 'delight' comes from the number of times that I finally get code to compile only to have an 'ah-ha!' moment. Of course! This absolutely banana makes sense! Why did I not see it before? This is all so obviously the 'correct' way to do this.

And then there is the phenomena that others have talked about: the low defect rate. Once the code compiles, it so often 'just works'. This is particularly true if one is employing the built-in unit testing mechanism and a Continuous Integration process that includes build steps for integration testing and further static analysis.

I guess that we won't be abandoning Python any time soon for teaching, prototyping and small, discrete analytical data/science tasks. But for Sent-ai-ance, Rust will be our standard for Machine Learning in Production (MLP) going forward.