Technolution Rust 2019 Roadmap

optie1

11 January 2019

We rely on Rust. So do our customers, sometimes without even knowing it. We build software that is installed on devices that go out into the world, sometimes never to be seen again in years or even decades. But 15 years from now, we may get a request to make a change to that software.

Historically we have been very successful doing just that, first in C and later in Java. Slowly but steadily, we are coming to rely on Rust to be able to create maintainable mission-critical software for our customers.

Therefore our proposal for the 2021 roadmap: Reliability.

This blog post reflects on our journey up to the 2018 edition and outlines our view on Rust's progression in the coming years.

Reflecting on the 2018 edition

The release of the 2018 edition of Rust was pretty uneventful for us. Sure, various tools were made stable which made our daily work easier, and we removed a ton of extern crate ... lines from our code bases. That was pretty much it.

But the journey as a whole was very different from what developers coming from C (which is a large part of our current Rust developers) are used to. They are not used to tracking the state of the programming language they are using. Hardly anything ever changes, so why should they? Working with Rust from 2017 to 2018 was really something else. In order to make the developers in our company familiar with the many changes, we held regular sessions to go through the 'This Week in Rust' newsletters and the release changelogs and to point out the things that were important for our day-to-day development work. Compared to C and even Java, things were changing at a breakneck speed. This has had its effect on our roadmap vision below.

Continued focus on productivity

When it comes to productivity, the Rust theme for 2018, we think that Rust is not there yet. Sure, progress was made with language features like the ? operator and the new module system, but developers switching from languages like Java or the .NET-based languages are in for a shock when it comes to the productivity they can get out of their editor/IDE.

Using the current state of Visual Studio Code and IntelliJ as a reference, even the most basic features, like code completion, jumping to the definition of a symbol and searching for usage of a symbol, are unreliable. Automatic code refactoring, a great productivity gain when refactoring code bases, is completely absent.

On the bright side, the language in itself is already a productive choice for writing stable systems software compared to C. On top of that, cargo, with all its limitations, is a breath of fresh air compared to anything the Java and C communities have created in the last two decades.

Productivity is a very strong argument when trying to convince tech companies to switch to a new programming language. Therefore, we hope that, after 2019, we will be able to say with even greater confidence that we can rely on Rust's productivity.

Slow down with the syntax

We would like to see a slowdown in language evolution. We have been training colleagues who have been working in C for over 25 years, and they are having a hard time learning and effectively using all the new features that Rust has to offer today. We believe that Rust has left the innovation phase and is currently in the early adopter phase. In this phase, a new type of developer is coming to the ecosystem. This is the type of developer whose main focus is getting the job done and who is less focused on using the latest and greatest that technology has to offer. They rely on the idea that the knowledge they currently have will get them through the year. They may be unpleasantly surprised to learn that the software they have written just months ago is considered 'old style' by now.

On the other hand, there are some widely supported changes coming that really should be added to the language. Yes, I am referring to features like *async/await* and *const generics*. But after implementing those, it may be time to consider taking a break for a while.

So after 2019 we would like to rely on longevity of knowledge of the Rust language.

Improving the toolchain

The simple fact that we can reliably install clippy and rustfmt using rustup takes a lot of hassle away from maintaining the systems we built. Also, starting a new project and adding it as a dependency into a bigger codebase takes a fraction of the time it would take in Java or .NET. As said before, we like cargo.

When it comes to building software, the main limitations in cargo are related to other things than turning Rust code into executables. Post-build steps for example. And pre-build steps in dependencies.

While we were used to simply throwing away all previous built artifacts in C and Java on build servers, we have learned that cargo is very reliable in detecting changed artifacts and just re-building those parts that have changed. This saves a lot of unnecessary compile time on developer machines and build servers.

Reproducibility, even years from now, is important to us. This year we will be implementing a caching layer for crates.io to guarantee the availability of previously downloaded crates for years to come.

So after 2019 we would like to say that we can rely on the Rust toolchain for maintainable software and reproducible binaries.

Trusting the Crates Ecosystem

Although the crates ecosystem greatly improves productivity compared to C, we are missing one big feature here: the ability to reliably know where a crate is coming from and if it landed on our systems unchanged.

Rust is a programming language that has the potential to drive the world's most critical systems. Safety-critical and security-critical systems in industry, aerospace, defense and network infrastructure. Since we are very active in these applications areas, we have to be very aware of the strict requirements for reviewing *every line of source code getting into the final system*. The great thing about using cargo is that pulling in dependencies from crates.io is a breeze. At the same time, this is also its greatest risk.

The biggest risk is in the crates that are widely used, but do not need big new features anymore. They are in a maintenance phase, where only small changes are necessary to keep the library up-to-date. This is when the original authors tend to lose interest and are happy to hand over the repository to anyone willing to do the work. Changes will only be seen by a single developer.

Compared to NPM, known from its recent malicious code injection by a maintainer in a widely used library, we are already one step ahead. We get the actual code from crates.io, not some pre-compiled library. This means that vetting the code has value. The problem with an ecosystem as big as crates.io is now, is that not all developers can check all of the code for every version. But as a community we can share that effort.

We want to see a reputation system that gives everyone the option to put a digital signature on a particular version of a library on crates.io. Signatures from the original author(s), other individual developers, companies, auditing authorities. Everyone. An open system that shares the burden of checking code and that gives each member of the community the tools to check if a piece code was vetted, and by whom it was vetted. Adding reputation requirements to build definitions will prevent developers from accidentally including an un-vetted library in their software.

This is basically a call for a multiple-eyes principle. Those eyes should be both human and automated. Automated eyes act like Rust's bors, but for crates.io. An example of this is automatic checking for unsafe in libraries that did not previously do that. Or libraries that are suddenly accessing a network, itself or its dependencies, that did not previously do that. Let's not go the NPM way.

Another problem with the current crates ecosystem is the single namespace for all libraries. After 70 years of software engineering we all know that putting everything in one big namespace is not sustainable. Any chance of fixing that?

So, after 2019 we would like to be on the way to rely on each others code and reviews.

Embedded development

We see Rust as the best chance to (largely) replace C in embedded development. But embedded development comes with challenges that greatly differ from building applications on mainstream OS's. There is no ABI between the application and the OS. The OS (if any) is compiled into the embedded application and loaded as a whole onto the device.

To build embedded applications in a productive way, the board that contains the peripherals required for the project needs to be supported by the vendor with high quality drivers (which is a problem in our experience) and a feature-rich RTOS. The embedded code we currently have in Rust uses C-bindings to the underlying RTOS and drivers. It takes too much time to re-write all the drivers ourselves in Rust, so we start with a vendor-supported framework like FreeRTOS, Mbed, Nuttx or RTEMS and build from there.

We believe that, for Rust to be successful in the embedded space, the road to follow is to start with a feature-rich RTOS, build tons of drivers and aim for vendor support. The current initiatives around the embedded-hal, rtfm and the drivers initiatives are the right steps on this road. But vendor support...

Embedded developers get their board, their initial code and sometimes even their IDE's from the vendor. This is the ecosystem that needs to be replaced when taking over the embedded world.

So looking into the future, we would like to rely on Rust for embedded development in the entire stack.

Training & Certification

We started working with Rust early 2017. We were using the first edition of the Rust Book and we were learning on the job. That was the training for our first three Rust programmers. The next three programmers were lucky enough to start with the second edition of the Rust Book. In the fourth quarter of 2018 we gathered a group of eight experienced C programmers, all having additional Python experience, and took them through an course of our own design. In Q1 2019 we will train another group of eight programmers, bringing the total of colleagues with at least basic Rust knowledge up to twenty two (!).

Creating training material is hard. Evaluating if you did a good job with the training is even harder. When going from early adopter phase to an early maturity phase, with larger commercial adoptation, we need training material that is well thought through. Also, we would like to be able to check whether job applicants (for temp jobs usually) have any kind of certification. Although our company does not see certification as a strong testimonial that someone can be productive at writing good quality code, it is at least proof that someone has taken the trouble to learn the syntax.

In the open spirit of Rust, wouldn't it be possible to create open training material? Ideally, we would like to have training material that differentiates between programmers who are basically new to programming and programmers who come from various existing languages (C, Java/.NET, Javascript, ...). And again, in the same open spirit, wouldn't it be great to have a way to test a developer's basic knowledge about the language?

Closing

Rust's borrow checker is quite an innovation in our field and takes the fear out of systems programming. Still, we think Rust's *hack without fear* slogan can be broadened.

We do not want to fear the speed at which new language features are introduced. We do not want to fear code coming from crates.io. We do not want to fear for a lack of productivity. We do not want to fear training and hiring new colleagues.

We want to rely on Rust, in all its aspects.

 

On behalf of all our Rust developers,

Erwin Gribnau.

 

Contact:

sergedevos-website
  • Serge de Vos
  • CTO

Related items

'Rust' provides well developed memory management

Read more

Publication

Cloudia unlocks Norwegian smart energy meters

Read more

Project

Membership in NIDV corresponds to Technolution’s security ambitions

Read more

News

Tiny OS ThreadX ported to Risc-V

Read more

Project