This is part II in a series on writing a Mender-client in Rust. Read part I.
In this post we are going to explore how to try and create an embedded version of the Mender-Rust binary which we created in the project from the previous post in this series. The original goal is to get it below 1 MB, but this sounds a bit ambitious to me. Let us see how we fare in the world of binary minimization.
If you want to have a look at someone taking this to the extreme, then for inspiration have a look at:
Reducing the size of the final Mender-Rust binary
Part of the goal of the Mender-Rust project is to have a binary available for low-resource devices. At the time of writing, this goal is not satisfied, as the resulting debug binary is:
➜ Mender-Rust git:(master) ls -lh target/debug/mender-rust
-rwxr-xr-x 2 olepor olepor 67M sep. 22 12:35 target/debug/mender-rust
67(!) MB. So it is huge. However, this is a debug build, and not applicable, but still makes me wonder what is actually in there to make the binary so huge (Maybe a good post for another time?). Building for release, will give us a smaller binary like so:
➜ Mender-Rust git:(master) cargo build --release
Finished release [optimized] target(s) in 0.07s
➜ Mender-Rust git:(master) ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 8,8M sep. 22 13:27 target/release/mender-rust
Then stripping the resulting binary results in:
➜ Mender-Rust git:(master) strip target/release/mender-rust
➜ Mender-Rust git:(master) ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 5,5M sep. 22 13:28 target/release/mender-rust
Which is a fair bit smaller, but still larger than what we want, as the size of the original, and fully featured mender-client, which is written is in Go, is approximately 8 MB. However, there are quite a few things that can be done in order to reduce the binary size of the resulting binary.
The first thing we can do to reduce the size of the binary is to compile it with 'opt-level = 's'', which will increase our compilation time, but try to optimize the size of the resulting binary. Let's have a look at what it does:
First add the feature to the release build in cargo:
[profile.release]
opt-level = 's'
Which, before and after stripping, gives a binary size of:
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 8,1M sep. 22 13:37 target/release/mender-rust
➜ Mender-Rust git:(master) ✗ strip target/release/mender-rust
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 4,5M sep. 22 13:37 target/release/mender-rust
Next, let's try the 'opt-level = 'z'' option, which will even further try to optimize the resulting binary size.
First add the feature to the release build in cargo:
[profile.release]
opt-level = 'z'
Which, before and after stripping, gives a binary size of:
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 8,5M sep. 22 13:42 target/release/mender-rust
➜ Mender-Rust git:(master) ✗ strip target/release/mender-rust
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 4,6M sep. 22 13:43 target/release/mender-rust
Which gives me a slightly larger binary, than with the optimization-level set at 's', which is supposed to provide less binary size optimization. Curious...
The next thing we can do to reduce the size of the binary is to compile it with 'link-time-optimization', which will increase our compilation time, but decrease the binary size. Let's have a look at what it does:
First add the feature to the release build in cargo:
[profile.release]
opt-level = 's'
lto = true
Then
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 5,6M sep. 22 13:59 target/release/mender-rust
➜ Mender-Rust git:(master) ✗ strip target/release/mender-rust
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 3,6M sep. 22 14:07 target/release/mender-rust
Where the final binary has now shrinked to 3.6 MB, which is significantly smaller, but still too big for our taste.
[profile.release]
opt-level = 'z'
lto = true
Then
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 5,8M sep. 22 14:12 target/release/mender-rust
➜ Mender-Rust git:(master) ✗ strip target/release/mender-rust
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 3,6M sep. 22 14:13 target/release/mender-rust
Which is the exact same result given by the 'opt-level = s' option in the previous case.
The next attempt will set the 'codegen-units = 1' variable, to further increase compilation times, and try to reduce the final binary size.
[profile.release]
opt-level = 's'
lto = true
codegen-units = 1
And
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 5,0M sep. 22 14:18 target/release/mender-rust
➜ Mender-Rust git:(master) ✗ strip target/release/mender-rust
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 3,3M sep. 22 14:18 target/release/mender-rust
So the binary size does keep shrinking. Let's see if changing to the 'z' option now makes for further optimization.
[profile.release]
opt-level = 'z'
lto = true
codegen-units = 1
Now
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 5,2M sep. 22 14:22 target/release/mender-rust
➜ Mender-Rust git:(master) ✗ strip target/release/mender-rust
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 3,2M sep. 22 14:22 target/release/mender-rust
And finally the 'z' option did manage to give better results than the 's' option.
At this point we have run out of regular optimization options -- meaning that to further decrease the size of the binary, we must remove some functionality. A standard way of doing this is to remove the backtrace of the stack on a panic for our release binary. Let's give it a shot:
[profile.release]
opt-level = 'z'
lto = true
codegen-units = 1
panic = 'abort'
Then
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 4,7M sep. 22 14:28 target/release/mender-rust
➜ Mender-Rust git:(master) ✗ strip target/release/mender-rust
➜ Mender-Rust git:(master) ✗ ls -lh target/release/mender-rust
-rwxr-xr-x 2 olepor olepor 3,0M sep. 22 14:29 target/release/mender-rust
And we have set a new record, bringing the binary down to 3 MB.
However, this is still too big, and I am running out of ideas. A quick google search returns, which has gone through basically the exact same footsteps that I have outlined above, and beyond. One further trick, that did not occur to me is that the standard rust library comes pre-compiled (makes sense), and hence, does not heed the optimization options that we have fed it at build time above.
This is a pending RFC in Rust, but still seems to be a way off for now. Therefore we have to resort to 'Xargo' tool, which is originally meant to build rust for platforms which does not match any of the regular target triples.
Thus after installing 'Xargo', and adding the configuration file 'Xargo.toml' with:
[dependencies]
std = {default-features=false}
the resulting binary is
➜ Mender-Rust git:(master) ✗ ls -lh target/x86_64-unknown-linux-gnu/release/mender-rust
-rwxr-xr-x 2 olepor olepor 3,2M sep. 22 15:01 target/x86_64-unknown-linux-gnu/release/mender-rust
➜ Mender-Rust git:(master) ✗ strip target/x86_64-unknown-linux-gnu/release/mender-rust
➜ Mender-Rust git:(master) ✗ ls -lh target/x86_64-unknown-linux-gnu/release/mender-rust
-rwxr-xr-x 2 olepor olepor 2,6M sep. 22 15:05 target/x86_64-unknown-linux-gnu/release/mender-rust
Where another 400 KB has been stripped off the final executable. At this point there is not really much else we can do, in order to reduce the size of the final executable. One could resort to compression, and various other tricks from the demo scene, but this is a bit over the top for this project I think, and hence it will not be pursued further.
One simple path though, is to look at what actually constitutes our binary. This can be done with the 'cargo-bloat' tool. Let's give it a trial run.
➜ Mender-Rust git:(master) ✗ cargo bloat --release -n 10
Compiling ...
Analyzing target/release/mender-rust
File .text Size Crate Name
0.8% 2.5% 36.9KiB mender_rust mender_rust::StateMachine:...
0.7% 2.2% 32.7KiB unicode_normalization unicode_normalization::tab...
0.7% 2.2% 32.7KiB unicode_normalization unicode_normalization::tab...
0.6% 1.8% 26.4KiB reqwest h2::proto::connection::Con...
0.5% 1.7% 25.1KiB unicode_normalization unicode_normalization::tab...
0.5% 1.7% 25.1KiB unicode_normalization unicode_normalization::tab...
0.4% 1.5% 21.5KiB encoding_rs encoding_rs::variant::Vari...
0.4% 1.4% 19.9KiB regex regex::exec::ExecBuilder::...
0.4% 1.4% 19.9KiB mender_artifact mender_artifact::MenderArt...
0.4% 1.2% 18.0KiB http http::header::name::parse_hdr
25.2% 82.4% 1.2MiB And 5961 smaller methods. ...
30.6% 100.0% 1.4MiB .text section size, the fi...
And for the largest crates
➜ Mender-Rust git:(master) ✗ cargo bloat --release --crates
Compiling ...
Analyzing target/release/mender-rust
File .text Size Crate
6.1% 20.0% 293.2KiB std
5.9% 19.3% 281.7KiB reqwest
3.1% 10.1% 147.2KiB unicode_normalization
1.8% 5.8% 84.2KiB regex
1.6% 5.3% 78.0KiB regex_syntax
1.4% 4.4% 64.9KiB mender_artifact
1.3% 4.3% 62.6KiB h2
0.9% 2.9% 42.8KiB aho_corasick
0.8% 2.6% 38.7KiB mender_rust
0.8% 2.6% 38.4KiB http
0.8% 2.5% 37.1KiB [Unknown]
0.7% 2.3% 33.8KiB hyper
0.5% 1.7% 25.5KiB encoding_rs
0.5% 1.6% 23.5KiB url
0.4% 1.3% 18.9KiB publicsuffix
0.4% 1.3% 18.8KiB serde_json
0.3% 1.1% 16.2KiB futures
0.3% 0.9% 13.3KiB tar
0.3% 0.9% 12.9KiB miniz_oxide
0.2% 0.8% 11.2KiB time
2.5% 8.2% 119.7KiB And 56 more crates. Use -n N to show more.
30.6% 100.0% 1.4MiB .text section size, the file size is 4.7MiB
Which shows that the 'reqwest' crate along with it's dependencies, are taking up a lot of space. This probably means that, in order to further reduce the size of the binary, the reqwest dependency must go. One option is to rely simply on the functionality contained in the standard library, but that is a post for another time.
Farewell!
Ole
