logo

Static Vs Dynamic Linking for very large and complex software.

61 points InsanityBlossom
View on Reddit ↗

Hi,

At work, I write C++ extensions for DCC (Digital Content Creation) software like SideFX Houdini and Nvidia Omniverse.

I love and use Rust whenever I can, mostly for hobby projects and small-ish tools and I'd prefer it 9/10 times to C++, however I can't stop thinking about how Rust can be used to build really complex software if we want it to ever "replace" C++ :)

I'm curious about feasibility/practicality/future strategy for building complex, modular and extensible Sofware in Rust, given its strong leaning towards static linking and lack of stable ABI.

This post is not so much about Rust, rather about the beaten-up topic about static VS dynamic linking and close vs open source.

I'll be using SideFx Houdini as an example of a state-of-the-art piece of software in terms of engineering complexity. It consists of dozens of in-house libraries and modules as well as a handful of 3rd-party libraries many of which are beasts on their own. It's very modular and incredibly extensible via plugins.

Modern DCC C++ software consists of many small-ish components which are often separate DLLs. This enables:

  • Patching/Hot fixes - ability to recompile a single module and ship it to the users.
    • Prety much impossible with static linking? Shipping the whole new single binary is not an option (Sofware like Houdini ships with 1.3 GB of binary files \[custom + 3rd-party\])
  • Plugin architecture - ability to distribute SDK/ modding toolkit.
    • It's very, very hard/impossible due to static linking, otherwise one need to invent a complex FFI-based plugin architecture or use something like "abi\_stable" crate. It looks interesting, but has anyone tried it in production? What would be the overhead?
  • 3rd-party libraries - ability to use commercial, close-sourced binaries.
    • Rust would need to have a stable ABI for this + some sort of interface (header) files.

Side question: What can be done to enable Rust for closed-source development while still providing some sort of an API crate - analogous of public header files in C++?

Don't get me wrong, I'm not bashing Rust here, and I understand and appreciate the benefits of static-linking. Static linking is perfectly suitable for many, many use cases, but apparently not all. But I'm having hard times imagining how static-linking can be scaled to the level of what Houdini is.

What do people think about this? Will Rust need to eventually define a stable ABI to enable this? I know it's technically possible to build dynamic libraries in Rust and ship a bunch of rlib files, but "possible" is one thing and "feasible/practical/common" is very different.

Cheers!