Python, C++ and the productivity myth ? and some bits on Nvidia CUDA

Note : I am saying all of this in the context of scientific and numerical computing !

This will be kinda long post !

These are some of my thoughts about rust.

Python is one of the most used programming language which is not being used different application domains such as basic scripting, web development to scientific computing and high performance computing. But Python compared to it's competitors like Java, Go, etc. So then why is it so popular and why especially in scientific computing and machine learning domains. Some of the main reasons most people would mention are

1. Easy to write and friendly syntax
2. Availability of ecosystem of libraries for scientific computing and machine learning

These are the two main reasons why Python is the most used language in that domain, but everyone knows that Python is terribly slow and in scientific computing that is the only thing you don't want to have (also the GIL issue). So to fix this people came up with libraries written in C/C++ and then call them from a Python wrapper like numpy. Numpy and Scipy were the two main libraries which excelled Python into being one of the most used libraries in scientific domain. Then came IronPython kernel REPLs and Jupyter notebook's which made it easy to do fast prototyping and sharing your results as runable reports using the Jupyter notebook. But the issue persist with Python having a strong dependency on C/C++ codebases that run in the backend. If anything would to go south your are pretty much be trying to debug large C/C++ codebases which themselves have 3rd party dependencies on different kernel libraries like OpenBLAS, intel MKL, etc. People would argue that Python is a good choice since it's easy and the scientist with no real CS experience won't be able to write C++ code would not be trying to understand and debug template heavy C++ libraries and link time errors.

Another big issue with Python is the lack of static type signatures so all your type inference happens at runtime and although you can use type annotation now with a decent IDE support it's still not a true static type system.

<center>Python Example</center>

def add(a : float, b : float) -> float:
    return a + b

<center>Rust example</center>

fn add(a : f64, b : f64) -> f64 {
    return a + b;
}

So with type annotations you are pretty much writing typed code without any strong type checking. Dynamic Pseudo code style of writing code is good for one off scripts but not for a large scale project.

Since loops are the main point of optimization in numerical programs Python has numba which can use Python bytecode to convert it into LLVM IR which can then be used to execute the program as a JIT with support for both CPU and GPU codegen via LLVM. But by that point if you're adding a type annotation convention and the ability to execute machine code why not just use a strongly typed compiled programming language. So to me this seems be an issue, Python = easy because dynamic type system but then add no effect type annotation but people won't shift to using a more strongly typed language (an exception would be in using a REPL (Read Eval Print Loop) but people don't do serious software development entirely in a REPL like ipython).

I have on multiple occasions experienced linking error and other types of issues which working with Python and pip and in R as well. If the main benifit of using Python is that you don't need to use C/C++ but the whole backend is written in C++ then I would pray that the backend has no serious error or else I would have to debug C++ eventually (and C++ and cmake is famously developer friendly /s).

So the main reason to choose Python for scientific computing are

1. Easy to write :
   But you also won't get any type system help to manage large projects or any
   compile time checks. I get people can easily write Python but then you would
   assume they can fix C++ errors too!
2. Easy to prototype :
   Sharing your work using Jupyter notebooks and developing prototypes is easy in
   Python most of the time (unless something goes wrong in the packages you
   installed or the installation)

This introduces the two language problem, in which you prototype in one language (mostly a dynamic scripting language like Perl, PHP, Python) and then rewrite it in C/C++/Fortran/Asm (a lot of low level inner loop are written as computational kernels in platform specific assembly for cache locality optimization and other assembly specific optimization).

Now enters Julia a programming languages introduced in 2012 to solve this issue. It does this by being a JIT compiled language with a garbage collector (unlike Python's reference counting Julia uses Mark-and-sweep GC). You can gain pretty decent performance gain in pure Julia compared to C/C++/Fortran.

But Julia itself has issues with program correctness and GC could be an issue in real time systems such as robotics etc and when you need minimal runtime and AOT (Ahead of time compilation) compilation to a specific hardware (a new domain specific programming language embedded in Python called Taichi uses C backend since Python doesn't support AOT compilation).

Well you need C++ again (back to square one).

So now I think Rust can offer both high level productivity and fast performance and AOT compilation to different platforms by default.

At first glance Rust supports

1. Better library organization and a dependable build system (cargo) :
   C/C++ are pretty bad at this even with Cmake.

2. Both backend library and the frontend end user uses Rust :
   In Python all you are doing is kinda metaprogramming the C++ backend. In Pytorch
   you can use Python and then dispatch this to the backend Aten tensor runtime.
   Julia fixes this since all the Julia libraries are written in just Pure Julia so
   if you know Julia, you can easily understand the backend implementation too.
   Rust can be the same but offer better developer experience with a better type
   system and compile time error.

3. Rust provides memory safety without using a GC so can be used in Real time system
   such as robotics and other embedded application which depend on these machine
   learning libraries.

4. Software bugs for some reason are acceptable, developers just assume their
   libraries could have serious memory bugs and concurrency bugs but they still get
   shipped (Taichi's backend in written in C/C++ and is meant to be used in mobile
   applications but this can introduce memory bugs and increase the attack surface
   for bad actors). Rust can improve both of these issues at compile time in most
   cases.

Rust to me seems to be a very good contender for scientific applications such as numerical analysis and machine learning, etc. Type driving development with rust's type system should be encouraged more to catch semantic error at compile time (see Idris for more on type drivin development).

On the side note

Fortran 95 among C/C++/Fortran languages has the most sound syntax for high performance computing. Most people think Fortran is a dead language but it's still used in mostly earth system science and geology, etc.

Rust still lacks in HPC and GPGPU and accelerator domains. Rust needs more community and vendor support from companies like AMD, Nvidia, etc to better adopt Rust in that domain. Nvidia has a monopoly in GPUs and also on Machine learning framework since GPUs are essential to the success of any Deep learning or Machine learning library so all libraries prioritize CUDA (a proprietary API from Nvidia) and support for AMD is quiet murky (Pytorch has support for AMD HIP).

Facebook's Pytorch I would assume is the most used Deep learning framework. Pytorch when it first came had a researcher friendly benefit over tensorflow 1.0 called define by run API (which was actually first introduced in 2015 by chainer, chainer is now retired and Pytorch has the monopoly now).

These were just some shower thoughts I had about Rust and it's benefit over other languages in scientific computing ecosystem.

If people and companies can do an insane level of engineering to make python faster they can try to adopt rust more in their ecosystem as well.

Edit : this is not about writing one off scripts for doing basic analysis and data processing and visualization but more about deployment and runtime. Anything you would do in a jupyter type notebook and in the ipython REPL is more suited towards scripting languages like Python.

For example Taichi is not something you would develop in a REPL its a tool for writing serious programs and then deploying them on various hardware. Building a big program and then dispatching it all to C++ can be done in just Rust.