Comparison of MATLAB Versions for CLI and GUI Performance
Although the MATLAB core has essentially stayed the same throughout the years, the MATLAB language and runtime has with every release been extended with more functionality as well as featured significant improvements to the interpreter and Just-In-Time (JIT) compilation engine. As developer of the FEATool Multiphysics simulation toolbox which features 10-years of both graphical user interface (GUI) and command line interface (CLI) backwards compatibility with MATLAB (back to R2009b), it would be very useful to be able to advise users which MATLAB version(s) to recommend in different situations for performance and/or other reasons.
With this in mind, and as no official or comprehensive benchmarks of MATLAB versions are available, both GUI and CLI script performance is compared for MATLAB versions R2011b (the subjectively snappiest, most responsive, and currently preferred version in the developers opinion) and R2019a (the latest version at the time of writing). Timings and tests are performed with the FEATool Multiphysics toolbox on a typical laptop with the Microsoft Windows 10 operating system and Intel CPU. The FEATool software is used for benchmarking as the toolbox features a large number of pre-defined and built-in validation test cases and fully automated (GUI) tutorials which can be used to automatically and consistently measure performance.
MATLAB GUI Performance
In the first step, the GUI model test suite is used for timing
runs. This test suite runs about 40 pre-defined multi-physics
simulation models (the test suite can be run by starting FEATool with
featool testt command). The examples are completely automated so
that all GUI callbacks are automatically and consistently run just as
a user typically would do manually. The timing averages from five runs
can be seen in the table below.
|MATLAB GUI Tutorial Tests|
|3142 s||2486 s|
From the results it is clear that MATLAB R2011b runs significantly
faster (around 25%) than the latest version R2019a even with the
improved JIT optimization as the subsequent MATLAB versions have. Note
that the MATLAB GUITool UI framework used by FEATool has
been specifically developed and written to run fast as possible using
procedural code. (Due to backwards compatibility the framework only
set(h,'prop',val) calls, and not struct
call options introduced with handle graphics hg2. Also these results
do not apply to the newer web graphics backend introduced in the last
The Mathworks own GUI Layout Toolbox was also tested in development and at the time shown to have significant overhead and cost due to the object oriented design, and just not performant enough for even moderately complex GUIs such as the ones use by FEATool.
MATLAB CLI Performance
As these tests include everything from running the GUI to CAD geometry
and model pre-processing, mesh generation, linear equation solving,
plotting and visualization, we can not from the tests above
conclusively tell what is slower in newer versions of MATLAB. As
FEATool also is fully scriptable (one GUI action maps to an equivalent
MATLAB function, and FEATool allows saving the modeling process in
both binary and script formats) we can also run the tests without the
GUI for comparison. The corresponding averaged results for five runs
are shown below (starting FEATool with
featool testm runs all the
available script models in the FEATool examples directory).
|MATLAB CLI Examples Tests|
|812 s||976 s|
Without using any GUI functionality the results are now reversed with MATLAB R2019a being somewhat faster. As the FEATool code base is both highly vectorized and optimized, and since Finite Element Analysis (FEA) simulations makes significant use of pre-compiled numerical libraries and sparse linear solvers such as BLAS, LAPACK, Umfpack/SuiteSparse etc. which are not really influenced by JIT improvements, a quite modest 10% speedup is maybe what one can expect from any JIT improvements (instead of a major increase in speed).
With the automated GUI and CLI simulation tests we have been able to establish that current (R2019a) MATLAB JIT optimizations have resulted in a modest speed increase on average, however older versions of MATLAB are significantly more responsive for using GUI functionality. To FEATool users we can therefore recommend using version 2011b when using the GUI. As FEATool also supports saving and exporting models to standard MATLAB m-script files, subsequent simulations can be run on the command line in a newer version of MATLAB if compute speed is critical. Also if a larger study involving many simulation runs is to be carried out it is recommended to use a xargs batch script to simulations them in parallel.