Build timings from auto-modularised VS solution with external and internal libs

To investigate how well modules work with MSVC without rewriting projects and putting up with malfunctioning intellisense, I made a little python script to modularise a whole VS solution (yes, *I* made it, using real bio-neurons).

The script lets me try out different modularisation strategies on a VS Solution on a per-project basis:

* Header-based, don't modularise. If dependencies are modules, they will be imported.
* PartitionsMI: Partition-per-header using standard module cpp units.
* PartitionsPI: Partition-per-header using MSVC partition cpp units.
* Submodules: Module-per-header
* SubmodulesII: Move implementation to interface (not always possible due to cyclic imports)

Modularised code is surrounded with `extern "C++" { export {`.

The code is a super secret incomplete game engine/game. But here's some info:

* External libs, including: Boost (unordered flat map), std, d3d, imgui, physx, spdlog, <windows>.
* Wrapper module for each.
* Solution projects: CommonStuff, Scripting, GameData, Graphics, Anim, SceneGraph, SceneManager, Entity, App
* Some projects don't have much code. Others have quite a bit. Enough to give the build system something to parallelise.

All projects built using debug config. Benchmarks average 3 or more timings, outliers ignored. CPU is 12600, 12 threads. Multipliers are speedup.

| |CommonStuff|Scripting|GameData|Graphics|Anim|SceneGraph|SceneManager|Entity|
|:-|:-|:-|:-|:-|:-|:-|:-|:-|
|Header-based, no pch|4.59s|5.68s|7.46s|2.92s|40.74s|24.94s|2.86s|3.19s|
|Header-based, pch|3.61s|5.30s|6.55s||19.38s|12.53s|||
|Extlib modules + CommonStuff submodules||3.60s|4.78s|1.62s|18.55s|9.41s|1.57s|2.09s|
|PartitionsMI|4.25s||8.09s|2.54s|||||
|PartitionsPI|4.02s|6.04s|7.09s|2.53s|33.83s|15.97s|2.66s|4.10s|
|Submodules|3.59s|4.75s|6.05s|2.26s|26.80s|14.31s|2.12s|3.15s|
|SubmodulesII|3.44s||7.93s||||||

| |CommonStuff|Scripting|GameData|Graphics|Anim|SceneGraph|SceneManager|Entity|
|:-|:-|:-|:-|:-|:-|:-|:-|:-|
|Header-based, no pch|1.00x|1.00x|1.00x|1.00x|1.00x|1.00x|1.00x|1.00x|
|Header-based, pch|1.27x|1.07x|1.14x||2.10x|1.99x|||
|Extlib modules + CommonStuff submodules||1.58x|1.56x|1.80x|2.20x|2.65x|1.82x|1.53x|
|PartitionsMI|1.08x||0.92x|1.15x|||||
|PartitionsPI|1.14x|0.94x|1.05x|1.15x|1.20x|1.56x|1.08x|0.78x|
|Submodules|1.28x|1.20x|1.23x|1.29x|1.52x|1.74x|1.35x|1.01x|
|SubmodulesII|1.33x||0.94x||||||

| |App|Single file|Full Rebuild| |App|Single file|Full Rebuild|
|:-|:-|:-|:-|:-|:-|:-|:-|
|Header-based, no pch|65.19s|3.97s|145.49s||1.00x|1.00x|1.00x|
|Header-based, app pch|24.74s|2.15s|108.92s||2.64x|1.85x|1.34x|
|Header-based, mostly pch|||78.87s||||1.84x|
|Extlib modules|26.82s|2.35s|58.89s||2.43x|1.69x|2.47x|
|Extlib modules + CommonStuff partitions|25.89s|2.47s|||2.52x|1.61x||
|Extlib modules + CommonStuff submodules|24.53s|2.23s|57.50s||2.66x|1.78x|2.53x|
|Extlib modules + most libs as submodules|31.14s|2.97s|||2.09x|1.34x||
|Full submodules, except app|34.61s|2.95s|88.90s||1.88x|1.35x|1.64x|
|Full submodules|64.02s|6.22s|142.98s||1.02x|0.64x|1.02x|

* "Most libs" = CommonStuff + Scripting + GameData + Graphics + Anim + SceneGraph.
* "Mostly pch" = PCH for CommonStuff, Scripting, GameData, Anim, SceneGraph, App.
* "Single file" is recompiling one moderately complex file in App. \~930 lines, lots of includes.
* "Full Rebuild" excludes extlib module wrappers.

Detailed timings of CommonStuff:

Header-based:

1> 210 ms LIB 1 calls
1> 4459 ms CL 2 calls
04.874 seconds

PartitionsMI:

1> 125 ms LIB 1 calls
1> 199 ms MSBuild 16 calls
1> 269 ms CppClean 1 calls
1> 798 ms SetModuleDependencies 10 calls
1> 1232 ms CL 2 calls
1> 1439 ms MultiToolTask

1 calls
04.295 seconds

Submodules:

1> 141 ms MSBuild 16 calls
1> 775 ms SetModuleDependencies 10 calls
1> 1015 ms CL 2 calls
1> 1132 ms MultiToolTask 1 calls
03.427 seconds

The CL entry is for cpp files and it's 4.39x as fast as header-based. Great!

But it's the ixx dependency scanning and compilation (MultiToolTask) that brings things down. Things that aren't there with header-based.

MultiToolTask *is* parallelised though. Reducing thread count slows it down. And you can open up the msbuild binlog in the msbuild log viewer to see the parallelisation of ixx compilation. See which modules are on the end of the DAG slowing things down.

So, I don't know how MultiToolTask could be sped up, but, msbuild seems to be waiting for all ixx files before moving onto cpp files, so that's some parallelism left on the table.

Detailed timings of App:

Header-based, app pch:

1> 196 ms MSBuild 13 calls
1> 1000 ms Link 1 calls
1> 23270 ms CL 16 calls
24.708 seconds

Extlib modules + CommonStuff submodules:

1> 199 ms SetModuleDependencies 18 calls
1> 606 ms MSBuild 27 calls
1> 1197 ms Link 1 calls
1> 21877 ms CL 15 calls
24.078 seconds

Full submodules:

1> 459 ms MSBuild 27 calls
1> 1324 ms Link 1 calls
1> 4646 ms SetModuleDependencies 18 calls
1> 12593 ms MultiToolTask 1 calls
1> 49327 ms CL 15 calls
01:09.245 minutes

Slow. But looking at the log viewer, ixx compilation seems very well parallelised.

And 4.6s just to scan for dependencies. VS isn't launching a whole CL process for each file is it?

Also, why does CL take so long? All modules have been compiled by that point, so why would it be much slower?

Recompiling a single file from App:

Header-based, no pch:

1> 46 ms SetModuleDependencies 10 calls
1> 120 ms MSBuild 5 calls
1> 3757 ms CL 1 calls
04.095 seconds

Header-based, app pch:

1> 56 ms SetModuleDependencies 10 calls
1> 121 ms MSBuild 5 calls
1> 1793 ms CL 2 calls
02.059 seconds

Extlib + CommonStuff modules (as submodules):

1> 121 ms SetModuleDependencies 18 calls
1> 216 ms MSBuild 17 calls
1> 1793 ms CL 1 calls
02.162 seconds

Full submodules, except app:

1> 518 ms MSBuild 17 calls
1> 631 ms SetModuleDependencies 18 calls
1> 2211 ms CL 1 calls
03.095 seconds

Full submodules:

1> 559 ms MSBuild 17 calls
1> 696 ms SetModuleDependencies 18 calls
1> 1891 ms MultiToolTask 1 calls
1> 3221 ms CL 1 calls
06.331 seconds

That MultiToolTask is quite slow, checking that all the modules in the project are up to date it seems. Does it have to be this slow?

"CL.exe will run on 0 out of 224 file(s) in 224 batches. Startup

phase took 2040.9372ms."

Anyway.

Conclusion:

* I now have the data to know how I should go about modularising a codebase for build perf.
* Surprisingly, there is a middle-ground between 0% modules and 100% modules where you get optimal build performance. It seems extlibs and your common lib should be modularised and nothing else. Or maybe my internal lib headers are too small to benefit.
* Because of that, I might be able to have a header-based fallback for intellisense without it spreading to the rest of the code.
* Multiple modules > Partitions. Partitions have worse performance. I assume that's because they need to have an extra module at the end of the DAG.
* Their only advantage is shared module attachment across multiple ixx files, if you don't use `extern "C++"`. As I said before, I wish you could have "public" partitions or multiple modules with the same name attachment.
* MSVC partition cpp files are slightly faster vs module cpp files. But why. Cpp files wait for all ixx files to compile anyway. Noise in the data?
* MSBuild parallelises as it should, it seems, except that cpp files wait for all ixx files. Could improve performance by merging the tasks?
* cpp compilation can be much faster, but overall build perf brought down by ixx files and scanning.
* Moving implementation to ixx files doesn't always help. Need more data.
* And it would be ideal if build systems avoided the build cascade when only changing implementation details.
* The more modules you add to the solution, the more bloated the build system feels. Lots of scanning, lots of ixx checking. Can performance be improved?
* PCH can be beaten in some cases.

Problems:

* Intellisense. Red squiggles everywhere. Not only `import std` issues, but Intellisense is highly sensitive to errors in imported modules. It cannot limp along like it can with headers.
* A simple example would be missing a semicolon after a struct def in a header and in a module. Intellisense can still use the struct from the header, but not the module.
* Or in working code, something from `import std` that Intellisense can't handle.
* This means that parsing issues with Intellisense may be viral when using modules. Intellisense must either be perfect or tolerant.
* From a previous modularisation attempt, I removed the combination of explicit template instantiations of a class with constrained friend functions, due to ICE, which has now become a bogus compiler error. Bug not fixed.
* Lots of linker warnings from dllexport-ed manual RTTI data. Bug not fixed.

At least the compiler functions well enough to produce a working program with no further changes. That's pretty good.

https://redd.it/1va18lf
@r_cpp

Top 7 desktop app frameworks in 2026
https://teamdev.com/mobrowser/blog/top-7-desktop-app-frameworks-in-2026/

https://redd.it/1vaprni
@r_cpp