" like optimizing memory layout and data types/representations based on partial evaluation / profile feedback."
They already can. The issue is usually one of what is allowable within language semantics, not of compiler optimization technology.
One reason you may see this as neglected is that outside of, say, polyhedral loop optimizations, research in the 80's and 90's (and sometimes earlier) did a really really good job of exploring this area because fortran allowed so much freedom.
I know C and C++ can't because the language standard forbids it. Anyone knows the reason why? It should be hard but doable. That's, of course, assuming everyone properly uses macros like offsetof instead of (shudder) hand-calculated offsets, does not use type punning to access the first members of structs, etc. The compiler will certainly need to store the memory layout to enable separate compilation of translation units so that's a complication. Any other issues I haven't thought of?
Actually, the C (and I believe C++) language standards both allow reorganizing struct layouts, except for the first member which has to be in first place (for tagged unions).
The struct layout is fixed by platform-specific ABI documents, which must specify the layout precisely so that structs can be passed between separately compiled code, e.g. between applications and dynamically linked libraries.
Changing ABIs is virtually impossible, so...
It would be nice to see opt-in optimizations of struct layouts, e.g. by annotating structs with a packing-style attribute. Though the best gains would probably be obtained from profile-guided optimizations for cache line optimizations, and those can only really be done with link-time optimizations for structs that are not passed outside the linker's scope.
C99 TC3 §6.7.2.1p13 "structure and union specifiers":
"Within a structure object, the non-bit-field members and the units in which bit-fields reside have addresses that increase in the order in which they are declared."
The as-if rule allows compilers to perform this optimization, provided they elide the optimization for those structs that have addresses of members taken/used, though. This requires whole program analysis of course (as in clang / gcc "LTO" compilation mode). And of course there could be language extensions to loosen this rule further on a per-struct basis.
The compiler is free to change the layout (even from AOS to SOA) as long as it can prove that the program can't tell the difference. Of course this is hard to prove in practice even with whole program optimization.
Besides the role of grouping a bunch of related fields, structures are also regularly used to define file formats, network protocols, or in OS development for data structures interpreted by the hardware. All these things would break without a defined memory layout.
Compilers and languages of course co-evolve. And we already have fairly widespread production use of invasive language extensions/dialects designed to enable use of parallelism, such as OpenMP and CUDA. And older language extensions that have ove time become everyday stuff, like SIMD intrinsics.
So I don't think "languages don't support it" is a good excuse.
"And we already have fairly widespread production use of invasive language extensions/dialects designed to enable use of parallelism, such as OpenMP and CUDA"
I'm not sure you are really saying anything that proves me wrong?
Using those extensions, there are optimizing compilers that will perform very heavy memory layout optimization.
Which does, in fact, go to show that it took "invasive language extensions" to make the original language able to be optimized in this fashion?
IE you invasively changed the original semantics, and now it made these things possible. How does that not show that the original language semantics were not the problem, which is what i said?
Maybe OpenMP and CUDA do more optimizations than I thought they did. Do they perform AoS->SoA reorganizations, conversions from pointer members to inline members, or range analysis to choose smaller data types?
Admittedly foremost I was thinking about entirely different kinds of data and layout optimizations: Herarchical data, pointer heavy structures, and potential compressed representations. Things like converting node pointers into 16-bit indices into a node array.
> Maybe OpenMP and CUDA do more optimizations than I thought they did. Do they perform [...] range analysis to choose smaller data types?
CUDA compiler developer here. We definitely perform range analysis on values stored in registers. This is an important optimization.
Via scalar replacement of aggregates, we can sometimes also replace a struct with a set of scalars. Once we do that, we can again perform range analysis on those scalars.
We can't change structs that don't get SROA'ed because of limitations of the language. For one thing, essentially any memory we write to the GPU's global memory can be read from the CPU side, and it's basically impossible to tell what is and isn't read, so we have to keep the memory layout the same.
"We can't change structs that don't get SROA'ed because of limitations of the language. For one thing, essentially any memory we write to the GPU's global memory can be read from the CPU side, and it's basically impossible to tell what is and isn't read, so we have to keep the memory layout the same.
"
Which is precisely a language semantic problem, since there are plenty of languages where you can change SOA to AOS.
In fact, it's wildly common among high perf fortran compilers
Sun's optimizing compiler did AoS->SoA reorganization, data member inlining/ordering, and range analysis in a whole program optimization mode, even for C++(!) without any extra #pragma, when it can correctly prove no aliasing and no pointer escaping. That was nearly 15 years ago. It could also transform an array of pointers to malloc'ed arrays into a single 2d array.
They already can. The issue is usually one of what is allowable within language semantics, not of compiler optimization technology.
One reason you may see this as neglected is that outside of, say, polyhedral loop optimizations, research in the 80's and 90's (and sometimes earlier) did a really really good job of exploring this area because fortran allowed so much freedom.