MSVC std::vector and aligned data

Started by
21 comments, last by mrbastard 12 years, 5 months ago
I switched to the DirectXMath library when I switched to D3D11, and I'm running into this issue of Microsoft's std::vector implementation being incompatible with aligned data types. Looking around the 'Net it looks like the standard fixes are to avoid using std::vector, use a custom allocator, patch [s]vector::rebind[/s] vector::resize to take a const reference, or a combination of the latter two measures. But all the relevant pages I found were from 2008 or older.

So, what does the GD.net community suggest? And does anyone have a custom aligned allocator, or do I need to learn to write my own?
Advertisement
Well, one solution would be not to use MSVC at all. If you are doing cross platform projects, you might as well be using MinGW on Windows, since you are likely using GCC already on other operating systems.

If you are not targeting Non-Microsoft OSes at all, you can still use MinGW.

If you must use MSVC, then you can replace the horrible Microsoft STL implementation with STLport.

I switched to the DirectXMath library when I switched to D3D11, and I'm running into this issue of Microsoft's std::vector implementation being incompatible with aligned data types. Looking around the 'Net it looks like the standard fixes are to avoid using std::vector, use a custom allocator, patch vector::rebind to take a const reference, or a combination of the latter two measures. But all the relevant pages I found were from 2008 or older.

So, what does the GD.net community suggest? And does anyone have a custom aligned allocator, or do I need to learn to write my own?

If you have a specific additional memory allocation requirement (like a specific alignment) and you want to use a standard library container, you have to replace the default allocator with one that satisfies the additional requirement. This is not specific to any one C++ runtime. It is why allocators are a customization point in the standard library.

It is not difficult to write something that satisfies the allocator requirements, but you might want to have a good reference handy (like Josuttis) just in case.

Stephen M. Webb
Professional Free Software Developer


[If you have a specific additional memory allocation requirement (like a specific alignment) and you want to use a standard library container, you have to replace the default allocator with one that satisfies the additional requirement. This is not specific to any one C++ runtime. It is why allocators are a customization point in the standard library.

It is not difficult to write something that satisfies the allocator requirements, but you might want to have a good reference handy (like Josuttis) just in case.


No, the problem OP is having isn't related to memory allocators, it's related to the fact that Microsoft's STL uses the wrong parameter types in functions like vector::resize, effectively making their STL completely incompatible with aligned types, regardless of what allocator is used.

If you must use MSVC, then you can replace the horrible Microsoft STL implementation with STLport.

MSVC has a fairly decent standard library implementation (the days of the lawsuits that prevented Microsoft from bundling a conformant library with MSVC6 after the previous C++ standard was released have been over for quite some time).

STLport does not come close to conforming to the current C++ standard.

Stephen M. Webb
Professional Free Software Developer

One solution is to use a std::vector replacement. For example, the bullet physics source code contains a class called [font="Courier New"]btAlignedObjectArray[/font], which is designed to solve your problem.
No, the problem OP is having isn't related to memory allocators, it's related to the fact that Microsoft's STL uses the wrong parameter types in functions like vector::resize, effectively making their STL completely incompatible with aligned types, regardless of what allocator is used.
No, the problem OP is having is related to standard allocators not satisfying non-default allocation requirements -- this is the same problem no matter what compiler/STL implementation you're using.
For example, on GCC, if you use the non-standard [font="Courier New"]__attribute__[/font] syntax to specify a non-default alignment value, then [font="Courier New"]std::vector[/font] won't work out of the box any more, due to the fact that this kind of structure alignment is non-standard.

Secondly, your information about Microsoft's 'STL' being non-standards compliant is incorrect, as of about 5 years ago.
No, the problem OP is having is related to standard allocators not satisfying non-default allocation requirements -- this is the same problem no matter what compiler/STL implementation you're using.


MSVC (9 and 10, at least) both pass resize's _Val parameter by value, not const reference. Consequently, even if you write a custom allocator, you get a compiler error warning that the parameter cannot be aligned. So, whilst the allocator is needed, it seems disingenuous to say that the library implementation is entirely faultless.
[TheUnbeliever]

MSVC has a fairly decent standard library implementation (the days of the lawsuits that prevented Microsoft from bundling a conformant library with MSVC6 after the previous C++ standard was released have been over for quite some time).

STLport does not come close to conforming to the current C++ standard.


I was recommended STLport by multiple users here at GD.net just a year ago when I hade the exact same issue when using MSVC's srd::vector with 16 byte aligned float structures. I had an appropriate allocator made up, but because MSVC's STL passes by value in vector::resize, it is impossible to get anything buy compiler errors out of it.

This problem doesn't exist with STLport or GNU STL, so I stand by my statements.
MSVC (9 and 10, at least) both pass resize's _Val parameter by value, not const reference.
Does the standard specify the signature as: [font="Courier New"]void resize(size_type sz, T c = T()) [/font]?
On my current project, the 3 different compilers that we use all comply with the above signature.

[quote name='Bregma' timestamp='1317695717' post='4868808']
MSVC has a fairly decent standard library implementation (the days of the lawsuits that prevented Microsoft from bundling a conformant library with MSVC6 after the previous C++ standard was released have been over for quite some time).

STLport does not come close to conforming to the current C++ standard.


I was recommended STLport by multiple users here at GD.net just a year ago when I hade the exact same issue when using MSVC's srd::vector with 16 byte aligned float structures. I had an appropriate allocator made up, but because MSVC's STL passes by value in vector::resize, it is impossible to get anything buy compiler errors out of it.

This problem doesn't exist with STLport or GNU STL, so I stand by my statements.
[/quote]

Standing by observation is a good thing but understanding the problem is probably more important in this case. Basically the problem has nothing to do with the STL of VC but in a fundamental issue of C++ and how to interpret a semi-vague standard in this area. The STL definition of an allocator has nothing to do with memory alignment, worse, by definition the amount of allocation is requested by using "sizeof". So, a fully compliant STL only allows you to specify the initial address of a vector by overloading the allocator, it will (by standard) use sizeof to figure out how much memory to allocate for each element and as such the delta between elements in that memory. The STL versions you mention support "hacks" which work with padded structures instead of the basic "sizeof", it makes sense because that is "usually" what you want, but it is actually "against" standards compliance.

C++ doesn't support "alignment" directly, that's all done via extensions using pragma's, attributes etc. All such concepts are compiler specific and while rather stupid, VC's STL implementation doesn't take into account for these items. Stupid versus non compliant are very different things in this case. VC's STL since 2k8 versions have been exceptionally compliant.

The difference here is critical. VC's STL is actually the more compliant in this case, unfortunately it is not the most usable for what you want to do with it. Which version is better? One is very compliant and the others are cheating to be more usable. Given all the complexities in C++, I prefer more compliance so there are fewer invisible things going on behind my back.

This topic is closed to new replies.

Advertisement