MSVC std::vector and aligned data

Started by
21 comments, last by mrbastard 12 years, 5 months ago

[quote name='TheUnbeliever' timestamp='1317696821' post='4868817']MSVC (9 and 10, at least) both pass resize's _Val parameter by value, not const reference.
Doesn't the standard specify the signature as: [font="Courier New"]void resize(size_type sz, T c = T()) [/font]?
On my current project, the 3 different compilers that we use all comply with the above signature.
[/quote]

I don't have access to a copy of the standard, so I can't honestly say. It's entirely plausible. In that case, maybe the word 'fault' isn't right to use, certainly not of the library. My point was really just that the combination of library and compiler mean that writing a custom allocator is insufficient to solve the problem - you must also patch vector to take a const reference. This was definitely the case with MSVC9. The library in 10 has stuck with the same signature, but perhaps the compiler allows alignment of function parameters. I can't remember if I've tried.

So, yeah - perhaps my wording is a little strong, but there's at least one additional problem which we don't have enough information (define 'incompatible', and which development environment are we using) to rule out. :o)
[TheUnbeliever]
Advertisement

Standing by observation is a good thing but understanding the problem is probably more important in this case. Basically the problem has nothing to do with the STL of VC but in a fundamental issue of C++ and how to interpret a semi-vague standard in this area. The STL definition of an allocator has nothing to do with memory alignment, worse, by definition the amount of allocation is requested by using "sizeof". So, a fully compliant STL only allows you to specify the initial address of a vector by overloading the allocator, it will (by standard) use sizeof to figure out how much memory to allocate for each element and as such the delta between elements in that memory. The STL versions you mention support "hacks" which work with padded structures instead of the basic "sizeof", it makes sense because that is "usually" what you want, but it is actually "against" standards compliance.

C++ doesn't support "alignment" directly, that's all done via extensions using pragma's, attributes etc. All such concepts are compiler specific and while rather stupid, VC's STL implementation doesn't take into account for these items. Stupid versus non compliant are very different things in this case. VC's STL since 2k8 versions have been exceptionally compliant.

The difference here is critical. VC's STL is actually the more compliant in this case, unfortunately it is not the most usable for what you want to do with it. Which version is better? One is very compliant and the others are cheating to be more usable. Given all the complexities in C++, I prefer more compliance so there are fewer invisible things going on behind my back.


Thanks for the clarification. Standards aside, OP as well as many of us find ourselves needing dynamic arrays of aligned data, so this is an issue which needs a solution.
Another (mostly) drop-in replacement for [font="'Courier New"]std::vector[/font] is [font="'Courier New"]rde::vector[/font].

So, yeah - perhaps my wording is a little strong, but there's at least one additional problem which we don't have enough information (define 'incompatible', and which development environment are we using) to rule out. :o)

"Incompatible" means I'm getting this error:
error C2719: '_Val': formal parameter with __declspec(align('16')) won't be aligned
I apologize for not posting it earlier.

And I'm using MSVC 2010 Express on a Win7 64-bit machine (but compiling for 32-bit debug).

I guess I'm going to look into alternative containers and custom allocators and see which I prefer.
"Incompatible" means I'm getting this error:
error C2719: '_Val': formal parameter with __declspec(align('16')) won't be aligned
I apologize for not posting it earlier.


Writing a custom allocator won't fix this: you either need to, standard be damned, patch the line in the header (the error presumably points to it) or use a replacement container. Either way, you will need a custom allocator as well.
[TheUnbeliever]

I don't have access to a copy of the standard, so I can't honestly say. It's entirely plausible.

For the record, ISO-14882 says this.
[23.3.6.2](11) [font="Courier New"]void resize(size_type sz, const T& c);[/font]

If the Microsoft implementation does not meet this requirement, it is nonconforming (ie. broken). Very unfortunate. The OP's problem will still obtain with other implementations because he needs to provide a custom allocator.

Also, for the record, STLport does not implement about 50% of the current standard library.

Stephen M. Webb
Professional Free Software Developer


[quote name='TheUnbeliever' timestamp='1317701806' post='4868848']
So, yeah - perhaps my wording is a little strong, but there's at least one additional problem which we don't have enough information (define 'incompatible', and which development environment are we using) to rule out. :o)

"Incompatible" means I'm getting this error:
error C2719: '_Val': formal parameter with __declspec(align('16')) won't be aligned
I apologize for not posting it earlier.

And I'm using MSVC 2010 Express on a Win7 64-bit machine (but compiling for 32-bit debug).

I guess I'm going to look into alternative containers and custom allocators and see which I prefer.
[/quote]

The above error you received is a problem with the Visual studio compiler cannot guarantee 16 byte alignment for variables that are copied on the stack. A fix to your problem is to compile your application as a 64 bit compile and you will not receive any errors. I would suggest going for 64 bit compiles as that is going to be the dominant os version.

You can read this http://thetweaker.wordpress.com/2010/05/05/stdvector-of-aligned-elements/
Wisdom is knowing when to shut up, so try it.
--Game Development http://nolimitsdesigns.com: Reliable UDP library, Threading library, Math Library, UI Library. Take a look, its all free.

If the Microsoft implementation does not meet this requirement, it is nonconforming (ie. broken). Very unfortunate.

After doing some more informed Googling, it appears that this is a relatively recent change to the standard, and that Microsoft actually complied here while many other vendors chose not to for the sake of usability. I can post a link later today that goes into more detail.

Edit: Here's the blog entry and followup I was talking about. I'm not sure when the revision occurred, but it looks like Microsoft's implementation of std::vector::resize() was compliant until the standard was revised.

A fix to your problem is to compile your application as a 64 bit compile and you will not receive any errors.

I was under the impression that one could not compile for 64-bit applications with the Express edition of MSVC.

Thanks for the clarification. Standards aside, OP as well as many of us find ourselves needing dynamic arrays of aligned data, so this is an issue which needs a solution.


I completely agree that we need a solution, sorry if I sounded like I was defending some gallant adherence to standards, that's not the case at all. I actually use a custom vector container because of these problems. If STL had been written with a policy based front end (i.e. not just the allocator but also internal behavior), it would have greatly helped correct the deficiencies and allow stl containers to adapt better to ongoing changes and requirements in programming. Unfortunately they were not and now they are starting to show deficiencies. (Personally, I thought they had a LOT of deficiencies for optimization reasons and hence wrote my own in many places as drop in replacements when it makes sense.)

Look at the basic problems with std::vector as examples:

1. Pod items are detected automatically and in cases where something "looks" non-pod but could be treated as pod, you can't do anything to change the STL behavior of copy construct/delete, which in "many" cases is stupid overhead. Deal with it or use a non-stl container.
2. No way to define stable/non-stable erasure. 90% of the time I don't care about stable erasures in a vector and as such some STL variations supply "fast_erase" or related functions which just move the last element to the erased elements location instead of shifting everything down by one. Of course, unless you target a single platform, you can't always use this and that causes big performance differences between platforms/libraries.
3. POD detection is so flakey given the lack of decent metadata in C++ that a huge portion of the time STL guesses wrong and by definition "most" of your std::vector's are doing stupid work behind your back.

This is a huge problem with C++ that needs to be corrected to make stl something even performance critical applications can/should use. Adding the concept of "alignment" requirements to C++ is also extremely important.

This topic is closed to new replies.

Advertisement