| 20 issues of porting C++ code on the 64-bit platform | |
|
20 issues of porting C++ code on the 64-bit platform
IntroductionWe offer you to read the article devoted to the port of the program code of 32-bit applications on 64-bit systems. The article is written for programmers who use C++ but it may be also useful for all who face the port of applications on other platforms. One should understand properly that the new class of errors which appears while writing 64-bit programs is not just some new incorrect constructions among thousands of others. These are inevitable difficulties which the developers of any developing program will face. This article will help you to be ready for these difficulties and will show the ways to overcome them. Besides advantages, any new technology (in programming and other spheres as well) carries some limitations and even problems of using this technology. The same situation can be found in the sphere of developing 64-bit software. We all know that 64-bit software is the next step of the information technologies development. But in reality, only few programmers have faced the nuances of this sphere and developing 64-bit programs in particular. We won’t dwell on the advantages which the use of 64-bit architecture opens before programmers. There are a lot of publications devoted to this theme and the reader can find them easily. The aim of this article is to observe thoroughly those problems which the developer of 64-bit programs can face. In the article you will learn about:
To make the following text easier for understanding let’s remember some types first which we can face. (see table N1).
We’ll use term "memsize" type in the text. By this we’ll understand any simple integer type which is capable to keep a pointer and changes its size according to the change of the platform dimension form 32-bit to 64-bit. The examples memsize types: size_t, ptrdiff_t, all pointers, intptr_t, INT_PTR, DWORD_PTR. We should say some words about the data models which determine the correspondence of the sizes of fundamental types for different systems. Table N2 contains data models which can interest us.
On default in this article we’ll consider that the program port will be fulfilled from the system with the data model ILP32 on systems with data model LP64 or LLP64. And finally, 64-bit model in Linux (LP64) differs from that in Windows (LLP64) only in the size of long type. So long as it is their only difference, to summarize the account we’ll avoid using long, unsigned long types, and will use ptrdiff_t, size_t types. Let’s start observing the type errors which occur while porting programs on the 64-bit architecture
In all books devoted to the development of the quality code it is recommended to set a warning level of warnings shown by the compiler on as high level as possible. But there are situations in practice when for some project parts there is a lower diagnosis level set or it is even set off. Usually it is very old code which is supported but not modified. Programmers who work over the project are used to that this code works and don’t take its quality into consideration. Here it is a danger to miss serious warnings by the compiler while porting programs on the new 64-bit system. While porting an application you should obligatory set on warnings for the whole project which help to check the compatibility of the code and analyze them thoroughly. It can help to save a lot of time while debugging the project on the new architecture. If we won’t do this the simplest and stupidest errors will occur in all their variety. Here it is a simple example of overflow which occurs in a 64-bit program if we ignore warnings at all. unsigned char *array[50]; unsigned char size = sizeof(array); 32-bit system: sizeof(array) = 200 64-bit system: sizeof(array) = 400 2. Use of the functions with a variable number of argumentsThe typical example is the incorrect use of printf, scanf functions and their variants: 1) const char *invalidFormat = "%u"; size_t value = SIZE_MAX; printf(invalidFormat, value); 2) char buf[9]; sprintf(buf, "%p", pointer);In the first case it is not taken into account that size_t type is not equivalent to unsigned type on the 64-bit platform. It will cause the printing of an incorrect result in case if value > UINT_MAX. In the second case the author of the code didn’t take into account that the pointer size may become more than 32-bit in future. As a result this code will cause the buffer overflow on the 64-bit architecture. The incorrect use of functions with a variable number of arguments is a typical error on all the architectures, not only on 64-bit ones. This is related to the fundamental danger of the use of the given C++ language constructions. The common practice is to refuse them and use safe programming methods. We recommend you strongly to modify the code and use safe methods. For example, you may replace printf with cout, and sprintf with boost::format or std::stringstream. If you have to support the code which uses functions of scanf type, in the control lines format we can use special macros which open into necessary modifiers for different systems. An example:
// PR_SIZET on Win64 = "I"
// PR_SIZET on Win32 = ""
// PR_SIZET on Linux64 = "l"
// ...
size_t u;
scanf("%" PR_SIZET "u", &u);
3. Magic numbersIn the low-quality code there are often magic numbers the mere presence of which is dangerous. During the migration of the code on the 64-bit platform these magic numbers may make the code inefficient if they participate in operations of calculation of address, objects size or bit operations. Table N3 contains basic magic numbers which may influence the workability of an application on a new platform.
You should study the code thoroughly in search of magic numbers and replace them with safe numbers and expressions. To do so you can use sizeof() operator, special values from Let’s look at some errors related to the use of magic numbers. The most frequent is a record of number values of type sizes.
Let’s return to the error with INVALID_RESULT. The use of the number 0xFFFFFFFFu causes the failure of the execution of "len == (size_t)(-1)" condition in a 64-bit program. The best solution is to change the code in such a way that it doesn’t need special marker values. If you cannot refuse them for some reason or think it unreasonable to correct the code fundamentally just use fair value -1.
Double type as a rule has size 64 bits and is compatible with IEEE-754 standard on 32-bit and 64-bit systems. Some programmers use double type for storing of and work with integer types.
It is possible that an approximate value can be used in your program, but to be on the safe side we’d like to warn you about possible effects on the new architecture. And in any case it is not recommended to mix integer arithmetic with floating-point arithmetic.
Bit shifting operations can cause a lot of troubles during the port from the 32-bit system on the 64-bit one if used inattentively. Let’s begin with an example of a function which defines the bit you’ve chosen as 1 in a variable of memsize type.
Pay attention that "1" has int type and during the shift on 32 positions an overflow will occur as it is shown on picture 2.
To correct the code it is necessary to make the constant "1" of the same type as the variable mask.
A large number of errors during the migration on 64-bit systems are related to the change of a pointer size in relation to the size of usual integers. In the environment with the data ILP32 usual integers and pointers have the same size. Unfortunately the 32-bit code is based on this supposition everywhere. Pointers are often casted to int, unsigned int and other types improper to fulfill address calculations.
You should understand exactly that one should use only memsize types for integer pointers form. Preference should be given to uintptr_t type for it shows intentions more clearly and makes the code more portable saving it from changes in future
Let’s look at two small examples.
The following code won’t hide and will show up at the first execution.
In the end I’d like to mention that it will be a bad style to store the pointer address into types which are always equal 64-bits. One will have to correct the code shown further again when 128-bit systems will appear.
The peculiarity of a union is that for all members of the union the same memory area is allocated that is they overlap. Although the access to this memory area is possible with the use of any of the elements the element for this aim should be chosen so that the result won’t be senseless.
One should pay attention to the unions which contain pointers and other members of memsize type.
When there is a necessity to work with a pointer as an integer sometimes it is convenient to use the union as it is shown in the example, and work with the number form of the type without using explicit conversions.
Sometimes it is necessary (or just convenient) in programs to present array items in the form of the elements of a different type. Dangerous and safe type conversions are shown in the following code.
On the 64-bit system we got "2 17179869187" in the output for it is value 17179869187 which is situated in the first item of sizetPtr array (see picture 3). In some cases we need this very behavior but usually it is an error.
The correction of the described situation consists in the refuse of dangerous type conversions by modernizing the program. Another variant is to create a new array and copy values of the original one into it.
If there are big derived class graphs with virtual functions in your program, there is a risk to use inattentively arguments of different types but these types actually coincide on the 32-bit system. For example, in the base class you use size_t type as an argument of a virtual function and in the derived class type unsigned. So this code will be incorrect on the 64-bit system.
But an error like this doesn’t necessarily hide in big derived class graphs and here it is one of the examples.
The correction consists in the use of the same types in the corresponding virtual functions.
An important point during the port of a software solution on the new platform is succession to the existing data exchange protocol. It is necessary to provide the read of the existing projects formats, to carry out the data exchange between 32-bit and 64-bit processes etc.
Mostly the errors of this kind consist in the serialization of memsize types and data exchange operations using them.
The use of types of volatile size
It is unacceptably to use types which change their size depending on the development environment in binary interfaces of data exchange. In C++ language all the types don’t have distinct sizes and consequently it is not possible to use them all for these aims. That’s why the developers of the development means and programmers themselves develop data types which have an exact size such as __int8, __int16, INT32, word64 etc.
The use of such types provides data portability between programs on different platforms although it needs the use of odd ones. The three shown examples are written inaccurately and this will show up on the changing of the capacity of some data types from 32-bit to 64-bit. Taking into account the necessity to support old data formats the correction may look as follows.
Ignoring of the byte order
Even after the correction of volatile type sizes you may face the incompatibility of binary formats. The reason is a different data presentation. Most frequently it is related to a different byte order.
The byte order is a method of recording of bytes of multibyte numbers (see also picture 4). The little-endian order means that the recording begins with the lowest byte and ends with the highest one. This record order was acceptable in the memory of PCs with x86-processors. The big-endian order – the recording begins with the highest byte and ends with the lowest one. This order is a standard for TCP/IP protocols. That’s why the big-endian byte order is often called the network byte order. This byte order is used by processors Motorola 68000, SPARC.
While developing the binary interface or data format you should remember about the byte order. If the 64-bit system on which you are porting a 32-bit application has a different byte order you’ll just have to take it into account in your code. For conversion between the big-endian byte order and the little-endian one you may use functions htonl(), htons(), bswap_64 etc.
If you use bit fields you should keep in mind that the use of memsize types will cause the change of sizes of structures and alignment. For example, the structure shown further will have size 4 bytes on the 32-bit system and 8 bytes on the 64-bit one.
So be attentive while working with bit fields. To avoid the described effect in our example we can simply use the explicit conversion from obj.a type to size_t type.
The first example.
One should take care to avoid possible overflows in pointer arithmetic. For this purpose it’s better to use memsize types or the explicit type conversion in expressions which carry pointers. Using the explicit type conversion we can rewrite the code in the following way.
In a 64-bit program 0xFFFFFFFFu value will be added fairly to the pointer and the result will be that the pointer will be outbound of the array. And while getting access to the item of this pointer we’ll face troubles.
To avoid the shown situation, as well as in the first case, we advise you to use only memsize types in pointer arithmetic. Here are two variants of the code correction:
This kind of errors is separated from the others for better structuring of the account because indexing in arrays with the use of square brackets is just a different record of address arithmetic observed before.
In programming in language C and then C++ a practice formed to use in the constructions of the following kind variables of int/unsigned types:
The given code won’t process in a 64-bit program an array containing more than UINT_MAX items. After the access to the item with UNIT_MAX index an overflow of the variable Index will occur and we’ll get infinite loop.
To persuade you entirely in the necessity of using only memsize types for indexing and in the expressions of address arithmetic, I’ll give the last example.
Programmers often make a mistake trying to correct the code in the following way:
If you want to correct the code without changing types of the variables participating in the expression you may use the explicit type conversion of every variable memsize type:
Mixed use of memsize and non-memsize types in expressions may cause incorrect results on 64-bit systems and be related to the change of the input values rate. Let’s study some examples.
Another frequent error is a record of the expressions of the following kind:
Let’s give an example of a small code which shows the danger of inaccurate expressions with mixed types (the results are got with the use Microsoft Visual C++ 2005, 64-bit compilation mode).
The order of the calculation of an expression with operators of the same priority is not defined. To be more exact, the compiler can calculate sub-expressions in such an order which it considers to be more efficient even if sub-expressions cause (side effect). The order of the appearing of side effects is not defined. Expressions including communicative and association operations (*, +, &, |, ^), may be converted in a free way even if there are brackets. To assign the strict order of the calculation of the expression it is necessary to use the explicit temporary variable.
That’s why if the result of the expression should be of memsize type, only memsize types must participate in the expression. The right variant:
Mixed use of types may occur in the change of the program logic.
If you need to return the previous behavior you should change the variable val_2 type.
Observing the previous kind of errors related to mixing of simple integer types and memsize types, we surveyed only simple expressions. But similar problems may occur while using other C++ constructions too.
extern int Width, Height, Depth;
size_t GetIndex(int x, int y, int z) {
return x + y * Width + z * Width * Height;
}
...
MyArray[GetIndex(x, y, z)] = 0.0f;
In case if you work with large arrays (more than INT_MAX items) the given code may behave incorrectly and we’ll address not those items of the array MyArray we wanted. In spite the fact that we return the value of size_t type "x + y * Width + z * Width * Height" expression is calculated with the use of int type. We suppose you have already guessed that the corrected code will look as follows:
And here it is one more example but now we’ll observe not the returned value but the formal function argument.
During the port of 32-bit programs on the 64-bit platform the change of the logic of its work may be found which is related to the use of overload functions. If the function is overlapped for 32-bit and 64-bit values the access to it with the argument of memsize type will be compiled into different calls on different systems. This method may be useful as, for example, in the following code:
We think you understand this kind of errors and that you should pay attention to the call of overload functions transferring actual arguments of memsize type.
Processors work more efficiently when they deal with data aligned properly. As a rule the 32-bit data item must be aligned at the border multiple 4 bytes and the 64-bit item at the border 8 bytes. The try to work with unaligned data on processors IA-64 (Itanium) as it is shown in the following example, will cause exception.
#pragma pack (1) // Also set by key /Zp in MSVC
struct AlignSample {
unsigned size;
void *pointer;
} object;
void foo(void *p) {
object.pointer = p; // Alignment fault
}
If you have to work with unaligned data on Itanium you should indicate this to the compiler. For example, you may use a special macro UNALIGNED:
On the architecture x64 during the access to unaligned data exception does not occur but you should avoid them either. Firstly, because of the essential slowing down of the speed of the access to these data, and secondly, because of a high probability of porting the program on the platform IA-64 in future.
Let’s look at one more example of the code which does not take into account the data alignment.
The correct calculation of the size should look as follows:
Always use these macros to get a shift in the structure without relying on your knowledge of the sizes of types and the alignment. Here it is the example of the code with the correct calculation of the structure member address:
Catch and handle exceptions when integer types participate is not a good programming practice while working in C++ language. For such aims you should use more informative types, for example classes derived from class std::exception. But sometimes one has to work with less quality code as it is shown further.
While developing a 64-bit application, be sure to bear in mind the changes of the environment in which it will be performed. Some functions will become outdated and it will be necessary to replace them with some variants. GetWindowLong is a good example of such function in the Windows operation system. Pay your attention to the constants referring to the interaction with the environment in which the program is functioning. In Windows the lines containing "system32" or "Program Files" will be suspect.
Be accurate with explicit type conversions. They may change the logic of the program execution when types change their capacity of cause the loss of significant bits. It is difficult to adduce typical examples of errors related to the explicit type conversion for they are very different and specific for different programs. You have become acquainted with some errors related to the explicit type conversion earlier.
Error diagnosis
The diagnosis of the errors occurring while porting 32-bit programs on 64-bit systems is a difficult task. The port of a not very quality code written without taking into account peculiarities of other architectures, may demand a lot of time and efforts. That’s why we’ll pay some attention to the description of methods and means which may simplify this task.
Unit test
Unit test have fought well-earned respect among programmers long ago. Unit tests will help to check the correctness of the program after the port on a new platform. But there is one nuance which you should keep in mind.
Unit test may not allow you to check the new ranges of input values which become accessible on 64-bit systems. Unit tests are originally developed in such a way that they can be passed in a short time. And the function which usually works with an array with the size of tens of Mb, will probably process tens of Kb in unit tests. It is justified for this function in tests may be called many times with different sets of input values. But suppose you have a 64-bit variant of the program. And now the function we study is processing more than 4 Gb of data. Surely there appears a necessity to raise the input size of an array in the tests either up to sizes more than 4 Gb. The problem is that the time of passing the tests will increase greatly in such a case.
That’s why while modifying the sets of tests keep in mind the compromise between speed of passing unit tests and the fullness of the checks. Fortunately, there are other methods which can help you to make sure of the efficiency of your applications.
Code review
Code review is the best method of searching errors and improving code. Combined thorough code review may help to get rid of the errors in the program completely which are related to the peculiarities of the development of 64-bit applications. Of course, in the beginning one should learn which errors exactly one should search, otherwise the review won’t give good results. For this purpose it is necessary to read this and other articles devoted to the port of programs from 32-bit systems on 64-bit ones, in time. Some interesting links concerning this topic can be found in the end of the article.
But this approach to the analysis of the original code has on significant disadvantage. It demands a lot of time and because of this it is actually inapplicable on large projects.
The compromise is the use of static analyzers. A static analyzer can be considered to be an automated system for code review where a fetch of potentially dangerous places is created for a programmer so that he could carry out the further analysis.
But in any case it is desirable to provide several code reviews for combined teaching the team to search for new kinds of errors occurring on 64-bit systems.
Built-in means of compilers
Compilers allow to solve some problems of searching defect code. They often have built-in different mechanisms for diagnosing errors observed. For example, in Microsoft Visual C++ 2005 the following keys may be useful: /Wp64, /Wall, and in SunStudio C++ key –xport64.
Unfortunately, the possibilities they provide are often not enough and you should not rely only on them. But in any case it is highly recommended to enable the corresponding options of a compiler for diagnosing errors in the 64-bit code.
Static analyzers
Static analyzers are a fine means to improve quality and safety of the program code. The basic difficulty related to the use of static analyzers consists in the fact that they generate quite a lot false warning messages about potential errors. Programmers being lazy by nature use this argument to find some way not to correct the found errors. In Microsoft this problem is solved by including necessarily the found errors in the bug tracking system. Thus a programmer just cannot choose between the correction of the code and tries to avoid this.
We think that such strict rules are justified. The profit of the quality code covers the outlay of time for static analysis and corresponding code modification. This profit is achieved by means of simplifying the code support and reducing the time of debugging and testing.
Static analyzers may be successfully used for diagnosing many of the kinds of errors observed in the article.
The authors know 3 static analyzers which are supposed to have means of diagnosing errors related to the port of programs on 64-bit systems. We would like to warn you at once that we may be mistaken about the possibilities they have, moreover, these are developing products and new versions may have great efficiency.
If you read these lines we are glad that you’re interested. We hope the article has been useful for you and will help to simplify the development and debugging of 64-bit applications. We will be glad to receive your opinions, remarks, corrections, additions and will surely include them in the next version of the article. The more we’ll describe typical errors the more profitable will be the use of our experience and getting help.
For more articles of the same, please visit our website.
Resources
Discuss this article in the forums
See Also: © 1999-2009 Gamedev.net. All rights reserved. Terms of Use Privacy Policy
|