• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.


  • Content count

  • Joined

  • Last visited

Community Reputation

1471 Excellent

About Code_Analysis

  • Rank

Personal Information

  1. Our company develops, promotes, and sells the PVS-Studio static code analyzer for C/C++ programmers. However, our collaboration with customers is not limited solely to selling PVS-Studio licenses. For example, we often take on contract projects as well. Due to NDAs, we're not usually allowed to reveal details about this work, and you might not be familiar with the projects names, anyway. But this time, we think you'll be excited by our latest collaboration. Together with Epic Games, we 're working on the Unreal Engine project. This is what we're going to tell you about in this article: https://www.unrealengine.com/blog/how-pvs-studio-team-improved-unreal-engines-code    
  2. On March 19, 2014, Unreal Engine 4 was made publicly available. Subscription costs only $19 per month. The source codes have also been published at the github repository. Since that moment, we have received quite a number of e-mails, twitter messages, etc., people asking to check this game engine. So we are fulfilling our readers' request in this article; let's see what interesting bugs the PVS-Studio static code analyzer has found in the project's source code. Unreal Engine The Unreal Engine is a game engine developed by Epic Games, first illustrated in the 1998 first-person shooter game Unreal. Although primarily developed for the first-person shooters, it has been successfully used in a variety of other genres, including stealth, MMORPGs, and other RPGs. With its code written in C++, Unreal Engine features a high degree of portability and is a tool used by many game developers today. The official website: https://www.unrealengine.com/ The Wikipedia article: Unreal Engine. Analysis methodology for an nmake-based project There exist certain difficulties regarding analysis of the Unreal Engine project. To check it, we had to use a new feature recently introduced in PVS-Studio Standalone. Because of that, we had to postpone the publication of this article a bit so that it would follow the release of the new PVS-Studio version with this feature. I guess many would like to try it: it allows programmers to easily check projects that make use of complex or non-standard build systems. PVS-Studio's original working principle is as follows:You open a project in Visual Studio.Click the "Start" button.The Visual Studio-integrated plugin collects all the necessary information: which files need to be analyzed, which macros are to be expanded, where the header files location, and so on.The plugin launches the analyzer module itself and outputs the analysis results.What's special about Unreal Engine 4 is that it is an nmake-based project, therefore it can't be checked by the PVS-Studio plugin. Let me explain this point. Unreal Engine is implemented as a Visual Studio project, but the build is done with nmake. It means that the plugin cannot know which files are compiled with which switches. Therefore, analysis is impossible. To be exact, it is possible, but it will be somewhat of an effort (see the documentation section, "Direct integration of the analyzer into build automation systems"). And here's PVS-Studio Standalone coming to help! It can work in two modes: You obtain preprocessed files in any way and let the tool check them.Its monitoring compiler calls and get all the necessary information. It is the second mode that we are interested in now. This is how the check of Unreal Engine was done: We launched PVS-Studio Standalone.Clicked "Compiler Monitoring".Then we clicked "Start Monitoring" and made sure the compiler call monitoring mode was on.We opened the Unreal Engine project in Visual Studio and started the project build. The monitoring window indicated that the compiler calls were being tapped.When the build was finished, we clicked Stop Monitoring, and after that the PVS-Studio analyzer was launched. The diagnostic messages were displayed in the PVS-Studio Standalone window. Hint. It is more convenient to use Visual Studio instead of the PVS-Studio Standalone's editor to work with the analysis report. You only need to save the results into a log file and then open it in the Visual Studio environment (Menu->PVS-Studio->Open/Save->Open Analysis Report). All that and many other things are described in detail in the article "PVS-Studio Now Supports Any Build System under Windows and Any Compiler. Easy and Right Out of the Box". Do read this article please before you start experimenting with PVS-Studio Standalone! Analysis results I found the Unreal Engine project's code very high-quality. For example, developers employ static code analysis during the development, which is hinted at by the following code fragments: // Suppress static code analysis warning about a // potential comparison of two constants CA_SUPPRESS(6326); .... // Suppress static code analysis warnings about a // potentially ill-defined loop. BlendCount > 0 is valid. CA_SUPPRESS(6294) .... #if USING_CODE_ANALYSIS These code fragments prove that they use a static code analyzer integrated into Visual Studio. To find out more about this tool, see the article Visual Studio 2013 Static Code Analysis in depth: What? When and How? The project authors may also use some other analyzers, but I can't say for sure. So their code is pretty good. Since they use static code analysis tools during the development, PVS-Studio has not found many suspicious fragments. However, just like any other large project, this one does have some bugs, and PVS-Studio can catch some of them. So let's find out what it has to show us. Typos static bool PositionIsInside(....) { return Position.X >= Control.Center.X - BoxSize.X * 0.5f && Position.X = Control.Center.Y - BoxSize.Y * 0.5f && Position.Y >= Control.Center.Y - BoxSize.Y * 0.5f; } PVS-Studio's diagnostic message: V501 There are identical sub-expressions 'Position.Y >= Control.Center.Y - BoxSize.Y * 0.5f' to the left and to the right of the '&&' operator. svirtualjoystick.cpp 97 Notice that the Position.Y variable is compared to the Control.Center.Y - BoxSize.Y * 0.5f expression twice. This is obviously a typo; the '-' operator should be replaced with '+' in the last line. Here's one more similar mistake in a condition: void FOculusRiftHMD::PreRenderView_RenderThread( FSceneView& View) { .... if (View.StereoPass == eSSP_LEFT_EYE || View.StereoPass == eSSP_LEFT_EYE) .... } PVS-Studio's diagnostic message: V501 There are identical sub-expressions 'View.StereoPass == eSSP_LEFT_EYE' to the left and to the right of the '||' operator. oculusrifthmd.cpp 1453 It seems that the work with Oculus Rift is not well tested yet. Let's go on. struct FMemoryAllocationStats_DEPRECATED { .... SIZE_T NotUsed5; SIZE_T NotUsed6; SIZE_T NotUsed7; SIZE_T NotUsed8; .... }; FMemoryAllocationStats_DEPRECATED() { .... NotUsed5 = 0; NotUsed6 = 0; NotUsed6 = 0; NotUsed8 = 0; .... } PVS-Studio's diagnostic message: V519 The 'NotUsed6' variable is assigned values twice successively. Perhaps this is a mistake. Check lines: 86, 88. memorybase.h 88 Structure members are initialized here. A typo causes the NotUsed6 member to be initialized twice, while the NotUsed7 member remains uninitialized. However, the _DEPRECATED() suffix in the function name tells us this code is not of much interest anymore. Here are two other fragments where one variable is assigned a value twice:V519 The HighlightText variable is assigned values twice successively. Perhaps this is a mistake. Check lines: 204, 206. srichtextblock.cpp 206V519 The TrackError.MaxErrorInScaleDueToScale variable is assigned values twice successively. Perhaps this is a mistake. Check lines: 1715, 1716. animationutils.cpp 1716 Null pointers I pretty often come across null pointer dereferencing errors in error handlers. No wonder: these fragments are difficult and uninteresting to test. In Unreal Engine, you can find a null pointer dereferencing error in an error handler too: bool UEngine::CommitMapChange( FWorldContext &Context ) { .... LevelStreamingObject = Context.World()->StreamingLevels[j]; if (LevelStreamingObject != NULL) { .... } else { check(LevelStreamingObject); UE_LOG(LogStreaming, Log, TEXT("Unable to handle streaming object %s"), *LevelStreamingObject->GetName()); } .... } PVS-Studio's diagnostic message: V522 Dereferencing of the null pointer 'LevelStreamingObject' might take place. unrealengine.cpp 10768 We want to print the object name when an error occurs. But the object doesn't exist. Here's another fragment with null pointer dereferencing. It's all much more interesting here. Perhaps the error appeared because of an incorrect merge. Anyway, the comment proves that the code is incomplete: void FStreamingPause::Init() { .... if( GStreamingPauseBackground == NULL && GUseStreamingPause ) { // @todo UE4 merge andrew // GStreamingPauseBackground = new FFrontBufferTexture(....); GStreamingPauseBackground->InitRHI(); } } PVS-Studio's diagnostic message: V522 Dereferencing of the null pointer 'GStreamingPauseBackground' might take place. streamingpauserendering.cpp 197 A few more words about null pointers Almost in every program I check, I get a pile of V595 warnings (examples). These warnings indicate the following trouble: A pointer is dereferenced first and only then is checked for being null. That's not always an error, but this code is highly suspicious and needs to be checked anyway! The V595 diagnostic helps us reveal slip-ups like this: /** * Global engine pointer. * Can be 0 so don't use without checking. */ ENGINE_API UEngine* GEngine = NULL; bool UEngine::LoadMap( FWorldContext& WorldContext, FURL URL, class UPendingNetGame* Pending, FString& Error ) { .... if (GEngine->GameViewport != NULL) { ClearDebugDisplayProperties(); } if( GEngine ) { GEngine->WorldDestroyed( WorldContext.World() ); } .... } PVS-Studio's diagnostic message: V595 The 'GEngine' pointer was utilized before it was verified against nullptr. Check lines: 9714, 9719. unrealengine.cpp 9714 Notice the comment. The global variable GEngine may be equal to zero, so it must be checked before it can be used. And there is such a check indeed in the function LoadMap(): if( GEngine ) Unfortunately, this check is executed only after the pointer has been already used: if (GEngine->GameViewport != NULL) There were quite a number of V595 warnings for the project (about 82). I guess many of them are false positives, so I won't litter the article with the samples and cite them in a separate list: ue-v595.txt. Excess variable declaration This error is pretty nice. It is about mistakenly declaring a new variable instead of using an already existing one. void FStreamableManager::AsyncLoadCallback(....) { .... FStreamable* Existing = StreamableItems.FindRef(TargetName); .... if (!Existing) { // hmm, maybe it was redirected by a consolidate TargetName = ResolveRedirects(TargetName); FStreamable* Existing = StreamableItems.FindRef(TargetName); } if (Existing && Existing->bAsyncLoadRequestOutstanding) .... } PVS-Studio's diagnostic message: V561 It's probably better to assign value to 'Existing' variable than to declare it anew. Previous declaration: streamablemanager.cpp, line 325. streamablemanager.cpp 332 I suspect the code must look like this: // hmm, maybe it was redirected by a consolidate TargetName = ResolveRedirects(TargetName); Existing = StreamableItems.FindRef(TargetName); Errors in function calls bool FRecastQueryFilter::IsEqual( const INavigationQueryFilterInterface* Other) const { // @NOTE: not type safe, should be changed when // another filter type is introduced return FMemory::Memcmp(this, Other, sizeof(this)) == 0; } PVS-Studio's diagnostic message: V579 The Memcmp function receives the pointer and its size as arguments. It is possibly a mistake. Inspect the third argument. pimplrecastnavmesh.cpp 172 The comment warns us that it is dangerous to use Memcmp(). But actually it all is even worse than the programmer expects. The point is that the function compares only a part of the object. The sizeof(this) operator returns the pointer size; that is, the function will compare the first 4 bytes in a 32-bit program and 8 bytes in a 64-bit program. The correct code should look as follows: return FMemory::Memcmp(this, Other, sizeof(*this)) == 0; But that's not the only trouble with the Memcmp() function. Have a look at the following code fragment: D3D11_STATE_CACHE_INLINE void GetBlendState( ID3D11BlendState** BlendState, float BlendFactor[4], uint32* SampleMask) { .... FMemory::Memcmp(BlendFactor, CurrentBlendFactor, sizeof(CurrentBlendFactor)); .... } PVS-Studio's diagnostic message: V530 The return value of function 'Memcmp' is required to be utilized. d3d11statecacheprivate.h 547 The analyzer was surprised at finding the Memcmp() function's result not being used anywhere. And this is an error indeed. As far as I get it, the programmer wanted to copy the data, not compare them. If so, the Memcpy() function should be used: FMemory::Memcpy(BlendFactor, CurrentBlendFactor, sizeof(CurrentBlendFactor)); A variable assigned to itself enum ECubeFace; ECubeFace CubeFace; friend FArchive& operatorGetMembers().GetAllocatedSize()) : 0; } PVS-Studio's diagnostic message: V502 Perhaps the '?:' operator works in a different way than it was expected. The '?:' operator has a lower priority than the '+' operator. materialshared.h 224 This code is pretty complicated. To make the explanation clearer, I have composed a simplified artificial sample: return A() + B() + C() + uniform ? UniformSize() : 0; A certain size is being calculated in this code. Depending on the value of the uniform variable, either UniformSize() or 0 should be added. But the code actually works in quite a different way. The priority of the addition operators '+' is higher than that of the '?:' operator. So here's what we get: return (A() + B() + C() + uniform) ? UniformSize() : 0; A similar issue can be found in Unreal Engine's code. I suspect the program calculates something different than the programmer wanted it to. Mess-up with enum I didn't feel like describing this case at first as I would have to cite quite a large piece of code. But then I overcame my laziness, so please be patient too. namespace EOnlineSharingReadCategory { enum Type { None = 0x00, Posts = 0x01, Friends = 0x02, Mailbox = 0x04, OnlineStatus = 0x08, ProfileInfo = 0x10, LocationInfo = 0x20, Default = ProfileInfo|LocationInfo, }; } namespace EOnlineSharingPublishingCategory { enum Type { None = 0x00, Posts = 0x01, Friends = 0x02, AccountAdmin = 0x04, Events = 0x08, Default = None, }; inline const TCHAR* ToString (EOnlineSharingReadCategory::Type CategoryType) { switch (CategoryType) { case None: { return TEXT("Category undefined"); } case Posts: { return TEXT("Posts"); } case Friends: { return TEXT("Friends"); } case AccountAdmin: { return TEXT("Account Admin"); } .... } } The analyzer generates a few V556 warnings at once on this code. The reason is that the switch operator has a variable of the EOnlineSharingReadCategory::Type type as its argument. At the same time, case operators work with values of a different type, EOnlineSharingPublishingCategory::Type. A logical error const TCHAR* UStructProperty::ImportText_Internal(....) const { .... if (*Buffer == TCHAR('\"')) { while (*Buffer && *Buffer != TCHAR('\"') && *Buffer != TCHAR('\n') && *Buffer != TCHAR('\r')) { Buffer++; } if (*Buffer != TCHAR('\"')) .... } PVS-Studio's diagnostic message: V637 Two opposite conditions were encountered. The second condition is always false. Check lines: 310, 312. propertystruct.cpp 310 The programmer intended to skip all text in double quotes. The algorithm was meant to be like this:Once the program comes across a double quote, a loop is started.The loop keeps skipping characters until stumbling across the next double quote.The error is about the pointer failing to be referenced to the next character after the first double quote is found. As a result, the second double quote is found right away, too, and the loop doesn't start. Here is simpler code to clarify the point: if (*p == '\"') { while (*p && *p != '\"') p++; } To fix the error, you need to change the code in the following way: if (*p == '\"') { p++; while (*p && *p != '\"') p++; } Suspicious shift class FMallocBinned : public FMalloc { .... /* Used to mask off the bits that have been used to lookup the indirect table */ uint64 PoolMask; .... FMallocBinned(uint32 InPageSize, uint64 AddressLimit) { .... PoolMask = ( ( 1 FORCENOINLINE void Set(....) { .... if ( DefinitionPtr == NULL ) { WidgetStyleValues.Add( PropertyName, MakeShareable( new DefinitionType( InStyleDefintion ) ) ); } else { WidgetStyleValues.Add( PropertyName, MakeShareable( new DefinitionType( InStyleDefintion ) ) ); } } PVS-Studio's diagnostic message: V523 The 'then' statement is equivalent to the 'else' statement. slatestyle.h 289 Miscellaneous What's left is just diverse subtle issues which are not very interesting to discuss. So let me just cite a few code fragments and corresponding diagnostic messages. void FNativeClassHeaderGenerator::ExportProperties(....) { .... int32 NumByteProperties = 0; .... if (bIsByteProperty) { NumByteProperties; } .... } PVS-Studio's diagnostic message: V607 Ownerless expression 'NumByteProperties'. codegenerator.cpp 633 static void GetModuleVersion( .... ) { .... char* VersionInfo = new char[InfoSize]; .... delete VersionInfo; .... } PVS-Studio's diagnostic message: V611 The memory was allocated using 'new T[]' operator but was released using the 'delete' operator. Consider inspecting this code. It's probably better to use 'delete [] VersionInfo;'. windowsplatformexceptionhandling.cpp 107 const FSlateBrush* FSlateGameResources::GetBrush( const FName PropertyName, ....) { .... ensureMsgf(BrushAsset, TEXT("Could not find resource '%s'"), PropertyName); .... } PVS-Studio's diagnostic message: V510 The 'EnsureNotFalseFormatted' function is not expected to receive class-type variable as sixth actual argument. slategameresources.cpp 49 Conclusions Using the static analyzer integrated into Visual Studio does make sense but it is not enough. The authors should consider using specialized tools in addition to it, for example our analyzer PVS-Studio. If you compare PVS-Studio to VS2013's analyzer, the former detects 6 times more bugs. Here you have the proof: Comparison of static code analyzers: CppCat, Cppcheck, PVS-Studio and Visual Studio;Comparison methodology. I invite all those who want their code to be high-quality to try our code analyzer. P.S. I should also mention that the errors described in this article (except for microoptimizations) could theoretically have been found by the lightweight analyzer CppCat as well. A one-year license for CppCat costs $250; annual renewal costs $200. But it wouldn't do in this particular case because it is lightweight and lacks the necessary functionality to monitoring compiler launches, which is a crucial requirement when checking Unreal Engine. However, the CppCat analyzer may well satisfy the authors of small projects.
  3. Hello, we are the developers of the PVS-Studio static code analyzer. We have created a new software product CppCat and are going to tell you about it in this article. Some time ago we imagined that we had never had PVS-Studio yet retained the experience of developing static analysis tools for C/C++ code. Our minds thus refreshed, we made a new static analyzer just the way we wanted it to be - simple and easy to use. You will also be glad to know that it costs $250 per installation. Background of the new product We always did our best to keep PVS-Studio easy to use and understand. But it inevitably grew to lose its simplicity as it acquired new functionality. For example, such is the ID field in the error table: some find it useful while others are confused by it. People mix it up with the diagnostic number and wonder why there can be IDs like 3, 7, 23, 25 - and what about the rest? The answer is simple and obvious: they are hidden; for example, the "64-bit" diagnostic set is turned off. Similar issues are with the settings as well. It is important in the case of large projects to have the option of choosing among preprocessors (Clang or Visual C++). It allows a user to greatly enhance the speed of analysis for certain projects. But those who are only getting started with the tool may be confused by this option. They select Clang and expect to get warnings generated by it. Some even send us indignant letters blaming us for selling an expensive add-on for Clang. Just in case, follow this link to find out how exactly we utilize Clang. You know, it's difficult to sell our analyzer to a programmer who mistakenly thinks that it is just a wrapper for Clang . This is how a vague setting gives rise to great confusion. We programmers are smart and sensible guys. But when it comes to learning new tools, we often resemble ordinary users. And it's quite okay: there's just too much information around us and you usually don't have enough time and strength to spend on learning every technological innovation. So, although programmers' tools are pretty intricate products, learning them should be as simple as handling a calculator is. Otherwise, you risk your tool just remaining unnoticed. People just won't have enough time to comprehend it. And that's what often happens to PVS-Studio. It's entirely our own fault that PVS-Studio has grown so complicated. And we have failed to find a way of returning the interface to the simplicity it possessed before. So, we created a new product: CppCat. This tool is so simple and streamlined that you will never get lost among its settings. Learning to use it takes less time than reading this article. Yes, it lacks quite a few features, but those are still there in PVS-Studio that will also remain. We just offer a cheaper carrot grater to those who don't need an expensive multi-function food processor. Now, let's point out the main idea once again. We've created an easy to learn and use tool for static analysis of C/C++ code. It will make your first steps into the static analysis methodology as simple as possible. And thanks to its simplicity and relatively low price, CppCat should go on to become a popular tool which will be as indespencible to the software developers, as a safety net for the rope-walker! Functionality The CppCat tool is designed to detect suspicious fragments in program code written in C/C++. The main functions of the analyzer are: Analysis of projects; Automatic analysis of files after compilation. That's all. The analyzer does exactly what it is designed for: checking the code and telling the programmer which fragments need closer examination. The tool can only work in Visual Studio (2010, 2012 and 2013). We decided to drop using the phrase "the analyzer detects bugs" too often. You see, any analyzer produces false positives. But when the analyzer generates warnings on correct code, it doesn't mean that you don't have to do anything with that code at all. We came across an interesting association in one article. The analyzer reveals code fragments with the "smell". "Smelly" code is not necessarily incorrect. It just means that it contains some anomalies that may cause some confusion when the code is maintained by other developers. "Smelly" code may lead to errors after refactoring: a programmer responsible for modifying the code may fail to understand how certain functions work and bring in some defects by a mistake. Thus, we believe that every code fragment that CppCat finds suspicious must be reviewed and improved. By clearing the code of its "smell", you will greatly help your coworkers. However, warnings do need to be suppressed sometimes. We offer a number of methods to do that - see the CppCat manual. Readers may still wonder: in what ways exactly is CppCat different from PVS-Studio? Here's the answer in the form of a summary table: Table 1. Comparing features of PVS-Studio against CppCat. On one hand, CppCat lacks much. But on the other, it retains all the functions necessary for everyday use in your work. The main ideological difference between CppCat and PVS-Studio is the following: CppCat licenses are individual. You install it on your computer and start using it. It doesn't provide functionality useful in team work - for example, you cannot set CppCat to run night checks. This and other functions of that kind are provided by PVS-Studio. PVS-Studio is intended for large, complex projects. CppCat can check large projects too - there's no restriction concerning the project size. It's just that CppCat doesn't have some of the additional functions and ships under a different licensing policy. CppCat can be used in companies (and large ones as well) in the same way - but you just need to purchase several licenses (and we offer discounts for buying several licenses at a time!). Pricing policy It's very simple with the prices. One copy of the product costs $250. The license is not floating - it is bound to one computer (hardware ID). We offer discounts depending on how many licenses you buy at a time: 1 - 4 licenses: $250.00 5 - 24 licenses: $225.00 25+ licenses: $212.50 The license is valid during 1 year. On the expiration, it can be renewed at 80% of the initial price (i.e. $200). Downloading, trying and purchasing Nothing complicated about that too: Visit the product website: http://www.cppcat.com Support is provided via e-mail: team@cppcat.com The full-function trial version can be used for 7 days. And what about PVS-Studio? We go on developing PVS-Studio as well. It is still relevant and all its aspects, as well as the pricing policy, remain the same. Conclusion We are launching a sale concurrently with our new product's release. You get a 5% discount when buying CppCat during 5 days (up to January, 19). Follow this link for the bargain.  
  4. Thank you. Written by stupidity. I'm a little fix this.
  5. Abstract About a year ago we published in our blog a series of articles on development of Visual Studio plugins in C#. We have recently revised those materials and added new sections and now invite you to have a look at the updated version of the manual. Creating extension packages (plug-ins) for Microsoft Visual Studio IDE appears as quite an easy task at the first sight. There exist an excellent MSDN documentation, as well as various articles, examples and a lot of other additional sources on this topic. But, at the same time, it could also appear as a difficult task when an unexpected behavior is encountered along the way. Although it can be said that such issues are quite common to any programming task, the subject of IDE plug-in development is still not thoroughly covered at this moment. We develop PVS-Studio static code analyzer. Although the tool itself is intended for C++ developers, quite a large fragment of it is written in C#. When we just had been starting the development of our plug-in, Visual Studio 2005 had been considered as modern state-of-the-art IDE. Although, at this moment of Visual Studio 2012 release, some could say that Visual Studio 2005 is not relevant anymore, we still provide support for this version in our tool. During our time supporting various Visual Studio versions and exploring capabilities of the environment, we've accumulated a large practical experience on how to correctly (and even more so incorrectly!) develop IDE plug-ins. As holding all of this knowledge inside us was becoming unbearable, we've decided to publish it here. Some of our solutions that seem quite obvious right now were discovered in the course of several years. And the same issues could still haunt other plug-in developers. The following topics will be covered: basic information on creating and debugging MSVS plug-ins and maintaining these extensibility projects for several versions of Visual Studio inside a common source code base;overview of Automation Object Model and various Managed Package Framework (MPF) classesextending interface of the IDE though the automation object model's API (EnvDTE) and MPF (Managed Package Framework) classes with custom menus, toolbars, windows and options pages;overview of Visual Studio project model; Atmel Studio IDE, which is based on Visual Studio Isolated Shell, as an example of interaction with custom third-party project models.utilizing Visual C++ project model for gathering data needed to operate an external preprocessor/compiler, such as compilation arguments and settings for different platforms and configurations; A list of more detailed in-depth references for the article covered here are available at the end of each topic through the links to MSDN library and several other external resources. The articles will cover the extension development only for Visual Studio 2005 and later versions. This limitation reflects that PVS-Studio also supports integration to Visual Studio starting only from version 8 (Visual Studio 2005). The main reason behind this is that a new extensibility API model was introduced for Visual Studio 2005, and this new version is not backward-compatible with previous IDE extensibility APIs. Creating, debugging and deploying extension packages for Microsoft Visual Studio 2005/2008/2010/2012 This item contains the overview of several different methods for extending Visual Studio IDE functionality. The creation, debugging, registration and end-user deployment of Visual Studio extension packages will be explained in detail. Creating and debugging Visual Studio and Visual Studio Isolated Shell VSPackage extension modules There exists a number of ways to extend Microsoft Visual Studio features. On the most basic level it's possible to automate simple routine user actions using macros. An add-In plug-in module can be used for obtaining an access to environment's UI objects, such as menu commands, windows etc. Extension of IDE's internal editors is possible through MEF (Managed Extensibility Framework) components (starting with MSVS 2010). Finally, a plug-in of the Extension Package type (known as VSPackage) is best suited for integrating large independent components into Visual Studio. VSPackage allows combining the environment automation through Automation Object Model with usage of Managed Package Framework classes (such as Package). In fact, while Visual Studio itself provides only the basic interface components and services, such standard modules as Visual C++ or Visual C# are themselves implemented as IDE extesnions. In its earlier versions, PVS-Studio plug-in (versions 1.xx and 2.xx to be precise, when it was still known as Viva64) existed as an Add-In package. Starting with PVS-Studio 3.0 it was redesigned as VSPackage because the functionality Add-in was able to provide became insufficient for the tasks at hand and also the debugging process was quite inconvenient. After all, we wanted to have our own logo on Visual Studio splash screen! VSPackage also provides the means of extending the automation model itself by registering user-defined custom automation objects within it. Such user automation objects will become available through the same automation model to other user-created extensibility packages, providing these packages with access to your custom components. This, in turn, allows third-party developers to add the support of new programming languages and compilers through such extensions into the IDE and also to provide interfaces for the automation of these new components as well. Besides extending the Visual Studio environment itself, VSPackage extensions could be utilized for addition of new features into Visual Studio Isolated\Integrated shells. Isolated\integrated shell provides any third-party developer with the ability to re-use basic interface components and services of Visual Studio (such as a code editor, autocompletion system etc.), but also to implement the support of other custom project models and\or compilers. Such a distribution will not include any of Microsoft proprietary language modules (such as Visual C++, Visual Basic and so on), and it could be installed by an end-user even if his or her system does not contain a previous Visual Studio installation. An isolated shell application will remain a separate entity after the installation even if the system contains a previous Visual Studio installation, but an integrated shell application will be merged into the preinstalled version. In case the developer of isolated\integrated shell extends the Visual Studio automation model by adding interfaces to his or her custom components, all other developers of VSPackage extensions will be able to add such components as well. Atmel Studio, an IDE designed for the development of embedded systems, is an example of Visual Studio Isolated Shell application. Atmel Studio utilizes its own custom project model which, in turn, itself is the implementation of a standard Visual Studio project model for the MSBuild, and the specific version of the gcc compiler. Projects for VSPackage plug-in modules. Creating the extension package. Let's examine the creation of Visual Studio Package plug-in (VSPackage extension). Contrary to Add-In plug-ins, developing VS extension packages requires the installation of Microsoft Visual Studio SDK for a targeted version of IDE, i.e. a separate SDK should be installed with every version of Visual Studio for which an extension is being developed. In case of the extension that targets Visual Studio Isolated\Integrated Shell, an SDK for the version of Visual Studio on which such a shell is based will be required. We will be examining extension development for the 2005, 2008, 2009 and 2012 versions of Visual Studio and Visual Studio 2010 based Isolated Shells. Installation of Visual Studio SDK adds a standard project template for Visual Studio Package (on the 'Other Project Types -> Extensibility' page) to VS template manager. If selected, this template will generate a basic MSBuild project for an extension package, allowing several parameters to be specified beforehand, such as a programming language to be used and the automatic generation of several stub components for generic UI elements, such as menu item, an editor, user tool window etc. We will be using a C# VSPackage project (csproj), which is a project for managed dynamic-link library (DLL). The corresponding csproj MSBuild project for this managed assembly will also contain several XML nodes specific to a Visual Studio package, such as VSCT compiler and IncludeinVSIX (in later IDE versions). The main class of an extension package should be inherited from the Microsoft.VisualStudio.Shell.Package. This base class provides managed wrappers for IDE interaction APIs, implementation of which is required from a fully-functional Visual Studio extension package public sealed class MyPackage: Package { public MyPackage () {} ... } The Package class allows overriding of its base Initialize method. This method receives execution control at the moment of package initialization in the current session of IDE. protected override void Initialize() { base.Initialize(); ... } The initialization of the module will occur when it is invoked for the first time, but it also could be triggered automatically, for example after IDE is started or when user enters a predefined environment UI context state. Being aware of the package's initialization and shutdown timings is crucial. It's quite possible that the developer would be requesting some of Visual Studio functionality at the moment when it is still unavailable to the package. During PVS-Studio development we've encountered several such situations when the environment "punished us" for not understanding this, for instance, we are not allowed to "straightforwardly" display message boxes after Visual Studio enters a shutdown process. Debugging extension packages. Experimental Instance. The task of debugging a plug-in module or extension intended for an integrated development environment is not quite a trivial one. Quite often such environment itself is utilized for plug-in's development and debugging. Hooking up an unstable module to this IDE can lead to instability of the environment itself. The necessity to uninstall a module under development from the IDE before every debugging session, which in turn often requires restarting it, is also a major inconvenience (IDE could block the DLL that needs to be replaced by a newer version for debugging). It should be noted that a VSPackage debugging process in this aspect is substantially easier than that of an Add-In package. This was one of the reasons for changing the project type of PVS-Studio plug-in. VSPackage solves the aforementioned development and debugging issues by utilizing Visual Studio Experimental Instance mechanism. Such an experimental instance could be easily started by passing a special command line argument: "C:\Program Files (x86)\Microsoft Visual Studio 10.0\ Common7\IDE\devenv.exe" /RootSuffix Exp An experimental instance of the environment utilizes a separate independent Windows registry hive (called experimental hive) for storing all of its settings and component registration data. As such, any modifications in the IDE's settings or changes in its component registration data, which were made inside the experimental hive, will not affect the instance which is employed for the development of the module (that is your main regular instance which is used by default). Visual Studio SDK provides a special tool for creating or resetting such experimental instances -- CreateExpInstance. To create a new experimental hive, it should be executed with these arguments: CreateExpInstance.exe /Reset /VSInstance=10.0 /RootSuffix=PVSExp Executing this command will create a new experimental registry hive with a PVSExp suffix in its name for the 10th version of IDE (Visual Studio 2010), also resetting all of its settings to their default values in advance. The registry path for this new instance will look as follows: HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\10.0PVSExp While the Exp suffix is utilized by default for package debugging inside VSPackage template project, other experimental hives with unique names could also be created by the developer at will. To start an instance of the environment for the hive we've created earlier (containing PVSExp in its name), these arguments should be used: "C:\Program Files (x86)\Microsoft Visual Studio 10.0\ Common7\IDE\devenv.exe" /RootSuffix PVSExp A capacity for creating several different experimental hives on a single local workstation could be quite useful, as, for example, to provide a simultaneous and isolated development of several extension packages. After installing the SDK package, a link is created in the Visual Studio program's menu group for resetting the default Experimental Instance for this version of the IDE (for instance, "Reset the Microsoft Visual Studio 2010 Experimental Instance"). In case of extension targeting an Isolated Shell, the issues with the "corruption" of the development environment are irrelevant, and so there is no need for Experimental Instance utilization. But, in any case, the faster you'll figure out how the debugging environment works, the fewer issues you'll encounter in understanding how plug-in initialization works during development. Registering and deploying Visual Studio extension packages Registering a VS extension package requires registering a package itself, as well as registering all of the components it integrates into the IDE (for example, menu items, option pages, user windows etc.). The registration is accomplished by creating records corresponding to these components inside the main system registry hive of Visual Studio. All the information required for registration is placed, after building your VSPackage, inside a special pkgdef file, according to several special attributes of the main class of your package (which itself should be a subclass of the MPF Package class). The pkgdef can also be created manually using the CreatePkgDef. This tool collects all of the required module registration information from these special attributes by the means of .NET reflection. Let's study these registration attributes in detail. The PackageRegistration attribute tells the registration tool that this class is indeed a Visual Studio extension package. Only if this attribute is discovered will the tool perform its search for additional ones. [PackageRegistration(UseManagedResourcesOnly = true)] The Guid attribute specifies a unique package module identifier, which will be used for creating a registry sub-key for this module in Visual Studio hive. [Guid("a0fcf0f3-577e-4c47-9847-5f152c16c02c")] The InstalledProductRegistration attribute adds information to 'Visual Studio Help -> About' dialog and the loading splash screen. [InstalledProductRegistration("#110", "#112", "1.0", IconResourceID = 400)] The ProvideAutoLoad attribute links automatic module initialization with the activation of a specified environment UI context. When a user enters this context, the package will be automatically loaded and initialized. This is an example of setting module initialization to the opening of a solution file: [ProvideAutoLoad("D2567162-F94F-4091-8798-A096E61B8B50")] The GUID values for different IDE UI contexts can be found in the Microsoft.VisualStudio.VSConstants.UICONTEXT class. The ProvideMenuResource attribute specifies an ID of resource that contains user created menus and commands for their registration inside IDE. [ProvideMenuResource("Menus.ctmenu", 1)] The DefaultRegistryRoot attribute specifies a path to be used for writing registration data to the system registry. Starting with Visual Studio 2010 this attribute can be dropped as the corresponding data will be present in manifest file of a VSIX container. An example of registering a package for Visual Studio 2008: [DefaultRegistryRoot("Software\\Microsoft\\VisualStudio\\9.0")] Registration of user-created components, such as toolwidows, editors, option pages etc. also requires the inclusion of their corresponding attributes for the user's Package subclass. We will examine these attributes separately when we will be examining corresponding components individually. It's also possible to write any user-defined registry keys (and values to already existing keys) during package registration through custom user registration attributes. Such attributes can be created by inheriting the RegistrationAttribute abstract class. [AttributeUsage(AttributeTargets.Class, Inherited = true, AllowMultiple = false)] public class CustomRegistrationAttribute : RegistrationAttribute { } The RegistrationAttribute-derived attribute must override its Register and Unregister methods, which are used to modify registration information in the system registry. The RegPkg tool can be used for writing registration data to Windows registry. It will add all of the keys from pkgdef file passed to it into the registry hive specified by the /root argument. For instance, the RegPkg is utilized by default in Visual Studio VSPackage project template for registering the module in the Visual Studio experimental hive, providing convenient seamless debugging of the package being developed. After all of the registration information have been added to the registry, Visual Studio (devenv.exe) should be started with '/setup' switch to complete registration for new components inside the IDE. Deploying plug-ins for developers and end-users. Package Load Key. Before proceeding to describe the deployment process itself, one particular rule should be stressed: Each time after a new version of the distribution containing your plug-in is created, this new distribution should be tested on a system without Visual Studio SDK installed, as to make sure that it will be registered correctly on the end-user system. Today, as the releases of early versions of PVS-Studio are past us, we do not experience these kinds of issues, but several of these early first versions were prone to them. Deploying a package for Visual Studio 2005/2008 will require launching of regpkg tool for a pkgdef file and passing the path to Visual Studio main registry hive into it. Alternately, all keys from a pkgdef can be written to Windows registry manually. Here is the example of automatically writing all the registration data from a pkgdef file by regpkg tool (in a single line): RegPkg.exe /root:Software\Microsoft\VisualStudio\9.0Exp "/pkgdeffile:obj\Debug\PVS-Studio-vs2008.pkgdef" "C:\MyPackage\MyPackage.dll" After adding the registration information to the system registry, it is necessary to start Visual Studio with a /setup switch to complete the component registration. It is usually the last step in the installation procedure of a new plug-in. Devenv.exe /setup Starting the environment with this switch instructs Visual Studio to absorb resource metadata for user-created components from all available extension packages, so that these components will be correctly displayed by IDE's interface. Starting devenv with this key will not open its main GUI window. We do not employ RepPkg utility as part of PVS-Studio deployment, instead manually writing required data to the registry by using our stand-alone installer. We chose this method because we have no desire of being dependent on some external third-party tools and we want full control over the installation process. Still, we do use RegPkg during plug-in development for convenient debugging. VSIX packages Beginning from Visual Studio 2010, VSPackage deployment process can be significantly simplified through the usage of VSIX packages. VSIX package itself is a common (Open Packaging Conventions) archive containing plug-in's binary files and all of the other auxiliary files which are necessary for plug-in's deployment. By passing such archive to the standard VSIXInstaller.exe utility, its contents will be automatically registered in the IDE: VSIXInstaller.exe MyPackage.vsix VSIX installer could also be used with /uninstall switch to remove the previously installed package from a system. A unique GUID of the extension package should be used to identify such package: VSIXInstaller.exe /uninstall: 009084B1-6271-4621-A893-6D72F2B67A4D Contents of a VSIX container are defined through the special vsixmanifest file, which should be added to plug-in's project. Vsixmanifest file permits the following properties to be defined for an extension:targeted Visual Studio versions and editions, which will be supported by the plug-in;a unique GUID identifier;a list of components to be registered (VSPackage, MEF components, toolbox control etc.);general information about the plug-in to be installed (description, license, version, etc.);To include additional files into a VSIX container, the IncludeInVSIX node should be added to their declarations inside your MSBuild project (alternately, they could also be marked as included into VSIX from their respective property windows, by opening it from Visual Studio Solution Explorer). true In fact, the VSIX file could be viewed as an almost full-fledged installer for extension packages on the latest versions of Visual Studio (2010 and 2012), allowing the extensions to be deployed by a "one-click" method. Publishing your VSIX container in the official Visual Studio Gallery for extensions allows end-users to install such packages through the Tools -> Extension Manager IDE dialog. VSIX allows the extension to be deployed either for one of the regular Visual Studio editions, or for the isolated\integrated shell based distributions. In case of developing an extension for isolated shell application, instead of the Visual Studio version the VSIX manifest file should contain a special identification string for the targeted environment. For example, the identification string for Atmel Studio 6.1 should be "AtmelStudio, 6.1". But, if the extension you are developing utilizes only common automation model interfaces (such as the ones for the text editor, abstract project tree and so on), and does not require any of the specific ones (for example, interfaces for Visual C++ projects), then it is possible for you to specify several different editions of Visual Studio, as well as an isolated shell based ones, in the manifest file. This, in turn, will permit you to use a single installer for a wide range of Visual Studio based applications. This new VSIX installation procedure in Visual Studio 2010 does substantially alleviate package deployment for end-users (as well as for developers themselves). Some developers even had decided to support only VS2010 IDE and versions above it, if only not to get involved with the development of a package and installer for earlier IDE versions. Unfortunately, several issues can be encountered when using VSIX installer together with Visual Studio 2010 extension manager interface. For instance, sometimes the extension's binary files are not removed correctly after uninstall, which in turn blocks the VSIX installer from installing/reinstalling the same extension. As such, we advise you not to depend upon the VSIX installer entirely and to provide some backup, for example by directly removing your files from a previous plug-in installation before proceeding with a new one. Package Load Key Each VSPackage module loaded into Visual Studio must possess a unique Package Load Key (PLK). PLK key is specified through the ProvideLoadKey attribute for the Package subclass in 2005/2008 versions of the IDE. [ProvideLoadKey("Standard", "9.99", "MyPackage", "My Company", 100)] Starting with Visual Studio 2010, the presence of a PLK, as well as of the ProvideLoadKey attribute respectively, in a package is not required, but it can still be specified in case the module under development is targeting several versions of MSVS. The PLK can be obtained by registering at the Visual Studio Industry Partner portal, meaning it guarantees that the development environment can load only packages certified by Microsoft. However, systems containing Visual Studio SDK installed are exceptions to this, as Developer License Key is installed together with the SDK. It allows the corresponding IDE to load any extension package, regardless of validity of its PLK. Considering the aforementioned, one more time it is necessary to stress the importance of testing the distribution on a system without Visual Studio SDK present, because the extension package will operate properly on developer's workstation regardless of its PLK correctness. Extension registration specifics in the context of supporting several different versions of Visual Studio IDE By default, VSPackage project template will generate an extensibility project for the version of Visual Studio that is used for the development. This is not a mandatory requirement though, so it is possible to develop an extension for a particular version of IDE using a different one. It also should be noted that after automatically upgrading a project file to a newer version through devenv /Upgrade switch, the targeted version of the IDE and its corresponding managed API libraries will remain unchanged, i.e. from a previous version of Visual Studio. To change the target of the extension to another version of Visual Studio (or to register an extension into this version to be more precise), you should alter values passed to the DefaultRegistryRoot attribute (only for 2005/2008 IDE versions, as starting from Visual Studio 2010 this attribute is no longer required) or change the target version in the VSIX manifest file (for versions above 2008). VSIX support appears only starting from Visual Studio 2010, so building and debugging the plug-in targeted for the earlier IDE version from within Visual Studio 2010 (and later) requires setting up all the aforementioned registration steps manually, without VSIX manifest. While changing target IDE version one should also not forget to switch referenced managed assemblies, which contain COM interface wrappers utilized by the plug-in, to the corresponding versions as well. Altering the IDE target version of the plug-in affects the following Package subclass attributes:the InstalledProductRegistration attribute does not support overloading of its constructor with a (Boolean, String, String, String) signature, starting from Visual Studio 2010;the presence of DefaultRegistryRoot and ProvideLoadKey attributes is not mandatory starting from Visual Studio 2010, as similar values are now specified inside VSIX manifest; References MSDN. Experimental Build.MSDN. How to: Register a VSPackage.MSDN. VSIX Deployment.MSDN. How to: Obtain a PLK for a VSPackage.MZ-Tools. Resources about Visual Studio .NET extensibility.MSDN. Creating Add-ins and Wizards.MSDN. Using a Custom Registration Attribute to Register an Extension.MSDN. Shell (Integrated or Isolated). Visual Studio Automation Object Model. EnvDTE and Visual Studio Shell Interop interfaces. This item contains an overview of Visual Studio Automation Object Model. Model's overall structure and the means of obtaining access to its interfaces through DTE/DTE2 top level objects are examined. Several examples of utilizing elements of the model are provided. Also discussed are the issues of using model's interfaces within multithreaded applications; an example of implementing such mechanism for multithreaded interaction with COM interfaces in managed code is provided as well. Introduction Visual Studio development environment is built upon the principles of automation and extensibility, providing the developers using it with the ability of integrating almost any custom element into the IDE and allowing for an easy interaction with its default and user-created components. As the means of implementing these tasks, Visual Studio users are provided with several cross-complementing toolsets, the most basic and versatile among these is the Visual Studio Automation Object Model. Automation Object Model is represented by a series of libraries containing a vast and well-structured API set which covers all aspects of IDE automation and the majority of its extensibility capabilities. Although, in comparison to other IDE extensibility tools, this model does not provide access to some portions of Visual Studio (this applies mostly to the extension of some IDE's features), it is nonetheless the most flexible and versatile among them. The majority of the model's interfaces are accessible from within every type of IDE extension module, which allows interacting with the environment even from an external independent process. Moreover, the model itself could be extended along with the extension of Visual Studio IDE, providing other third-party developers with an access to user-created custom components. Automation Object Model structure Visual Studio automation model is composed of several interconnected functional object groups covering all aspects of the development environment; it also provides capabilities for controlling and extending these groups. Accessing any of them is possible through the top-level global DTE interface (Development Tools Environment). Figure 1 shows the overall structure of the automation model and how it is divided among functionality groups. Figure 1 -- Visual Studio Automation Object Model (click the picture to zoom in) The model itself could be extended by user in one of the following groups:project models (implementing new project types, support for new languages);document models (implementing new document types and document editors)code editor level models (support for specific language constructs)project build-level modelsAutomation model could be extended from plug-ins of VSPackage type only. All of the automation model's interfaces could be conventionally subdivided into two large groups. 1st group are the interfaces of the EnvDTE and Visual Studio Interop namespaces, these interfaces allow interactions with basic common components of the IDE itself, such as tool windows, editors, event handling services and so on. 2nd group are the interfaces of the specific project model. The figure above specifies this interface group as late-bound properties, i.e. these interfaces are implemented in a separate dynamically loaded library. Each standard (i.e. the one that is included in a regular Visual Studio distribution) project model, such as Visual C++ or Visual Basic, provides a separate implementation for these interfaces. Third-party developers are able to extend the automation model by adding their own custom project models and by providing an implementation of these automation interfaces. Also worth noting is that the interfaces of the 1st group, which was specified above, are universal, meaning that they could be utilized for interaction with any of the project models or Visual Studio editions, including the integrated\isolated Visual Studio shells. In this article we will examine this group in more detail. But still, despite the model's versatility, not every group belonging to the model could be equally utilized from all the types of IDE extensions. For instance, some of the model's capabilities are inaccessible to external processes; these capabilities are tied to specific extension types, such as Add-In or VSPackage. Therefore, when selecting the type for the extension to be developed, it is important to consider the functionality that this extension will require. The Microsoft.VisualStudio.Shell.Interop namespace also provides a group of COM interfaces, which can be used to extend and automate Visual Studio application from managed code. Managed Package Framework (MPF) classes, which we utilized earlier for creating a VSPackage plugin, are actually themselves based on these interfaces. Although theses interfaces are not a part of EnvDTE automation model described above, nevertheless they greatly enhance this model by providing additional functionality for VSPackage extensions, which is otherwise unavailable for extensions of other types. Obtaining references to DTE/DTE2 objects. In order to create a Visual Studio automation application it is necessary to obtain access to the automation objects themselves in the first place. To accomplish this, first of all it is necessary to hook up the correct versions of libraries containing the required managed API wrappers in the EnvDTE namespace. Secondly, the reference to the automation model top-level object, that is the DTE2 interface, should be obtained. In the course of Visual Studio evolution, several of its automation objects had been modified or received some additional functionality. So, to maintain a backward compatibility with existing extension packages, new EnvDTE80, EnvDTE90, EnvDTE100 etc. namespaces were created instead of updating the interfaces from the original EnvDTE namespace. The majority of such updated interfaces from these new namespaces do maintain the same names as in the original ones, but with addition of an ordinal number at the end of the name, for example Solution and Solution2. It is advised that these updated interfaces should be utilized when creating a new project, as they do contain the most recent functionality. It's worth noting that properties and methods of DTE2 interface usually return object references with types corresponding to the original DTE, i.e. accessing dte2.Solution will return Solution and not the Solution2 as it would seem. Although these new EnvDTE80, EnvDTE90, EnvDTE100 namespaces do contain some of the updated functionality as mentioned above, still it is the EnvDTE interface that contains the majority of automation objects. Therefore, in order to possess access to all of the existing interfaces, it is necessary to link all versions of the managed COM wrapper libraries to the project, as well as to obtain the references to DTE and also to DTE2. The way of obtaining top-level EnvDTE object reference is dependent upon the type of IDE extension being developed. Let's examine 3 of such extension types: Add-In, VSPackage and an MSVS-independent external process. Add-In extension In the case of an Add-In extension, access to the DTE interface can be obtained inside the OnConnection method which should be implemented for the IDTExtensibility interface that provides access to the extension-environment interaction events. The OnConnection method is called at the moment when the module is loaded by the IDE; it can happen either when the environment is being loaded itself or after the extension was called for the first time in the IDE session. The example of obtaining the reference follows: public void OnConnection(object application, ext_ConnectMode connectMode, object addInInst, ref Array custom) { _dte2 = (DTE2)application; ... } An Add-In module can be initialized either at the moment of IDE start-up, or when it is called for the first time in current IDE session. So, the connectMode can be used to correctly determine the moment of initialization inside the OnConnection method. switch(connectMode) { case ext_ConnectMode.ext_cm_UISetup: ... break; case ext_ConnectMode.ext_cm_Startup: ... break; case ext_ConnectMode.ext_cm_AfterStartup: ... break; case ext_ConnectMode.ext_cm_CommandLine: ... break; } As in the example above, Add-In could be loaded either simultaneously with the IDE itself (if the startup option in the Add-In manager is checked), when it is called the first time or when it is called through the command line. The ext_ConnectMode.ext_cm_UISetup option is invoked only for a single time in the plug-in's overall lifetime, which is during its first initialization. This case should be used for initializing user UI elements which are to be integrated into the environment (more on this later on). If an Add-In is being loaded during Visual Studio start-up (ext_ConnectMode.ext_cm_Startup), then at the moment OnConnect method receives control for the first time, it is possible that the IDE still is not fully initialized itself. In such a case, it is advised to postpone the acquisition of the DTE reference until the environment is fully loaded. The OnStartupComplete handler provided by the IDTExtensibility can be used for this. public void OnStartupComplete(ref Array custom) { ... } VSPackage extension For VSPackage type of extension, the DTE could be obtained through the global Visual Studio service with the help of GetService method of a Package subclass: DTE dte = MyPackage.GetService(typeof(DTE)) as DTE; Please note that the GetService method could potentially return null in case Visual Studio is not fully loaded or initialized at the moment of such access, i.e. it is in the so called "zombie" state. To correctly handle this situation, it is advised that the acquisition of DTE reference should be postponed until this interface is inquired. But in case the DTE reference is required inside the Initialize method itself, the IVsShellPropertyEvents interface can be utilized (also by deriving our Package subclass from it) and then the reference could be safely obtained inside the OnShellPropertyChange handler. DTE dte; uint cookie; protected override void Initialize() { base.Initialize(); IVsShell shellService = GetService(typeof(SVsShell)) as IVsShell; if (shellService != null) ErrorHandler.ThrowOnFailure( shellService.AdviseShellPropertyChanges(this,out cookie)); ... } public int OnShellPropertyChange(int propid, object var) { // when zombie state changes to false, finish package initialization if ((int)__VSSPROPID.VSSPROPID_Zombie == propid) { if ((bool)var == false) { this.dte = GetService(typeof(SDTE)) as DTE; IVsShell shellService = GetService(typeof(SVsShell)) as IVsShell; if (shellService != null) ErrorHandler.ThrowOnFailure( shellService.UnadviseShellPropertyChanges(this.cookie) ); this.cookie = 0; } } return VSConstants.S_OK; } It should be noted that the process of VSPackage module initialization at IDE startup could vary for different Visual Studio versions. For instance, in case of VS2005 and VS2008, an attempt at accessing DTE during IDE startup will almost always result in null being returned, owning to the relative fast loading times of these versions. But, one does not simply obtain access into DTE. In Visual Studio 2010 case, it mistakenly appears that one could simply obtain an access to the DTE from inside the Initialize() method. In fact, this impression is a false one, as such method of DTE acquisition could potentially cause the occasional appearance of "floating" errors which are hard to identify and debug, and even the DTE itself may be still uninitialized when the reference is acquired. Because of these disparities, the aforementioned acquisition method for handling IDE loading states should not be ignored on any version of Visual Studio. Independent external process The DTE interface is a top-level abstraction for Visual Studio environment in the automation model. In order to acquire a reference to this interface from an external application, its ProgID COM identifier could be utilized; for instance, it will be "VisualStudio.DTE.10.0" for Visual Studio 2010. Consider this example of initializing a new IDE instance and when obtaining a reference to the DTE interface. // Get the ProgID for DTE 8.0. System.Type t = System.Type.GetTypeFromProgID( "VisualStudio.DTE.10.0", true); // Create a new instance of the IDE. object obj = System.Activator.CreateInstance(t, true); // Cast the instance to DTE2 and assign to variable dte. EnvDTE80.DTE2 dte = (EnvDTE80.DTE2)obj; // Show IDE Main Window dte.MainWindow.Activate(); In the example above we've actually created a new DTE object, starting deven.exe process by the CreateInstance method. But at the same time, the GUI window of the environment will be displayed only after the Activate method is called. Next, let's review a simple example of obtaining the DTE reference from an already running Visual Studio Instance: EnvDTE80.DTE2 dte2; dte2 = (EnvDTE80.DTE2) System.Runtime.InteropServices.Marshal.GetActiveObject( "VisualStudio.DTE.10.0"); However, in case several instances of the Visual Studio are executing at the moment of our inquiry, the GetActiveObject method will return a reference to the IDE instance that was started the earliest. Let's examine a possible way of obtaining the reference to DTE from a running Visual Studio instance by the PID of its process. using EnvDTE80; using System.Diagnostics; using System.Runtime.InteropServices; using System.Runtime.InteropServices.ComTypes; [DllImport("ole32.dll")] private static extern void CreateBindCtx(int reserved, out IBindCtx ppbc); [DllImport("ole32.dll")] private static extern void GetRunningObjectTable(int reserved, out IRunningObjectTable prot); public static DTE2 GetByID(int ID) { //rot entry for visual studio running under current process. string rotEntry = String.Format("!VisualStudio.DTE.10.0:{0}", ID); IRunningObjectTable rot; GetRunningObjectTable(0, out rot); IEnumMoniker enumMoniker; rot.EnumRunning(out enumMoniker); enumMoniker.Reset(); IntPtr fetched = IntPtr.Zero; IMoniker[] moniker = new IMoniker[1]; while (enumMoniker.Next(1, moniker, fetched) == 0) { IBindCtx bindCtx; CreateBindCtx(0, out bindCtx); string displayName; moniker[0].GetDisplayName(bindCtx, null, out displayName); if (displayName == rotEntry) { object comObject; rot.GetObject(moniker[0], out comObject); return (EnvDTE80.DTE2)comObject; } } return null; } Here we've acquired the DTE interface by identifying the required instance of the IDE in the table of running COM objects (ROT, Running Object Table) by its process identifier. Now we can access the DTE for all of the executing instances of Visual Studio, for example: Process Devenv; ... //Get DTE by Process ID EnvDTE80.DTE2 dte2 = GetByID(Devenv.Id); Additionally, to acquire any project-specific interface (including custom model extensions), for example the CSharpProjects model, through a valid DTE interface, the GetObject method should be utilized: Projects projects = (Projects)dte.GetObject("CSharpProjects"); The GetObject method will return a Projects collection of regular Project objects, and each one of them will contain a reference to our project-specific properties, among other regular ones. Visual Studio text editor documents Automation model represents Visual Studio text documents through the TextDocument interface. For example, C/C++ source code files are opened by the environment as text documents. TextDocument is based upon the common automation model document interface (the Document interface), which represents file of any type opened in Visual Studio editor or designer. A reference to the text document object can be obtained through the Object field of the Document object. Let's acquire a text document for the currently active (i.e. the one possessing focus) document from IDE's text editor. EnvDTE.TextDocument objTextDoc = (TextDocument)PVSStudio.DTE.ActiveDocument.Object("TextDocument"); Modifying documents The TextSelection document allows controlling text selection or to modify it. The methods of this interface represent the functionality of Visual Studio text editor, i.e. they allow the interaction with the text as it presented directly by the UI. EnvDTE.TextDocument Doc = (TextDocument)PVSStudio.DTE.ActiveDocument.Object(string.Empty); Doc.Selection.SelectLine(); TextSelection Sel = Doc.Selection; int CurLine = Sel.TopPoint.Line; String Text = Sel.Text; Sel.Insert("test\r\n"); In this example we selected a text line under the cursor, read the selected text and replaced it with a 'test' string. TextDocument interface also allows text modification through the EditPoint interface. This interface is somewhat similar to the TextSelection, but instead of operating with the text through the editor UI, it directly manipulates text buffer data. The difference between them is that the text buffer is not influenced by such editor-specific notions as WordWrap and Virtual Spaces. It should be noted that both of these editing methods are not able to modify read-only text blocks. Let's examine the example of modifying text with EditPoint by placing additional lines at the end of current line with a cursor. objEditPt = objTextDoc.StartPoint.CreateEditPoint(); int lineNumber = objTextDoc.Selection.CurrentLine; objEditPt.LineDown(lineNumber - 1); EditPoint objEditPt2 = objTextDoc.StartPoint.CreateEditPoint(); objEditPt2.LineDown(lineNumber - 1); objEditPt2.CharRight(objEditPt2.LineLength); String line = objEditPt.GetText(objEditPt.LineLength); String newLine = line + "test"; objEditPt.ReplaceText(objEditPt2, newLine, (int)vsEPReplaceTextOptions.vsEPReplaceTextKeepMarkers); Navigating the documents VSPackage modules are able to obtain access to a series of global services which could be used for opening and handling environment documents. These services could be acquired by the Package.GetGlobalService() method from Managed Package Framework. It should be noted that the services described here are not part of the EnvDTE model and are accessible only from a Package-type extension, and therefore they could not be utilized in other types of Visual Studio extensions. Nonetheless, they can be quite useful for handling IDE documents when they are utilized in addition to the Documents interface described earlier. Next, we'll examine these services in more detail. The IVsUIShellOpenDocument interface controls the state of documents opened in the environment. Following is the example that uses this interface to open a document through path to a file which this document will represent. String path = "C:\Test\test.cpp"; IVsUIShellOpenDocument openDoc = Package.GetGlobalService(typeof(IVsUIShellOpenDocument)) as IVsUIShellOpenDocument; IVsWindowFrame frame; Microsoft.VisualStudio.OLE.Interop.IServiceProvider sp; IVsUIHierarchy hier; uint itemid; Guid logicalView = VSConstants.LOGVIEWID_Code; if (ErrorHandler.Failed( openDoc.OpenDocumentViaProject(path, ref logicalView, out sp, out hier, out itemid, out frame)) || frame == null) { return; } object docData; frame.GetProperty((int)__VSFPROPID.VSFPROPID_DocData, out docData); The file will be opened in a new editor or will receive focus in case it already has been opened earlier. Next, let's read a VsTextBuffer text buffer from this document we opened: // Get the VsTextBuffer VsTextBuffer buffer = docData as VsTextBuffer; if (buffer == null) { IVsTextBufferProvider bufferProvider = docData as IVsTextBufferProvider; if (bufferProvider != null) { IVsTextLines lines; ErrorHandler.ThrowOnFailure(bufferProvider.GetTextBuffer( out lines)); buffer = lines as VsTextBuffer; Debug.Assert(buffer != null, "IVsTextLines does not implement IVsTextBuffer"); if (buffer == null) { return; } } } The IVsTextManager interface controls all of the active text buffers in the environment. For example we can navigate a text document using the NavigateToLineAndColumn method of this manager on a buffer we've acquired earlier: IVsTextManager mgr = Package.GetGlobalService(typeof(VsTextManagerClass)) as IVsTextManager; mgr.NavigateToLineAndColumn(buffer, ref logicalView, line, column, line, column); Subscribing and handling events Automation objects events are represented by the DTE.Events property. This element references all of the common IDE events (such as CommandEvents, SolutionEvents), as well as the events of separate environment components (project types, editors, tools etc.), also including the ones designed by third-party developers. To acquire a reference for this automation object, the GetObject method could be utilized. When subscribing to the DTE events one should remember that this interface could be still unavailable at the moment of extension being initialized. So it is always important to consider the sequence of your extension initialization process if the access to DTE.Events is required in the Initialize() method of your extension package. The correct handling of initialization sequence will vary for different extension types, as it was described earlier. Let's acquire a reference for an events object of Visual C++ project model defined by the VCProjectEngineEvents interface and assign a handler for the removal of an element from the Solution Explorer tree: VCProjectEngineEvents m_ProjectItemsEvents = PVSStudio.DTE.Events.GetObject("VCProjectEngineEventsObject") as VCProjectEngineEvents; m_ProjectItemsEvents.ItemRemoved += new _dispVCProjectEngineEvents_ItemRemovedEventHandler( m_ProjectItemsEvents_ItemRemoved); MDI windows events The Events.WindowEvents property could be utilized to handle regular events of an environment MDI window. This interface permits the assignment of a separate handler for a single window (defined through the EnvDTE.Window interface) or the assignment of a common handler for all of the environment's windows. Following example contains the assignment of a handler for the event of switching between IDE windows: WindowEvents WE = PVSStudio.DTE.Events.WindowEvents; WE.WindowActivated += new _dispWindowEvents_WindowActivatedEventHandler( Package.WE_WindowActivated); Next example is the assignment of a handler for window switching to the currently active MDI window through WindowEvents indexer: WindowEvents WE = m_dte.Events.WindowEvents[MyPackage.DTE.ActiveWindow]; WE.WindowActivated += new _dispWindowEvents_WindowActivatedEventHandler( MyPackage.WE_WindowActivated); IDE commands events The actual handling of environment's commands and their extension through the automation model is covered in a separate article of this series. In this section we will examine the handling of the events related to these commands (and not of the execution of the commands themselves). Assigning the handlers to these events is possible through the Events.CommandEvents interface. The CommandEvents property, as in the case of MDI windows events, also permits the assignment of a handler either for all of the commands or for a single one through the indexer. Let's examine the assignment of a handler for the event of a command execution being complete (i.e. when the command finishes its execution): CommandEvents CEvents = DTE.Events.CommandEvents; CEvents.AfterExecute += new _dispCommandEvents_AfterExecuteEventHandler(C_AfterExecute); But in order to assign such a handler for an individual command, it is necessary to identify this command in the first place. Each command of the environment is identified by a pair of GUID:ID, and in case of a user-created commands these values are specified directly by the developer during their integration, for example through the VSCT table. Visual Studio possesses a special debug mode which allows identifying any of the environment's commands. To activate this mode, it is required that the following key is to be added to the system registry (an example for Visual Studio 2010): [HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\10.0\General] "EnableVSIPLogging"=dword:00000001 Now, after restarting the IDE, hovering your mouse over menu or toolbar elements with CTRL+SHIFT being simultaneously pressed (though sometime it will not work until you left-click it) will display a dialog window containing all of the command's internal identifiers. We are interested in the values of Guid and CmdID. Let's examine the handling of events for the File.NewFile command: CommandEvents CEvents = DTE.Events.CommandEvents[ "{5EFC7975-14BC-11CF-9B2B-00AA00573819}", 221]; CEvents.AfterExecute += new _dispCommandEvents_AfterExecuteEventHandler(C_AfterExecute); The handler obtained in this way will receive control only after the command execution is finished. void C_AfterExecute(string Guid, int ID, object CustomIn, object CustomOut) { ... } This handler should not be confused with an immediate handler for the execution of the command itself which could be assigned during this command's initialization (from an extension package and in case the command is user-created). Handling the IDE commands is described in a separate article that is entirely devoted to IDE commands. In conclusion to this section it should be mentioned that in the process of developing our own VSPackage extension, we've encountered the necessity to store the references to interface objects containing our handler delegates (such as CommandEvents, WindowEvents etc.) on the top-level fields of our main Package subclass. The reason for this is that in case of the handler being assigned through a function-level local variable, it is lost immediately after leaving the method. Such behavior could probably be attributed to the .NET garbage collector, although we've obtained these references from the DTE interface which definitely exists during the entire lifetime of our extension package. Handling project and solution events (for VSPackage extensions) Let's examine some of the interfaces from the Microsoft.VisualStudio.Shell.Interop namespace, the ones that permit us to handle the events related to Visual Studio projects and solution to be more precise. Although these interfaces are not a part of EnvDTE automation model, they could be implemented by the main class of VSPackage extension (that is the class that was inherited from Package base class of Managed Package Framework). That is why, if you are developing the extension of this type, these interfaces a conveniently supplement the basic set of interfaces provided by the DTE object. By the way, this is another argument for creating a full-fledged VSPackage plugin using MPF. The IVsSolutionEvents could be implemented by the class inherited from Package and it is available starting from Visual Studio version 2005, and the isolated\integrated shells based applications. This interface permits you to track the loading, unloading, opening and closing of projects or even the whole solutions in the development environment by implementing such of its methods as OnAfterCloseSolution, OnBeforeCloseProject, OnQueryCloseSolution. For example: public int OnAfterLoadProject(IVsHierarchy pStubHierarchy, IVsHierarchy pRealHierarchy) { //your custom handler code return VSConstants.S_OK; } As you can see, this method takes the IVsHierarchy object as an input parameter which represents the loading project. Managing of such objects will be examined in another article devoted to the interaction with Visual Studio project model. The IVsSolutionLoadEvents interface, in a similar fashion to the interface described above, should be implemented by the Package subclass and is available to versions of Visual Studio starting from 2010 and above. This interface allows you to handle such interesting aspects as batch loading of project groups and background solution loadings (the OnBeforeLoadProjectBatch and OnBeforeBackgroundSolutionLoadBegins methods), and also to intercept the end of this background loading operation as well (the OnAfterBackgroundSolutionLoadComplete method). Such event handlers should come in handy in case your plug-in needs to execute some code immediately after its initialization, and, at the same time, the plug-in depends on projects\solutions that are loaded inside the IDE. In this a case, executing such a code without waiting for the solution loading to be finished could lead to either incorrect (incomplete) results because of the incompletely formed projects tree, or even to runtime exceptions. While developing PVS-Studio IDE plug-in, we've encountered another interesting aspect of VSPackage plug-in initialization. Then one Package plug-in enters a waiting state (for instance, by displaying a dialog window to the user), further initialization of VSPackage extensions is suspended until the blocking plug-in returns. So, when handling loading and initialization inside the environment, one should always remember this possible scenario as well. And finally, I want to return one final time to the fact, that for the interface methods described above to operate correctly, you should inherit your main class from theses interfaces: class MyPackage: Package, IVsSolutionLoadEvents, IVsSolutionEvents { //Implementation of Package, IVsSolutionLoadEvents, IVsSolutionEvents ... } Supporting Visual Studio color schemes If the extension you are developing will be integrated into the interface of the development environment, for instance, by creating custom tool windows or document MDI windows (and the most convenient way for such an integration is a VSPackage extesnion), it is advisable that the coloring of your custom UI components should match the common color scheme used by Visual Studio itself. The importance of this task was elevated with the release of Visual Studio 2012, containing two hugely opposite color themes (Dark and Light) which the user could switch "on the fly" from the IDE options window. The GetVSSysColorEx method from Visual Studio Interop interface IVsUIShell2 could be utilized to obtain environment's color settings. This interface is available to VSPackage plugins only. IVsUIShell2 vsshell = this.GetService(typeof(SVsUIShell)) as IVsUIShell2; By passing the the __VSSYSCOLOREX and __VSSYSCOLOREX3 enums to the GetVSSysColorEx method, you can get the currently selected color for any of Visual Studio UI elements. For example, let's obtain one of the colors from the context menu's background gradient: uint Win32Color; vsshell.GetVSSysColorEx((int)__VSSYSCOLOREX3.VSCOLOR_COMMANDBAR_MENU_BACKGROUND_GRADIENTBEGIN, out Win32Color); Color BackgroundGradient1 = ColorTranslator.FromWin32((int)Win32Color); Now we can use this Color object to "paint" our custom context menus. To determine the point in time at which the color theme of your components should be reapplied, you can, for example, utilize events of the environment command responsible for opening of IDE's settings window (Tools -> Options). How to subscribe your handlers to such an event was described earlier in this article. But if you are, for some reason, unable to utilize the IVsUIShell2 object (for instance, in case you are developing a non-VSPackage extension), but at the same time you still need to support Visual Studio color themes, then it is possible to obtain color values for environment's various UI components directly from the system registry. We will not cover this approach in the article, but here you can download a free and open-source tool designed for Visual Studio color theme editing. The tool is written in C# and it contains all the code required for reading and modifying Visual Studio 2012 color themes from the managed code. Interacting with COM interfaces from within a multithreaded application Initially PVS-Studio extension package had not contained any specific thread-safety mechanisms for its interaction with Visual Studio APIs. At the same time, we had been attempting to confine the interactions with this APIs within a single background thread which was created and owned by our plug-in. And such approach functioned flawlessly for quite a long period. However, several bug reports from our users, each one containing a similar ComExeption error, prompted us to examine this issue in more detail and to implement a threading safety mechanism for our COM Interop. Although Visual Studio automation model is not a thread-safe one, it still provides a way for interacting with multi-threaded applications. Visual Studio application is a COM (Component Object Mode) server. For the task of handling calls from COM clients (in our case, this will be our extension package) to thread-unsafe servers, COM provides a mechanism known as STA (Single-Threaded Apartment) model. In the terms of COM, an Apartment represents a logical container inside a process in which objects and threads share the same thread access rules. STA can hold only a single thread, but an unlimited number of objects, inside such container. Calls from other threads to such thread-unsafe objects inside STA are converted into messages and posted to a message queue. Messages are retrieved from the message queue and converted back into method calls one at a time by the thread running in the STA, so it becomes possible for only a single thread to access these unsafe objects on the server. Utilizing Apartment mechanism inside managed code The .NET Framework does not utilize COM Apartment mechanics directly. Therefore, when a managed application calls a COM object in the COM interoperation scenarios, CLR (Common Language Runtime) creates and initializes Apartment container. A managed thread is able to create and enter either an MTA (Multi-Threaded Apartment, a container that, contrary to STA, can host several threads at the same time), or an STA, though a thread will be started as an MTA by default. The type of the Apartment could be specified before thread is launched: Thread t = new Thread(ThreadProc); t.SetApartmentState(ApartmentState.STA); ... t.Start(); As an Apartment type could not be changed once thread had been started, the STAThread attribute should be used to specify the main thread of a managed application as an STA: [STAThread] static void Main(string[] args) {...} Implementing message filter for COM interoperation errors in a managed environment As STA serializes all of calls to the COM server, one of the calling clients could potentially be blocked or even rejected when the server is busy, processing different calls or another thread is already inside the apartment container. In case COM server rejects its client, .NET COM interop will generate a System.Runtime.InteropServices.COMException ("The message filter indicated that the application is busy"). When working on a Visual Studio module (add-in, vspackage) or a macro, the execution control usually passes into the module from the environment's main STA UI thread (such as in case of handling events or environment state changes, etc.). Calling automation COM interfaces from this main IDE thread is safe. But if other background threads are planned to be utilized and EnvDTE COM interfaces are to be called from these background threads (as in case of long calculations that could potentially hang the IDE's interface, if these are performed on the main UI thread), then it is advised to implement a mechanism for handling calls rejected by a server. While working on PVS-Studio plug-in we've often encountered these kinds of COM exceptions in situations when other third-party extensions were active inside the IDE simultaneously with PVS-Studio plug-in. Heavy user interaction with the UI also was the usual cause for such issues. It is quite logical that these situations often resulted in simultaneous parallel calls to COM objects inside STA and consequently to the rejection of some of them. To selectively handle incoming and outgoing calls, COM provides the IMessageFilter interface. If it is implemented by the server, all of the calls are passed to the HandleIncomingCall method, and the client is informed on the rejected calls through the RetryRejectedCall method. This in turn allows the rejected calls to be repeated, or at least to correctly present this rejection to a user (for example, by displaying a dialog with a 'server is busy' message). Following is the example of implementing the rejected call handling for a managed application. [ComImport()] [Guid("00000016-0000-0000-C000-000000000046")] [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)] public interface IMessageFilter { [PreserveSig] int HandleInComingCall( int dwCallType, IntPtr hTaskCaller, int dwTickCount, IntPtr lpInterfaceInfo); [PreserveSig] int RetryRejectedCall( IntPtr hTaskCallee, int dwTickCount, int dwRejectType); [PreserveSig] int MessagePending( IntPtr hTaskCallee, int dwTickCount, int dwPendingType); } class MessageFilter : MarshalByRefObject, IDisposable, IMessageFilter { [DllImport("ole32.dll")] [PreserveSig] private static extern int CoRegisterMessageFilter( IMessageFilter lpMessageFilter, out IMessageFilter lplpMessageFilter); private IMessageFilter oldFilter; private const int SERVERCALL_ISHANDLED = 0; private const int PENDINGMSG_WAITNOPROCESS = 2; private const int SERVERCALL_RETRYLATER = 2; public MessageFilter() { //Starting IMessageFilter for COM objects int hr = MessageFilter.CoRegisterMessageFilter( (IMessageFilter)this, out this.oldFilter); System.Diagnostics.Debug.Assert(hr >= 0, "Registering COM IMessageFilter failed!"); } public void Dispose() { //disabling IMessageFilter IMessageFilter dummy; int hr = MessageFilter.CoRegisterMessageFilter(this.oldFilter, out dummy); System.Diagnostics.Debug.Assert(hr >= 0, "De-Registering COM IMessageFilter failed!") System.GC.SuppressFinalize(this); } int IMessageFilter.HandleInComingCall(int dwCallType, IntPtr threadIdCaller, int dwTickCount, IntPtr lpInterfaceInfo) { // Return the ole default (don't let the call through). return MessageFilter.SERVERCALL_ISHANDLED; } int IMessageFilter.RetryRejectedCall(IntPtr threadIDCallee, int dwTickCount, int dwRejectType) { if (dwRejectType == MessageFilter.SERVERCALL_RETRYLATER) { // Retry the thread call immediately if return >=0 & // File.NewFile Mytext /t:"General\Text File" /e:"Source Code (text) Editor" A command's syntax generally complies with the following rules:command's name and arguments are separated by a spacearguments containing spaces are wrapped by double quotes The caret (^) is used as an escape characterOne-character abridgments for command names can be combined, as for example, /case(/c) and /word(/w) could be presented as /cwWhen using the 'command' command-line switch, name of a command with all of its arguments should be wrapped by double quotes: devenv.exe /command "MyGroup.MyCommandName arg1 arg2" For the sake of convenience, a command could be associated with an alias: >alias MyAlias File.NewFile MyFile Commands integrated into IDE by PVS-Studio extension can be utilized through the /command switch as well. For example, this mode could be used for the integration of our static analysis into the automated build process. Our analyzer itself (PVS-Studio.exe) is a native command-line application, which operates quite similar to the compiler, i.e. it takes a path to the file containing source code and its compilation arguments and then it outputs analysis results to stdout/stderr streams. It's quite obvious that the analyzer could easily be integrated directly into the build system (for instance, into a system which is based on MSBuild, NMake or even GNU Make) at the same level where C/C++ compiler is being called. Of course, such integration already provides us, by its own definition, with complete enumeration of all of the source files being built, with all of their compilation parameters. In turn, this allows for a substitution (or supplementation) of a compiler call by call to the analyzer. Although the described scenario is fully supported by PVS-Studio.exe analyzer, it still requires a complete understanding of build system's internals as well as an opportunity to modify a system in the first place, which could be problematic or even impossible at times. Therefore, the integration of the analyzer into the build process can be performed in a more convenient way, on a higher level (i.e. at the level of Continuous Integration Server), by utilizing Visual Studio extension commands through the /command switch, for example, by using the PVS-Studio.CheckSolution command to perform analysis on MSVS solution. Of course, such use case is only possible when building Visual C++ native project types (vcproj/vcxproj). In case Visual Studio is started from a command line, the /command switch will be executed immediately after the environment is fully loaded. In this case, the IDE will be started as a regular GUI application, without redirecting its standard I/O streams to the console that was used to launch the environment. It should be noted that, in general, Visual Studio is a UI based development environment and so it is not intended for command line operations. It is recommended to employ Microsoft MSBuild utility for building inside build automation systems, as this tool supports all of native Visual Studio project types. Caution should be applied when using Visual Studio /command switch together with non-interactive desktop mode (for example when calling IDE from a Windows service). We've encountered several interesting issues ourselves when we were evaluating the possibility of integrating PVS-Studio static analysis into Microsoft Team Foundation build process, as Team Foundation operates as a Windows service by default. At that moment, our plug-in had not been tested for non-interactive desktop sessions and was incorrectly handling its child windows and dialogs, which in turn led to exceptions and crashes. But Visual Studio itself experienced none of such issues, almost none to be more precise. The case is, Visual Studio displays a particular dialog for every user when it is started for a first time after an installation, and this dialog offers the user to select a default UI configuration. And it was this dialog that Visual Studio displayed for a LocalSystem account, the account which actually owns the Team Foundation service. It turns out that the same dialog is 'displayed' even in the non-interactive desktop mode, and it subsequently blocks the execution of the /command switch. As this user doesn't have an interactive desktop, he is also unable to close this dialog normally by manually starting the IDE himself. But, in the end, we were able to close the dialog manually by launching Visual Studio for LocalSystem account in the interactive mode through psexec tool from PSTools utilities. Creating and handling commands in VSPackage. Vsct files. VSPackage extension utilizes Visual Studio command table (*.vsct) file for creating and managing commands that it integrates into the IDE. Command tables are text files in XML format which can be compiled by VSCT compiler into binary CTO files (Command Table Output). CTO files are then included as a resources into final builds of IDE extension packages. With the help of VSCT, commands can be associated with menus or toolbar buttons. Support for VSCT is available starting from Visual Studio 2005. Earlier IDE versions utilized CTC (Command Table Compiler) files handling their commands, but they will not be covered in this article. In a VSCT file each command is assigned a unique ID -- CommandID, a name, a group and a quick access hotkey combination, while its representation in the interface (if any) is specified by special flags. Let's examine a basic structure of VSCT file. The root element of file is 'CommandTable' node that contains the 'Commands' sub-node, which defines all of the user's commands, groups, menu items, toolbars etc. Value of the "Package" attribute of the "Commands" node must correspond with the ID of your extension. The "Symbols" sub-node should contain definitions for all identifiers used throughout this VSCT file. The 'KeyBindings' sub-node contains default quick access hotkey combinations for the commands. ... ... ... ... The 'Buttons' node defines the commands themselves by specifying their UI representation style and binding them to various command groups. Pict TextOnly IconAndText DefaultDisabled My Command 1 The 'Menus' node defines the structure of UI elements (such as menus and toolbars), also binding them to command groups in the 'Groups' node. A group of commands bound with a 'Menu' element will be displayed by the UI as a menu or a toolbar. Sub Menu 1 And finally, the 'Groups' element organizes user's IDE command groups. To include vsct file into MSBuild-based VSPackage project, it is necessary to insert the following node used for calling VSCT compiler into your csproj project file (note, that in the auto-generated project created from an SDK template, a vsct file will be already included in a project): Menus.ctmenu In order to integrate a user-defined command or command group to one of the standard Visual Studio command groups, it is necessary to specify for your group the identifier of such standard group in the parent node. For example, top integrate your commands to a context menu of a project in the solution explorer window: As you can see, the standard IDM_VS_CTXT_PROJNODE is being used here. Following this link, you can discover a list of standard IDs for Visual Studio command groups. Next, the ProvideMenuResource attribute of your Package-derived class should point to this node that you've inserted into your project earlier: [ProvideMenuResource("Menus.ctmenu", 1)] ... public sealed class MyPackage : Package Assigning handlers to the commands defined in a VSCT file is possible through a service that is available through the IMenuCommandService. A reference for it can be obtained by the GetService method of your Package subclass: OleMenuCommandService MCS = GetService(typeof(IMenuCommandService)) as OleMenuCommandService; Let's examine an example in which we assign a handler to a menu command (this command should be declared in a vsct file beforehand): EventHandler eh = new EventHandler(CMDHandler); CommandID menuCommandID = new CommandID(guidCommand1CmdSet, id); //ID and GUID should be the same as in the VCST file OleMenuCommand menuItem = new OleMenuCommand(eh, menuCommandID); menuItem.ParametersDescription = "$"; MCS.AddCommand(menuItem); To obtain command's arguments while handling its invocation, the EventArgs object should be casted into OleMenuCmdEventArgs: void CMDHandler(object sender, EventArgs e) { OleMenuCmdEventArgs eventArgs = (OleMenuCmdEventArgs)e; if (eventArgs.InValue != null) param = eventArgs.InValue.ToString(); ... } Handling commands through EnvDTE.DTE interfaces. The EnvDTE.DTE automation object allows for a direct manipulation (creation, modification and execution) of commands through the dte.Commands interface and dte.ExecuteCommand method. Utilizing the Automation Object Model for invoking, modifying and creating IDE commands, as opposed to using VSCT mechanism available only for VSPackage, allows the interaction with IDE commands from within Add-In extension packages as well. The DTE automation object allows a direct creation, modification and invocation of commands through the DTE.Commands interface. A command can be directly added to the IDE by Commands.AddNamedCommand method (but only for an Add-In extension): dte.Commands.AddNamedCommand(add_in, "MyCommand", "My Command", "My Tooltip", true); The command added in this way will be preserved by the IDE -- it will reappear in the menu after IDE restart, even if the extension which created the command is not loaded itself. That's why this method should only be utilized during the first initialization of an Add-In module, after its installation (this is described in the section dedicated to Visual Studio Automation Object Model). The OnConnection method of an Add-In contains a special initialization mode which is invoked only for a single time in the module's entire lifetime. This method can be used to integrate UI elements into the IDE: public void OnConnection(object application, ext_ConnectMode connectMode, object addInInst, ref Array custom) { switch(connectMode) { case ext_ConnectMode.ext_cm_UISetup: ... break; ... } } The EnvDTE.Command interface represents a single IDE command. This interface can be used to modify a command which it references. It permits managing IDE commands from either a VSPackage, or an Add-In module. Let's obtain a reference to the EnvDTE.Command object for our custom command MyCommand1 and utilize this interface to assign a 'hot-key' to it for a quick access: EnvDTE.Command MyCommand1 = MyPackage.DTE.Commands.Item("MyGroup.MyCommand1", -1); MyCommand1.Bindings = new object[1] { "Global::Alt+1" }; The quick-access combination assigned to MyGroup.MyCommand1 will now be available through 'Keyboard, Environment' environment settings dialog. As was mentioned before, Visual Studio command is not a UI element by itself. The Commands.AddCommandBar method allows the creation of such UI elements, as main menu items, toolbars, context menus and the association of these elements with user-created commands. CommandBar MyToolbar = dte.Commands.AddCommandBar("MyToolbar1", vsCommandBarType.vsCommandBarTypeToolbar) as CommandBar; CommandBar MyMenu = dte.Commands.AddCommandBar("MyMenu1", vsCommandBarType.vsCommandBarTypeMenu) as CommandBar; CommandBarButton MyButton1 = MyCommand1.AddControl(MyToolbar) as CommandBarButton; MyButton1.Caption = "My Command 1"; The Delete method of Command/ CommandBar objects could be utilized to remove a command or toolbar from IDE. MyCommand1.Delete(); In general, it is not recommended creating commands each time an Add-In plug-in is loaded and removing them each time it is un-loaded, as such behavior could slow down the initialization of IDE itself. Even more, in case the OnDisconnect method is somehow interrupted in the process, it is possible that the user commands will not be completely deleted from the IDE. That is why it is advised that the integration, and subsequent removal, of IDE commands should be handled at the times of module's installation/uninstallation, as for example, by obtaining DTE interface reference from a stand-alone installer application. The initialization of Add-In modules and acquisition of DTE references is thoroughly described in the article devoted to EnvDTE Automation Object Model. Any IDE command (either custom or default one) could be called by the ExecuteCommand method. Here is the example of invoking our custom MyCommand1 command: MyPackage.DTE.ExecuteCommand("MyGroup.MyCommand1", args); To handle command execution, an Add-In extension should be derived from the IDTCommandTarget interface and it should also implement the Exec method: public void Exec(string commandName, vsCommandExecOption executeOption, ref object varIn, ref object varOut, ref bool handled) { handled = false; if(executeOption == vsCommandExecOption.vsCommandExecOptionDoDefault) { if(commandName == "MyAddin1.Connect.MyCommand1") { ... handled = true; return; } } } References MSDN. Visual Studio Commands and Switches.MSDN. Visual Studio Command Table (.Vsct) Files.MSDN. Designing XML Command Table (.Vsct) Files.MSDN. Walkthrough: Adding a Toolbar to the IDE.MSDN. How VSPackages Add User Interface Elements to the IDE.MZ-Tools. HOWTO: Adding buttons, commandbars and toolbars to Visual Studio .NET from an add-in.MSDN. How to: Create Toolbars for Tool Windows. Visual Studio tool windows This item covers the extension of Visual Studio IDE through integration of a custom user tool window into the environment. Discussed are the issues of window registration and initialization in VSPackage and Add-In plug-in modules, hosting of user components and handling of window's events and states. Introduction Tool windows are child windows of Visual Studio MDI (Multiple Document Interface) and they are responsible for presenting various pieces of information to the user. Solution Explorer and Error List are the examples of tool windows. Usually tool windows' contents are not associated with any files and do not contain any editors, as separate document windows are reserved for such tasks. For instance, PVS-Studio extension package integrates several tool windows into the IDE, with Output Window being the primary one. All other of its tool windows can be opened from this main window, as, for example, a search window for the grid. PVS-Studio Output Window itself can be opened from Visual Studio main menu (PVS-Studio -> Show PVS-Studio Output Window), but it also will be invoked automatically each time the analysis starts. In most cases IDE creates and utilizes just a single instance for each one of its tool windows, and this instance will be preserved until IDE itself needs to shut down. Therefore, pressing the 'close' button on a tool window does actually hide it, and when this window is invoked for the second time, it becomes visible again, thus preserving any data that it contained before being 'closed'. But still, is it possible to crate Multi-Instance tool windows in the IDE, which are the windows that can exist in several instances at once. A tool window can also be associated with a certain UI context (as the so called dynamic window), and such window will be automatically displayed when the user enters this context. Integration of a tool window into the IDE is supported by VSPackage and Add-In extensions (although the methods for it are different); it requires the specification of the window's initial settings and its registration in the system registry. Registering and initializing user tool windows A VSPackage project template that is installed together with Visual Studio SDK allows you to create a sample tool window in the extension project which this template generates. Such a project should already contain all of the basic components which will be described below, so it could be conveniently used as a sample for experimenting with Visual Studio tool window integration process for VSPackage plug-ins. Registering, initializing and invoking a tool window in VSPackage Registering a custom user window in the environment requires writing of the data that defines this window into a special section of Visual Studio registry hive. This process can be automated by generating a pkgdef file that can contain all of the required window registration information. The contents of this pkgdef file can be specified through special registration attributes of your Package subclass. The immediate registration of a user-created tool window into VSPackage extension is handled by ProvideToolWindowattribute of Package subclass: [ProvideToolWindow(typeof(MyWindowPane), Orientation = ToolWindowOrientation.Right, Style = VsDockStyle.Tabbed, Window = Microsoft.VisualStudio.Shell.Interop.ToolWindowGuids.Outputwindow, MultiInstances = false, Transient = true, Width = 500, Height = 250, PositionX = 300, PositionY = 300)] Let's examine several parameters of this attribute. The Typeof parameter points to user implementation of the window's client area (a subclass of ToolWindowPane). The MultiInstances parameter enables the Multi-Instance mode for a window, in which multiple instances of the window can be opened simultaneously. The Orientation, Size and Style parameters specify the initial position of a window when it is opened for the first time by the user. It should be noted that the position specified by these parameters will only be used once, when a tool window is displayed for the first time; at all of the subsequent iterations of opening this window, the IDE will be restoring its screen position from the previous one, that is the position before a window was closed. The Transient parameter indicates whether the window will be automatically opened after Visual Studio environment is loaded in case it already have been opened during the previous session of the IDE. It should also be remembered that the initialization of a user window by VSPackage (the initialization itself will be covered later) does not necessarily occur at the same moment as the initialization of a Package subclass for which we provided this registration attribute. For example, after implementing a tool window for PVS-Studio plug-in, we've encountered an issue in which our custom window was automatically opened (but not focused/displayed) and placed among other window tabs at the bottom of the main window, and it was done immediately after Visual Studio started up, even though we've passed the Transient=true parameter to the ProvideToolWindow attribute. Although the plug-in itself is always initialized at IDE start-up, the window had not been fully initialized until after a first call to it, which was evident by the corrupted icon on aforementioned tab. A dynamic visibility context can be specified for a window by the ProvideToolWindowVisibilityattribute: [ProvideToolWindowVisibility(typeof(MyWindowPane), /*UICONTEXT_SolutionExists*/"f1536ef8-92ec-443c-9ed7-fdadf150da82")] In this example, the window is set to be automatically displayed when the user enters the "Solution Exists" UI context. Take a note that each one of user's tool window requires a separate attribute and a window's type should be passed as a first argument to it. The FindToolWindow method of a Package subclass can be utilized to create and display a tool window from a VSPackage extension. This method returns a reference to the specified tool window object, creating it if necessary (for instance, in case a single-instance window is called for a first time). Following is the example of invoking a single-instance tool window: private void ShowMyWindow(object sender, EventArgs e) { ToolWindowPane MyWindow = this.FindToolWindow(typeof(MyToolWindow), 0, true); if ((null == MyWindow) || (null == MyWindow.Frame)) { throw new NotSupportedException(Resources.CanNotCreateWindow); } IVsWindowFrame windowFrame = (IVsWindowFrame) MyWindow.Frame; ErrorHandler.ThrowOnFailure(windowFrame.Show()); } In this example, the window will be created in case it is called for the first time, or the window will be made visible in case it had been created before and then hidden. The FindToolWindow's third argument of the bool type specifies whether a new instance of a window should be created if the method was unable to find an already existing one. To create a Multi-Instance tool window, the CreateToolWindow method can be used. It allows the creation of a window with a pre-defined identifier. An example of invoking such window: private void CreateMyWindow(object sender, EventArgs e) { for (int i = 0; ; i++) { // Find existing windows. var currentWindow = this.FindToolWindow(typeof(MyToolWindow), i, false); if (currentWindow == null) { // Create the window with the first free ID. var window = (ToolWindowPane)this.CreateToolWindow(typeof(MyToolWindow), i); if ((null == window) || (null == window.Frame)) { throw new NotSupportedException(Resources.CanNotCreateWindow); } IVsWindowFrame windowFrame = (IVsWindowFrame)window.Frame; ErrorHandler.ThrowOnFailure(windowFrame.Show()); break; } } } Note that in this example the FindToolWindow method receives 'false' value as its third argument, i.e. we are searching for an unoccupied index before initializing a new window instance. As was mentioned above, the environment will preserve position of a window after it is closed. But if, for whatever reason, it is necessary to specify the size and position of a window, it could be achieved through the SetFramePos method of the IVsWindowFrame interface: Guid gd = Guid.Empty; windowFrame.SetFramePos(VSSETFRAMEPOS.SFP_fDockBottom, ref gd, 20, 20, 200, 200); A call to the SetFramePos() should always be made only after the Show() method is executed. Creating and invoking a window from Add-In extension A user tool window can be initialized from an Add-In extension with the help of the EnvDTE Windows2 interface: public void OnConnection(object application, ext_ConnectMode connectMode, object addInInst, ref Array custom) { _applicationObject = (DTE2)application; _addInInstance = (AddIn)addInInst; EnvDTE80.Windows2 window; AddIn add_in; object ctlobj = null; Window myWindow; // Get the window object add_in = _applicationObject.AddIns.Item(1); window = (Windows2)_applicationObject.Windows; // This section specifies the path and class name of the windows // control that you want to host in the new tool window, as well as // its caption and a unique GUID. string assemblypath = "C:\\MyToolwindow\\MyToolWindowControl.dll"; string classname = " MyToolWindowControl.MyUserControl"; string guidpos = "{E87F0FC8-5330-442C-AF56-4F42B5F1AD11}"; string caption = "My Window"; // Creates the new tool window and inserts the user control into it. myWindow = window.CreateToolWindow2(add_in, assemblypath, classname, caption, guidpos, ref ctlobj); myWindow.Visible = true; } In the example above, a user tool window was created using the MyToolWindowControl.MyUserControl as a client area control. The MyToolWindowControl.MyUserControl class could either be located in the same assembly as the add-in that initializes it, or it could be provided by a stand-alone assembly with a full COM visibility (though the 'Register for COM Interop' option in project settings). The regular composite UserControl subclass could be utilized as MyUserControl. Implementing a user toolwindow in a VSPackage module Tool window consists of a frame border and a client area. A frame is provided by the environment and is responsible for performing docking with other interface objects of the environment, as well as for size and position of the window itself. A client area is a pane, controlled by a user, which houses the contents of a window. Tool windows can host user-created WinForms and WPF components and are capable of handling regular events, such as OnShow, OnMove, etc. A user tool window, or its client area to be more precise, can be implemented by inheriting the class representing a standard empty IDE window -- ToolWindowPane [Guid("870ab1d8-b434-4e86-a479-e49b3c6797f0")] public class MyToolWindow : ToolWindowPane { public MyToolWindow():base(null) { this.Caption = Resources.ToolWindowTitle; this.BitmapResourceID = 301; this.BitmapIndex = 1; ... } } The Guid attribute is used to uniquely identify each custom user window. In case a plug-in module creates several windows of different types, each one of them should be identified by its own unique Guid. A ToolWIndowPane subclass can be subsequently modified and host user-controlled components. Hosting user components A base ToolWindowPane class implements an empty tool window of the environment. Inheriting from this class allows hosting user-created WinForms or WPF components. Up until Visual Studio 2008 version, tool windows only provided a native support for WinForms user components, although it still was possible to host WPF components through the WPF Interoperability ElementHost object. Starting from Visual Studio 2010, tool windows themselves are based on WPF technology, although they still provide a backward compatibility for hosting of WinForms components. To host a user-created WinForms component inside a user tool window, the Window property of the ToolWindowPane base class should be overridden: public MyUserControl control; public MyToolWindow():base(null) { this.Caption = Resources.ToolWindowTitle; this.BitmapResourceID = 301; this.BitmapIndex = 1; this.control = new MyUserControl(); } public override IWin32Window Window { get { return (IWin32Window)control; } } In the example above, the MyUserControl object is a regular composite component of the System.Windows.Forms.UserControl type and it can host any other user component inside itself. UserControl can also host WPF components by using WPF ElementHost object. Starting from Visual Studio 2010, WPF components can be hosted by tool windows natively. To do this, a reference to the WPF component should be passed to the Content property of a base class: public MyToolWindow():base(null) { this.Caption = Resources.ToolWindowTitle; this.BitmapResourceID = 301; this.BitmapIndex = 1; base.Content = new MyWPFUserControl(); } Please note that using the two methods described above simultaneously is not possible. When a reference to WPF component is assigned to the base.Content property, an overridden Window property is ignored. The main PVS-Studio 'Output' window of our extension plug-in hosts a virtual grid based on SourceGrid open-source project. This window provides an interface for handling the results of static analysis. The grid itself is bound to a regular ADO.NET table of the System.Data.Datatable type, which is utilized for storing analysis results. Until 4.00 version of PVS-Studio extension, it utilized a regular IDE 'Error List' window, but as the analyzer evolved, the capabilities of this default window became insufficient. Apart from being un-extendable with such specific static analysis UI elements as, for example, false positive suppression and filtering mechanisms, the Error List is itself basically a 'real' grid, as it stores all of the displayed elements inside itself. Therefore, this grid only permits an adequate handling of 1-2k messages at a time, performance wise, as a greater number of messages already can cause quite a noticeable lag to the environment's UI. On the other hand, our own practice of using static analysis on relatively large projects, such as Chromium or LLVM, demonstrated that a total number of diagnostic messages (taking into account all of the marked false alarms and low-lever user diagnostics as well) could easily reach tens of thousands or even more. Therefore, by implementing a custom output window, based on virtual grid that is connected to a DB table, PVS-Studio is able to display and provide convenient handling for hundreds of thousands of diagnostic messages at once. Also, the ability for a convenient and flexible filtering of the analysis results is quite an important aspect of handling a static analyzer, as the manual examination even of only such a "tiny" amount of messages as 1-2k is nearly impossible for a single user. The storage of analysis results in a Datatable object by itself provides quite a convenient filtering mechanism based on a simple SQL queries, even more so because the results of such queries become visible immediately inside the bound virtual grid. Handling tool windows events A client area of a tool window (represented by our ToolWindowPane subclass) can process the regular events of user-interface interactions. The IVsWindowFrameNotify3 interface can be used for subscribing to window events. Let's provide an example of implementing this interface: public sealed class WindowStatus: IVsWindowFrameNotify3 { // Private fields to keep track of the last known state private int x = 0; private int y = 0; private int width = 0; private int height = 0; private bool dockable = false; #region Public properties // Return the current horizontal position of the window public int X { get { return x; } } // Return the current vertical position of the window public int Y { get { return y; } } // Return the current width of the window public int Width { get { return width; } } // Return the current height of the window public int Height { get { return height; } } // Is the window dockable public bool IsDockable { get { return dockable; } } #endregion public WindowStatus() {} #region IVsWindowFrameNotify3 Members // This is called when the window is being closed public int OnClose(ref uint pgrfSaveOptions) { return Microsoft.VisualStudio.VSConstants.S_OK; } // This is called when a window "dock state" changes. public int OnDockableChange(int fDockable, int x, int y, int w, int h) { this.x = x; this.y = y; this.width = w; this.height = h; this.dockable = (fDockable != 0); return Microsoft.VisualStudio.VSConstants.S_OK; } // This is called when the window is moved public int OnMove(int x, int y, int w, int h) { this.x = x; this.y = y; this.width = w; this.height = h; return Microsoft.VisualStudio.VSConstants.S_OK; } // This is called when the window is shown or hidden public int OnShow(int fShow) { return Microsoft.VisualStudio.VSConstants.S_OK; } /// This is called when the window is resized public int OnSize(int x, int y, int w, int h) { this.x = x; this.y = y; this.width = w; this.height = h; return Microsoft.VisualStudio.VSConstants.S_OK; } #endregion } As evident by this sample code above, the WindowsStatus class implementing the interface is able to process such window state changes, as the alterations in window's size, position, visibility properties and so on. Now, let's subscribe our window for handling these events. It requires the OnToolWindowCreated method to be overridden in our ToolWindowPane subclass: public class MyToolWindow: ToolWindowPane { public override void OnToolWindowCreated() { base.OnToolWindowCreated(); // Register to the window events WindowStatus windowFrameEventsHandler = new WindowStatus(); ErrorHandler.ThrowOnFailure( ((IVsWindowFrame)this.Frame).SetProperty( (int)__VSFPROPID.VSFPROPID_ViewHelper, (IVsWindowFrameNotify3)windowFrameEventsHandler)); } ... } Controlling window state A window state can be controlled through event handlers of our IVsWindowFrameNotify3 implementation. The OnShow method notifies the extension package about changes in tool window's visibility state, allowing to track the appearance of the window to a user, when, for example, user switches windows by clicking on window tabs. Current visibility state could be obtained by the fShow parameter, which corresponds to the __FRAMESHOW list. The OnClose method notifies about the closure of a window frame, allowing to define IDE behavior in case of this event with the pgrfSaveOptions parameter, which controls the default document saving dialog (__FRAMECLOSE). The OnDockableChange method informs the package on window's docking status changes. The fDockable parameter indicates whether a window is docked to another one; other parameters control window's size and position before and after the docking event. The parameters of OnMove and OnSize methods provide window's coordinates and size while it is being dragged or resized. References MSDN. Kinds of Windows.MSDN. Tool Windows.MSDN. Tool Window Essentials.MSDN. Tool Window Walkthroughs.MSDN. Arranging and Using Windows in Visual Studio.MZ-Tools. HOWTO: Understanding toolwindow states in Visual Studio. Integrating into Visual Studio settings This item covers the extension of Visual Studio by integrating into its 'Settings' dialog pages. Option page registration and integration into the IDE for different kinds of extension packages will be examined, as well as the means to display various standard and user-created components inside a custom settings page. Also covered are the ways of accessing environment settings through Visual Studio Automation model and preservation mechanism for option pages. Introduction Visual Studio employs a single unified dialog window to provide an access to the settings of its various components. This window is available through the IDE Tools -> Options main menu item. A basic element of Visual Studio settings is an Options Page. The Options dialog window arranges its pages in a tree-like structure according to the membership of the pages in their respective functional groups. Each one of these pages could be uniquely identified by the name of its group and its own name. For example, Visual Basic source code editor settings page is "Text Editor, Basic". Extension packages are able to access and modify the values of various settings from option pages registered in the IDE. They can also create and register their own custom options pages in the environment through the automation object model and MPF classes (Managed Package Framework, available only to VSPackage extensions). Visual Studio contains an embedded mechanism for preserving the state of its settings objects; it is enabled by default, but can be overridden or disabled. Creating and registering user options pages It can be useful for a Visual Studio extension plug-in to be associated with one or several custom options pages from the Tools->Options dialog window. Such a tool for configuring an extension will conform to the environment's UI paradigm and is actually quite convenient for handling your extension's settings from within the IDE itself. The methods of implementing and integrating custom user options page into the IDE can vary, as they depend upon the type of the extension being developed and the technology being used (either an automation model or MPF). Integrating settings through an MPF class Managed Package Framework allows creating custom options pages by inheriting from the DialogPage. As the environment loads each of its options pages independently when accessing the corresponding section of the Tools->Options dialog, each page must be implemented with an independent object as a result. The object which implements a custom page should be associated with your VSPackage through the ProvideOptionPage attribute of the corresponding Package subclass. [ProvideOptionPageAttribute(typeof(OptionsPageRegistration), "MyPackage", "MyOptionsPage", 113, 114, true)] This attribute designates the names for the options page itself and for group that it belongs to, as it should be displayed in the IDE options dialog. A separate attribute should be used for every custom page that is to be integrated by the extension. In fact, this attribute is used to provide a registration for the page through pkgdef file and it does not directly affect the execution in any other way. For the user options page to be correctly rendered by the environment, the page should be registered in the following node of the system registry: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\\ ToolsOptionsPages Here is the version number of Visual Studio IDE, 10.0 for example. This record will be automatically created when ProvideOptionPage attribute is utilized. It should be noted that a correct uninstallation of an extension package also requires purging all of the records that this extension had written to the system registry before, including the ones belonging to its options pages. As the versions of Visual Studio IDE starting from 2010 can utilize VSIX packages to deploy/uninstall VSPackage plug-ins, the VSIX installer will automatically perform such registry operations according to its pkgdef file. But earlier versions of IDE may require manual registry cleaning, for instance by a stand-alone installer application. The 6th bool-type argument of the attribute's constructor allows the user's custom options page to be registered as an automation object. This exposes the page to the Automation Object Model, providing an access to its settings through the EnvDTE interfaces for other third-party plug-ins. Registering an automation object requires the creation of several records in the system registry (it is performed automatically when using the aforementioned attributes) in the following nodes: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\ Import/Export Settings, by the default implementation of the SaveSettingsToXml method which also can be overridden if necessary. Of course, integrating a page into Visual Studio settings dialog is not the exclusive or mandatory way of creating configuration interface for an IDE plug-in. If the capabilities of a regular PropertyGrid are insufficient and there are no future plans to utilize the embedded mechanism for settings preservation, then it could be quite reasonable to implement an IDE-independent settings dialog. The advantages of this approach are high portability (for instance, a plug-in that could be used with multiple IDEs) and complete control over the dialog window itself, which in turn substantially alleviates the support of various end-user configurations. On the downside, such solution makes your settings inaccessible to third-party developers through the automation object model. For configuring its settings, PVS-Studio extension package utilizes a custom state preservation mechanism that operates through an external XML file, so that the options pages which the plug-in integrates into the IDE are provided only as means for displaying and modifying these internal settings. Initially, the embedded settings preservation functionality of Visual Studio created conflicts with PVS-Studio's own settings mechanism in the earlier versions of the plug-in, leading to setting de-synchronization issues. This demonstrated to us that even in the presence of an independent settings management inside the extension, it still may be necessary to override some of Visual Studio regular mechanisms (maybe even by an empty method). Integrating settings through an Add-In XML definition A user options page can be integrated into the IDE through an independent XML definition of an Add-In extension. The contents of such a user page should be implemented as a user component, for example as an System.Windows.Forms.UserControl. This component is not associated with an Add-In itself, thus it can be implemented either inside the extension's assembly or as an independent library altogether. An add-in XML file could even be created for such user component alone, without any definitions for an Add-In extension. Let's examine an XML definition for an Add-In module which also contains a definition for a user's custom options page. Microsoft Visual Studio Macros 10.0 Microsoft Visual Studio 10.0 My Add in My Addin 1 c:\MyAddIn1\MyAddin1.dll MyAddin1.Connect 0 1 0 c:\MyAddIn1\MyAddin1.dll MyAddin1.UserControl1 A description for custom options page is located inside the node. The sub-node points to the library which contains a user component for the client area of the page. The contains the full name of a user component in a Namespace.ClassName format. The and nodes define position of a user page inside the Tools->Options tree-like structure by specifying page's group and personal names. Any existing group names, as well as a new one, can be used as a value. As evident by the example, a user MyAddin1.UserControl1 component is located inside the same assembly as the add-in itself, though this is not a mandatory requirement. Visual Studio loads a page after it is opened by a user for the first time through the Options dialog window. As opposed to the integration of a page through the Managed Package Framework, the description of a page is stored within an XML description addin file, so the page will be initialized only after the environment discovers such a file. Visual Studio reads addin files which are available to it immediately after start-up. The Environment -> Add-In/Macross Security options page specifies the paths which are used for addin discovery. Contrary to custom option pages implemented through inheriting the MPF classes, such high level approach to the integration does not register such a page as an automation object, and so it does not provide ways to access the page's contents through automation object model or to utilize the embedded state preservation mechanism of the environment. Accessing option pages through the automation Visual Studio Automation Object Model provides the means of accessing various system settings of the Tools->Options dialog, excluding some of the pages, such as 'Dynamic Help' and 'Fonts and Colors pages (they are available through separate APIs). User-created custom option pages are also available through the automation model in case they are registered as automation objects themselves (as described in the previous section). The get_Properties method can be utilized to obtain the necessary settings: Properties propertiesList = PVSStudio.DTE.get_Properties("MyPackage", "MyOptionsPage"); The option page can be identified by its own name and the name of group it belongs to. Here is the example of obtaining value for a specific property: Property MyProp1 = propertiesList.Item("MyOption1"); The value of the property can be accessed and modified through the MyProp1.Value. The ShowOptionPage method of the Package MPF subclass can be used to open and display the custom options page inside the Options window. MyPackage.ShowOptionPage(typeof(MyOptionsPage)); As evident by the example, this method takes the type of a user-created DialogPage subclass. However, if it is required to open any other page which is not part or your extension project, a standard IDE page for example, then it could be located by its GUID identifier available at this registry branch: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\9.0\ ToolsOptionsPages\\ Here is the name of a page inside the Tools -> Options dialog. Following is the example of opening a standard TextEditor -> GeneralIDE settings page through the IMenuCommandService global service: string targetGUID = "734A5DE2-DEBA-11d0-A6D0-00C04FB67F6A"; var command = new CommandID(VSConstants.GUID_VSStandardCommandSet97, VSConstants.cmdidToolsOptions); var mcs = GetService(typeof(IMenuCommandService)) as MenuCommandService; mcs.GlobalInvoke(command, targetGUID); In fact, this code is equivalent to the execution of the Tools.Options IDE command. It can be invoked through the ExecuteCommand method of the EnvDTE.DTE: dte.ExecuteCommand("Tools.Options", "734A5DE2-DEBA-11d0-A6D0-00C04FB67F6A"). References MSDN. Options Pages.MSDN. State Persistence and the Visual Studio IDE.MSDN. User Settings and Options.MSDN. Registering Custom Options Pages.MSDN. Providing Automation for VSPackages. Visual Studio project model This item covers the structure of Visual Studio project model and its implementation on the example of Visual C++ (VCProject). Also included are the cases of using the project model for enumeration of project elements and obtaining their compilation properties through the corresponding configurations. Isolated Shell based Atmel Studio environment will be examined as an example of a third-party implementation of the project model. Introduction Visual Studio project model is a collection of interfaces describing the properties of a compiler, linker and other build tools, as well as the structure of MSVS-compatible projects themselves, and it is connected with the Visual Studio Automation Object Model through the VCProjects late-bound properties. Visual C++ project model extends the standard Visual Studio project model, providing access to the specific functionality of Visual C++ (vcproj/vcxproj) project types. Visual C++ project model is a stand-alone COM component available through the VCProjectEngine.dll assembly, which could also be used independently outside of Visual Studio development environment. It is possible for third-party developers to create custom implementations of the project model, adding the support of new languages and compilers into Visual Studio. Project model structure Visual Studio provides an extendable project-neutral object model that represents solutions, projects, code objects, documents, etc. Every MSVS project type has a corresponding project automation interface. Every tool in the environment that has a project also has an object of the 'Project' type associated with it. Standard Visual C++ project model also complies with this general automation project model scheme: Projects |- Project -- Object(unique for the project type) |- ProjectItems (a collection of ProjectItem) |- ProjectItem (single object) -- ProjectItems (another collection) |- Object(unique for the project type) The 'Projects' interface provides an ensemble of abstract objects of the 'Project' type. The 'Project' interface defines an abstract project, i.e. it can reference an object from any project model that complies with the standard scheme. Any peculiar properties of a specific model should be defined through a special interface which is unique only to this model alone. A reference for such an object could be acquired through the Project.Object property. For instance, specific properties of Visual C++ project could be obtained through the VCProject interface, and for the Atmel Studio model it will be the AvrGCCNode interface: Project proj; ... VCProject vcproj = proj.Object as VCProject; AvrGCCNode AvrGccProject = proj.Object as AvrGCCNode; It is possible to obtain a list of all projects loaded in IDE and belonging to any project model type through the dte.Solution.Projects field; projects belonging to a particular model can be acquired through the DTE.GetObject method (see the example below for Visual C++ model): Projects vcprojs = m_dte.GetObject("VCProjects") as Projects; To obtain projects of all types, the following code could be used: Projects AllProjs = PVSStudio.DTE.Solution.Projects; The ProjectItems interface represents an ensemble of abstract solution tree elements of ProjectItem type. Similar to the Project interface, the ProjectItem can define any kind of element; it can even contain the same ProjectItems collection inside itself (accessible through the ProjectItem.ProjectItems) or it can be a Project altogether. An object unique for a specific project model can be obtained through the ProjectItem.Object field. For instance, a Visual C++ source code file is represented by a VCFile type, and Atmel Studio source file be the AvrGccFileNode interface: ProjectItem projectItem; ... VCFile file = projectItem.Object as VCFile; AvrGccFileNode file = projectItem.Object as AvrGccFileNode; An embedded project can be obtained in a similar manner when such an element of the hierarchy represents a project: Project proj = projectItem.Object as Project; Recursively walking all elements of a Solution tree's branch The interface for controlling hierarchies IVsHierarchy can be used to perform a passing of Solution tree's branch. This interface provides an access to abstract nodes of a tree, each one of which in turn could be a leaf, a container of elements or a link to another hierarchy. Each tree node is uniquely identified through the DWORD identifier VSITEMID. Such identifiers are unique within the scope of a single hierarchy and possess a limited lifetime within it. A hierarchy object can be obtained for a tree branch of a single project through the VsShellUtilities.GetHierarchy method: public static IVsHierarchy ToHierarchy(EnvDTE.Project project) { System.IServiceProvider serviceProvider = new ServiceProvider(project.DTE as Microsoft.VisualStudio.OLE.Interop.IServiceProvider); Guid guid = GetProjectGuid(serviceProvider, project); if (guid == Guid.Empty) return null; return VsShellUtilities.GetHierarchy(serviceProvider, guid); } In the example above, the hierarchy was obtained for a project through its GUID identifier. Consider the example of obtaining this GUID identifier for a project: private static Guid GetProjectGuid(System.IServiceProvider serviceProvider, Project project) { if (ProjectUnloaded(project)) return Guid.Empty; IVsSolution solution = (IVsSolution)serviceProvider.GetService(typeof(SVsSolution)) as IVsSolution; IVsHierarchy hierarchy; solution.GetProjectOfUniqueName(project.FullName, out hierarchy); if (hierarchy != null) { Guid projectGuid; ErrorHandler.ThrowOnFailure( hierarchy.GetGuidProperty( VSConstants.VSITEMID_ROOT, (int)__VSHPROPID.VSHPROPID_ProjectIDGuid, out projectGuid)); if (projectGuid != null) { return projectGuid; } } return Guid.Empty; } The IEnumHierarchies interface permits obtaining all of the hierarchies for projects of a particular type through the solution. GetProjectEnum method. Here is an example of obtaining the hierarchies for every Visual C++ project in a solution tree: IVsSolution solution = PVSStudio._IVsSolution; if (null != solution) { IEnumHierarchies penum; Guid nullGuid = Guid.Empty; Guid vsppProjectGuid = new Guid("8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942"); //You can ask the solution to enumerate projects based on the //__VSENUMPROJFLAGS flags passed in. For //example if you want to only enumerate C# projects use //EPF_MATCHTYPE and pass C# project guid. See //Common\IDL\vsshell.idl for more details. int hr = solution.GetProjectEnum( (uint)(__VSENUMPROJFLAGS.EPF_LOADEDINSOLUTION | __VSENUMPROJFLAGS.EPF_MATCHTYPE), ref vsppProjectGuid, out penum); ErrorHandler.ThrowOnFailure(hr); if ((VSConstants.S_OK == hr) && (penum != null)) { uint fetched; IVsHierarchy[] rgelt = new IVsHierarchy[1]; PatternsForActiveConfigurations.Clear(); while (penum.Next(1, rgelt, out fetched) == 0 && fetched == 1) { ... } } } As evident by the example above, the GetProjectEnum method provides hierarchies for projects based on a project kind specified by the GUID identifier. GUID identifiers for regular Visual Studio/MSBuild project types can be obtained here. The penum.Next() method allows us to enumerate all project hierarchies we've acquired (the rgelt array). It should be remembered that user-created project models could possess their own unique identifiers in case they define a new project type for themselves. Such custom user identifiers can be obtained from XML project files of the corresponding models. But our own experience in developing PVS-Studio IDE plug-in demonstrates that an opposite situation is quite possible as well, that is, when a user-created project type uses a GUID from one of the stock project types, usually the one from which it was derived. In particular, we've encountered a VCProject type that was extended to provide development for Android platform. As a result, this project model extension had caused crashes in our plug-in because it did not provide several properties which are otherwise present in VCProject model (OpenMP for example) through the automation API. An intricacy of this situation is that such an extended project model type cannot be differentiated from a regular one, and thus, it is quite hard to correctly process it as well. Therefore, when you are extending a project model through your custom types, to avoid such conflicts with various IDE components (including other third-party extensions as well), it is always important to remember the necessity of providing means to uniquely identify your types. Possessing an IVsHierarchy for the project, we are able to recursively enumerate all the elements of such solution tree branch through the hierarchy.GetProperty method, which in turn provides us with the specified properties for each one of the hierarchy nodes: EnumHierarchyItemsFlat(VSConstants.VSITEMID_ROOT, MyProjectHierarchy, 0, true); ... public void EnumHierarchyItemsFlat(uint itemid, IVsHierarchy hierarchy, int recursionLevel, bool visibleNodesOnly) { if (hierarchy == null) return; int hr; object pVar; hr = hierarchy.GetProperty(itemid, (int)__VSHPROPID.VSHPROPID_ExtObject, out pVar); ProjectItem projectItem = pVar as ProjectItem; if (projectItem != null) { ... } recursionLevel++; //Get the first child node of the current hierarchy being walked hr = hierarchy.GetProperty(itemid, (visibleNodesOnly ? (int)__VSHPROPID.VSHPROPID_FirstVisibleChild :(int)__VSHPROPID.VSHPROPID_FirstChild), out pVar); Microsoft.VisualStudio.ErrorHandler.ThrowOnFailure(hr); if (VSConstants.S_OK == hr) { //We are using Depth first search so at each level we recurse //to check if the node has any children // and then look for siblings. uint childId = GetItemId(pVar); while (childId != VSConstants.VSITEMID_NIL) { EnumHierarchyItemsFlat(childId, hierarchy, recursionLevel, visibleNodesOnly); hr = hierarchy.GetProperty(childId, (visibleNodesOnly ? (int)__VSHPROPID.VSHPROPID_NextVisibleSibling : (int)__VSHPROPID.VSHPROPID_NextSibling), out pVar); if (VSConstants.S_OK == hr) { childId = GetItemId(pVar); } else { Microsoft.VisualStudio.ErrorHandler.ThrowOnFailure(hr); break; } } } } private uint GetItemId(object pvar) { if (pvar == null) return VSConstants.VSITEMID_NIL; if (pvar is int) return (uint)(int)pvar; if (pvar is uint) return (uint)pvar; if (pvar is short) return (uint)(short)pvar; if (pvar is ushort) return (uint)(ushort)pvar; if (pvar is long) return (uint)(long)pvar; return VSConstants.VSITEMID_NIL; } A ProjectItem object that we've acquired for each one of the tree's nodes will allow us to obtain its corresponding Visual C++ object (as well as the object for any other custom project model) through the Object filed, as was described earlier. Enumerating all projects in solution tree DTE.Solution.Projects interface can be used to enumerate all projects in the solution: if (m_DTE.Solution.Projects != null) { try { foreach (object prj in m_DTE.Solution.Projects) { EnvDTE.Project proj = prj as EnvDTE.Project; if (proj != null) WalkSolutionFolders(proj); } } } Besides projects, Solution tree can also contain folder nodes (Solution Folders). They should also be taken into account while processing each Project element: public void WalkSolutionFolders(Project prj) { VCProject vcprj = prj.Object as VCProject; if (vcprj != null && prj.Kind.Equals(VCCProjectTypeGUID)) { if (!ProjectExcludedFromBuild(prj)) { IVsHierarchy projectHierarchy = ToHierarchy(prj); EnumHierarchyItemsFlat(VSConstants.VSITEMID_ROOT, projectHierarchy, 0, false); } } else if (prj.ProjectItems != null) { foreach (ProjectItem item in prj.ProjectItems) { Project nextlevelprj = item.Object as Project; if (nextlevelprj != null && !ProjectUnloaded(nextlevelprj)) WalkSolutionFolders(nextlevelprj); } } } Projects that are excluded from the build should be inspected separately, as they are not accessible through the automation model after being unloaded from the IDE: public bool ProjectExcludedFromBuild(Project project) { if (project.UniqueName.Equals("", StringComparison.InvariantCultureIgnoreCase)) return true; Solution2 solution = m_DTE.Solution as Solution2; SolutionBuild2 solutionBuild = (SolutionBuild2)solution.SolutionBuild; SolutionContexts projectContexts = solutionBuild.ActiveConfiguration.SolutionContexts; //Skip this project if it is excluded from build. bool shouldbuild = projectContexts.Item(project.UniqueName).ShouldBuild; return !shouldbuild; } Enumerating selected elements The DTE.SelectedItems interface can be used to enumerate solution elements which are selected in the Solution Explorer window. foreach (SelectedItem item in items) { VCProject vcproj = null; if (item.Project != null) { vcproj = item.Project.Object as VCProject; if (vcproj != null && item.Project.Kind.Equals("{" + VSProjectTypes.VCpp + "}")) { IVsHierarchy projectHierarchy = ToHierarchy(item.Project); PatternsForActiveConfigurations.Clear(); EnumHierarchyItemsFlat(VSConstants.VSITEMID_ROOT, projectHierarchy, 0, false, files, showProgressDialog); } else if (item.Project.ProjectItems != null) { //solution folder if (!ProjectUnloaded(item.Project)) WalkSolutionFolders(item.Project); } } else if (item.ProjectItem != null) { //walking files ... else if (item.ProjectItem.ProjectItems != null) if (item.ProjectItem.ProjectItems.Count > 0) WalkProjectItemTree(item.ProjectItem); } } private void WalkProjectItemTree(object CurrentItem) { Project CurProject = null; CurProject = CurrentItem as Project; if (CurProject != null) { IVsHierarchy projectHierarchy = ToHierarchy(CurProject); PatternsForActiveConfigurations.Clear(); EnumHierarchyItemsFlat(VSConstants.VSITEMID_ROOT, projectHierarchy, 0, false); return; } ProjectItem item = null; item = CurrentItem as ProjectItem; if (item != null) { ... if (item.ProjectItems != null) if (item.ProjectItems.Count > 0) { foreach (object NextItem in item.ProjectItems) WalkProjectItemTree(NextItem); } } } Visual C++ project model. Configurations and properties of projects and files To this moment, we have been examining a general, exterior part of Visual Studio project model, declared mostly in EnvDTE namespace, which is available from any version of project model implementation. Now, let's talk about one of the regular implementations of this model: Microsoft Visual C++. It is declared in the VisualStudio.VCProjectEngine namespace. Visual C++ project model implements a general Visual Studio project model, so interfaces described in the previous chapters can be utilized for projects of this particular model as well. Interfaces that are specific to Visual C++ model are defined inside the Microsoft.VisualStudio.VCProjectEngine.dll assembly. This assembly should be added to the references of the extension that is being developed. Visual C++ physically stores build configurations (compilation and linking parameters, pre-build and post-build steps, external tool command lines, etc.) for C/C++ source files inside its XML-based project files (vcproj/vcxproj). These settings are available to Visual Studio users through property page dialogs. Each combination of project's build configuration (Debug, Release, etc.) and platform (Win32, IA64, x64, etc.) is associated with a separate collection of settings. Although majority of the settings are defined at the project level, it is possible to redefine separate properties for each individual file (file properties are inherited from its project by default). The list of properties which can be redefined at the file level is dependent upon the type of a file in question. For example, only the ExcludedFromBuild property can be redefined for header files, but cpp source files permit the redefinition for any of its compilation properties. Obtaining configurations Visual C++ project model presents property pages through the VCConfiguration (for a project) and VCFileConfiguration (for a file) interfaces. To obtain these objects we will start from a ProjectItem object which represents an abstract Solution tree element. ProjectItem item; VCFile vcfile = item.Object as VCFile; Project project = item.ContainingProject; String pattern = "Release|x64"; if (String.IsNullOrEmpty(pattern)) return null; VCFileConfiguration fileconfig = null; IVCCollection fileCfgs = (IVCCollection)vcfile.FileConfigurations; fileconfig = fileCfgs.Item(pattern) as VCFileConfiguration; if (fileconfig == null) if (fileCfgs.Count == 1) fileconfig = (VCFileConfiguration)fileCfgs.Item(0); In the example above we've acquired a file configuration for VCFile object (which represents a C/C++ header or a source file) by passing a configuration pattern (configuration's name and platform) to the Item() method. Build configuration pattern is defined on the project level. The following example demonstrates the acquisition of active configuration (the one that is selected in IDE) of a project. ConfigurationManager cm = project.ConfigurationManager; Configuration conf = cm.ActiveConfiguration; String platformName = conf.PlatformName; String configName = conf.ConfigurationName; String pattern = configName + "|" + platformName; return pattern; The ActiveConfiguration interface should be handled with care. Quite often we've encountered exceptions when calling it from our PVS-Studio IDE extension package. In particular, this field sometimes becomes inaccessible through the automation object model when a user is building a project, or in the presence of any other heavy user interaction with Visual Studio UI. As there is no assured way of predicting such user actions, it is advised to provide additional error handlers for such 'bottlenecks' when accessing settings with automation model. It should be noted that this particular situation is not related to COM exception handling that was described in the previous article dedicated to EnvDTE interfaces, and it is probably related to some internal issues within the automation model itself. Next, let's acquire the configuration for a project that contains the file in question: VCConfiguration cfg=(VCConfiguration)fileconfig.ProjectConfiguration; While the interfaces representing configurations themselves contain settings only from the 'General' tab of the property pages, references for individual build tools can be acquired through the VCConfiguration.Tools and VCFileConfiguration.Tool interfaces (note that a single file contains settings respectively for only one build tool). Let's examine the VCCLCompilerTool interface representing the C++ compiler: ct = ((IVCCollection)cfg.Tools).Item("VCCLCompilerTool") as VCCLCompilerTool; ctf = fileconfig.Tool as VCCLCompilerTool; Now let's acquire the contents of, for example, the AdditionalOptions field belonging to the compiler tool, using the Evaluate method to process any macros that we can encounter within its value: String ct_add = fileconfig.Evaluate(ct.AdditionalOptions); String ctf_add = fileconfig.Evaluate(ctf.AdditionalOptions); Property Sheets Property sheets are XML files with a *.props extension. They allow an independent definition of project's build properties, i.e. the command line parameters for various building tools, such as a compiler or a linker. Property sheets also support inheritance and can be used for specifying build configurations for several projects at once, i.e. the configuration defined inside the project file itself (vcproj/vcxproj) could inherit some of its properties from single or multiple props files. To handle property sheets, Visual C++ project model provides the VCPropertySheet interface. A collection of VCPropertySheet objects can be obtained through the VCConfiguration.PropertySheets field: IVCCollection PSheets_all = fileconfig.PropertySheets; Similarly, the PropertySheets field of the VCPropertySheet interface provides a reference to a collection of child property sheet files for this object. Let's examine the recursive enumeration of all of the project's property sheets: private void ProcessAllPropertySheets(VCConfiguration cfg, IVCCollection PSheets) { foreach (VCPropertySheet propertySheet in PSheets) { VCCLCompilerTool ctPS = (VCCLCompilerTool)((IVCCollection)propertySheet.Tools).Item( "VCCLCompilerTool"); if (ctPS != null) { ... IVCCollection InherPSS = propertySheet.PropertySheets; if (InherPSS != null) if (InherPSS.Count != 0) ProcessAllPropertySheets(cfg, InherPSS); } } } In the example above we've obtained an object of VCCLCompilerTool type (that is compilation settings) for PropertySheet on every level. In this way we could gather all compilation parameters defined in every property sheet, including the embedded ones. The VCPropertySheet interface does not contain means to evaluate macros within its fields, so as a work-around, the Evaluate method from the project's configuration can be used instead. But, such approach could also lead to the incorrect behavior in case the value of the macro being evaluated is related to the props file itself. For instance, several MSBuild macros which were introduced in the MSBuild version 4 could also be utilized inside vcxproj projects from Visual Studio 2010. Let's take the MSBuildThisFileDirectory macro that evaluates as a path to the directory containing file in which it is used. Now, evaluating this macro through the cfg.Evaluate will result in a path to the vcxproj file, and not to props file, which actually does contains this macro. All of the property sheets in Visual C++ project can be divided between user and system files. By user files we understand the props files which were created and added to the project by a user himself. But even an empty template-generated MSVC project often includes several property sheets by default. These system props files are utilized by the environment to specify various compilation parameters which were set inside the project's property page interface by the user. For example, setting up the CharacterSet property to use Unicode manually in the Property Page interface will result in the appearance of a special property sheet in the 'Property Sheets' window which will define several preprocessor symbols (Unicode, _Unicode), and this properties subsequently will be inherited by the project. Therefore when processing properties from inside a Property sheet, one should always remember that compilation symbols defined in system props files are also returned by their corresponding property in the project's configuration through the automation API. Evidently, processing these two simultaneously while gathering compilation arguments can result in a duplication of such arguments. Atmel Studio project model. Compilation settings in project toolsets. We have examined an implementation of Visual Studio project model for C/C++ projects from Microsoft Visual C++, the model that is included into Visual Studio distribution by default. But Visual Studio automation model can be extended by adding interfaces for interacting with other custom project models from third-party developers (such custom project model can actually be implemented as a VSPackage extension). Therefore, if a developer of custom project model does provide interfaces for it, when it will be possible to interact with it the same way that we've interacted with Visual Studio regular models, such as Visual C++ one, as was described earlier. For example, let's examine the interfaces of the project model that is provided by Atmel Studio IDE, the environment intended for development of embedded solutions. Now, you could probably ask - how does the topic of this article concern Atmel Studio, and we were always examining the interactions with Visual Studio? But, the truth is - Atmel Studio itself is the Visual Studio isolated shell application. We will not be discussing the essence of isolated shells here, and I will only mention that it is possible to develop the same kinds of extensions and plugins for isolated shells, as it is possible for the regular Visual Studio versions. You can familiarize yourself with some of the specifics regarding the development of Visual Studio plug-ins, including the isolated shell applications, in the previous chapter. Now, Atmel project model itself is the implementation of a standard Visual Studio project model. And, exactly as for the Visual C++ projects, the common interfaces could be utilized with Atmel Studio projects as well. The interfaces that are specific for this model are declared in the AvrGCC.dll, AvrProjectManagement.dll and Atmel.Studio.Toolchain.Interfaces.dll files. These files can be obtained by downloading the Atmel Studio Extension Developer's Kit (XDK). Atmel Studio project model physically stores its build parameters in cproj project files which themselves at the same time serve as project files for MSBuild system (as all of the standard project types in Visual Studio). Atmel project model supports C/C++ languages and utilizes special editions of GCC compilers for the purpose of building its source files. Types of projects and toolchains Atmel Studio provides 2 types of projects: C and C++ projects. Please note that these project types possess different GUIDs, so this should be taken into account when traversing the project tree. Atmel project model also provides 2 compile tool chains - GNU C compiler and GNU C++ Compiler, each one possessing a separate set of options. It should be noted that while C projects can hold only C compiler options, the toolchain of C++ project consists of both C and C++ options sets. The appropriate compile options set will be selected by the build system automatically during compilation according to the extension of the source file being built. This means that in the case of a "mixed" project, 2 compile tool sets will be utilized at the same time! The available option sets for each individual project can be obtained through the ProjectToolchainOptions interface. ProjectItem item; ... AvrGccFileNode file = item.Object as AvrGccFileNode; AvrGCCNode project = file.ProjectMgr as AvrGCCNode; AvrProjectConfigProperties ActiveProps = project.ConfigurationManager.GetActiveConfigProperties(); ProjectToolchainOptions ToolChainOptions = ActiveProps.ToolchainOptions; if (ToolChainOptions.CppCompiler != null) //Toolchain compiler options for C++ compiler if (ToolChainOptions.CCompiler != null) //Toolchain compiler options for C Compiler Obtaining compilation settings To extract individual compilation settings themselves, the CompilerOptions object that we've obtained earlier (a common base type for CppCompilerOptions and CCompilerOptions) can be utilized. Some of the settings could be taken directly from this object, such as Include paths of a project: CompilerOptions options; ... List Includes = options. IncludePaths; Please note that a part of the settings are shared between all project types (i.e. between the C and C++ ones). An example of such common settings is the one holding system includes: List SystemIncludes = options. DefaultIncludePaths; But most of other settings are available only through the OtherProperties property of the Dictionary type. As evident by the type of this property, each setting (a key) corresponds to a list of one or more values. However, if you wish to obtain a whole command line passed to MSBuild from the project (and to the compiler thereafter), and not the individual settings values, such command line can be immediately obtained through the CommandLine property (which is a lot easier than in the case of VCProjectEngine!): String RawCommandLine = this.compilerOptions.CommandLine; It should be noted that General settings, such as system includes, still will not be present in a command line obtained in this way. Also worth noting is that either individual setting values or the whole command line obtained in this way could still contain a number of unexpanded MSBuild macros. The GetAllProjectProperties method of the AvrGCCNode interface can be utilized to resolve such macros: AvrGCCNode project; ... Dictionary MSBuildProps = new Dictionary(); project.GetAllProjectProperties().ForEach(x =>MSBuildProps.Add(x.Key, x.Value)); Now we can replace each macro we encounter with the corresponding values from MSBuildProps collection. Each file in the project can possess additional build parameters, additional compiler flags to be more precise. We can obtain such flags through the AvrFileNodeProperties interface: AvrGccFileNode file; ... AvrFileNodeProperties FileProps = file.NodeProperties as AvrFileNodeProperties; String AdditionalFlags = FileProps.CustomCompilationSetting; References MSDN. Visual C++ Project Model.MSDN. Project Modeling.MSDN. Automation Model Overview.
  6. Not so long ago one of our colleagues left the team and joined one company developing software for embedded systems. There is nothing extraordinary about it: in every firm people come and go, all the time. Their choice is determined by bonuses offered, the convenience aspect, and personal preferences. What we find interesting is quite another thing. Our ex-colleague is sincerely worried about the quality of the code he deals with in his new job. And that has resulted in us writing a joint article. You see, once you have figured out what static analysis is all about, you just don't feel like settling for "simply programming". Forest Reserves I find an interesting phenomenon occurring in the world nowadays. What happens when a software development department turns into a secondary entity not closely related to the company's basic area of activity? A forest reserve appears. However significant and critical the company's area of activity may be (say, medicine or military equipment), a small swamp appears anyway, where new ideas get stuck and 10-year old technologies are in use. Here you are a couple of extracts from the correspondence of a man working in the software development department at some nuclear power plant: And then he says, "What for do we need git? Look here, I've got it all written down in my paper notebook." ... And do you have any version control at all? 2 men use git. The rest of the team use numbered zip's at best. Though it's only 1 person with zip's I'm sure about. Don't be scared. Software developed at nuclear power plants may serve different purposes, and no one has abolished hardware security yet. In that particular department, people collect and process statistical data. Yet the tendency to swamping is pretty obvious. I don't know why it happens, but the fact is certain. What's interesting, the larger the company, the more intense the swamping effect. I want to point it out that stagnation in large companies is an international phenomenon. Things are quite the same abroad. There is an article on the subject, but I don't remember its title. I spent quite a time trying to find it, but in vain. If somebody knows it, give me the link please so that I could post it. In that article, a programmer is telling a story about him having worked at some military department. It was - naturally - awfully secret and bureaucratic - so much secret and bureaucratic that it took them several months to agree on which level of access permissions he could be granted to work on his computer. As a result, he was writing a program in Notepad (without compiling it) and was soon fired for inefficiency. Foresters Now let's get back to our ex-colleague. Having come to his new office, he was struck with kind of a cultural shock. You see, after spending so much time and effort on studying and working with static analysis tools, it's very painful to watch people ignore even compiler warnings. It's just like a separate world where they program according to their own canons and even use their own terms. The man told me some stories about it, and most of all I liked the phrase "grounded pointers" common among the local programmers. See how close they are to the hardware aspect? We are proud of having raised inside our team a skilled specialist who cares about the code quality and reliability. He hasn't silently accepted the established situation; he is trying to improve it. As a start, he did the following. He studied the compiler warnings, then checked the project with Cppcheck and considered preventing typical mistakes in addition to making some fixes. One of his first steps was preparing a paper aiming at improving the quality of the code created by the team. Introducing and integrating a static code analyzer into the development process might be the next step. It will certainly not be PVS-Studio: first, they work under Linux; second, it's very difficult to sell a software product to such companies. So, he has chosen Cppcheck for now. This tool is very fine for people to get started with the static analysis methodology. I invite you to read the paper he has prepared. It is titled "The Way You Shouldn't Write Programs". Many of the items may look written pretty much in the Captain Obvious style. However, these are real problems the man tries to address. The Way You Shouldn't Write Programs Issue 1 Ignoring compiler warnings. When there are many of them in the list, you risk easily missing genuine errors in the lately written code. That's why you should address them all. Issue 2 In the conditional statement of the if operator, a variable is assigned a value instead of being tested for this value: if (numb_numbc = -1) { } The code is compiled well in this case, but the compiler produces a warning. The correct code is shown below: if (numb_numbc == -1) { } Issue 3 The statement using namespace std; written in header files may cause using this namespace in all the files which include this header, which in turn may lead to calling wrong functions or the occurrence of name collisions. Issue 4 Comparing signed variables to unsigned ones: unsigned int BufPos; std::vector ba; .... if (BufPos * 2 < ba.size() - 1) { } Keep in mind that mixing signed and unsigned variables may result in: overflows; occurrence of always true or always false conditions and, as a consequence, infinite loops; a value larger than INT_MAX may be written into a signed variable (and it will be negative); an int-variable participating in addition/subtraction/etc. with an unsigned variable becomes unsigned too (so that negative values turn into large positive ones); other unexpected nice things The foregoing code sample incorrectly handles the situation of the ba array being empty. The expression ba.size() - 1 evaluates to an unsigned size_t value. If the array contains no items, the expression evaluates to 0xFFFFFFFFu. Issue 5 Neglecting usage of constants may lead to overlooking hard-to-eliminate bugs. For example: void foo(std::string &str) { if (str = "1234") { } } The = operator is mistakenly used instead of ==. If the str variable were declared as a constant, the compiler would not even compile the code. Issue 6 Pointers to strings are compared instead of strings themselves: char TypeValue [4]; ... if (TypeValue == "S") {} Even if the string S is stored in the variable TypeValue, the comparison will always return false. The correct way to compare strings is to use the special functions strcmp or strncmp. Issue 7 Buffer overflow: memset(prot.ID, 0, sizeof(prot.ID) + 1); This code may cause several bytes of the memory area right after prot.ID to be cleared as well. Don't mix up sizeof() and strlen(). The sizeof() operator returns the complete size of an item in bytes. The strlen() function returns the string length in characters (without counting the null terminator). Issue 8 Buffer underflow: struct myStruct { float x, y, h; }; myStruct *ptr; .... memset(ptr, 0, sizeof(ptr)); In this case only N bytes will be cleared instead of the whole *ptr structure (N is the pointer size on the current platform). The correct way is to use the following code: myStruct *ptr; .... memset(ptr, 0, sizeof(*ptr)); Issue 9 Incorrect expression: if (0 < L < 2 * M_PI) { } The compiler doesn't see any error here, yet the expression is meaningless, for you will always get either true or false when executing it, the exact result depending on the comparison operators and boundary conditions. The compiler generates a warning for such expressions. The correct version of this code is: if (0 < L && L < 2 * M_PI) { } Issue 10 unsigned int K; .... if (K < 0) { } ... if (K == -1) { } Unsigned variables cannot be less than zero. Issue 11 Comparing a variable to a value it can never reach. For example: short s; ... If (s==0xaaaa) { } The compiler produces warnings against such things. Issue 12 Memory is allocated with the help of new or malloc, while forgotten to be freed through delete/free correspondingly. It may look something like this: void foo() { std::vector *v1 = new std::vector; std::vector v2; v2->push_back(*v1); ... } Perhaps it was the pointer to std::vector that used to be saved into v2 before. Now, due to modifications of some code parts, it is no longer needed and there are just int values being saved. At the same time, memory allocated for v1 is not freed, as that was not needed in earlier times. To fix the code we should add the statement delete v1 at the end of the function, or use smart pointers. Even better is to bring refactoring to an end, making v1 a local object, since you no longer need to pass it anywhere: void foo() { std::vector v1; std::vector v2; v2->push_back(v1[0]); ... } Issue 13 Memory is allocated through new[] and freed through delete. Or, vice versa, memory is allocated through new and freed through delete[]. Issue 14 Using uninitialized variables: int sum; ... for (int i = 0; i < 10; i++) { sum++; } In C/C++, variables are not initialized to zero by default. Sometimes code only seems to run well, which is not so - it's merely luck. Issue 15 A function returns a reference or pointer to local objects: char* CreateName() { char FileName[100]; ... return FileName; } On leaving the function, FileName will refer to an already-freed memory area, since all the local objects are created on the stack, so it's impossible to handle it correctly any further. Issue 16 Values returned by functions are not checked, while they may return an error code or -1 in case of an error. It may happen that a function returns an error code, us continuing to work without noticing and reacting to it in any way, which will result in a sudden program crash at some point. Such defects take much time to debug after that. Issue 17 Neglecting usage of special static and dynamic analysis tools, as well as creation and usage of unit-tests. Issue 18 Being too greedy for adding some parentheses in math expressions, which results in the following: D = ns_vsk.bit.D_PN_ml + (int)(ns_vsk.bit.D_PN_st)
  7. We haven't used PVS-Studio to check games for a long time. So, this time we decided to return to this practice and picked out the MTA project. Multi Theft Auto (MTA) is a multiplayer modification for PC versions of the Grand Theft Auto: San Andreas game by Rockstar North that adds online multiplayer functionality. As Wikipedia tells us, the specific feature of the game is "well optimized code with fewest bugs possible". OK, let's ask our analyzer for its opinion. Introduction This time I decided to omit the texts of diagnostic messages generated by PVS-Studio for every particular defect. I comment upon examples anyway, so if you want to find out in which particular line and by which diagnostic rule a certain bug was found, see the file mtasa-review.txt. When looking through the project, I noted in the mtasa-review.txt file those code fragments which I found suspicious and used it to prepare the article. Important! I added only those code fragments which I personally didn't like. I'm not a MTA developer, so I'm not familiar with its logic and principles. That's why I must have made a few mistakes attacking correct code fragments and missing genuine bugs. Also, when studying certain fragments, I felt lazy indeed to describe some slightly incorrect printf() function calls. So, I'm asking MTA Team developers not to rely on this article and consider checking the project by themselves. It is pretty large, so the demo version of PVS-Studio won't be enough. However, we support free open-source projects. If you're an open source developer, contact us and we'll discuss the question of giving you a free registration key. So, Multi Theft Auto is an open-source project in C/C++: project website; source code; MTA Wiki. Analysis was performed by the PVS-Studio 5.05 analyzer: tool page; download (the demo version's only limitation concerns the number of click jumps from the message list to source code); bug database. Now let's see what bugs PVS-Studio has managed to find in the game. They aren't numerous, and most of them are found in rarely-used parts of the program (error handlers). It's no wonder: most bugs are found and fixed through other, more expensive and slow, methods. To use static analysis properly is to use it regularly. By the way, PVS-Studio can be called to analyze recently modified and compiled files only (see incremental analysis mode). This mechanism allows the developer to find and fix many bugs and misprints immediately, which makes it much faster and cheaper than detecting errors through testing. This subject was discussed in detail in the article "Leo Tolstoy and static code analysis". It's a worthy article, and I do recommend reading the introduction to understand the ideology of using PVS-Studio and other static analysis tools. Strange Colors // c3dmarkersa.cpp SColor C3DMarkerSA::GetColor() { DEBUG_TRACE("RGBA C3DMarkerSA::GetColor()"); // From ABGR unsigned long ulABGR = this->GetInterface()->rwColour; SColor color; color.A = ( ulABGR >> 24 ) && 0xff; color.B = ( ulABGR >> 16 ) && 0xff; color.G = ( ulABGR >> 8 ) && 0xff; color.R = ulABGR && 0xff; return color; } By mistake '&&' is used instead of '&'. The color is torn into bits and pieces to leave only 0 or 1. The same problem is found in the file "ccheckpointsa.cpp". One more problem with colors. // cchatechopacket.h class CChatEchoPacket : public CPacket { .... inline void SetColor( unsigned char ucRed, unsigned char ucGreen, unsigned char ucBlue ) { m_ucRed = ucRed; m_ucGreen = ucGreen; m_ucRed = ucRed; }; .... } Red is copied twice, while blue is not copied at all. The fixed code should look like this: { m_ucRed = ucRed; m_ucGreen = ucGreen; m_ucBlue = ucBlue; }; The same problem is found in the file cdebugechopacket.h. By the way, quite a number of bugs of the game are duplicated in two files which, I suspect, refer to the client-side and the server-side correspondingly. Do you feel the great power of the Copy-Paste technology? :). Something Wrong with utf8 // utf8.h int utf8_wctomb (unsigned char *dest, wchar_t wc, int dest_size) { if (!dest) return 0; int count; if (wc < 0x80) count = 1; else if (wc < 0x800) count = 2; else if (wc < 0x10000) count = 3; else if (wc < 0x200000) count = 4; else if (wc < 0x4000000) count = 5; else if (wc Create (....); if ( pPlayer ) { .... } else { // Tell the console CLogger::LogPrintf( "CONNECT: %s failed to connect " "(Player Element Could not be created.)\n", pPlayer->GetSourceIP() ); } .... } If the object player can't be created, the program will attempt printing the corresponding error message into the console. It will fail because it's a bad idea to use a null pointer when calling the function pPlayer->GetSourceIP(). Another null pointer is dereferenced in the following fragment: // clientcommands.cpp void COMMAND_MessageTarget ( const char* szCmdLine ) { if ( !(szCmdLine || szCmdLine[0]) ) return; .... } If the szCmdLine pointer is null, it will be dereferenced. The fixed code must look like this, I suppose: if ( !(szCmdLine && szCmdLine[0]) ) The following code fragment I like most of all: // cdirect3ddata.cpp void CDirect3DData::GetTransform (....) { switch ( dwRequestedMatrix ) { case D3DTS_VIEW: memcpy (pMatrixOut, &m_mViewMatrix, sizeof(D3DMATRIX)); break; case D3DTS_PROJECTION: memcpy (pMatrixOut, &m_mProjMatrix, sizeof(D3DMATRIX)); break; case D3DTS_WORLD: memcpy (pMatrixOut, &m_mWorldMatrix, sizeof(D3DMATRIX)); break; default: // Zero out the structure for the user. memcpy (pMatrixOut, 0, sizeof ( D3DMATRIX ) ); break; } .... } Very nice Copy-Paste. The function memset() must be called instead of the last memcpy() function. Uncleared Arrays There are a number of errors related to uncleared arrays. They all can be arranged into two categories. The first includes unremoved items, the second includes partial array clearing errors. Unremoved Items // cperfstat.functiontiming.cpp std::map < SString, SFunctionTimingInfo > m_TimingMap; void CPerfStatFunctionTimingImpl::DoPulse ( void ) { .... // Do nothing if not active if ( !m_bIsActive ) { m_TimingMap.empty (); return; } .... } The function empty() only checks whether or not the container contains items. To remove items from the m_TimingMap container one should call the clear() function. Another example: // cclientcolsphere.cpp void CreateSphereFaces ( std::vector < SFace >& faceList, int iIterations ) { int numFaces = (int)( pow ( 4.0, iIterations ) * 8 ); faceList.empty (); faceList.reserve ( numFaces ); .... } Some more similar bugs are found in the file cresource.cpp. If you have started reading the article from the middle and therefore skipped the beginning, see the file mtasa-review.txt to find out exact locations of all the bugs. Partial Array Clearing Errors // crashhandler.cpp LPCTSTR __stdcall GetFaultReason(EXCEPTION_POINTERS * pExPtrs) { .... PIMAGEHLP_SYMBOL pSym = (PIMAGEHLP_SYMBOL)&g_stSymbol ; FillMemory ( pSym , NULL , SYM_BUFF_SIZE ) ; .... } Everything looks alright at the first sight. But FillMemory() will in fact have no effect. FillMemory() and memset() are different functions. Have a look at this fragment: #define RtlFillMemory(Destination,Length,Fill) \ memset((Destination),(Fill),(Length)) #define FillMemory RtlFillMemory The second and the third arguments are swapped. That's why the correct code should look like this: FillMemory ( pSym , SYM_BUFF_SIZE, 0 ) ; The same thing is found in the file ccrashhandlerapi.cpp. And here is the last error sample of this type. Only one byte gets cleared. // hash.hpp unsigned char m_buffer[64]; void CMD5Hasher::Finalize ( void ) { .... // Zeroize sensitive information memset ( m_buffer, 0, sizeof (*m_buffer) ); .... } Asterisk '*' should be removed: sizeof (m_buffer). Uninitialized Variable // ceguiwindow.cpp Vector2 Window::windowToScreen(const UVector2& vec) const { Vector2 base = d_parent ? d_parent->windowToScreen(base) + getAbsolutePosition() : getAbsolutePosition(); .... } The variable base initializes itself. Another bug of this kind can be found a few lines ahead. Array Index out of Bounds // cjoystickmanager.cpp struct { bool bEnabled; long lMax; long lMin; DWORD dwType; } axis[7]; bool CJoystickManager::IsXInputDeviceAttached ( void ) { .... m_DevInfo.axis[6].bEnabled = 0; m_DevInfo.axis[7].bEnabled = 0; .... } The last line m_DevInfo.axis[7].bEnabled = 0; is not needed. Another error of this kind // cwatermanagersa.cpp class CWaterPolySAInterface { public: WORD m_wVertexIDs[3]; }; CWaterPoly* CWaterManagerSA::CreateQuad ( const CVector& vecBL, const CVector& vecBR, const CVector& vecTL, const CVector& vecTR, bool bShallow ) { .... pInterface->m_wVertexIDs [ 0 ] = pV1->GetID (); pInterface->m_wVertexIDs [ 1 ] = pV2->GetID (); pInterface->m_wVertexIDs [ 2 ] = pV3->GetID (); pInterface->m_wVertexIDs [ 3 ] = pV4->GetID (); .... } One more: // cmainmenu.cpp #define CORE_MTA_NEWS_ITEMS 3 CGUILabel* m_pNewsItemLabels[CORE_MTA_NEWS_ITEMS]; CGUILabel* m_pNewsItemShadowLabels[CORE_MTA_NEWS_ITEMS]; void CMainMenu::SetNewsHeadline (....) { .... for ( char i=0; i SetFont ( szFontName ); m_pNewsItemShadowLabels[ i ]->SetFont ( szFontName ); .... } .... } At least one more error of this kind can be found in the file cpoolssa.cpp. But I decided not to describe it in the article because that would be a pretty large sample and I didn't know how to make it brief and clear. As I've already said, this and all the rest of the bugs can be found in the detailed report. The Word 'throw' is Missing // fallistheader.cpp ListHeaderSegment* FalagardListHeader::createNewSegment(const String& name) const { if (d_segmentWidgetType.empty()) { InvalidRequestException( "FalagardListHeader::createNewSegment - " "Segment widget type has not been set!"); } return ....; } The correct line is throw InvalidRequestException(....). Another code fragment. // ceguistring.cpp bool String::grow(size_type new_size) { // check for too big if (max_size() SaveReadableDepthBuffer(); .... } The programmer wanted to check a particular bit in the Flag variable. By mistake he wrote the '|' operation instead of '&'. This results in the condition being always true. A similar mess-up is found in the file cvehiclesa.cpp. Another bug in a check is found here: unsigned_value < 0. // crenderitem.effectcloner.cpp unsigned long long Get ( void ); void CEffectClonerImpl::MaybeTidyUp ( void ) { .... if ( m_TidyupTimer.Get () < 0 ) return; .... } The Get() function returns the value of the unsigned unsigned long long type. It means that the check m_TidyupTimer.Get () < 0 is pointless. Other errors of this type can be found in the files csettings.cpp, cmultiplayersa_1.3.cpp and cvehiclerpcs.cpp. This Code May Work, but You'd Better Refactor It Many PVS-Studio diagnostics detected bugs which will most likely in no way manifest themselves. I don't like describing such bugs because they are not interesting. So, here are just a couple of examples. // cluaacldefs.cpp int CLuaACLDefs::aclListRights ( lua_State* luaVM ) { char szRightName [128]; .... strncat ( szRightName, (*iter)->GetRightName (), 128 ); .... } The third argument of the strncat() function refers, instead of the buffer size, to the number of characters you can put into the buffer. A buffer overflow can theoretically occur here, but in practice it will most probably never happen. This type of error is described in detail in the V645 diagnostic's description. The second example. // cscreenshot.cpp void CScreenShot::BeginSave (....) { .... HANDLE hThread = CreateThread ( NULL, 0, (LPTHREAD_START_ROUTINE)CScreenShot::ThreadProc, NULL, CREATE_SUSPENDED, NULL ); .... } In many game fragments, the functions CreateThread()/ExitThread() are used. This is in most cases a bad idea. You should use the functions _beginthreadex()/_endthreadex() instead. For details on this issue see the V513 diagnostic's description. I Have to Stop Somewhere I have described only a part of all the defects I noticed. But I have to stop here: the article is already big enough. See the file mtasa-review.txt for other bug samples. There you will find bugs which I haven't mentioned in the article: identical branches in the conditional operator if () { aa } else { aa }; checking a pointer returned by the new operator for being a null pointer: p = new T; if (!p) { aa }; a poor way of using #pragma to suppress compiler warnings (instead of push/pop); classes contain virtual functions but no virtual destructors; a pointer gets dereferenced first and only then checked for being a null pointer; identical conditions: if (X) { if (X) { aa } }; miscellaneous. Conclusion The PVS-Studio analyzer can be efficiently used to eliminate various bugs at early development stages both in game projects and projects of any other type. It won't find algorithmic errors of course (it needs AI to do that), but it will help save a lot of time programmers usually waste searching for silly mistakes and misprints. Developers actually spend much more time on finding plain defects than they may think. Even debugged and tested code contains numbers of such errors, while 10 times more of them get fixed when writing new code.
  8. We decided to write several small posts on how C/C++ programmers play with fire without knowing it. The first post will be devoted to an attempt to explicitly call a constructor. Programmers are lazy creatures. That's why they tend to solve a task using minimal amounts of code. This aim is praiseworthy and good. But the main point is to not get too involved in the process and stop at the right time. For example, programmers are too lazy to create a single initialization function in a class so that it could be called from various constructors later. They think: "What do I need an extra function for? I'd rather call one constructor from the other". Unfortunately, sometimes programmers can't solve even such a simple task. It is to detect such unsuccessful attempts that I'm implementing a new rule in PVS-Studio. Here is, for instance, a code sample I have found in the eMule project: class CSlideBarGroup { public: CSlideBarGroup(CString strName, INT iIconIndex, CListBoxST* pListBox); CSlideBarGroup(CSlideBarGroup& Group); ... } CSlideBarGroup::CSlideBarGroup(CSlideBarGroup& Group) { CSlideBarGroup( Group.GetName(), Group.GetIconIndex(), Group.GetListBox()); } Let's examine more attentively how the last constructor is implemented. The programmer decided that the code CSlideBarGroup( Group.GetName(), Group.GetIconIndex(), Group.GetListBox()); simply calls the other constructor. Nothing of the kind. A new unnamed object of the CslideBarGroup type is created and destroyed right afterwards. It appears that the programmer has actually called the other constructor. But he/she has not quite done the same thing he/she intended: the class fields remain uninitialized. Such errors are just half the trouble. Some people do know how to call the other constructor really. And they do it. I wish they didn't know :) For instance, the above given code could be rewritten in this way: CSlideBarGroup::CSlideBarGroup(CSlideBarGroup& Group) { this->CSlideBarGroup::CSlideBarGroup( Group.GetName(), Group.GetIconIndex(), Group.GetListBox()); } or in this way: CSlideBarGroup::CSlideBarGroup(CSlideBarGroup& Group) { new (this) CSlideBarGroup( Group.GetName(), Group.GetIconIndex(), Group.GetListBox()); } Now one data initialization constructor is really calling the other constructor. If you see a programmer doing this, deal him/her one flick on his/her forehead for yourself and one more flick on my behalf. The cited examples contain very dangerous code and you should understand well how they work! Being written for the purpose of petty optimization (programmers are too lazy to write a separate function), this code might do more harm than good. Let's see more closely why such constructs sometimes work but most often don't. class SomeClass { int x,y; public: SomeClass() { new (this) SomeClass(0,0); } SomeClass(int xx, int yy) : x(xx), y(yy) {} }; This code will work correctly. It is safe and works well, since the class contains primary data types and is not a descendant of other classes. In this case, a double constructor call is harmless. Let's consider another example where an explicit constructor call causes an error (the sample is taken from this discussion on the StackOverflow website): class Base { public: char *ptr; std::vector vect; Base() { ptr = new char[1000]; } ~Base() { delete [] ptr; } }; class Derived : Base { Derived(Foo foo) { } Derived(Bar bar) { new (this) Derived(bar.foo); } } When we call the new (this) Derived(bar.foo); constructor, the Base object is already created and fields initialized. The repeated constructor call will cause a double initialization. A pointer to the newly allocated memory area will be written into ptr. As a result, we get a memory leak. The result of double initialization of an object of the std::vector type cannot be predicted at all. But one thing is obvious: such code is inadmissible. Conclusion An explicit constructor call is needed only in very rare cases. In common programming practice, an explicit constructor call usually appears due to a programmer's wish to reduce the code's size. Don't do that! Create an ordinary initialization function. This is how the correct code should look: class CSlideBarGroup { void Init(CString strName, INT iIconIndex, CListBoxST* pListBox); public: CSlideBarGroup(CString strName, INT iIconIndex, CListBoxST* pListBox) { Init(strName, iIconIndex, pListBox); } CSlideBarGroup(CSlideBarGroup& Group) { Init(Group.GetName(), Group.GetIconIndex(), Group.GetListBox()); } ... }; P.S. Explicit call of one constructor from the other in C++11 (delegation) The new C++11 standard allows you to perform the call of constructors from other constructors (known as delegation). It enables you to create constructors that use the behavior of other constructors without added code. This is an example of correct code: class MyClass { std::string m_s; public: MyClass(std::string s) : m_s(s) {} MyClass() : MyClass("default") {} };
  9. While analyzing the source codes of various programs I can't help creating associations that each program has a tendency to certain diseases. In many projects you can easily make out patterns of incorrect code that can be found in different project files. In some programs these are Copy-Paste errors, while in others it's "unsigned_integer < 0"-like checks. Each project has its own disease. The sore of the next project (called MAME) we have checked is the memset() function. MAME is an emulator application designed to recreate the hardware of arcade game systems in software to preserve gaming history by preventing vintage games from being lost or forgotten [1]. Although almost all the project files have the ".c" extension, MAME is actually a C++ project. The source code's size is rather large - 110 Mbytes. Checking MAME with PVS-Studio was impossible before because it is built with MinGW on Windows. MinGW is a native software port of the GNU Compiler Collection (GCC) under Microsoft Windows [2]. It means that PVS-Studio has to provide correct support of the special features of the GCC syntax and special keywords. Support of MinGW has been available in PVS-Studio since version 4.70. It is not full yet, but it is enough to check most projects. MAME was one of the first projects to be analyzed. Note. While performing the analysis, there will be a lot of similar false reports. The odd code fragments are located in several macros widely used in various project parts. It seems at first that there are only false positives - scattered useful messages just get lost among them. However, you can easily fix it by adding just a few comments to suppress the warnings triggered by the macros. See the "Suppression of false alarms" section in the documentation to find out how to do that. Now let's study the errors we have detected. Incompletely cleared arrays As we have already said, you can find a lot of fragments in the MAME project where the memset function is used incorrectly. A typical mistake is filling only a part of an array. Consider a simple example: UINT32 m_pstars_regs[16]; static DRIVER_INIT( pstar ) { ... memset(state->m_pstars_regs, 0, 16); ... } PVS-Studio: V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_pstars_regs'. pgm.c 4458 Number 16 means the number of items in the "m_pstars_regs" array. But it is the number of bytes being filled in the buffer that should be passed into the memset function. As a result, only a part of the array is filled with zeroes. This is the correct code: memset(state->m_pstars_regs, 0, 16 * sizeof(UINT32)); The mistake is trivial. Programmers often think that there are few trivial errors in their programs (see the second myth [3]). It's not so. It is very simple and silly mistakes that make up the largest part of errors found in programs. Do you think the error shown above is a single one? No. Here you are at least 8 other fragments where instances of the same mistake can be found: V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_kb_regs'. pgm.c 4975 V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_kb_regs'. pgm.c 4996 V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_kb_regs'. pgm.c 5056 V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_oldsplus_ram'. pgm.c 5780 V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_oldsplus_regs'. pgm.c 5781 V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_sysreg'. rungun.c 399 V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_ttl_vram'. rungun.c 400 V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_playfield_code'. malzak.c 392 In the example above, the number of items was defined by an absolute number. It is bad. You'd better calculate the array size instead of using constants. Unfortunately, it doesn't help to avoid the error we're discussing. UINT16 m_control_0[8]; #define ARRAY_LENGTH(x) (sizeof(x) / sizeof(x[0])) static MACHINE_RESET( tumbleb ) { ... memset(state->m_control_0, 0, ARRAY_LENGTH(state->m_control_0)); } PVS-Studio: V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_control_0'. tumbleb.c 2065 The ARRAY_LENGTH macro is used to calculate the number of array items. Again, it's incorrect. The programmer should have calculated the array size, not number of items it contains. There are two ways to fix it. The first: memset(state->m_control_0, 0, sizeof(state->m_control_0)); The second: memset(state->m_control_0, 0, ARRAY_LENGTH(state->m_control_0) * sizeof(UINT16)); These are some other fragments where arrays are not filled correctly in the same way: V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_pmac_read'. megadriv.c 7156 V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_pmac_write'. megadriv.c 7157 V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_cart_is_genesis'. megatech.c 426 V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_vol_ctrl'. nycaptor.c 841 V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_rotate_ctrl'. wgp.c 949 V512 A call of the 'memset' function will lead to underflow of the buffer 'state->m_vreg'. othldrby.c 237 Misfortunes with the memset() function are over here, though I may have missed some mistakes. But it's time for another, as scary, function memcpy(). Incorrect use of the memcpy() function Let's look at a code that causes an array overrun: #define CHD_SHA1_BYTES 20 #define CHD_V4_HEADER_SIZE 108 #define CHD_MAX_HEADER_SIZE CHD_V4_HEADER_SIZE static chd_error header_read(...., chd_header *header) { UINT8 rawheader[CHD_MAX_HEADER_SIZE]; ... memcpy(header->parentsha1, &rawheader[100], CHD_SHA1_BYTES); ... } PVS-Studio: V512 A call of the 'memcpy' function will lead to the '& rawheader[100]' buffer becoming out of range. chd.c 1870 The 'rawheader' array consists of 108 bytes. We want to copy its contents from byte 100 on. The trouble is we will reach outside the array boundaries. We can copy only 8 bytes, yet 20 bytes are actually copied. Unfortunately, I don't know how to fix this code, since I'm not familiar with the program logic. When using the memset() function, it often happens that only a part of an array is filled. Correspondingly, when you use the memset() function, there may often be errors causing only a part of an array to be copied. Consider the following sample: UINT16 m_spriteram16[0x1000]; UINT16 m_spriteram16_buffered[0x1000]; static WRITE32_HANDLER( deco32_buffer_spriteram_w ) { deco32_state *state = space->machine().driver_data(); memcpy(state->m_spriteram16_buffered, state->m_spriteram16, 0x1000); } PVS-Studio: V512 A call of the 'memcpy' function will lead to underflow of the buffer 'state->m_spriteram16_buffered'. deco32.c 706 That's a small function. But it has an error. I think you have already guessed that multiplication by sizeof(UINT16) is missing. This is the correct code: memcpy(state->m_spriteram16_buffered, state->m_spriteram16, 0x1000 * sizeof(UINT16)); The same error here: V512 A call of the 'memcpy' function will lead to underflow of the buffer 'state->m_spriteram16_2_buffered'. deco32.c 726 Misprints and Copy-Paste In any project you can see misprints and errors caused by using the Copy-Paste technology. There are few of them in some projects and quite many in others. In MAME these errors are not numerous, yet they are there. Let's study some of them. static WRITE8_HANDLER( tms70x0_pf_w ) { ... if( ((cpustate->pf[0x03] & 0x80) == 0) && ((data & 0x80) == 0x80 ) ) { ... } else if( ((data & 0x80) == 0x80 ) && ((cpustate->pf[0x03] & 0x80) == 0) ) { ... } ... } PVS-Studio: V517 The use of 'if (A) {...} else if (A) {...}' pattern was detected. There is a probability of logical error presence. Check lines: 577, 584. tms7000.c 577 If you look close, you will notice that the first and the second conditions are identical. They have different order of comparisons, but it doesn't influence the result in any way. Consider the following example. class device_debug { device_disasm_interface *m_disasm; ... int min_opcode_bytes() const { return (m_disasm != NULL) ? m_disasm->max_opcode_bytes() : 1; } int max_opcode_bytes() const { return (m_disasm != NULL) ? m_disasm->max_opcode_bytes() : 1; } } PVS-Studio: V524 It is odd that the body of 'max_opcode_bytes' function is fully equivalent to the body of 'min_opcode_bytes' function (debugcpu.h, line 150). debugcpu.h 151 The max_opcode_bytes() function is identical to the min_opcode_bytes() function. This is most likely incorrect. I suppose that the min_opcode_bytes() function was intended to be written as follows: int min_opcode_bytes() const { return (m_disasm != NULL) ? m_disasm->min_opcode_bytes() : 1; } Here are some other code fragments which are most probably misprints: V583 The '?:' operator, regardless of its conditional expression, always returns one and the same value: ",(%d,". 9900dasm.c 670 V517 The use of 'if (A) {...} else if (A) {...}' pattern was detected. There is a probability of logical error presence. Check lines: 549, 579. cdrom.c 549 V501 There are identical sub-expressions 'offset != (0x370 >> 1)' to the left and to the right of the '&&' operator. decoprot.c 118 V501 There are identical sub-expressions 'offset != (0x3c0 >> 1)' to the left and to the right of the '&&' operator. decoprot.c 118 V501 There are identical sub-expressions 'offset != 0x2c / 2' to the left and to the right of the '&&' operator. decoprot.c 240 V501 There are identical sub-expressions 'offset != 0xe' to the left and to the right of the '&&' operator. decoprot.c 447 Undefined behavior Quite a number of warnings generated by PVS-Studio for this project refer to shift operations. These operations lead to undefined behavior. Of course, when you use particular compilers, your code can work properly for many years. That's why we can call these errors potential. They may reveal themselves when moving to a different platform, compilers or optimization switches. To learn more about it, please see the article: "Wade not in unknown waters. Part three." [4]. Consider a couple of samples causing undefined behavior. The first sample: #define ATARIRLE_PRIORITY_SHIFT 12 #define ATARIRLE_PRIORITY_MASK \ ((~0 < ATARIRLE_PRIORITY_SHIFT) & 0xffff) PVS-Studio: V610 Undefined behavior. Check the shift operator 'm_color1_mask |= 1 < i; ... } ... } PVS-Studio: V610 Undefined behavior. Check the shift operator 'pen_usage && transparency == STV_TRANSPARENCY_PEN) { ... } if( gfx ) { ... } ... } PVS-Studio: V595 The 'gfx' pointer was utilized before it was verified against nullptr. Check lines: 2457, 2483. stvvdp2.c 2457 At times I come across some strange code about which I cannot tell for sure if it has an error or not. Maybe there is a Copy-Paste mistake. And maybe everything is correct and the two code branches are actually intended to be identical. Here is an example: static DEVICE_START( deco16ic ) { ... if (intf->split) deco16ic->pf2_tilemap_16x16 = tilemap_create_device(device, get_pf2_tile_info, deco16_scan_rows, 16, 16, fullwidth ? 64 : 32, fullheight ? 64 : 32); else deco16ic->pf2_tilemap_16x16 = tilemap_create_device(device, get_pf2_tile_info, deco16_scan_rows, 16, 16, fullwidth ? 64 : 32, fullheight ? 64 : 32); ... } PVS-Studio: V523 The 'then' statement is equivalent to the 'else' statement. deco16ic.c 943 Regardless of the condition, one and the same action is performed. Here is another similar sample: int compute_res_net(int inputs, int channel, const res_net_info *di) { ... if (OpenCol) { rTotal += 1.0 / di->rgb[channel].R; v += vOL / di->rgb[channel].R; } else { rTotal += 1.0 / di->rgb[channel].R; v += vOL / di->rgb[channel].R; } ... } PVS-Studio: V523 The 'then' statement is equivalent to the 'else' statement. resnet.c 628 Conclusion As usual, I will stress that these are probably quite not all the errors that PVS-Studio can find in MAME. The task of this article is to show that PVS-Studio is learning to check crossplatform projects. To know how exactly you can integrate into the make-file, please see the documentation. You may also ask us if you have any troubles analyzing projects built with MinGW. P.S. Review of analysis results currently implies that you need the Visual Studio environment where you can open the report and study it. Manual analysis of the report is very effortful. Maybe we will make a special tool in future that will allow you to conveniently review the report and perform code navigation without having Visual Studio installed. References Wikipedia. MAME. http://www.viva64.com/go.php?url=900 Wikipedia. MinGW. http://www.viva64.com/go.php?url=901 Myths about static analysis. The second myth - expert developers do not make silly mistakes. http://www.viva64.com/en/b/0116/ Wade not in unknown waters. Part three. http://www.viva64.com/en/b/0142/
  10. This time we will discuss virtual inheritance in C++ and find out why one should be very careful using it. See other articles of this series: N1, N2, N3. Initialization of Virtual Base Classes First let's find out how classes are allocated in memory without virtual inheritance. Have a look at this code fragment: class Base { ... }; class X : public Base { ... }; class Y : public Base { ... }; class XY : public X, public Y { ... }; It's pretty clear: members of the non-virtual base class Base are allocated as common data members of a derived class. It results in the XY object containing two independent Base sub-objects. Here is a scheme to illustrate that: Figure 1. Multiple non-virtual inheritance. When we deal with virtual inheritance, an object of a virtual base class is included into the object of a derived class only once. Figure 2 shows the structure of the XY object in the code fragment below. class Base { ... }; class X : public virtual Base { ... }; class Y : public virtual Base { ... }; class XY : public X, public Y { ... }; Figure 2. Multiple virtual inheritance. It is at the end of the XY object that memory for the shared sub-object Base is most probable to be allocated. The exact implementation of the class depends on the compiler. For example, the classes X and Y may store pointers to the shared object Base. But as far as I understand, this practice is out of use nowadays. A reference to a shared sub-object is rather implemented through offset or as information stored in the virtual function table. The "most derived" class XY alone knows where exactly a sub-object of the virtual base class Base is to be allocated. That's why it is the most derived class which is responsible for initializing all the sub-objects of virtual base classes. XY constructors initialize the Base sub-object and pointers to it in X and Y. After that, all the remaining members of the classes X, Y and XY are initialized. Once the XY constructor has initialized the Base sub-object, the X and Y constructors are not allowed to re-initialize it. The particular way it will be done depends on the compiler. For example, it can pass a special additional argument into the X and Y constructors to tell them not to initialize the Base class. Now the most interesting thing which causes much confusion and a lot of mistakes. Have a look at the following constructors: X::X(int A) : Base(A) {} Y::Y(int A) : Base(A) {} XY::XY() : X(3), Y(6) {} What number will the base class's constructor take as an argument - 3 or 6? None! The constructor XY initializes the virtual sub-object Base yet does that implicitly. It is the Base constructor which is called by default. As the XY constructor calls the X or Y constructor, it doesn't re-initialize Base. That's why Base is not being called with an argument passed into it. Troubles with virtual base classes don't end here. Besides constructors, there are also assignment operators. If I'm not mistaken, the standard tells us that an assignment operator generated by the compiler may assign values to a sub-object of a virtual base class multiple times or once. So, you just don't know how many times the Base object will be copied. If you implement your own assignment operator, make sure you have prevented multiple copying of the Base object. The following code fragment is incorrect: XY &XY::operator =(const XY &src) { if (this != &src) { X::operator =(*this); Y::operator =(*this); .... } return *this; } This code leads to double copying of the Base object. To avoid this, we should add special functions into the X and Y classes to prevent copying of the Base class's members. The contents of the Base class are copied just once, in the same code fragment. This is the fixed code: XY &XY::operator =(const XY &src) { if (this != &src) { Base::operator =(*this); X::PartialAssign(*this); Y::PartialAssign(*this); .... } return *this; } This code will work well, but it still doesn't look nice and clear. That's the reason why programmers are recommended to avoid multiple virtual inheritance. Virtual Base Classes and Type Conversion Because of the specifics of how virtual base classes are allocated in memory, you can't perform type conversions like this one: Base *b = Get(); XY *q = static_cast(b); // Compilation error XY *w = (XY *)(b); // Compilation error A persistent programmer, though, will achieve that by employing the operator reinterpret_cast: XY *e = reinterpret_cast(b); However, the result will hardly be of any use. The address of the beginning of the Base object will be interpreted as a beginning of the XY object, which is quite a different thing. See Figure 3 for details. The only way to perform a type conversion is to use the operator dynamic_cast. But using dynamic_cast too often makes the code smell. Figure 3. Type conversion. Should We Abandon Virtual Inheritance? I agree with many authors that one should avoid virtual inheritance by all means, as well as common multiple inheritance. Virtual inheritance causes troubles with object initialization and copying. Since it is the "most derived" class which is responsible for these operations, it has to be familiar with all the intimate details of the structure of base classes. Due to this, a more complex dependency appears between the classes, which complicates the project structure and forces you to make some additional revisions in all those classes during refactoring. All this leads to new bugs and makes the code less readable. Troubles with type conversions may also be a source of bugs. You can partly solve the issues by using the dynamic_cast operator, but it is too slow and if you have to use it too often in your code it means that your project's architecture is probably very poor. Project structure can be almost always implemented without multiple inheritance. After all, there are no such exotica in many other languages, and it doesn't prevent programmers writing code in these languages from developing large and complex projects. We cannot insist on total refusal of virtual inheritance: it may be useful and convenient at times. But always think twice before making a heap of complex classes. Growing a forest of small classes with shallow hierarchy is better than handling a few huge trees. For example, multiple inheritance can be in most cases replaced by object composition. Good Sides of Multiple Inheritance OK, we now understand and agree with the criticism of multiple virtual inheritance and multiple inheritance as such. But are there cases when it can be safe and convenient to use? Yes, I can name at least one: Mix-ins. If you don't know what it is, see the book "Enough Rope to Shoot Yourself in the Foot" [3] A mix-in class doesn't contain any data. All its functions are usually pure virtual. It has no constructor, and even when it has, it doesn't do anything. It means that no troubles will occur when creating or copying these classes. If a base class is a mix-in class, assignment is harmless. Even if an object is copied many times, it doesn't matter: the program will be free of it after compilation. References Stephen C. Dewhurst. "C++ Gotchas: Avoiding Common Problems in Coding and Design". - Addison-Wesley Professional. - 352 pages; illustrations. ISBN-13: 978-0321125187. (See gotchas 45 and 53). Wikipedia. Object composition. Allen I. Holub. "Enough Rope to Shoot Yourself in the Foot". (You can easily find it on the Internet. Start reading at section 101 and further on).
  11. I'm going on to tell you about how programmers walk on thin ice without even noticing it. Let's speak on shift operators [lessthan][lessthan], >>. The working principles of the shift operators are evident and many programmers even don't know that using them according to the C/C++ standard might cause undefined or unspecified behavior. You can read the previous articles here: [1], [2]. Excursus to the history A bit of history first. The necessity of bit shifting operations is evident to any programmer. Anyone sooner or later faces the need to handle individual bits and bit masks. However, shift operators are much more popular among programmers than they should be. The reason is that you can multiply and divide numbers by powers of two. For example, the "X [lessthan][lessthan] 3" operation will multiply X by 8. In the past, the advantage of this number multiplication/division method lied in the speed of its work. I've just got a book from the dusty shelf with a description of assembler commands for processors from 8086 to 80486. I've found a table with the number of clock cycles necessary to perform various instructions. Multiplying a 16-bit register by a memory cell using the MUL instruction takes about 124-139 clock cycles on the 8086 processor! A shift of a 16-bit register by N digits using the SHL instruction takes 8+4*N clock cycles on the 8086 processor. That is, it will take 72 clock cycles at worst. You could get a noticeable speed gain by using various tricks handling bitwise operations when calculating arithmetic expressions. This is what became the reason for massively using shifts - first in assembler, and then in C and C++. The first C/C++ compilers were simple. You could get a performance gain by explicitly prompting the compiler to use a shift instead of multiplication or division instructions in certain places. As processors were developing, shift operators were of use for a long time. On the 80486 processor, multiplication now took about 26 clock cycles. Seems like it became much better, doesn't it? But a shift operator took just 3 clock cycles at that time and again appeared to be better than multiplication. Fortunately, most of these forced optimizations have been forgotten by now. First, compilers have become smarter and now use an optimal instruction set to calculate arithmetic expressions. Second, processors have undergone great changes too. Pipelines, branch predictions, register renaming and many other things have appeared. That's why an ordinary programmer of nowadays cannot tell for sure how much time will take the execution of a certain instruction. But it's clear that if some fragments of code are not ideal, you may not even notice it. The processor will split instructions into micro-instructions and start executing them in parallel. To be honest, I don't make out now how it all goes on there. I've come to the understanding that it's no longer reasonable to know all the subtleties starting with the Intel Pentium processor. So, I've concluded that one should not think that one knows better how to write optimized code and use shifts and bitwise operations wherever possible. It's not necessarily true that you can make the code faster than the compiler's optimizer can, but you can tell for sure that the program will become complicated and difficult to understand in that case. Everything said above doesn't mean that you cannot benefit from bitwise operations anymore. There are many interesting and useful tricks [3]; just don't get too fond of them. Undefined behavior It all began when I decided to create more diagnostics related to undefined behavior [4] and unspecified behavior [5] in PVS-Studio. It took me rather little time and effort to create a rule to detect incorrect use of shift operators. And after that I had to stop and think it over. It turned out that programmers are very fond of shifts. They use them in every way they can, which often leads to undefined behavior from the viewpoint of the coding standard. But theory is one thing and practice is another. Is there sense in persecuting code that has been faithfully serving you for many decades and gone through many compilers? That's a difficult question. Despite the code being incorrect, compilers adhere to some secret agreement and process it uniformly. After pondering over it for a long time, I finally decided to leave this diagnostic rule in PVS-Studio without making any exceptions to it. If there are too many complaints from users, maybe I will change my mind. However, perhaps users will be satisfied by the capability of disabling this diagnostic or use other methods of warning suppression. By the way, it is these painful thoughts that made me write the article. I hope that you will find the information I'm going to show you interesting and useful. So, let's see what the C++11 standard has to say about shift operators: The shift operators [lessthan][lessthan] and >> group left-to-right. shift-expression [lessthan][lessthan] additive-expression shift-expression >> additive-expression The operands shall be of integral or unscoped enumeration type and integral promotions are performed. 1. The type of the result is that of the promoted left operand. The behavior is undefined if the right operand is negative, or greater than or equal to the length in bits of the promoted left operand. 2. The value of E1 [lessthan][lessthan] E2 is E1 left-shifted E2 bit positions; vacated bits are zero-filled. If E1 has an unsigned type, the value of the result is E1 * 2^E2, reduced modulo one more than the maximum value representable in the result type. Otherwise, if E1 has a signed type and non-negative value, and E1*2^E2 is representable in the result type, then that is the resulting value; otherwise, the behavior is undefined. 3. The value of E1 >> E2 is E1 right-shifted E2 bit positions. If E1 has an unsigned type or if E1 has a signed type and a non-negative value, the value of the result is the integral part of the quotient of E1/2^E2. If E1 has a signed type and a negative value, the resulting value is implementation-defined. It is sad to read such texts. But don't worry - now we will study various issues by examples. The simplest case leading to undefined behavior is the situation when the right operand has a negative value. For example: int A = 10; int B = A
  12. This time I want to speak on the 'printf' function. Everybody has heard of software vulnerabilities and that functions like 'printf' are outlaw. But it's one thing to know that you'd better not use these functions, and quite the other to understand why. In this article, I will describe two classic software vulnerabilities related to 'printf'. You won't become a hacker after that but perhaps you will have a fresh look at your code. You might create similar vulnerable functions in your project without knowing that. STOP. Reader, please stop, don't pass by. You have seen the word "printf", I know. And you're sure that you will now be told a banal story that the function cannot check types of passed arguments. No! It's vulnerabilities themselves that the article deals with, not the things you have thought. Please come and read it. The previous post can be found here: Part one. Introduction Have a look at this line: printf(name); It seems simple and safe. But actually it hides at least two methods to attack the program. Let's start our article with a demo sample containing this line. The code might look a bit odd. It is, really. We found it quite difficult to write a program so that it could be attacked then. The reason is optimization performed by the compiler. It appears that if you write a too simple program, the compiler creates a code where nothing can be hacked. It uses registers, not the stack, to store data, creates intrinsic functions and so on. We could write a code with extra actions and loops so that the compiler lacked free registers and started putting data into the stack. Unfortunately, the code would be too large and complicated in this case. We could write a whole detective story about all this, but we won't. The cited sample is a compromise between complexity and the necessity to create a code that would not be too simple for the compiler to get it "collapsed into nothing". I have to confess that I still have helped myself a bit: I have disabled some optimization options in Visual Studio 2010. First, I have turned off the /GL (Whole Program Optimization) switch. Second, I have used the __declspec(noinline) attribute. Sorry for such a long introduction: I just wanted to explain why my code is such a crock and prevent beforehand any debates on how we could write it in a better way. I know that we could. But we didn't manage to make the code short and show you the vulnerability inside it at the same time. Demo sample The complete code and project for Visual Studio 2010 can be found here: . const size_t MAX_NAME_LEN = 60; enum ErrorStatus { E_ToShortName, E_ToShortPass, E_BigName, E_OK }; void PrintNormalizedName(const char *raw_name) { char name[MAX_NAME_LEN + 1]; strcpy(name, raw_name); for (size_t i = 0; name != '\0'; ++i) name = tolower(name); name[0] = toupper(name[0]); printf(name); } ErrorStatus IsCorrectPassword( const char *universalPassword, BOOL &retIsOkPass) { string name, password; printf("Name: "); cin >> name; printf("Password: "); cin >> password; if (name.length() < 1) return E_ToShortName; if (name.length() > MAX_NAME_LEN) return E_BigName; if (password.length() < 1) return E_ToShortPass; retIsOkPass = universalPassword != NULL && strcmp(password.c_str(), universalPassword) == 0; if (!retIsOkPass) retIsOkPass = name[0] == password[0]; printf("Hello, "); PrintNormalizedName(name.c_str()); return E_OK; } int _tmain(int, char *[]) { _set_printf_count_output(1); char universal[] = "_Universal_Pass_!"; BOOL isOkPassword = FALSE; ErrorStatus status = IsCorrectPassword(universal, isOkPassword); if (status == E_OK && isOkPassword) printf("\nPassword: OK\n"); else printf("\nPassword: ERROR\n"); return 0; } The _tmain() function calls the IsCorrectPassword() function. If the password is correct or if it coincides with the magic word "_Universal_Pass_!", then the program prints the line "Password: OK". The purpose of our attacks will be to have the program print this very line. The IsCorrectPassword() function asks the user to specify name and password. The password is considered correct if it coincides with the magic word passed into the function. It is also considered correct if the password's first letter coincides with the name's first letter. Regardless of whether the correct password is entered or not, the application shows a welcome window. The PrintNormalizedName() function is called for this purpose. The PrintNormalizedName() function is of the most interest. It is this function where the "printf(name);" we're discussing is stored. Think of the way we can exploit this line to cheat the program. If you know how to do it, you don't have to read further. What does the PrintNormalizedName() function do? It prints the name making the first letter capital and the rest letters small. For instance, if you enter the name "andREy2008", it will be printed as "Andrey2008". The first attack Suppose we don't know the correct password. But we know that there is some magic password somewhere. Let's try to find it using printf(). If this password's address is stored somewhere in the stack, we have certain chances to succeed. Any ideas how to get this password printed on the screen? Here is a tip. The printf() function refers to the family of variable-argument functions. These functions work in the following way. Some amount of data is written into the stack. The printf() function doesn't know how many data is pushed and what type they have. It follows only the format string. If it reads "%d%s", then the function should extract one value of the int type and one pointer from the stack. Since the printf() function doesn't know how many arguments it has been passed, it can look deeper into the stack and print data that have nothing to do with it. It usually causes access violation or printing trash. And we may exploit this trash. Let's see how the stack might look at the moment when calling the printf() function: Figure 1. Schematic arrangement of data in the stack. The "printf(name);" function's call has only one argument which is the format string. It means that if we type in "%d" instead of the name, the program will print the data that lie in the stack before the PrintNormalizedName() function's return address. Let's try: Name: %d Password: 1 Hello, 37 Password: ERROR This action has little sense in it for now. First of all, we have at least to print the return addresses and all the contents of the char name[MAX_NAME_LEN + 1] buffer which is located in the stack too. And only then we may get to something really interesting. If an attacker cannot disassemble or debug the program, he/she cannot know for sure if there is something interesting in the stack to be found. He/she still can go the following way. First we can enter: "%s". Then "%x%s". Then "%x%x%s" and so on. Doing so, the hacker will search through the data in the stack in turn and try to print them as a line. It helps the intruder that all the data in the stack are aligned at least on a 4-byte boundary. To be honest, we won't succeed if we go this way. We will exceed the limit of 60 characters and have nothing useful printed. "%f" will help us - it is intended to print values of the double type. So, we can use it to move along the stack with an 8-byte step. Here it is, our dear line: %f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%x(%s) This is the result: Figure 2. Printing the password. Click on the picture to enlarge it. Let's try this line as the magic password: Name: Aaa Password: _Universal_Pass_! Hello, Aaa Password: OK Hurrah! We have managed to find and print the private data which the program didn't intend to give us access to. Note also that you don't have to get access to the application's binary code itself. Diligence and persistence are enough. Conclusions on the first attack You should give a wider consideration to this method of getting private data. When developing software containing variable-argument functions, think it over if there are cases when they may be the source of data leakage. It can be a log-file, a batch passed on the network and the like. In the case we have considered, the attack is possible because the printf() function receives a string that may contain control commands. To avoid this, you just need to write it in this way: printf("%s", name); The second attack Do you know that the printf() function can modify memory? You must have read about it but forgotten. We mean the "%n" specifier. It allows you to write a number of characters, already printed by the printf() function, by a certain address. To be honest, an attack based on the "%n" specifier is just of a historical character. Starting with Visual Studio 2005, the capability of using "%n" is off by default. To perform this attack, I had to explicitly allow this specifier. Here is this magic trick: _set_printf_count_output(1); To make it clearer, let me give you an example of using "%n": int i; printf("12345%n6789\n", &i); printf( "i = %d\n", i ); The program's output: 123456789 i = 5 We have already found out how to get to the needed pointer in the stack. And now we have a tool that allows us to modify memory by this pointer. Of course, it's not very much convenient to use it. To start with, we can write only 4 bytes at a time (int type's size). If we need a larger number, the printf() function will have to print very many characters first. To avoid this we may use the "%00u" specifier: it affects the value of the current number of output bytes. Let's not go deep into the detail. Our case is simpler: we just have to write any value not equal to 0 into the isOkPassword variable. This variable's address is passed into the IsCorrectPassword() function, which means that it is stored somewhere in the stack. Do not be confused by the fact that the variable is passed as a reference: a reference is an ordinary pointer at the low level. Here is the line that will allow us to modify the IsCorrectPassword variable: %f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f%f %n The "%n" specifier does not take into account the number of characters printed by specifiers like "%f". That's why we make one space before "%n" to write value 1 into isOkPassword. Let's try: Figure 3. Writing into memory. Click on the picture to enlarge it. Are you impressed? But that's not all. We may perform writing by virtually any address. If the printed line is stored in the stack, we may get the needed characters and use them as an address. For example, we may write a string containing characters with codes 'xF8', 'x32', 'x01', 'x7F' in a row. It turns out that the string contains a hard-coded number equivalent to value 0x7F0132F8. We add the "%n" specifier at the end. Using "%x" or other specifiers we can get to the coded number 0x7F0132F8 and write the number of printed characters by this address. This method has some limitations, but it is still very interesting. Conclusions on the second attack We may say that an attack of the second type is hardly possible nowadays. As you see, support of the "%n" specifier is off in contemporary libraries by default. But you may create a self-made mechanism subject to this kind of vulnerabilities. Be careful when external data input into your program manage what and where is written into memory. Particularly in our case, we may avoid the problem by writing the code in this way: printf("%s", name); General conclusions We have considered only two simple examples of vulnerabilities here. Surely, there are much more of them. We don't make an attempt to describe or at least enumerate them in this article; we wanted to show you that even such a simple construct like "printf(name)" can be dangerous. There is an important conclusion to draw from all this: if you are not a security expert, you'd better follow all the recommendations to be found. Their point might be too subtle for you to understand the whole range of dangers on yourself. You must have read that the printf() function is dangerous. But I'm sure that many of you reading this article have learned only now how deep the rabbit hole is. If you create an application that is potentially an attack object, be very careful. What is quite safe code from your viewpoint might contain a vulnerability. If you don't see a catch in your code, it doesn't mean there isn't any. Follow all the compiler's recommendations on using updated versions of string functions. We mean using sprintf_s instead of sprintf and so on. It's even better if you refuse low-level string handling. These functions are a heritage of the C language. Now we have std::string and we have safe methods of string formatting such as boost::format or std::stringstream. P.S. Some of you, having read the conclusions, may say: "well, it's as clear as day". But be honest to yourself. Did you know and remember that printf() can perform writing into memory before you read this article? Well, and this is a great vulnerability. At least, it used to. Now there are others, as much insidious.
  13. This article is about C++. "We decided to write several small posts on how C/C++" - I mean, there will be several articles. Some will be about C++, the other about C/C++.