Jump to content

  • Log In with Google      Sign In   
  • Create Account

FREE SOFTWARE GIVEAWAY

We have 4 x Pro Licences (valued at $59 each) for 2d modular animation software Spriter to give away in this Thursday's GDNet Direct email newsletter.


Read more in this forum topic or make sure you're signed up (from the right-hand sidebar on the homepage) and read Thursday's newsletter to get in the running!


lib hell


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
12 replies to this topic

#1 fir   Members   -  Reputation: -456

Like
0Likes
Like

Posted 26 May 2014 - 04:15 AM

Im not quite see it clearly (got not so much experience) and maybe some will help me to claryfy this picture

 

Im mostly talking about windows platform (though I may be courious if on other platforms things looks better)

 

As far as i know there is no standarisation of lib (static libraries format)

so there we got some static lib-hell (my own name for this) where given static lib may only works with some compiler say VS or say GCC and not across them all (as it should be) [I am speeking on one specyfic system

mostly win32 or win64]

 

1) is this true?

2) do this incompatibility occur between across verisons of compilers 

or it only occurs between different compilers (I mean can lib not works if 

made with older version of compiller with the newer version of the same 

brand compiler or in one compiler it will work for all verions)

3) are the gcc/vs/intel/what else partially compatible - or in general mostly not compatible

4) is there a way of mending a incompatible lib in some way ?

 

tnx for the answer (it seem to me tah it may be hard to answer thiss fully but at least partially - i would like to build some wiev on this static lib - hell  in windows, how the things look like)

 

 

 



Sponsor:

#2 BitMaster   Crossbones+   -  Reputation: 4437

Like
2Likes
Like

Posted 26 May 2014 - 05:41 AM

I would strongly advise compiling a static library with the exact compiler and the exact build settings you intend to use it with. You might be able to work without that provided you have a pure C library, but I don't have significant experience in that area.

As soon as C++ enters the picture it's just not worth it. Even for the same version of the same compiler setting a different flag can break ABI-compatibility (for example the safe iterator define on earlier MSVCs would do that). That can lead to extremely annoying, very subtle, hell to find problems.
On the other hand the Windows version of gcc had one or two (unintentional) ABI breaks some time between 4.8.0 to 4.8.2. Check the changelogs for details.

Also note that is not a problem limited to static libraries. Unless you can make several restrictions regarding choice of runtime and other build settings an interface between dynamic libraries is also extremely limited (regarding allowed types and memory ownership).

On Windows you can usually get around compiling your own libraries if you stick to MSVC and you do not divert from the standard 'Debug' and 'Release' configurations. I would still suggest you get into the habit of compiling your own libraries though. You will need it sooner or later anyway and it's better to get skill points in compile-foreign-code without a really nasty deadline at your back.

#3 Jan2go   Members   -  Reputation: 956

Like
1Likes
Like

Posted 26 May 2014 - 06:33 AM

1) is this true?

Yes.

2) do this incompatibility occur between across verisons of compilers 
or it only occurs between different compilers (I mean can lib not works if 
made with older version of compiller with the newer version of the same 
brand compiler or in one compiler it will work for all verions)

Incompatibilities between different compilers are quite likely. It may work when using only different version of the same compiler (VS doesn't allow it though, don't know about other compilers).

3) are the gcc/vs/intel/what else partially compatible - or in general mostly not compatible

From what I know Intel is compatible with VS (not vice versa though). And I was able to use a MSVC dll with MinGW, however in most cases this will not work properly.

4) is there a way of mending a incompatible lib in some way ?

If you have access to the source code you may try to compile it with the compiler you need. If you don't have the source you are out of luck in the most cases.

#4 imoogiBG   Members   -  Reputation: 1247

Like
1Likes
Like

Posted 26 May 2014 - 10:52 AM


As far as i know there is no standarisation of lib (static libraries format)

 

*.lib/a file is just an archive that contains *.obj files (+ eventually some minimalistic metadata header file). 

EDIT: And those obj files are passed to the linker. So YES the sample compilation settings is a *MUST(only few exceptions are possible. I personally don't allow them in my build system, It is obvious that they will make you kill yourself).


Edited by imoogiBG, 26 May 2014 - 04:20 PM.


#5 SeanMiddleditch   Members   -  Reputation: 7261

Like
3Likes
Like

Posted 26 May 2014 - 05:09 PM

It's not usually considered a "static lib hell" because you have almost no good reason to actually try mixing different .lib files. Linux makes it more common, but that's only because of the walled garden repository approach of most Linux distros (if they recompile anything in the repository, they usually recompile every single package in the entire distro for the repository, too). One of the many arguments Linux folks try to use to bar commercial/binary software is that it may be compiled against different versions of static libs (and then you run into problems with IPC protocol version changes, data version changes, etc.). Really, on any OS, the only static libs you should try to use at all are the ones you compiled yourself for that application, and you should keep those to an absolute minimum (especially if the libs need to interact with the wider system in any way).

 

Note that .lib files on Windows are a little bit more tricky due to how .DLLs work. Older .lib files that are shims for a .DLL will typically work in newer compilers. This is because the .lib contains (almost) no code.

 

If anything, the real hell with Windows libraries it that there's no standard for architecture names and configurations.  e.g. one lib might have its 32-bit debug name be foo32d.lib while another is just bar.lib while their 64-bit release names are foo.lib and bar64r.lib. It makes it harder to have a single unified project configuration. This is true for DLLs as well. Nuget for C++ (CoApp) for Windows is an attempt to solve this that still has a lot of ground to cover.



#6 BGB   Crossbones+   -  Reputation: 1554

Like
2Likes
Like

Posted 26 May 2014 - 05:30 PM

Im not quite see it clearly (got not so much experience) and maybe some will help me to claryfy this picture
 
Im mostly talking about windows platform (though I may be courious if on other platforms things looks better)
 
As far as i know there is no standarisation of lib (static libraries format)
so there we got some static lib-hell (my own name for this) where given static lib may only works with some compiler say VS or say GCC and not across them all (as it should be) [I am speeking on one specyfic system
mostly win32 or win64]
 
1) is this true?

 
simple answer: yes.
 
 
less simple answer: there is some informal/de-facto standardization of LIB and OBJ file-formats, at least as far as Windows goes (as AR archives and COFF objects, known as OBJ/LIB with MSVC and O/A with GCC). however different compilers have various minor format differences (in terms of structures both within LIB/AR and the COFF objects), making things problematic.

some compilers had also used different formats entirely (for example, a few older compilers had used a 32-bit version of the OMF format and its associated LIB format instead of COFF/AR). there are also differences in the contents of various header-files (ex: what exactly is a "FILE"?), there are differences in terms of parts of the C ABI, and often a very different C++ name mangling scheme and ABI (so C++ object-files are usually incompatible).
 
the end result is that while sometimes it is possible to force things to link between compilers, usually the result will either not work (at all or at least not correctly) and/or be prone to crashing.
 
 

2) do this incompatibility occur between across verisons of compilers 
or it only occurs between different compilers (I mean can lib not works if 
made with older version of compiller with the newer version of the same 
brand compiler or in one compiler it will work for all verions)

it can apply between compiler versions for the same compiler as well.

for example, both MSVC and GCC have changed their ABIs in various ways between compiler versions, so code linked between different compiler versions is prone to not work and/or result in crashes.

this is generally less of an issue for DLLs, as the basic C calling conventions (cdecl and stdcall) are pretty much frozen, and also both MSVC and GCC agree in most areas WRT the basic C ABI, and MS is adverse to changing things which will break existing software.

however, the C++ ABI is compiler specific, and there are a lot of minor edge cases where the C ABI differs between compilers (namely WRT passing/returning structs, the handling of types like "long double" and "__m128", ...), so some care is still needed.

it may also be observed that many/most Windows libraries tend to use a fairly constrained C subset for their APIs (and COM and similar are built on top of the C ABI).

 

3) are the gcc/vs/intel/what else partially compatible - or in general mostly not compatible
4) is there a way of mending a incompatible lib in some way ?
 
tnx for the answer (it seem to me tah it may be hard to answer thiss fully but at least partially - i would like to build some wiev on this static lib - hell  in windows, how the things look like)

when possible, build things from source all with the same version of the same compiler.

Edited by BGB, 26 May 2014 - 05:35 PM.


#7 fir   Members   -  Reputation: -456

Like
-1Likes
Like

Posted 27 May 2014 - 02:43 AM

so it looks bad..

 

Is there any physical reason against the making lib (static libs format) (saying on one platform) working across all compilers and verisons?

 

Seem no.. (at least speaking about c, I am using c interface libraries only)

 

so this is bad, and this is guilt of compiler society - it should be standarized and this hell wouldnt be existing - every lib for win32 would be working with every compiler - i would call it normal situation 

 

[as to compiling libs from sources.. you seem to suggest everyone is doing that? I am not good at it.. Probably it brings its own bag of obstacles in the box , ?]

 

as to reasons (when speaking about c interface libs, I am only use) I understand the main source of incompatibility are just the symbol names (underscoress and so), Or there is something other above that?          I assume most compilers use coff libs today, old borland omf I incidentally know a bit because back then i was using b55 heavy by a couple of years, but (Im not sure) isnt it is conversable to coff with some tool (agner for as i vaguelly remember had such a tool 

on his page)

 

So as i said I do not see it clearly but It vaguelly seems to me that reasons of this incompatibility between static libs are not so much Heavy.. Is there something more than can be spoiled above just the linker symbol names? is someone here able to answer this?



#8 frob   Moderators   -  Reputation: 22787

Like
2Likes
Like

Posted 27 May 2014 - 09:21 PM

In practice it isn't a big deal.

Even when you are using the same compiler, you can have the problems when you build it twice with incompatible options.

Build everything from the source yourself. If you have a system that you cannot build from source (which is often a bad idea) then do what you can to remain compatible while trying to get the source. Note that even the major compilers provide source code to their standard libraries so you can build them from source if you need.

Check out my book, Game Development with Unity, aimed at beginners who want to build fun games fast.

Also check out my personal website at bryanwagstaff.com, where I write about assorted stuff.


#9 Hodgman   Moderators   -  Reputation: 31939

Like
2Likes
Like

Posted 27 May 2014 - 09:45 PM


In practice it isn't a big deal.
Assuming you've got the source to everything cool.png

 

If you're using closed-source C++ libraries, you need to be sure that the vendor has compiled it using the exact same compiler (and compiler options) to what you need...

It would probably be better to just ask the vendor to give you a C version of their library instead laugh.png



#10 frob   Moderators   -  Reputation: 22787

Like
0Likes
Like

Posted 27 May 2014 - 11:55 PM

In practice it isn't a big deal.

Assuming you've got the source to everything :cool: 
In two decades, with teams as small as 5 people, I have not seen it as an actual problem. Maybe it is a problem for other people.

It is easily avoided, either by getting contracts that require source code or by using major reputable companies that provide builds in all shapes and sizes, and as part of the contract will provide them for you as needed.

Games have a fun feature that approximately zero major games use the standard memory allocation systems. External libraries that fail to use custom allocators for everything tend to cause no end of problems, so 3rd parties typically ship with source for that reason alone.


I only remember one time (in 21 years) not having either source or compatible builds was an issue, and it was a simple matter: We told our boss (the studio owner) that we could not use the product due to lack of source and to cancel the contract; we were going to cut the features that relied on the library. The following day their technology lead contacted our lead programmer directly begging for us to take them back (apparently they only had 2-3 total customers), and after passing it on to the company owner that we could use the product if we were allowed to view and potentially modify the source, we not only got the source but negotiated about half the cost as well.

Check out my book, Game Development with Unity, aimed at beginners who want to build fun games fast.

Also check out my personal website at bryanwagstaff.com, where I write about assorted stuff.


#11 fir   Members   -  Reputation: -456

Like
0Likes
Like

Posted 28 May 2014 - 02:32 AM

 


In practice it isn't a big deal.
Assuming you've got the source to everything cool.png

 

If you're using closed-source C++ libraries, you need to be sure that the vendor has compiled it using the exact same compiler (and compiler options) to what you need...

It would probably be better to just ask the vendor to give you a C version of their library instead laugh.png

 

well maybe i do not wrote it clearry but I was asking about c interface libs? you talking about c++?

 

As i said It seem to me that this c (c interface) libs are also incompatible (has incompatible interfaces?) but also I suspect that this incompatibility  is not major - isnt this just a metter of symbols?

specifically i wold be most curious what is a reason between incompatibility of c - like static libs made with VS against those of GCC/mingw



#12 fir   Members   -  Reputation: -456

Like
0Likes
Like

Posted 28 May 2014 - 02:45 AM

 

 

In practice it isn't a big deal.

Assuming you've got the source to everything cool.png
In two decades, with teams as small as 5 people, I have not seen it as an actual problem. Maybe it is a problem for other people.
 

 

This big is relative ;/ I understand what you say , but still maybe it is not very big deal but still some deal ;\ Really static libs are done to work, as i said if they could work it is bad that they do not work ;\

 

Also it is really so easy to complie sources?, i am using mingw  (presently i dont even know how to use make) Is there easy to compile sources made with VS, they probably use different build systems or something.. I got not a practice in it

(also some sources (for quite interestin things) are made with some old compilers, i remember vaguelly i was trying to get some lib for c-interpreter (so I could use c scripts, it was fun idea, big advantages) wasnt it cint or something? but it was old and I get no skills to compile it, if this would be just lib (even old) i could use it, but old (and messy as big sources are always not redeable and undertstandable to me) sources 

seemed unusable to me

 

what with that ? ;/



#13 kunos   Crossbones+   -  Reputation: 2207

Like
3Likes
Like

Posted 28 May 2014 - 03:11 AM

 

i am using mingw  (presently i dont even know how to use make) Is there easy to compile sources made with VS, they probably use different build systems or something.. I got not a practice in it

 

 

these problems are easy to predict. Choosing a minor and unused platform (mingw in this case) also means being ready to fight.

You choose a minor platform and here you are complaining about it.. it's all in your doing. If you were using what everybody else is using on Windows, you wouldn't even notice the problem.

I don't know which particular lib are you trying to move over to MinGW, some are compatible with it (for example I use glfw and glew with MinGW because I need to link with Go programs... it's hacky to be gentle), some are not designed to be build with anything but VS.

 

Anyway.. none of this would be a problem if you were using the standard for Windows development, which is Visual Studio.. so, at the end of the day, it's all self inflicted pain you are complaining about.


Stefano Casillo
Lead Programmer
TWITTER: @KunosStefano
AssettoCorsa - netKar PRO - Kunos Simulazioni




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS