An Open Source pro math lib.

Started by
5 comments, last by Charles B 20 years, 1 month ago
My indy team and mostly the writer at the moment, has developped a powerful technology aimed at helping game developpers. It's of really optimal speed (*), based on a virtual SIMD (no, not mainly virtual C++ classes, ... of course) that lets one write a single code as if he had an enhanced portable SSE. And this will work on any platform. It beats my own hand written SIMD asm (**) since it blends everything available (intrisics, asm, compiler tricks) with the strengths of the compiler. It also beats the vectorizers though it can benefit of them. Well in one word : It blends human and computer intelligence to produce the best math code possible with max productivity atm. Now the technical constraints, since 95% is in headers for perfs(inline and defines), no black box is possible. So it's a non sense protecting the IP. Open Source seems to be the only way. I wish to receive some feedback here about the potential 'buisiness' issues : - Do you think we can get some support of the big industry ? For instance Apple and Motorola (Altivec) should encourage it. It would contribute to make PC 3D games painlessly portable on their OS/platforms. - Do you think a company could try to reap the main ideas from the headers and try to patent some elements or try to thieve the technology and put it into a black box ? (through a (plugable) compiler à la Intel compiler for instance) - Can I raise some support from the dev community ? Normally it's a typical win-win deal. The lib is made to be extended in size (new functions added) so that everyone will benefit of high speed asserted and benchmarked algorithms. For instance typical convex/convex col det routines, or root extraction, etc ... (*) Most vector or higher ops are in 1,3, or 10 cycles. Example : vAABoxPlaneDist is 4 cycles on SSE, 6 on 3DNow, 8 on 586 FPU. This routine shows human knowledge, SIMD, and compiler powers optimally combined. It's the standard example. (**) asm inlining is frozen cut and paste. The compiler can optimize C code more in the global context. Minimalize register pressure, eliminate unnecessary operations, etc ... [edited by - Charles B on March 6, 2004 3:25:17 AM]
"Coding math tricks in asm is more fun than Java"
Advertisement
Goodness knows about the business issues, but a question that does form in my head:

Q) Why is this better than a vectorising compiler ? (VectorC, VC++ 7.1 (vs.net 2003), GCC ??)

In answer to your qns:
1) no idea, you''d have to suck it and see - btw do you have any demos (benchmarks?) you could release - even better if it was with source (even if minus your library stuff)

2) Depends on the ethics of the company involved; see the whole SCO/linux/IBM mess; the problem is how would you know for anything closed source?

3) As in doing some kind of OSS dev type thing perhaps based off sourceforge.net ? that would be good.
Why is this better than a vectorising compiler ?

At start I have tortured codes, compiler options, preprocessing tricks, calling conventions in every way to challenge optimal handwritten optimal asm.

Then I found the relevant principles and parameters under the strict law of rdtsc. It really brings something new else I would not have spent nearly 2 months on it.

- benchmarks : every function is benchmarked/written in parallel under every environement settings. I started with it. The bench and precision tests are part of the lib (to help contributors). It also generates html docs and reports automatically. Plus a lot of ways to assert math in Debug under various defines.

- It's better than handwritten asm inlined when many basic vec ops are involved in a routine because it works in pair with the C/C++ compiler. It makes contextual optimizations and register allocation whereas with inline asm register roles and instruction scheduling are frozen.

- It beats vectorizers because there is some wisdom and SIMD logic in how functions such as xAABoxPlaneDist are written. When you write floating point code and then launch a vectorizer, there lacks something. The writer does not know precisely what's preferable for vectorization and SIMD.

Well in one word it blends coder knowledge and compiler capacities in the best possible balance. And this work is cached for user coders, helping productivity.

Some ideas about benches :
vAABoxPlaneDist = 3-4 cycles (SSE)
Sqrt8 : 8 cycles (ANSI C - 586 FPU)

There are 4 levels of coding for each function, between pseudo asm style and C++ OOP.


[edited by - Charles B on March 7, 2004 7:55:31 AM]
"Coding math tricks in asm is more fun than Java"
Technology licensing is totally different than selling middleware and you''re seem to be stuck in the middle. You''ll have to ask yourself: what kind of business am I in?

Technology licensing involves protecting your IP in a legal way (i.e. patents), and entering into licensing agreements to co-develop the technology into your customer''s applications (i.e. _their_ middleware).

Otherwise you are in the middleware business yourself; that means selling a product that encapsulates your technology . You will have to select in which middleware category (AI, physics, simulation, ...) your technology can generate a sustainable competitive advantage and then grow your offerings once you are dominating that part of the market. It''s a big undertaking, but it''s the only path than ensures your technology is protected if it cannot be patented.

Your third option is to throw in the towel and make it Open Source in the feeble attempt to gain some personal notoriety...
Technically, two reasonable possibilities (b1,b2) are :

a) make the lowest levels of the lib ''Lambda'' completly Open Source.
(Lambda is a pseudo name for this thread, Lambda is mostly centered at the virtual SIMD system)

b1) make some middleware on the top of Lambda.
- separate the higher level in a different library that uses Lambda. (for instance physics/collision detection)

or

b2) make commercial games using Lambda (and keep these col. det. routines for our team).

Now I have no precise idea of the practical or theorical consequences of (a+b1) or (a+b2) from a buisiness or legal perspective.

a+b1) For instance a) made public can be technically and, as a consequence, commercially a reinforcement of b1). Since some people are accustommed to a) they can more easilly try a demo of b1.

a+b2) We can upgrade the online game continuously by recompiling it with fresher versions of Lambda. So contributors benefit from our work and also help us in our regular buisiness.

"Coding math tricks in asm is more fun than Java"
> make the lowest levels of the lib ''Lambda'' completly Open Source.
> make some middleware on the top of Lambda

Again, what business are you in? and remember who you ultimately compete with...

You seem to think that the technology will sell all by itself once smart-enough game developers realize its advantages. To my knowledge, only a very small fraction will truly understand its meaning to game performance and explore it. At worst, it''s the middleware providers that could rip your technology open and integrate it into _their_ middleware; their APIs got faster without any hefty investment in API changes and documentation.



what business are you in ?
It''s not well defined in your categorization. There is no comparable equivalent in the traditionnal buisiness. We''re a group of indies. We all develop our own libs, hold the ips wether Open Source or proprietary. We make an indy game in the hope of promoting the various libs, in win-win cooperative deal. Each lib is owned by one or two member of the game project. The game will be sold through the www. The game code itself .. well it would be a bit tricky to explain, but anyway it''s another issue.



You seem to think that the technology will sell all by itself once smart-enough game developers realize its advantages.
No I am not that optimistic. I know how most companies produce games, the recurrent middleware issue, etc ...

That''s why we make a game to prove the efficiency of each of the components, the math lib highly included. We''ll also start a web site soon to expose this technology. The power of the math lib for instance will be shown through my terrain engine that will demonstrate things never seen before. When I''ll explain that this techno is made of portable code and one single source for every hardware, some ears may listen with more attention. Very high FPS on a TNT2 and a 500MHz machine, for a terrain looking better than other engines on the latest hw should be convincing enough.


At worst, it''s the middleware providers that could rip your technology open and integrate it into _their_ middleware;

Yes it''s possible. But if it''s open source (I don''t remember the exact licence) and they use it for commercial products, they''ll bypass law if they do not negotiate with the creators. The code generates highly reckonizable patterns in output machine code (asm). Many useful routines have only a limited number of conformations for optimal performances. It''s de facto a watermarking capacity. Then it''s easy to scan .exe or libs and seek more in detail when a hit is found. I know a counter attack could be you spied our code. But as you can see it''s autoimated, it''s not a harmful reverse engineering. Of course then lawyers battles is really another issue. But if it''s open source, rumours based on technical proofs about potential thefts can also be harmful for the offender.

their APIs got faster without any hefty investment in API changes and documentation.

Right but the goal is not to block such possibilities. The goal is balance and equity. For instance if they bring new features to the lib, why not. I am not planning some rapacious buisiness. My goal is also altruist, if I can help indy dev, I am happy. The more serious indy groups start up the more the global situation in the game buisiness may change. This would let more opportunities to indy developpers like us to find niches on the www buisiness.
"Coding math tricks in asm is more fun than Java"

This topic is closed to new replies.

Advertisement