Jump to content
  • Advertisement
Sign in to follow this  
  • entries
    7
  • comment
    1
  • views
    1110

Compressing LDraw database

Sign in to follow this  
piecuch.p

1102 views

One of the main goal for QLMesh was to add some new formats I have been working with quite often, like Photoshop files of bdf fonts.

For 3D it is LDraw formats and DAZ Studio models.

LDraw is one of my favourite. I am currently working on extending Assimp to support .ldr and .mpd files. One of the major challenge is actually not drawing but embedding library definitions into the plugin. Original library it is about 250MB (compressed to ~40MB). That's quite large for Quicklook plugin. I started to work on some heavy compression/optimalization and current result is:

 
-rw-r--r--  1 piecuchp  staff    40M May 12 17:18 parts.db
-rw-r--r--  1 piecuchp  staff   2.2M May 12 17:18 parts.db.gz

That's much better. 2MB can be easily embedded into plugin, eg. using assembler module like this:

 
bits 64

section .rodata

global _ldrawlib
global _ldrawlib_end
global _ldrawlib_size

_ldrawlib:      incbin "parts.db.gz"
_ldrawlib_end:
_ldrawlib_size: dd $-_ldrawlib

and later build with e.g. nasm: 

/opt/local/bin/nasm -fmacho64 ldraw_lib.asm -o ldraw_lib.o

 

PS1

Sometimes less is more. Working on reading gzip stream, I had to remove one of the compression optimisation. The uncompressed file is slightly bigger, but compressed one much smaller:

-rw-r--r--  1 piecuchp  staff    41M Jun 17 12:03 parts.db
-rw-r--r--  1 piecuchp  staff   1.5M Jun 17 12:03 parts.db.gz


PS2

Sadly, this is not the end of the story :) I had to increase the precision of the float numbers in the database (it is now 17 bits - sign:8bit:8bit) - it increased the size but also significantly affected the compression ratio:

-rw-r--r--  1 piecuchp  staff    67M Jul 11 08:55 parts.db
-rw-r--r--  1 piecuchp  staff    41M Jul 11 08:55 parts.db.gz

Seems like I am gonna have to live with such database for a while.

Screen Shot.jpg

Mech2.jpg

Sign in to follow this  


0 Comments


Recommended Comments

There are no comments to display.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Advertisement
  • Advertisement
  • Blog Entries

  • Similar Content

    • By jbadams
      Linear interpolation (sometimes called 'lerp' or 'mix') is a really handy function for creative coding, game development and generative art.
      The function interpolates within the range [start..end] based on a 't' parameter, where 't' is typically within a [0..1] range.

       
      For example, divide 'loop time' by 'loop duration' and you get a 't' value between 0.0 and 1.0.
      Now you can map this 't' value to a new range, such as `lerp(20, 50, t)` to gradually increase a circle's radius, or `lerp(20, 10, t)` to gradually decrease its line thickness.

       
      Another example: you can use linear interpolation to smoothly animate from one coordinate to another. Define a start point (x1, y1) and end point (x2, y2), then interpolate the 'x' and 'y' dimensions separately to find the computed point in between.

       
      Or use linear interpolation to spring toward a moving target. Each frame, interpolate from the current value to the target value with a small 't' parameter, such as 0.05.
      It's like saying: walk 5% toward the target each frame.

       
      A more advanced example, but built on the same concept, is interpolating from one color (red) to another (blue).
      To do this, we interpolate the (R, G, B) or (H, S, L) channels of the color individually, just like we would with a 2D or 3D coordinate.

       
      Another quick example is to choose a random point along a line segment.
      There are lots of ways to use linear interpolation, and lots more types of interpolation (cubic, bilinear, etc). These concepts also lead nicely into areas like: curves, splines and parametric equations.

       
      Source code for each of these examples is available here: https://gist.github.com/mattdesl/3675c85a72075557dbb6b9e3e04a53d9
       
      About the author:
      Matt DesLauriers is a creative coder and generative artist based in London. He combines code and emergent systems to make art for the web, print media, and physical installations.
       
      Note:
      This brief introduction to lerp was originally published as a Twitter thread and is republished here with the kind permission of the original author.
       
       
    • By Iris_Technologies
      I had some doubts about hex formats(assembler output) and linkers:

      1.- So, I disassembly a raw binary(no ELF, PE, etc... headers) X64 assembly code and i got that result:

       
      0: 66 89 c8 mov ax,cx 3: e8 00 00 00 00 call 8 <gh> 0000000000000008 <gh>: 8: 66 89 c2 mov dx,ax I understand how Byte Offset works('66' is the byte ID 0, '89' is 1, 'c8' is 2 and on 3 the call instruction starts(that is why '3:' is there)) but, by that logic, shouldn't 'call gh' be translated to 'e8 00 00 00 00 00 00 00 08' instead of 'e8 00 00 00 00' since the byte offset of the first instruction of gh, which is 'mov   dx, ax' is 8 and the output is 64 bits?
       
      2.- Using the example of above, if endianness is little end., how the assembler would swap the bytes, by each instruction? Like:

       
      Original, no endiannes { 66 89 c8 e8 00 00 00 00(in case that would be correct and i'm wrong in the question 1.-) 66 89 c2 } to { c8 89 66 00 00 00 00 e8 c2 89 66 } 3.- And then, the big end. would be like the original, without endiannes, code of the question 2.-?
      4.- Suppose that i mark gh as .globl, then, would the assembler create a map table file where gh is in 'e8 00 00 00 00'(again, in case that would be correct and i'm wrong in question 1.-), and linker will look into such map file, and if another object file calls gh, the linker will then translate call gh as either 'e8 00 00 00 00'?
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!