• entries
7
• comment
1
• views
1110

# Compressing LDraw database

1102 views

One of the main goal for QLMesh was to add some new formats I have been working with quite often, like Photoshop files of bdf fonts.

For 3D it is LDraw formats and DAZ Studio models.

LDraw is one of my favourite. I am currently working on extending Assimp to support .ldr and .mpd files. One of the major challenge is actually not drawing but embedding library definitions into the plugin. Original library it is about 250MB (compressed to ~40MB). That's quite large for Quicklook plugin. I started to work on some heavy compression/optimalization and current result is:


-rw-r--r--  1 piecuchp  staff    40M May 12 17:18 parts.db
-rw-r--r--  1 piecuchp  staff   2.2M May 12 17:18 parts.db.gz

That's much better. 2MB can be easily embedded into plugin, eg. using assembler module like this:


bits 64

section .rodata

global _ldrawlib
global _ldrawlib_end
global _ldrawlib_size

_ldrawlib:      incbin "parts.db.gz"
_ldrawlib_end:
_ldrawlib_size: dd \$-_ldrawlib

and later build with e.g. nasm:

/opt/local/bin/nasm -fmacho64 ldraw_lib.asm -o ldraw_lib.o

PS1

Sometimes less is more. Working on reading gzip stream, I had to remove one of the compression optimisation. The uncompressed file is slightly bigger, but compressed one much smaller:

-rw-r--r--  1 piecuchp  staff    41M Jun 17 12:03 parts.db
-rw-r--r--  1 piecuchp  staff   1.5M Jun 17 12:03 parts.db.gz

PS2

Sadly, this is not the end of the story  I had to increase the precision of the float numbers in the database (it is now 17 bits - sign:8bit:8bit) - it increased the size but also significantly affected the compression ratio:

-rw-r--r--  1 piecuchp  staff    67M Jul 11 08:55 parts.db
-rw-r--r--  1 piecuchp  staff    41M Jul 11 08:55 parts.db.gz

Seems like I am gonna have to live with such database for a while.

There are no comments to display.

## Create an account

Register a new account

• ### Similar Content

Linear interpolation (sometimes called 'lerp' or 'mix') is a really handy function for creative coding, game development and generative art.
The function interpolates within the range [start..end] based on a 't' parameter, where 't' is typically within a [0..1] range.

For example, divide 'loop time' by 'loop duration' and you get a 't' value between 0.0 and 1.0.
Now you can map this 't' value to a new range, such as lerp(20, 50, t) to gradually increase a circle's radius, or lerp(20, 10, t) to gradually decrease its line thickness.

Another example: you can use linear interpolation to smoothly animate from one coordinate to another. Define a start point (x1, y1) and end point (x2, y2), then interpolate the 'x' and 'y' dimensions separately to find the computed point in between.

Or use linear interpolation to spring toward a moving target. Each frame, interpolate from the current value to the target value with a small 't' parameter, such as 0.05.
It's like saying: walk 5% toward the target each frame.

A more advanced example, but built on the same concept, is interpolating from one color (red) to another (blue).
To do this, we interpolate the (R, G, B) or (H, S, L) channels of the color individually, just like we would with a 2D or 3D coordinate.

Another quick example is to choose a random point along a line segment.
There are lots of ways to use linear interpolation, and lots more types of interpolation (cubic, bilinear, etc). These concepts also lead nicely into areas like: curves, splines and parametric equations.

Source code for each of these examples is available here: https://gist.github.com/mattdesl/3675c85a72075557dbb6b9e3e04a53d9

Matt DesLauriers is a creative coder and generative artist based in London. He combines code and emergent systems to make art for the web, print media, and physical installations.

Note:
This brief introduction to lerp was originally published as a Twitter thread and is republished here with the kind permission of the original author.

1.- So, I disassembly a raw binary(no ELF, PE, etc... headers) X64 assembly code and i got that result:

0: 66 89 c8 mov ax,cx 3: e8 00 00 00 00 call 8 <gh> 0000000000000008 <gh>: 8: 66 89 c2 mov dx,ax I understand how Byte Offset works('66' is the byte ID 0, '89' is 1, 'c8' is 2 and on 3 the call instruction starts(that is why '3:' is there)) but, by that logic, shouldn't 'call gh' be translated to 'e8 00 00 00 00 00 00 00 08' instead of 'e8 00 00 00 00' since the byte offset of the first instruction of gh, which is 'mov   dx, ax' is 8 and the output is 64 bits?

2.- Using the example of above, if endianness is little end., how the assembler would swap the bytes, by each instruction? Like:

Original, no endiannes { 66 89 c8 e8 00 00 00 00(in case that would be correct and i'm wrong in the question 1.-) 66 89 c2 } to { c8 89 66 00 00 00 00 e8 c2 89 66 } 3.- And then, the big end. would be like the original, without endiannes, code of the question 2.-?
4.- Suppose that i mark gh as .globl, then, would the assembler create a map table file where gh is in 'e8 00 00 00 00'(again, in case that would be correct and i'm wrong in question 1.-), and linker will look into such map file, and if another object file calls gh, the linker will then translate call gh as either 'e8 00 00 00 00'?
×