Jump to content
  • Advertisement

nemequ

Member
  • Content Count

    8
  • Joined

  • Last visited

Community Reputation

149 Neutral

About nemequ

  • Rank
    Newbie

Personal Information

  • Interests
    Programming

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. It's been a while since I worked with one of these, but I remember the protocol being quite simple… Basically, every so often you poll the device. The poll includes a couple bytes of data; one of them tells the bill acceptor what bills you want to enable acceptance of, the other instructs the bill acceptor to do with bills (escrow it in the acceptor, move it to the stacker, or return it). The response from the acceptor will tell you the status such as if there is a bill in escrow and what its denomination is. If you're curious about the details, Pyramid has a relatively good document on their web site explaining the protocol; lots of bill and coin acceptors use the same protocol (I seem to remember that it was created by CashCode, but I could be mistaken), but I used Pyramid's documentation to write my implementations. I'm not saying you should roll your own implementation; if they already have a library you can use that sounds like a good idea. Rather, I'm trying to explain why you'll need to regularly interact with it. I'm not familiar with Unity, but you can probably do that by creating a background thread or integrating it into an event loop.
  2. // Custom types#if !defined(abc_global)#define abc_global static#define abc_inline inline#define abc_internal static#endif"single-header library" is a bit ambiguous; is it a library with 1 or more C files and a single header, or is the entire library contained in the header? Assuming the latter, all your functions should be static, and maybe some should be inline too (but keep in mind "inline" is a C99 thing, so if you want to support C89 you'll need to hide it behind an ifdef). If you omit the "static" you're likely to end up with collisions if multiple files include your library. abc_internal is pretty misleading. Usually when people see "internal" they think something marked with __attribute__((visibility("hidden"))) (GCC-like compilers); i.e., usable by the entire library/executable (not just the current compilation unit), but the symbol isn't exposed publicly to other code. I'm not sure what "global" is supposed to indicate. #if !defined(abc_u64)typedef unsigned long long abc_u64;...#endifThat's wrong on a lot of platforms. uint64_t is unsigned long long on Windows and OS X, but it's usually unsigned long on other platforms. This is a particularly infuriating issue because they're *mostly* compatible (same size, alignment, etc.), so the compiler won't generally warn you, except for when it will (like when you're working with pointers to abc_u64 instead of an actual abc_u64). in general you should use just use stdint.h, as others have mentioned. It's really nice to be able to use a standard name instead of having to remember a different name for each API, which may or may not be compatible with what you expect. The major exception is if you need the library to work with versions of MSVC older than 2013, since Microsoft didn't actually ship stdint.h (or a lot of other C99 features) until then. If that's the case, you should typedef your type to unsigned __int64 on MSVC, not unsigned long long. Unfortunately a lot of people stick to ancient versions… I routinely encounter people using 8.0 (circa 2005) and older. How much patience you have for them is up to you; mine is just about worn out. For everyone else it's usually pretty safe to assume stdint.h is available, but you can always check __STDC_VERSION__ and possibly the compiler version and/or libc version. If that's still not enough, the best solution I know of is to use limits.h to try to find a type with MIN/MAX values equal to what you expect, but that opens you back up to the issue of incompatible types. One very common convention (used by glib, among lots of others), is CamelCase for types, UPPER_CASE for macros, and lower_case_with_underscores for functions. So you would have #define ABC_FOO 1729 typedef struct { uint64_t value } AbcBar; void abc_baz(void); Mixing notations (i.e., abc_MyAwesomeFunction instead of AbcMyAwesomeFunction, or abc_FOO instead of ABC_FOO) is pretty uncommon (and, IMHO, very annoying). Sure it is, C just has different name spaces for different types of identifiers. You just need to do something like struct MyStruct { uint32_t value; }; void my_struct_set_value(struct MyStruct* my_struct, uint32_t value) { my_struct->value = value; } Note the "struct MyStruct" instead of just "MyStruct" on the function. Personally, I prefer to create typedefs, but there is something to be said for avoiding them; it does help improve code readability a bit. When you see "struct Foo", "enum Bar", or "union Baz" you at least have a good idea of what to expect. That's especially true for "enum Bar" since by convention the values will start with "BAR_". The reason I prefer typedefs is consistency; you can't really create aliases for structs in C other than typedefs, so there are situations where you have no choice but to *not* have the struct/union/enum keyword. Those cases aren't all that common, but IMHO consistency is much more important than avoiding typedefs.
  3.   I run everything directly from disk (using memory mapped files), but I use CPU time not wall clock so I/O shouldn't really be a big factor.  Loading everything into memory isn't really feasible since several of the machines don't have enough memory to do it—even when using mmap for the input and output several of them end up thrashing pretty badly (which kills wall-clock time but doesn't significantly alter CPU time).  I think trying to get I/O out of the equation is really the only sane thing to do, since the costs vary so much.  Once you get to that point you should really be doing your own benchmarks (which, BTW, Squash makes very easy).     Yep, you might want to take a look at LZHAM and Brotli—they are both designed to be very asymmetrical with faster decompression speeds.  Brotli is more of a work in progress right now, but in some time it could be pretty nice.
  4.   Because the existing corpora suck—they don't include most of the things people are actually compressing these days, which makes them of limited use for benchmarking.  I want a corpus that most people can use to get a decent idea of what codecs might be appropriate for their use case, and in order to do that I need a reasonably diverse corpus which is likely to contain some data similar to what they need to compress.   Take a look at The Squash Benchmark, which will probably be the first user for the new corpus—it's also probably the most exhaustive compression benchmark around (others have more codecs, but aren't run on nearly as many data sets or machines).  It currently contains data from three corpora (Silesa, Canterbury, and enwik8), plus some extra data from Snappy's tests because the standard corpora didn't cover it.  28 data sets in all, and its still missing very common things like log files, RPC requests/responses, ODF/OOXML files, and data from games like compressed textures and 3D models.  It has a lot of plain text, though almost all of it is in English, but how often to you need to compress a plain text file, relatively speaking (not JSON, XML, or anything like that… things like the collected works of Charles Dickens)?     It seems like you're assuming here that I'm trying to create a new compression codec, I'm not.  That said, I disagree with a lot of the stuff here, so…   First off, there are definitely trade-offs between compression speed, decompression speed, ratio, memory usage, but it's not like there is some equation where you plug in the compression ratio and decompression speed and get back the compression speed.  The type of data you're trying to compress plays a huge role in what codec to choose, as does the architecture.  Going back to the Squash Benchmark, look at some different pieces of data.  The performance varies wildly based on the type of data.  Check out density's speed for enwik8 on x86_64, then look at what happens when you feed it binary data like x-ray, or already compressed data like fireworks.jpeg.   You mention 7zip, which is basically LZMA.  Compare that to LZHAM—similar ratios and compression speeds, but LZHAM has much better decoding speed.  LZHAM was actually designed for games, so they could achieve LZMA-like compression but still be fast enough to decompress without too big of a performance hit.  Brotli is in much the same situation as LZHAM, though performance is a bit more variable.  Obviously there is room for improvement over LZMA in terms of speed.  Now look at ZPAQ and bsc—they both dominate LZMA in terms of ratio, and bsc isn't prohibitively slow.  Again, plenty of room for improvement over LZMA.   Sticking with games, take a look at RAD Game Tools' Oodle.  They have plenty of customers, so obviously there is a demand for good compression codecs from game studios.     It kind of depends on your definition of "everybody".  You might want to take a look at the README in that project—basically I'm trying to get something that 99% of people will find useful, at least as a starting point.  I'm ignoring plenty of use cases, like DNA databases, geological data, oil exploration, etc.  These are actually big users of compression who pay for a lot of the development, but like I mentioned by goal isn't to create a new codec, it's something to help most people decide on one.     That would be nice—then I could just abandon the project without remorse.  Unfortunately it's not the case :(
  5.   wav->mp3/ogg/opus is outside the scope of what I'm working on (which is lossless, general purpose compression).  Honestly I think lossy audio compression is more interesting, but it's not what the corpus is for.  I've filed an issue about 3D models, and I'll start trying to find some information about the formats different engines use—hopefully add some of the output from one of them to the corpus.  My current idea is to try to get a 3D model from one of the Blender foundation's open movie projects, then figure out how to get that exported as an asset from Unity or something.     I suspected an NDA was why you didn't just come out and say it in you initial post; thanks for taking the time to research this, I have a feeling a lot of people in the compression community weren't aware, maybe we can get some better encoders.   According to http://www.redgamingtech.com/xbox-one-sdk-leak-part-3-move-engines-memory-bandwidth-performance-tech-tribunal/ the LZ is LZ77.  It also seems to imply that it is either DEFLATE, zlib, or gzip (though I'm not sure how reliable that article is…) which would make sense.  If that's true, I would definitely be looking at zopfli for things you only have to compress when building a release but decompress regularly.     Right, but I don't really think much can be done about the issue.  Using that feature will obviously restrict people to the supported codecs, and people will probably be more interested in in writing improved implementations of the relevant codecs, but AFAICT from the perspective of developing a corpus nothing changes; you are still using the same type of data.
  6. Good point—in this case I'm definitely more interested in the published files.  The development files could be an interesting data set, but the published files are important to a lot more people.  I'm not trying to create a corpus with every different type of data available, I'm trying to create something which will be relevant for the vast majority of people.   As for memory mapped files, I think the only reasonable thing to is ignore the issue.  Either such cases cannot benefit from compression or the algorithm is constrained by the hardware, so there isn't really anything we can do here but focus on the use cases which could be served by compression.   The fact that LZW is used for that is interesting—thanks for that.  I've been generally uninterested in adding LZW support to Squash, I'll have to rethink that.  Any chance you could share the names of some of the platforms which can decode LZW like that?
  7. Compressed textures are definitely something I'm interested in including, though I'm a bit unsure about the format…  it seems to me (very much an outsider) that things are moving away from S3TC towards something like ETC2 or ASTC, perhaps it would be better to use one of those?   As for audio, do you compress the OGG/MP3 (or, trying to look towards the future again, Opus), or are you just talking about the compression being the WAV->OGG/MP3 encoding process?  If you do compress the OGG/MP3, is that just for the installer or also for your installed data?   3D model data seems like it would be a very good idea, but I don't understand the "processed through a pipeline, resulting in fitting data for the game engine)" part.  Does each game engine have its own format, and that is what you compress?   Finally, for all this stuff, is it all lumped together in some sort of container file which is then all compressed/decompressed at once, or is each file compressed independently?
  8. Sorry, to be clear I'm not looking for more implementations of compression algorithms—well, I am, but that's not what this thread is about.  What I am trying to find is the type of data that game developers typically need to compress so I can include it in a set of data I'm putting together to test different algorithms.   In addition to being used by developers to judge which compression algorithm(s) might be appropriate for them, people developing codecs will almost certainly be using this data to help tune their compressors, so getting something like the type of data you want to compress into this corpus should help make codecs better at handling that type of data—meaning better compression for games.  AFAIK none of the currently used corpora (such as the Canterbury Corpus, Silesa Compression Corpus, Calgary Corpus, etc.) include any game data.
  9. Recently, I put together a benchmark for general-purpose lossless data compression.  While I was doing so it occurred to me that the standard corpora aren't very representative of the type of data people actually use data compression for these days, and part of what is missing is data from games.   To address this I'm putting together a new corpus and I would like to make sure it includes data relevant for game developers. Since I have virtually no experience with game development I was hoping some of the developers around here could tell me what kind of data they usually use compression for, as well as how they use it (i.e., bundled with other content and the entire container is compressed, or is each piece of data compressed individually), so I can include something like it in the corpus.   For a list of things I'm already thinking about, please take a look at the project's issue tracker.  If you have any suggestions, thoughts on the existing items, ideas for data sources, etc., I would definitely like to know.
  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!