Jump to content
  • Advertisement


  • Content Count

  • Joined

  • Last visited

Community Reputation

484 Neutral

About LarryTheKing

  • Rank

Personal Information

  1. LarryTheKing

    PinWheel Encryption

      Impossible, each block is processed independently. I think I wasn't clear in my description of what a block is. A block is 256 bytes; the picture divided into 9 areas, does not show 9 blocks, but rather 768.   Maybe this will help with my explanation of what a "block" is     *Pretend I was able to draw 768 lines ( which each represent a block )   Sorry if I've been a bit hard to understand, my mind is focused on Thanksgiving   But, in the case where data is divided as such, and each block is independently encrypted, does this make sense?
  2. LarryTheKing

    PinWheel Encryption

      Well, i know you asked not to point it out, buuut... i think that's the biggest flaw in your algorithm. Encrypting identical data should NOT return identical value...   Anyway, i guess that code is just for your personal usage so, who care. No one is probably gonna try to break your code, unless you ask them to. Imo that's good enough for personal use.   My first encryptor was showing a similar flaw (you could see parts of a bitmap image in the noisy background), then i just created another algorithm, which isn't perfect, but at least give no clue as in your image, everything always look 100% noisy, but even then, ppl started pointing flaws about it. (You definitely should read this post)   For me, i guess that just good enough for my need, althrough i never used it yet for anything, it was more for learning than anything else. For the fun of it, i encrypted your image using my algo. discussed above, and it gave me this. That's what you should aim.   PS: dont take me too seriously, im no encryption expert...       What I'm trying to say is that for a set of inputs - the passcode being X and the 256 byte block being  Y - the algorithm should satisfy the requirements of a "function" : for a given set of inputs the function should return a specific output F(x, y) == z always   If two blocks are identical, Y1 == Y2, and they are being encrypted with the same pass code, then F(X, Y1) == F(X, Y2) == some value z Our ability to decrypt the data, given the correct passcode, relies on this fact.   Now if I wanted two identical blocks encrypted using the same passcode to not return equivalent values, I would need to add another "variable" to the function. So If I added the block's number as variable W so, F(w, x, y) == z then two identical blocks, being encrypted with the same pass code would not give identical results. I tried this and the results were promising :     However, I decided that the algorithm shouldn't care or know which block it was dealing with, as an "attacker" could easily later determine this "variable" and use it to weaken the integrity of the system   I'd rather have only two inputs X and Y that can't be determined by looking at the encrypted data, than three variable W, X and Y, where one of the variables could be determined.   There is the problem, it is not enough if the set of all passwords allows any bit to appear anywhere. You would need to make sure that for a single password the bits are shuffled around enough that they can be anywhere, but as can be seen from your pictures thats not happening. I would think that is because of an inherent flaw of your pinwheel thing, that moves the bits in a very predictable way that mostly depends on the source data and not much on the password. And you know, xor obfuscation is the first thing everyone tries, but cause of its property of revealing a source bit by double xor-ing it with the same bit it will never make a good encryption in that way you use it, even if your method is a bit more elaborate than usually seen.     A bit can take multiple paths to end up in the same spot. By taking these alternative paths though, the other 2047 bits also take alternative paths.   The movement, or spinning if you will, is determined only by the passcode, and there is only one combination of spins, only one passcode, that will decrypt all of the blocks. Even if you know that the data was "spun", you don't know which path it took. I imagine this path on a massive tree where each branch has 256 child branches, and each of those child branches has 256 child branches and so on. The number of unique combination for a given password length n is 256n where hopefully n is some large number like 24   Also, there is more than XOR'ing going on, there is twisting, turning, and in the newer version, turbulence ( noise ) being added to the mix. To decrypt, each byte must not only be XOR'd with every byte it had been, which would have to determined some how, but it must do so in the correct order as other operations such as rolling and the turbulence take place. In other words, you must follow your exact path up the tree to retrieve the original data.   Yes, this is only a pet-project, so it's not likely anyone other than myself will ever use it. Still, I'm happy to see other people's ideas.
  3. LarryTheKing

    PinWheel Encryption

    So, I've made a few changes that seem to fix the two biggest issues: I've introduced a "Turbulence" phase to combat patterns, and I've also added an "Avalanche" phase to increase sensitivity:   The results: A difference of a single bit will result in dramatic variation.   Small test: A 256 byte file was created with all bytes = 0xFF and a second, similar file was created where all bytes = 0xFF except for the last byte which = 0xFE   Here are the visualizations of the two files: (Note they are 16 x 16 pixels and are quite difficult to see on a white background) File A :  File B :    Both files were encrypted using the same passcode: Result A :  Result B :    Both results appear as noise; however each is different.   And finally we have the "worst case scenario" triangle image/data test:   Please note that the vertical lines are the result of blocks of data being EXACTLY the same in the original data. As in one row of pixels being exactly the same as another: Note: Each row of pixels in the solid green block is exactly the same, as is the case wherever else there are vertical lines; encrypting identical datas returns identical results. In other words, please don't point out the "clear pattern of vertical lines"    Now, adding both the turbulence and avalanche phases to the algorithm results in a ~ 40% increase in computation time.   The new code is attached ( PNWL.h )   Also, I'm still looking for a good passcode hashing function. If you feel like testing the new algorithm, use a longer password, or better yet, to emulate a hashcode, generate a 24 byte null terminated string  randomly and use that.   Again thank you.   Edit: Forgot to attach the new code XD
  4. LarryTheKing

    PinWheel Encryption

    Excellent, I wasn't expecting this much of a response. This is great   OK, so the way I see it, here are the main concerns: Hash The password! Why bother spinning the data? What are these patterns I see? Black and white! Correlation between password strength and quality. So, in relative order:   1 : Yes! hashing the password would be a good idea. I could generate a large number ( lets call it X) of bytes from a password and then use this as the "key". The question is how large should the hash be? It clearly should be larger than 12 bytes, 24 bytes maybe? The number of possible combinations from spinning the data appears to be 256n where n is the number of characters in the key. Thus 25624 yeilds 6.27 x 1057 possible combinations. If we assume that a computer can evaluate 1 billion combinations per second, an attempt to try every single combination of 24 bytes, or brute force a password, could still take up to 1041 years.   2 : You guessed correctly when you said "to obfuscate the data." By performing these spins, a byte, or a variant there of, can end up literately anywhere within the block! On a smaller scale, a bit can end up anywhere within the block as-well, because remember we're rolling too! With 256 bytes, there are 2048 bits. There are, by definition, 2048! ways to organize 2048 bits, and by spinning and rolling we can achieve any of these combinations. In other-words, by spinning and rolling we set the maximum number of possibilities to 2048! or around 105895 possibilities.   The other reason to spin the blocks is to create multiple solutions, which sounds counter intuitive! Just because you find a solution to a block, doesn't mean that it'll work for any other block. So for example lets say that we have 256 bytes, where the first byte is 0 and the rest are 1. Next we have another 256 bytes where the last byte is 1 and the rest are 0. Finally we encrypt both of the data with the same password, say "boat", Now because we spun the data, there is a massive number of ways to spin the data in reverse to get the byte '1' back into the first slot, one of these ways is, obviously, to follow the pattern generated by the password "boat", however other pass-codes may work like say "hamburger" The trick is that even though the pass-code "hamburger" successfully decrypts the first set of 256 bytes, it WILL not decrypt the second set! Only the passcode "boat" will do that.   Furthermore, even if you had the original as well as the encrypted version of a file, you would still need to try MANY ( up to  25624 ) combinations of passcodes to find the one that works for every block and crack the password.   3 : Ah yes the patterns in the triangles... maybe I should have been more specific in my captions. Of course there will be patterns, this is a worse case scenario test. The data in that image is LARGELY repetitive, in fact each row of 256 pixels, the width of the image, is usually close to identical to the row above it. Encrypting blocks that only have a 1 byte difference between one and another will unfortunately yield somewhat similar results. This effect is compounded by the fact that the change between rows is fairly regular. If I reduce the size of the image to 246 x 246 pixels and then encrypt with an 8 char password, we get this:   Which while not perfect, you can still somewhat see the triangles, looks more like noise.   To spice things up, and prevent these types of patterns, the amount of roll could be varied. Part of the issue is that most ASCII pass-codes limit the amount of roll as the high bit is usually 0.   4 : This one is kind of silly. It's black and white because it's a black and white bitmap! - Remember I only encrypted the image/data portion of each bitmap. No amount of encryption is going to change the fact that in a grayscale bitmap, each pixel is determined by one and only byte and must be somewhere between black and white. Here's what it would have looked like, had it not been grayscale, but 24bpp (COLOR) with an 8 char password :   5 : The correlation between password length and quality was actually intentional. I figured that longer passwords yielding better encryption was ideal. After reading some of the feedback I've gotten, I now know this to be false. By hashing the passcodes ( See # 1 ) we can remove this correlation.   Sorry if my explanations haven't been the best.   Again, thank you everybody who has spoken thus far. Any other ideas and comments would be appreciated.
  5. Edit: See the newest version in this post New phases have been added: Turbulence and Avalanche.   --   Hello everybody, I'm looking for some feedback on an encryption technique/algorithm I've been working on for the past few days: PinWheel (PNWL) encryption. Now, I've found that explaining how the technique works is a challenge in and of itself, so please bear with me – I've even included lots of pictures. Before I get to how the algorithm works, here are some statistics:   Basic Information About PNWL: Operates on 256 Byte Blocks Makes heavy use of XOR “Spins” the data to achieve encryption Strength of encryption is exponentially proportional to the password length Essentially PNWL works by splitting up 256 bytes of data into sized blocks. Sort of like this:     Thus, one block of 256 bytes (the main block) contains four blocks of 64 bytes; each of these blocks contains four blocks of 16 bytes, and similarly each of these blocks contain four blocks of 4 bytes.   To encrypt the data each block's content is quartered and then spun clockwise. As the quartered block spins, its content is internally XOR'd.   This hierarchy of spins is repeated for each character in the password. Furthermore, the magnitude of each spin is determined by the respective char.     The only exception to the “Spin” technique are the Block4's, which instead “roll.” The amount of roll is determined by a set of magic numbers: MAGIC[4][4] = { { 1, 3, 5, 7}, { 1, 7, 2, 9}, { 2, 3, 5, 7}, { 1, 9, 9, 6} }; To encrypt:   For each character in the password: Roll Block4's Left Spin Bloc16's Spin Block64's Spin the Block256 To decrypt:   Reverse the password, and then for each character: Spin the Block256 in reverse Spin the Block64's in reverse Spin the Block16's in reverse Roll Block4's Right Anyways enough talk; here's the code, which also attached (note: requires SSE3) //Copyright (C) 2013 Laurence King // //Permission is hereby granted, free of charge, to any person obtaining a //copy of this software and associated documentation files (the "Software"), //to deal in the Software without restriction, including without limitation //the rights to use, copy, modify, merge, publish, distribute, sublicense, //and/or sell copies of the Software, and to permit persons to whom the //Software is furnished to do so, subject to the following conditions: // //The above copyright notice and this permission notice shall be included //in all copies or substantial portions of the Software. // //THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, //INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A //PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT //HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION //OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE //SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. #pragma once #include <intrin.h> #ifdef _MSC_VER #define ALIGN( n ) __declspec( align( n ) ) #else #define ALIGN( n ) alignas( n ) #endif #define PNWL_MASK1 0x3 #define PNWL_MASK2 0xC #define PNWL_MASK3 0x30 #define PNWL_MASK4 0xC0 namespace PinWheel { typedef int int32; typedef unsigned int uint32; // PNWL Magic constants const uint32 MAGIC[4][4] = { { 1, 3, 5, 7}, { 1, 7, 2, 9}, { 2, 3, 5, 7}, { 1, 9, 9, 6} }; ALIGN(16) struct Block16 { union { uint32 Data[4]; __m128i vData; }; void Spin0 (void); void Spin1 (void); void Spin2 (void); void Spin3 (void); void rSpin0 (void); void rSpin1 (void); void rSpin2 (void); void rSpin3 (void); }; ALIGN(16) struct Block64 { union { uint32 _Data[16]; Block16 Blocks[4]; __m128i vData [4]; }; void Spin0 (void); void Spin1 (void); void Spin2 (void); void Spin3 (void); void rSpin0 (void); void rSpin1 (void); void rSpin2 (void); void rSpin3 (void); }; ALIGN(16) struct Block256 { union { uint32 _Data[64]; __m128i _vData[16]; Block16 _Block16[16]; Block64 Blocks[4]; }; void Spin0 (void); void Spin1 (void); void Spin2 (void); void Spin3 (void); void rSpin0 (void); void rSpin1 (void); void rSpin2 (void); void rSpin3 (void); void Forward(const char *); void Reverse(const char *); }; #define ROTATE_LEFT(x, n) (((x) << (n)) | ((x) >> (32-(n)))) #define ROTATE_RIGHT(x, n) (((x) >> (n)) | ((x) << (32-(n)))) void Block16::Spin0(void) { Data[3] ^= Data[0]; Data[2] ^= Data[0]; Data[1] ^= Data[0]; Data[0] = ~Data[0]; } void Block16::Spin1(void) { Data[3] ^= Data[0]; vData = _mm_shuffle_epi32(vData, _MM_SHUFFLE(2, 1, 0, 3)); } void Block16::Spin2(void) { Data[3] ^= Data[0]; Data[2] ^= Data[0]; Data[0] = ~Data[0]; vData = _mm_shuffle_epi32(vData, _MM_SHUFFLE(1, 0, 3, 2)); } void Block16::Spin3(void) { Data[3] ^= Data[0]; Data[2] ^= Data[0]; Data[1] ^= Data[0]; vData = _mm_shuffle_epi32(vData, _MM_SHUFFLE(0, 3, 2, 1)); } void Block16::rSpin0(void) { Data[0] = ~Data[0]; Data[1] ^= Data[0]; Data[2] ^= Data[0]; Data[3] ^= Data[0]; } void Block16::rSpin1(void) { vData = _mm_shuffle_epi32(vData, _MM_SHUFFLE(0, 3, 2, 1)); Data[3] ^= Data[0]; } void Block16::rSpin2(void) { vData = _mm_shuffle_epi32(vData, _MM_SHUFFLE(1, 0, 3, 2)); Data[0] = ~Data[0]; Data[2] ^= Data[0]; Data[3] ^= Data[0]; } void Block16::rSpin3(void) { vData = _mm_shuffle_epi32(vData, _MM_SHUFFLE(2, 1, 0, 3)); Data[3] ^= Data[0]; Data[2] ^= Data[0]; Data[1] ^= Data[0]; } void Block64::Spin0(void) { vData[3] = _mm_xor_si128(vData[0], vData[3]); vData[2] = _mm_xor_si128(vData[0], vData[2]); vData[1] = _mm_xor_si128(vData[0], vData[1]); } void Block64::Spin1(void) { __m128i val_a = vData[0]; vData[0] = _mm_xor_si128(vData[3], val_a); vData[3] = vData[2]; // _mm_xor_si128(vData[2], val_a); vData[2] = vData[1]; // _mm_xor_si128(vData[1], val_a); vData[1] = val_a; } void Block64::Spin2(void) { __m128i val_ab = vData[0]; vData[0] = _mm_xor_si128(vData[2], val_ab); vData[2] = val_ab; val_ab = vData[1]; vData[1] = _mm_xor_si128(vData[3], val_ab); vData[3] = val_ab; } void Block64::Spin3(void) { __m128i val_a = vData[0]; vData[0] = _mm_xor_si128(vData[1], val_a); vData[1] = _mm_xor_si128(vData[2], val_a); vData[2] = _mm_xor_si128(vData[3], val_a); vData[3] = val_a; } void Block64::rSpin0(void) { vData[3] = _mm_xor_si128(vData[0], vData[3]); vData[2] = _mm_xor_si128(vData[0], vData[2]); vData[1] = _mm_xor_si128(vData[0], vData[1]); } void Block64::rSpin1(void) { __m128i val_a = vData[1]; vData[1] = vData[2]; vData[2] = vData[3]; vData[3] = _mm_xor_si128(vData[0], val_a); vData[0] = val_a; } void Block64::rSpin2(void) { __m128i val_ab = vData[2]; vData[2] = _mm_xor_si128(vData[0], val_ab); vData[0] = val_ab; val_ab = vData[3]; vData[3] = _mm_xor_si128(vData[1], val_ab); vData[1] = val_ab; } void Block64::rSpin3(void) { __m128i val_a = vData[3]; vData[3] = _mm_xor_si128(vData[2], val_a); vData[2] = _mm_xor_si128(vData[1], val_a); vData[1] = _mm_xor_si128(vData[0], val_a); vData[0] = val_a; } void Block256::Spin0(void) { _vData[0x4] = _mm_xor_si128(_vData[0x0], _vData[0x4]); _vData[0x8] = _mm_xor_si128(_vData[0x0], _vData[0x8]); _vData[0xC] = _mm_xor_si128(_vData[0x0], _vData[0xC]); _vData[0x5] = _mm_xor_si128(_vData[0x1], _vData[0x5]); _vData[0x9] = _mm_xor_si128(_vData[0x1], _vData[0x9]); _vData[0xD] = _mm_xor_si128(_vData[0x1], _vData[0xD]); _vData[0x6] = _mm_xor_si128(_vData[0x2], _vData[0x6]); _vData[0xA] = _mm_xor_si128(_vData[0x2], _vData[0xA]); _vData[0xE] = _mm_xor_si128(_vData[0x2], _vData[0xE]); _vData[0x7] = _mm_xor_si128(_vData[0x3], _vData[0x7]); _vData[0xB] = _mm_xor_si128(_vData[0x3], _vData[0xB]); _vData[0xF] = _mm_xor_si128(_vData[0x3], _vData[0xF]); } void Block256::Spin1(void) { __m128i val_ = _vData[0]; _vData[0x0] = _mm_xor_si128(val_, _vData[0xC]); _vData[0xC] = _vData[0x8]; _vData[0x8] = _vData[0x4]; _vData[0x4] = val_; val_ = _vData[1]; _vData[0x1] = _mm_xor_si128(val_, _vData[0xD]); _vData[0xD] = _vData[0x9]; _vData[0x9] = _vData[0x5]; _vData[0x5] = val_; val_ = _vData[2]; _vData[0x2] = _mm_xor_si128(val_, _vData[0xE]); _vData[0xE] = _vData[0xA]; _vData[0xA] = _vData[0x6]; _vData[0x6] = val_; val_ = _vData[3]; _vData[0x3] = _mm_xor_si128(val_, _vData[0xF]); _vData[0xF] = _vData[0xB]; _vData[0xB] = _vData[0x7]; _vData[0x7] = val_; } void Block256::Spin2(void) { __m128i val_ = _vData[0]; _vData[0x0] = _mm_xor_si128(val_, _vData[0x8]); _vData[0x8] = val_; val_ = _vData[1]; _vData[0x1] = _mm_xor_si128(val_, _vData[0x9]); _vData[0x9] = val_; val_ = _vData[2]; _vData[0x2] = _mm_xor_si128(val_, _vData[0xA]); _vData[0xA] = val_; val_ = _vData[3]; _vData[0x3] = _mm_xor_si128(val_, _vData[0xB]); _vData[0xB] = val_; val_ = _vData[4]; _vData[0x4] = _mm_xor_si128(_vData[0x8], _vData[0xC]); _vData[0xC] = val_; val_ = _vData[5]; _vData[0x5] = _mm_xor_si128(_vData[0x9], _vData[0xD]); _vData[0xD] = val_; val_ = _vData[6]; _vData[0x6] = _mm_xor_si128(_vData[0xA], _vData[0xE]); _vData[0xE] = val_; val_ = _vData[7]; _vData[0x7] = _mm_xor_si128(_vData[0xB], _vData[0xF]); _vData[0xF] = val_; } void Block256::Spin3(void) { __m128i val_ = _vData[0]; _vData[0x0] = _mm_xor_si128(val_, _vData[0x4]); _vData[0x4] = _mm_xor_si128(val_, _vData[0x8]); _vData[0x8] = _mm_xor_si128(val_, _vData[0xC]); _vData[0xC] = val_; val_ = _vData[1]; _vData[0x1] = _mm_xor_si128(val_, _vData[0x5]); _vData[0x5] = _mm_xor_si128(val_, _vData[0x9]); _vData[0x9] = _mm_xor_si128(val_, _vData[0xD]); _vData[0xD] = val_; val_ = _vData[2]; _vData[0x2] = _mm_xor_si128(val_, _vData[0x6]); _vData[0x6] = _mm_xor_si128(val_, _vData[0xA]); _vData[0xA] = _mm_xor_si128(val_, _vData[0xE]); _vData[0xE] = val_; val_ = _vData[3]; _vData[0x3] = _mm_xor_si128(val_, _vData[0x7]); _vData[0x7] = _mm_xor_si128(val_, _vData[0xB]); _vData[0xB] = _mm_xor_si128(val_, _vData[0xF]); _vData[0xF] = val_; } void Block256::rSpin0(void) { _vData[0x4] = _mm_xor_si128(_vData[0x0], _vData[0x4]); _vData[0x8] = _mm_xor_si128(_vData[0x0], _vData[0x8]); _vData[0xC] = _mm_xor_si128(_vData[0x0], _vData[0xC]); _vData[0x5] = _mm_xor_si128(_vData[0x1], _vData[0x5]); _vData[0x9] = _mm_xor_si128(_vData[0x1], _vData[0x9]); _vData[0xD] = _mm_xor_si128(_vData[0x1], _vData[0xD]); _vData[0x6] = _mm_xor_si128(_vData[0x2], _vData[0x6]); _vData[0xA] = _mm_xor_si128(_vData[0x2], _vData[0xA]); _vData[0xE] = _mm_xor_si128(_vData[0x2], _vData[0xE]); _vData[0x7] = _mm_xor_si128(_vData[0x3], _vData[0x7]); _vData[0xB] = _mm_xor_si128(_vData[0x3], _vData[0xB]); _vData[0xF] = _mm_xor_si128(_vData[0x3], _vData[0xF]); } void Block256::rSpin1(void) { __m128i val_ = _vData[4]; _vData[0x4] = _vData[0x8]; _vData[0x8] = _vData[0xC]; _vData[0xC] = _mm_xor_si128(val_, _vData[0x0]); _vData[0x0] = val_; val_ = _vData[5]; _vData[0x5] = _vData[0x9]; _vData[0x9] = _vData[0xD]; _vData[0xD] = _mm_xor_si128(val_, _vData[0x1]); _vData[0x1] = val_; val_ = _vData[6]; _vData[0x6] = _vData[0xA]; _vData[0xA] = _vData[0xE]; _vData[0xE] = _mm_xor_si128(val_, _vData[0x2]); _vData[0x2] = val_; val_ = _vData[7]; _vData[0x7] = _vData[0xB]; _vData[0xB] = _vData[0xF]; _vData[0xF] = _mm_xor_si128(val_, _vData[0x3]); _vData[0x3] = val_; } void Block256::rSpin2(void) { __m128i val_ = _vData[8]; _vData[0x8] = _mm_xor_si128(val_, _vData[0x0]); _vData[0x0] = val_; val_ = _vData[9]; _vData[0x9] = _mm_xor_si128(val_, _vData[0x1]); _vData[0x1] = val_; val_ = _vData[0xA]; _vData[0xA] = _mm_xor_si128(val_, _vData[0x2]); _vData[0x2] = val_; val_ = _vData[0xB]; _vData[0xB] = _mm_xor_si128(val_, _vData[0x3]); _vData[0x3] = val_; val_ = _vData[0xC]; _vData[0xC] = _mm_xor_si128(_vData[0x0], _vData[0x4]); _vData[0x4] = val_; val_ = _vData[0xD]; _vData[0xD] = _mm_xor_si128(_vData[0x1], _vData[0x5]); _vData[0x5] = val_; val_ = _vData[0xE]; _vData[0xE] = _mm_xor_si128(_vData[0x2], _vData[0x6]); _vData[0x6] = val_; val_ = _vData[0xF]; _vData[0xF] = _mm_xor_si128(_vData[0x3], _vData[0x7]); _vData[0x7] = val_; } void Block256::rSpin3(void) { __m128i val_ = _vData[0xC]; _vData[0xC] = _mm_xor_si128(val_, _vData[0x8]); _vData[0x8] = _mm_xor_si128(val_, _vData[0x4]); _vData[0x4] = _mm_xor_si128(val_, _vData[0x0]); _vData[0x0] = val_; val_ = _vData[0xD]; _vData[0xD] = _mm_xor_si128(val_, _vData[0x9]); _vData[0x9] = _mm_xor_si128(val_, _vData[0x5]); _vData[0x5] = _mm_xor_si128(val_, _vData[0x1]); _vData[0x1] = val_; val_ = _vData[0xE]; _vData[0xE] = _mm_xor_si128(val_, _vData[0xA]); _vData[0xA] = _mm_xor_si128(val_, _vData[0x6]); _vData[0x6] = _mm_xor_si128(val_, _vData[0x2]); _vData[0x2] = val_; val_ = _vData[0xF]; _vData[0xF] = _mm_xor_si128(val_, _vData[0xB]); _vData[0xB] = _mm_xor_si128(val_, _vData[0x7]); _vData[0x7] = _mm_xor_si128(val_, _vData[0x3]); _vData[0x3] = val_; } void Block256::Forward(const char * key) { for(char c = *(key++); c != 0; c = *(key++)) { uint32 amnt0 = c & PNWL_MASK1; uint32 amnt1 = (c & PNWL_MASK2 ) >> 2; uint32 amnt2 = (c & PNWL_MASK3 ) >> 4; uint32 amnt3 = (c & PNWL_MASK4 ) >> 6; #pragma region BLOCK4 for(int i = 0; i < 64; i+=4) { _Data[i] = ROTATE_LEFT( _Data[i] , MAGIC[amnt3][0] ); _Data[i + 1] = ROTATE_LEFT( _Data[i + 1] , MAGIC[amnt3][1] ); _Data[i + 2] = ROTATE_LEFT( _Data[i + 2] , MAGIC[amnt3][2] ); _Data[i + 3] = ROTATE_LEFT( _Data[i + 3] , MAGIC[amnt3][3] ); } #pragma endregion #pragma region BLOCK16 switch (amnt0) { case 0: for(int i = 0; i < 16; i++) _Block16[i].Spin0(); break; case 1: for(int i = 0; i < 16; i++) _Block16[i].Spin1(); break; case 2: for(int i = 0; i < 16; i++) _Block16[i].Spin2(); break; case 3: for(int i = 0; i < 16; i++) _Block16[i].Spin3(); break; } #pragma endregion #pragma region BLOCK64 switch (amnt1) { case 0: Blocks[0].Spin0(); Blocks[1].Spin0(); Blocks[2].Spin0(); Blocks[3].Spin0(); break; case 1: Blocks[0].Spin1(); Blocks[1].Spin1(); Blocks[2].Spin1(); Blocks[3].Spin1(); break; case 2: Blocks[0].Spin2(); Blocks[1].Spin2(); Blocks[2].Spin2(); Blocks[3].Spin2(); break; case 3: Blocks[0].Spin3(); Blocks[1].Spin3(); Blocks[2].Spin3(); Blocks[3].Spin3(); break; } #pragma endregion #pragma region BLOCK256 switch (amnt2) { case 0: Spin0(); break; case 1: Spin1(); break; case 2: Spin2(); break; case 3: Spin3(); break; } #pragma endregion } } // Expects the key to already have been reversed void Block256::Reverse(const char * rKey) { for(char c = *(rKey++); c != 0; c = *(rKey++)) { uint32 amnt0 = c & PNWL_MASK1; uint32 amnt1 = (c & PNWL_MASK2 ) >> 2; uint32 amnt2 = (c & PNWL_MASK3 ) >> 4; uint32 amnt3 = (c & PNWL_MASK4 ) >> 6; #pragma region BLOCK256 switch (amnt2) { case 0: rSpin0(); break; case 1: rSpin1(); break; case 2: rSpin2(); break; case 3: rSpin3(); break; } #pragma endregion #pragma region BLOCK64 switch (amnt1) { case 0: Blocks[0].rSpin0(); Blocks[1].rSpin0(); Blocks[2].rSpin0(); Blocks[3].rSpin0(); break; case 1: Blocks[0].rSpin1(); Blocks[1].rSpin1(); Blocks[2].rSpin1(); Blocks[3].rSpin1(); break; case 2: Blocks[0].rSpin2(); Blocks[1].rSpin2(); Blocks[2].rSpin2(); Blocks[3].rSpin2(); break; case 3: Blocks[0].rSpin3(); Blocks[1].rSpin3(); Blocks[2].rSpin3(); Blocks[3].rSpin3(); break; } #pragma endregion #pragma region BLOCK16 switch (amnt0) { case 0: for(int i = 0; i < 16; i++) _Block16[i].rSpin0(); break; case 1: for(int i = 0; i < 16; i++) _Block16[i].rSpin1(); break; case 2: for(int i = 0; i < 16; i++) _Block16[i].rSpin2(); break; case 3: for(int i = 0; i < 16; i++) _Block16[i].rSpin3(); break; } #pragma endregion #pragma region BLOCK4 for(int i = 0; i < 64; i+=4) { _Data[i] = ROTATE_RIGHT( _Data[i] , MAGIC[amnt3][0] ); _Data[i + 1] = ROTATE_RIGHT( _Data[i + 1] , MAGIC[amnt3][1] ); _Data[i + 2] = ROTATE_RIGHT( _Data[i + 2] , MAGIC[amnt3][2] ); _Data[i + 3] = ROTATE_RIGHT( _Data[i + 3] , MAGIC[amnt3][3] ); } #pragma endregion } } } And here is how you would encrypt some data: PinWheel::Block256 * blocks = reinterpret_cast<PinWheel::Block256 *>(memblock); for(int i = 0; i < blockcount; i++) { blocks[i].Forward(password.data()); } Now for some visual examples of PNWL encryption in action: (For illustration purposes, these were created by encrypting the image portion of either 24bpp bitmaps or grayscale bitmaps)   Mona: [spoiler]Original: Short Password (4 char):  Medium Password ( 8 char):  Long Password (12 char):  [/spoiler]   Simple Triangles: [spoiler]Original: Short Password ( 4 char): Medium Password (8 char): Long Password (12 char): [/spoiler]   Flower ( grayscale bitmap) [spoiler]Original:   Short Password (4 char):   Medium Password ( 8 char ) :   Long Password ( 12 char ) :   [/spoiler]   Where I can see improvement: PNWL was designed to make use of SIMD commands, however it can be done without them. I don't have a processor that supports AVX2, but I predict a 30% boost if it was used, for example, on the Roll portion. Furthermore, multithreading could yield excellent returns   Attached is the source code for PNWL and a quick console app to test it out.   Thank you
  6. Thinking up a renderqueue
  7. LarryTheKing

    Model Format: Concept and Loading

      So you're saying store the vertex layouts in the model, and have the meshes index to the matching layout. That might work. After a quick thought thought, 100 bytes per mesh * a mean of 8 meshes per model * about 1000 models per level = ~781 KB. (averages not from any real statistics) Not nearly as bad as I had imagined.   The other thing I could do is store the vertex layouts in separate files. That way meshes using the same layout would just use the same file, which I could cache. If I did it this way I would only have to have one copy of each unique vertex layout in memory and on disk.   Regarding what a model would contain: the idea was that a Model would just be a generic class to store related Meshes. A mesh would have an index buffer, N vertex buffers, and N resources / textures. This way a model could contain, for example, all of the meshes that composed an automobile. These models would then be used by objects that would also have a material. Now of course some checking would have to be done to make sure that the model would be compatible with the material. Luckily each mesh has a MeshMask, 16 bits specifying what attributes the mesh contains; the mesh header also keeps track of the number of TexCoord and vertex color streams as well as the number of texture resources. To check if a model and a material were compatible, I would just loop over each mesh and check if its mask and header were compatible with the materials requirements. A level would then contain many objects that could be culled, sorted, and drawn.   My intent was for the model to be able to store pretty much any type of graphic meshes;, whether it contains a weighted charter,  a tree, a sword, or a spaceship shouldn't matter as the model is just a container for meshes. A model should also be able to store thousands of meshes(up to 65, 536) assuming the file doesn't become over 4GB in size. I don't think I'm missing anything that would prevent this, but I could very well have skipped right over something.   Again, thank you for your ideas.
  8. LarryTheKing

    Model Format: Concept and Loading

    Yep, the model is really just a container of meshes. The textures are stored in other files - the model just contains the names of the textures so they can be loaded. Similarly, all other types/objects, ie bones and animations, will be - I haven't gotten to this stage yet - stored in separate files.   The way I have my vertex buffers set up is that a mesh can have multiple vertex buffers, but doesn't necessarily need to. Each vertex buffer can also contain interleaved attributes. When the files are opened in my exporter tool, you can create new, split up, and delete vertex buffers; however, it defaults to two vertex buffers per mesh.   So the default could look like this: Buffer 1 - Position Buffer 2 - TexCoord, Color, Normal, Tangent, Binormal...   The buffers could be split to look like anything really. ie: Buffer 1 - Position, Color1 Buffer 2 - TexCoord1, TexCoord2, Color2 Buffer 3 - Normal, Tangent, Binormal The big reason, I opted for split vertex buffers, is fast shadowing. I just have to bind the buffer that contains the position data (which I force to always be the first buffer).   I am worried, though, about how I should deal with getting the VertexLayout to the renderer. I'm not sure if I should store it per mesh, load it similarly to how I'm loading textures, or just have each material specify. The vertex layout itself could end up being 100 bytes per mesh, so storing it per-mesh doesn't really seem ideal.   Any ideas? Thank you for your input.
  9. LarryTheKing

    Model Format: Concept and Loading

      Yeah, I'm adjusting the loader so that they are treated more as offsets. I'm not storing the textures in the file, the blob just contains the character data for the texture's name. The model loader has no idea how to load the textures, it just request the texture from the texture cache. The cache  can then load the texture from where ever. You're right about not putting the DXGI_FORMAT in there, I'll adjust that.     All right. It looks like I'll have to adjust quite a few things in the library as I've just been tacking the dllexports on to make visual studio be quiet.   Thanks for all of the feedback!
  10. LarryTheKing

    Model Format: Concept and Loading

    All right! It's good to know that you can offset a pointer with an int. I also like the idea of offsets relative to the position of the offset, although I wonder what would happen if you had a negative offset; Theoretically it wouldn't matter and would still work.   Now, regarding the custom deleter, the deleter would call a "delete" function of the model, correct? Would it then cast the Model* to a char* before finally calling delete[] foochar?   I assume the deleter would look something like this:   void DeleteModel( *Model) { Model->Shutdown(); delete[] (char*)Model; }   Thankyou.   EDIT: Your offset template is a really good idea! It took me a moment to understand what it was doing - still new to C++
  11. Hello all, I'm looking for some feedback regrading a model format and loader I've been working on over the past few days. I'm transitioning to C++, coming from C# and this has been a great exercise thus far, but I feel this post might end up a bit lengthy.   Some background as to what I wanted to accomplish with the file format: The model can contain a large number of meshes. Each mesh can have an arbitrary number of vertex buffers; not that a mesh should have 8, 16, or even 65,535 vertex streams, just that it could. Each mesh can have an arbitrary number of textures; again, see above. Each mesh and the model itself and should have some sort of bounding volume. Fast loading; the model should use local pointers and require minimal live processing. Possibility for compression   After some research, it appears that loading directly into memory and then adjusting local pointers seems to be one of the fastest ways to load an object. So the entire format reflects the objects that make up my model.   A model contains: a pointer to some ModelTextures, a pointer to the MeshHeaders, a pointer to the MeshCullDatas and a pointer to the MeshDrawDatas.   I've tried to implement some Data Oriented Design – a very different concept coming from C#. I've split up the meshes into arrays of data needed for different operations: Culling and Drawing.   Furthermore, I'm attempting to implement this as part of my content manager, so a ModelTexture, is really just a wrapper around a shared_ptr<Texture2D> that is retrieved from another content cache.   All right, so here is what the model format looks like, I made a diagram! *sorry it's so tall...     The actual files are exported from a tool I've written in C#. I'm loading Collada files via Assimp, calculating any user requested data and displaying the model via SharpDX in a WinForms app.   In the end, the model gets exported to the file by the exporter first writing each mesh data object to “virtual memory,” adjusting all of the pointers and finally using a binary writer to spit out the finished file.   Pretty straight forward for me as I'm used to C#, but the scary stuff happens when we get to actually loading the file in C++.   First I load the entire file with an ifstream into a char[]. Then I cast the char[] to a Model. Now I need to offset the local pointers so that the model will work in memory; however! I read somewhere that you can't add pointers in C++, only subtract them, but to offset local pointers you needed to add! After much internet searching, I finally found an object ptrdiff_t that I could retrieve from a pointer, add to, and then cast back to a pointer. The question then became, “Is this legal, what I'm doing?” For a full day I pondered before quizzically deciding that it should? be legal. I mean how else would you offset pointers when you shouldn't just cast to an int? The next problem arrived when I realized that I needed to somehow delete the model from memory as well. Again not sure, as I had casted a char[] to a model, if I could delete the model. I pretended I could and wrote the destructor. Miraculously it seemed to work! The “Memory” window in Visual Studio seemed to show that the object had successfully been deleted, although I'm still not sure if I need to call delete on the model's pointers as they weren't created with new.   So now, I have all this code for loading a model, but I'm not sure if it's legal, safe, or even sensible!   Enough talk though, here's the code for loading the model:   std::shared_ptr<Ruined::Graphics::Model> ModelLoader::Load(const std::string &name) { Ruined::Graphics::Model * model; std::ifstream file (m_BaseDirectory + name, std::ios::in|std::ios::binary|std::ios::ate); if (file.is_open()) { // Get the file's total size unsigned int size = file.tellg(); // Create a char[] of the size to load the file into char* memblock = new char [size]; // Seek to the beginning and read the file file.seekg (0, std::ios::beg); file.read (memblock, size); // Finally close the file file.close(); // Cast the char[] to a Ruined::Graphics::Model pointer model = static_cast<Ruined::Graphics::Model *>((void*)memblock); // The location of the model in memory ptrdiff_t memOffset = (ptrdiff_t)model; // Offset the model's local pointers // Mesh Headers ptrdiff_t intOffset; model->MeshHeaders = (Ruined::Graphics::MeshHeader*)(memOffset + (ptrdiff_t)model->MeshHeaders); // Mesh Culling Datas // intOffset = (ptrdiff_t)model->MeshCullDatas; model->MeshCullDatas = (Ruined::Graphics::MeshCullData*)(memOffset + (ptrdiff_t)model->MeshCullDatas); // Mesh Drawing Datas // intOffset = (ptrdiff_t)model->MeshDrawDatas; model->MeshDrawDatas = (Ruined::Graphics::MeshDrawData*)(memOffset + (ptrdiff_t)model->MeshDrawDatas); // Model's Ruined::Graphics::ModelTexture pointer // intOffset = (ptrdiff_t)model->Textures; model->Textures = (Ruined::Graphics::ModelTexture*)(memOffset + (ptrdiff_t)model->Textures); // Load the model's textures for(int t = 0; t < model->TextureCount; t++) { // Offset TextureName pointers // intOffset = (ptrdiff_t)(model->Textures[t].TextureName); model->Textures[t].TextureName = (char*)(memOffset + (ptrdiff_t)(model->Textures[t].TextureName)); // Load the texture model->Textures[t].TextureContent = p_TextureCache->Load(model->Textures[t].TextureName); } HRESULT hresult; Ruined::Graphics::MeshDrawData * tempMeshD = nullptr; for(int m = 0; m < model->MeshCount; m++) { // Build the buffers tempMeshD = &model->MeshDrawDatas[m]; // Offset Index Buffer // intOffset = (ptrdiff_t)tempMeshD->IndexBuffer; tempMeshD->IndexBuffer = (ID3D11Buffer*)(memOffset + (ptrdiff_t)tempMeshD->IndexBuffer); // Offset Vertex Buffer // intOffset = (ptrdiff_t)tempMeshD->VertexBuffer; tempMeshD->VertexBuffers = (ID3D11Buffer**)(memOffset + (ptrdiff_t)tempMeshD->VertexBuffers); // Offset Strides // intOffset = (ptrdiff_t)tempMeshD->Strides; tempMeshD->Strides = (unsigned int*)(memOffset + (ptrdiff_t)tempMeshD->Strides); // Offset Resources intOffset = (ptrdiff_t)tempMeshD->Resources; tempMeshD->Resources = (ID3D11ShaderResourceView**)(memOffset + intOffset); // Convert Resources * to unsigned int * unsigned int * index = (unsigned int*)(memOffset + intOffset); // Assign to the poingters from the model's textures for(int t = 0; t < model->MeshHeaders[m].ResourceCount; t++) tempMeshD->Resources[t] = model->Textures[index[t]].TextureContent.get()->p_shaderResourceView; // Desc for the index buffer D3D11_BUFFER_DESC indexBufferDesc; indexBufferDesc.Usage = D3D11_USAGE_DEFAULT; indexBufferDesc.ByteWidth = tempMeshD->IndexCount * (tempMeshD->IndexFormat == DXGI_FORMAT_R16_UINT ? sizeof(unsigned short) : sizeof(unsigned int)); indexBufferDesc.BindFlags = D3D11_BIND_INDEX_BUFFER; indexBufferDesc.CPUAccessFlags = 0; indexBufferDesc.MiscFlags = 0; D3D11_SUBRESOURCE_DATA indexData; indexData.pSysMem = tempMeshD->IndexBuffer; indexData.SysMemPitch = 0; indexData.SysMemSlicePitch = 0; hresult = p_Graphics->GetDevice()->CreateBuffer(&indexBufferDesc, &indexData, &tempMeshD->IndexBuffer); if(FAILED(hresult)) { OutputDebugStringA("Failed to create Index Buffer"); } // Create each vertex buffer Ruined::Graphics::MeshBufferDesc * tempDesc = (Ruined::Graphics::MeshBufferDesc*)(tempMeshD->VertexBuffers); for(unsigned int b = 0; b < tempMeshD->VertexBufferCount; b++) { // Each buffer gets a desc D3D11_BUFFER_DESC bufferDesc; bufferDesc.Usage = D3D11_USAGE_DEFAULT; bufferDesc.ByteWidth = tempDesc[b].BufferWidth; bufferDesc.BindFlags = D3D11_BIND_VERTEX_BUFFER; bufferDesc.CPUAccessFlags = 0; bufferDesc.MiscFlags = 0; // Each buffer needs a subresource data D3D11_SUBRESOURCE_DATA subData; subData.pSysMem = (void*)((ptrdiff_t)tempDesc[b].Data + memOffset); subData.SysMemPitch = 0; subData.SysMemSlicePitch = 0; hresult = p_Graphics->GetDevice()->CreateBuffer(&bufferDesc, &subData, &(tempMeshD->VertexBuffers[b])); if(FAILED(hresult)) { OutputDebugStringA("Failed to create Vertex Buffer"); } } } } else { std::string errorMsg = "Failed to load Model: "; errorMsg += m_BaseDirectory + name + "\n"; OutputDebugStringA(errorMsg.c_str()); // Set model equal to something model = new Ruined::Graphics::Model(); } std::shared_ptr<Ruined::Graphics::Model> sModel(model); return sModel; } So that makes a little more sense, here is Model.h: #include "MeshDrawData.h" #include "ModelTexture.h" #include <memory> namespace Ruined { namespace Graphics { // Combine Model Header and Model Pointers struct __declspec(dllexport) Model { public: // Header unsigned short _FILETYPE; unsigned short _FILEVERSION; unsigned int ModelSize; unsigned short TextureCount; unsigned short MeshCount; DirectX::BoundingBox BoundingBox; // Pointers ModelTexture * Textures; MeshHeader * MeshHeaders; MeshCullData * MeshCullDatas; MeshDrawData * MeshDrawDatas; public: Model(void); ~Model(void); }; } } #endif   Here is ModelTexture.h: #pragma once #ifndef _MODELTEXTURE_H #define _MODELTEXTURE_H_ // Includes // #include "Texture2D.h" #include <memory> namespace Ruined { namespace Graphics { struct __declspec(dllexport) ModelTexture { public: char* TextureName; std::shared_ptr<Texture2D> TextureContent; }; } } #endif     Here are the mesh objects: #ifndef _MESHCULLDATA_H_ #define _MESHCULLDATA_H_ #include <DirectXCollision.h> namespace Ruined { namespace Graphics { struct __declspec(dllexport) MeshCullData { public: DirectX::BoundingBox BoundingBox; }; } } #endif #ifndef _MESHHEADER_H_ #define _MESHHEADER_H_ #include "MeshBufferDesc.h" namespace Ruined { namespace Graphics { enum MeshMask : unsigned short { Undefined = 0x0000, Texture = 0x0001, UVCoord = 0x0002, Color = 0x0004, Normal = 0x0008, Tangent = 0x0010, Binormal = 0x0020, BoneIndices = 0x0040, BoneWeights = 0x0080, SplitBuffers = 0x0100, AlphaBlend = 0x4000 }; struct __declspec(dllexport) MeshHeader { MeshMask Mask; unsigned char UVStreamCount; unsigned char ColorStreamCount; unsigned short ResourceCount; }; } } #endif #ifndef _MESHDRAWDATA_H_ #define _MESHDRAWDATA_H_ #include <d3d11.h> namespace Ruined { namespace Graphics { struct __declspec(dllexport) MeshDrawData { public: unsigned int VertexBufferCount; ID3D11Buffer ** VertexBuffers; unsigned int * Strides; ID3D11Buffer * IndexBuffer; DXGI_FORMAT IndexFormat; ID3D11ShaderResourceView ** Resources; unsigned int IndexCount; }; } } #endif #ifndef _MESHBUFFERDESC_H_ #define _MESHBUFFERDESC_H_ namespace Ruined { namespace Graphics { // Used for creating vertex buffers. // Only accessed at load time. struct __declspec(dllexport) MeshBufferDesc { public: unsigned int BufferWidth; void * Data; }; } } #endif Lastly here is the Model destructor: Model::~Model(void) { if(Textures != nullptr) { for(int t = 0; t < TextureCount; t++) { Textures[t].TextureContent.reset(); Textures[t].TextureName = nullptr; } } if(MeshDrawDatas != nullptr) { for(int m = 0; m < MeshCount; m++) { if(MeshDrawDatas[m].IndexBuffer != nullptr) { MeshDrawDatas[m].IndexBuffer->Release(); MeshDrawDatas[m].IndexBuffer = nullptr; } for(int v = 0; v < MeshDrawDatas[m].VertexBufferCount; v++) { if(MeshDrawDatas[m].VertexBuffers[v] != nullptr) { MeshDrawDatas[m].VertexBuffers[v]->Release(); MeshDrawDatas[m].VertexBuffers[v] = nullptr; } } } } }   Holly cow! That's one long post. If anyone could take the time to read this, even just part of it, and lend me a hand, I would be very thankful.
  12. LarryTheKing

    Point-light normal issue

    Well, I feel like a complete idiot! I found the error... I wasn't sending the normal-buffer to the graphics card! I have no idea how I missed that Oh, well. Live and learn! -Thank you to everyone who looked this over
  13. LarryTheKing

    Point-light normal issue

    I meant to add - since they're at ~90 degrees to each other, at least one component of the normal should be very different [/quote] You're right is odd.. All I'm doing to create the normal buffer is multiplying each vertex's normal by the World matrix. Then moving the [-1, 1] to the [0, 1] range. It seems pretty straight forward. [source lang="cpp"]PreVertexShaderOutput PreVertexShaderFunction(PreVertexShaderInput input) { PreVertexShaderOutput output; float4 worldPosition = mul(input.Position, World); float4 viewPosition = mul(worldPosition, View); output.Position = mul(viewPosition, Projection); output.Depth.xy = output.Position.zw; output.Normal = mul(input.Normal, World); return output; } PrePixelOutput PrePixelShaderFunction(PreVertexShaderOutput input) { PrePixelOutput output; output.Normal.xyz = (normalize(input.Normal).xyz * 0.5f) + 0.5f; output.Normal.a = 1; output.Depth = input.Depth.x / input.Depth.y; return output; }[/source] Yet something is throwing off my normals. The walls should appear flat. Here are some more pics: Any ideas?
  14. LarryTheKing

    Point-light normal issue

    I'm pretty positive that the calculated scene-world-position is correct, and the Light position is a parameter, also in world space. I had been doing attenuation using just the distance between these two points, which had worked fine; leading me to think my problem is elsewhere.
  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!