Im not saying that i don't want to use another known and tested algorithm, i just wanted to know the most noticables things i could improve this algorithm on.

In fact, i've found a simple aes-256 bits algorithm in c, and might include it in my own library later too.

What i meant is, unless a cryptographier tried to decript my files using my encryptor, which is nobody i know of in the remote area where i live, could crack it. Hell i've made it and couldn't even crack it given lots of rounds. But i get your point. The government would probably crack it fast enough...

The thing is, my goal with this code is more about speed than security, if im worried about security, then ill use a well known method in this case.

The only thing i ever cracked this way was a simple xor algorithm of jpeg files on a cd with each byte encoded using 1 pass and the same xored value everywhere haha.

After i've found the first 4-5 bytes it was rather easy... But then on another cd they used more round and/or more xored value and then i gived up. That was a long time ago though, like 15 years ago. That just about the level of security im after, something not too simple, but not too complicated too so it take a long time to encrypt, and can deter most noob attackers like me.

Actually, now when I think about it, your algorithm is much worse than that. Since you only encrypt a byte at a time, and the sequence of random numbers is fully known, you effectively only have a 8-bit XOR cipher. Not only is brute force viable, but even manual brute force BY HAND is viable since you effectively only have an 8-bit key and thus 256 combinations to manually examine.

There's a flaw in that reasoning...

Think of the "one time pad" encryption scheme. In the one-time-pad scheme, you have a seemingly random sequence of essentially infinite length to encrypt the data with. Now, you need only encrypt one byte at a time, heck even 1 bit if you like, and this is known to be uncrackable. It's essentially a key of infinite length, or rather the key is as long as the data.

Now relating that back to the proposed scheme here...
The sequence of random numbers is only "fully known" in the sense that the PRNG does eventually repeat its sequence, and so the key length is not infinite. But with the PRNG chosen here, that sequence is extremely long, and would not repeat for the length of the data being encrypted, making it effectively as secure as the one-time pad in that respect. However, the weakness is that where is starts has 2^32 possibilities. Basically you don't really know the sequence at all, but you it is one of X possible sequences.

Calling it effectively an 8-bit key is thus very much incorrect. Your earlier post about the key being 2^32 in size was correct though, because it's like choosing between 2^32 keys of a very long length.

Oh and this entire idea is not new to me, not in the slightest, although I never did bother implementing it yet.
Personally I would use a larger seed. Good PRNG algorithms have an internal state that is much larger than 32-bits. XORShift for example has 128 bits internally, and I'd probably use that and initialise all 128 bits to a 128-bit hash of the password, if I were doing this.
I'd also XOR 32-bits at a time, not because it's more secure, but because it's more efficient.

Well, im simply trying to find a way to encrypt the data that pass through the network of my RemotePC project, for example, a kind of VNC so easy to use that even my mother can use it, everything work fine, it's a finished project, i was just wondering if i should change my encryption scheme. Im mostly trying to hide the handshaking part of the connection, along with screenshots and keypress, mouse events ect. Of course im assuming that both side are "good" actor otherwise encryption is pretty useless. Im more concern about what they call the "man in the middle" attack, like someone sniffing out your packet, although firewalls pretty much eliminated most trojan based virus. Im not saying that a firewall is infallible but that it does a pretty good job at what it do. Im pretty sure an attack on my software, which only me and my mother own is almost inconsivable, it just make my mind feel a little better about it. By encrypting the data, how could they guess the data transfer protocol?

And i also wanted to make file encryption more random-like, instead of leaving a "trace" of the original image, as i was saying so, at the very least, an average hacker at the file couldn't guess what's inside, defeating the encryption in the first place.

Im fully aware that my code isin't very secure, i just do this to experiment mostly. I might try other tested encryption technique too when i have the time, i just love to do everything in my projects by myself, that's all.

Just use a known asymmetric encryption scheme. Secure network transmission is a well-explored problem, and believe me when I say you're just begging for someone to come along and break your attempted encryption in a matter of minutes of reverse engineering.

So what algorithm would be fast AND secure enough for this kind of thing? (network encryption)

So what algorithm would be fast AND secure enough for this kind of thing? (network encryption)

SSL/TLS with a decent cipher suite.

There are an immense number of pitfalls you can (and will) fall into if you try to design your own encryption algorithm and protocol. A few of them:

- inadequate privacy, this just means your cipher sucks and can be broken by simple cryptanalysis or brute force

- inadequate integrity, someone can modify your stuff in transit and you haven't implemented secure checks for this

- inadequate authentication, someone can impersonate whoever you are trying to communicate with (by intercepting the TCP connection and imposing himself as a middle man) and you wouldn't even know

- replay attacks

- weak password exchange schemes

- side channel attacks

- reverse engineering

- and a thousand more threats you and I cannot even begin to conceive

Also, in cryptography, you always prepare for the worst. Saying that only you and your mother are going to use the application is *not* a valid argument. If that is the case, and you are not anticipating anyone trying to attack your protocol, why even bother? You may as well do nothing at all, since you don't know what you are going to defend against. Otherwise, you need to set up a realistic threat model (for instance, I agree it is unlikely your protocol will be targeted by an APT - advanced persistent threat, that's stuff like governments and security agencies - but a lone bored hacker might come across your router, scan it, find an open port, eavesdrop on your protocol, learn it, break it, and then start doing lots of damage).

With some cryptography training it is plausible to create your own limited, specialized cryptographic high-level protocol for when nothing else really works, but is still requires a lot of work and peer review, and that does not include whole ciphers and hash functions. If you do that for anything other than learning and fun (which is a fine goal in and of itself, by the way - cryptography is fascinating), then you have failed as a responsible developer.