Jump to content
  • Advertisement

Archived

This topic is now archived and is closed to further replies.

try_catch_this

Is a 100% accurate file download accelerator possible?

This topic is 5293 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Download Accelerator Plus Fresh Download Getright I used them all before (some I abandoned because they contained spyware) and the work pretty well for the most part. But the problem is when I try to download files that are about 300 megs the are frequently corrupted. I have a 56k connection which means its takes about 2 - 3 days to download one of these 300 meg files ( for example the direct x sdk ) I think the problem comes from multiple resuming but dosent the download client have a way of calculating the check sum fo see if an error occured? I am in the process of making a file update module that I plan to use for downloading the latest release of all my files to those interested. I want to know if it is too optimistic to think that I can get a file to be downloaded with 100% accuracy.
gdipong oglpong Online (43% complete) If you got any requests or suggestions post here

Share this post


Link to post
Share on other sites
Advertisement
Its perfectly possible to get 100% accurate results for downloaded files, it just might take a little longer due to the extra overhead. IIRC, TCP only uses simple checksums on each packet for checking for corrupt data, and with a large file the odds of getting a damaged packet which still appears to be valid increase.

I suppose a quick fix would be to download files over SSL which includes a more robust MAC method for checking data accuracy.

Share this post


Link to post
Share on other sites
check out bittorrent; it''s essentially doing the same thing, only using other clients instead of multiple download streams from the same server...which could cause even more problems, but it verifies the download itself. from the official bittorrent faq:

quote:
How do I know the download isn''t corrupted?

BitTorrent does cryptographic hashing (SHA1) of all data. When you see "Download succeeded!" you can be sure that BitTorrent has already verified the integrity of the data. The integrity and authenticity of a BitTorrent download is as good as the original request to the tracker. Checking the MD5/CRC32/other hash of a file downloaded via BitTorrent is redundant.


Share this post


Link to post
Share on other sites
That''s what they do now; the problem is you don''t know the file is corrupted until it''s finished and you calculate the md5 for it. If they don''t match, what do you do? Delete the file and start over...

The download-resumers would need to be smarter, and re-start the download a few kilobytes back from the current position and verify the data coming down matches what''s already on disk. If it doesn''t, it needs to start over then.

Share this post


Link to post
Share on other sites
quote:
Original post by Magmai Kai Holmlor
That''s what they do now; the problem is you don''t know the file is corrupted until it''s finished and you calculate the md5 for it. If they don''t match, what do you do? Delete the file and start over...

The download-resumers would need to be smarter, and re-start the download a few kilobytes back from the current position and verify the data coming down matches what''s already on disk. If it doesn''t, it needs to start over then.


What I meant is add an MD5 to each PACKET, so that if a packet is corrupt, you request it again.

Share this post


Link to post
Share on other sites

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!