Is a 100% accurate file download accelerator possible?
Author
Download Accelerator Plus
Fresh Download
Getright
I used them all before (some I abandoned because they contained spyware) and the work pretty well for the most part. But the problem is when I try to download files that are about 300 megs the are frequently corrupted.
I have a 56k connection which means its takes about 2 - 3 days to download one of these 300 meg files ( for example the direct x sdk )
I think the problem comes from multiple resuming but dosent the download client have a way of calculating the check sum fo see if an error occured?
I am in the process of making a file update module that I plan to use for downloading the latest release of all my files to those interested. I want to know if it is too optimistic to think that I can get a file to be downloaded with 100% accuracy.
gdipong
oglpong Online (43% complete)
If you got any requests or suggestions post here
Its perfectly possible to get 100% accurate results for downloaded files, it just might take a little longer due to the extra overhead. IIRC, TCP only uses simple checksums on each packet for checking for corrupt data, and with a large file the odds of getting a damaged packet which still appears to be valid increase.
I suppose a quick fix would be to download files over SSL which includes a more robust MAC method for checking data accuracy.
I suppose a quick fix would be to download files over SSL which includes a more robust MAC method for checking data accuracy.
check out bittorrent; it''s essentially doing the same thing, only using other clients instead of multiple download streams from the same server...which could cause even more problems, but it verifies the download itself. from the official bittorrent faq:
quote:How do I know the download isn''t corrupted?
BitTorrent does cryptographic hashing (SHA1) of all data. When you see "Download succeeded!" you can be sure that BitTorrent has already verified the integrity of the data. The integrity and authenticity of a BitTorrent download is as good as the original request to the tracker. Checking the MD5/CRC32/other hash of a file downloaded via BitTorrent is redundant.
That''s what they do now; the problem is you don''t know the file is corrupted until it''s finished and you calculate the md5 for it. If they don''t match, what do you do? Delete the file and start over...
The download-resumers would need to be smarter, and re-start the download a few kilobytes back from the current position and verify the data coming down matches what''s already on disk. If it doesn''t, it needs to start over then.
The download-resumers would need to be smarter, and re-start the download a few kilobytes back from the current position and verify the data coming down matches what''s already on disk. If it doesn''t, it needs to start over then.
Author
Finding out about bittorrent leaves me in a situation where I wonder if I may have to reinvent the wheel.
I wonder if there is an opensource network libary I can use to make the update module without having to learn how to do these kinda thins by my self.
oglpong Online (47% complete)
gdipong
If you got any requests or suggestions post here
I wonder if there is an opensource network libary I can use to make the update module without having to learn how to do these kinda thins by my self.
oglpong Online (47% complete)
gdipong
If you got any requests or suggestions post here
quote:Original post by Magmai Kai Holmlor
That''s what they do now; the problem is you don''t know the file is corrupted until it''s finished and you calculate the md5 for it. If they don''t match, what do you do? Delete the file and start over...
The download-resumers would need to be smarter, and re-start the download a few kilobytes back from the current position and verify the data coming down matches what''s already on disk. If it doesn''t, it needs to start over then.
What I meant is add an MD5 to each PACKET, so that if a packet is corrupt, you request it again.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
