Sign in to follow this  

[web] Low-Cost Hosting w/ lots of Storage, Transfer/Mo, and Bandwidth?

This topic is 4376 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

On a site that was quite helpful to me (wc3sear.ch), the owner has stated that he has more content to put up but can't afford more space with his current host. In an attempt to help (and to satisfy my curiosity), I've been looking around for hosts that might meet the site owner's needs. The site is mainly a file host, so the main resources needed are storage, high monthly transfer limits (or unmetered), and a decent amount of bandwidth. The site has a lot of traffic (wc3sear.ch stats), exceeding 1000GB one month in the last year and maintaining an average monthly usage of around 850GB. This means that the ideal hosting solution would be a shared plan, because the actual machine resources won't be fully utilized by the site (unless the machine is relatively low-powered) and other hosting solutions (virtual or dedicated servers) cost far more, especially when you add in buying extra bandwidth (because the amount needed is around 1250GB to support some growth and any usage peaks, which is an odd number). Few hosts offer anywhere near that much transfer, much less on a shared hosting plan. I've found a few that offer amounts in that area, but many of them have silly rules (such as $20/mo for the first 1000GB, then $6 per GB over, which for Aug 2005 would have cost $144 extra) or are unwilling to give details (such as one host that claims unlimited transfer/mo, but won't give any hints as to what kind of bandwidth the machines have besides stating the company, which also does virtual and dedicated servers, has multiple OC-192 lines). So far, the 3 hosts I've found that almost meet the needs are GoDaddy, CHIHost, and 1and1. GoDaddy is the one that charges outrageous amounts for extra bandwidth without providing any other method to increase bandwidth available (such as combining multiple hosting accounts). CHIHost is the one that claims unmetered, but won't give me any hints as to what their servers can handle, so I'm thinking about buying a month myself and doing some stress testing with the help of certain huge communities. Also, half their replies included > instead of > for quoting, but did not set the content type to HTML, which worries me somewhat as to their technical competency (even more so since, after I pointed it out, they said something to the effect of "yes, we use plain text only" yet still included the error). I haven't talked to 1and1 yet, but personally I've had some bad experiences with them so I'm reluctant to suggest them as a possible option or to investigate them further for viability. I've already tried the "Host Quote" on WebHostingTalk, but unfortunately it doesn't go high enough in the drop downs. I put my actual requirements in the comments section, but it seems to be entirely ignored because of all the replies, only CHIHost actually had any plans that came close to what was needed. I made sure to kindly thank each automated reply for wasting my time =-( Does anybody have any ideas? The price the person gave me for what they pay currently was $800/yr, and they have unlimited transfer. I'm not sure how much space he has currently (I've asked, but no reply yet), but he mentions that he needs 4GB more so I figure quite a bit. Any suggestions would be greatly appreciated, whether they be for actual hosts, ways to find them, or any other options. [Edited by - Extrarius on December 19, 2005 11:10:32 AM]

Share this post


Link to post
Share on other sites
For practicality, is it possible to host torrents rather than serving the files directly? It looks like it would save a ton of transfers.


Hosting over a terabyte per month is going to cost a lot, no matter where you look. I'm guessing you want around 6 or 7 Mbps dedicated lines.

When you're looking at that transfer range, web sites are not your best source of information. You need to actually call and discuss various hosting options.

Consider XMission.com

Connectivity is good:
# LEVEL3 - Internet Backbone connection. (155Mb\OC3)
# AT&T - Internet Backbone connection. (155Mb\OC3)
# UUNET - Internet Backbone connection. (45Mb\DS3)
# XO - Internet Backbone connection. (45Mb\DS3)
# PAIX - Internet peering connections in Palo Alto, CA. (45Mb\DS3)
# CommIX - Internet peering connections in Utah. (45Mb\DS3)

Toll-free: 1.877.964.7746
in Salt Lake City: 801.539.0852
sales@xmission.com

If you decide to go with them, send me a PM so I can get a referral kickback.

Or look for something local in your area. I have no idea where The Psionic Ward is located in the world. [grin]

frob.

Share this post


Link to post
Share on other sites
You guys are as bad as the hosting companies =-(

I realize this isn't a normal need, but I certainly wouldn't be willing to spend anything like $129/mo just to host files for the public without some incentive to do so.

I'd rather buy the $20/mo 1and1 plan that offers 30GB of space and 1500GB of transfer each month, which seems to be the only reasonable (if it is reasonable) hosting solution that might work, and trust me when I say I don't want to put foward 1and1 as a possibility.

The limit is $800/yr, because that is the amount the owner is already paying, and I'm betting he'd be much happier to spend even less.

Any ideas besides 'buy the expensive plans'?

Quote:
Original post by kanzler
1&1 claims to have 120 GB managed server hosting with 1,500 GB bandwidth for $129/month. That sounds like a pretty good deal.[...]
Not as good as the "1&1 Developer" hosting package that offer 1500GB of transfer and 30GB of space for only $20/mo

Share this post


Link to post
Share on other sites
Quote:
Original post by Extrarius
Any ideas besides 'buy the expensive plans'?


For a terabyte of uploads per month? At least a continuous 3.2Mbps upload rate, and probably 7-10Mbps minimum burst rate preferred? For under $67/month?!

Nope.

You get what you pay for.

frob.

Share this post


Link to post
Share on other sites
Quote:
Original post by frob
Quote:
Original post by Extrarius
Any ideas besides 'buy the expensive plans'?


For a terabyte of uploads per month? [...]
Each of the hosts I listed offer shared plans with at least 1000GB/mo of transfer for under $20/mo. I'm just looking for additional such plans or other inexpensive options.

Share this post


Link to post
Share on other sites
I just browsed through the Terms & Conditions of 1&1. In section 2.1.5 they state: "Bandwidth use, including but not limited to data retrieval from Your Web Site, e-mail traffic, and downloads, shall not exceed six gigabytes per month. Your combined Mailbox use per account shall not exceed twenty-five gigabytes per month." How does that relate to the 1,500 GB bandwidth they offer?

Share this post


Link to post
Share on other sites
Looking at the hourly stats of August 2005 (month of most transfer from Jan 2005 to Nov 2005), there was a maximum average hourly transfer of 1592736 KB, which gives 442.43 KB/sec. Presuming that by KB the stats site means 1024 8-bit bytes, a 4Mbps connection would handle peak usage with up to a maximum of 15% overhead, which would require each TCP packet to be 427 bytes long under the worst-case condition of a maximum-sized(64 bytes) TCP header in each packet.

I'm sure you could get slightly higher burst transfer requirements if you took the time to correlated the hourly and daily statistics and build some kind of statistical model, but that's too much much work for me =-)

Share this post


Link to post
Share on other sites
As mentioned in the first post, I'd still look at ways to reduce your bandwidth requirements rather than looking for ways to get more bandwidth.

I didn't bother to look at your stats page at first, but now I have.

Based on your stats page, over half your traffic is coming from a single page, your home page, which is over 100KB in size! That's lunacy to have that big of a / page.

I'm going to scream this as loud as I can:


You are wasting a half terabyte of data transfers with your stupid home page design!!




Getting that file down to 20K would have saved 450 GB in August!

Link in your javascript, rather than embedding it in the page
Use style sheets more than you already are
Stop using tables, if you can manage it
Use shorter names. One file name, "1133892430ss1131727905Image2.jpg", is over 60MB when transferred 2219657 times.

class="collapsbarunused" occurs 186 times. It only occurs inside a certain type of table. Moving that style to the table (with an obfuscated name like class="t") would drop another 2 GB of data.

Similarly for your names: id="topdownloadstools_pos15", "gfx/border/bottom_left_s.gif", "gfx/border/bottom_right_s.gif", "style="font-size:8pt;font-family:Arial;", and hundreds of other things spread all over that page.



You don't need 1TB of transfer per month. You don't even need 500GB of transfer per month.

You need to hire a better web programmer.



frob.

Share this post


Link to post
Share on other sites
Two things:

1) site design actually matters when it comes to transfer costs

2) if you want premium performance, you have to pay a premium price

If you have so many visitors, Google text ads might pay for hosting for you.

The networking & multiplayer forum FAQ has some suggestions for hosting providers to try, without any specific endorsements. If you want to do 2000 GB/month, you can get that for about $99/month from ServerBeach, which is fairly reputable (but that's more than $800/year).

Share this post


Link to post
Share on other sites
frob:
1) The site isn't mine in any respect, so your emphasis-formatting is wasted on me. I never bothered looking at the things you mention, but I will pass your suggestions along.

2) index.php does everything (as far as the web page goes, just about every link on the site is to index.php with different parameters), and it appears the stat system folds it into the stats for "/", so it isn't quite as bad as it seems. Consider that for every single download, a user must navigate several pages (all index.php plus parameters) to get to a single download, and it becomes apparant that even with a simple theme, there would be a relatively large amount of traffic going through "/".

I agree, though, that there is much room for improvment, and I'll let the owner know as much.

hplus0603:
The site already has google ads, at least on the front page.
I don't think what is needed is exactly "premium performance", though I guess it's a matter of perspective scale. Certainly, what I'm looking for would be the highest end shared plans.
2000 GB/mo for only $99 is the best deal I've heard (presuming it includes more than just the transfer =-), and I'll look into it.

Share this post


Link to post
Share on other sites
Im not a web designer/programmer or anything, but i had a thought:

you say you get 2000 GB for 99$ in one host, and 1000GB for 20$ at another,

why not get two of the other host? that will be 2000GB for 40$
and 40$ per month is way below the 800$...
you can maybe host half the files on one of them and half on the other.

i know it cant be this simple... what did i miss?

Share this post


Link to post
Share on other sites
Quote:
Original post by Iftah
Im not a web designer/programmer or anything, but i had a thought:

you say you get 2000 GB for 99$ in one host, and 1000GB for 20$ at another,

why not get two of the other host? that will be 2000GB for 40$
and 40$ per month is way below the 800$...
you can maybe host half the files on one of them and half on the other.

i know it cant be this simple... what did i miss?
It would be possible, except that with GoDaddy (the one that offers 1000GB), you can't combine accounts (in the sense of buying two and treating them like one). You could do so manually as you suggest, but then you have to worry about one of the two running out of bandwidth, so you'd have to shift files as different ones become more and less popular. To say the least, that would be a pain and would require constant attention.

Share this post


Link to post
Share on other sites
You said that they guy now pays $800 a year, right. Well, for that kind of cash you can go dedicated. Here's one I have good experceinces with: Layered Technology. The link goes to a $65/month dedicated server with 1024 GB bandwidth limit. The machine ain't too powerfull (AMD Sempron 2600 with 512 MB RAM and 80 GB harddrive) but you said you don't need that. They offer far more bandwidth though. The standard packages go up to 33 Terrabyte (33000 GB). I think if you need even more than that, they can provide if you ask [smile]

Share this post


Link to post
Share on other sites
From my experiences GoDaddy is the devil unless you plan to stay with them FOREVER. Moving a domain from them is more painful then wisdom teeth being removed with no drugs by a drunk bum on the street with no tools.

Just trying to steer you in the right direction. You can also look at getting a second cheap host and doing some files on there and others on another server and split the bandwidth.

Share this post


Link to post
Share on other sites
Sander: While dedicated is possible for under $800/year, dedicated with enough bandwidth is not. To add only 200GB to the plan (because the site exceeded 1000GB peak), the price more than doubles to $140/mo, and that still doesn't include a control panel(which few people can do without).


Mike2343: Why do you say that? If you transferred the domain several years ago, it was required for them to get all kinds of stuff from you by whoever is in charge of domain names (forgot the name), but these days it just takes an email and a few replies/clicks because the head company changed the requirements. Anyways, GoDaddy is the least expensive company for domains that has competent support IME, so I don't plan on transferring the many domains I have with them anywhere else.

Share this post


Link to post
Share on other sites
Quote:
Original post by Extrarius
Sander: While dedicated is possible for under $800/year, dedicated with enough bandwidth is not. To add only 200GB to the plan (because the site exceeded 1000GB peak), the price more than doubles to $140/mo, and that still doesn't include a control panel(which few people can do without).


You could contact them about the possibility of only paying extra in the months that you actually exceeded the 1024 GB/month limit. That goes for all your hosting plans BTW, not just LayeredTech. A TerraByte is pretty much the "standard" limit out there. If you want more for cheap, negotiate. Especially if you don't need it all the time

Also, LayeredTech provides Plesk for free, which is every bit as good as CPanel.

Share this post


Link to post
Share on other sites
Since it's game related. Why does'nt he partner with Fileplanet or another game mod/demo hosting site. I'm sure they would welcome the traffic and revenue from signups.

gamershell
actiontrip
filefront

etc

Share this post


Link to post
Share on other sites
Quote:
Original post by Sander
[...]You could contact them about the possibility of only paying extra in the months that you actually exceeded the 1024 GB/month limit. That goes for all your hosting plans BTW, not just LayeredTech. A TerraByte is pretty much the "standard" limit out there. If you want more for cheap, negotiate. Especially if you don't need it all the time[...]
Sure, but most sites charge outrageous amounts, such as GoDaddy's charging $6/GB. If LayeredTech allowed it to be payed for per GB (at ~$0.38, rougly the 200GB/$75), it'd be the least I've seen charged for going over. Usually, though, hosts have a separate and distinct price for overages that is several times the base cost for a plan with that much space. I guess I might as well email them, though.

Share this post


Link to post
Share on other sites
i still think the twice cheap host has a chance,
cant you web programmers make a script to balance the load?

for example (pseudo code)

hostA = 0.5
hostB = 1-hostA

// for each link (download file?)
onClickLink() {
if (rand(0..1) < hostA)
gotoLinkInHostA;
else
gotoLinkInHostB;
}



this way you can change the load on the two host with one parameter.
perhaps you can automate it so that it will keep track of used bandwidth and set the param to balance the load? (and even maybe stop the downloading when nearing the limit).
I dont know how limited the scripts languages are, but the logic seems simple.

Iftah.

Share this post


Link to post
Share on other sites
The logic is simple, yes, but if you do it like that then you don't get doubled space because each file is in two places, so you have to pay for more space on a single account which usually is bought per 100MB for a high price, which makes it not worth it.
The only way the idea would be viable would be to have only part of the site at each URL, and then files would have to be moved based on popularity, and moving the file would take time and bandwidth itself. It could MAYBE be automated, but it would still take a long time to transfer the file and would use bandwidth so it couldn't be done constantly.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Id say K.I.S.S

since apparently you cant find a single server to put everything on:

Simply put the less popular downloads on the extra server with less space and bandwidth. Im sure you already have statistics for which are popular and which are not.

Share this post


Link to post
Share on other sites
What i would do (this is just me tho...)

Make a \index.dat file, which stores which parts of which files are where. (ie. Randomfile-part2.dat is at www.something.com\Shared\Randomfile-part2.dat)

Your prog then downloads \index.dat, reads off where each part is of the file.

It then downloads a few (like 2-3) simultaniously off different websites. (ie. downloading part 1,3,and 72).

This then lets you download faster. (they do a similar thing in limewire, kazaa, ect. so your not capped by the other peers upload speeds), and it lets you share the burden over many different servers.

Then you just shove them together in one big file at the end. (which is pretty easy to do).

Overall time for project: One day (including debugging, testing and converstion)

From,
Nice coder

Share this post


Link to post
Share on other sites
Nice Coder: You are suggesting either "load balancing" (if done on the server side, which is _very_ expensive) or that a special download program be required, which would be insanely annoying to all the users of the site. Not only that, but it would requrie a lot of work to set up and maintain, and would would require more than a day of work due to the numerous versions that would be required to support various operating systems.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
well it would seem like your screwed, since all your doing is shooting down every idea right away and not contributing anything yourself besides a "That wont work, stupid" attitude. There are SEVERAL good solutions and tips on this page (especially the mention of reducing your start page), good luck asking for more help since your only contributing a VERY negative attitude to the problem. Since you know everything, why dont you come up with a solution on your own from now on.

I recommend giving up, because its not your problem and your not going to be satisfied with any solution that either takes more effort or costs more.

Share this post


Link to post
Share on other sites

This topic is 4376 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this