19 replies to this topic

[MarkS]

OK, I know that it is a world-wide network of computers. My question is actually more low level and more or less concerns ISPs.

The way I look at an ISP is that it acts as a gate. On one side of the gate is the mysterious "source" and my computer is on the other side. The ISP allows data to travel from "the source" to my computer and vice versa.

The question is, what is "the source"? What does the ISP connect to? Where does the connection come from? Since an ISP is really nothing more than large computers with industrial routers, why am I unable to directly connect to "the source"? Why is it not possible to connect my router to this "source" and bypass the ISP?

It is a question that I have always wondered and have never found an answer.

[Matias Goldberg]

Why is it not possible to connect my router to this "source" and bypass the ISP?

It is. Just hook up a cable to that "source" and you'll be done.
Problem is, that source may be on the other side of the world, so that must've been hell of a long cable. It is very expensive and inefficient for you (the only gain you'll get is direct access). We pay ISPs, many ISP just actually pay to the (usually) same wholesale provider; which sell by bandwidth.

With this system costs are reduced significantly and those wholesale providers are wired together. Sometimes these "providers" depends on the Government, that varies per Country.

Usually the communication is done all through satellite, with optical fiber cables going underground/underwater as a backup (satellite communication can be very unstable, depends on weather conditions on the ground, environmental hostility (i.e. a bird nest inside the satellite, I'm not kidding this happens too often), proper orientation, satellite availability and outer space conditions (i.e. solar flares).
Optical fibers are used as a backup because they are very expensive to maintain (underwater cables are somehow specially attractive for some creatures) and don't usually provide the same level of bandwidth. Not to mention it requires signal strength to be mantained, the longer it is, the more expensive it becomes.

Feel free to pay a giant bill to hook my router to yours if you want direct access to my PC. It's not a server so you won't find it very interesting though.

How the ISP knows which ISP to query to contact it's client , which will be the "source", based on just the IP number is a completely different question, and has to do about how the IP (Internet Protocol) works. TCP and UDP are built on IP as it's base.
Beware though, IP is an old protocol and works much more chaotic than one would think. Often people who know how it works ask themselves how the internet manages to hold together and stay functioning.

Cheers
Dark Sylinc

Edit:
To summarize, the connection goes (usually!) like this:

• PC -> ISP A
• ISP A -> Wholesale provider X
• Wholesale provider X -> Wholesale provider Y
• Wholesale provider Y -> ISP B
• ISP B -> PC "Source"

There aren't many providers (typically 1 to 3 per country). This communication is overly simplified since the IP protocol actually asks a lot of people before reaching the end, jumping (wasting) on a lot of computers first before reaching what you've called the "source". And the communication back may not follow the same route. Not even the same packet you've sent may follow the same route. Your ISP A doesn't magically know ISP B is your source's provider, so it has to start asking. And sometimes, those who have been asked don't know the answer but are aware of someone who may know, so they ask them too...

[MarkS]

Ah. So, basically you're saying the answer is "magic". Gotcha!

[SiCrane]

What ISPs mostly connect to is each other. The really simple version is that each ISP is given a block of IP addresses. They then assign the IP addresses to their own hardware and to their customers. Then each ISP maintains connections to one or more other ISPs. Through various protocols the ISPs tell each other which IPs they have access to. For more reading you might want to look up BGP (Border Gateway Protocol), which is mostly how ISPs communicate with each other. For comparison you may want to look up interior gateway protocols like OSPF or IS-IS.

[jbadams]

Ah. So, basically you're saying the answer is "magic". Gotcha!

From the user's point of view, yes! This is why we call it "the cloud", and even display it that way on network diagrams -- we just plug in at one end and are able to communicate with a device at the other -- we don't generally know or care the specifics of what happens in between.

[Sirisian]

It's like a bunch of tubes. You pay to have a tube to your house so you can put stuff in it and send stuff. The term ISP is kind of hiding a lot of the details. There are hubs at street corners which aggregate and multiplex your data to put it in a single larger tube. Then the tubes go to ISP "offices". Then you have the giant fiber networks from chicago to washington DC to california etc. It ends up being a very complicated graph resulting in routing fun and a huge theory for proactive routing. If you have the time go take a few undergraduate/graduate level networking courses. They are fun. Also read Computer Networking a Top Down Approach. It's a very good primer.

[Eelco]

Usually the communication is done all through satellite, with optical fiber cables going underground/underwater as a backup (satellite communication can be very unstable, depends on weather conditions on the ground, environmental hostility (i.e. a bird nest inside the satellite, I'm not kidding this happens too often), proper orientation, satellite availability and outer space conditions (i.e. solar flares).
Optical fibers are used as a backup because they are very expensive to maintain (underwater cables are somehow specially attractive for some creatures) and don't usually provide the same level of bandwidth. Not to mention it requires signal strength to be mantained, the longer it is, the more expensive it becomes.

It seems to me this opinion is rather wrong. Both satelite and cable are optical technology. Except that the cable takes a shorter and much more well defined path leading to vastly better signal to noise, can be boosted anywhere along the path without the constraints of outer-space technology, and probably many others im forgetting now.

The vast majority of data travels by cable. Only if you are on an oilrig does your data go by satelite; you pay dearly for it, and it lags like hell.

[Antheus]

The question is, what is "the source"? What does the ISP connect to? Where does the connection come from?

The internet backbone. It works the same way as your home connection, just the cable is bigger, routers have more blinking lights and it's slightly regulated. Also, very expensive.

A simplified version is: internet connections between continents, countries and cities are one very thick cable. Then there is a house at either end with a bunch of very fancy routers into which all local ISPs connect with their cables. And that's it. And all the traffic from that country, city, even continent, goes through that one cable. So when that gets cut, millions of machines lose all connectivity.

Why is it not possible to connect my router to this "source" and bypass the ISP?

You can. It's expensive, regulated and supervised, but you can. Most of the big universities are directly on the backbone, that is likely the easiest way.

[samoth]

bird nest inside the satellite, I'm not kidding this happens too often

Alien space bird breeding attack? The amazing feat of flying that high left aside, where do they find twigs to build their nests?

By the way, I can ping most machines on the other side of the Atlantic Ocean in little over 100-120ms, which strongly suggests that the data does not go via sattelite, which would be around 240ms one-way due to the mere speed of light (so 480ms round trip, assuming no additional router delay).

[Matias Goldberg]

bird nest inside the satellite, I'm not kidding this happens too often

Alien space bird breeding attack? The amazing feat of flying that high left aside, where do they find twigs to build their nests?

Hahaha, sorry I was talking about the satellite dish. Today dishes are smaller because they're cheaper and easier to maintain; if there's some signal/bandwidth limitation, just add one more.
But the "ancient" dishes from the 60's onwards when they constructed everything in gigantic size, costing several millions, are still being used. Birds tend to build their nests near the center of the dish, and less often where the receiver is, provided it is big enough.
It's a PITA for the engineers in the ground stations because it's the last place they want to check. Orientation & satellite's route can be check from your desk; while a nest means going out and climbing to the dish to visually inspect it.
If it's raining, (side-note: ground station's locations are carefully chosen to reduce any environmental noise or hazard, like earthquakes, volcanos, radio interference, snow; but no place on earth offers a natural protection from everything at once) then it's easier to blame the weather conditions for the bad signal than rather go out with the rain and start climbing. Like I said, a major PITA.

By the way, I can ping most machines on the other side of the Atlantic Ocean in little over 100-120ms, which strongly suggests that the data does not go via sattelite, which would be around 240ms one-way due to the mere speed of light (so 480ms round trip, assuming no additional router delay).

Yeah, most likely over cable. Satellites have terrible latency. Perhaps I made a wrong choice of words. Used as "secondary" is probably a better word than "backup". Wholesale providers often supply all kinds of data (TV, Phone, VoIP, cell phone, Internet), not just internet. When satellites go down cable receives a lot of stress: TV channels down intermitently, failed international calls, slow internet (if not solved before peak hours).

[Bregma]

Ah. So, basically you're saying the answer is "magic". Gotcha!

No, no. It's just sufficiently advanced technology.

[Eelco]

Perhaps I made a wrong choice of words. Used as "secondary" is probably a better word than "backup". Wholesale providers often supply all kinds of data (TV, Phone, VoIP, cell phone, Internet), not just internet. When satellites go down cable receives a lot of stress: TV channels down intermitently, failed international calls, slow internet (if not solved before peak hours).

No, you did not make a wrong choice of words: it is in fact your very opinion that is wrong, as they say.

http://en.wikipedia.org/wiki/Submarine_communications_cable#Importance_of_submarine_cables

99% of all traffic going through underseas cables makes them neither 'backup' nor 'secondary'. If anything, it is the other way around, if you take the generous interpretation of calling less than one percent of capacity a meaningful backup, that is.

[Matias Goldberg]

99% of all traffic going through underseas cables makes them neither 'backup' nor 'secondary'. If anything, it is the other way around, if you take the generous interpretation of calling less than one percent of capacity a meaningful backup, that is.

Hi, this has the potential of becoming a flame war, so I'll be brief.
The article in Wikipedia fails to cite that quote. There are two citations in that parragraph, and neither of them talk about the 1% vs 99%. Furthermore Wikipedia talks about cables as if they were all already made and links to articles about underwater cables, some which have actually started working last year, and some others yet operational.
Despite this, there's a shift to submarine cables, that's true; but I'll like to see reliable sources regarding the traffic comparison.
Optic fibers offer significantly lower latency and more bandwidth, but they're very expensive; and have their own set of problems: underwater earthquakes, shark bites, and anchor damage.

Well, enough getting out of topic. Regardless of whether satellite or cable is used, the story is the same. Client -> ISP -> Wholesale -> ***magic*** -> Wholesale -> ISP -> Client.
I had a whitepaper that clearly explained how that magic works, but you'll have to excuse me as I can't find it. At least I may point you some directions, Dijkstra's algorithm has to do with it. Follow the link to "routing" in Wikipedia.
We use a derived version from Dijkstra's in games BTW, called "A*" which is used for pathfinding.

[Eelco]

The article in Wikipedia fails to cite that quote.

Fair enough; that places it on equal footing with your quote.

Given this divergence of opinions, and keeping an open mind as to which is the more credible source here, we should at the very least warn the impressionable parts of our audience that the jury is still out on this one, lest theyd be rash and accept your worldview without further questioning. Because then they would be completely wrong.

Well, enough getting out of topic.

Yeah. Admitting your mistake is probably the shortest way out.

[SiCrane]

I had a whitepaper that clearly explained how that magic works, but you'll have to excuse me as I can't find it. At least I may point you some directions, Dijkstra's algorithm has to do with it. Follow the link to "routing" in Wikipedia.

*cough*

For more reading you might want to look up BGP (Border Gateway Protocol), which is mostly how ISPs communicate with each other. For comparison you may want to look up interior gateway protocols like OSPF or IS-IS.

[Matias Goldberg]

I had a whitepaper that clearly explained how that magic works, but you'll have to excuse me as I can't find it. At least I may point you some directions, Dijkstra's algorithm has to do with it. Follow the link to "routing" in Wikipedia.

*cough*

For more reading you might want to look up BGP (Border Gateway Protocol), which is mostly how ISPs communicate with each other. For comparison you may want to look up interior gateway protocols like OSPF or IS-IS.

Sorry. I was feeling guilty for going off topic starting with my 2nd post, with little contribution to the original questions (it was unclear whether he was doubting about how routing works, or how communications are made internationally); so I had to throw something.

Edit: It's sad to see Eelco's and my post both with -1; it makes it look like we've both thumbed us down each other. Thumbing Eelco's post up.

[samoth]

The article in Wikipedia fails to cite that quote.

Wikipedia could probably take as circumstantial evidence the entirely accidential cable cuts that are not related to any covert operations that don't exist or wars or political motives, and regularly happen by accident.

Such as in 2005, when Pakistan was totally cut off the world, or 2008 when FLAG FEA, GO-1, SEA-ME-WE 3, and SEA-ME-WE-4 were all accidentially cut at the same time, pulling the plug for large parts of northern Africa and Middle East (basically the entire Arab world), India, and many Asian countries (some of which funnily denied the obvious fact) for two weeks. Communication during that time was extremely limited to alternative routes such as via SEA-ME-WE-3, which was accidentially cut in 2010 to a similar effect.

Ah. So, basically you're saying the answer is "magic".

No, no. It's just sufficiently advanced technology.

This advanced technology is surprisingly simple (one could say quite primitive). However, that is it's very ingeniuity, and its strength. It is of course a little more complicated, but in principle it is as easy as this:
Do I know the receiver? Does anyone else here know them? Yes --> forward to destination. No --> pass to default route, someone else will know...

[speciesUnknown]

What I don't understand is, what is the cloud? a lot of marketing materials talk about it as if its a thing as opposed to stuff, and I suspect its actually stuff, but i don't quite get what it is.

[Hodgman]

What is the postal service?

[speciesUnknown]

What is the postal service?

A socialist conspiracy. If it were left to the free market, it wouldn't exist.