http://en.wikipedia.org/wiki/Side-scrolling_game
and
http://en.wikipedia.org/wiki/Platform_game
give a description and some pictures of side scrolling games.

Often the player stays in the middle of the screen and then the background moves behind him. The items on the floor (bonus items or enemies etc) scroll along also. Enemies may move as well so you need to take that into account.

So when the player presses the right arrow to go towards the righthand side of the screen then the background image/tiles scroll to the left to simulate the player walking right.

There are special cases where the player reaches the end of the scrolling area and then they can walk to the edge of the sceen.

Other implementations allow the player to move within a small area in the middle of the screen and when they reach the edge of this small area then the background starts scrolling like described above.

Hope this helps.

You can load one huge picture, or you can chop it up into sections and only load at most two pictures at once (The player may be on the edge of two smaller images hence the need to have two images loaded at once).

Or you can create the background out of tiles. For this one you would have a 2D array for the background and each space in the array would point to a particular tile image. This is quite a common way of doing it. There should be some tutorials on this somewhere.

Edit: There is a free book online (it's quite old now) that goes through the steps of making a game. It's in C if I recall but the ideas are still valid now. There is even a chapter on side scrolling. Book

There are a few ways to go about it, here's an easy way of thinking of it:

-Store object positions as x, y, and z coordinates. (the z is for depth or layer which can be important if things overlap)
-Store the current top left of the camera's display space as x, y.
-When rendering things, first you should do some figuring to make sure you don't attempt to draw everything in the world, basic culling/clipping procedures etc. And then you can add x and y to each object you draw.
-When your character moves (being an object with its own x, y, and z position), move your camera with them by adding to the camera's x and y coordinates.

Yes, chop up big pictures into smaller ones and tile them... I'm not sure what you're using to display this stuff, but usually huge textures are not a great idea.

If you are using opengl or directx you can do matrix translations based on x and y instead of manually adding things...

This is about as simple as I can describe it...