It sounds like you could benefit from 'spatial partitioning'. There are different types of spatital partitioning algorithms and they are often best suited to slightly different tasks so pick one that suits best.

Some common methods are:
Quad-trees
Oc-trees
BSP
KDtrees
ABTs

I suggest you research some/all of these using [google] and pick the one that best suits your application.
BSPs are getting a bit old but well documented,whilst ABTs are (in comparison) brand new but much less help is availble (in fact 90% of all infos on ABTs are here at GDnet!)
Good general purpose methods are Quad-trees(good for terrain) and Oc-trees(Oc-trees are an extension to Quad-trees so it might be easier to learn Quad_trees first) and they are fairly simple to implement.

Now being more specific to you question: You could use for example a quad-tree to partition your (3D) world and then only those portions of the world in direct sight need to be rendered (boost performance lots!) and those sections of the world close-by, whether in-view or out-view, can be updated for physics & collision detection but those sections far away (or far away AND out of view maybe) may not require any/as much process because you can assume that any affect of physics will not be noticed from the players current position (again, boosts processing).

Hope that was some help [smile]