Using OpenGL might be a good option if you want to target hardware cross-platform.
Instead of learning everything about 3D programming, I recommend focusing on 2D and 3D file format implementations (rasterize them to screen), shaders, animation support from 3D programs such as Collada as one example, voxels, input/controller support such as mouse and keyboard or others, camera views, and procedural generated surfaces such as terrain. You might want to get started early with Oculus Rift, which has a reasonable cost developers version. Bullet Physics is a good thing to plan to implement down the road.
Don't spend too much time on one area but be balanced long term. Better to create a sphere on a simple terrain plain that bounces off another sphere, or something like that, instead of exhausting yourself on trying to get one thing great looking before moving to another aspect of 3D.
Remember to really focus on developing a creative workflow pipeline in the long term. Anything else is trivial pursuit for a beginner or intermediate developer. Workflow pipeline for your efforts is very important if you aspire to go pro in 3D.