No knowledge of any programming language
C++, C#, and VB are all pretty good and included in Visual Studio 2012 express. Personally I like Java, which has a very similar feel to C# without the presence of Microsoft. You are going to need to download the Eclipse IDE, or the Netbeans IDE. These work on all major operating systems and a few you have never heard of, so your probably ok.
After making your first "hello world" program, if you like the language you should buy a book and dive in further.
Java, lacking lambda expressions, feels nothing like C# to me. But then, I use a LOT of lambdas.
There really are a bunch of different ways you can go. Probably one of the easier ones that gets you started, doesn't take a lot of tools, and is easy to show off your results, is HTML 5 and JavaScript. You technically have everything you need on your computer already: a text editor and a browser. Some tools like a syntax highlighting editor will help make the job easier, but you never really have to worry about keeping up to date on the latest versions of compilers and what not.
Java, lacking lambda expressions, feels nothing like C# to me. But then, I use a LOT of lambdas.
Well lambdas are coming in the next Java version, I've heard.
@FezileNkuna: What kind of programs or software do you want to develop? Or put another way, what kind of programs or software having been thinking of making?
Playground: http://play.golang.org/
Documentation: http://golang.org/doc/
You can learn in one tab, and experiment in another!
I second the recommendation for starting with Python.
Oh wait, five posts and nobody has recommended it yet? Then allow me to be the first. 
Java for me personally was kind of a pain to work with. Only started learning it to develop android apps. Created a game using that + Andengine (btw Andengine is pretty awesome even though it uses Java) and then found Unity. Picked up C# fairly quickly, and I'm enjoying programming a lot more.
Python, is pretty good from what I've been hearing. I took a look at some tutorials, and am not a big fan of the syntax though. Anyway, you can use it with Pygame to create games.
Personally, I recommend C#. I think this language is great for beginners. Plus, you get to use Unity once you get better at it, and want to start creating games. Unity builds to Windows, Mac, Linux, Unity Web Player, Flash, Android and iOS. Just 1 code, for all these platforms.
Java, lacking lambda expressions, feels nothing like C# to me. But then, I use a LOT of lambdas.
Well lambdas are coming in the next Java version, I've heard.
Like they were supposed to be in 6 and 7? Seeing is believing.
Google Go has an in-browser Go compiler you can play with. Nothing to install or break.
Playground: http://play.golang.org/
Documentation: http://golang.org/doc/
You can learn in one tab, and experiment in another!
I second the idea to start with Go. You don't need to download anything. Have a look at the step-by-step tutorial at http://tour.golang.org/#1.
Java, lacking lambda expressions, feels nothing like C# to me. But then, I use a LOT of lambdas.
I don't think the availability of lambda expressions is a criteria for what first language to learn.
