First good bit of cheapness is fifty bucks free from Chase.
1. Go here.
2. When you get your coupon in email, print it and take it to your local Chase branch and open a free account.
It did take a little finagle-ing, as I had to explain to 'em that being self-employed and getting direct deposit were incompatible, but they worked around it. According to some of the fatwallet folks, a few paypal deposits will qualify you.
Anyway, I opened my account last week and my free $50 has already showed up.
Next is a backup solution. My current solution consists of an external USB hard drive and EZ Back-It Up. It's a simple backup program that'll mirror some or all of a disk's folder-structure on another drive. I have it doing a daily mirror of the "my documents" directory into seven different folders on the external drive, labeled "monday", "tuesday", etc. That way if I do something dumb I can roll back files for up to seven days.
That works fine, but it still leaves a problem if there's something like a fire, big power-surge, etc. that'll wipe out the machine and the drive. For that, I'm using Carbonite Backup. It's an extension to Windows Explorer that backs up stuff over the internet to Carbonite's servers. To back up a folder, you just right-click the folder and say "I wanna back this up", and it'll queue it up for backup. It'll seriously bog your machine down for the first big backup, but after that it just quietly monitors changed files in your marked folders and uploads 'em to Carbonite's servers. It's $3.75 a month if you prepay for 24 months. Highly recommended.