40 replies to this topic

[mrbastard]

Your original suggestions of flash or svg would work fine for a mock-up. If you want advice on how to practice coding, you're going to have to say what sort of level you want to code at and what languages you're familiar with.

[Shinkage]

(responding for people who genuinely are interested in OS development)

I think x86 is an absolutely horrible platform to start learning this kind of low level stuff on. Honestly, f you want to learn how to make an OS, for the love of god don't do it on an x86 machine. If you're really serious about it, start with an embedded SoC (system on a chip) evaluation board (like the Beagleboard). There are a few things that make these kind of systems a whole lot easier to do low-level work on, especially if you're just learning:

• They're designed with that kind of work in mind. In the embedded world it's pretty much assumed that you're going to want to talk directly to peripherals (MMU, GPIO, etc...) and so all that stuff is pretty thoroughly documented, right down to the register bit layout and usage procedures.
• Everything in one place. With an SoC, basically the entire computer system is all stuffed into a single chip, with a single reference manual to figure it all out.
• JTAG. There's really no substitute for even a cheap USB JTAG debugger when it comes to doing the kind of low level stuff where standard debugging isn't an option. With a JTAG debugger you can literally step through every instruction the processor executes, one by one, anywhere, any time. Boot loader, kernel code, you can debug it all with JTAG. No 3 beeps and wondering what went wrong like Ryan_001 had to deal with.
• GPIO+logic analyzer. Toggle a GPIO pin (basically just a metal pin on the board that you can very quickly switch high/low voltage) and watch exactly when it happens on a logic analyzer. With enough pins hooked up to an analyzer, you can get a really good idea of what's going on. It's like printf's, but way way better because the timing is precise down to the nanosecond range. You can even encode more sophisticated messages using I2C or SPI or such and have the logic analyzer decode them (again, like printf's but even better). Also, unlike printf's, it'll work in any execution context like, for example, a harware interrupt handler (don't know how I'd debug those without my trusty logic analyzer).
• Simple instruction set. You're going to have to do some work in assembly. I, personally, find ARM an absolute joy to used compared to x86 when it comes to working in assembly.

For anybody who actually is interested in learning this kind of stuff, it's (my humble opinion) by far the most enjoyable kind of software engineering out there. That said, it's not cheap. The above equipment, for example:

SoC Board: http://beagleboard.org/hardware-xm
OMAP processor, and TI documentation tends to be some of the better.
JTAG Debugger: http://www.tincantools.com/product.php?productid=16153&cat=251&page=1
Every JTAG debugger on earth is a pain in the ass to get working. This is no different.
Logic Analyzer: http://www.pctestinstruments.com/
Fantastic piece of equipment. Really just works perfectly.

It'll set you back close to a grand and it's about the cheapest setup you could put together, but there's really no better setup for learning how the real nitty-gritty of low level software engineering works. Plus you get to feel like a mad scientist with wires and pins going everywhere.

[ATC]

@ Shrinkage ::

Very good and interesting post... I +1'ed it!

However, I'd like to point out a few things... For starters, while your idea is great and makes me want to try it myself, which of the two is cheaper: 1) free download of Sun's Virtualbox and/or free download of Microsoft's Virtual PC 2) going out and buying physical hardware like you describe? Obviously, #1 is a lot cheaper and more accessible for "the crowd". Secondly, most of us programmers are writing software for x86/64 systems. And when we want to learn about OS development it's usually not to try to become a professional OS developer (very tough market to get into, dominated by Microsoft) but to go on an "academic" venture; to become more knowledgeable and skillful programmers on our primary development platform. So for those two reasons alone I would not go as far as saying that the x86 platform is a "horrible" choice... I think that if you're in OS development for academic reasons you should work with the platform you do most of your userland/everyday programming on... and if you're in it to make a profession out of OS development you should work with the platform you intend to support.

However, I must agree that learning the x86/64 architecture is quite difficult; the learning curve is steep, the manuals are thick and Intel is not exactly what I'd call a "n00b-friendly" company lol. There are lots of little tricky caveats, traps and pitfalls... lots of weird things you must learn, like setting up a GDT (global descriptor table) as an off-the-wall example... But if you learn this stuff you walk away knowing what makes your PC and your primary OS (be it Windows, Linux, etc) tick. It's hard to even describe in words how incredibly valuable that can be to a PC programmer... say you're writing an performance-critical function that needs to move and/or compare large blocks of memory as quickly as possible. Without experience in low-level/systems programming, you're like to just to take the most direct, straight-forward (and naive) approach to your algorithm. With such experience, however, you know how the CPU, data bus and RAM work... you know how memory is moved in/out of registers, to and from RAM and understand virtual memory and paging/caching... you would also know that on 32-bit architecture the CPU registers are 32-bits in size and natively move d-words (32-bits of data) faster than it can move 64-bits (and vice-versa on x64 platforms). Armed with that sort of knowledge you will almost always know how to implement the best possible routines. However, if you're a PC programmer by trade and all your OS development experience is on some completely different platform there can be quite a few gaps in your knowledge...

So I encourage people to work with the platform that's right for them... and if you're a PC developer then you may as well bite the bullet and learn the x86/64 platform. If, however, you are wanting to make a career or serious hobby out of OS dev then choose the platform you need to work with and heed Shrinkage's advice... and learn more than one platform!

Regards,

--ATC--

[Ohforf sake]

I wrote my first OS (multithreading, virtual memory, rudimentary syscalls) on an ARM SoC at university, so there are cheaper ways to get access to this kind of rig. Although we didn't have JTAG or a logic analyzer. IMSW we had a serial cable and a LED, which is not as comfy but still enough.

I never tried writing an OS for x86 so my comparisons might be wrong here, but the ARM ISA (I used ARMv4) is a lot nicer then x86 ISA and you can not avoid writing some parts in assembly. Also in our chip, we could disable certain functionalities (cache, MMU, ...) in order to get everything right one step at a time. I don't know, if intel lets you globally disable the caches "just like that".

So for the very first steps in OS programing I'm with Shinkage. Get yourself an ARM SoC or find someone who lets you work with his. A university course is even better because you get help when you are stuck. If afterwards, you still want to write OSs and maybe do something bigger, then you can still go x86 (or whatever platform is your favorite).

[Olof Hedman]

Very good and interesting post... I +1'ed it!

[Lots of text]

The point with the simple system is that everything you learn is actually directly applicable on the more complex x86 system.
Sure, ports memory areas and bitpatterns isn't the same, but thats not really the interesting part of it...
Thats just whatever the hardware is right now, and should (mostly) be handled by drivers, working as plugins to abstract hardware so the OS doesn't have to care about those details.

With simpler hw you can focus better on the actual OS programming problems.
Then when you've got those down, you can move onto more complex hardware.
Doing it on an actual physical device is a bit more rewarding imo.

My first programming steps was taken on a hacked Texas Instruments calculator, running z80 (Ti83) and 68000 (TI89) assembler.
There wasn't even a c-compiler available, very good experience for learning how a computer system works

Old iPods are also fun hardware, its basically a SoC with a nice screen and some buttons, and they are pretty easy to upload your own bootloader and code to, I've played a bit with my old (2006) iPod NANO and installed linux on it. (and run some OS-less code too)

So theres lots of options if you dont want to spend money on a beagle board, though with those you get the advantage of good documentation and a big community.

OP: Sorry for all this off topicness

Edited by Olof Hedman, 28 October 2012 - 03:53 AM.

[Ohforf sake]

Come to think of it, the most important and valuable piece of hardware in my case was a very easy to press and physically robust (as in frustration proof) reset button :-)

[Shinkage]

I'll try to address your points individually:

For starters, while your idea is great and makes me want to try it myself, which of the two is cheaper: 1) free download of Sun's Virtualbox and/or free download of Microsoft's Virtual PC 2) going out and buying physical hardware like you describe?

ARM processors can not be virtualized on a PC, but they can be emulated with QEMU, which is actually a good way to start now that you mention it. It'll let you get off the ground with stuff like figuring out how to get a bootloader working without the trouble of having to constantly flash new code onto an embedded board. A good way to get your feet wet before diving in and buying a $150 board and then figuring out a good process for flashing new builds to it, etc... and then buying the even more expensive debug and analysis equipment.

In fact, I'd say QEMU is probably a better way to start then the setup I suggested, I just hadn't thought of it!

Secondly, most of us programmers are writing software for x86/64 systems. And when we want to learn about OS development it's usually not to try to become a professional OS developer (very tough market to get into, dominated by Microsoft) but to go on an "academic" venture; to become more knowledgeable and skillful programmers on our primary development platform. So for those two reasons alone I would not go as far as saying that the x86 platform is a "horrible" choice... I think that if you're in OS development for academic reasons you should work with the platform you do most of your userland/everyday programming on... and if you're in it to make a profession out of OS development you should work with the platform you intend to support.

This is actually precisely why I'd say x86 is a bad choice. If you're in it for academic reasons, what you should really be interested in is the general theory behind everything you're doing, rather than the minute implementation details. Believe it or not, the vast majority of what you learn working on any even vaguely comparable platform (i.e. 32-bit von Neumann architecture) will be equally applicable wherever you go when it comes to userland. Things like the gritty details of how to manage your system's MMU are largely irrelevant outside of OS design, but the general principles behind how a paged MMU works are very useful and pretty consistent among most modern platforms.

Assuming you buy the story that what you learn will be equally applicable, then I can almost guarantee you'll have an easier time actually getting off the ground on a simple SoC. Can't really underestimate how having the entire system on one chip with one detailed reference manual can simplify figuring everything out.

Also in our chip, we could disable certain functionalities (cache, MMU, ...) in order to get everything right one step at a time. I don't know, if intel lets you globally disable the caches "just like that".

This is actually a very important point that I thought I'd quote for emphasis here. I also don't know if that's the case on Intel, but it can make quite the difference.

[6677]

Instead of a beagleboard you could use a raspberry pi. There is a tutorial on developing an OS in ARM assembly for it too.



The extend of my OS development ends with using cosmos: http://cosmos.codeplex.com
Fun stuff. I also had a semi working command shell written in DCPU16 once aswell but it was very buggy and I lacked motivation to complete it.

[Bentm's Games]

can u people quit discussing os development. All i want is something to create an inmteractive program. If coding is needed, I would prefer python or java and btw, whats svg

[6677]

If you want an interactive program. Any language will do.

Pygame will do it for python. Slick will do it for java. XNA with C#. Anything. Even a simple windows form application might work to some extent.

[Cromulent]

Instead of a beagleboard you could use a raspberry pi. There is a tutorial on developing an OS in ARM assembly for it too.

I was going to suggest the Raspberry Pi as well. It is a very capable board for $35. You can't really go wrong with it. 700Mhz ARM chip, 10/100 Ethernet, HDMI out, 2x USB 2 ports, audio out, an SD card slot and 512Mb of RAM. What more do you need?

[nife87]

...Raspberry PI...

AFAIK they only make model A and B with 128/256 MB RAM, respectively, though with a model C in mind. But it is a great tool/toy with credit card dimensions, requiring a mere 5 W from an ordinary AC/DC charger and even contains a GPU capable of decoding 1080p (x264) without hassle. Unfortunately, it is only produced in small production batches (still waiting for my own copy since mid-July), since they initially expected a lot less demand from private buyers.

[Cromulent]

AFAIK they only make model A and B with 128/256 MB RAM, respectively, though with a model C in mind.

Incorrect.

What’s the difference between Model A and Model B?
Model A has 256Mb RAM, one USB port and no Ethernet (network connection). Model B has 512Mb RAM, 2 USB port and an Ethernet port.

http://www.raspberrypi.org/faqs

[ATC]

can u people quit discussing os development. All i want is something to create an inmteractive program. If coding is needed, I would prefer python or java and btw, whats svg

Well, your thread suggests you were interested in OS development... if you're not, then I have no idea what you're trying to do (or better yet, why or what the point is)... You're initial question has already been answered though, just use whatever graphics API you're best with... Anything will work...

[akesterson]

TL;DR - you don't need to program if you want to "mock up" an operating system. A design idea can be done in flash, a workable demo can be done in Java or any other programming language on top of an existing OS (this is what Nintendo did for their Gecko emulation, and AmiOS - the AmigaOS open source project - has been doing for a while).

For those who were led here by the (slightly misleading) "Create an operating system" title, here are some links to how others are using qemu and gdb to develop low-level system code for ARM processors on their cheap, readily-available PC hardware:

http://stackoverflow.com/questions/2651075/what-do-i-need-for-development-for-an-arm-processor
http://stackoverflow.com/questions/2611624/low-level-qemu-based-debugging

Have fun.

[Heath]

can u people quit discussing os development. All i want is something to create an inmteractive program. If coding is needed, I would prefer python or java and btw, whats svg

You could write a python program with pyglet or whatever that runs other python chunks, with a homescreen and everything that looks like what you might want. It would teach you more about the language itself, even if you never want to use that program for anything practical.

[recompile]

Y'know, I'd be interested in buying a good book on writing an OS, if anyone wants to write one and put it up on Amazon. I'd probably learn a lot from it, if nothing else.

It's an antique by today's standards, but I remember it being handy: Developing Your Own 32-Bit Operating System

It worked surprisingly well with this much older, but extremely handy book: Programmer's Problem Solver for the IBM PC, XT, & AT

Hmmm... Maybe I need to find new books...

[6677]

...Raspberry PI...

AFAIK they only make model A and B with 128/256 MB RAM, respectively, though with a model C in mind. But it is a great tool/toy with credit card dimensions, requiring a mere 5 W from an ordinary AC/DC charger and even contains a GPU capable of decoding 1080p (x264) without hassle. Unfortunately, it is only produced in small production batches (still waiting for my own copy since mid-July), since they initially expected a lot less demand from private buyers.

I'd chase your purchase up. They have more manufacturing plants running in larger batches now and no longer have any order delays, havent done since early august. Infact a friend ordered a model B when they announced the 512mb upgrade, on his doorstep in 2 days.

Model A still isn't shipping yet (no idea why) but is now using the 256mb chipset from the original model B.
Model B is now shipping on a revision 2 PCB with 512mb RAM instead of the original 256 of first revision boards. Price is the same.



The SoC used only has a single USB port supporting acting as a slave aswell as host. On the model A they break this port straight out into a single USB connector. On the B they run this through an onboard hub first which splits it off into 2 ports and an ethernet port (through yet another chip, so yes, technically ethernet is a USB device although I think the firmware in the GPU does some trickery to expose the hub as a network driver and 2 dedicated ports) which does limit the bandwidth of USB devices considerably, also impacts ethernet performance.

Some of the GPU firmware is now open source.





Back on topic, yes OP, your virtual OS is just a graphical app which just happens to have the functions of a media player and a text editor and god knows what else. There are plenty of them out there. Any graphics library will do the job fine with any language capable of playing audio. Easiest way to do it would probably be to use .net win forms application, create one main form which is your desktop and set it to be an MDI parent, other forms can then be opened inside it.

[AMenard]

The Tannenbaum "Modern Operating System" book includes the listing of Minix in C. I never had the guts to type it in

[demonkoryu]

I found An Operating Systems Vade Mecum (Second Edition) extremely valuable. And yes, it is a free download; although the original download page has vanished.