Sign in to follow this  
game of thought

Basic operating system development

Recommended Posts

Hello, before anybody states it i realise how massive this is. I have an idea of an incredibly basic os containing maybe six commands, a text editor and an interpreted language interpreter. I read on osdev about various string and interger parsing and have a plan for it.

I have questions to those few who have done this before:

1: how long did it take you to create what i have stated above?

2:how long would it take to implement a basic api(i read about one on the gdn logie maybe a week ago)?

3: can you share a link about using cds as bootable media(looked around, i could only find floppy disk an usb sadly)?

4: should i write the kernel in x86 asm or c - libraries( i know both)?

As i said before i realise the enormity of the above task. If you wish to know how this connects to game development, i like the idea of working on something for something i have done, rather than a pre existing os because i would rather be satisfied by my achievement rather than anybody playing it.

Thank you for your time.

P.S. again i know this is probably crazy

Share this post


Link to post
Share on other sites
I really don't think you realize how difficult this is.

I mean, to be honest, I never made an OS and I most likely never will make one(at least not one that does anything useful).

To make an OS from scratch you'll need to know quite a bit about hardware for interrupts, how kernels work, how they manage memory, linkers, parsing, be very good at a programming language, and knowing Assembly is almost always a requirement to get some things done.

You'd need to know Assembly to write a bootloader, you'll need good coverage of hardware for interrupts(which are signals to the processing device on what it needs to do), how the kernel should work with driver software, if any, and the I/O stream will not be easy either. The booting process can vary, and a lot of the times it is off putting to people on the whole process because a lot of the work is constituted in hex, and Assembly is really complicated to even many good programmers.

I'm sure many people could fill the blanks even better, but the point is that if you plan on writing an operating system you hopefully are a really good programmer to [b]say the least[/b].

It's not impossible, and I've even been reading up/learning the process behind it, and it's extremely far fetched to the majority of basic computer programmers.

PS: You say you "like the idea of working on something for something i have done."

Unfortunately, using Notepad saves the enormous time and pain than making an operating system host a text editing program in memory to write code to run on the same platform you could have written the code in in the first place.

But, by all means, don't give up the idea of building an operating system. You never know. It's just such a big time killer to bother with the development of one to just use a text editor to write code to use on the same OS you neglected to make it on initially. Edited by Pointer2APointer

Share this post


Link to post
Share on other sites
I agree with pointer2apointer, what is the point of this? If it is a learning excercise it's completely useless. You will probably never need to do anything that this involves in game programming, or programming in general. And it isn't a matter of time. It is excruciatingly complicated (From what I've read) And involves learning many different and extremely low level programming languages. Unless this is a requirement I don't recommend it, however you could still try and experiment with it.

Share this post


Link to post
Share on other sites
I took an OS course in my degree based off the [url="http://www.stanford.edu/class/cs140/projects/pintos/pintos.html"]pintos[/url] academic kernel, and it was a beast. The task was broken into four main sections that spanned the entire semester (with teams of 5), was boatloads of work, and that was [i]after[/i] the basic kernel had already been built: we added on to the existing infrastructure (well, virtual memory was empty before we touched it, but, yea). OS programming is a daunting task. If you have the right resources and a LOT of time, coupled with a very solid understanding of C and assembly, there's nothing saying it's impossible. It's some incredibly low-level debugging in many cases, and ultimately served to impress upon me how pleasant programming can be a few levels up from the metal.

Share this post


Link to post
Share on other sites
1. Depends on your experience.
2. I don't really understand the question ("gdn logie"?) but I guess the answer will be the same as #1.
3. Sadly I have no information here. Perhaps [url="http://www.google.com/search?q=osdev+bootable+cd"]Google knows[/url]?
4. I would choose C, for as much as possible.

Share this post


Link to post
Share on other sites
[quote name='game of thought' timestamp='1347817636' post='4980666']
1: how long did it take you to create what i have stated above?

2:how long would it take to implement a basic api(i read about one on the gdn logie maybe a week ago)?

3: can you share a link about using cds as bootable media(looked around, i could only find floppy disk an usb sadly)?

4: should i write the kernel in x86 asm or c - libraries( i know both)?
[/quote]

1. I've written some very small very basic kernels in the past which didn't even come close to what you're describing. You have to realize that OS development isn't like regular application development as you have to write a huge amount of boilerplate code before you can even start working on something remotely interesting. There will be no libraries available to you to develop your system with, if you want a standard library for developing applications on your custom system you will have to take the time to port an existing library to your platform, which can be a really serious amount of work in itself. The time it takes to develop your system will vastly depend on your programming expertise, your knowledge of the hardware platform you're developing for and the design of your kernel. You should also prepare for days or even weeks of digging through various obscure and sometimes horribly written hardware reference manuals.

2. This is pretty much the same as question 1?

3. Trust me, stick with floppies in the beginning, you don't want to prepare an entire CD image each time you do a kernel build. It will be a while before your system exceeds the 1.44MB you get in a standard 3,5 inch floppy, and you'll mainly be working in a virtual environment for easier debugging and for faster reboots, so you can just mount your floppy images in your virtual machine. If you want to move on to a physical machine you'll still want to stick with floppies as long as possible as they are perfectly rewritable and very easy to switch between machines (ie. your development machine and your target machine).

4. If you're going for x86 development you'll probably be stuck in real mode when your boot loader passes control over to your kernel, so you'll want to use ASM in this stage, unless you fancy using archaic C compilers which can compile 16-bit real-mode code. Once you're in protected mode or higher (ie. long mode) you can use C with a compiler of choice.
ASM and C are not the only options here though, any language which can compile down to native code could be used to develop an OS, although some additional setup could be required for some languages. I've personally used both C and C++ for OS development, but I've seen it done in other languages as well. Some even go as far as to only use a native language to get a lightweight VM set up so they can use managed code to develop the rest of their kernel.


Do not expect any actual results in your OS development endeavours in the first couple of months as it is definitely not an instant gratification discipline. The environment you will be working in will be extremely barebones, the options for debugging are extremely limited, there will be no system in place which protects you and your machine from yourself (I've heard stories of people destroying hardware, mostly monitors, by messing with hardware programming in a custom OS), there will be no descriptive error messages when something goes wrong (you should actually expect no error messages at all, or just random garbage dumps) and you just need a huge amount of dedication and patience to get something remotely useful up and running.

Also, don't expect there to be a lot of tutorials or well-explained reference materials to be lying around, communities like osdev.org can provide some basics, although the community itself can be quite harsh for newcomers. Also be aware of the fact that for some pieces of hardware there will be absolutely no reference material available at all, as a lot of hardware developers aren't keen on giving away documentation of their products to just anyone. Your safest bet is to develop for rather old hardware for which documentation has been published freely, or which have been properly reversed engineered by hardware enthusiasts.


It's definitely not impossible to write your own OS, but I would recommend you only do this if it's just for your personal interest and if you are really really really patient and really really really dedicated to get this system developed. There will be no commercial applications for your OS, nor should you expect anyone to show any real interest in it beside maybe some individuals in the OSDev hobbyist communities. Edited by Radikalizm

Share this post


Link to post
Share on other sites
Even with plenty of time on your hands it's going to be very harsh and tough.

Try and answer some of these questions, and they will probably determine if you're able to complete this feat anytime soon:

1.Do you know in any good sense how GUIs work with OS kernels, how hardware interrupts work, how memory in kernels works, how kernels allocate user space for the user portion of programs to execute under a GUI or program/window structure for this process or purpose of using a text-editor?

If not, you'll need to:

1.Read up more on how operating systems work.

2.Read a LOT more on how kernels work.

3.Read how GUIs work.

4.Read more terminology involved in user space and kernel space, kernel architecture, and how kernels manage memory.

2.Do you know how the kernel communicates with the hardware?

If not, you'll need to:

1.Learn low-level programming, possibly Assembly, will help. Not to mention it can have other benefits(speed for one).

2.Learn about CPU hardware. A processor has a bunch of regions called [i]registers. [/i]

You should read this if you're not familiar with some basic processor architecture: [url="http://en.wikipedia.org/wiki/Processor_register"]http://en.wikipedia.org/wiki/Processor_register[/url]

Multi-core processor designs through Assembly and other higher-level programming environments are going to be much more tedious than single-core ones for optimum development.

Point is, you need to know your computer anatomy before you can give useful life to an effective operating system, and learning computer anatomy is not so easy because each computer's anatomy can vary.

A first step: research your current computer's hardware, and then learn more specifically from there. Most importantly, though, would be the processor, input and output, and memory to start with in the learning process.

Also, I'd try and learn about your BIOS, booting options, how exactly you plan on booting the kernel, etc.

It's not a type-and-go process like writing code on an already built OS is with everything you need taken care of for you(especially the ugly stuff that would make many game programmers facepalm).

Hope I helped somewhat!

Share this post


Link to post
Share on other sites
[quote name='Pointer2APointer' timestamp='1347917230' post='4981030']
1.Do you know in any good sense how GUIs work with OS kernels, how hardware interrupts work, how memory in kernels works, how kernels allocate user space for the user portion of programs to execute under a GUI or program/window structure for this process or purpose of using a text-editor?
[/quote]

GUIs? There's an enormously long road to follow before one can even start to think about writing a graphical system for an OS. It's probably a very good idea to not incorporate a graphical system at all for a first OS. Same goes for multi-core kernel design as it's hard enough to get a kernel to play nicely with a single CPU.

Knowledge of an assembly language for your hardware platform is a must, I'll even go as far as to say that you need quite some expertise in it as well, even if you're writing the majority of your system in a language like C. A solid understanding of an assembly language implies a good knowledge of what a CPU actually does under the hood, and that understanding is absolutely crucial if you're going to write a system which runs entirely on bare metal.

OS development is one of the most advanced topics you could tackle in computer science with almost no rewarding results, and it'd be wise to keep this in mind at all times.

Share this post


Link to post
Share on other sites
It is worth considering a 'dumbed-down' VM, designed for learning OS development.

One of my professors wrote [url="https://code.google.com/p/clown/"]CLOWN[/url], which was used in our Operating Systems design course. It's basically a much-simplified x86-style processor with minimal added complexity (just RAM, console I/O, hard drive, and interrupts). Takes an awful lot of the pain away, compared to learning on real hardware.

Share this post


Link to post
Share on other sites
OS development is very exciting and rewarding, often filled with Fun. YOLO.

Here's some tips:

(1) stick to C as much as possible, and limit your assembly to inline assembly where you can (and that's almost everywhere)
(2) start with an existing bootloader to get up and running (grub, redboot, das u-boot with the latter two particularly suited to ARM)
(3) use an emulator like qemu or bochs rather than real hardware until you've at least booted to a userland command interpreter because they tend to follow published specs better than real hardware and you can use gdb instead of jtag for debugging

Good luck.

Share this post


Link to post
Share on other sites
I cant give any advice because I dont even know the slightest thing involved in making an OS, however I gotta say it sounds like a fun thing to do [img]http://public.gamedev.net//public/style_emoticons/default/biggrin.png[/img] I am sure you will learn a lot from it

[quote name='game of thought' timestamp='1347817636' post='4980666']
As i said before i realise the enormity of the above task. If you wish to know how this connects to game development, i like the idea of working on something for something i have done, rather than a pre existing os because i would rather be satisfied by my achievement rather than anybody playing it.
[/quote]

They say making an engine is like reinventing the wheel, therefore making an OS would be like creating the universe.

Have fun with your project and good luck [img]http://public.gamedev.net//public/style_emoticons/default/smile.png[/img]

Edit: Doing a quick search I came across this, not sure if its useful at all http://www.academicearth.org/courses/operating-systems-and-system-programming Edited by Dynamo_Maestro

Share this post


Link to post
Share on other sites
[quote name='xenobrain' timestamp='1347932254' post='4981098']
"Operating Systems: Design and Implementation" is the definitive text on the subject. Grab a copy of the book and the latest source for Minix 3 and you'll be off to a good start.
[/quote]
Thanks, like linus, didnt he based the original linix kernel off minix or something

Share this post


Link to post
Share on other sites
[quote name='game of thought' timestamp='1347949400' post='4981158']
[quote name='xenobrain' timestamp='1347932254' post='4981098']
"Operating Systems: Design and Implementation" is the definitive text on the subject. Grab a copy of the book and the latest source for Minix 3 and you'll be off to a good start.
[/quote]
Thanks, like linus, didnt he based the original linix kernel off minix or something
[/quote]

+1 on that one, it's great. Microkernels bring about some really interesting issues.
On the rest: I think most people replying are overly negative, it's not that hard. Especially if you make your bootloader take you to pmode directly, plus, if you want fancy resolutions GRUB (which is the loader I've used the most) can hand you off in some VESA mode.
Booting from a floppy is easy-peasy, I think USB can even emulate a floppy drive...? Anyway, if you want to split your system into many files without implementing a floppy driver and filesystem, you can - wait for it - let GRUB load it as a module. So easy!
What do you mean by a "basic API"? Syscalls to support libc, for example? If you strip down your memory protection, you don't need any API.
To write an editor (assuming console-based), you need to implement screen scrolling (maybe you can get this in hardware though), moving the carret, setting letters on-screen, etc. If you want a proper GUI, you have to draw the characters yourself, pixel by pixel.
For the languages, some people like to make their whole system in asm but I prefer to stick with C as long as I can. Some things need to be in asm (multitasking and configuration thingies, for example).
It sounds like a fun project, and not crazy at all!

Share this post


Link to post
Share on other sites
[quote name='patrrr' timestamp='1347957052' post='4981178']
What do you mean by a "basic API"? Syscalls to support libc, for example?
[/quote]
i mean like a basic graphical library, like curses or allegro. Should of clarified that.

And i am not planning a "GUI" as such, more of a command line interface, a la DOS or the (as far as i know) the original Unix.

Share this post


Link to post
Share on other sites
For a more complete experience, it might be useful to learn via [url="http://en.wikipedia.org/wiki/Xv6"]xv6[/url], a simple Unix-like teaching operating system.

Here are some relevant resources:
http://pdos.csail.mit.edu/6.828/2011/xv6.html
http://pdos.csail.mit.edu/6.828/2011/xv6/book-rev6.pdf // even if you don't decide to use xv6 for learning, I'd still recommend to read Chapter 0
https://github.com/duckinator/xv6#readme

6.828 @ MIT:
http://pdos.csail.mit.edu/6.828/
// note: this links to the current version; the 2011 version above might have more lectures available

CS422/522 @ Yale:
http://zoo.cs.yale.edu/classes/cs422/2011/info#texts
http://zoo.cs.yale.edu/classes/cs422/2011/sched
http://zoo.cs.yale.edu/classes/cs422/2011/reference

W4118 @ Columbia:
http://www.cs.columbia.edu/~junfeng/12sp-w4118/syllabus.html

See also V6 (more of historical interest, since xv6 has replaced it):
http://www.lemis.com/grog/Documentation/Lions/

See also:
Operating Systems: Three Easy Pieces: http://pages.cs.wisc.edu/~remzi/OSTEP/

Share this post


Link to post
Share on other sites
[quote name='game of thought' timestamp='1347985519' post='4981291']
[quote name='patrrr' timestamp='1347957052' post='4981178']
What do you mean by a "basic API"? Syscalls to support libc, for example?
[/quote]
i mean like a basic graphical library, like curses or allegro. Should of clarified that.

And i am not planning a "GUI" as such, more of a command line interface, a la DOS or the (as far as i know) the original Unix.
[/quote]
Well, Unix preceded DOS by a couple of decades, and yes DOS did kind of copy the idea of a simple kernel API and a command-line shell from it, because it was a good design.

The way an OS like Unix or DOS works is that they have a simple basic API that userspace makes calls into the kernel to perform I/O and process control. Sitting on top of that is generally a set of more portable language bindings; on Unix that's a C library (often called "libc") but there was no such thing on the original DOS and you had to make kernel calls using interrupts from your higher-level language.

I would recommend you go with the Unix way: there are a number of small libc implementations (eg. newlib, ulibc) in which you just have to provide some 5 or 6 basic kernel API calls and the rest just appears. You can then build a cross-development GCC toolchain using that runtime and start building an entire POSIX system, including the classic Unix shell. All the heavy lifting is already done, all you have to do is write the kernel.

It's just that easy. Edited by Bregma

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this