• entries
72
51
• views
60135

OS and Game Development

## OSDev Series: Chapter 20

Thank you!

20 chapters has gone by in the OS development series. A little over a year sense it has started on this very site. I want to give a public thank you to everyone - it is the support that I get from readers and members of communities that helps keep the series going.

Thank you everyone!!

OSDev Series Chapter 20

Chapter 20 has been officially released. It covers:

• FDC and FDD History
• Disk Layout
• CHS and LBA
• FDD Structure
• FDC and FDD Hardware
• Interfacing with the FDC
• FDC Registers and Commands
• Programming the FDC

It also now includes a demo that is built from the Textual User Interface (TUI) that we have built from the previous chapter. We build a floppy driver in this demo, and impliment a read command to allow us to read from any sector on disk.

Check it out here.

Please feel free to let me know if there is any comments or suggestions!

OSDev Series Updates - Revision 3

I have also added navigational icons to the chapters of the series to help making navigating the series easier for the readers. More updates for the series are also planned including bug fixes, additional content, and more as I impliment Revision 3 of the OS development series.
• ## OSDev Series: Chapter 20

OSDev Series Chaoter 20

I cannot believe it... I am at chapter 20 of the OS development series going on to chapter 21. It all started with a forum post here on GDNet and is continuing on thanks to the great support that it gets. I thank all of the supporters of the series [grin]

Chapter 20 covers almost everything about floppy drive programming including:

• FDD history
• Disk layout
• CHS, LBA
• FDD Structure
• FDC Hardware
• FDC Registers and Commands
• FDC Interfacing
• FDC Programming

The demo adds a read command to the TUI demo developed in the last chapter that allows you to read any sector on disk using the floppy driver developed in the chapter.

OSDev Series: Chapter 20
OS Dev Series Base Site

If anyone has any questions, comments, suggestions, or ideas, please feel free to let me know [grin]
• ## Small update on Neptune

Neptune Kernels new Debug Screen - Running in VPC

Hello everyone,

This is just a small update on Neptune. I have finally gotten native Win32 compatible resources implemented in the kernel. (Well, just images anyways but still [grin]) Thanks to this, the kernel can now use bitmaps for different things without loading anything from disk. This also makes development easier as we can just use an image program and resource scripts [smile]

Also, thanks to boot time DLLs, the kernel itself is still a microkernel and has no internal drivers for anything. DLLs are used by the kernel but are separate programs.

With these additions, the kernel is still being cleaned up to better fit the new design.

I suppose thats all for now [smile] I know it somewhat looks like Windows XP do to the colors. What can I say? I like those colors [grin] I am always looking for suggestions though if you have any!

Hello folks, this is just a small update on Neptune!

We have been working on Neptune a bit in the last few weeks. The Kernel now supports native Win32-complaint bitmap resources. Thus we can create images in any image program and create a simple resource script to compile into the kernel for it to work [smile]

I really like this current setup as it allows the kernel to work with resources without needing to load anything from disk. And do to it using a VGA DLL, the kernel itself doesnt do anything at all with hardware so it follows a microkernel design.

With regards to the colors; they do look alot like XPs. What can I say, I like how they look [grin] but if anyone does have any suggestions please feel free to share!

I guess thats all for now [smile]

## Neptune Boot Loader 2.0

Hello everyone,

I have gotten the 2nd rewrite of Neptune Boot Loader complete. Thanks to the new Boot Library, the boot loader itself took only two days to make [smile]

It is capable of booting multiple operating systems through an LST file (which is basically an INI file in disguise.), and a dynamic boot menu. Supports any mountable volume and filesystem that the boot library nativity supports. About 6,000 lines smaller then the previous version as well thanks to the boot library.

Click here for a snapshot and the documentation on the new and improved boot loader [smile]

## Neptune Boot Library & OSDev Series Chapter 20

Hello everyone! [smile]

I know that I should start posting more often again - I have been very busy with work and my projects. I have still been visiting this site every day though. I am hoping to get back to a regular posting schedule soon, I promise! [grin]

OS Dev Series Chapter 20 - FDC Programming

The 20th (Yes, 20th!) installment has been released. I plan another update very soon which will include a demo.

This chapter looks at:

• FDC and FDD History
• Disk Layout
• CHS, LBA
• FDD Structure
• FDC Hardware
• Interfacing with the FDC
• FDC registers and commands

This chapter can be reached at here (Clicky!). Please let me know if there are any comments or suggestions!

I am still thinking of what should be next. I am considering between DMA and a C FAT12 minidriver chapter. I don't like filesystem programming so I may go DMA first [wink]

Update - Neptune Boot Library

The Neptune Boot Library helps reorganize the way Neptune boots and loads. It provides a very portable and hardware independent way of interfacing with display (text output), loading files (abstract filesystem access and volume mounting), PE loading, dynamic library loading, generic disk access, and a standard method of error handling.

I originally got the idea from Windows Vistas booting design. The Boot Library interfaces with the system BIOS (or can be extended to support EFI) and is written in C. Only one function - io_services () - is architecture dependent do to its usage of the system BIOS. It also supports INI and Neptune LST configuration file parsing through a standard function very similar to the Win32 APIs GetProfileString () method.

This library makes it very easy to develop C boot loaders and startup applications using Microsoft Visual C++ 2008, or any compilier (hopefully) that can support Win32 compliant PE executable images.

I am still considering if I will release this library to the generic public or not though... It certainly makes things much easier and reduces code duplication alot.

I suppose thats all for an update for now. As always any comments are welcome! [grin]
• ## Project Neptune

Make backups now!

I have lost almost 600-800 lines of code this week do to an annoying thing that happened in Visual Studio.

I copied two files from one project to another thinking that they were truly copied...Then I deleted most of the contents of the copied files (they were not needed in the new project), saved them, and closed them. Wondering where my other errors are coming from, turns out I basically ruined both copies of the files. Ugh! To make things worse: I could not get them back or revert the changes. The good news is that I did have a backup a few weeks ago and was able to recover from it.

Always remember to make backups often! You never know what might happen!!

Change in codename

"MicroOS" and "mos" seem to be fairly common codenames for "Microcomputer operating system". In an effort to stand out a little more I decided to drop this codename and remove "OS" part completely.

As of current, the new code name is Project Neptune.

Boot.ini

The boot loader now has (almost) full support for working with boot time scripts. This allows us to manually configure how we want to boot; boot time options; boot types; and additional commands dynamically. All of the internal command parsers commands are supported.

There is no special reason why I named the file "Boot.ini"...I just cannot seem to find a nice name for the file besides the above. If I name it "Boot.scr" for "Boot script", Windows will think its a screen saver :p If anyone here has any suggestions, feel free to share. [smile]

New TUI

I have also upgraded the TUI to something a little more pleasant to the eye and (hopefully) more user friendly.

I suppose that's all for now. ...Been a busy week.

## OSDev Series: Chapter 19

OSDev Series: Chapter 19

Chapter 19 has been released. There is no demo yet, but it should be released within a day or two (hopefully). It covers:
• Keyboard History
• Keyboard Layout
• Keyboard Protocols
• Keyboard Encoder
• Keyboard Controller
• Scan Code Sets
• Keyboard IO Port mapping; Commands; Interfacing
• Keyboard IRQ
The demo is also the first interactive demo in the series. It uses the new keyboard driver developed in this chapter to interface with the user through a Command Line Interface (CLI).

Chapter 19: Keyboard Programming
OSDev Series Base Site

As always, any input is always appreciated [smile]

## MoS Boot Loader

Hey everyone! [smile]

I have rewritten my systems bootloader from scratch to provide more flexible way of booting either MoS or existing operating systems, with better support for BIOS or EFI booting from different boot devices, filesystems, and architecture abstractions. Of course, there is GRUB and LILO; but my goals are different which is why I have developed my own.

Here we are. 11K lines of code later and the main menu has just been completed. I figured that I have not posted in a little while, so why not?

Heres the new boot menu:

This is with the second option selected (and enter pressed). Because its not a valid system, it displays an error. This is just a test to insure the menu rendering code worked fine.

Highlighting and selecting the first option brings the user into the internal command interpreter (Which almost none of the commands are implemented yet. Im still trying to determine how I should proceed from here) :

I got the idea of the internal command line from GRUB (Which I would like to see BOOTMGR have as well.) Im thinking of having the bootloader look in the current boot drive for a MOS.INI file and running it as a boot up script if it exists to determine how to boot. MOS.INI would contain the operating system options that would be displayed in the main menu (Which is dynamically rendered).

Currently, it only supports reading from FAT12. However future support for FAT16 and FAT32 are planned very soon as well as partitions. The current design of the boot loader software already has partitions in place; just not fully implemented yet.

Written almost entirely in C++ and designed kind of like NTLDR and BOOTMGR I am very happy with the current progress of the bootloader so far. [smile]

Now, back to coding...

## OSDev Series: Tutorial 18

OSDev Series: Tutorial 18: Virtual Memory Management

Sometimes these tutorials just get fun to write [wink] Tutorial 18 covers alot of topics including:

• Virtual Memory
• Memory Management Unit (MMU)
• Translation Lookaside Buffer (TLB)
• PAE and PSE
• Paging Methods
• Pages and Page Faults
• The Page Table
• The Page Directory Table
• Implementing Paging

It develops a virtual memory manager that manages and provides an effective interface for managing and working with pages, page tables, page directories, and virtual memory in general.

OS Development Series: Tutorial 18: Virtual Memory Management

I will be updating the tutorial with some more content and the demo within the next couple of days. Until then I am wanting to hear feedback on readability, ease of reading, and ways to improve both the text and code within the tutorial.

Please let me know what you think! This tutorial builds off the physical memory manager build in the previous tutorial.
• ## Uninitilized pointer worked for weeks!

Yeesh!

Today has started out as a typical day of programming. After making some modifications to the code to fix some warnings and split the code into smaller routines, I rebuild and test.

invalid instruction!

Okay I thought. Ill just undo the changes and try it again:

general protection fault!

Eek! At this point I was having no idea what was going on. The code worked fine for weeks without any problems; what can cause this to happen now? I decided to go back even further to see if it was any of the newer code:

invalid instruction!

*ugh*. After going through my code, I found out I was working with a pointer that was never initialized. The system uses a single singleton class called core that provides a basic set of routines that need to be used across all other classes. Thus everything uses it. All of them obtain the pointer from this routine:
core* core::get () {	return m_core;}

Keep in mind that this is self hosted running in ring 0. And all classes are using m_core to print strings, call bios interrupts, and more...and it worked fine for weeks until now.

The fix for it was simple--just have m_core point to a known valid address. Not having a memory manager nor wanting to point to specific locations in memory, I decided to have the compilier do it:
core* core::get () {	if (!m_core) {		static core pkCore;		m_core = &pkCore;	}	return m_core;}

After this change everything seems to be working fine now.

I have no clue of how this managed to work before just fine (even on raw hardware) without any problems :/ I am glad the problem turned up now though before it became a bigger issue!

If anyone knows how it managed to work for weeks ill give you a free cookie [wink]

## MoS's merger.exe

Alright... My systems boot loader consists of two separate programs. The first is a 16 bit/32 bit flat binary assembly language program (startup.sys). The second program interfaces with the first--its the bulk of the bootloader and is a 32 bit MSVC++ program. (OSLoader.exe).

The problem here is that we are trying to abstract the filesystem the bootloader is using behind a common interface. However it is not really possible to do this inside of startup.sys (Although it is possible in osloader.exe). So, what I wanted was a way to take the file loading code completely out of startup.sys.

I wanted a nicer way for this to work. So I decided to create a small program that will generate an image similar to Windows NTLDR and BOOTMGR. It can combine any number of files into one--I call it merger.exe.

Using our new merger.exe program, I updated the boot code to allow it to work as a single image file: Behold! MoS's KRNLDR boot loader! Now KRNLDR consists of both startup.sys and OSLoader.exe all in one file. Startup.sys has also been stripped of all loading code as it doesnt need it anymore because OSLoader.exe is directly after it in memory. Thus the only disk read/write code (and the only filesystem code) is in an abstract set of classes within OSLoader.exe.

While the system itself is closed source, Im leaving merger.exe to the public in case anyone is interested in it. There are a few problems with it, but for the most part it seems to work fine for my needs.

The command line for it is merger.exe outfile.bin file1 file2 file_n, where merger.exe is the programs name and outfile.bin is the output file location.

#include #include #include //! program nameconst std::string _strProgName = "merge.exe";//! program interfaceclass prog {	std::fstream	m_fOut;	void display_parm_help ();public:	bool init (std::string& fIn);	int run (int argc, char** argv);	virtual ~prog ();};//! display help info if incorrect command line is usedvoid prog::display_parm_help () {	std::cout << "Incorrect command line syntax" << std::endl;	std::cout << "Syntax: " << _strProgName << " output_file file1 file2 file_n" << std::endl;}//! inits the program//! in/ fIn - filename of output file to work withbool prog::init (std::string& fIn) {	//! open up the file in binary mode (and cleared out), create it if its not there	m_fOut.open(fIn.c_str(), std::ios_base::in | std::ios_base::out | std::ios_base::binary | std::ios_base::trunc); 	//! if file is not open, there was an error, bail out	if (!m_fOut.is_open ()) {		std::cout << "Unable to open output file \'" << fIn << "\', aborting..." << std::endl;		return false;	}	return true;}//! dtor called at program exitprog::~prog () {	//! close the output file if its still open	if (m_fOut.is_open ())		m_fOut.close ();}//! main program coreint prog::run (int argc, char **argv) {	//! argc containes number of command line paramaters passed.	//! this program requires at least 4: program_name.exe, outfile, file1 and file2	if (argc < 4) {		//! its not, so bail out		display_parm_help ();		return 1;	}	//! paramater 1=output file name	std::string str (argv[1]);	//! initialize the program w/ the output file name. If fails return error	if (!init (str))		return 1;	//! stores number of bytes written to output file	int outLength = 0;	//! for all of the input files in the paramater list...	for (int i=2; i		std::cout << "Merging file '" << argv << "'..." << std::endl;		//! open the file in binary mode		std::fstream fIn (const_cast <char*> (argv), std::ios_base::in | std::ios_base::binary);		//! get size of file in bytes, store in length		fIn.seekg (0, std::ios::end);		int length = fIn.tellg();		fIn.seekg (0, std::ios::beg);		//! read the file into buffer		char* buffer = new char [length];		fIn.read (buffer, length);		//! ...and write the buffer to the output file		m_fOut.write (buffer, length);		//! add number of bytes written and cleanup		outLength+=length;		delete [] buffer;		fIn.close ();	}	std::cout << outLength << " bytes written to " << argv[1] << ", completed" << std::endl;	return 0;}int main (int argc, char** argv) {	//! just run the program passing in the command line paramamters	prog merger;	return merger.run (argc, argv);}

...Now back to some coding [wink]

## io_services ()

Im going to be working on my real system as well as Tutorial 18 during this weekend. Before that, however, I decided to share the knowledge that I have learned while writing my io_services() routine and the new design for MoS's 2nd stage bootloader.

MoS's 2nd stage bootloader (Startup.sys) is composed of two programs. The first program is a pure binary 16bit/32 bit program. It switches to protected mode very early and uses io_services() for Bios calls for basic system initialization and to load the 2nd program of stage 2. io_services() is defined in this program, and is passed to the 2nd program.

The 2nd program (OSLoader.exe) is the bulk of the bootloader developed in Microsoft Visual C++ 2008. It uses io_services() for any hardware access. We have already seen an example of this in action in a previous post.

io_services() needs to work with both assembly language and C. Because of this, I created a local version of the structure that is passed to the routine (REGS).

There are a few important differences between how real mode and protected mode work, so we also need to save some important states:

/mos/boot/startup/i386/startup/su.asm
; protected mode stack locationstack dd 0; real mode ivt pointerivt_ptr:   istruc idt_ptr      at idt_ptr.m_size, dw 0      at idt_ptr.m_base, dd 0   iend; protected mode idt pointerpmode_idt:   istruc idt_ptr      at idt_ptr.m_size, dw 0      at idt_ptr.m_base, dd 0   iend; local copy of registersREGS_LOCAL:istruc REGS   at REGS.m_ax, dw 0   at REGS.m_bx, dw 0   at REGS.m_cx, dw 0   at REGS.m_dx, dw 0   at REGS.m_si, dw 0   at REGS.m_di, dw 0   at REGS.m_es, dw	0   at REGS.m_cs, dw	0   at REGS.m_ss, dw	0   at REGS.m_ds, dw	0   at REGS.m_flags,dw 0   at REGS.m_reserved, dw 0iend

Now for the fun stuff...the io_services() routine. With this routine does is pretty basic: Saves the current protected mode state, makes a local < 1MB copy of the INPUT REGS structure, switches to protected mode and calls the interrupt with the information passed to the routine inside of the INPUT REGS parameter.

Bochs does not seem to like me directly jumping from 32 bit protected mode to 16 bit real mode (without running into invalid LOCK prefix instructions), so I needed to add an additional jump into 16 bit protected mode. It seems to work now on all emulators, virtual machines, and real hardware that I have tried.

/mos/boot/startup/i386/startup/su.asm
;----------------------------------------------;	void io_services (int function, REGS* in, REGS* out);		-16bit input/output services;;	int function: 16bit interrupt number to call;	REGS* in: input registers for interrupt call;	REGS* out: output registers from interrupt call;----------------------------------------------io_services:	pushad	cld	mov	esi, [esp+32+8]		mov	edi, REGS_LOCAL	mov	ecx, 6	rep	movsd	mov	[stack], esp	cli	sidt	[pmode_idt]	call	disable_paging	mov	word [ivt_ptr+idt_ptr.m_size], 0xffff	mov	dword [ivt_ptr+idt_ptr.m_base], 0	lidt	[ivt_ptr]	jmp	CODE_SEL_16BIT:.pmode16	; jmp into 16bit protected modebits 16.pmode16:	call	rmode_enable	jmp	0:.rmode; jmp into real mode.rmode:	xor	ax, ax	mov	fs, ax	mov	gs, ax	mov	ds, ax	mov	es, ax	mov	ss, ax	mov	ax, [REGS_LOCAL+REGS.m_ax]	mov	bx, [REGS_LOCAL+REGS.m_bx]	mov	cx, [REGS_LOCAL+REGS.m_cx]	mov	dl, [esp+32+4]	; get interrupt to call and set it	mov	byte [.call_service+1], dl	mov	dx, [REGS_LOCAL+REGS.m_dx]	mov	si, [REGS_LOCAL+REGS.m_si]	mov	di, [REGS_LOCAL+REGS.m_di]	mov	es, [REGS_LOCAL+REGS.m_es].call_service:	int	0; invoke interrupt. Above code modifies int to call	pushfd	pop	dword [REGS_LOCAL+REGS.m_flags]	mov	word [REGS_LOCAL+REGS.m_reserved], 0	mov	[REGS_LOCAL+REGS.m_ax], ax	mov	[REGS_LOCAL+REGS.m_bx], bx	mov	[REGS_LOCAL+REGS.m_cx], cx	mov	[REGS_LOCAL+REGS.m_dx], dx	mov	[REGS_LOCAL+REGS.m_si], si	mov	[REGS_LOCAL+REGS.m_di], di	mov	[REGS_LOCAL+REGS.m_es], es	call	pmode_enable	; jmp back to protected mode	jmp	CODE_SEL:.returnbits 32.return:	mov	ax, DATA_SEL	mov	fs, ax	mov	gs, ax	mov	ds, ax	mov	es, ax	mov	ss, ax	mov	esp, [stack]	cld	mov	esi, REGS_LOCAL	mov	edi, [esp+32+12]	mov	ecx, 6	rep	movsd	call	enable_paging	popad	lidt	[pmode_idt]	ret

I guess thats it for my io_services() routine [smile] This routine seems to work very well. However, it will be a performance penalty as we are switching between three processor modes, two copies, and saving/restoring states. Its quite effective though as it allows us to use BIOS interrupts in 32 bit programs in C or assembly language. (note: This will only work if you are in supervisor mode)

If anyone has an suggestions to improve it, I would very much like to hear them!

## OSDev Series Tutorial 17

OS Development Series Tutorial 17: Physical Memory Management

Tutorial 17 has been fully released! This tutorial covers alot of topics and the development of a full physical memory manager (PMM). Topics include:

• Physical Memory concepts
• Translation Lookaside Buffer (TLB)
• Memory Management Unit (MMU)
• Memory Managers
• Abtaining memory information from BIOS
• Developing a physical memory manager

• OS Development Series: Tutorial 17

I want to cover every single thing about how memory works. If anyone has additional information, comments, suggestions, or ideas to improve this tutorial please let me know!

The demo should work with Microsoft Visual C++ 2005 and 2008 (tested on both), Bochs, VirtualPC, and real hardware.

## MoS Updates OS Dev Series Tutorial 17

MoS: New functionality

Okay... what I wanted to do is provide a method similar to DOS's int86() routine. The basic idea is that--in our 32 bit protected mode C++ program, we can call our routine (boot_services() ) to call BIOS and other real mode interrupts from protected mode.

With this new routine added, I am expanding my bootloader with much more functionality then was previously possible.

Here is it in action. Here, io_services(), when called, goes into real mode and uses the BIOS to perform an INT 0x10 call. It then switches back into protected mode while outputting the output registers to us so we can abtain information.

void _cdecl osloader_entry ( void (_cdecl *io_services)(uint32_t function, INTR* in, INTR* out) ) {        //! get current video mode	INTR in, out;	in.h.ah=0xf;	io_services (0x10, &in, &out);	DebugPrintf ("oldmode: 0x%x", out.h.al);	//! switch video mode (classic mode 0x13! Woo-hoo!!!)	in.x.ax = 0x0013;	io_services (0x10,&in, &out);	//! clear display to blue	memset ((void*)0xA0000,0x1,320*200);}

The code for io_services() is needing to be cleaned up a little. It is passed by the 2nd stage boot loader to the OS Loader program by a function pointer. This method seems to work for all interrupts...BIOS, VESA, etc..

Oh, and Mode 0x13 will never die[cool][grin]

OS Development Series: Tutorial 17...thats right, 17

The tutorial has been released. Its demo should be up tomorrow or Saturday. I am considering whether I should announce it on the forums or not though.

## PMM Completed and FPU Support

Alot of nice updates: The physical memory manager is completed and I added basic floating point support for our MSVC++ runtime library.

Floating Point Support

float f = bootinfo->m_memorySize / 1024.0f;
Do to runtime support, many compilers attempt to call runtime functions. For example, in MSVC++ 2005 the above line of code calls _ftol2_sse(). If this is not defined, you will get linker errors.

Heres my attempt at it:
//! called by MSVC++ to convert a float to a longlong __declspec (naked) _ftol2_sse() {	int a;	_asm {		fistp [a]		mov	ebx, a		ret	}}
It seems to work with my tests. The above uses the FPU to convert the floating point value on the stack (pushed by MSVC++) to an integer, and returns it.

Physical Memory Manager (PMM)

The physical memory manager has been completed today. I dunno if I will add anything else to it. The above demo will probably not be the one for the series. I dunno ;)

Here is the current interface:
extern	void	pmmngr_init (size_t);extern	void	pmmngr_init_region (physical_addr, size_t);extern	void*	pmmngr_alloc_block ();extern	void	pmmngr_free_block (void*);extern	size_t pmmngr_get_memory_size ();extern	uint32_t pmmngr_get_use_block_count ();extern	uint32_t pmmngr_get_free_block_count ();extern	uint32_t pmmngr_get_block_count ();
A "block of memory" is simply a 4KB block of physical memory. The PMM uses an internal bit array structure that represents each block of memory. Because each block is represented by a single bit, its a very small structure.

Not all physical memory is available for us to use. Some areas of memory may be being used by memory mapped devices, other areas may contain physical memory holes in the physical address space (PAS).

Because of this, by default all memory is set to "used". The kernel obtains available regions of memory from the boot loaders multiboot_info structure and uses pmmngr_init_region() to initialize regions that are safe to use. (The multiboot_info is another new addition to the demo that allows the boot loader to pass information to the kernel)

After an area is initialized, simply call pmmngr_alloc_block to allocate a free block of physical memory and pmmngr_free_block() to release it so it may be used again.

## VirtualPC supported... again ^^

VirtualPC Compatibility problems fixed

As some of our readers may know, our system has had some problems running on VirtualPC, but working fine on Bochs and on certain hardware.

Well... The problem is resolved! The new code now boots and works fine with both VirtualPC and Bochs. I am planning on testing it with VMWare next to see how that holds up.

All the system currently does is self-boot and displays some memory info for its physical memory manager...nothing much [smile]

OSDev Series: Should I write a book?

I have had a few people suggest turning the OSDev Series into an actual book. I was not thinking too much about it as it seemed like a big undertaking.

...That is, until I realized the OSDev Series is already over 300 pages. O_O Jeeze, I never realized how much I typed when writing those tutorials!

Considering that I am currently updating every tutorial, perhaps I should make it more so an actual book instead? Mabey provide more content, lists and tables, more friendly context?

I dunno. Its interesting how the series has turned out... I never thought of ending up sticking with MSVC++ as much as I am. Perhaps I am just used to and like it.

What do you think? Turn it into a book (Probably both an ebook and printable one)? If I do, are there anything that you would want added or changed?

Im kind of stuck on what I should do here... [smile]

Yey...

Bugs, bugs, and more bugs. Don't you wish we can just spray all of them to eliminate the bug problem for good? Oh well...

There are just two small updates: 2 bug fixes, and the updating of Tutorial 17. Besides that, Eveh3D Scene Editor is being rewritten using the wxWidgets framework.

Also, we are going to be revising all of the tutorials soon along with updating bug fixes in the demos. Any and all suggestions or comments welcome!

Here we go...!!

OSDev Series: Tutorial 17 Update

For those who are following our OS Dev Series, another change is in effect for Tutorial 17. Tutorial 17 was originally planned (and written) to cover Memory and Virtual Memory. However, I now conclude that this is a bad idea.

In order to effectively manage virtual memory, we will first need to effectively manage physical memory. This, of course, is where two memory managers come in: A Physical Memory Manager and a Virtual Memory Manager.

Both of these are going to be discussed as separate tutorials:

OSDev Tutorial 17: Physical Memory Management
OSDev Tutorial 18: Virtual Memory Management

I feel that this will make the tutorials a little less cluttered and easier to follow. It will also allow me to focus on memory managers, rather then just "paging and virtual memory", giving them more of a direction.

Lets hope it turns out well...

The PE kernel loading fix was caused by the bootloader incorrectly parsing the PE's header information. Its all fix now :) And it also allows us to NOT have to use that Order.txt trick (Please see OSDev MSVC++ Tutorial for more info)

Heres the new kernel parsing code for those that are interested. BASE is the base address that the program is located at. fileSig is just "PE". I added the test to insure a valid program was being executed. If not, it will display an error and halt the system.

	;-------------------------------------------;	;   Execute Kernel.                   	    ;	;-------------------------------------------;	mov		ebx, [BASE+60]   ; e_lfanew is a 4 byte offset address of the PE header; it is 60th byte. Get it		add 		ebx, BASE	 ; Add base address. EBX now points to file sig (PE00)	; test the file sig to insure its a valid PE format	mov		esi, ebx	mov 		edi, fileSig	cmpsw	je		.execute	mov		ebx, msgUnknown	call		Puts32	cli	hlt.execute:	; jump over to optional header (Although it isnt optional o.0 )	add		ebx, 24	mov		eax, [ebx]	 ; _IMAGE_FILE_HEADER is 20 bytes + size of sig (4 bytes)	add		ebx, 20-4	 ; address of entry point	mov		ebp, dword [ebx] ; get entry point offset in code section		add		ebx, 12		 ; image base is offset 8 bytes from entry point	mov		eax, dword [ebx] ; add image base	add		ebp, eax	cli	jmp		ebp		 ; Execute Kernel

This should call the entry point routine of a 32 bit PE executable. ie; in my case it calls my C++'s kernels mainCRTStartup() routine.

memset() bug

The original code was this. I'll first post the bug to see if anyone see's it ;)

//! sets count bytes of dest to valvoid *memset(void *dest, char val, size_t count){    unsigned char *temp = (unsigned char *)dest;	for( ; count != 0; count--, temp[count] = val, temp++);	return dest;}

Thats it! Do you see it? I don't know what I was thinking when I wrote temp++ o.0 The fix is just to remove the temp++.

---

I suppose thats it for now..

## Executing an array as code

When reading This thread the first post seems like they were looking for a method of treating an array as a function.

In whatever case, everyone who reads that thread knows a scripting language is the best tool here. I decided "what the hey" and try it anyways :)

Here it is for anyone who is interested. The most ugliest hack that will ever go through your eyes:

char code[3] = {	"\x90"		//nop	"\xC3"		//ret};int main () {	void (*funct)() = (void (__cdecl *)(void))(void*) &code;	funct();}

This executes the instructions found in the code array as if it was a function. It only executes a NOP (No operation) instruction and RET instruction (to return) though.

Wee...! The fun with C++! [grin] Though I prefer inline assembly much better then this :)

## Argh... C++

Okay...

So I am writing a codec for ms3d model files for my engine. Pretty routine procedure. I have two files... EvehMs3d.h and EvehMs3d.cpp for the codec (along with Eveh3D's standard Link.h and Dll.cpp files for proper bindings).

Everything good so far. Then it hit... BAM! I get linker errors. Not to hard, I thought. The weird thing? There was no reason for them.

Here are the linker errors:
3>Linking...3>EvehMs3d.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: __thiscall Eveh3D::Scene::MeshCodec::MeshCodec(void)" (__imp_??0MeshCodec@Scene@Eveh3D@@QAE@XZ) referenced in function "public: __thiscall Eveh3D::EvehMs3d::MeshCodec::MeshCodec(void)" (??0MeshCodec@EvehMs3d@Eveh3D@@QAE@XZ)3>EvehMs3d.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: virtual __thiscall Eveh3D::Scene::MeshCodec::~MeshCodec(void)" (__imp_??1MeshCodec@Scene@Eveh3D@@UAE@XZ) referenced in function "public: virtual __thiscall Eveh3D::EvehMs3d::MeshCodec::~MeshCodec(void)" (??1MeshCodec@EvehMs3d@Eveh3D@@UAE@XZ)3>..\Debug\Plugins\EvehMs3d.dll : fatal error LNK1120: 2 unresolved externals

It is basically saying that it cannot find the import symbols (Which are defined in Eveh3D's MeshCodec.h.) However, that was, of course included, or else I would be receiving compile time errors instead.

The problem seems to be do to the compile time bindings of the symbolic names within Eveh3D's MeshCodec.h. All I needed to do was create a 1 line source file that included MeshCodec.h...And it resolved all problems.

In my opinion, this is a waste. I would be expecting #including a header file would resolve import and export symbols (They are behind EVEH_EXPORT symbol). I guess not...

Has anyone else experienced this issue? (Where they would need to create a new source file with about 1-2 lines to include another to fix linker issues?)

Oh, and did I mention the best part? It took about 3hrs to figure out. Ugh!!

Oh well, least its working now. *runs back to coding*

## Project Update and Possible x64 support to OS

I decided to post a little update on the current progress of Eveh3D. Eveh3D is almost at its v. 0.1 beta release, which we cannot wait for!

This is the current screenshot of the Eveh3D Scene Editor, which will be used in designing the levels that will be used within our current RPG, and future projects.

I guess thats all for now... We am hoping to release another in-house test demo for the editor and engine around the end of the month. ...I just hope we can make it.

Because of this um.. "crunch mode" that I am in, I had less time to work on the OS dev series. However, We are thinking of merging the OS dev series OS with another system to better support both x86 and x64 platform architectures. We are still deciding on this, though.

It would certainly make the system better in the long run, and support more features. ...At the cost of adding more complexity to the series' tutorials.

I dunno, I guess we will see what happens.

## Eveh3D Editor and Cleanup

*Ugh* cleanup.

I am to admit that some of the code was left in for testing. I am sure many members here know what I am talking about... When you put in code for testing, then completely forget about it. Ugh.

Anywhoo...

Eveh3D World Editor

As some of you may know, I have decided to start work on an editor for our engine. I have not talked much about it until now. Its great for both a test for the engine as a whole, and for us to finalize the engine's structure.

...Not to mention, we will need to do this anyway. This editor will basically allow us to import different images, terrains, skyboxes, and edit varies of different things (lights, light position, properties of these objects, etc.) that will be stored within a scene graph. When saving, it will save the scene graph nodes as an XML file that can be read by the engine.

This will allow the game to load a single scene very easily. The games' scripts will decide when and what scenes to load based off the level that the game is in. The exact details of everything is still being decided, so we are always looking for more options here, and how we should store the scene information.

Current pics

Some objects being rendered using different properties; stored within the scene graph. Please note the scene graph view window is not finished yet.

Rendering a white sphere, with lighting, solid rendering, Not scaled

Rendering a *.tga 32bpp compressed textured cube, with lighting, solid rendering, Not scaled. Here you can also see log console window

Lighting, Texturing, and Material management has been all fully integrated into the engine.

Sphere and cube scene nodes are also integrated into the engine for use by the scene manager. You can see them being rendered in the above pics.

Geomipmapping

We are going to treat the terrain as a standard scene node, along with the other objects. The terrain will be rendered using an LoD algorithm via geographic mipmapping technique. Lets hope it turns out well...

OSDev Series - Tutorial 15

This tutorial should be completed either today or tomorrow. I'll keep you posted...

The tutorial itself is completed. I just need to finish working on the demo.

## Eveh3D Scene Editor

Hello World! ...Again!

We have decided to finally turn our demo tests software into the beginning development stages for Eveh3D's Scene Editor.

We are considering of using XML to use within our scene file formats. We are still considering on how we would like to approach everything, however.

We need opinions!

There are alot of different formats and methods out there that we can use to store our scene information. We like the way Irrlicht stores its scene information, but it seems to have a big drawback. How can we find a way to store all scene data within a single file without any dependencies?

i.e., Not having the scene file refer to what other file contains an objects mesh.

We are currently looking for any suggestions that may help with us in deciding these factors.

Thanks to all or anyone with suggestions :)

## OSDev Series Tutorial 14 and 15 Updates

OS Dev Series Tutorial 14: Basic CRT and Code Design

Yey!! Its here!!

This tutorial covers some basic groundwork of our system, and some C++ runtime (CRT) library routines, some basic debug output routines, and code design and stressing good programming practices that is required knowledge for large scale software. There is also a complete demo using NASM and MSVC++ for the system bootloader and kernel, with each of its programs (Kernel, Hardware Abstraction Layer, and CRT) as separate library projects for easier support for us using these as dynamic libraries later if we like. (Hal.dll, Crt.dll)

If you ever wondered how variable length arguments work, or even basic text output via a self developed printf() routine, this is for you... ;)

Tutorial 14: Basic CRT and Code Design
OS Development Series Base Site

OS Dev Series Tutorial 15 Plans

With all of the basic code and layout completed, it is time to get back to the details... Tutorial 15 is planned to cover:

Tutorial 15: Kernel - Errors, Exceptions, Interruptions

>Error Handling Concepts
>Exception Handling Concepts
>Interrupts
>IRs (Interrupt Routines), ISR's (Interrupt Service Routines), IRQs (Interrupt Requests)
>Interrupt Handling Concepts
>Software Interrupts
>Hardware Interrupts
>INT n, STI, and CLI instructions, IDTR cpu register
>Interrupt Descriptor Table (IDT), and LIDT instruction
>Hardware Abstraction Layer (HAL): Abstracting an interrupt management interface

Weee...its going to be fun :)

## Terrains, Terrains...

Hey everyone,

The terrain itself is almost complete... This is some newer pics of our 2d/3d hybrid game engine in action. These use our targa library and opengl ports. (EvehTga.dll and EvehGL.dll)

We are still working on the terrain to give it more detail. the terrain itself is purposely not scaled completely, so its polygonal in some areas. We will fix this very soon.

Here is an earlier pic of a test terrain for our terrain file codec (EvehTer.dll), which was used to design these terrains, in both solid and wireframe modes:

We are looking for more methods that we can use to add more content to this terrain. If anyone has any suggestions to help out, please feel free to let us know!