Jump to content

  • Log In with Google      Sign In   
  • Create Account

frob

Member Since 12 Mar 2005
Online Last Active Today, 09:29 PM

#5188482 Heap Error

Posted by frob on Today, 02:07 AM


Oh Just a thought while posting this if this is called on another thread is it the same object as in the main thread.?????????

As this is For Beginners, I'm going to recommend you rip out everything thread-related in your program right now. 

 

Threading adds concurrency bugs and race conditions. If you thought this class of bugs was bad, concurrency bugs are far worse, since they are the same type of bugs compounded across a dimension of additional processes.

 

Remember when I wrote "Coming in after memory corruption errors and race conditions, I'd rank it as probably the third biggest source of the nasty evil nemesis bugs that can haunt a code base for months or even years before isolating it and finding a fix" above? 

 

What you are saying is "I'm having trouble with the third-worst source of nasty bugs.  Now I want to throw in the second-most evil source of nightmare fuel, and both of those sources can trigger the absolute worst of the three: memory corruption bugs.  STOP. DO NOT PASS GO.

 

"Threading" and "For Beginners" are a terrible mix unless you enjoy spending your days hunting seemingly random crashes rather than learning useful material.

 

I strongly recommend you remove everything related to threading in your personal project, and stick with the much simpler realm of linear programming.




#5188463 Heap Error

Posted by frob on Today, 12:07 AM

While those are great examples of why it happens, usually the code is much more tricky to diagnose than those three line examples. 

 

The patterns are the same but the bugs can be in very different parts of code. The allocation can be handled in one library, then the pointers go through your program, and you pass the pointers to a second library which maintains an out-of date pointer.  

 

Also, they don't necessarily happen immediately. The condition can happen ages later.  It is possible that some portion of the code was holding a pointer that was freed minutes later, or even hours or possibly days later in a long running program. 

 

"Its Complicated" is the truth. Sometimes they are very easy to find.  But sometimes they are nightmares. Coming in after memory corruption errors and race conditions, I'd rank it as probably the third biggest source of the nasty evil nemesis bugs that can haunt a code base for months or even years before isolating it and finding a fix.




#5188459 What is bad about a cache miss since a initial cache miss is inevitable

Posted by frob on Yesterday, 11:25 PM

Caching is one of those magical things that take some understanding.

 

 

* It is not so much that a single cache miss is bad. As you wrote, they sadly happen quite frequently.

* It is much more that a single cache hit is awesome. And we are used to modern compilers and processors being very awesome.

 

Due to the complex nature of the modern CPU the exact benefit of a cache hit is hard to get an exact benefit, but usually it makes the operation approximately free.  Again: having the data already loaded usually results in free instructions.  Since you can often store 4 values in a cache line but only pay the cost for a single load, buy one get three free.

 

 

Good: If your code and data is able to operate entirely in the cache, you can see enormous speed improvements. 

Bad: If your code and data doesn't play nicely with cache, you will see performance plummet since you pay full price for everything.

 

 

Let's make a little thought experiment:  Imagine data carefully and artificially designed so every single memory access was a cache miss, and needed to be fetched from main memory.  (Or just imagine the data cache was turned off.)

 

The exact time required varies based on the chipset, the motherboard, the ram chip's timings, and more. You've got those numbers on your memory chips like "9-11-11-28" or whatever, those come in to play, too. And just for fun, we'll say you've got DDR3-1600 memory since they usually take 5 nanoseconds per cycle. So if you get a trip to memory, you incur at least these costs:

 

CAS latency, roughly 12 nanoseconds. It just takes this long to go through.

Time to read and issue the command, from the memory chip above that is 5ns (from the 200MHz) * 11 cycles = 55 nanoseconds minimum.

Time to charge a row on the memory chip, again 11, so 11*5ns = 55 nanosecond minimum. 

And the final number is the cycles of overhead so the chip can basically change gears between requests. We'll hit this every time since in this scenario nothing is cached and we need to wait for each row. So 28 * 5 = 140 ns minimum.

 

Then you've got overhead for getting out of the chip, processing along the system bus, and more, but we'll just discount that right now. We'll round down and call that number 250 ns for one trip out to those DDR3 memory chips instead of the cache.

 

So by incurring a cache miss out to main memory, you just cost your process about 250 nanoseconds. 

 

And if your cache miss was something swapped out to disk, it can potentially take a LONG time to get that data, especially if the disk has spun down. On a slow spinning disk that was sleeping, the cost of your cache miss might be multiple seconds.

 

Normally that many nanoseconds is not a particularly enormous amount of time by itself, lots of things cost nanoseconds. Virtual function dispatch, for example, has a cost of roughly 10 nanoseconds. We usually don't lament the cost of a single virtual function because the cost of implementing the same functionality through other means will be at least as expensive. But with virtual functions and other costs, those nanoseconds are spent buying us functionality.  With a cache miss, we pay a cost and get nothing in return.

 

So in our contrived little program every single memory read and memory write incurs a full trip out to main memory.

 

A simple little addition function, x = a + b + c, gives three loads and a store, or 4 trips to memory, costing 1 microsecond in overhead.

 

The problem is when it adds up quickly.  If *EVERYTHING* is a cache miss out to main memory, then *EVERYTHING* gets a 250 nanosecond penalty.  

 

4000 trips to main memory mean 1 millisecond lost. That is about 10% of a graphics frame on a modern game that you just spent in overhead.

 

Roughly 40,000 trips to main memory means you just paid the cost of a full graphics frame accomplishing no productive work.

 

So in our thought experiment with a non-caching processor, if you want a fast frame rate you are limited to around 30,000 total operations per frame.  It is not because the processor is slow -- the processor can handle the work quite handily -- it is because the cost of trips to main memory dwarfs all other performance concerns.

 

Now, we'll remember that modern chips don't work like the thought experiment. Importantly, they do have three layers of cache turned on. The systems transfer larger memory blocks than a single byte over to multiple level of caches, often modern chips have L1, L2, and L3 caches. The fastest cache is tiny, and they get incrementally slower but also larger. So thankfully you need to make trips out to main memory very rarely.

 

Many times when you have a cache miss on the die it will jump out one level of cache and find the data right there, almost instantly available. That fetch serves as a warning to the cache levels outside it to prefetch the following blocks of memory. So even then a cache miss is not 250ns it may be only 30ns or 80ns or 100ns or whatever because it is already in the L1 cache, or the L2 cache, or the L3 cache, and not sitting out in main memory. 

 

 

One of the biggest differences between Intel's i3/i5/i7 and similar chipsets, the thing that far and away is the biggest factor in their compute-bound performance, is the cache size. On the current Haswell chipset the cache sizes are 2 MB on the lowly Celeron, 3MB on most of the i3, either 4 or 6 MB on the i5, 8MB on the i7, and 20MB cache on the i7 Extreme.  On the server lineup (e3, e5, and e7) they range from 8MB to 45MB of cache. Cache is the way the chip gets fed. It doesn't matter how fast the processor is if the processor is sitting around waiting for data to compute with. A bigger faster cache is usually the most critical performance enhancer for compute-intensive systems. 

 

The secondary feature that interacts with caching is hyperthreading. Hyperthreading (as explained in the article lined above) basically just has the same processor inside and bolts on a second instruction decoder. It lets you run two processes on the same internal core. That way when one process is stalled by a cache miss or other performance hit, one process can stall waiting for data while the other process continues to run with data in the cache.  Of course, if both processes are compute-intensive they both end up fighting for cache space.  If you are not compute-bound hyperthreading can give nice speedups. When you are already fighting over cache space in compute-bound threads, hyperthreading doesn't help matters.  That is why the second biggest difference between the different Intel processors is hyperthreading, which is basically just taking advantage of properties of the cache.

 

 

Hopefully that will give a bit deeper understanding about CPU caches. 

 

TL;DR:  Individual cache misses are inevitable but every cache miss has a cost with no benefit. Better cache use and bigger caches both provide better performance for compute-intensive tasks.




#5188398 Is there a way for me to see my game`s performance or frame rate?

Posted by frob on Yesterday, 03:25 PM


Some engines calculate the framerate from the frametime (framerate = 1000 / frametime_in_milliseconds, or framerate = 1 / frametime_in_seconds). This is cool but I don't like the flickering framerate as numbers change every single frame.

 

I love a one-second interval update showing three values with actual time spent:  min / avg / max

 

For example, you might have these microseconds:    8413 us / 9341 us / 9642 us

 

Seeing a spread of times gives a lot more insight.  For some made up illustrative examples:

 

8413 us / 9341 us / 9642 us  <-- Comfortably 100 frames per second, very consistent times each frame.

1534 us / 9458 us / 27314 us <-- Roughly 100 frames per second, but erratic timing with occasionally very slow frames.

0 us / 16000 us / 875345 us   <-- Exactly 60 frames per second, but still a slideshow since 59 frames are instant and one frame is a full second. Fire everyone.

 

Giving both the min and max let you see much more than just the average. They help identify bottlenecks and dropped frames that may not be as obvious with just an average alone.




#5188397 Portfolio Review Request

Posted by frob on Yesterday, 03:10 PM

I see none of the major common problems, and instead see lots of personal decisions, examples of code, transferable skills, and showing rather than telling.  That is quite good up front.

 

 

Like the web site. It communicates well, although it isn't super pretty. I'd call it functional, unless you are going for a web-based ui job.

 

The site could use videos (or links to videos) in addition to the pictures, but what you've got is quite telling. It has everything needed to make a decision right up front.  You let me see the code and see an executable, both are good. 

 

Linking directly to your github account can be a double-edged sword, since it makes it very clear about what you do over time. Seeing you've had 115 contributions, and the work was almost entirely clustered in July and August, I can make various kinds of judgments based on that information that may or may not reflect your work, and I can see how much you accomplish over time.

 

The resume is two pages, but with your earliest achievement being a scholarship in 2005 that may be too much, but it is a choice you can make. I would consider condensing it down to one page, but the acceptable length is both a regional trend and a personal preference. It feels like it has a lot of white space trying to be a filler to make it two pages, but it is readable and does not appear filled with fluff.

 

The resume gives a lot of transferable skills and tells me both what you did and how you did it. In that regard it is better than most who try to break in. I'm not terribly thrilled about the colored boxes and don't know how they will react inside fancy HR tools, but that is entirely your call.

 

 

Looks good to me, and I would certainly read it over more than once in the initial pruning. (That is the brutal pass where over 90% get thrown out, and roughly half of those get thrown out with about five seconds of looking over the page.) 




#5188184 C++ header inclusion question

Posted by frob on 20 October 2014 - 02:17 PM

Agreed about the duplicate include guard names. It is a fairly common error.
 
An alternate solution is to use:

#pragma once

 
The #pragma commands are system specific commands, but "#pragma once" support is very nearly universal, and is supported on all the major compilers.
 
There are a small number of cases where that fails, such as if a file exists twice in your source tree and is pulled in once from each location. Don't do that.

 

If you are sticking with inclusion guards, add some unique value beyond the file name. A guid works, the time you made the file, or even just a collection of random banging on the keyboard.  Something like #define header1_h_filemadebyfrob10202014

 

The older versions of Visual Studio used to ship with a macro that generated inclusion guards with both the file name and a guid to prevent duplication.  I would recommend using that type of macro... ... except the Visual Studio team decided to abandon the "best in class" software and is currently on a race-to-the-bottom of cutting critical features and making the non-essential features (like scanning the code for inline documentation) block the developer from important tasks (like code editing), and in their process of self-sabotage they removed macro support. If you are on VS2010 or earlier you can still have that option. Personally I recommend sticking with VS2008, the last version of Visual Studio where they actually cared about the developers, and plugging in a new compiler. Then you'll have that inclusion guard macro in the built-in collection.




#5188080 Does Valve have a good working methodology?

Posted by frob on 19 October 2014 - 11:08 PM

Politics is not inherently bad, but it can become bad. Office politics generally is the use of power inside an organization.

I've seen it broken down many ways over the years, but the best I've seen is PIRPET:

* Positional. The boss is in charge, and exercises the power through distributing information and work.
* Informational. The office secretary has much power through wielding and sharing information.
* Resources. Those who control resources have significant power, and share it by allowing access to resources. They may not be those people above.
* Performance. You know the person, you can ask them for a little help with something and they'll give awesome results.
* Expertise. Similar to informational above, but more like the senior person who has seen everything before and can mentor you through anything.
* Traits. Basically under the advice "be beautiful". The power comes from their personal traits, from charisma to strength to perfect hair. Many people don't like it, but this is a source of power.

Each of them can be used negatively to install roadblocks or prevent people they don't like from reaching goals.

They are trying to reduce positional power, since it tends to serve more as a restrictive negative thing rather than a positive thing. Many people with positional power are stingy with the power, instead of enabling those under them to get the jobs done they hoard and prevent others from doing things.

Note that all the other elements of office politics and office power are alive and well under their structure.


#5188072 Class with static compiler const member variable?

Posted by frob on 19 October 2014 - 10:30 PM

So you're looking at a do it yourself job for RTTI?

You can make a variable the way you describe, and you can use your own integers and other numeric constants. A switch should work with known values that are legal in a switch (basically integers since it devolves to a jump table) and you can use direct comparisons also as you described above.

But really, it looks like you are looking for an alternate solution to something that is already provided to you.


#5188064 The basic structure of game programming

Posted by frob on 19 October 2014 - 08:34 PM

That is true for a subset of games. Most of the interactive console style games follow it on a broad spectrum.

Many of the popular games follow the model.

However, it is not true for most games on the market. Many are event driven, turn driven, input driven, or compute driven, rather than a tight loop of render/simulate.

For a few....

Text adventures are not render/simulate.

Chess and checkers and forms-based games are not.

At the extreme, Deep Blue, a massive chess simulator, is not based on a render/simulate loop.

A large percentage of mobile games are turn based where they take some action and wait for input, are not tight render/simulate loops since that rapidly drains batteries.

Many online games may present a client that does its own graphical things but under the hood are REST-based turn oriented games where the data tells the database to run a small update, and the client happens to be visual but is unnecessary; the client may present lots of graphical and visually interesting effects but they are completely irrelevant to the actual game component.

The loop adds a lot of difficulty because you are suddenly in the realm of soft realtime requirements. You must update regularly or it is a defect. That means you need to restructure all your algorithms to run within the time frame, or structure them to span frames somehow.

So while the tight render/simulate loop is one of many possible options, it is also one of the more compute-expensive and annoying routes to implement.


#5188063 Is there a specific name for fake-SP-via-MP game architecture?

Posted by frob on 19 October 2014 - 08:19 PM

AFAIK this originated with the original Quake engine, and is what virtually every FPS since has modeled it's netcode after.

Predates that.

Many early console games from the 1980s were written such that the number of players could be anything, trivially replaced with either an AI or a human.

Among other things, this could be enabled the 'attract' mode in arcades, where the game would play against itself. It could also allow for an arbitrary number of humans where AI players were easily swapped in. Insert a coin and the player routine could be replaced.


#5187932 Steam OS survey - reflection of reality?

Posted by frob on 18 October 2014 - 11:27 PM


Naturally this is viewed through the filter of "people who use Steam". For those of you who see actual real sales of your games outside of Steam, do you observe similar distribution figures?
Partly because game players tend to favor it.  But also partly because of virtualization.

 

Lots of people who use a Mac and play games will have a virtual install of Windows. Lots of people on Linux will use a VirtualBox installation (with the DirectX extensions) or WINE (with WineD3D).  Sometimes it works great, sometimes not so much. Lots of games work just fine inside it.

 

A side effect of that kind of portability is that the target becomes that much more valuable. More people show up as using Windows since they are inside a Windows virtual machine.

 

The effect happens in other places, too. When a group wants to have their file formats or their systems become popular one of the best things they can do to help themselves is to write importers/exporters for their competitors.  Make it easy for other groups to move in to your product, and reduce the fears that they will be locked in to your minority-share product.




#5187886 Inexperienced Programmer looking for Help

Posted by frob on 18 October 2014 - 03:23 PM

What do you want to do?

Right now it looks like you are trying to be the "idea guy". That isn't a real job, and everybody in the industry already has more ideas than they can possibly implement.

If you want to be a programmer, as it looks like, then take the idea and put it on the shelf for a while. Start with the basics. Start with text based games. The simple "guess the number" and "tic tac toe" and "connect four" style games. Do this while at the same time as learning how to program by reading programming books.

Programming is the application of data structures and algorithms, so to be a good programmer you need to master the core structures and algorithms.

And you need math, since nearly everything in games relies on it. In 2D worlds you need a solid grasp on trig, algebra, and statistics. In 3D worlds you need a solid understanding of linear algebra in addition to those above. Get them either through studying on your own or study at a school, but you will need them to be a good programmer.

The path from learning to program to be reaching professional skill levels usually takes multiple years.

If programming is not your thing, there are other disciplines available to you. The other major, reliable paths to break in to the industry are through making models and through making animations. There are other paths in, but "game designer" is not an entry level position. Programmer, modeler, and animator are reliable. Level builder, writer, QA, and other paths can also get you in, but they are less reliable. But again, these are also multi-year paths.


Alternatively, if professional game development is not your goal, and you just want hobby toys to play with, Google can help you find many different isometric game engines. You may or may not be able to understand how they work, so try a few and see what feels comfortable.


#5187734 Encoder/Decoder Licensing (Audio/Video) - On Mobile

Posted by frob on 17 October 2014 - 04:36 PM

Do you need an encoder, or just the decoder on the device?

 

Video decoding tends to be quite resource intensive. However, both devices have API support for various mpeg4 decoding, with android.media and the iOS AVPlayer both supporting H.263 and H.264 AVC, both of those have some degree of support.

 

If you need something more game-friendly, Bink is less pricey than MP4 licenses, but still has a pricetag. If you are going through a major publisher you may be able to use one of their bulk licenses, which can make things convenient for little guys.




#5187729 Question about programming portfolio

Posted by frob on 17 October 2014 - 04:21 PM


I've never once ever looked at an engineering applicant's "portfolio" and I haven't heard any colleagues at other companies doing so, either.
I've looked at a very small number of them, when the person asked to have it looked at, and only AFTER they have made it to the short stack.

 

There is usually no time to look at them when they are still in the big stack. The goal is to get from 100 or 200 or more down to around 10. Pruning the first stack is brutal. There are many people who have both a degree AND industry experience, so unless we are creating a position specifically to pick up some entry level or intern-type workers, there just isn't time to see how awesome your portfolio is.

 

There are some times (rarely) where I'll see something that catches my eye and I'll move it to the small stack.

 

One of them, a decade ago and very memorable, was a recent graduate. He made a game for a senior project. Half of his resume was devoted to the project, including headlining that his senior project was sponsored by both Microsoft and NVidia, and submitted to assorted big competitions. And he was the local SIGDA president, and had picked up GDC scholarship.  So I took the bait and looked at it.

 

The name was memorable ("Modern Warfare") and a few seconds on Google shows the program (100MB download, back in an era when that needed a serious warning) is still up, along with the web page for the senior project.

 

If you look at the game and realize this was his senior project back in 2003, you'll understand how he managed to get multiple job offers from that little senior project.

 

 

The hard part is making it to the short list.

 

Once you've made the short list, if you think your demo is awesome enough to be shown, then push the issue.




#5187464 Unity3D Board Game development

Posted by frob on 16 October 2014 - 01:36 PM

Online
multi-player
Board game (like monopoly)
Windows
MacOS
Mobile version


Interesting list.

Do you know how to make online games? Do you know how to build multiplayer software? Do you know how to program for Windows? Do you know how to program on Mac? And actually have macs to test on? Which mobile platforms? Do you have equipment for all the platforms you want to target?
 
It is an ambitious list, to say the least.
 

Thanks, and please can you write some links for a tutorials - i just don't know how start write it

I suggest you start there.

Learn by building something small. Then experiment with something bigger.

In this case, you might start by using Unity to make a simple board game.

Then perhaps how to target multiple platforms.

Then perhaps figure out how to use networking components.

Then maybe figure out how to make online multiplayer games.

And after a few years when you are comfortable with it all, make your game for real.




PARTNERS