I was playing snooker yesterday as a total beginner and am trying to learn where exactly I should be aiming the cue ball. I was struck by the idea that if you could build an application which knew where all the balls were on the table, it could tell you where to aim and show this in a simple 3D world.

Then I thought, could you use the iPad camera to let your app automatically figure out where the balls are after each shot, so you merely tell it which ball yo want to send in to the pocket and it tells you where to aim?

Then I thought, could you use augmented reality and the other camera so it can record your shot and show you what you did wrong... or even detect the cue position before you take the shot and give you directions to line up perfectly... put the ipad on the table opposite where you are standing so you can see the screen as you prepare your shot.

You see all these types of things in mobile games these days but the accuracy seems bad - augmented reality apps tend to shudder around like the position tracking is pretty lousy.

Anyone like to weigh in whether this is likely to be achievable... or which parts are achievable, which parts are likely to be very hard or impossible?

I have actually seen a student project that did exactly this about 10 years ago. Obviously it wasn't using an iPad (they didn't exist) it was using a couple of PCs, about half a million pounds worth of camera equipment and lasers overlaid on top of the snooker tables to show the correct angles to hold the queue and to strike the ball.

I think there was also an iPhone app shown on some show in the UK where they had Jeremy Clarkson and Stephen Fry playing pool and they used an augmented reality app on an iPhone. The problem was that you had to actually clip the phone to the cue using a gorilla pod and also have an ipad on a stand at the other end of the table.

You probably can do what you are asking with an iPad in conjunction with another device, possibly a kinect.

But letting it tell you that your aim is off…won’t people notice the guy behind you holding up your iPhone and him telling you to correct yourself left or right?

For this to work as I have in mind, it would have to require only an iPad (or iPhone) and not clipping your phone to your cue. Lugging a laptop with Kinect to your local club doesn't sound practical to me!

I can look at the table from just about any angle and understand where all the balls are but I have no idea how hard this kind of image recognition is... as I say it's a nice closed system so I was rather hoping I could take a single photo and work out ball positions - or put the ipad opposite me on the cushion and it could automatically recognise what was going on.