Jump to content
  • Advertisement
Sign in to follow this  
kristenrogers.kr75

Pokémon GO Style Augmented Reality With Vuforia

This topic is 659 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

79_pokemon-go-logo.jpg

 

1. Introduction

 

Thanks to the wild success of Pokémon GO, augmented reality (AR) is getting a lot of attention in the mobile development world. The idea of augmented reality isn’t new, but only now is the technology finally getting close to maturity and commercial viability. But what is augmented reality exactly?

 

This a really broad concept that can take many different forms. The idea can be applied to entertainment, science, military, education, you name it. What these applications all have in common, though, is a real-time connection between the physical world and the digital.  

 

Concretely, AR is often realized in interactive experiences that overlay a camera feed with 2D or 3D objects, informed by some sensor data. 

 

However, creating an engaging augmented reality experience from scratch can be a real challenge. Fortunately, there are some solutions available that can make this job easier. One of the most solid augmented reality tools out there is Vuforia, an AR library that is compatible with Android, iOS, UWP, and some brands of smart glasses. 

In this tutorial we'll explore augmented reality using Vuforia. We'll explore Vuforia's features, possibilities, and challenges, and we'll also take a look at  how the SDK works and its main functions. We won't explore any code in this tutorial—future tutorials in this series will dig into coding in Vuforia, with step-by-step guides targeting all the library's main features.

 

2. Vuforia

 

Originally developed by Qualcomm and recently bought by PTC, Vuforia is on its sixth version. Its SDK is constantly evolving and is compatible with Android, iOS, UWP, and Unity. Unfortunately, Vuforia isn't open source, but its price range is reasonable, and there is no up-front cost for development or education. You can create almost any kind of AR experience on the most popular mobile platforms using Vuforia

 

2.1. How Does It Work?

 

Vuforia uses the device's camera feed combined with accelerometer and gyroscope data to examine the world. Vuforia uses computer vision to understand what it 'sees' on the camera to create a model of the environment. After processing the data, the system can roughly locate itself in the world, knowing its coordinates: where is up, down, left, right, and so on. What you do with this depends on your development goals.

There are multiple possibilities:

  • Using World coordinates: Any kind of object can be exhibited using some arbitrary reference, ignoring real obstacles but considering the World localization obtained. The object can be out of view and continue to "exist" in the same position. This is a system similar to the one used by Pokémon GO.
  • Recognizing targets: Vuforia can look for specific images as "targets". VuMarks, for example, a QR code-style image highly recognizable by the system, can be used as anchor points or references for any kind of object projected into the world. A magazine could be 'expanded' with this resource, with its pages opening rich interactions like videos, audio, and so on.
  • Recognizing simple objects:  Simple objects like boxes and cylinders can be recognized and used as anchor points. This method is useful for recognizing packages and making them interactive.
  • Recognizing complex objects: Vuforia can also look for complex objects, like toys, computer parts, gadgets, and so on. Those objects are previously scanned, conforming to specific requirements, and can later be recognized by the system. This functionality could be used, for example, to turn toys into live creatures, or to create interactive assistance for mechanics or service people.
  • Looking for words: English words can also be understood by Vuforia. The system can look for specific words and target some kind of interaction on them. This could be useful for learning tools for children or for language translation.
  • Recognizing the world's terrain: One of the most powerful features available on Vuforia is the ability to recognize the world as it is. Vuforia lets the user scan their surroundings and can do some hardcore processing to interpret the world by creating a 3D computer vision of the real world and its objects. Imagine a Pokémon GO-like game where the Pokémons can hide behind real objects. Wouldn't that be awesome?

2.2. Developing With Vuforia

 

Vuforia is compatible with multiple systems. Its software development kits (SDKs) are available for Android, iOS, UWP, and Unity. Once you download and install the SDK, you'll need to sign up for a developer account and create an app key before you can start to code.

Even though Vuforia is compatible with many systems, the easiest way to create engaging AR experiences using its SDK is definitely using Unity. Don't get me wrong, you can access almost everything that Vuforia offers when developing directly for Android or iOS. However, the process is much simpler when done using the prefabs offered on Vuforia's SDK for Unity.

 

3. Vuforia Targets

 

As mentioned before, Vuforia can seek 'targets' on the camera feed to create anchor points or references to be used by AR experiences. Those targets can assume many forms, and any file that has been previously processed by Vuforia target manager can be recognized. Targets can be created by the developer or created during execution by the user.

 

3.1. Image Targets

 

Any kind of image can be a Vuforia Image Target. However, the more detailed and intricate the image, the better it is to be recognized by the algorithm. A lot of factors will be part of the recognizing calculation, but basically the image must have a reasonable level of contrast, resolution, and distinguishing elements. A blue sky photograph wouldn't work very well, but a picture of some grass would work gracefully. Image Targets can be shipped with the application, created later and uploaded to the application through a cloud system, or directly created on the app by the user.

image-target.jpg

 

3.2. VuMarks

 

VuMark works very similarly to a QR Code; however, it's much more versatile. It can assume many shapes and colors and adapt to a great number of environments. Its biggest advantage is that it's in total conformity with Vuforia's recognizing algorithm and will be easily 'found' by any Vuforia AR application. Vuforia also offers an Adobe Illustrator plugin that creates VuMarks.

 

3.3. Cylinder and Cube Targets

 

It's also possible to use cubes and cylinders as Image Targets. This is very appropriate to create engagement with product packages. Once recognized, the primitive object is used as an anchor point, and it seems to exist for the system's computer vision, allowing virtual objects to interact directly with the primitive. The virtual object could track the real object's position and orientation, for example.

 

objs.jpg

 

3.4. Objects as Targets

 

One of the most interesting features of Vuforia is the possibility to recognize complex objects. A toy, a phone, a computer board, and other kinds of objects can be used as targets. 

To use this resource, you have to scan the object using the Vuforia Object Scanner tool, which runs only on Samsung Galaxy S6 and S7. The data scan is uploaded to Vuforia Target Manager, and the scanned object will be recognized by any compatible device using that data.

3.5. Managing Targets

There are multiple ways to create a target: 

  • The application can ship with a series of targets embedded.
  • The app can receive targets online through the cloud system offered by Vuforia.
  • The user themselves can use the device's camera to create a target. 

As a developer, you don't have much control about how the user will create their target; all that you can do is provide the user with some tips about how to chose an appropriate image as a target. However, the situation is different for the targets processed using the Target Manager.target-manager2.jpg

Vuforia's developer portal provides us with the Target Manager tool to help manage all our apps' targets. The manager is very simple to use: you create a database to hold the targets, upload a file that corresponds to a target, and wait until the file is processed. The target will receive a unique ID and a score related to how recognizable it is and will then become part of the database, available for download or for storage in the cloud. 

There are three different types of target databases:

  • Device Databases are local databases of images or object targets that are stored on the user's device.
  • VuMark Databases are local databases of VuMarks that are stored on the user's device.
  • Cloud Databases are databases of Image Targets stored online and queried over the internet.

4. Smart Terrain

 

When developing a Vuforia app on Unity, there is a really exciting option available, the Smart Terrain. It allows an application to replicate 3D meshes of objects seen by the camera. 

The process works like some kind of scan, where the user uses the device's camera as a 3D scanner. As the camera scans through the ambient environment, the application creates 3D models of the recognized objects, giving the application the possibility to adjust the scenario to the real world, creating a deeply engaging experience.

 

5. Conclusion

 

There are hundreds of possibilities to explore with augmented reality, and we're just starting to scratch its surface. Many believe that AR will be a part of our future and that we'll use it on a day-to-day basis. This is a field that promises to grow a lot in the coming years, and Vuforia provides us with cool tools to create engaging experiences.

 

Did you find this article helpful? Don’t forget to drop your feedback in the comments section below.
Suggest for you:
 
 
 
Edited by kristenrogers.kr75

Share this post


Link to post
Share on other sites
Advertisement

If you had been a member of this community you would have realized this is great content for an article, now it's just a annoying post that will be lost by the time I am free to read it. :(

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!