After being hown in public at Nokia World and presented at ISMAR (winning best-demo, which is very gratifying - good work Petros!) I guess I can share a bit about what our team at Nokia Research Center has been working on, and why I think it’s significant for mixed and augmented reality right now. Unfortunately we don’t have our own shiny video just yet, so here’s one courtesy of mynokiablog.com.
Nokia City Scene is a mobile mixed-reality service (running on the Nokia N900) for viewing panoramic street imagery on mobile devices. It’s quite a lot like Google Streetview, and in addition it has full 3D understanding of the environment - our pipeline and backend teams have integrated 3D building models, terrain data with the panoramas, so we can tell if you’ve clicked on a building (and which building), on the ground, or in empty space, and respond accordingly. Building models also allow us to place content on and correctly aligned with the building facade, no matter which panorama you view the content from. Similarly, we can compute a visibility model for any panorama, determine occlusion etc.
One interesting thing about these kinds of panoramic mirror worlds is they allow AR style interaction locally without having to hold your phone in front of your face, and even more interestingly they allow AR style interaction remotely. If AR takes off in a big way, a certain set of interactions will become common, and people will expect to be able to interact with locations with those interactions, whether those places are their current location, or remote.
While the tracking problem is insufficiently solved to allow really good precise tracking in outdoor environments, using a mirror world instead of AR for city-based outdoor use skirts two problems: inaccurate content placement by users, and inaccurate positioning of content for user consumption. It also allows users to avoid the “pointing the camera” style interaction, which some people find uncomfortable in public spaces.
Finally I suspect that a general tracking solution will be built upon these kinds of complete, 3D city models, so it’s exciting to anticipate that this might be the foundations for a general urban tracking solution.
