The first public release of our mirror worlds work comes out just in time for ISMAR 11! Right now it’s Nokia N9 only, and we only have a handful of cities covered, but if you have an N9 get it! I’d be especially interested in hearing what people who follow this blog think.
What isn’t presented to the general public is our overall idea of how this fits into the mixed and augmented reality space, I’ll try to get around to that soon, but right now I need to finish my slide deck for our workshop tomorrow.
I’m very excited to be involved co-organizing a workshop at ISMAR this year on enabling large scale MAR. If you’re interested in participating, the research topics include (but are not restricted to):
3D geo-referenced data (images, point clouds, and models)
Algorithms for object recognition from large databases of geo-referenced data
Algorithms for object tracking in outdoor environment
Multi-cue fusion to achieve improved performance of object detection and tracking
Novel representation schemes to facilitate large-scale content distribution
3D reasoning to support intelligent augmentation
Novel and improved mobile capabilities for data capture (device sensors), processing, and display
Applications, experiences, and user interface techniques.
The organizing and program committees are stellar, it’s a real honour to be working with this group, and I’m looking forward to the event on September 26th immensely. Special thanks to Matei Stroila of Navteq Research for putting together the site and the call for submissions while most of us were on vacation!
After being hown in public at Nokia World and presented at ISMAR (winning best-demo, which is very gratifying - good work Petros!) I guess I can share a bit about what our team at Nokia Research Center has been working on, and why I think it’s significant for mixed and augmented reality right now. Unfortunately we don’t have our own shiny video just yet, so here’s one courtesy of mynokiablog.com.
Nokia City Scene is a mobile mixed-reality service (running on the Nokia N900) for viewing panoramic street imagery on mobile devices. It’s quite a lot like Google Streetview, and in addition it has full 3D understanding of the environment - our pipeline and backend teams have integrated 3D building models, terrain data with the panoramas, so we can tell if you’ve clicked on a building (and which building), on the ground, or in empty space, and respond accordingly. Building models also allow us to place content on and correctly aligned with the building facade, no matter which panorama you view the content from. Similarly, we can compute a visibility model for any panorama, determine occlusion etc.
One interesting thing about these kinds of panoramic mirror worlds is they allow AR style interaction locally without having to hold your phone in front of your face, and even more interestingly they allow AR style interaction remotely. If AR takes off in a big way, a certain set of interactions will become common, and people will expect to be able to interact with locations with those interactions, whether those places are their current location, or remote.
While the tracking problem is insufficiently solved to allow really good precise tracking in outdoor environments, using a mirror world instead of AR for city-based outdoor use skirts two problems: inaccurate content placement by users, and inaccurate positioning of content for user consumption. It also allows users to avoid the “pointing the camera” style interaction, which some people find uncomfortable in public spaces.
Finally I suspect that a general tracking solution will be built upon these kinds of complete, 3D city models, so it’s exciting to anticipate that this might be the foundations for a general urban tracking solution.
