AUG|RE

It’s interesting to see the discussion around the Google...

Fri, 06 Apr 2012 17:50:40 +0300

It’s interesting to see the discussion around the Google heads-up display. The most obvious thing to point out is that so far the UI they present isn’t augmented reality. There’s nothing there that really “blends” or “mixes” into the environment - everything obscures. Whether for pragmatic or stylistic reasons some of the easier possible examples of aligning the data with the view have been omitted: routes aren’t shown projected on the ground, and POIs aren’t rendered seamlessly on the objects.

Personally I don’t think a video like this is going to sell anyone on the idea of head mounted displays. HUDs offer a real potential for magic. They have the ability to reveal a world quite similar to our own, but populated everywhere with figments of the imaginations of artists and thinkers. I always thought if people started wearing HUDs you’d look at strangers and think “I wonder what he’s seeing”. I hope it’s more than calendar reminders and routing information.

http://www.physorg.com/news/2012-03-birds-evolved-compass.html

Mon, 19 Mar 2012 08:19:55 +0200

http://www.physorg.com/news/2012-03-birds-evolved-compass.html:

Birds literally have augmented vision.

This is wonderful video [from Timo Arnall at Berg London] but I...

Mon, 06 Feb 2012 22:52:48 +0200

This is wonderful video [from Timo Arnall at Berg London] but I think seeing all of that computer vision at once makes it easy for us to project a bit too much facility on the robots and their vision.

Before we get too overwhelmed by the abilities of modern computer vision it’s worth remembering that everything without overlaid graphics is functionally invisible to those specific algorithms, and what is visible has zero higher perceptual understanding. A pattern has been matched, no more.

[I’m going to put a disclaimer here that I took CogSci 101 a long time ago, and then jumped right in to trying to read EEG, so my understanding is both shallow and beyond its shelf-life and in need of refresh]

There are a host of disorders in humans categorized as “agnosia” or ‘not knowing’. Most of what we know about our own visual system, like much else of what we know about our own cognition, comes mostly from examining cases like these where it has gone wrong. [This has a certain morbid fascination for me: somewhere out there are cognitive psychologists hoping against hope for someone in their vicinity to come down with a slightly novel stroke.]

One of the sub-disorders is “associative agnosia”, which is where someone who can see perfectly well has difficulty identifying objects visually (though allowing them to hold the object, or telling them the name of the object might trigger instant recognition). There is no semantic understanding of what is viewed.

Another is ‘integrative agnosia”, where sufferers cannot recognize things at once, but must piece together an object by considering the recognizable parts.

Then there are cases of category specific agnosia, where those afflicted have extreme difficulty recognizing something like animals, but might be quite good at recognizing man-made objects. This seems bizarre until we consider we have additional mental models of many man-made objects due to our interactions. I can’t say I’ve had many interactions with most of the animal kingdom (honest, guv’ner).

The thing to remember is robots suffer from all of these in particular, and agnosia in general: they don’t know what they’re looking at, and that makes it incredible difficult to program them to look for it reliably.

In the grand scheme of things, we’re really just nibbling on the very edges of our own perceptual capabilities, from many fields all at once. I tend to think that solving any particular field of robotics like computer vision eventually expands until it’s an AI-complete task - that really solving computer vision requires solving artificial intelligence. There’s a feedback loop between the knowing and the seeing that just isn’t there.

After reading Paintwork I’m reminded of and sensitive to...

Sun, 29 Jan 2012 23:31:17 +0200

After reading Paintwork I’m reminded of and sensitive to the fact that people respond to the mastery-of-the-thing inherent in analog, but damn, I would like to see this in AR.

Art that might be ARt

Tue, 15 Nov 2011 17:14:28 +0200

I’m always keeping an eye out for pieces that either might already be AR, or would translate very well into AR. Two artists caught my eye recently.

The first is illustrator and artist Anna Emelia, whose work is very cosy and intimate which might translate well into the “doll house” perception of desktop hand-held AR for instance. She recently submitted this image on her blog, which I like very much.

The second is German street artist Tasso. Below is just one of many pieces he has that might translate very well into AR. How terrifying, for instance, might this one be, with a bit of interactivity and some sound:

ScandAR 2011

Thu, 10 Nov 2011 09:46:50 +0200

ScandAR 2011:

If you work, study or research AR in the Nordics check out the Call for Participation for the Second Annual Scandinavian Workshop on Augmented Reality, which will be held at Aalborg University, Dec 8th and 9th.

Nice projective AR art piece. What appeals to me about...

Wed, 09 Nov 2011 00:52:40 +0200

Nice projective AR art piece. What appeals to me about projective AR is it lends itself more naturally to shared experience. It’s a really interesting area in that it’s the intersection of hardware, software and sculpture - this piece is presented as a Light Sculpture. I hope that sort of naming takes off, because the research community nomenclature is starting to feel like an impediment.

Nokia City Scene is now publicly available

Tue, 25 Oct 2011 23:24:53 +0300

Nokia City Scene is now publicly available:

The first public release of our mirror worlds work comes out just in time for ISMAR 11! Right now it’s Nokia N9 only, and we only have a handful of cities covered, but if you have an N9 get it! I’d be especially interested in hearing what people who follow this blog think.

What isn’t presented to the general public is our overall idea of how this fits into the mixed and augmented reality space, I’ll try to get around to that soon, but right now I need to finish my slide deck for our workshop tomorrow.

Nokia Maps 3D using WebGL

Fri, 21 Oct 2011 14:43:18 +0300

Nokia Maps 3D using WebGL:

Our colleagues in Berlin have been doing amazing things with the same sort of data we’ve been using to build our mobile mirror world applications - really nice work guys.

Some updates on Suwappu from Dentsu London. Interesting to note...

Fri, 07 Oct 2011 16:48:14 +0300

Some updates on Suwappu from Dentsu London. Interesting to note that they redesigned the characters faces slightly to get better features for tracking. This may become an important aspect of character design in future - having to run your character through feature detector algorithms to make sure it can be used with AR apps and toys.

Part of an artist’s manifesto from the East Side Gallery...

Wed, 21 Sep 2011 00:32:00 +0300

Part of an artist’s manifesto from the East Side Gallery on the Berlin Wall. Struck a chord.

Enabling Large-Scale Outdoor Mixed Reality and Augmented Reality ISMAR workshop

Mon, 08 Aug 2011 14:27:11 +0300

Enabling Large-Scale Outdoor Mixed Reality and Augmented Reality ISMAR workshop:

I’m very excited to be involved co-organizing a workshop at ISMAR this year on enabling large scale MAR. If you’re interested in participating, the research topics include (but are not restricted to):

3D geo-referenced data (images, point clouds, and models)

Algorithms for object recognition from large databases of geo-referenced data

Algorithms for object tracking in outdoor environment

Multi-cue fusion to achieve improved performance of object detection and tracking

Novel representation schemes to facilitate large-scale content distribution

3D reasoning to support intelligent augmentation

Novel and improved mobile capabilities for data capture (device sensors), processing, and display

Applications, experiences, and user interface techniques.

The organizing and program committees are stellar, it’s a real honour to be working with this group, and I’m looking forward to the event on September 26th immensely. Special thanks to Matei Stroila of Navteq Research for putting together the site and the call for submissions while most of us were on vacation!

Can you imagine what you could do with see through and/or...

Wed, 27 Jul 2011 10:11:00 +0300

Can you imagine what you could do with see through and/or projected AR added to the mix? It would be the whisical, funtime opposite of Daniel Suarez’ icy Daemon.

There’s a couple of things to note here: this is an “mp3 experiment” - but the mp3 really only serves to initiate, synchronize, and contextualize via the voice and music. AR could work like this - detractors confuse wanting to augment things with more or less wanting to replace all other kinds of stimulation, and that’s not necessarily in the best interest of the user.

Then there’s the element of “being in on it” while perplexing the general public that’s part of the fun of flashmobs, that actually plays to the user-exclusivity of portable AR presentation.

The key thing, thought, is the motivation in the disembodied introduction - “follow my instructions and we’ll all have a pleasant time together” - this is about having a remarkable shared experience that really depends on careful planning and scripting by the organizers, and preparation and willingness from the participants, rather than the technology which is really only a facilitator.

"Fluent, convivial satiation of as many senses as possible"

Mon, 04 Jul 2011 15:17:53 +0300

“Fluent, convivial satiation of as many senses as possible”

- A good friend’s brilliant answer to “what makes a good meal” - but damn if that isn’t the perfect answer to “what would make a good augmented experience”.

AR Books

Tue, 28 Jun 2011 00:40:18 +0300

As an example of how fast things are moving in AR consider augmented books. In 2008 I was blown away by “Haunted Book” by Camille Sherrer of ECAL and the ever amazing EPFL CV lab. I can’t seem to find a video of that work, but “Magic Book” by the same artist looks similar:

This ran on a laptop, and the graphics are strictly flat sprites. The natural feature tracking was ground-breaking in 2008 - even recent commercial works such as the rather nice DibiDogs book prefer markers.

The augmentations on the laptop / book in hands dichotomy makes for awkward interaction, and should recognized for what it is: a limitation imposed by incapable mobile devices, rather than a design decision (though the kids are clearly unfased).

Three years on from “Haunted Book”, Helen Papagiannis has a new augmented pop-up book, with interactive fully 3D models, natural feature tracking running on iOS mobile devices.

The tracking isn’t a research project either, but off-the-shelf from Metaio. It feels like the technology is just maturing to the point where artists and creators can get on with what they do without the technology being a massive consideration, or getting in the way too much.

As with most technologies, the mechanisms for reading and consuming are maturing faster than those for writing and creating. The next challenge is to make creating AR works as easy as consuming this piece. Maybe we’ll be there when we have an AR sketch book (if that even makes any sense).

I have a copy of the DibiDogs book, and I hope I can get a copy of “Who’s afraid of Bugs” shipped to Finland. I’m waiting to try them out on my friends’ tech-addled kids, who weren’t even born when I first saw “Haunted Book” back in 2008.

Performance aids

Wed, 22 Jun 2011 00:08:34 +0300

I was at Sónar festival in Barcelona this weekend, and one of the things that stood out for me what how important visuals have become for live electronic music artists. It makes sense electronic acts would lead the way here, both because of familiarity with technology, and a certain boring guy-with-laptop sameness of performance that doesn’t create energy like a live rock-band can. The visuals are a performance aid.

This is the sound/visual control island at the night venue’s main stage - notable because it’s as deep as it is wide, with four rows of computers and technicians, and only one of the four rows seems to deal with sound, while the other three layers of technology all control lights, screens, lasers and other visuals. Visuals require three times as many resources as sound engineering at a music festival. One act was delayed ten minutes purely because his windows laptop wouldn’t play nice with the projectors - lack of visuals was a sufficient impediment to performance to hold up the show.

There were a couple of different styles of visuals seeming (to me) to be trying to be volumetric:

a) The coloured lasers seemed to be trying to “carve out” visual spaces and shapes - certainly (they aren’t making any discernable pattern at the endpoint of the beam, seen from about 11 seconds into the video), and they aren’t doing the usual laser-show trick of projecting things like grids on smoke.

b) Straight-ahead projection onto a 3D shape. The mesh-screen in conjunction with the back-screen make it feel like the band are performing in the middle of a textured cube.

c) This is harder to describe, but the singer was using a strobe as if to reach out and “touch” the audience.

It felt like, if given working projective AR tech at low installation costs, many artists would jump at the chance for a more immersive show. Certainly I see Kinect-ish technology having an impact here in coming years. The nice thing about AR from a festival-artist’s perspective is you can have a “set” without necessarily bringing and installing it. Cell-phone or HMD based displays won’t cut it though; this needs to be something the whole audience can share.

There's no such thing as "Virtual Space"

Mon, 13 Jun 2011 17:06:43 +0300

tl;dr: “Virtual space” is a poor metaphor, we shouldn’t take it too literally, especially when hypothesizing about the legal ramifications of putting things “in it”.

The topic of who owns virtual space (in front of existing advertising billboards outdoors, in museums and other curated spaces) comes up often in discussions about AR and its implications, and for me this is a symptom of people taking AR metaphors too literally. When we imagine physical spaces as if they have a single contiguous virtual volume, we can end up projecting the limitations of physical space onto digital data. This is unnecessary, inappropriate, and leads to confusion.

What’s actually happening with (visual) AR? Well, we have a view on the world, and we have some geo-spatial data comprising a thing to render, an understanding of how to render it, and metadata describing where in a coordinate frame it is addressed. At some point, we composite a suitable rendering of that data with a view of the real environment (our actual view, a video stream, etc). Essentially we are using a spatial descriptor, be it a pose reading from sensors, or feature points from a camera image, to generate an address in a data representation scheme, and we’re fetching data proximal to that address in a given database. We may “align” the rendering with the view, but this happens in 2D at the rendering stage.

The geo-spatial data is infinite: we can create any number of database instances that will return any number of different data objects for the same coordinate. Equally, creating an geo-located AR 3D model doesn’t “use-up” “virtual space” that could be occupied by someone else, and you can have multiple objects associated with a given coordinate. Understanding this makes the idea of geo-spatial digital property claims ridiculous (that over 2000 potentially coincident AR layers happily coexist simultaneously on the same platform drives home the point). So even if we hang on to “virtual space”, even just as a metaphor, “virtual spatial layers” is better - it emphasizes the multiplicity (even within platforms).

Examining one of the excellent hypothetical questions:

So what would happen if someone painted an AR beard on the Mona Lisa

The beard would reside not in the space between you and the Mona List, but as a data entry in a particular database, run by a particular service, viewable with a particular application (today). Nothing happens, but it becomes possible to find and view the AR beard if so desired. The process of rendering the beard happens entirely on your device (whatever it may be). And as discussed previously, the viewing is inherently opt-in.

Considered like this, the legal issues look more likely to surround copyright and the combination of the virtual overlay and the overlaid taken together as comprising a derivative work than geospatial positioning. Copyright issues are likely to be irrelevant in public commons, and in cases where there is no copyrighted object being derived from (e.g. if you place your AR piece in an empty piece of museum). (Of course, IANAL).

So, AR metaphors shouldn’t be taken too literally, and it would be much better to promote non-metaphorical thinking about AR when considering legal ramifications even hypothetically, to try to limit the scope to well-defined issues like copyright and derivative works, rather than to promote “virtual space” thinking which is liable to cloud the issues and potentially lead to much more restrictive and insane legislation like taking ownership of ranges of addressing schemes that correspond to physical volumes, or partitioning ranges to different AR vendors etc.

That said I’d really like to get other people’s take on this.

La Macular by SUPERBIEN (by SOHfestival) More (abstract)...

Mon, 06 Jun 2011 15:03:28 +0300

La Macular by SUPERBIEN (by SOHfestival)

More (abstract) projective AR. The Sydney Opera house makes for a very striking canvas.

MVI_7513-desktop by thesystemis on Flickr.(IR, Kinect based)...

Mon, 30 May 2011 15:50:30 +0300

MVI_7513-desktop by thesystemis on Flickr.

(IR, Kinect based) Face tracking, plus projection AR. Ubiquitous projection in performance based or experiential spaces (night-clubs, concerts etc) is something to start considering - your device can hand over the augmenting to the environment where available - especially interesting for self-augmenting.

I’m plugging away at a City Scene demo right now, but...

Tue, 26 Apr 2011 21:37:46 +0300

I’m plugging away at a City Scene demo right now, but I’d rather throw it all away and start figuring out how to make AR art with a similar feeling to that shown: intimate, precisely located, and cute. Definitely leaving some physical props in there.

via @vuokko