标签: augmented

As you are surely aware (unless you've been living under a rock), I am enamoured with things like Augmented Reality and Virtual Reality (in the form of games like Obduction and headsets like the Oculus Rift ).   Well, I just received a message from my chum Rich Quinnell, who is the Editor in Chief over at IoT World. In his email, Rich said "Hi Max, I thought you might like to take a look at this Panopticam 360-degree camera ."     This little beauty -- which is the work of a company called Figure Digital -- is a spherical device adorned with 36 cameras. Using this little scamp, you could take totally immersive "surround" images that could be viewed using the Oculus Rift.   Suppose we take this one step further -- shoot an entire film using a Panopticam and then watch it using an Oculus Rift. In this case, you could be standing in the middle of the room tilting your head up and down or turning it left or right (or turning your body completely around) to see something completely different. In fact, you could watch the same film over and over again looking in different directions each time.   Actually, knowing how modern films require so much in the way of special effects and large numbers of lighting and sound technicians and so forth, I'm not sure how well this would work for something like an action movie. On the other hand, I can really envisage the most amazing educational videos taking you to places like the Great Pyramid in Egypt. And I can also envisage the most amazing computer games...   How about you? What do you think would be a good use for the Panopticam?

In my recent blog on this topic, we discussed some of the delivery mechanisms that might be used to present us with augmented reality information. The next question, of course, is what sort of augmented reality information do we expect and/or want to be delivered?   The impression I have after talking to a number of people is that they expect augmented reality data to be reasonably benign. They also expect it to be a "pull" type model in which they will have the ability to decide just how much information comes their way. The sort of thing a lot of people think of when you say "augmented reality" is looking at the world through their smartphone in a supermarket and being alerted to an interesting sale, for example.     Another popular theme is to be able to look through one's handheld device at something like a restaurant, and to then be provided with information like today's menu, dining hours, customer reviews, and so forth.     For myself, I'm looking a bit farther out. Take navigation, for example. Today's GPS systems that you get in cars are, in many ways, incredible, but it's still a pain having to look down at a small screen, and the spoken directions are often not as clear as one might like ("Stay right and then turn left"). Even "Turn right in 250 yards" may not be optimum if you can see multiple right turns in close proximity to each other.   There's also the fact that a lot of graphical presentations try too hard and can overwhelm the user with too much information. On the other hand, a simple, well-conceived graphic truly can be worth 1,000 words.     Now, there are some very interesting heads-up-display technologies around, but I'm thinking beyond them. I'm not sure what delivery mechanisms will ultimately become available to us (from Google Glass to fully immersive Oculus Rift-type headsets to active contact lenses and beyond). Whatever the delivery mechanism turns out to be, I am envisaging something that we don’t have to hold in our hands, that can overlay graphical data anywhere in our visual field (we'll leave things like audio, tactile, and olfactory versions of augmented reality as topics for another day), and that is always-on (we'll return to this last point in a while).   The really interesting thing to me is the sort of augmented reality information that might be made available to us. The following are a few ideas off the top of my head -- I would be very interested to hear your thoughts on these, along with other ideas you have of your own.   Let's start with a simple one. Suppose you are in a foreign country and you look at an indecipherable sign. Suppose your augmented reality system is aware of the languages you know and don’t know, and automatically performs the translation and superimposes it over the original. Wouldn’t that be cool?   In the case of navigation, we typically think of an automotive experience as depicted above, but I've spent a lot of time on foot wandering around strange cities. Let's assume that I've given my augmented reality system the name "Maximillian" and that responds to (only my) spoken commands. I can envisage exiting a hotel in San Francisco, saying something like "Maximillian, show me the way to Fisherman's Wharf," and having a faint dotted line or arrows or whatever be unobtrusively overlaid on the scene in front of me.   When I was a student living in Sheffield, England, all of my friends would be calling each other early in the evening to decide where we would be going that night. This was before the days of cellphones, so once a plan had been made, everyone tended to stick to it (unlike my son and his friends, whose plans seem to undergo a continuous morphing process). Once one was out and about, there was no way to know where everyone else was.   Now consider an augmented reality scenario. Suppose I could see recent virtual tracks (lines or arrows hovering just above the ground) left by my friends. These tracks could be color-coded for each friend, and -- by default -- could fade away over time. Alternatively, as I passed a bar, for example, some form of a visual indication could inform me if anyone I knew was in there. Or perhaps I could call up a bird's eye view of the immediate area to see where all of my friends were currently located. (Note that many of these capabilities are already available as smartphone apps -- I'm simply extrapolating them to an augmented reality delivery system.)   Apropos of nothing at all, have you seen the latest quadcopter drone from Parrot? It's called the Bebop (who can comment first as to why this name has a special significance to me?). Check out this BBC video . It's well worth watching the whole thing, but an especially interesting part starts around the 1'34" mark when the operator uses an Oculus Rift to see the world through the eyes of the drone. Judging by his exclamations this is pretty impressive. I would really like to see what he is seeing. But we digress… Suppose you were in the market for a new house. It's a Saturday morning and you are walking or driving around a new subdivision. Can you imagine an augmented reality scenario whereby details of the houses that are available for sale appear superimposed over them, including things like price, size, number of rooms, and special features? How about being able to quickly access additional information like the builder's reputation with the better business bureau and the quality of the local schools?   This all sounds quite benign and useful, doesn’t it? But now suppose that by simply looking at the neighboring houses that had already been sold, you could access details about their occupants. How much did they pay for their houses, for example? How many people live there? How many children are there (and what ages)? Does the home owner's association have liens on any of the properties? Where do the occupants work? Have any of them been convicted of any crimes?   I have a terrible head for names and faces. Things get really bad if it's someone I used to work with at a company 20 years ago, or perhaps someone I met at a conference a few years ago. At official functions, people like the Queen of England have assistants who whisper the names of the people she's about to meet into her ear. I would love for my Maximillian augmented reality system to be able to do the same thing.   As I'm walking through the mall, for example, its facial recognition software could identify people with whom I am in some way acquainted and indicate them to me (perhaps using flashing arrows over their heads -- maybe using different colors to indicate different levels of acquaintanceship). As we approached Maximillian could superimpose additional information on the scene (or whisper nuggets of knowledge in my ear) as to the person's name along with the names of their partner and children and so forth. It would be especially useful if Maximillian reminded me that we used to work at Company XYZ 20 years ago. Maybe Maximillian could access LinkedIn to see where my acquaintance works now. Perhaps Maximillian could also check Facebook and Twitter and alert me to any additional information that may be of interest, such as the fact that it's my acquaintance's birthday, or the fact he just got divorced and 30 seconds ago he Tweeted that he's going to beat up on the next person he sees.   All of this is a double-edged sword, of course. If you can access information about other people, then presumably they have the same ability. How much information would you be comfortable sharing with the world? Do you want everyone to know how much you ate for lunch, or how many martinis you quaffed for breakfast? If you have young children, it would be advantageous to you if any registered sex offenders appeared with a big red warning mark flashing over their heads. Imagine being able to tell your kids to run away if they ever see this sign. Of course, I'm sure the registered sex offenders would be somewhat less enthusiastic about all of this.   Another thing to consider is a "push" model in which information is pushed at you via your augmented reality system. Suppose you are walking past a store, for example, and they can detect who you are and determine your name, age, sex, and so forth. Maybe they can also access your personal interests and preferences based on your past purchasing experiences. It wouldn’t take much effort to discover that I'm partial to Hawaiian shirts or that I have been known to quaff the occasional beer, for example. Now suppose that the store can start projecting visual and audible adverts into your augmented reality system. On the bright side, I already hear voices whispering "You really deserve to own that flashing LED thingy, whatever it is" when I pass an electronics store, at least now I would know that the voices were real.   It seems to me that if it becomes possible for all of this information to be made accessible to the world, then we are also going to have to be able to filter what we see and what we allow to be seen. If we want our friends to be able to locate us, then we allow our systems to report on our location, but we also need to have the ability to drop below the radar, as it were.   I have so many thoughts bubbling around in my head at the moment that I don't know where to go next. Maybe this would be a good time to throw the floor open for everyone else to share their thoughts on this topic. What do you think? How much of what I've talked about here do you consider to be plausible? How much do you consider to be inevitable? And what sort of timeline do you think we are looking at here? As always, I welcome your comments and questions.

I have recently been thinking about the future of augmented reality (AR). I really believe this technology is racing toward us faster than most people think.   Just today, for example, my chum Jay Dowling sent me a link to something called iOnRoad , which turns your smartphone into a personal driving assistant that can tell you what you are doing wrong. That will be handy for those times when my wife can't be there to perform this task for me.     This comes as an app that you can download to your iPhone or Android phone. Using the phone's camera, augmented by machine vision algorithms, the iOnRoad app provides a range of personal driving assistance functions, including augmented driving, collision warning, and black box-like video recording.   As per my previous columns, suppose this sort of AR capability -- showing the time gap between you and the car in front -- didn't require a smartphone or a heads-up display on the windscreen. Suppose it were projected directly on to one's retina?   I don't know about you, but I think that having a wealth of AR data available at one's fingertips (or on one's eyeballs) could be an incredible experience. It seems like the sort of thing that you can't imagine when you haven't got it, and that you can't imagine living without once you have it.   When I started writing my first book in 1992, I spent just about every evening and weekend for a year at my local library using the hard-copy Encyclopedia Britannica as my main reference source to check facts like the year George Boole invented Boolean algebra and when he was born and died. This was before the first popular web browser, Mosaic, appeared on the scene and before the vast majority of folks had heard the term "Internet."   Today, we all have unbelievable amounts of information available at our fingertips. While watching House Hunters with my wife, for example, I will use Google Earth on my iPad to get additional information about where our hopeful house hunters are hunting. I would guess that no more than 10 minutes typically passes -- at work or at home -- without my consulting the Internet about something or other. Way back in 1887, in his autobiographical Souvenirs d'Enfance et de Jeunesse , Ernest Renan (1823-1892) famously noted: "The simplest schoolboy is now familiar with facts for which Archimedes would have sacrificed his life." Goodness only knows what Ernest would have said about the information available to us today.   And I think we have only seen the tip of the iceberg. I don't think most of us can conceive of the way things will be in the not-so-distant future.   I do have a question. What happens if we get used to relying on all this stuff, and a disaster like a mega-computer virus brings down all our systems? What will we do if the data stops flowing?

Have you seen the original Terminator movie starring Arnold Schwarzenegger portraying a cyborg sent from the future to kill someone in our time? On a number of occasions during the movie, we saw the world through the cyborg's eyes, with additional information being overlaid on the scene he was observing.   This was an early depiction of augmented reality, which refers to a live direct or indirect view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphical, and textural data.   About 12 years ago I saw some very interesting work in the augmented reality arena, including a pilot flying a plane with a computer screen in front of him instead of a window (his co-pilot did have a window and was poised to take over if there were any problems).   A photorealistic 3D model of the outside world was displayed on the screen in front of the pilot. This was augmented with imagery from a variety of radar systems (so if something flew in front of the plane, it would appear on the screen).   The interesting thing was that the pilot was attempting a real-world landing at a small airport in a tricky, mountainous terrain. Displayed on the screen in front of the pilot was a "corridor" of rectangles in the form of glowing lines that were superimposed on the scene. It looked like a computer game, with the pilot flying his real-world plane down the "corridor."   The idea was that, even at night or on a foggy day, the pilot could be presented with a scene that looked as though the world was in bright daylight. My understanding is that pilots consistently achieved better landings using this experimental technology than they did using more traditional approach techniques.   Of course, this was some time before the days of smartphones and tablets and us all having GPS-based location information at our fingertips. There are already a variety of forms of augmented reality that are commercially available, with many more on the way. Personally, I don’t think that many of us really have a clue as to how pervasive augmented reality has the potential to become.   Take a look at this YouTube video , for example. This depicts a variety of possible augmented reality scenarios in which a young lady looks at the world through the screen of her smartphone.   When she points it at the sky, she is presented with the immediate weather forecast, along with a simulation of raindrops falling on her screen. When she points the smartphone at an art gallery, she is informed as to current exhibits. Looking at a tram via the smartphone returns details as to destinations and schedules. Pointing the smartphone at a hotel provides an indication as to which rooms are currently occupied and which remain available.   What other sorts of augmented reality information could be presented to us? Well, let's return to that in a moment but -- before we go there -- let's first consider how this augmented reality information might be presented to us. One approach -- as we've already seen in the video above -- is to view the world through the screen of a smartphone or tablet computer.     Another alternative might be something along the lines of Google Glass . This has the advantage of being relatively unobtrusive, but it doesn’t really offer a fully immersive experience.     I think a lot of it depends on the quantity and quality of augmented data that is made available to us. If there were to be sufficient data, then it wouldn’t surprise me if -- sometime in the not-so-distant future -- we were to see people walking around in public sporting a full-up Oculus Rift -type display.     What? You think this couldn’t happen? Well, all I can say is that I bet when you saw the original Star Trek featuring Lieutenant Uhura with an oddly-shaped communicator module stuck in her ear, you would never have expected to actually see people strolling down the street with flashing Bluetooth earpieces.     As an aside, my 70-year-old mother-in-law has long hair. She also has a Bluetooth earpiece linked to her smartphone, and this system is set to automatically turn on if anyone calls. I had no idea she was so in tune with the times. We were strolling around a supermarket together when she suddenly commenced to have what seemed to be a one-way conversation with herself, arguing vehemently and giving herself orders. I honestly feared she had gone insane (it turned out she was talking with her administrative assistant; the jury's still out on the question of insanity). But we digress...   The advantage of wearing something like an Oculus Rift is that you do get a fully immersive experience. Also, you can change reality to suit your whims. It could be midnight in the real world, but your display could present the scene around you as though it were midday, or vice versa, of course. The downside is that it's somewhat bulky and you would tend to look like a bit of a plonker. On the other hand, once enough people are doing it, this could become the new reality (pun intended), in which case it wouldn’t be long before we were inundated with designer models.   What about the farther future? Well, there are already experiments going on to embed LEDs in contact lenses. I can imagine a time when contact lenses have the capability to project high-resolution textual and graphical imagery directly onto their owners' retinas. At some stage, it wouldn’t surprise me if it became possible to have such equipment embedded in the eye itself.   What do you think regarding the various technologies that might be used to present augmented reality information? Do you agree with my thoughts above? Do you vehemently disagree? Can you offer any other suggestions? Please share your thoughts in the comments below.   Also, earlier on in this column, I said "What other sorts of augmented reality information could be presented to us?" Don’t answer this now, because I'm going to post Part 2 of this miniseries tomorrow, but you could certainly start noodling on this question and we'll see if what you come up with matches my hopes and fears.

  Virtualization is a term that has taken some time to internalize for me. My introduction with virtualization was as far back as 1992 when Windows 3.1 introduced the 386 enhanced mode, and the ability to run multiple DOS sessions with each session being independent of each other but having a separate 640KB space available to each virtual DOS session (can you believe we had operating systems that ran in 128KB?) . My “play” machine is an Intel I7 based notebook with 8GB RAM, 1TB Hard disk, runs Windows 8 Pro and using the Hyper-V virtualizer runs for me a virtualized Windows 7, and Ubuntu 12.04 (64 bit) concurrently. My Ubuntu virtual environment quite often runs a virtual android device too (recursive virtualization?). Everything running at once, and I’ll be damned…it’s faster than my “work” notebook (Intel I5, 4GB) running my work operating system…OUTLOOK! At the Consumer Electronics Show (CES 2013) earlier this month, the best demo that I saw was at the Ubuntu booth. They had a Samsung Galaxy S III phone, that when you plug it into a docking station (Display, keyboard, mouse) would run a complete Ubuntu desktop version concurrently with the Android that is already running on the phone. There has been a lot of talk about Ubuntu’s other announcement of building a phone operating system but this one got my interest. Think of the possibilities, you can carry your entire computing environment around with you, and plug it into another device to give it context. So based on what you plug it into the device can morph to what you want !! This was running a quad-core ARM Cortex-A9, and piggybacks off some modifications to the Android kernel. Newer devices launching this year are going to be running quad-core ARM Cortex A15, which has the ability to run each core separately, and you can actually run a separate operating system on each of the cores. I don’t really expect to see a virtualized IOS/Android/Windows phone anytime soon, but think about the performance that you can put in the background.  I got to discussing the demo and it’s possibilities with Pat McGowan, Director of Engineering and Product Strategy at Canonical (company the ships ubuntu). A lot of this is already available and can be used in your designs, the potential is huge… The smartphone has become the window into the world around us. With features like these the smartphone will be able to acquire context, of where we are and what we want to do, and provide today what we only saw in the movies. An ancient Chinese curse says “May you live in Interesting times…” We most definitely are ....   - Samyeer Metrani