标签: vision

One of the things I talked about in my Not Your Grandmother's Embedded Systems session at ESC Silicon Valley was the topic of embedded vision. The example I often use is that of a next-generation electric toaster. The first time you power this on, it will cheerily greet you by saying: "Hello, what's your name?" I would, of course, answer: "You can call me Max the Magnificent!"   When I eventually come to drop in a couple of slices of bread, the toaster will recognize me, take note of the type of bread I'm using, and inquire: "Hello Max the Magnificent, how do you prefer this type of bread to be toasted?" I may respond by saying something like: "I like my toast to be well done." When the toast pops out, the toaster might say something like: "How's that?" I may respond "That's just right" or "Perhaps a tad darker next time" or "Not quite so dark, if you please."   The next time I go to make some toast, we can dispense with the dialog. The toaster will once again recognize me, it will recognize the bread, and it will present my taste buds with just the experience they are looking for. Similarly, the toaster will learn my preferences for bagels, baguettes, croissants, and so on, and it will do the same for all of the members of my household.   Now, when I give this sort of talk, some people are under the impression that this technology is a long way out; I believe it's closer than they think. But how might one go about adding embedded vision to one's systems? Well, I just received an email from my chum Rick Curl. The title of this email was "Who is watching you?" Inside, Rick pointed me toward a flyer for OMRON's Human Vision Components (HVC) modules .     The HVC module integrates OMRON’s best in class image sensing technology (OKAO Vision) along with a camera, processor, and external interface, all onto a single 60mm x 40mm PCB. The module boasts 10 functions as follows: - Human body detection - Hand detection - Face detection - Face recognition - Age estimation - Gender estimation - Facial pose estimation - Gaze estimation - Blink detection - Expression/mood estimation   The HVC module uses serial communication via UART to communicate its findings in real-time to your main system, which can use this information as the basis for its actions. In the case of a flat screen display showing an advert in a store, for example, the system may present different adverts based on the age and/or gender of the person viewing the display. Similarly, a vending machine may base its food and beverage recommendations on the age and/or gender of the customer.   In the case of my hypothetical toaster, it may base it's responses on the user's expression. If it sees a "happy face," it may bask in the glow of a job well done, while a sad or angry expression may prompt it to say: "Oh dear, did I do something wrong?"   I can envisage so many applications for this sort of thing. Take my Caveman Diorama project, which is to be presented in a 1950s television cabinet, for example. In an earlier column on this topic, I mentioned that there will be a variety of audio effects, like the sound of the waterfall and the sound of the cavemen chatting. I also mentioned the idea of having a tiny camera located at the back of the cave. When a visitor bends down to look into the cave, an image from the inside of the cave showing the visitor's face looking into the TV set will be presented as a live feed onto a screen mounted on the wall above the television.     Well, now imagine what we could do with one of OMRON’s HVC modules. Suppose we have a kid looking into the scene. If the kid smiles, the conversation inside the diorama could sound happy and include laughter; if the kid then frowns or looks puzzled, the tone of the conversation inside the diorama could change to reflect this; and so on and so forth.   While you're mulling on that, check out this video showing example HVC applications.   I have to admit that I'm very enthused by all of this. New applications keep on popping into my mind. Take my Inamorata Prognostication Engine, for example. This little beauty is intended to predict whether the radiance of my wife's smile will fall upon me. (Of course, if Gina ever discovers the true purpose of this device, then I really won't need a Prognostication Engine to determine her mood :-)   Now, suppose that Gina comes to visit me in my office and pauses to peruse and ponder the Prognostication Engine, which sits in the bay outside. If the Prognostication Engine were equipped with an HVC module, it could detect and identify Gina's face and also determine her mood. If she starts off smiling, but soon begins to frown, and then starts to march toward my office, we could arrange things such that my door automatically closes and locks itself and an "On Air" sign lights up outside my door. This should give Gina pause for thought while also giving me sufficient time to escape through the window (LOL).   What about you? What applications spring to your mind for these little beauties?

Albert Einstein once said , "Everything should be as simple as possible, but not simpler." I would never claim to have his level of insight -- or such an awesome head of hair -- but as an engineer, I wholeheartedly agree with this statement.   It definitely applies when it comes to computer-controlled vision systems for parts inspection. When designing a system, it can be tempting to try and include every inspection that could possibly be done. If, for instance, the machine is already being designed to check whether a bearing is missing a roller or ball, why not also inspect for the roundness of the bearing's housing or even check for surface defects? These extra steps should be simple, because the camera's already there, right?   Wrong. You rarely get anything extra for free. Engineers who work on vision systems know what I am talking about. But sometimes other people erroneously believe that just putting a camera on it constitutes an effective inspection setup.   When adding tasks for a vision system, a major challenge is lighting and camera position. Just because the light is properly set up for one inspection doesn't mean it will be correct for anything else. The result is inevitably some sort of compromise, which is never ideal. Of course, one could add a robot or other device to position all the parts for inspection, but it would add significant complexity and cost.   Another drawback of piggbacking features on to a vision system is that troubleshooting becomes harder with each additional inspection. It's incredibly frustrating to tweak a machine to pass one inspection, only to find that it often causes a malfunction elsewhere in the program.   On my first job, the inspection machinery that I oversaw was set up primarily to check for a missing pin or roller in a bearing. But it was also set up to check several other things that technically were not required or even necessary.   I was eventually able to program all these machines to look only for a missing pin -- with no loss in parts quality, interestly enough.   Later, an outside contractor was called in to update the machinery. He offered to expand its inspection capabilities. I wanted that code to remain as simple as possible. Some of these machines used a PlayStation-like controller for adjustment, so I seriously didn't want to have to do more than was absolutely necessary.   As for making things as simple as possible, but not simpler, an inspection camera shouldn't physically be more adjustable than necessary. To that end, I wish vision companies would start making holes for dowel pins standard on all their products. This feature would simplify the setup, allowing one to determine the precise camera position within a thousandth of an inch or so.   Jeremy Cook is a manufacturing engineer with 10 years of experience and has a BSME from Clemson University.

Albert Einstein has been quoted as saying , "Everything should be as simple as possible, but not simpler." I would never claim to have his level of insight -- or such an awesome head of hair -- but as an engineer, I wholeheartedly agree with this statement.   It definitely applies when it comes to computer-controlled vision systems for parts inspection. When designing a system, it can be tempting to try and include every inspection that could possibly be done. If, for instance, the machine is already being designed to check whether a bearing is missing a roller or ball, why not also inspect for the roundness of the bearing's housing or even check for surface defects? These extra steps should be simple, because the camera's already there, right?   Wrong. You rarely get anything extra for free. Engineers who work on vision systems know what I am talking about. But sometimes other people erroneously believe that just putting a camera on it constitutes an effective inspection setup.   When adding tasks for a vision system, a major challenge is lighting and camera position. Just because the light is properly set up for one inspection doesn't mean it will be correct for anything else. The result is inevitably some sort of compromise, which is never ideal. Of course, one could add a robot or other device to position all the parts for inspection, but it would add significant complexity and cost.   Another drawback of piggbacking features on to a vision system is that troubleshooting becomes harder with each additional inspection. It's incredibly frustrating to tweak a machine to pass one inspection, only to find that it often causes a malfunction elsewhere in the program.   On my first job, the inspection machinery that I oversaw was set up primarily to check for a missing pin or roller in a bearing. But it was also set up to check several other things that technically were not required or even necessary.   I was eventually able to program all these machines to look only for a missing pin -- with no loss in parts quality, interestly enough.   Later, an outside contractor was called in to update the machinery. He offered to expand its inspection capabilities. I wanted that code to remain as simple as possible. Some of these machines used a PlayStation-like controller for adjustment, so I seriously didn't want to have to do more than was absolutely necessary.   As for making things as simple as possible, but not simpler, an inspection camera shouldn't physically be more adjustable than necessary. To that end, I wish vision companies would start making holes for dowel pins standard on all their products. This feature would simplify the setup, allowing one to determine the precise camera position within a thousandth of an inch or so.   Jeremy Cook is a manufacturing engineer with 10 years of experience and has a BSME from Clemson University.

I've invited my colleague Brian Dipert, editor-in-chief of the Embedded Vision Alliance, to share his perspective on various face analysis algorithms used in embedded vision. He regularly discovers and reports on interesting embedded vision applications, some of which he discusses here. Face recognition—the technology that enables cameras (and the computers behind them) to identify people automatically, rapidly, and accurately—has become a popular topic in movies and television. Consider the 2002 blockbuster Minority Report . If you've seen it (and if you haven't, you definitely should), you might recall the scene where Tom Cruise's character, Chief John Anderton, is traversing a shopping mall. After scanning his face, sales kiosks greet him by name and solicit him with various promotions. Lest you think this is just a futuristic depiction, the British supermarket chain Tesco is now making it a reality. Plenty of other real-life face recognition implementations exist. Consider Facebook's tag suggestions, an automated system that identifies Friends' faces each time you upload a photo (a facility likely enhanced by the company's 2012 acquisition of Face.com), or Apple's iPhoto software, which automatically clusters pictures containing the same person. Don't forget the face recognition-based unlock option supported in the last few Android releases (likely enabled by Google's 2011 acquisition of Pittsburgh Pattern Recognition) and available on iOS via third-party applications. And the new Microsoft Xbox One and Sony PlayStation 4 game consoles support face recognition-based user login and interface customisation via their camera accessories (included with the Xbox One, optional with the PS4). Face recognition has made substantial progress in recent years, but it's admittedly not yet perfect. Some of its limitations are due to an insufficiently robust database of images. And some of its limitations are a result of algorithms not yet able to compensate fully for things like off-centre viewing angles, poor lighting, or subjects who are wearing hats or sunglasses or sport new facial hair or makeup. Ironically, face recognition's inability to identify people with guaranteed reliability provides privacy advocates with solace. However, other face analysis technologies are arguably more mature, enabling a host of amazing applications, and they are useful for addressing privacy concerns, since they don't attempt to identify individuals. For example, face analysis algorithms can accurately discern a person's gender. This capability is employed by electronic billboards that display varying messages depending on whether a man or woman is looking at them, as well as by services that deliver dynamically updated reports on meeting-spot demographics. Face analysis techniques can also make a pretty good guess as to someone's age bracket. Intel and Kraft harnessed this capability last year in developing a line of vending machines that dispense free pudding samples only to adults. More recently, the Chinese manufacturing subcontractor Pegatron used it to screen job applicants, flagging those who may be less than 15 years old, so it can avoid hiring underage workers. The mainstream press tends to latch on to any imperfection as a broad-brush dismissal of a particular technology. As engineers, we know how oversimplistic such an approach is. While RD and product developers continue to pursue the holy grail of 100% accurate face recognition, other face analysis techniques are sufficiently mature to support numerous compelling uses. How will you leverage them in your next-generation system designs? Visit the Embedded Vision Alliance website for plenty of application ideas, along with implementation details and supplier connections. Jeff Bier is the founder of the Embedded Vision Alliance.

I've invited my colleague Brian Dipert to share his perspective on various face analysis algorithms used in embedded vision. As editor-in-chief of the Embedded Vision Alliance, he regularly discovers and reports on interesting embedded vision applications, some of which he discusses here. Face recognition—the technology that enables cameras (and the computers behind them) to identify people automatically, rapidly, and accurately—has become a popular topic in movies and television. Consider the 2002 blockbuster Minority Report . If you've seen it (and if you haven't, you definitely should), you might recall the scene where Tom Cruise's character, Chief John Anderton, is traversing a shopping mall. After scanning his face, sales kiosks greet him by name and solicit him with various promotions. Lest you think this is just a futuristic depiction, the British supermarket chain Tesco is now making it a reality. Plenty of other real-life face recognition implementations exist. Consider Facebook's tag suggestions, an automated system that identifies Friends' faces each time you upload a photo (a facility likely enhanced by the company's 2012 acquisition of Face.com), or Apple's iPhoto software, which automatically clusters pictures containing the same person. Don't forget the face recognition-based unlock option supported in the last few Android releases (likely enabled by Google's 2011 acquisition of Pittsburgh Pattern Recognition) and available on iOS via third-party applications. And the new Microsoft Xbox One and Sony PlayStation 4 game consoles support face recognition-based user login and interface customisation via their camera accessories (included with the Xbox One, optional with the PS4). Face recognition has made substantial progress in recent years, but it's admittedly not yet perfect. Some of its limitations are due to an insufficiently robust database of images. And some of its limitations are a result of algorithms not yet able to compensate fully for things like off-centre viewing angles, poor lighting, or subjects who are wearing hats or sunglasses or sport new facial hair or makeup. Ironically, face recognition's inability to identify people with guaranteed reliability provides privacy advocates with solace. However, other face analysis technologies are arguably more mature, enabling a host of amazing applications, and they are useful for addressing privacy concerns, since they don't attempt to identify individuals. For example, face analysis algorithms can accurately discern a person's gender. This capability is employed by electronic billboards that display varying messages depending on whether a man or woman is looking at them, as well as by services that deliver dynamically updated reports on meeting-spot demographics. Face analysis techniques can also make a pretty good guess as to someone's age bracket. Intel and Kraft harnessed this capability last year in developing a line of vending machines that dispense free pudding samples only to adults. More recently, the Chinese manufacturing subcontractor Pegatron used it to screen job applicants, flagging those who may be less than 15 years old, so it can avoid hiring underage workers. The mainstream press tends to latch on to any imperfection as a broad-brush dismissal of a particular technology. As engineers, we know how oversimplistic such an approach is. While RD and product developers continue to pursue the holy grail of 100% accurate face recognition, other face analysis techniques are sufficiently mature to support numerous compelling uses. How will you leverage them in your next-generation system designs? Visit the Embedded Vision Alliance website for plenty of application ideas, along with implementation details and supplier connections. Jeff Bier is the founder of the Embedded Vision Alliance.