标签: electronics

The "Things" in our world are becoming increasingly complicated, and the failure modes a whole lot more interesting ("interesting" as in the old curse, "may you live in interesting times"). In the world of IoT, what is fixing broken things going to be like? Let's look at where we are, and extrapolate.   My "check engine soon" light came on when I was driving home from the (first) visit to the auto shop (an oxygen sensor problem, supposedly fixed). I took it back so they could look at it again and reset it -- it came on again on the way home from the second trip, as I'm writing this, so it goes back for a third visit now. This reminds me of my high-tech, high-efficiency, computerized heating/air-conditioning unit, which has been down for almost a month, with six tech visits so far, interleaved with ordering parts from the manufacturer. It's still not working. On the sixth visit they brought in an expert on this unit, and the manager is coming out with him for visit seven after ordering enough parts to rebuild the unit if necessary, and the manager swears he won't leave until we're happy. OK, we have an extra bedroom, I'm ready. But as makers of complex stuff, how can we all do our part to keep this sort of thing from happening with our own products?   Well, education is important, but it has to be the right education. I didn't think any of the techs who worked on my A/C were incompetent, but they were under time pressure to get in and get out, didn't want to install new parts when old ones were fine, and made a couple of basic debug process errors. I'm sure that all these guys know lots more than I ever will about A/C, but some somewhat-understandable (and a few "huh?") mistakes were made that led to extra trips. The first was that the first six visits were by different techs, a scheduling process failure.   This was a hard failure, the best kind when identifying a problem. What if the problem was intermittent, or had only shown up when the unit was controlled over the Internet or by the power company's mesh radio network, and the problem involved interaction among networking, security, radios, and software? And maybe even malware? How many trips would that take?   The length of time this is taking brings attention to the required practical debugging skills needed to minimize fix-time for low-to-medium-tech electronics-controlled devices. Those skills aren't ubiquitous. And worse, even high-efficiency residential heat pumps are relatively simple, compared to networked devices with multiple processors, multiple radios, a few million (or tens of million) lines of code, and chips the size of your fingernail that are so dense that if you de-cap them they don't even look interesting any more like chips used to. (The Museum of Modern Art in NYC once had an exhibit of blow-ups of complex chips -- they were exquisite modern art, many of them signed by their artists right there on the silicon).   So now, let's talk about IoT -- WHO'S GOING TO FIX THIS SMART STUFF WHEN IT BREAKS? Think about what fixing broken IoT devices will be like. The more responsibility you give to electronics, the more annoying a failure can be -- with great power comes great responsibility, as Spiderman says. Downtime can affect lives in annoying and even profound ways. Defect opportunities increase exponentially with product complexity, and while increasing reliability of individual components and good modularity helps offset that, exponentials are hard to beat in the fullness of time. And many of these babies are complex, and connected to other things that are complex. What new skills will be needed to diagnose and fix them?   We need to be thinking about *which* fundamentals and *which* debugging skills are needed in a world of IoT, and how to train people on those skills. And the designers of IoT products have to design them with service in mind. If the field-replaceable-unit is cheap, fine, put in a new one -- but what if it's built into your house? Or your factory? Or it's your refrigerator? Or the problem is somewhere else in the network? It's a simple fact that some field guys might not know Ohm's Law or the difference between WPA-PSK and WEP, that there may be malware involved in a problem, that techs can't pull out a protocol analyzer and diagnose a failed handshake on an encrypted link, etc., etc. So they might have little choice but to just start replacing components until the problem goes away, despite the cost and repair time -- when in fact the root cause might be a mis-typed port number or password, or that the router vendor once shipped revs of software with DLNA turned off and THIS guy is running it (or the owner, aware of security problems with DLNA, had turned it off manually). Sometimes the problem gets fixed totally by accident, without ever knowing what it was so you can keep it from happening again. I don't know about you, but I don't like that future much. Come to think of it, I'm living in it now, and it's getting pretty hot in the house.   The spread of IoT devices into society will go much slower than the people who sell them would like unless we think forward to the support structure it's going to take to fix these things when they break. Otherwise, the backlash from field problems will slow adoption of new tech for everyone (fear is a great demotivator). How do you quickly fix your radio-controlled, Internet-connected door lock product that has trapped someone in their garage? Or help someone when your Internet-connected smart refrigerator product sometimes forgets to order milk? Think ahead to how your product support and a tech will handle such problems. Remote- and self- diagnosis, mail-in programs, and online self-diagnosis charts to reduce truck-rolls is certainly a requirement, when they work, but they won't always (bad power supply, loose connection, interfering radios next door, ...). And never forget security -- if you make bypassing security easy for techs ("For the super-user password, look on P. 2 of the service manual"), you might make it easy for bad guys. (More venting on that subject another day!)   The more complicated things are, the more skilled the diagnostic/repair people have to be, and/or the easier it has to be to service a failure. A big part of the fix is to teach fundamental skills well, so techs have the basics of how things actually work to fall back on to figure stuff out, even if they've never seen it before -- not just "how to fix device x", but "basic things that anything that does what x does MUST do right". And you have to build escalation into the service process -- you can't send Superman to open every hard-to-open jar of pickles because there's only one Superman and there are a lot of pickle jars, and maybe he's got other jobs more pressing. You need to know how and when to escalate, and how to do it fast.   So what does the IoT equivalent of teaching basic physics like refrigeration cycles and Ohm's Law to mid-level techs look like? It certainly involves a more-than-cursory understanding of software, security, and networking -- far beyond just knowing about specific implementations (Linux, IP, ...), though that too. What else?   And what does a serviceable IoT device look like? Perhaps it exposes test points that can quickly explain why it's not talking (LEDs and test points are cheap, phone calls and truck rolls are expensive). When it can talk, it self-diagnoses and tells the tech where it hurts in language that a tech with basic skills can understand, even without knowing that device. It calls in sick, if it can -- maybe you can even fix it before the failure affects the customer. It provides key information that makes it easy to decide when to escalate to the next level up in the service pyramid. It lets authorized parties diagnose it, ideally remotely, without becoming insecure. What else?   Postscript Though we're still offline, the A/C problem was tentatively identified as being caused by the power supply failing in a way that over-voltaged multiple FRUs, and although each part had been replaced with a known-good unit with no resulting change in behavior, the entire set of failed units was never replaced with known-good units *at the same time*. The techs were following common practice (modulo a couple of goofs) in an attempt to isolate and correct a single failure at the lowest cost. If the current diagnosis is correct, instead of seven visits the repair would optimally have taken four visits if the parts weren't stocked locally, or two if they were and the expert made the second visit with parts in hand. From my selfish viewpoint, 1-2 weeks instead of 3-4.   Steve Bunch is a CS/EE software architect with experience in operating systems, system architecture, networking, radio, and software security, and was responsible for the creation of the embedded software platform that was used in several hundred million cellphones. He is currently involved in a startup operating in the networking spa ce.

Water flow is often used to illustrate basic electronics in the form of analogies. In this blog, after learning the hard way, I will attempt to explain basic water flow in terms of electronics.   It started one morning with a wet carpet just outside the laundry room and the water heater. The water was warm, and the puddle started at the base of the water heater which had no drip pan. We assumed that the water heater had sprung a leak, which is a typical mode of their behavior.   While I had done some copper pipe soldering many years ago, I felt that installing a new water heater was probably beyond my basic plumbing skills, so we had the heater replaced by professionals. They showed up at 9:00 a.m. and had the job done by noon, as my wife reported while I was at work. My draining the tank ahead of time might have helped.   But then about 3:00 p.m. my wife called back to report that the carpet was still getting soaked even after sucking up the water with the carpet cleaner. And the puddle was hot, so this was not a residual puddle from water still hidden under the wall. Uh-oh -- sounds like a leaky pipe inside the wall.   The heater hot water outlet entered the laundry room wall about six feet above floor level, and I assumed it went straight down from there. I had two choices: I could drain and move the new water heater and cut through the wall behind it, or I could investigate from the other side of the wall (which happened to be the rear inside of a kitchen cabinet). Due to laziness, I chose the latter to start with.   After measuring from the laundry room side of the wall I used a three-inch hole saw to cut out the rear of the kitchen cabinet and through the drywall halfway from floor level, and luckily got centered over the pipe the first time. It was wet. Then I drilled another 3-inch diameter circular cutout two feet higher -- the pipe was dry.   Bingo! I now knew approximately the location of the leak -- about two feet above the concrete floor. It was very fortunate that the leak was not below the concrete slab! After cutting out a larger section of wall and peeling back all the thermal insulation materials, I saw what is in the picture below. So I set up a small funnel and short drainage hose into a large cooking pot to collect the leakage while pondering (no pun intended) what to do:     The sliced-open tin can was planned as a heat shield for my brand-new propane torch to solder the pinhole closed. Yeah, right.   I shut off the cold water intake valve to the water heater, opened all the household hot water faucets, and waited for the above stream to subside. And waited... and waited... and waited.   After many fruitless hours I finally concluded that maybe the shower and kitchen faucets that combined the hot and cold feeds into a single outlet might be allowing the still-pressurized cold feed to leak over into the hot system. Something like this with a "leaky" lower diode, consider the incoming cold water pressure as akin to a positive DC voltage:     My hypothesis was later confirmed by a friendly, long-experienced, retired plumber at the local home improvement center. He referred to it as "cross-feed," similar to what we know as "cross-talk." Ask these guys, they have a lot of good tips for novices. And it did work -- shutting off the outdoor curbside main cold water feed stopped the flow from the pin hole a few minutes later.   But now the situation was a lot more serious -- lack of hot water is an annoyance (cold showers, etc.), but NO water is a magnitude more of a problem. We take for granted how easy it is to flush a toilet, and it's not always mellow yellow. (Note to renovators -- always make sure your new shower/bath/kitchen faucets have two individual hot/cold controls.)   No big deal, I thought, just solder the pinhole closed once the water stops leaking out. Well, we fluxed and tried over and over again. The solder simply would not flow. Having done a lot of electronics soldering, I recognized that even with the torch going full blast the pipe was not getting hot enough. And judging from the steam hissing from the pinhole, there was still standing water in the pipe at the pinhole level, and being at the lowest elevation it had nowhere else to go.   I tried snaking some 1/4-inch icemaker tubing as a siphon down the hot water pipe that by now had been disconnected from the heater (thankfully the new heater had been installed with flexible pipe with threaded attachments instead of the original soldered piping) and sucking on it to remove the standing water (gross!). It seemed to work -- for a few minutes. Then the water level rose back up to the pinhole, I could still see a small trickle leaking out again. In electronic terms, here is what was happening:     Both the hot and cold systems had water stored in the higher-elevation pipes, represented by the capacitors. Discharging at the pinhole let the remaining stored water drain back to the level of the pinhole through the "leaky-diode faucets" and slowly escape. This would take forever to drain out through the tiny pinhole, or to be sucked out through the mouth-operated siphon, which I wasn't looking forward to doing again.   The only way to speed up the discharge process was to bite the bullet and remove the water heater again, cut into the drywall behind it, and with a brand-new, mini-hacksaw made for this purpose cut three inches of the bad section out of the pipe so that a six-inch "repair coupling" could be soldered in. This allowed most of the standing water to discharge quickly, but only down to the level of the lower cut-off pipe. Now the situation looked like this:   The capacitor voltage (water level) cannot discharge below the forward bias voltage of the silicon diode. A residual charge will remain since the electrons have nowhere to go. The standing water in the lower pipe also had nowhere to go, and the pipes still could not be soldered.   To discharge a capacitor, it must have a current path. A negative current source can discharge it. In this water case, a wet/dry shop vac coupled to the cut-off pipe through a duct-taped "impedance matching network" to seal the air flow for maximum suction transfer, along with opened water faucets throughout the house, did the job nicely. With no more standing water I was finally able to solder the new section of pipe in.   Thank goodness for duct tape and shop-vacs!   In retrospect, had I thought of using the wet/dry vac first, I could have simply attached it to the hot water pipe removed from the top of the water heater and applied the suction there. Would have saved me a lot of work, and I would have been able to solder the pinhole through the rear wall of the kitchen cabinet instead of moving the heater and cutting into the drywall.   Next time I will know better. Maybe this experience will help another neophyte with a similar plumbing emergency.   Glen Chenier is an old-timer who grew up with vacuum tubes and therefore has been amazed and fascinated by the many advances in the electronics industry since.

Water flow is usually used to explain basic electronics in the form of analogies. In this blog, after learning the hard way, I will attempt to explain basic water flow in terms of electronics.   It started one morning with a wet carpet just outside the laundry room and the water heater. The water was warm, and the puddle started at the base of the water heater which had no drip pan. We assumed that the water heater had sprung a leak, which is a typical mode of their behavior.   While I had done some copper pipe soldering many years ago, I felt that installing a new water heater was probably beyond my basic plumbing skills, so we had the heater replaced by professionals. They showed up at 9:00 a.m. and had the job done by noon, as my wife reported while I was at work. My draining the tank ahead of time might have helped.   But then about 3:00 p.m. my wife called back to report that the carpet was still getting soaked even after sucking up the water with the carpet cleaner. And the puddle was hot, so this was not a residual puddle from water still hidden under the wall. Uh-oh -- sounds like a leaky pipe inside the wall.   The heater hot water outlet entered the laundry room wall about six feet above floor level, and I assumed it went straight down from there. I had two choices: I could drain and move the new water heater and cut through the wall behind it, or I could investigate from the other side of the wall (which happened to be the rear inside of a kitchen cabinet). Due to laziness, I chose the latter to start with.   After measuring from the laundry room side of the wall I used a three-inch hole saw to cut out the rear of the kitchen cabinet and through the drywall halfway from floor level, and luckily got centered over the pipe the first time. It was wet. Then I drilled another 3-inch diameter circular cutout two feet higher -- the pipe was dry.   Bingo! I now knew approximately the location of the leak -- about two feet above the concrete floor. It was very fortunate that the leak was not below the concrete slab! After cutting out a larger section of wall and peeling back all the thermal insulation materials, I saw what is in the picture below. So I set up a small funnel and short drainage hose into a large cooking pot to collect the leakage while pondering (no pun intended) what to do:     The sliced-open tin can was planned as a heat shield for my brand-new propane torch to solder the pinhole closed. Yeah, right.   I shut off the cold water intake valve to the water heater, opened all the household hot water faucets, and waited for the above stream to subside. And waited... and waited... and waited.   After many fruitless hours I finally concluded that maybe the shower and kitchen faucets that combined the hot and cold feeds into a single outlet might be allowing the still-pressurized cold feed to leak over into the hot system. Something like this with a "leaky" lower diode, consider the incoming cold water pressure as akin to a positive DC voltage:     My hypothesis was later confirmed by a friendly, long-experienced, retired plumber at the local home improvement center. He referred to it as "cross-feed," similar to what we know as "cross-talk." Ask these guys, they have a lot of good tips for novices. And it did work -- shutting off the outdoor curbside main cold water feed stopped the flow from the pin hole a few minutes later.   But now the situation was a lot more serious -- lack of hot water is an annoyance (cold showers, etc.), but NO water is a magnitude more of a problem. We take for granted how easy it is to flush a toilet, and it's not always mellow yellow. (Note to renovators -- always make sure your new shower/bath/kitchen faucets have two individual hot/cold controls.)   No big deal, I thought, just solder the pinhole closed once the water stops leaking out. Well, we fluxed and tried over and over again. The solder simply would not flow. Having done a lot of electronics soldering, I recognized that even with the torch going full blast the pipe was not getting hot enough. And judging from the steam hissing from the pinhole, there was still standing water in the pipe at the pinhole level, and being at the lowest elevation it had nowhere else to go.   I tried snaking some 1/4-inch icemaker tubing as a siphon down the hot water pipe that by now had been disconnected from the heater (thankfully the new heater had been installed with flexible pipe with threaded attachments instead of the original soldered piping) and sucking on it to remove the standing water (gross!). It seemed to work -- for a few minutes. Then the water level rose back up to the pinhole, I could still see a small trickle leaking out again. In electronic terms, here is what was happening:     Both the hot and cold systems had water stored in the higher-elevation pipes, represented by the capacitors. Discharging at the pinhole let the remaining stored water drain back to the level of the pinhole through the "leaky-diode faucets" and slowly escape. This would take forever to drain out through the tiny pinhole, or to be sucked out through the mouth-operated siphon, which I wasn't looking forward to doing again.   The only way to speed up the discharge process was to bite the bullet and remove the water heater again, cut into the drywall behind it, and with a brand-new, mini-hacksaw made for this purpose cut three inches of the bad section out of the pipe so that a six-inch "repair coupling" could be soldered in. This allowed most of the standing water to discharge quickly, but only down to the level of the lower cut-off pipe. Now the situation looked like this:   The capacitor voltage (water level) cannot discharge below the forward bias voltage of the silicon diode. A residual charge will remain since the electrons have nowhere to go. The standing water in the lower pipe also had nowhere to go, and the pipes still could not be soldered.   To discharge a capacitor, it must have a current path. A negative current source can discharge it. In this water case, a wet/dry shop vac coupled to the cut-off pipe through a duct-taped "impedance matching network" to seal the air flow for maximum suction transfer, along with opened water faucets throughout the house, did the job nicely. With no more standing water I was finally able to solder the new section of pipe in.   Thank goodness for duct tape and shop-vacs!   In retrospect, had I thought of using the wet/dry vac first, I could have simply attached it to the hot water pipe removed from the top of the water heater and applied the suction there. Would have saved me a lot of work, and I would have been able to solder the pinhole through the rear wall of the kitchen cabinet instead of moving the heater and cutting into the drywall.   Next time I will know better. Maybe this experience will help another neophyte with a similar plumbing emergency.   Glen Chenier is an old-timer who grew up with vacuum tubes and therefore has been amazed and fascinated by the many advances in the electronics industry since.  

My background is long and varied. I won't go into it too much here, as it's mostly not pertinent. The thing is that, like many other engineers, I've picked up plenty of things over the years that have stuck with me and let me connect some of the dots. It's interesting to pick up things we consider common sense and later learn we're members of a small minority who actually know these things. For example, before meeting me, the group of guys who work with me right now had never heard of a threek (a three-tined fork), a twok (a two-tined fork), or a fivek (a five-tined fork). As you may know from an image that's been bouncing around the Internet for a while, a one-tined fork—or a onek—is also known as a chopstick. Getting (slightly) back on the path, I recall my first undergraduate digital electronics lab in which we were told to go breadboard a design. We were to start with a logic function, perform the Karnaugh mappings, and build the circuit using 74LS-series logic. (Does anyone still do this?) We worked up our circuits and then be-bopped over to the lab to implement them using real devices. Now, I most certainly was not the brightest bulb in the room, but I found it more than just a bit amusing when a bunch of my classmates couldn't figure out the relationship between the two items shown below—a datasheet and a "chip." On that particular day, I rapidly turned into a lab assistant—explaining to the younger kids (I had well more than 6-7 years of experience as a technician at this point) the relationship between a handful of integrated circuits and their related datasheets. Over time, I've run into plenty of experienced engineers who don't know the things I think I know. This is not to knock on anyone (or tout myself, as you already know I'm a dope). We all have our interests and expertise, and we all have holes in our knowledge. Long gone are the days of the Renaissance Man. The sum of human knowledge has just gotten waay too big. The end result of all of this is that I've been hankering to make a video (no, not about threeks). You see, in my life, I've been a gearhead, an electronics technician, an engineering student, and an engineer—each of these for fairly lengthy times. Over these years, I've picked up little tidbits that I'd never really thought about before, but each one opened my eyes to the grand interconnectedness of the world. I've been wanting to make this video for several years, and I just got around to it. The video covers an interconnected route through semiconductor physics, with slight offshoots to things I find interesting, but get this— there's (almost) no math . Yeah, this video covers the most fundamental of concepts without the math to prove it, but I start with the basic PN junction diode and then cover BJTs, FETs, EEPROMs, and CCDS (and their control). I don't know if it's worth anything to anybody, but the most gloriously Magnificent Max suggested that I write something out and present it for your amusement. Primarily, this is for the beginner, and I try to speak in understandable words (even though most of what I spout is usually gibberish). If you have an hour and a half of your life to waste, and you are having problems sleeping, then maybe you'd like to take a look at a fat old man trying to explain some stuff. You might know someone who could actually benefit from watching this video. If you do get to watch the video, I'd be very interested in learning what you think about it. Tom Burke is a Senior Systems Engineer  

Hi everyone. My official name is Javier D. Garcia-Lasheras, but you can simply call me Javi. I'm a Spanish electronics hacker who has worked in a wide range of areas in the electronics and computing industries, from microelectronic research to operating systems. Throughout my career, two complementary issues have constantly attracted my attention: How can fundamental science and electronic technology mutually benefit each other, and what can one do to act as a catalyst for this synergy? Once upon a time, a scientist working alone was able to accomplish breakthrough discoveries or develop new technologies with a limited budget. Just think of the way Faraday changed the world by harnessing the power of electricity. Today the complexities associated with fundamental research lead us to use cooperative, multi-disciplinary development teams to cope with the huge budgets imposed by the new challenges we face. A superb example of this is the European Centre for Nuclear Research (CERN), where literally thousands of scientists and engineers from tens of countries work together in massive research facilities. In this environment, the first web server and web pages were developed by Tim Berners-Lee in 1989 for helping scientists share data and documents. This vague but exciting idea, initially intended for a scientific environment, has changed the ways in which we all communicate and live.   SM18, the CERN transnational facility for testing superconducting technologies. Building CERN's Large Hadron Collider (LHC), the world's biggest particle smasher, required the development of a new generation of radiation-hardened electronics, solid state devices, cryogenic superconductors, and big-data processing architectures. The LHC has finally led to the discovery of a candidate for the long-sought Higgs boson, a key piece of the standard model of fundamental particles. Only by pushing state-of-the-art technology to its limits have we been able to test our best theory for describing the quantum world across energies that range from absolute zero to near-Big Bang temperatures. Recently, I had the honour, the privilege, and the pleasure of spending a week at CERN working with a young team of hardware developers. While there, I had the opportunity to discover more about the work they are undertaking. They split their time between developing the electronic building blocks used to perform gargantuan physics experiments and determining the best way to bring this technology to the masses.   Javi at CERN. To give you a quick example, I was able to see a new generation of open EDA tools that rival their commercial counterparts. I also saw a brave new control and timing network protocol that outperforms the accuracy of the best measurement and control systems in the industry by orders of magnitude. But perhaps the most important thing is that all this amazing stuff is being released into the public domain by building a community-based open-hardware ecosystem and developing open licensing setups. To help the EE Times-Asia community become aware of all the opportunities that the big science domain is bringing to the world of electronics, I will be using my future columns to keep you on top of things. In the meantime, do you have any questions about how fundamental research may benefit one's day-to-day job as an electronics or computing engineer? Javier D. Garcia-Lasheras Open Science Activist