PICAXE-18X vs. Lightning

[vk6bgn]

9-AUG-08
Sub tropics – Outback Australia.

I was notified at 7:30pm at night that the transmitter at our local Amateur Radio repeater was stuck on. This was after our first fears lightning storm of the year went through earlier in the day around lunchtime.

I went to the repeater site and found the PICAXE-18X in the repeater controller card had pretty much "lost its way". No all, but most of the outputs were ON for no reason. Unfortunately the “PTT” (Push To Talk) to key up the TX radio was one of the outputs that was on. I reset all the power and the PICAXE based repeater controller seems to have come good. I also let the TX radio cool down for half an hour as it was probably on continuously for 5 - 6 hours! (The unintentional 100 percent duty cycle check on the TX radio and it's heat sink are now complete!!!

My questions:

What possibly happened to the PICAXE-18X??? It was just stuck in this state and was unresponsive to any DTMF commands I sent to it via my hand held VHF transceiver or anything I did until I resent the power. ((The RX radios squelch was not open. (no sound out of it's speaker) So the repeater was not in use. So no outputs should of been on.))

I would of thought the PICAXE works or it doesn’t. Why would a power reset fix this problem?

Is a PICAXE more susceptible to something like this since it has…. how would you say this…. bootstrap code in it?

Are PICs in general more susceptible to this sort of problem vs. STAMPS, AVR’s etc.

Should I really expect better results from the PICAXE or any sort of micro, PIC, IC etc. etc. as our suitably earthed radio tower and building probably took a direct hit of tens of millions of volts at tens of thousands of amps!

In my code I have a timer (more like a counter) to shut the TX radio off if it is keyed up for a long period of time. It’s presently set to 10 minutes. This feature works perfect under normal operation. But it’s now proven that I can’t count on this feature to shut the TX radio off if it transmits for extended periods of time when the PICAXE “looses its way”. Possibly I need something external to the PICAXE to sense the transmitter current draw and shut the TX radio off if it transmits for a long period. But this will probably be some sort of electronic and mechanical relay system which is susceptible to a lightning strike too. Possibly this is really a no win situation? Does the PICAXE vs. mother nature really have a chance!?!?!

One good thing to note, at least I didn’t have to replace a dead PICAXE!

"HamRadioAddict"

 

[Michael 2727]

Put a Valve/Tube watchdog in the circuit, I bet Stan would even get in on the act

It's not so much the VVVVV Volts from the lightning when you get a near miss or strike.
There's also the 50,000V or so Volts going ^^^^^^ a milisecond beforehand where the
two potentials collide 10m above the ground then slam back into the ground.

These upward VHV filaments have been captured on film and can be from any grounded
object within a 20m radius of the eventual hit.

Who knows what actually happens inside a micro with a HV hit.
Maybe the little neon blue guy on his motorbike inside had a stack and caused
an overflow error

 

[vk6bgn]

Put a Valve/Tube watchdog in the circuit, I bet Stan would even get in on the act

Who knows what actually happens inside a micro with a HV hit.
Maybe the little neon blue guy on his motorbike inside had a stack and caused
an overflow error

Michael, Yes, the ol’ electron valve. Virtually immune to lightning strikes and nuclear EMR pulses. The trusty 6L6GB or KT88 should do. Or an RCA-813 for larger jobs.

Hey, the little blue guy on the bike…. is he the one going around and round the 17 mile track at the particle accelerator?

Hum…. Stack over flow might explain it? I better read up on that. With a stack overflow, does the chip just lock-up and seems to go into la la land?

 

[westaust55]

DO not think you will ever know for sure what happened.

If the reset had gone low or the serial input had gone high somehow the first thing that happens is the putputs all go off/low.

Could have been high electromagnetic radiation "scrambling the PICAXE circuits.

From my days in the Kimberley when we had a telephone cable on top of the ground (too had to dig 10km in solid rock), a direct hit resulted in 100m of cable vapourising. A strike within say 20m or so would blow all the gas surge arrestors at each end of the cable in the early days and then at a later time when we went to the 3M scotch-lock type connectors (and no surge arrestors), there was enough energy transferred into the cable to heat the gel causing it to expand in the conenctors and pop them open.

So you can imagine the impact of a near stirke close to your PICAXE.

 

[hippy]

You'll never know exactly why the PICAXE locked-up but it could have happened to any microcontroller. All it takes is one bit of memory to be read wrong and the chip can go off and do something unexpected, randomly set and clear outputs and never resume what it was meant to do or respond to anything until reset. As long as the event which causes problems only affects code or data being used, not what is programmed into it, the chip will recover when reset or power-cycled.

A direct lighting strike is probably the worse that can happen and very hard to design against. The usual mechanism is to use an external watchdog which can check that the micro is working as expected and force a reset if it isn't. This usually has to be implemented in resilient hardware because a software solution can equally be affected by the event you are protecting against ( as happened with the internal timer ).

The most common watchdog is one which times out and causes a reset, the micro giving it a 'stay quiet' kick every now and again. If the watchdog doesn't see the kick it will reset the micro. A good watchdog will usually check that a micro is putting out an expected sequence of pulses of correct duration and so forth so a micro 'gone rogue' is unlikely to generate the necessary sequence when it has failed. Multiple watchdogs looking for different sequences give more robustness.

Ultimately it is a no-win situation because there can always be an event which is so overwhelming that nothing can recover from it, but it is possible to design for recovery up to a certain level of event.

On the issue of whether the PICAXE is more prone to lock-up than any other micro, in one way no ( any micro can go off and lose its course ), but in another way, yes. Losing course in a PICAXE can be more catastrophic than it may be in other micros because timeouts and other means of ensuring correct operation are dependant upon the firmware doing what it must do. In others, even if the main program has failed, timeouts may still keep working. In addition, hardware watchdogs which should remain unaffected by system failures are not used by the PICAXE firmware for that purpose.

A cautionary note to others

What was experienced here is why many of us here are extremely cautious about projects which use PICAXE in safety critical and business critical systems.

This may have been a considerable event almost guaranteed to cause failure, but a poorly designed system can suffer similar problems with far less significant events. This is a particular potential in an automotive system.

It's very important to note that the PICAXE failure caused secondary consequences which could have been severe. Prolonged transmitting didn't cause overheating and burn the entire station down, but it did disrupt correct operation. A PICAXE failure which affects any in-vehicle system can make it instantly more dangerous. And there doesn't always need to be a direct link to that outcome; a PICAXE failure causing something to overheat can cause a fuse to pop which something critical ( like power steering or power-assisted brakes ) may rely upon.

 

[vk6bgn]

Hippy,

Thanks for the technical reply and cautionary notes. All very well explained.

Cheers.
"The Addict"

 

[Michael 2727]

Hey, the little blue guy on the bike…. is he the one going around and round the 17 mile track at the particle accelerator?

Hammy, no that's his distant cousin Magnetron

 

[premelec]

Redundant circuits and watchdog timers can help but nearby lightning strikes can wipe out even vacuum tubes [valves] and their filaments are notoriously less reliable over long term use.

I built an anemometer integrator 40 years ago with plastic Resistor Transistor Logic - the leads into the trailer where the equipment was located had lots of protection but a nearby strike caused failure - looking inside my unit the plastic potting material had been blown off several of the ICs and the rest worked ok... This RTLwas before CMOS which is much more sensitive to ESD and has a finer less robust geometry... in case of lightening all bets are off -

The rocket guys use electric field sense equipment to tell when it's building and I suppose you could put in a sense unit of this sort in play which would shut down your eguipment with mechanical disconnects and lowers a copper hat on your circuits when lightening was too near... Meantime simple plugable repacement equipment to get you back on line should work... good luck!

 

[BeanieBots]

At the end of the day, any device which has 'memory' can be corrupted by strong electric or magnetic fields. That includes devices such as D-Type flip-flops which is in effect a memory device. That includes counters.
Any device which has an internal state set by an external stimulus can also be effected. This includes some designs of H-Bridge. Even the ULN28003 can be made to 'latch' if subjected to enough EMI. There are several different manufactureres of the ULN28003, some can latch, some can't. That's why engineers also specify supplier. Something purchasers never appreciate when they try to get the best bargain.

These are all the things you need to know when designing for 'fail safe' and 'mission critical'. The golden rule is "nothing with memory" but when that is not an option, there are ways around it but remember, your watchdog particularly needs to be failsafe.