433 MHz wireless link trick

womai · Feb 1, 2009

Hi all,

just wanted to share my recent discovery:

I'm sure many people know those cheap, simple 433 MHz transmitter/receiver pairs which are supposed to be a drop-in into a serial data line (receiver will simply mimic the level on the transmitter, so you connect the transmitted to the Picaxe which does serout, and the receiver to the Picaxe that does serin).

Again, anybody who worked with those devices knows about the necessity of a preamble (several 0x55's = 0b01010101 or 0xaa's = 0b10101010) to pre-condition the receiver, and a qualifier (like "data") preceding the actual data so the receiving Picaxe avoids reading in bogus data.

Now ... I did all that and my link still lost about one data packet out of four. My suspicion was that before any data comes, the Picaxe receives a random data stream from the receiver (because the gain control is wide open and any random radio noise shows up as data). Now if the Picaxe happens to be just in the middle of reading a bogus data byte when the true preamble starts, there is a chance it will never correctly synchronize to the start of the true bytes, and thus mis-read the qualifier --> data packet lost. Indeed that seems to be what is going on.

Now the simple solution (simple after I stumbled across while daydreaming and sweating for an hour on the treadmill

:

- do the preamble as usual, which gets the receiver conditioned

- then, SEND NOTHING for a little more than one data byte (1 start bit, 8 data bits, 1 stop bit, i.e. 10 bits in total or e.g. 4.2ms at 2400 baud). This will make sure that wherever the receiver was at the end of the preamble, it is now idle and eagerly waiting for the next start bit. Just don't wait much longer than one byte, otherwise the receiver will lose lock again.

- now send qualifier and data

End result: just sent over 10000 data packets across the room without losing a single one.

In Picaxe language, sender side:

serout 0, T2400_4, (0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55, 0x55,)
pause 5
serout 0, T2400_4, ("data", b0)

In Picaxe language, receiver side:

serin 0, T2400_4, ("data"), b0

Wolfgang

manuka · Feb 1, 2009

Wolfgang: This is very well presented,& findings are consistent with known typical 433 receiver charcteristics. Yet another victory for a well placed PAUSE, & it looks temptingly easy to trial & apply.

My usual approach has been to send a really decent remote RX "WAKE UP-WAKE UP-WAKE UP" preamble, usually "01010101", representing ASCII (dec.) 85 = capital "U". A unique "Station ID" qualifier preceeding the data then locks the tx & rx nicely. In fact I can't recall having any data losses under such a scheme, but maybe I've been lucky! Just which 433 cheapies are you using please? Stan

manie · Feb 1, 2009

@Wolfgang
Thanks for that, this will obviously be true for the Xbee modules as well, and it could well reduce the "resend" requirement ?
Manie

womai · Feb 1, 2009

Hi Stan,

actually an ID like you mention is part of my data packet. The packet consists of 4 bytes of data (with a few more 0x55's between the first two and second two, just to make absolutely sure the receiver does not lose lock if e.g. all data bytes are 0x00). The first two bytes are a counter index, the second the actual data to be transmitted. The sender sends each packet several times. It incremements the counter value for each new data packet. The receiver looks at the counter index; if it is new (i.e. not the same as in the last received packet), it processes the data. If it is the same as last time, that means it has already received that data and it does nothing. This scheme adds redundancy and improves the link reliability further, without the need for a transmission back from the receiver to the sender. But the moment it looks like my data link is virtually perfect even without this additional measure.

The modules I use are TLP 434A and RLP 434A from Laipac. I bought them about 2 - 3 years ago, I believe it was from Advanced Micro Circuits, for about $20 per transmitter/receiver pair. After some initial playing around (transmitting counter data over the full length of a breadboard - Marconi would have been proud of me

I never really did anything useful with them up to now.

manie, unfortunately I am not at all familiar with Xbees (apart from a basic knowledge of what they are), so I can't judge whether this is applicable there. I was under the expression the Xbee is using a much more elaborate protocol that takes care of such things?

Wolfgang

manie · Feb 1, 2009

Wolfgang yes, I believe Xbee does checks and balances but if you do it with built -in redundancy it will improve data integrity, especially if I want to "streeeeeetch" the transmission distance on this 1mW transciever to as far as possible...
Manie

manuka · Feb 1, 2009

XBee = indeed totally different! Those Laipac TX/RX units look near identical to numerous similar 433 cheapies I've got stashed away here, so I'll look into this as time allows. Code listing available? Thanks- Stan

EXTRA: This is reminding me of 1960s comms receiver SSB (Single SideBand) reception requiring clever "fast attack/slow decay" AGC (auto. gain control) . Our recent HopeRF workout has revealed "pause" & full buffer issues too.

Dippy · Feb 1, 2009

Manie, XBee does all the 'stuff' for you.
You just input data (<max buffer size). When Xbee detects a pause of whatever it is I've forgotten, then it assume EOD.
It then packetises it does all the preamble gubbins and other stuff and sends it.

You do nothing except shovel it in. This is one of the raison d'etres of XBee.

Many other 433/434 modules do some of the stuff for you.
So, it DEPENDS on which module you buy.
To determine things you MUST read the device DATA SHEET

But, good info from Wolfgang.
And, again, the specific implementation will depend on the product.
Not all receiver's electronics are the same...
And, again again, we're down to suck-it-and-see. Because exactly that, experience and perseverance has resulted in Wolfgang's discovery. Whereas most people would have been straight on the Forum asking...

womai · Feb 1, 2009

Also I should admit that my sender is not really a Picaxe (head bent down in shame

but a PIC16F887 programmed in C. Just need more processing power and that side, and I love MikroC's libraries for parallel LCDs and numeric keypads. Overall was a better fit for what I need.

But the receiver is indeed a Picaxe 08M. So far I had always used 18X and up, but had a few 08Ms lying around. And there's no simple implementation of the Picaxe's serial qualifier string in any C compiler library I know of, and I didn't feel like spending a day writing the code myself. On the Picaxe the receiver program including header checking etc. is less than 20 lines of code and took me less than an hour to write!

Dippy · Feb 1, 2009

You don't have to admit it Wolfgang do you?
But I'm sure it may:
A) Impress. (This could cause small boys to cry out "Ah, a C God!".. Neptune?)
B) Cause sharp intakes of breath.
C) Cause Stan to reminisce even more ... this usually results in posting pictures of breadboards covered in felt-tipped pen markings!

westaust55 · Feb 1, 2009

All good information to keep in mind, considering that I am about to experiment with some 433MHz modules.

moxhamj · Feb 1, 2009

Re "Now if the Picaxe happens to be just in the middle of reading a bogus data byte when the true preamble starts, there is a chance it will never correctly synchronize to the start of the true bytes, and thus mis-read the qualifier --> data packet lost."

Did you find the lost packet rate improved with this new code?

It might help to get inside the serin code at the assembly level. I think there are slight pauses between each byte anyway, but I'm not sure how it synchronises - off those pauses or off start bits or off a rolling last 8 bytes or something else.

Anyway, womai, you may well be on to something that could boost the range.

Dippy · Feb 1, 2009

Can you explain that last bit Drac?

There are so many variables with modules that you'll be fortunate to get a one size fits all.

Personally, I like working the RSSI into the equation, but sadly so many cheapos don't have that option.

It reminds me of when I had my first cat's whisker set in 1893 when I was only 20.

womai · Feb 1, 2009

Hi Dr Acula,

yes, there was a dramatic improvement in error rate. Before implementing the 5ms pause, it was around 25% (lost 1 packet out of 4), although at least I never saw a wrong packet make it through (thanks to the qualifier), so it was either "receive correct data" or "miss data completely". I worked around it by sending everything 10 times but of course I knew that Murphy tells that in any real application (or when demonstrating it to somebody else) it will miss data 11 times in a row or more within the first 3 minutes

With the 5ms pause in place I had it transmit for the last 10 hours, 2 packets per second, so after around 70000 packets still not a single error or missed data. (could transmit faster, but then I couldn't log it on the PC in real time through sertxd...).

Wolfgang

MFB · Feb 1, 2009

I have never used basic 433Mhz radio links in the way described above but what does come across is the level of redundancy used (all that preamble and multiple transmissions stuff). Given that these low cost/power links can normally only manage about 1200 baud, the total over-air data rate must be very slow indeed!

For higher rates I would recommend some form of simple modem techniques. FSK modulation/demodulation would only require two extra chips and be inherently more reliable. One advantage being that the receiver normally sees a carrier and can therefore respond to data without a preamble.

boriz · Feb 1, 2009

I have never used these RF thingies. But I might be using one soon on a mains electricity use monitoring project.

I understand the need for a equal mark-space type preamble and was planning on using something like the following to maintain a continuous 50% average :

At the TX… TX byte then NOT byte.

At the RX… RX byte1 and byte2, IF byte1 XOR byte2 = 255 THEN it’s a good byte, no corruption.

Think it would work?

KIGX · Feb 1, 2009

Wolfgang,

That sounds interesting, I'll have to give it a try. I have been sending temp and RH data over a 100m distance using a 433 link using cheapo ASK madenchina modules. I was getting a pretty good capture rate - unscientifically no more than a 5% missed packet rate but I was occasionaly getting corrupted data. I was using the "UUUUUUUUUUTW" pilfered from Dr_Ac as a preamble but I had erroneously assumed that if one byte in the data stream was corrupted they all would be corrupted so I simply embedded two numbers which added together would give me a known value which I would then check for. Needless to say I got the 'checksum' to check out OK but there was still corrupted data. (After coming in from chipping ice off the patio I discovered that the high for the day was +50C). Following a recent thread here, this morning I implemented a scheme where I simply added up all of the data being sent and then sent that checksum along with the data. Add up the data again at the receiving end and see if the sums match. Been transfering data every 5 seconds for several hours now and no corrupted data getting through. For this test the Txer is only 10m away and the recvr has a 4-element yagi pointed at the Txer. I haven't implemented anything in the data stream to keep the number of 1s and 0s even but it seems to work quite well and I haven't seen any missed packets.

I'll give your approach a shot in the next retest - it sounds like it could make my data even more bullit-proof.

Thanks

Bob

papaof2 · Feb 1, 2009

C is forgiven ;-)

womai said:
Also I should admit that my sender is not really a Picaxe (head bent down in shame but a PIC16F887 programmed in C. Just need more processing power and that side, and I love MikroC's libraries for parallel LCDs and numeric keypads. Overall was a better fit for what I need.

We'll allow you to be a "C god".

I've experienced that *once* myself - when my daughter was at Georgia Tech and a group project required processing an image in C for a machine vision class. One of the guys in the group was amazed: "Your dad knows C?" It was one of the few times I was able to help with college "homework" ;-)

John

womai · Feb 2, 2009

KIGX,

those modules can live with a few bytes that do not have balanced numbers of 1s and 0s. In my case, I see the output from the receiver become erratic again after ~10 byte periods at 2400 baud, so you want to keep the number of data bytes in a block well below that unless you use Manchester coding.

IMHO checksums, redundancy, and repeated transmission of the same data block shouldn't be used to mask an inherently bad (unreliable) link. That just invites disaster later on (small or large, depending on if it is your weather monitoring station or the Space Shuttle control

They should be used to add an additional layer of reliability to a link that in principle works well. That's why I spent the time drilling down to the root cause of the lost packets.

Boriz,

you got a pretty good idea here. Ideally one would use Manchester coding (each bit followed by its negation), but since the link can live with short unbalanced bit sequences your scheme will work fine - and the beauty is that is uses much less code or processing time on the Picaxe. Still, it won't do away with the need for the 10-bit pause I outlined.

MFB,

the data rate you can achieve really depends on how much effort you spend on implementation. In my current application I need about 1 set of 4 bytes per second, so the usable data rate is really secondary and I can freely "waste" bandwidth on repeated transmissions. On the other hand, losing even a single packet would be a near catastrophy.

If you wanted to maximize data rate, you could do the following:

- run at maximum baud rate (around 4800 baud for the RX/TX modules I use)
- use Manchester coding for the data so there's no need to insert additional 0x55's in the middle of a longer unbalanced data stream
- transmit as much of that Manchester coded data in one block (so the ratio between preamble bits and actual data bits becomes more favorable).

Further improvements (will need more memory and/or processing power than the Picaxe has, but could be implemented e.g. on a Microchip PIC16Fxx programmed in some compiled language):

- use 8bit/10bit encoding instead of Manchester coding --> 25% bandwidth overhead instead of 100%.

- keep the data stream running all the time; if there is data to send, good (i.e., send it), otherwise send some balanced "idle pattern" (keepalive pattern) (e.g. 0x55's) to keep the receiver conditioned.

The two last ideas aren't mine, but are used on several ultra-high bandwidth links (multi-Gbit/sec SerDes).

Wolfgang

manuka · Feb 2, 2009

"Your dad knows C?"

Next time tease them by asking if they know Fortran - responses are often "Fortran Who?

For thus who cringe at the low data speeds associated with all these overheads, it should be noted that many telemetry/telecommand setups spend most of their time OFF. A few extra seconds to gain/send valid data is trivial when maybe the last day was spent passively.

It's often far more crucial to have gold plated 100% reliability associated with slow data rates, than higher speeds & swampy data. The latter may be typical of say TV "Teletext", where a few errors & dropouts can be lived with-while the former could be life & death, at a PICAXE level perhaps typified by Dr_A's wireless water pump controllers. (His Australian region has recently had daily heat waves around ~43°C, & tragic heat related fatalities...)

papaof2 · Feb 2, 2009

The highest temperature I've had to deal with was 105F (41C) with a heat index of 109.9F (43.2C) in Memphis, Tennessee, in August 2007. You must be super cautious when the temperatures are that high. The water supply does need "gold plated 100% reliability" because it truly is a matter of life and death, so it's easy to understand Dr_A's search for reliable communications.

People who have near 100% reliable water service from local government or commercial suppliers often aren't aware that some places don't have that luxury and the people there must acquire their water by other means. I've always lived in urban areas, but did have a lot of exposure to "country cousins" who got their water via wells - some with a hand pump - and some of the houses still had a cistern where they had collected rainwater before the well was dug.

In 4 years at this house, we've experienced one interruption to the local water service - a contractor placing new storm sewer pipe cut the water supply line to the neighborhood. Some 30+ houses were without water for about 18 hours. This was not life threatening, as the temperatures were moderate that day and the grocery stores had plenty of bottled water. I put a 30 gallon barrel in the truck and filled the barrel at a house that had water, then carried water to refill the toilet one bucket at a time.

Our state (Georgia) has been in near-drought conditions for a couple of years (maybe 60% of normal rainfall) and outside water use has been severely curtailed or banned altogether. I collect rainwater for gardening (flowers in previous years, vegetables this year) but it has other potential uses (toilets and clothes washing, but not drinking - too much cleanup is needed to make it potable under current conditions).

I'm still working on my PICAXE-based irrigation pump controller to provide measured amounts of water on a regular schedule. It's about 6 months behind schedule because spinal fusion surgery tends to limit your activity for a while and being over 60 slows recovery even more :-(

John
Definition: "country cousin" - lived in a house that was four rooms and a path

moxhamj · Feb 3, 2009

@manuka: "It's often far more crucial to have gold plated 100% reliability associated with slow data rates, than higher speeds & swampy data. The latter may be typical of say TV "Teletext", where a few errors & dropouts can be lived with-while the former could be life & death, at a PICAXE level perhaps typified by Dr_A's wireless water pump controllers. (His Australian region has recently had daily heat waves around ~43°C, & tragic heat related fatalities...)"

Yes indeed, it has been hot here. And my pumping system failed on the hottest day. Filters had got clogged, and they had got clogged because the backflush had stopped working, and that had stopped working because a koala had chewed the wires (again!). Yet another reason to go wireless. And speed isn't important - even 1200 baud is plenty.

During the day I've been treating many people with heatstroke. People get quite confused with dehydration - one lady tried to tell me her dry mouth was because of the iceblock she had just had. The wildlife is suffering a bit too. I came home last night to find this little guy (see attached) spreadeagled on the door mat. After syringing in water 1ml at a time into his mouth (he wouldn't drink) we have now got him to the stage of eating again. Koalas tend to fight if there are more than one in a tree, and many have wounds like this one has. I'm guessing that is why he doesn't want to go back up in the tree.

The best tree is the one that has juicy leaves, and those are the trees that are getting the water - which then attracts the koalas - who then chew the wires. I hope they don't like chewing antennas...

manuka · Feb 3, 2009

Dr_A: A great read- & no doubt especially mind boggling for Brits digging themselves out of snow drifts. Your fame must be spreading amongst the Ozzie wildlife if they drop by the clinic like this.

As your night temps are a stiffling ~33°C, at least the human birth rate should be lower than normal 9 months off in November!

KIGX · Feb 4, 2009

Well, I spoke too soon. After solid communications at short range I put the Txer back out 100 m where it was before and everything has gone to pot this time. I had previously been using balanced 1s and 0s at that distance quite effectively but took it out. I'm only sending 5 data bytes now at 2400 baud but now I'm missing quite a few transmissions and sometimes the checksum doesn't work out. I guess its back to balancing the code and trying Wolgang's tricks.

Right now it's -10C and the snow is blowing all over the place. Seems like you can either freeze to death or bake to death. I have experienced 49C in california when the Santa Anas blow down off the mountains. Its not too bad cuz its so dry. You can put a dry ice acetone bath at -78C out on a bench and it won't get any frost on it... You can feel the moisture evaporating off your eyes. So, stay warm or stay cool but stay hydrated.

william47316 · Feb 3, 2011

the wireless units i made for my 8 digit display uses a header of 170's then an 85,170 plus 8 data bytes and a crude check value using the data bytes added together and divided by 8 its not nasa quality but it works quite well and picks up the major byte errors, minor errors sometimes pop through but its usually low end bit flips or shifts which dont affect the checksum too much,

at the receiver it just looks for a 170,85,170 and gets the data bytes and checksum, and does a local test and compares against the received value if they match it continues to send it out to the display if not it ignores that packet, get 20 and it tells you on the display and the display beeps at you

i've never had a problem with not having a pause between the preamble and main data, so long as there is a difference between one of the bytes in the qualifier so the qualifier doesnt bleed into the data bytes
and all this running at 4800 baud too, there have been the odd bad packets but aside from that its ~80% reliable

westaust55 · Feb 3, 2011

william_47316 said:
the wireless units i made for my 8 digit display uses a header of 170's then an 85,170 plus 8 data bytes and a crude check value using the data bytes added together and divided by 8 its not nasa quality but it works quite well and picks up the major byte errors, minor errors sometimes pop through but its usually low end bit flips or shifts which dont affect the checksum too much,

at the receiver it just looks for a 170,85,170 and gets the data bytes and checksum, and does a local test and compares against the received value if they match it continues to send it out to the display if not it ignores that packet, get 20 and it tells you on the display and the display beeps at you

i've never had a problem with not having a pause between the preamble and main data, so long as there is a difference between one of the bytes in the qualifier so the qualifier doesnt bleed into the data bytes
and all this running at 4800 baud too, there have been the odd bad packets but aside from that its ~80% reliable

“170” will not mean a lot for many folks without some though.
If it is indicated that 170 = $AA = %10101010 then it is tentatively more obvious that it is a sequence of 1’s and 0’s which give the receiving PICAXE a chance to “lock-into the data stream.
Alternatively $55 = %01010101 will work just as well.
The action of the “85” = $55 = %01010101 does provide a lot of difference. It is in reality only providing two consecutive 0’s followed by 6 alternating bits and two consecutive 1’s
Personally, if trying something in lieu of the pause , I would consider replacing the “85” with say %10001000 = $44 = 68dec to provide a more “distinctive” break.

if not it ignores that packet, get 20 and it tells you on the display

Can you clarify what “get 20” is/means.

Notwithstanding the above, you talk about the code you have developed with header, qualifier (170,85,170) data and checksum. Rather that just tell us you have done that, why do you not take the opportunity to post you code (or if already posted on this forum then include a link to it) as an example for others. Many newcomers are seeking working code.

Saying you have it is a bit like the many examples on "U-tube" where someone demonstrates a feature they have achieved but provides no real information on how they did it. Great for bragging rights but not much use to others. Then many newcomer end up coming here looking for help to emulate what they have seen elsewhere.

Pauldesign · Feb 3, 2011

If it is indicated that 170 = $AA = %10101010 then it is tentatively more obvious that it is a sequence of 1’s and 0’s which give the receiving PICAXE a chance to “lock-into the data stream.
Alternatively $55 = %01010101 will work just as well.

Technically speaking, the above patterns of 1's and 0's in any order and as long as they're balance (i.e. even number of 1's and 0's in the bit stream), possesses a crude form of Manchester coding, implemented unnoticed.

The new PICAXE series chips with built in RF feature should alleviate such issues.

hippy · Feb 3, 2011

william_47316 said:
i've never had a problem with not having a pause between the preamble and main data, so long as there is a difference between one of the bytes in the qualifier so the qualifier doesnt bleed into the data bytes
and all this running at 4800 baud too, there have been the odd bad packets but aside from that its ~80% reliable

Please see the following thread and particularly post #14 which illustrates the reasoning for the pause.

http://www.picaxeforum.co.uk/showthread.php?t=17531

Correct selection of preamble, pause and data qualifiers may make the difference between near 100% and 80% reliability.

william47316 · Feb 3, 2011

westaust55 said:
“170” will not mean a lot for many folks without some though.
If it is indicated that 170 = $AA = %10101010 then it is tentatively more obvious that it is a sequence of 1’s and 0’s which give the receiving PICAXE a chance to “lock-into the data stream.
Alternatively $55 = %01010101 will work just as well.
The action of the “85” = $55 = %01010101 does provide a lot of difference. It is in reality only providing two consecutive 0’s followed by 6 alternating bits and two consecutive 1’s
Personally, if trying something in lieu of the pause , I would consider replacing the “85” with say %10001000 = $44 = 68dec to provide a more “distinctive” break.

Can you clarify what “get 20” is/means.

Notwithstanding the above, you talk about the code you have developed with header, qualifier (170,85,170) data and checksum. Rather that just tell us you have done that, why do you not take the opportunity to post you code (or if already posted on this forum then include a link to it) as an example for others. Many newcomers are seeking working code.

Saying you have it is a bit like the many examples on "U-tube" where someone demonstrates a feature they have achieved but provides no real information on how they did it. Great for bragging rights but not much use to others. Then many newcomer end up coming here looking for help to emulate what they have seen elsewhere.

i do have the code on the forum but i have done some minor updates, i think it is floating around on the forum here

"Can you clarify what “get 20” is/means."
20 "bad packet" checks of the received serial

Code:

symbol checksum = b9
'disablebod
setfreq m8


TXloop:

serin 3,n2400, b1,b2,b3,b4,b5,b6,b7,b8

'calculate a crude checksum
w5 = b1 + b2 +b3 +b4+b5+b6+b7+b8
checksum = w5 / 8 'set a byte variable to the average value of the data input for checksum

serout 0,n2400, (85,85,85,85,85,85,85)
pause 10
serout 0, n2400,(":01",b1,b2,b3,b4,b5,b6,b7,b8,checksum) 'broadcast a quailifier to prep the receiver and burp the bytes out
pulsout 4,5000 'pulse out a confirmation LED to say its finished

goto txloop

receiver

Code:

setfreq m8
symbol checksum = b9
symbol loccheck = b12
symbol baddata = b13
high 4
mainloop:

serin 3,n2400, (":01"),b1,b2,b3,b4,b5,b6,b7,b8,checksum
w5 = b1 + b2 +b3 +b4+b5+b6+b7+b8
loccheck = w5 / 8 'set a byte variable to the average value of the data input for checksum
if loccheck = checksum then
serout 0,n2400, (b1,b2,b3,b4,b5,b6,b7,b8,#baddata)
pulsout 4,5000
end if
if loccheck <> checksum then
inc baddata
if baddata >= 20 then
serout 0,n2400, ("BAD",7,"DATA")
baddata = 0
end if
end if
goto mainloop

433 MHz wireless link trick

Senior Member

Senior Member

Senior Member

Senior Member

Senior Member

Senior Member

Moderator

Senior Member

Moderator

Moderator

New Member

Moderator

Senior Member

Senior Member

Senior Member

Member

Senior Member

Senior Member

Senior Member

Senior Member

New Member

Attachments

Senior Member

Member

New Member

Moderator

Senior Member

Technical Support

New Member