HSerial Code - Works sometimes, sometimes not...

[darosu]

Hi all again.

I am making good progress and have developed a 20X2 that can read Hserial data in, process it and send it out of a slower software serial port to my graphic LCD module. In the end this will include all of my code for icons, menus, i need for my application, without tying up my main PICAXE with lots of code and delays waiting for the GLCD to catch up...

My problem is that if i smatter it with delays it works, and without them it doesn't. My understanding is that it should work anyway as it only processes data once the HSERPTR has incremented.

The code in my recieving PIXACE 20X2 is as follows:

main:
	HSERSETUP B9600_16,%001
	gosub	startup			'LCD startup routine

		
newdatatest:	
			if HSERPTR=PTR then		'No new data has been recieved
				goto newdatatest		'Go loop until it has
			else
				pause 25			'New data is in, but wait for the data to settle?
			endif
			
								
			get PTR,TEMP1			'retieve that new byte
			inc PTR				'now increment the test byte for next test later
			
			if TEMP1<>EOT then		'if its not the end of the tranmsion then
				inc DATALENGTH		'Add 1 onto 'datalength so we know what to process later
				pause 15			'Another pause to improve reliability
				goto newdatatest		'go back and see if there is some more
			endif
			
		
								'If it was the end of tranmission, send it to LCD
		
			do
				if LCDSTAT=1 THEN exit    'Check if the LCD is ready 
			loop
	
			ptr=DATASTART					'Move pointer back to start of selected data
			
			For counter1=1 to DATALENGTH			'Loopthrou selected variables of scratchpad
				get ptr,temp1				'Get each byte
				SerOut LCDOUT, T2400_16, (temp1)	'Squirt it out to Serial
				inc ptr					'Move ptr down to next byte			
			next counter1					'Increment For/Next
	
			DATALENGTH = 0			'All data sent, reset DataLength for nect time
			DATASTART=PTR			'New DataStart to current pointer position (which is now back in the right place)
			
			goto newdatatest			'Return back to data-testing to see if we have another complete transmission to process?

With the two delays as shown in the code it works well with whatever timings the serial data is sent to it (as long as it doen;t get over 128 bytes before it can catch up)

However it will also work perfectly if i take at all delays, but space the incomming serial data with 800 x 0.25ms delay . (20 bytes making up one instruction, then delay, then next 20bytes etc...)

Is there something strange about the timings of the Harware serial? Eg a min time one has to wait after the HSERPTR has moved, or issues when you are 'GET'ting variables with the PTR while other data is coming in?

In an ideal world i wish to have no resteriction on how the data is sent from my main PICAXE and also no delays in processing.

Any comment/suggesting most welcome.

Thanks

Darosu

 

[lanternfish]

One of the things you have to remember about the GPS output is that the default settings may output a number of NMEA data strings seperated by a <CR> and <LF>.

As hserin uses the scratchpad as a serial buffer, depending on the number of NMEA data strings and their individual lengths, you may have some of the data overwritten by subsequent data.

You can tell the GPS to, say, only output GGA data. As this string is no more than 82 bytes long, you may be able to test for the <CR><LF> pair.

By turning hserin off while processing the data, and then turning it on again at the end of the processing, you effectively reset the buffer pointer to 0.

Does this make sense? nOr have I misinterpreted your question?

Cheers

 

[darosu]

I am afraid I think you might have grabbed on to the wrong end of the proverbial stick... but thanks for the reply none the less.

I am sending data myslelf from another PICAXE, not a GPS unit, therefore have full control of the data sent. However i wish to be able to send and recieve without any of the delays in my programme. All data send is terminated with as byte of '255' so that the 20X2 interpreter can know when to go and execute a command.

Currently i have to have 25 x 0.5ms before i act on new serial data in or i get a currupted output, similarly i must delay the processing speed too...

So my question really is:

1).Is there a minimum ammount of time between the Hardware Seaial Pointer moving and the data being robustly available on the scratchpad?

2). Is there any restrictions on reading the scratchpad while HSERIAL is also writing to it in another location.

nb. I can gaurentee that it is not buffer overun as I can cuase the failure before 127 bytes are sent, enven with only 40 or so.

Thanks

Darosu

 

[obroni]

Interesting it looks like you could be right about #1. It looks like hserptr moves before the entire byte is received into the scratchpad. I don't know how you can prove this however without someone who knows more about the inner workings confirming it.

Regarding #2, I don't believe their should be a problem with doing random read/writes on the scratchpad.

 

[hippy]

I don't see any evidence of hSerPtr moving before the data is written to scratchpad during testing and your code ( with and without pauses, SEROUT LCD changed to HSEROUT ) and the following seems to operate perfectly with whatever data I throw at it. My code simply echoes what's received (N9600) back out buffering data until a "Z" is seen. Functionally identical to what you are doing but with different data and optimised. I'm throwing large chunks of data at it at full speed.

Perhaps the probem is with the sender or too much data being received before you have processed it ?

#Picaxe 20X2
#Terminal 9600
#No_Data
#No_Table

Symbol dataPtr = w1

SetFreq M16
HSerSetup B9600_16,%111

HSerOut 0,( "Ready", CR, LF )
Do 
  Do
    Do : Loop until hSerPtr <> ptr
    Get ptr, b0
    ptr = ptr+1
  Loop Until b0 = "Z"
  Do
    Get dataPtr, b0
    dataPtr= dataPtr + 1 & $7F
    HserOut 0,(b0)
  Loop Until b0 = "Z"
Loop

 

[darosu]

Thanks for the responses...

I am little confused then. If i take all my pauses out of the receiving software, apart from one long one at the very beginnning just after initialising the HSerial backgorund recieve it would work... i.e. set up harware recieve, send it 100 bytes or so, split into 5 x 20bytes instructions. If it was given plenty of time for the data to be securly in then i could process it through as top speed with no errors...

maybe its my code, or could it be interference on by circuit? I have both the 40X2 running at 32mHZ sitting next to the 20X2 at 16Mhz on the same breadboard. I do have some decopling capacitors on across the power for each chip...

Darosu

 

[darosu]

OK guys - I have solved this one, so I thought I would update this post for completeness, just incase anyone else is as rubbish as me at this PICAXE melarchy.

So i tried games with running programme with and without backlight on the LCD display, thinking it was perhaps a lack of power from my PSU, but no change.

Next i added code to indicate the status of the scratchpad usage, so i have a 6 LED bargraph that shows the % usage of the buffer. If it detects a usage drop of more than 40 bytes is it is clear it has overflowed so all the lights flash and resets the LCD and the pic.

So from this although my code doesn't catch every overflow it gives a very good indication of the status of the scratchpad. This shows that everytime i get a strange results the buffer is 80% full or more. In fact it is clear it has overflowed and corrupted later data.

So I can now run at full speed with no delays in my code The only oddity is that i seem to overflow when only 100bytes or so of data has been recieved, but netherless if i stay below this it all works fine and dandy.

Thanks everyone for your help.

Darosu

 

[darosu]

OBIVIOUSLY I WRONG i haven't solved my problem.

Please Please someone help me, I am just about to pull out all by hair.

Situation:

40X2 with external 8Mhz resonator (running at 32MHZ)
HERSOUT PIN directly connected to 20X2 HSERIN PIN

Code on 40X2

#Picaxe 40X2
setfreq em32	'currently running at standard 32Mhs of 40X2

pause 3000

hsersetup B9600_32, %000
pause 15000
HSEROUT 0,("ONE TWO THREE FOUR FIVE SIX SEVEN EIGHT NINE TEN")

do 
loop

Very simple code on recieving 20X2:

#Picaxe 20X2
#Terminal 19200
setfreq m16


initialisation:
	HSERSETUP B9600_16,%001           'background recieve
	

main:	
		if HSERPTR=PTR then		'No new data has been recieved
		           goto main		'Go loop until it has
		
		endif
			
		'pause 100		'SEE TEXT BELOW REGARDING THIS PAUSE!
			
                get PTR,TEMP1			'retieve that new byte
			sertxd ("POINTER POS#",#PTR," = ",TEMP1,CR,LF)
			inc PTR				
			
			goto main

So this is what happens:

If pause of 100 or greater is included on 20X" for every byte sent to PC then 100% success
Any less than 'pause 100' any success drops very quickly.

With no pause I can get success at baud rates of 300 and 1200. Corruption at 2400 and above

Modifying the running frequency from 4, 8, 16, 32Mhz of 20X2 no difference

At 16Mhz, 9600 baud rate, Calibfreq -12,-8,-4,4,8,12 no effect.

Does anyone have any suggestions? All i want to do is send data at 9600 from one PICAXE to another without half of it getting lost in the wash...

Thanks

Darosu

 

[tarzan]

This code may help you.
http://www.picaxeforum.co.uk/showpost.php?p=112962&postcount=1

 

[hippy]

I think I've found the problem.

Your SERTXD in the 20X2 receiving loop is taking too long. I'm guessing that as it's bit-banged serial it has to disable interrupts to get its timing right and that's causing the problem. Replace it with SERTXD(TEMP1) and it's fine. Replace it with a long sequence of SERTXD each of which only outputs one byte and it's fine ...

sertxd("P")
sertxd("O")
sertxd("I")
sertxd("N")
sertxd("T")
sertxd("E")
sertxd("R")
sertxd(" ")
sertxd("P")
sertxd("O")
sertxd("S")
sertxd("#")
sertxd(" ")
sertxd(#PTR)
sertxd(" ")
sertxd("=")
sertxd(" ")
sertxd(TEMP1)
sertxd(CR)
sertxd(LF)

 

[hippy]

And ... the reason it worked with a PAUSE 100 in the 20X2 was because that allows it to receive all bytes before outputting them. Without the PAUSE the 20X2 is outputting while also still having to recieve.

 

[hippy]

The optimised version of your 'main:' would be ( couldn't be bothered to type in all those SERTXD's ) ...

main:
  Do
    Do : Loop Until hSerPtr <> ptr
    Do
       SerTxd( #ptr )
       SerTxd( "=" )
       SerTxd( @ptrinc )
       SerTxd( CR )
       SerTxd ( LF )
    Loop Until hSerPtr = ptr
  Loop

 

[darosu]

H I P P Y

Some excellent late night detective work. Thank you very much.

So my proposed method is not going to work then? Ideally i wanted to recieve serial information at 9600 (or at least a good speed) so my main chip can spirt out graphics information whenever it fancied and go get on with what it needs to.

The 20X2 recieving chip would store this in its buffer and play catchup sending at 2400 to my GLCD and waiting for the LCD status line was ready before sending it the next command.

If i am using SEROUT at 2400, even if i keep each command to a single byte it is likely to mess with incomming serial data?

Do we have anymore info on this clash of timing or a way round it.

I guess i could SEND and RECIEVE at 2400 using HARDWARE serial and this would be better than dropped packets of info (but this would tie up my sending chip for up to half a second (128bytes).

Thanks again Hippy

Darosu

 

[hippy]

If i am using SEROUT at 2400, even if i keep each command to a single byte it is likely to mess with incomming serial data?

It would seem so. Under-clocking the 20X2 so it SERTXD's at 2400 baud and corruption is present ...

#Picaxe 20X2
#Terminal 2400
#No_Data
#No_Table

Symbol B9600_2 = 51

SetFreq M2
HSerSetup B9600_2, %001

Do
  Do : Loop Until hSerPtr <> ptr
  Do
   SerTxd( @ptrinc )
  Loop Until hSerPtr = ptr
Loop

Do we have anymore info on this clash of timing

"It's the laws of physics, captain, ye canna change that"; at 2400 baud each byte sent takes 4167us and for that period interrupts need to be locked-out to ensure correct timing. At 9600 baud, bytes can be arriving every 1042us, so you could potentially lose 4 bytes while sending a 2400 baud byte.

The same issue arises at 4800 baud ( 2083us per byte ) and you could lose two 9600 bytes during that time. However, the receive UART hardware has double-buffering which allows one byte to have been 'missed' while another is still being received and still be able to retrieve those two bytes of data.

I suspect it was fluke the modified 4800 baud code worked. As ptr gets higher, SERTXD(#ptr) sends more bytes, more chance of collision, but that is offset by sending so much, taking so long, that the buffer was filled long before that could happen. If the sender sent more or we more quickly processed the data we would likely see the 4800 version start to corrupt.

Do we have ... a way round it.

Only by system design. Any baud rate changer will potentially have a problem when TX baud rate is half or less than RX baud rate.

You can drop the RX baud rate or buffer up data and then churn out the data. The latter is perhaps best as it exchanges the time taken to send data to the time between data sent. Either way you have to give the TX enough time to do its job before sending more data.

You can send a packet of data, wait, then send another, or you could pace the sending of each byte, so after each byte the converter is given enough time to have sent that byte.

There's going to have to be delays anyway or sending faster than the converter will put out will overflow the receive buffer.

For a dedicated microcontroller designed for baud rate conversion it's not so much a problem because one has complete control and it can be designed to handle the possibility of data being received during transmission, and yes or no, it can still keep the output bit-banging timing the same and correct. Though problems will arise if the data arrival handling time exceeds the bit-banged timing requirements. The same is true for any command which has required bit-banged timing and timing can be so tight it's not possible to do all the things necessary without some conflict in operation.

Buffering is the best way to to go and end of packet detection can be achieved without excessive delays, and handshaking to tell the transmitter it's 'Clear To Send' can be added to minimise delays ...

#Picaxe 20X2
#Terminal 2400
#No_Data
#No_Table

Symbol B9600_2 = 51

Symbol lastHSerPtr = w0

Symbol CTS = C.1 ' Leg 9 on 20X2 ( to LED or 40X2 )

SetFreq M2
HSerSetup B9600_2, %001

Do
  High CTS
  Do : Loop Until hSerPtr <> ptr
  Low CTS
  Do
    lastHSerPtr = hSerPtr
    Pause 1
  Loop Until lastHSerPtr = hSerPtr
  Do
    SerTxd( #ptr, "=", @ptrinc, CR, LF )
  Loop Until ptr = lastHSerPtr
  SerTxd( "================", CR, LF )
Loop

#Picaxe 40X2
#No_Data
#No_Table

Symbol CTS_IN = pinC.0 ( leg 18 on 40X2, leg 11 on 28X2 )

SetFreq EM32

HSerSetup B9600_32, %000

Do
  HSerOut 0,( "ONE TWO THREE FOUR FIVE SIX" )
  Pause 16000  
Loop

' Or ...

Do
  Do : Loop Until CTS_IN = 1
  HSerOut 0,( "ONE TWO THREE FOUR FIVE SIX" )
Loop

 

[darosu]

OK, looks like a plan is coming together...

The system is compounded in complexity by the fact that my 40X2 sender chip will also be recieving background hardware serial at 9600 from RFID chips. These cannot be told when to send as it sends as soon as it detects something.

This means that the data leaving my 40X2 must either:

1). leave very fast by SEROUT, but the max would be 38400 baud at 32MHz, therefore will be limited to 4 bytes before i have missed incoming data

2). leave by HSEROUT at 9600 (Baud dicatated by RFID).
Question: Are there any issues with HSEROUT effecting background hserin in? any clashing? I hope not as I think this is my best option


The 20X2 graphics chip will follow your Clear To Send approach, and will also allow the 40X2 permission to send whenever it is waiting for the GLCD to finish what it is doing, checking all data is in before moving on.

So if there are no issue with HSEROUT vs background HSERIN then I am good.

Thanks for all the advice

Darosu

 

[hippy]

There should be no issues with HSEROUT within a 'HSERIN' background receive loop, nor with SERTXD / SEROUT with a baud rate equal or faster than data received. You may not be able to send multiple bytes within a single SERTXD / SEROUT command, ie ...

SERTXD( b0, b1, b2, b3 )

but ...

SERTXD( b0 )
SERTXD( b1 )
SERTXD( b2 )
SERTXD( b3 )

should be okay, no data lost, missed or corrupted.

Although incoming RFID data may be arriving at any time, presumably there will be periods in between when there is no data at all. Buffering up received RFID data then processing it and passing it along within those gaps ( by any means ) would work.