20x2 scratch pad size

ol boy

Member
Is there a way to get more than 128 bytes for holding hser input data? I want to build a data logger to receive data from a ucontroller and store it to a SD Card. The incoming data size is 208 bytes. I only need about 50 bytes of that whole string of data. Can this be accomplished with a 20x2 or will I need to use a 28x2 instead?

Thanks Ryan
 

Goeytex

Senior Member
It depends upon how fast the data arrives. Do all 206 bytes come in at once? At what data rate ?

The scratchpad can be set up as a ring buffer and received data can be moved on the fly to available ram memory, that can hold the extra 78 bytes.
 

ol boy

Member
Yes all 208 bytes come in at once. 115200 baud rate. I might just go to the 28x2 for this exercise. How would the ring buffer work?

Thanks Ryan
 

Goeytex

Senior Member
115,200 is probably too fast for the Picaxe 20X2 keep up. I'll test it here tonight and see.

Basically a ring buffer is set up by comparing the values of ptr and hserptr. If they are not equal then byte are waiting to be processed . (Hserptr will be greater than PTR zero when it raps from 127 to zero). PTR can lag behind hserptr by no more than 128 bytes. So if too many bytes come in too fast, data will be lost. Processing overhead is the real killer here. Here's an example.

Code:
#Picaxe 20X2
#no_table
Setfreq  M64   '// Wide open for faster processing.  

Hsersetup B115200_64, %001
setintflags %00100000,%00100000  '// interrupt on hserial receive   

MAIN:

do
        '// waiting for hser background data  
loop


Interrupt: 
   do while ptr <> hserptr    '//  Ring buffer
     sertxd (@ptrinc," ")     '//  Process byte & increment ptr
   loop     ' //  until ptr = hserptr
 
 flags = 0                            ' // Clear the hserint flag
 setintflags %00100000,%00100000   ' // re enable interrupt on serial receive 
 return       '// Back to main
 

AllyCat

Senior Member
Hi,

If the 208 bytes are tightly packed at 115200 baud (with just one or two stop bits), then that very probably is too fast for a "conventional" 128 byte circular buffer in a PICaxe.

The basic calculation is that the data packet might take only 18 ms and you must process 80 bytes in that time to prevent an over-run. That's 225 us per byte, which "normalised" from a 64 MHz to a 4 MHz clock (for which some timing data is documented) becomes about 3,6 ms. A GOSUB .. RETURN takes nearly that long (I don't have data for an interrupt) or about 3 - 4 "typical" PICaxe Basic instructions. However, the @BPTRINC contruct is quite fast (for a PICaxe), so IMHO the best (possibly only) command worth attempting to empty the buffer is @BPTRINC = @PTRINC.

But if you are sure that it will always receive a packet of exactly 208 bytes, then you could process multiple bytes in the interrupt routine, perhaps two or eight at a time, or in the extreme 208. Then, if the ptr and hserptr were found to be no longer in sync, you would have to junk that packet of data and start a "recovery" routine.

Cheers, Alan.
 

ol boy

Member
Thanks

I'll have to rig up a test bread board to see if I can recieve the 208 byte pyload then send it back out the hserout pin. I've played with the @bptrinc and @ptr commands in the simulator to transfer 10 bytes from the scratch pad over to a set of RAM address in sequence. I forgot that the 20x2 only has 128 bytes of RAM to work with. I might just have to use a 28x2 so I can copy the hole scratch pad over to RAM then pick which bytes I want to send to open log uSD card widget.

I'll have to look at the com spec for the payload to see if I want any of the data after byte 127. How would I keep the ptr from rolling back to location 0 and just dump 128 to 208 bytes into location 127?

Thanks Ryan
 

Goeytex

Senior Member
I forgot that the 20x2 only has 128 bytes of RAM to work with. I might just have to use a 28x2 so I can copy the hole scratch pad over to RAM then pick which bytes I want to send to open log uSD card widget.
If using a 28X2, why do you need to move the bytes to user ram? The data can be sent directly from the scratchpad.

Q: How often does the 208 byte packet arrive?

How would I keep the ptr from rolling back to location 0 and just dump 128 to 208 bytes into location 127?
I am trying to understand what you are asking here but it doesn't make sense. You can't dump 128 bytes into a single location.
On the 20X2 SP and user ram, location 127 is the last location. The next location will be location 0.

With the scratchpad and background serial, hserptr points to the location where the next received byte is placed. It is automatically incremented after each received byte. If the hserptr started at zero, after 128 bytes have been received, the hserptr will be automatically incremented from 127 to 0. The next received byte will be placed in SP location 0 and will overwrite any data in that location with the new data. PTR will only roll over to zero if it is told to increment in the program.

Now ..... if you "preload" hserptr to location 48, The SP will receive 80 bytes starting at location 48 and ending at location 127. The hserptr will then be pointing to location 0 which has not yet been written. If we quickly move these first 80 bytes to user ram on-the-fly, then the next 128 bytes can go into the scratchpad starting at location 0 and ending at location 127. We are not overwriting anything because the first 80 bytes were copied into user ram.

So at this point the first 80 bytes are in user ram (BPTR) at locations 0 - 79. The next 128 bytes are in the scratchpad (PTR) at locations 0 - 128.

The code is tricky and must be timed properly. I have been playing around with this and have it working on a 20X2 at 9600 baud will a lot of room for increased speed. I'll know later tonight or tomorrow if it can work at 115200.

It is very important that you tell me how often the 208 byte packets arrive. ( and any other details that you have left out).
 

ol boy

Member
I'm using a Megasquirt2 ECU in my 64 For Falcon and the one feature the ECU lacks is on board data logging. I have added a bluetooth adapter to the UART lines on the ECU processor. There is an app called MSDroid that can poke (sends an A ascii char) into the ECU which in return dumps all 208 bytes from the outputpc array. These values are the most current values in the ECU at that time. As I'm thinking about it the total serial string will be longer than 208 bytes as there is other data sent, header data, and CRC16 check sum at the end. As was mentioned it take about 18ms to send 208 bytes at 115200 buad rate. I've scoped this in the past. As soon as the android app receives the last byte it pokes the ecu for the next dump of data. I've seen as high as 21 reads per second.

As for a simple data logger... if I could send out an ascii A to the ecu and receive the data dump then pick which set or range of data I want to store the openlog sd card. A data sample rate of about 5 hz is fine for me. Once the data is received and picked through and sent off to the sd card we can then request the next sample. Can I use the programming port output to send the A command and then use the hserout to send data off to the SD card widget?

I really need to read up on all the pointers and play around with the simulator. PE6 serial simulator is nice to have for testing.

Thanks for your help.
 

Goeytex

Senior Member
With your application, it makes no sense to twiddle about with a 20X2 and try to make it jump through hoops, when a 28x2 will do the job in a straight forward manner. You need to use a 28X2.

Get a 28X2 and I will help you with the code if you need help.
 

rossko57

Senior Member
... and CRC16 check sum at the end.
As that is available, it'd be daft not to verify the CRC on rxd data
That implies calaculating CRC on the fly (more overhead) - or not discarding data before whole packet received. Looking like a 28X2 job really.
 

srnet

Senior Member
As that is available, it'd be daft not to verify the CRC on rxd data
That implies calaculating CRC on the fly (more overhead) - or not discarding data before whole packet received. Looking like a 28X2 job really.
I used a CRC16 checksum for the HABAXE project.

Its calculated on the fly when manually sending RTTY at 200baud, the routine takes 10.5mS or so at 16mhz, but this is equivalent to two serial stop bits at 200baud, so the delay whilst the CRC is being calculated does not add any time overall.
 

hippy

Ex-Staff (retired)
As the MegaSquirt is open source; one option may be to modify the data sent from the ECU to include only the data you want to receive, or add that as an additional command response. It may be enough to simply limit the amount of data sent to 128 bytes or less.

The easiest option would seem to be to use a 28X2 rather than struggling to use a 20X2 or modifying the ECU.
 

ol boy

Member
Thanks for the quick responses. Changing the source code with in the ms2 is something I have thought about but that would mess up the way the tunning software would function. It uses all the incoming bytes to calculate other functions to aide in the tuning process.

In the mean time could uses the 20x2 in my transmission controller as a test setup and have it blast out the hand full of bytes to the SD logger to make sure the CSV file is created correctly.

With the openlog widget running on arduino base c software I suppose it could handle the action of requesting the data from the ms2 and then stripping the unwanted data then write the wanted data to the SD card....

I've seen a few projects in the forum that are sending data to this openlog widget. How come know one is using the spi to write directly to an SD card? What's the limiting factor in a picaxe based SD logger... Axelogger!
 

hippy

Ex-Staff (retired)
I've seen a few projects in the forum that are sending data to this openlog widget. How come know one is using the spi to write directly to an SD card? What's the limiting factor in a picaxe based SD logger... Axelogger!
The main problem is making it work. That requires an understanding of MMC/SD/SDHC card hardware and protocols, having the skills to do it plus the time to design, debug, test, and make it work robustly.

In theory it should be 'easy enough' and some work has been done interfacing MMC/SD/SDHC direct to PICAXE, but that doesn't mean it is necessarily plain sailing as there's a fair amount that needs to be done and done right and a lot which can go wrong. If something doesn't work as expected that has to be resolved which requires research and most importantly time.

Doing it, investing that time and effort and usefulness of it all, has to be set against simply using a ready made SD data logger module.

http://www.picaxeforum.co.uk/showthread.php?14881-Reading-text-files-from-an-SD-card
 

Goeytex

Senior Member
I've got this working at 115200 baud , on a 20X2 , where the first 80 bytes go to user ram and
the next 128 bytes remain in the scratch pad.

If you want the code. I will clean it up and post it. Let me know.

Goey
 

Goeytex

Senior Member
As promised. It works quite well.

I was doubtful that the 20X2 had the speed to capture and process the data at 115200 baud. But it has more than enough with some room to spare.

Code:
'// **********************************************************
'// * Data Logger Using Hser & User Ram & Scratchpad         *
'// * to store 208 bytes received from ECU                   *
'// ***********************************************************

#picaxe 20X2
#no_table
#terminal 38400
setfreq M32
pause 2000  '// Stabilize 

symbol cntr = b4  'counter for loop timer 
symbol error_flag = b5

         
sertxd ("Starting",cr,lf) '//  for debugging only

pause 16000 '// for debugging only  remove in final code 
   
   hsersetup B115200_64, %001
  
   setfreq m64
  
   hserptr = 48 '// preload herial pointer
   ptr  = 48    '// preload SP location pointer
   bptr = 48    '// preload User ram pointer
                '// these will overflow to zero after 80 bytes  
   
   error_flag = 0 '// just in case 
    
  MAIN:
  
    Do
    
         Gosub request_data
         pause 16000 '// every 2 seconds @ 64 Mhz    
        
    Loop       
     

Request_data:  
  
  error_flag = 1    '// data requested
  
  setintflags %00100000,%00100000
  cntr = 0
  
  hserout 0,("A") '//  Send "A" to tell the ECU to send data packet
     
    
     '// Wait for interrupt on hser receive
     '// If no interrupt after 20 counts then
     '// ECU did not respond. 
      
       
         do 
           inc cntr : pause 1
            if cntr >= 20 then
               error_flag = 3
            endif     
            
            if error_flag = 3 then gosub error_message
            if error_flag = 2 then gosub process_data
            if error_flag = 0 then exit
         loop
               
 
return '// to main loop
  

Process_Data:
     
     '//do  whatever you want here with data now captured
     '// in User Ram and SP Ram. 
     
     'access User ram with "peek" or "bptr / @bptr
     'access SP   ram with "get"  or "ptr" / @ptr
     
     setfreq m32     '//  so we can use the terminal at 38400
          
     '// User RAM (bptr) 
     for bptr = 48 to 127            ;Get  80 bytes from user ram starting at loc 48
         sertxd ("BPTR ",#bptr," = ", #@bptr,cr,lf)    ;display the bytes
     next   
       
     '// Scratchpad  (PTR)    
     for ptr = 0 to 127      '// Get 128 bytes from  scratchpad memory 
          sertxd ("PTR ",#ptr," = ",#@ptr,cr,lf)  '// Display the bytes
     next        
      
   setfreq m64  'back to 64 MHz for fast processing
   
   '// reset pointers for next packet   
   hserptr = 48
   ptr  = 48     
   bptr = 48
   error_flag = 0 '// Data was processed all clear 
   return    '// returns to main 
  

Interrupt:
'// -------------------------------------------------------
         
    '//  Let a few bytes arrive before processing so we don't
    '// get ahead of the incoming data. 
     pause 4
       
   
  '// store the first 80  bytes in user ram 44 - 127  
    do until ptr =  0   
        @bptrinc = @ptrinc
    loop
            
    '// The next 128 bytes go into the scratchpad 
     pause 200 '//   > 15ms @ 64Mhz  make sure all bytes received
   
   flags = 0                         '// clear interrupt flag 
   setintflags  %00100000,%00100000  '// enable the interruot 
   setfreq m64
   error_flag = 2  '// Data received 
   cntr = 0        '//clear the counter 
   return          '// returns to request data 
'// -------------------------------------------------------

    
 Error_Message:
'// --------------------------------------
 
 setfreq m32
 sertxd ("No Response From ECU",cr,lf)
 setfreq m64
 error_flag = 0   '// clear the flag
 return
'// --------------------------------------
This is the code used to send the 208 bytes, using a 08M2 to simulate the ECU.
Code:
'===================================
#Picaxe 08M2
#No_Data
Setfreq M32
Pause 100
'=====================================

high C.0 
pause 1000

hsersetup B115200_32, %000

MAIN:

w1 = $FFFF ; set up a non-valid value

 do 
  
     hserin w1 ; receive 1 byte into w1
     if w1 <> $FFFF then   
           If w1 = "A" then
           w1 = $FFFF
            gosub send_data 
           endif
     endif      
    pause 10    
    
 loop
     
Send_data:
'------------------------------------------------------------------

  '// This is 208 bytes
  
hserout 0, (255,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20, _
  21,22,23,24,25,26,27,28,29,30,32,32,33,34,35,36,37,38,39,40, _
  41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60, _
  61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80, _
  81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100, _
  101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120, _
  121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140, _
  141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160, _
  161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180, _
  181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200, _
  201,202,203,204,205,206,255)  
pause 500

goto main
 
Last edited:
Top