Another "discrepancy" between the Simulator and a real chip

AllyCat

Senior Member
Hi,

The background to this thread is the programs that I wrote in response to this recent enquiry.

The OP wanted to sort the bytes stored in five consecutive variables, but it seemed to me that the "correct" method would be to write generic code, using values accessed via the byte pointer (@bptr). Also that it could be quite "elegant" and compact to use the post-increment/decrement facility (@bptrinc....). Two versions are shown in that thread and appeared to simulate correctly.

However, some further simulations seemed to acquire a spurious "0", only when sorting the first set of digits. This is not yet resolved, but I decided that the FOR bptr with subsequent bptrinc/decs might be "a step too far", so I returned to a simpler DO ... LOOP WHILE ... structure. Thus my "final" program became:

Code:
#terminal 4800
#no_data

pause 2000            ; Give the terminal time to open
symbol FIRST = 2       ; Pointer to byte variables
symbol MIDDLE = 4
symbol LAST = 6

	sertxd("Start",cr,lf)       ; Confirm the program has started
do
	bptr = FIRST
	for bptr = FIRST to LAST         ; Create and Report random list
		random w4
		@bptr = w4 // 10		     ; Select the range for Random numbers
		sertxd(#@bptr," ")
	next
	sertxd(cr,lf)

sort:
	 bptr = FIRST
	do     
		if @bptrinc > @bptr then               ; Exchange @bptr with @bptr+1
;//		if @bptr < @bptrinc then               ; This is the structure required for real chips  //
			@bptr = @bptrdec xor @bptr	; )
			@bptr = @bptrinc xor @bptr	; } Swap without using an additional variable
			@bptr = @bptrdec xor @bptr	; )	
			goto sort
		endif
	loop while bptr < LAST
 
	for bptr = FIRST to LAST            ; Report final list
		sertxd(#@bptr," ")
	next
 
	bptr = MIDDLE
	sertxd(cr,lf,"Middle Value is ",#@bptr,cr,lf)
loop
But when downloaded to a real chip, the program didn't run at all. :(

To cut a long story short(er), the syntax required by the real chips is IF @bptr < @bptrinc THEN . Note not just the post-increment in the "wrong" place (for strict left to right execution) but the change of > to < . To me, it looks as if it's the Compiler that is "incorrect" and the Simulator is more to specification/expectations. :eek:

Any comments (except that I should write simpler program code) ?

Cheers, Alan.
 

johnlong

Senior Member
Hi Alan
Please do not start writing simpler coding
I do enjoy the what the hell--well bloody hell --- bugger my rags I didnt even think of that

regards
john
 
Last edited:

AllyCat

Senior Member
Hi,

Thanks for your kind words. I did want to persevere with the post-INC/DECs because they don't appear to add any "overhead" in terms of program bytes or execution time, relative to the basic @bptr variable/commands. I've updated the other thread with additional code and some interesting links; the execution time is almost half of the equivalent "conventional" algorithm and significantly faster than hippy's improved version. However, I had to go back and update that thread yet again because I'd used an @bptrdec = which worked perfectly in the PE5 and PE6 simulators and the PE6 downloader, but I later discovered NOT in a PE5 download.

A problem with debugging such code (if it just stops producing an output) is that it is difficult to determine what's happening (even in the simulator) when the bptr changes at some (unknown) point within a command. For example, if the bptr doesn't move (no inc/dec), then the < or > always fails, no swap takes place and the program hangs endlessly. I thought I might "trap" the = condition, but that can occur correctly if two adjacent bytes are the same, and of course swapping identical values (which don't need to be swapped) is a likely way to produce endlessly running code.

Then I had a "brainwave" that the IF .. ENDIF and IF .. GOTO contructs behave quite differently, so maybe the GOTO would work more "as expected". Well, the GOTO does behave differently, but only in that I haven't managed to make that structure work at all. :( However, in any case, I suspect that Rev Ed will not "fix" the "unexpected" behaviour, but only trap it as a "syntax error".

So I've taken the pragmatic view that it's probably not normally "sensible" to include more than one post-INC/DEC within a single command (line), and probably none where a conditional test is involved. The appeal of my original program was that it was fast and needed no temporary variables (b0 upwards), but in practice, adding two variables helps to speed up the algorithm even more (because there is less shuffling of the bptr) and of course it's easier to debug.

So, another entertaining excursion and I now have a sorting algorithm that is faster than even hippy's offering (albeit that it was 11 years ago). ;) I'll update one of the existing threads that I've previously linked, but haven't decided which is the more appropriate yet.

Cheers, Alan.
 

hippy

Technical Support
Staff member
To cut a long story short(er), the syntax required by the real chips is IF @bptr < @bptrinc THEN . Note not just the post-increment in the "wrong" place (for strict left to right execution) but the change of > to < . To me, it looks as if it's the Compiler that is "incorrect" and the Simulator is more to specification/expectations. :eek:
You are correct; the simulator behaves as one would expect it to. The compiler is also correct and the issue is a historical one which relates to how the firmware is and how execution code is stored.

The conditional "a > b" is held in the execution code as "b,a,>". In most cases the ordering of the two operands is not important but where '@bptrinc', '@bptrdec', '@ptrinc' and '@ptrdec' are used this ordering issue affects when the increment or decrement occurs.

For example "@bptrinc > @bptr"' is held in the execution stream as "@bptr,@bptrinc,>" so, when executed in sequence, the '@bptr' is executed before the '@bptrinc'.

By rewriting your code as "@bptr < @bptrinc" this generates the execution stream of "@bptrinc,@bptr,<" which has the same functionality but ensures the '@bptrinc' occurs before the '@bptr' is accessed a second time. While that executes as expected it will no longer simulate with the same result.

We will investigate the issue further and determine how best to resolve it.

In the meantime we would recommend, although slower, assigning variables which are then used in comparisons from '@...inc' or '@...dec' variables rather than using those directly within comparison statements. This will allow code to be simulated and executed as expected.

The alternative is to use #IFDEF to ensure correct simulation and execution, for example -

Code:
#IfDef SIMULATING
  If @bptrinc > @bptr Then
#Else
  If @bptr < @bptrInc Then
#EndIf
    :
  End If
That will however get complicated and the code will almost certainly have to change again when this issue has been resolved. Using -

Code:
Let bTemp = @bPtrInc
If bTemp > @bPtr Then
  :
End If
Will execute and simulate correctly, and will be correct in the future.
 

AllyCat

Senior Member
Hi hippy,

In the meantime we would recommend, although slower, assigning variables which are then used in comparisons .
Thanks for the explanation. Actually, the "temporary variables" approach proves to be faster, so I didn't need too much persuading to adopt that; The "resident" SWAP (pseudo command) is notoriously slow, the "triple XOR" trick somewhat faster and the "brute force" method with a single temporary variable is faster still. But using two separate variables is even faster, because the two "swapped" values can be written almost directly with just one inc or dec of the bptr.

However, there appears to be something else "strange" happening. I commented above that the "first execution" of the algorithm appeared to sometimes fail inexplicably, but I couldn't find a reason. However, in a more recent version of the program, I happened to allocate the byte immediately below the "data list" and discovered that it occasionally became corrupted. Although the algorithm may sometimes set the bptr to that location, it should never write to it.

Again to cut a long story short, I put a "trap" immediately after the IF @bptr < @bptrinc THEN .. to check if the pointer had strayed to the location below the data list. The full program was testing execution speed by repeatedly performing a group of 1000 sorts (loaded with randomised numbers). The first group of 1000 triggered the trap 6 times, the second none and the third once. But from then on, the sorting algorithm appeared to work perfectly. Furthermore this was repeatable; restart the program and again a sequence of 6, 0, 1 and no further errors. So what can cause a program to suffer very occasional "errors" for up to a minute or two, and then to work normally? I guess it might be an obscure bug in an interrupt handler, but I can't see why that might "fade away" after a few minutes. :confused:

My full program is too long and obscure to be worth posting now, and at the moment I've got sidetracked by the operation of the RANDOM function, which might be the cause of the issue and again doesn't behave "as (I) expected".

Cheers, Alan.
 

AllyCat

Senior Member
Hi,

Here is a "tidied up" version of the Program that I described in the previous post. The original version was intended to compare three sorting algorithms, timing them for lengths of 2 to 16 bytes, but I've removed the other algorithms and timing data, etc.. It performs a "repeats" number of sorts for each data-length, printing the last pass as a "sanity check" and a "#" character if the bptr is not where it was expected to be (pointing to the second byte in the byte-swap code).

The surprising thing (at least to me) is that it still reports "errors" at similar times during the execution, in a real PICaxe and the PE6 simulator. So in principle it should be possible to determine what's happening, but I'm not very familiar with the PE6 simulator, because the PE5 version is so much faster (at least on my PCs). Also, there are a great many iterations for the full code, so it's fortuante that the "errors" predominate very near the start.

Maybe the issue is related to the structure of my sorting algorithm which uses a hybrid arrangement of IFs and LOOPs. In particular, I can't see how to set up the indenting "correctly" (because the LOOP tests are out of sequence).

Code:
[color=Green]; Ascending sort of Random Numbers.  AllyCAt August 2017
; Occasional "errors" in bptr handling (#) in PE6 simulator and real 20M2 (PE5/6)[/color]
[color=Navy]#no_data
#terminal [b]4800[/b][/color]
[color=Blue]pause [/color][color=Navy][b]5000[/b][/color]

[color=Blue]Symbol [/color][color=Purple]Var1 [/color][color=DarkCyan]= [/color][color=Purple]b26[/color]
[color=Blue]Symbol [/color][color=Purple]Var2 [/color][color=DarkCyan]= [/color][color=Purple]b27[/color]
[color=Blue]symbol [/color][color=Purple]pass [/color][color=DarkCyan]= [/color][color=Purple]w10[/color]
[color=Blue]symbol [/color][color=Purple]repeats [/color][color=DarkCyan]= [/color][color=Purple]w11[/color]
[color=Blue]symbol [/color][color=Purple]seed [/color][color=DarkCyan]= [/color][color=Purple]w12[/color]
[color=Blue]symbol FIRST [/color][color=DarkCyan]= [/color][color=Navy][b]1[/b][/color]
[color=Blue]symbol [/color][color=Purple]last [/color][color=DarkCyan]= [/color][color=Purple]b19

   seed [/color][color=DarkCyan]= [/color][color=Navy][b]1
   [/b][/color][color=Blue]sertxd(cr[/color][color=Black],[/color][color=Blue]lf[/color][color=Black],[/color][color=Red]"Seed= "[/color][color=Black],#[/color][color=Purple]seed[/color][color=Blue])
   [/color][color=Purple]b0 [/color][color=DarkCyan]= [/color][color=Navy][b]100          [/b][/color][color=Green]; ** This affects the "errors" **[/color]
[color=Blue]for [/color][color=Purple]last [/color][color=DarkCyan]= [/color][color=Navy][b]2 [/b][/color][color=Blue]to [/color][color=Navy][b]16

   [/b][/color][color=Purple]repeats [/color][color=DarkCyan]= [/color][color=Navy][b]25      [/b][/color][color=Green]; 250 or larger for a "serious" timing test[/color]
[color=Blue]sertxd(cr[/color][color=Black],[/color][color=Blue]lf[/color][color=Black],[/color][color=Red]"Go "[/color][color=Blue])
sertxd([/color][color=Black]#[/color][color=Purple]Last[/color][color=Black],[/color][color=Red]"= "[/color][color=Blue])

   for [/color][color=Purple]pass [/color][color=DarkCyan]= [/color][color=Navy][b]1 [/b][/color][color=Blue]to [/color][color=Purple]repeats
      [/color][color=Blue]call [/color][color=Black]load                  [/color][color=Green]; Load a list of Random numbers
      [/color][color=Purple]bptr [/color][color=DarkCyan]= [/color][color=Blue]FIRST
      do 
         do[/color]
[color=Black]sorti:[/color]
[color=Navy]#ifdef [/color][color=Blue]simulating
            if [/color][color=Purple]@bptrinc [/color][color=DarkCyan]> [/color][color=Purple]@bptr [/color][color=Blue]then[/color]
[color=Navy][u]#else
            [/u][/color][color=Blue]if [/color][color=Purple]@bptr [/color][color=DarkCyan]< [/color][color=Purple]@bptrinc [/color][color=Blue]then [/color]
[color=Navy][u]#endif   [/u][/color]
[color=Blue]if [/color][color=Purple]bptr [/color][color=DarkCyan]= [/color][color=Blue]FIRST then [/color][color=Black]: [/color][color=Blue]sertxd([/color][color=Red]"#"[/color][color=Blue]) [/color][color=Black]: [/color][color=Blue]endif   [/color][color=Green]; Report error if bptr hasn't incremented
               [/color][color=Purple]@bptr [/color][color=DarkCyan]= [/color][color=Purple]@bptrdec [/color][color=DarkCyan]xor [/color][color=Purple]@bptr       [/color][color=Green];  ; bptr moves to 1
               [/color][color=Purple]@bptr [/color][color=DarkCyan]= [/color][color=Purple]@bptrinc [/color][color=DarkCyan]xor [/color][color=Purple]@bptr    [/color][color=Green]; bptr = 2
               [/color][color=Purple]@bptr [/color][color=DarkCyan]= [/color][color=Purple]@bptrdec [/color][color=DarkCyan]xor [/color][color=Purple]@bptr    [/color][color=Green]; bptr = 1     0 ok with PE6 (@bptrdec = ..)
               [/color][color=Blue]dec [/color][color=Purple]bptr                      [/color][color=Green]; bptr = 0      Required for PE5 compiler
         [/color][color=Blue][u]loop [/u]while [/color][color=Purple]bptr [/color][color=DarkCyan]=> [/color][color=Blue]first[/color]
[color=Green];;    if bptr => first then  ;sorti       ; Alternative GOTO version
            [/color][color=Blue]endif
      [u]loop [/u]while [/color][color=Purple]bptr [/color][color=DarkCyan]< [/color][color=Purple]last

   [/color][color=Blue][u]next [/u][/color][color=Purple]pass
   [/color][color=Blue]call [/color][color=Black]report [/color]

[color=Blue][u]next [/u][/color][color=Purple]last
   [/color][color=Blue]sertxd(cr[/color][color=Black],[/color][color=Blue]lf[/color][color=Black],[/color][color=Red]"Done"[/color][color=Blue])
   stop[/color]

[color=Black]report:
   [/color][color=Blue]for [/color][color=Purple]bptr [/color][color=DarkCyan]= [/color][color=Blue]FIRST to [/color][color=Purple]last
      [/color][color=Blue]sertxd([/color][color=Red]" "[/color][color=Black],#[/color][color=Purple]@bptr[/color][color=Blue])
   next [/color][color=Purple]bptr      
   [/color][color=Blue]return
 [/color]
[color=Black]load:
   [/color][color=Blue]for [/color][color=Purple]bptr [/color][color=DarkCyan]= [/color][color=Blue]FIRST to [/color][color=Purple]last
      [/color][color=Blue]random [/color][color=Purple]seed 
      @bptr [/color][color=DarkCyan]= [/color][color=Purple]seed [/color][color=Green];  // 20
   [/color][color=Blue]next [/color][color=Purple]bptr[/color]
[color=Blue]return[/color]

[color=Green]#rem
Output with repeats = 25:

Seed= 1
Go 2= ## 43 149
Go 3= ## 27 54 109
Go 4= # 113 143 199 227
Go 5=  29 58 117 212 234
Go 6=  100 144 178 200 217 236
Go 7=  86 89 117 171 172 213 234
Go 8=  41 44 83 89 101 148 178 202
Go 9=  55 110 114 149 155 185 202 220 229
Go 10= # 113 125 184 190 220 223 226 238 247 251
Go 11=  2 4 8 17 24 35 70 96 129 140 192
Go 12=  31 62 99 108 124 143 177 199 216 224 240 248
Go 13=  19 38 55 68 77 110 117 137 155 186 213 221 234
Go 14= # 48 63 76 96 127 152 166 211 233 244 250 253 254 255
Go 15=  41 47 72 82 94 121 144 148 151 164 188 202 203 229 242
Go 16=  2 5 9 11 19 23 38 46 48 76 92 96 129 132 152 192
Done

Seed= 100
Go 2= ### 71 163
Go 3=  57 115 231
Go 4= ## 117 186 212 234
Go 5= # 21 42 80 84 168
Go 6=  5 11 22 65 130 160
Go 7= ## 36 36 73 73 73 146 146
Go 8=  10 46 92 112 133 184 194 225
Go 9=  25 51 54 102 109 155 183 205 219
Go 10=  51 115 123 153 185 204 220 230 238 247
Go 11=  84 106 118 169 170 181 187 213 218 221 237
Go 12=  31 62 117 125 143 186 213 221 234 238 247 251
Go 13=  18 27 35 36 68 70 72 104 137 141 145 162 209
Go 14=  13 24 27 48 55 97 110 113 134 140 184 195 220 226
Go 15=  7 14 25 29 51 58 65 70 116 131 140 160 163 209 232
Go 16=  63 103 108 109 126 159 179 182 182 207 217 219 219 237 248 252
Done
#endrem[/color]
Thanks, Alan.
 

hippy

Technical Support
Staff member
The surprising thing (at least to me) is that it still reports "errors" at similar times during the execution, in a real PICaxe and the PE6 simulator. So in principle it should be possible to determine what's happening
Are the errors during real chip execution and in simulation in the same place ? Can you describe the nature of the errors and do they have anything in common ?

What would be best is if you could provide just a single example which fails which can then be studied because it is very difficult to replicate and analyse the issue otherwise. Indicating what the end result is and what it should have been will also help.
 

hippy

Technical Support
Staff member
Maybe the issue is related to the structure of my sorting algorithm which uses a hybrid arrangement of IFs and LOOPs. In particular, I can't see how to set up the indenting "correctly" (because the LOOP tests are out of sequence).
Thanks for noting that. What you effectively have when simulating is ...

Code:
do
  if @bptrinc > @bptr then
    @bptr = @bptrdec xor @bptr
    @bptr = @bptrinc xor @bptr
    @bptr = @bptrdec xor @bptr
    dec bptr
loop while bptr => FIRST
  endif
That is badly structured code. We will have to investigate why the compiler has not detected that as such but all bets are off as to what it would actually do on a real chip or within simulation.
 

AllyCat

Senior Member
Hi hippy,

Thanks for the reply. The only "error" I'm trapping is that the bptr is not where "expected" (it's at b1 instead of b2), which probably also causes a sorting failure (not specifically tested). The errors certainly "move around" when the random numbers are changed, but then the whole sorting sequence is different.

I haven't done much comparison between the simulator and the real chip because the simulator is so slow (and the chip comparatively fast) and I haven't got the hang of setting breakpoints (in PE6) yet. It's the fact that it occurs in the simulator at all that seems to rule out bugs caused by interrupts, etc. The other "oddity" is that they are so "rare" (one in many, many thousands).

I'm afraid I really still "think" in GOTOs but tried to put the algorithm in a more "acceptable" format. Also the IF @bptrinc > @bptr THEN GOTO ... didn't seem to work (reliably) at all (I was using only real chips at the time so debugging was impossible).

Sorry I'm mainly "on the road" today so can't provide much more information at the moment.

Cheers, Alan.
 

hippy

Technical Support
Staff member
Consider this, assuming it is somewhat how it is intended ...

Code:
bptr = FIRST
do
  if @bptrinc > @bptr then
    @bptr = @bptrdec xor @bptr
    @bptr = @bptrinc xor @bptr
    @bptr = @bptrdec xor @bptr
    dec bptr
  endif
loop while bptr => FIRST
Ignoring that some things may be swapped, at some point bptr will reach LAST, but the loop will keep repeating even when bptr is at or has passed LAST because it will always be greater than FIRST. It will continue to increment bptr, may do additional swapping, until bPtr reaches the end of RAM and wraps over to become zero.

This means that other RAM locations and variables after LAST may be corrupted while the program is running. And that's where all your variables are.
 

AllyCat

Senior Member
Hi,

assuming it is somewhat how it is intended ...
........... at some point bptr will reach LAST, but the loop will keep repeating even when bptr is at or has passed LAST because it will always be greater than FIRST. It will continue to increment bptr, may do additional swapping, until bPtr reaches the end of RAM and wraps over to become zero.

This means that other RAM locations and variables after LAST may be corrupted while the program is running. And that's where all your variables are.
Well yes, that's why I didn't write the instructions in that sequence. ;) Writing the "go back for the next iteration" as a GOTO (commented out in my example) appears to be legitimate (or at least not detectable as an "error" by the compiler?). Or maybe it needs an EXIT or a "jump forwards", but they would lead to slightly lower efficiency.

Certainly the bptr wasn't running amok as you propose. The only reason that I noticed the strange behaviour is that I inadvertently allocated a variable to the byte (b0) below the data list and the program started "miscounting" slightly (not crashing or hanging in a big way). A later issue was that when single-stepping through the simulator, the "misplaced" pointer occurs very fleetingly and the program then continues "as normal". Does the simulator have a "conditional breakpoint" (e.g. "IF bptr = 1), although that still might not help find what caused the pointer to be miscalculated in the first place ?

In practice, I now have a better algorithm (faster, "safer" and easier to debug), but I highlighted this issue because the PICaxe seems to be doing something very "strange". The program I posted performs around 100,000 comparisons, about a half invoking a SWAP of two bytes, yet the bptr is consistently "misplaced" on only about 6 or 7 occasions (and even then the "target" byte may not be actually written/corrupted). :confused:

Cheers, Alan.
 

AllyCat

Senior Member
Hi again,

Having "slept" on the problem, I think I can now "See the wood for the trees". My "error test" only looks for the bptr "falling off the end of the list" and pointing at b0 (with the data list from b1 upwards) where it's not expected to be. If we assume that the "fault" is an incorrect @bptrinc or @bptrdec , then that can only occur if bptr is nearby. This will occur much more of the time with a short search list than a long one. Note that this is an "insertion sort" (searching upwards and downwards), not a "bubble sort" (where the pointer continually starts from the low end).

In my demonstration code in #6, the errors are actually reported in passes 14, 15 and 21 or around 12% of the 25 trials. That seems too few for a "binary" (true/false) error or too many for a numerical error (involving byte values). That made me wonder about the XOR expressions, because it's used three times and 1/2 ^ 3 = 1/8 or ~12% ?

I'm struggling to use the PE6 simulator on my humble netbook with its 600 x 1024 screen, but have made some progress. First I moved the NEXT pass down a line so that all results are reported to the serial terminal. Also I've found that changing the random "seed" to 100 gives errors in the 3rd and 6th sorts, which are much quicker to get to in the simulator. A cursory inspection of the output data looks as if all the sorts are "successful", but what actually happens is that when the "error" occurs, the bptr swaps the data in b0 and b1 (hence the corruption of b0), so after the error symbol (#) the "results" list shows the last value in b0. not one of the two data values (which has got transferred into b0).

That's as far as I've got at the moment, but it does look as if maybe the INC in the @bptr = @bptrinc xor @bptr might be getting "lost" occasionally (or perhaps one of the decs is being executed twice?). I think I'll have to try to find a "bigger" computer, because this simulation really needs the Terminal, Editor, RAM and System (for bptr), etc. windows all open at the same time. :(


EDIT: OK, it looks as if the line " dec bptr ; bptr = 0 Required for PE5 compiler" is completely unnecessary in PE6. I thought it was because the previous line had been @bptrdec = .... (instead of just @bptr =..) in PE6, but that structure caused issues in PE5. With that line deleted, the program now appears to work correctly (only tested slowly in the PE6 simulator so far) but I'm surprised how such a "large" error caused such relatively minor disruption to the functioning of the algorithm.

Now I'll need to try the code in PE5, but of course I must first replace the RANDOM function in the PE5 simulator with the correct simulation code for a real chip (but that's for another thread in the Code Snippets section. ;) ).

Cheers, Alan.
 
Last edited:
Top