Superfast Datalogger intefacing with Vdrive2 module

[superworm]

Hi Everyone!

I've been using PICAXE for a relatively long time now, and am really finding alot of enjoyment and inspiration in it. I've made a fair few successful projects, and am now planning on the following! This is my first post to the PICAXE forum so bear with me if I seem a little green to you guys!


Basically I want to create a datalogger, using the normal readadc10 command and related potential divider circuit etc, the usual sort of set up.

I then want to output the ADC results to a USB pen drive through the vDrive2 module, using a variation of profmason's sample code. This will use the normal hserout commands etc.

All this seems ok, the catch comes here. I want to be running the PICAXE at 8MHz, and want to read the ADC (10 bit) pin over a period of 1 second. I will be using the minimal amount of code during the ADC scanning loop, ie read the ADC and then write to the USB drive, then repeat. But I also need the data to be read and then written to the USB drive at a rate as close as possible to 8MHz. Obviously time will be lost when reading the ADC pin and then output to the USB, but I was hoping that it would be minimal, as I couldn't find any official times that these instructions take. I can make the PICAXE run at 16MHz if required.

I obviously need the USB stick as memory, for write speed and memory size (EEPROM would be far to slow and would need far to many EEPROM chips to reach the amount of data i need to store). Plus there is a great appeal to being able to pull the stick out and read in on the PC.


I really hope people may be able to give me some advice on this one, as I really don't want to have to move away from PICAXE for speed issues.

Many Thanks!

Matt

 

[hippy]

Welcome to the PICAXE forum.

It sounds like an interesting project but I'm not sure you will be able to attain the sampling rate you may desire, though I'm not clear about what "I also need the data to be read and then written to the USB drive at a rate as close as possible to 8MHz" really means; 8 million analogue samples per second ?.

Each PICAXE command takes around 250us at 4MHz so just consecutive READADC's would be around 4,000 samples per second, 8,000 at 8MHz. Then there would be the overhead of getting the data over to the VDRIVE2. That would take at least as long as the READADC, then another command to put it into a loop, and a means to finish, so, even at 16MHz you would probably be looking at a sample rate of 4,000 per second or less. I don't know what the actual overhead of VDRIVE2 communications would be and it could have a significant impact.

 

[superworm]

Thanks for the quick reply hippy! Yes that's basically what I meant, around as close to 8 million samples as possible, but I did think it was a bit much to ask!

Think I'm going to have to look down other routes. Just found the LCS-1M pc-based oscilloscope that would actually be perfect I would imagine, built by womai based on the PICAXE. But he has stopped selling it due to collaborating with Rev Ed on a project which means I probably won't be able to get hold of one which is upsetting!

Anyone have any information on if they are available or not?

Thanks again hippy!

EDIT - Just found Technical's response on another thread (sorry for my ignorance on that one!).

I'm thinking it's going to be alot more expensive than when womai was selling it. He made it extremely affordable! Do you think it will be very expensive?

 

[hippy]

Eight million 10-bit ADC samples a second is a hard task for most microcontrollers, 125ns per sample during which time data has to also be transferred elsewhere. Dedicated - and frequently expensive - hardware is really the only way to go.

I have no knowledge of what price the PICAXE Oscilloscope (KIT120) will be though I would not expect it to be unreasonably priced.

 

[BCJKiwi]

Use the GHI Electronics' uALfat and communicate with hi2c. Must faster device and comms.

 

[superworm]

Hi BCJ!

I have just looked at your previous thread about the ualfat system, unfortunately I believe I may be out of my depth with this one sadly


I'm guessing also that even when using this system I would still fall well short of the sample rate I'm aiming for?

I think I may be better off with a PC based oscilloscope set up. I have a question about PC based scopes too if anyone can help -

If the PC based scope can read samples at a rate of 1MSamples/s and has a record length of 256 per channel does this mean if you ran it for a second it would only record 256 of the possible 1 million samples?

Sorry I don't fully understand what record length is, can anyone clear this up! Bet I'm being a complete idiot!

 

[lbenson]

If you're talking about Wolfgang's scope, which looks like it is about to be sold by Rev-Ed, then it doesn't do what you want. I have one (tho I'm not very experienced in its use), and like it, but it doesn't do datalogging of the sort you are looking for. You'ld have to look to Wolfgang for more details, but among many other issues, the PICAXE 28X in the scope communicates with the PC at, I believe, 9600 bits per second. Maybe it's twice that, but even at 19,200 bps, you're talking about a maximum of about 1,920 data bytes per second. If you want 10-bit ADC numbers, you could push a maximum of about 1,500 readings per second down that pipeline--but it doesn't work that way anyway--it has Wolfgang's custom software to provide an interface which looks like an oscilloscope.

Except for vttom's "Tip to Share" shown here -- http://www.picaxeforum.co.uk/showthread.php?t=11017 -- (up to 38,400bps), BCJKiwi's method is likely to be close to the fastest standard method of pumping out data for the PICAXE, and several orders of magnitude short of what you say you want. Not to mention that the picaxe itself is orders of magnitude short of being able to acquire the data and output it at the rate you want.

You may be well into black art territory to realize your expressed specification. What is it that needs such precision and speed?

 

[BCJKiwi]

Hspi will be the fastest datapump and uAlfat does support spi - Vdrive2 did not last time I looked. However I have not used spi.

 

[MORA99]

Vdrive2 can go to 3,000,000baud but the disk overhead is huge in comparison.
Although you can write many bytes at one go and not wait for reply, at some point you will have to wait before sending more bytes.

If you could store data for maybe 1 second in the picaxe and write it at once the delay could be minimal, but it will need hardware serial and very efficient code to check if vdrive is in sync.

EDIT:
Or you can just send one byte at a time without the \r so vdrive does not end the command,and then complete the write and check sync every so often.

 

[SilentScreamer]

Might be stupid and its probably impossible (I know very little about serial communication) but to get a faster baud could you use hsersetup? I don't know if its possible but if you can set it to 1 on a 16Mhz chip couldn't you get 2 million baud? Then use hserout to transmit the data at the new custom baud rate?

 

[lbenson]

The description of hsersetup in manual 2 shows a maximum baud rate of B115200 for a 40mHz part, which we don't have yet (the X2, due next month, is spec-ed to operate at 40mHz). This is +bits+, so around 11,000 +bytes+ per second. You say you want ADC10, 10 bits, so you either have to send each reading in two bytes or do compression, which slows you down further. So you are at least 1,000 times too slow for what you have specified, even for a picaxe several times faster than what is available now--and that's only for the maximum speed at which you could theoretically pump out data.

Here is a link to a module which says it totals 24 million samples per second on a total of 16 channels: http://www.l-3com.com/products-services/docoutput.aspx?id=586

I think you're into satellite telecommunications territory.

 

[superworm]

Yes, I was worried I was asking possibly too much!

Do you guys know if the arduino or similar may be able to handle the specs I require. I just need a 16bit ADC running at 1MS/s and for there to be standalone memory, so that the data can be read from a computer at a later date.

Or is it simply just too much to ask? Is there any way that a 16bit ADC could transfer its data straight into memory at 1MS/s without having to go through a micro?

 

[lbenson]

If memory serves, threads here have indicated that if you can get 12 good bits of reliable ADC at any number of samples per second, you will have done a good job of hobby engineering.

 

[MORA99]

Or is it simply just too much to ask? Is there any way that a 16bit ADC could transfer its data straight into memory at 1MS/s without having to go through a micro?

A few lines from a toprange atmel-avr 8bit mcu datasheet. (arduino is based on avr)
10-bit Resolution
0.5 LSB Integral Non-linearity
±2 LSB Absolute Accuracy
13 - 260 μs Conversion Time
Up to 76.9 kSPS (Up to 15 kSPS at Maximum Resolution)

And thats hardware driven,no software needed to do those measurements, still way below your target (if SPS means samples per second).

 

[superworm]

I think I've met my match on this one! Thanks ever so much for your help guys!

Theres only one last thing I can think of;

- Using womai's scope (soon to become a kit from Rev-Ed), which can handle up to 1MSample/s, is there anyway that I can use on board memory to take the results from his scope to look at on the the PC, probably using his software, at a later date?

Thanks once again guys!

 

[MORA99]

What about connecting a netbook to the scope, easy to bring in the field and can save the results on disk.

 

[nickelflipper]

There was a recent post on serial RAM/FRAM. Apparently the Ramtron FRAM chips can self write and increment on a user supplied clock and addresses (i.e. your adc data). Such a chip would handle your speed requirements, but would fall short on your storage at max 256k x 16 samples. If this is a follow up post to that, then you all ready know this.

The Ramtron datasheet says its a drop in replacement for SRAM, so that is probably the place to look. Good luck with your project.

 

[hippy]

- Using womai's scope (soon to become a kit from Rev-Ed), which can handle up to 1MSample/s, is there anyway that I can use on board memory to take the results from his scope to look at on the the PC, probably using his software, at a later date?

That's the principle I believe it works on; capture and buffer, send to the PC for display.

Whether it has enough ADC resolution ( 8-bit ? ) or can be extended to have enough memory I do not know.

Another thing to consider is how long it's going to take to get your samples to the PC; at 115200 baud, that's over 10 minutes for 1MB of samples. You may be able to go faster over a USB-to-serial link.

 

[Dippy]

I've only read bits of this thread, so bear with me.

S-worm:
You say 16bit ADC at 1MS/s ..... for how long? One second? One week?
This will determine RAM requirement.

I really think to get this you are going to have to look elsewhere.

Most low-end 'scope designs I have seen will sample a block, then display as said by hippy.
For cheaper scopes the block usually ties up with the GLCD size.
For big boy stuff they use a bigger buffer. (So your scope screen is a 'window' on the data buffer).

YOu may have to consider a manly micro +external ADC+parallel RAM or use the technique employed by Wolfgang.

You may also consider a dsPIC and learning C in an afternoon.

I've done work using dsPICs (for FFT) and if you roll your own ADC routine then 1MS/s is possible (in bursts).
(I think my own efforts were only 10 or 12 bit though, can't remember).
dsPICs have the great advantage of being able to automatically trigger a new sample via hardware settings and therefore can run faster vs software driven.
But don't expect to get this done in 5 minutes unless you can pinch the routines.

Also, as your bits increase, you will need to pay extra attention to component values, placement and PCB layout.

There maybe other micros out there that can do the job, but it'll be your job to find them and learn how to programme them.
I haven't looked but 1MS/s @ 16 bits is a healthy spec.

And, of course, storage is another bottleneck that needs overcoming. Certainly the newish video SD cards would be fast enough (technically) but the interface would be very tricky. Certainly well beyond the SPI method I used for 'standard' SD.

 

[hippy]

@ Dippy : The original call was for 8MSPS for one second, which begs the question ...

What is the application here ?

I'd have thought it rare to want 1us or faster sampling for such a long period but high speed sampling over a fairly short period would not be unreasonable.

 

[Dippy]

Oh. I missed the 1 second bit. I saw 8Mhz and then later on 1MSps.
I assumed there was some confusion between clock speed and sampling rate.

Quite a DIY project and will need a lot of searching, thought and input methinks.
I'll be interested to see how it pans out.
Could it be done economically with a variation of Wolfgang's method?

 

[MBrej]

Hi, the only way u'll be able to do the speed for the length of time you want is to use an external ADC and buffer memory (RAM), so that the ADC writes directly to the RAM for 1s, then the PIC will get the data from RAM and write it to whatever flash memory you are using.

You could use a serial system, although your only option for memory is FRAM, which comes in 2Mb max sizes, with about 256kb being reasonably priced. For a parallel system 16bit SRAM can be used, which is easily avaliable in 16Mb sizes needed for 16bit 1MSPS for 1s.

I have done this using the serial method with FRAM, but for your needs a parallel system is best. Also, with me i needed to get is as small as possible, and having 16bit data buses and even larger for the address bus, it was not really going to happen. My serial version has 8 ICs but only measures 20mm x 43mm (out of those 8 ICs are two accelerometers, quad opamp and a multiplexer and wont be needed in a basic logging application)

Matt

 

[superworm]

The specs I need are 1MS/s over a time frame of 1 second...

But as many people are saying, there are horrendous complications, especially if employing on-board memory to take the readings. I think I'm going to have to think outside the box on this one and come up with something else!

 

[superworm]

Thanks for you reply MBrej,

I am very interested in doing it the parallel way that you suggested, and then use the picaxe to just read the data into the serial port etc.

Do you have any examples of the parrallel system? Or even ADC and memory chips that could work well together...really struggling on this one! Any parallel circuit diagrams would be a great help even if basic!

Thanks ever so much to all you guys

 

[lbenson]

ADC parallel output interfaces

Far beyond my knowledge, but here is a google-scanned book with a section with the above title:

http://books.google.com/books?id=0aeBS6SgtR4C&pg=RA3-PA405&lpg=RA3-PA405&dq=parallel+output+ADC+interfaces&source=bl&ots=6yPdT2iaXW&sig=SbsH8q1Q_4evokTXgV84Y79xVYw&hl=en&ei=GpykSY-HNY-ctweK0aTZBA&sa=X&oi=book_result&resnum=2&ct=result

 

[MBrej]

I drew a quick block diagram in paint of a parallel system (attached). It is fairly simple, the PIC drives the process my providing a clock pulse, and so on each clock pulse the ADC takes a reading, outputs the data, the memory then saves that data, and the binary counter increments the address to save to for the next reading. When the PIC wants to read back the data, then save it to whatever flash device, the ADC is disabled, the memory is put into 'read' mode, the counter is reset, and on every clock pulse the memory outputs the value set by the counter, which the PIC reads and then writes it to the flash memory.

I think normally the ADC clocks out on the clock signal rising, and the memory saves the data on the clock pulse falling, so the ADC isnt modifying the data while the memory is saving it. The counter should increment at the same time as the ADC. You may need to invert the clock pulses to some devices to make this true.

The data bus from the memory is 'both ways,' and to stop the ADC outputting when the PIC wants to read the output from the memory, the ADC enable pin, when high, disables the output from the ADC, so only one device is in control of the data bus.

For ADCs, look at the ranges of maxim, linear and texas. Memory can be found at farnell, although i couldnt find DIP 16bit parallel memory, although you could always connect two 8bit devices together if you dont want to solder surface mount

Matt

 

[superworm]

Hi guys, me again!

Have started work on a parallel system, have found some 16bit parallel memory that I wish to use. My problem now is addressing it!

The memory has a 16bit address bus, and im looking for a 16-bit binary counter, but none are suitable because they do not show the upper and lower byte at the same time!

I want to cascade two syncronous 8bit binary counters to form a 16bit binary counter. Does anyone know how to do this?

It's all clocked by the pwmout on the picaxe at 1MHz

 

[moxhamj]

I was going to suggest cascading two 4040 counters. These are ripple counters so you can easily cascade the last stage into the clock of the next chip. However, the data does 'ripple' through the stages as it moves though, and at 1Mhz you might start to come up against the limits of the chip. You could try it at a much slower clock then ramp it up.

 

[womai]

superworm,

there may not be a need to reinvent the wheel. What you are trying to do is pretty much identical to what my LCS-1M Picaxe scope does, with the only exception that you want a 16-bit counter (my scope has only 8 bits) and accordingly 64 kBytes of RAM (my scope has only 2 kB per channel and uses only 256 bytes of that). With the upgrade to the Picaxe 28X2 the transfer rate can go up to 112 kbits/sec, but the Picaxe isn't fast enough to make full use of it during readback. My guess based on my Picaxe scope is you can transfer around 2 - 3 Kbytes/sec with very tight coding (look at the scope firmware code).

Forget the 16 bit resolution; that is stuff for expert, otherwise circuit noise will make the lower bits unusable. 8 bits is pretty easy (that's what my scope does), 12 bits with low noise would be a hell of an achievement. I'd start out small. The LCS-1M has an input amplifier so you can always increase the gain if you need to look at small signals.

So what you need to do is:

1. Have a good look at my scope schematic
2. add another 74hc393 and cascade its two halves to the existing one (the two halves of the first chip are cascaded so that should give a good idea how to do it - highest addres pin of previous half goes into /CLK input of next half).
3. Use a larger SRAM; either a larger chip (easiest solution), or you can use a bank of smaller chips but then need to add some address decoder logic (unncessarily difficult). Jameco has a suitable SRAM, it comes in SOJ and TSPO package which isn't very hard to solder); it has 64K x 16 bits so you could even log two channels at the same time; usage is pretty much identical to the SRAM the LCS-1M has: https://www.jameco.com/Jameco/Products/ProdDS/1927799.pdf
4. You can get the scope firmware (source code) from my scope website, that shows how to drive the hardware from the Picaxe. Espcially important are the ARM command and the READBACK command.

Wolfgang