I need to log about 60-80 bytes of data every 5ms. I am acquiring the data using an Arduino Mega (AVR 1280 @ 5.0V and 16.0Mhz), connected to various SPI devices. One of the SPI devices has a 'data ready' pin which goes low for 0.5ms at 200Hz indicating that it's data is ready to be acquired via SPI. I am using this signal to trigger the SPI reads of all of my devices. The data is in various formats, some is 32bit float, other is 16 bit ints and unsigned ints. The time it takes to collect all the data is about 0.3ms. Once I have received it, I do a few calculations and adjust a few PWM signals based on these calculations.

These calculations take about 1 ms. The final thing that I need to do before my 5ms period is up is to log the data. The data acquisition and calculations and PWM adjustment takes about 1.3ms, leaving at most 3.7ms to do all my data logging.

So I tried just using a photocell hooked up like one of the FSRs here, then I tried a speed test to see just how fast I could get data to print. While you can see data scroll in the serial monitor screen of Arduino, you can't save it to a file directly from there. I've found that CoolTerm is the easiest way to do this,. Mar 25, 2016 Logging data on an Arduino is very much a trivial every. But it’s not fast. 19 thoughts on “ Fast, Efficient Data Storage on an Arduino ”.

So basically, I need to send 60-80 bytes off via SPI or Serial to some sort of data logging device which won't drop any data or have any buffer overflow issues. From what I have read, some of the SD card writes can take as long as 150ms, while others are more like 40ms, so whatever device I send my data to needs to have adequate buffering to handle at much as 2400 bytes (150ms/5ms = 30 periods, 30periods*80bytes/period=2400 bytes) in some sort of buffer in case one of the writes takes a really long time. In order to send 80 bytes via a serial connection in less than 3.7ms, I need to be running at a 172973 bits per second or faster. This means I need at least 230. Building With Papercrete Pdf Free on this page. 4kbps capability from something using standard serial.

I would prefer something even faster that would give me more time to do other things besides log the data. My storage requirements are not huge. In a one minute experiment, I will be generating 960kb of data, and it would be nice to be able to do 10-20 experiments in a row. So small SD cards would certainly be adequate from a storage point of view. Ideally, I could send the data from the Arduino Mega to some SPI enabled with a SPI clock rate of around 1Mhz -- this would let me complete my data logging in less than 1ms, and would give me plenty of time to do other things in case I need to.

I have looked into OpenLog and Logomatic available from Sparkfun, but it really seems like my application would be pushing the limits of these devices and that I would likely run into problems with dropped data or buffer overflow issues. I would like something that has room to grow as far as data throughput goes, so that in case I end up needing to log a lot more data, it wouldn't be a huge problem.

Are there other products out there that would be able to handle the data rate and storage requirements I have? SD card or flash memory would be ideal, as I would like to be able to pop the card into my computer after experiments, but if there is some other device that would work, I could give up that convenience. Thanks, ~Luke. With TinyFATfs, I see times ~15ms for a sector write. To me, 'data logging' implies steady-state operation. The file is already open, sector pointer in the right place, etc.

60-80 bytes every 5ms. A sector of 512 bytes is then ~8 periods, or about 40ms. It sounds feasible to me. However, my code 'sits and waits' for the f_write() etc. I'd have to examine to see if the operation could be started and checked on later.

SD card will be SPI, as is your datastream source. So you'll need to do a dance there. Is 960kb 960Kb or 960KB?

8 seconds, or 8000? Vw Transporter Ply Lining Templates For Business on this page. Clawson, SD does have the write-bandwidth I need. But it needs to be buffered, because the writes occur in relatively big chunks, much bigger than 80 bytes at a time (at least this is my understanding).

My AVR has 8K internal SRAM, 4k EEPROM, and 128K flash. This isn't enough space to hold the data for one experimental run (960kb), so if I wanted to use this space on my AVR, I would have to implement some sort of buffering to prevent overflow, and deal with knowing the SD communication protocol. I want to avoid this, and was hoping there was something out there already that would just let me send data via SPI and not have to worry about buffering issues or low-level communication protocol issues to an SD card. I'll check out the AT45 Dataflash.

Theusch wrote: With TinyFATfs, I see times ~15ms for a sector write. To me, 'data logging' implies steady-state operation. The file is already open, sector pointer in the right place, etc. 60-80 bytes every 5ms. A sector of 512 bytes is then ~8 periods, or about 40ms. It sounds feasible to me. However, my code 'sits and waits' for the f_write() etc.

I'd have to examine to see if the operation could be started and checked on later. SD card will be SPI, as is your datastream source. So you'll need to do a dance there. Is 960kb 960Kb or 960KB? 8 seconds, or 8000? I'll have a look at TinyFATfs.

Thanks for that. By 960kb, I meant 960,000 bytes. I'm not sure what you are referring to when you write '8 seconds, or 8000?' Could you clarify? As far as the amount of allowable 'sitting and waiting' time, I really need to keep it under 3.7ms at the very most. Ideally I could send it out much faster (.