cuonghuynh
10/1/2018 - 3:48 PM

How to Import a large CSV file to our storage?

How to Import a large CSV file to our storage?

Chunk data

If you have a large CSV file, you very likely can't store it all in an array in memory before you start inserting the data into the DB, so you need to be inserting the data in chunks as you read it from the CSV file.

Queue chunks

You would be lost the data if the process unfortinately broken.

Log errors

You should know why it is broken. :)