Archived Forum Post

Index of archived forum posts

Question:

Need fastest technique/language for csv import, create sql statements, etc

Apr 05 '17 at 09:10

This may seem like a strange question but I am wanting to create a small application to use to import csv data, create sql insert statements. Some of the input files have non ascii characters, so I have to clean up the data also. In one case the file uses ~ (3 characters) for the field delimiter. I will just need a lot of options and as long as Chilkat can handle it, I'm in good shape.

My question is of the languages that I know or feel comfortable in, which could accomplish this the fastest? I currently am using vbscript, but it's much slower than what I want.

vbscript php activex php extension purebasic delphi dll delphi activex xojo plugin

I do already own xojo and purebasic if that helps, but if another option like delphi is going to be much faster, I'm fine with going that direction.

Thanks Alan


Answer

Those are a lot of options. I know vbscript and can say it won't read the files fast enough for your request. You need something that will read the files at a low level, and line by line. I am more comfortable with Visual FoxPro and C#. Both of these have the ability to read files line by line directly. I believe delphi will have the ability. Purebasic and xojo might.

I have done hundreds of custom data conversions pulling in different styles of CSV and otherwise delimited text files. Reading the file line by line, and processing each line as it is read has proven to be the fastest.

HTH, Tracy