import XML into SQLite - what's the most efficient method?

[orbs]

i have a set of XML files of this format:

<record>
    <col1>data</col1>
    <col2>data</col2>
    ...
    <colN>data</colN>
</record>
<record>
    <col1>data</col1>
    <col2>data</col2>
    ...
    <colN>data</colN>
</record>
...

i wrote a simple function, based on _StringBetween(), to read the data and rewrite it into a TSV file for post-processing.

i'm going to be processing a LOT of those files on regular basis. i'm considering import into SQLite database for more efficient storing and post-processing. it's going to be accumulated into a single table, where each row is a record and each column name is mentioned in the XML.

this question is concerned about the overall speed and simplicity of the import process:

is there an existing feature of SQLite for importing such XML? i.e. without using AutoIt to read and parse the text? like an XML parser that is native to SQLite?

thanks for any hints!

[jchd]

If your XML files always have such a simple 2D array-like structure, you can adopt one of the two approaches: use a faster way to read the XML into a 2D array (a one line regex would do) or transform the XML intoJSON form which then fits well the JSON1 extension.

Personally I'd use the first way. Post a sample real input file and DB.

[orbs]

yes, the XML structure is very simple. the posted structure is pretty much the way it actually is (col1..colN reworded, originally having a much more descriptive titles).

a regex seems a daunting task. all results of google "regex parse XML" leads nowhere or worse. besides, looking at _StringBetween() i see it already utilizes StringRegExp(), very useful because i provide it with <col1> and </col1> as the start and end substrings, in a loop col1..colN. not sure if a single regexp can successfully handle the entire XML parsing directly into a 2D array. i see StringRegExp() can "Return an array of arrays containing global matches including the full match (Perl / PHP style)." which may be what i'm looking for, but i haven't the slightest idea on how to approach this.

converting to JSON is something i'm about to consider, although at first glance it seems extra work for the same purpose, i doubt it's going to be more efficient.

so for now i'll stick to using _StringBetween().