Re: How to achieve scalability

Tom, that sounds fascinating. Where did you come across this and how
did you apply it? Is it literally string A, string B, and here's the
answer xyz?

I have managed to do something similar to what Carlos has done, in
this case for product lists being im****ted into a database in a
structured way. The fastest way with pure vba that I could find was to
simply use a 'dictionary' for the terms, and make sure that the
dictionary terms were unique to a specific data source. In this way
source A provides data in say a CSV, you do some column mapping so
that the 'im****t' routine knows which columns to place where, then
load the data into a 'staging area'. You then run the first set of
transformations that map the terms from the dictionary for that data
source against the terms in the staging area that came from that data
source. Where you go from there would depend on what you want as an
output. This method processes approx 25k records in about 8 mins on a
Pentium M 1.2Ghz. Just for the record, it takes a while to build the
dictionary for a specific data source - thats the annoying part. I
could probably make the routine faster if I spent some time eyeballing
the code, but what you have there could be used much more efficiently
indeed.

Please spill the beans!

Cheers

The Frog