Re: How to achieve scalability

My bad.  I meant to write "YUMMYTASTYWAFFLE".  I wanted to the example to 
demonstrate what we need to do, which is to take out common words like 
"cor****ation", "inc", "and", and "the".  We also strip out any non-
alphanumeric characters, so symbols like the "&", and punctuation and 
spaces go away.  Another step substitutes common abbreviations for certain

words.   

Tom van Stiphout <no.spam.tom7744@[EMAIL PROTECTED]
> wrote in
news:ek2m645fjac2qubbkkctfarqvo5stcj80q@[EMAIL PROTECTED]
 

> On Wed, 2 Jul 2008 00:45:11 +0000 (UTC), "Carlos Nunes-Ueno"
> <sullascl@[EMAIL PROTECTED]
> wrote:
> 
> Was that a real example? It seems difficult to come up with consistent
> rules that convert "The Yummy and Tasty Waffle Cor****ation" into
> "TASTYWAFFLE". Something like "Take the 4th and 5th word, and omit the
> spaces"? Can you tell us what kinds of rules you're applying for the
> conversion?
> 
> I have had good success with the Ratcliff/Obershelp algorithm that
> returns a similarity (a number between 0 and 1) between two strings. I
> checked and for your two company names the similarity is 0.75. Using
> some cutoff value you can narrow down the most similar companies and
> bunch them up that way.
> We recently implemented this algorithm as a .Net assembly in SQL
> Server 2005, and it is very fast. 10,000 comparisons in way less than
> 1 second.
> 
> I'm not at all convinced RegEx is the ticket here.
> 
> -Tom.
>