I'm trying to validate a series of words that are provided by users. I'm trying to come up with a scoring system that will determine the likelihood that the series of words are indeed valid words.
Assume the following input:
xxx yyy zzz
The first thing I do is check each word individually against a database of words that I have. So, let's say that
xxx was in the database, so we are 100% sure it's a valid word. Then let's say that
yyy doesn't exist in the database, but a possible variation of its spelling exist (say
yyyy). We don't give
yyy a score of 100%, but maybe something lower (let's say 90%). Then
zzz just doesn't exist at all in the database. So,
zzz gets a score of 0%.
So we have something like this:
xxx = 100% yyy = 90% zzz = 0%
Assume further that the users are either going to either:
- Provide a list of all valid words (most likely)
- Provide a list of all invalid words (likely)
- Provide a list of a mix of valid and invalid words (not likely)
As a whole, what is a good scoring system to determine a confidence score that
xxx yyy zzz is a series of valid words? I'm not looking for anything too complex, but getting the average of the scores doesn't seem right. If some words in the list of words are valid, I think it increases the likelihood that the word not found in the database is an actual word also (it's just a limitation of the database that it doesn't contain that particular word).
NOTE: The input will generally be a minimum of 2 words (and mostly 2 words), but can be 3, 4, 5 (and maybe even more in some rare cases).