-4
$\begingroup$

What algorithms do you know regarding text analyze/processing? Detailed, I need to "find out" the most used words, sentences, words' combinations, their category etc (these are just examples, moreover I need very strong mechanism to process a large text). Also need an intellectual searching by text "meaning", if that's possible.

Have you ever met with such kind problems, can you advice me and provide information to look for to find out/investigate more about this?

Thank you.

$\endgroup$
  • 1
    $\begingroup$ This question is a bit too open-ended, especially for an area as large as text analysis. I recommend closing. $\endgroup$ – Suresh Venkat Jul 7 '11 at 17:04
2
$\begingroup$

I think that Gusfield's book gives a very nice and complete overview of problems and solutions related to string matching. If you're interested in meaning too, you should have a look at what natural language processing has to offer (I cannot recommend a specific reference in that field, but Google will certainly help you).

$\endgroup$
1
$\begingroup$

In terms of searching by text meaning, arguably the most advanced search apparatus is IBM's Watson. There is a lot of information here. They don't keep a databse of meanings. Rather, they search for answers to questions where the information of the answer is arranged so that it has a high probability of corresponding to the information of the question. So they don't assume they know the meaning of either the question or the answer; the objective is to look for a likely connection.

$\endgroup$
0
$\begingroup$

Perhaps Apache's Lucene? http://lucene.apache.org/java/docs/features.html It must implement the algorithms you are looking for and in a way that scales.

$\endgroup$
  • $\begingroup$ The question is not asking for implementations but for algorithms. $\endgroup$ – Kaveh Jul 13 '11 at 0:16
  • $\begingroup$ Sorry.. I should have been more explicit in my statement. I meant that it's a good place to see an inventory of common algorithms and their implementations since the source is available and well documented. $\endgroup$ – unclejamil Jul 13 '11 at 13:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.