Find sentences with similar relative meaning from a list of sentences against an example one

Question

I want to be able to find sentences with the same meaning. I have a query sentence, and a long list of millions of other sentences. Sentences are words, or a special type of word called a symbol which is just a type of word symbolizing some object being talked about.

For example, my query sentence is:

Example: add (x) to (y) giving (z)

There may be a list of sentences such as: 1. the sum of (x) and (y) is (z) 2. (x) plus (y) equals (z) 3. (x) multiplied by (y) does not equal (z) 4. (z) is the sum of (x) and (y)

The example should match 1, 2, 4 but not 3.

Its not just math, its any sentence which can be compared to any other sentence based upon the meaning of the words. I need some way to have a comparison between a sentence and many other sentences to find the ones with the closes relative meaning.

Thanks! (the tag is language-design as I couldn't create any new tag)

Answer 1

I believe semantic analysis of a "spoken" language could be done quite efficiently, since words are grouped in finite sentences (Note: in this context language refers to natural and artificial human languages, not sets of input). The real problem is introducing a word's meaning to a machine, so that it associates a word or phrase with objects or ideas, like humans do. Furthermore, it is not unusual for the document's context to immensely affect the word's meaning. A good example is finding a word from your field and consider what that would mean to an outsider, e.g. the term "formal language".

A comparison of programming languages and natural languages amplify this fact. For many years we have compilers that efficiently transform programming languages to machine instructions (although they might not understand what the program really does). Creating a computer program (even an inefficient one) that can understand a human language is a great open problem in the field of Artificial Intelligence. Actually, it is one of the basic components in the famous Turing test, which allows to distinguish a human from a machine.

However, instead of AI, for practical purposes I would suggest studying Information Retrieval. These techniques are used for example, by search engines so that they find related documents to a specific query, using a variety of mathematical models. Techniques are also used so large datasets can be manipulated (since IR is used a lot in the Web). So if you are satisfied with the quality of the results given by your favourite search engine, Information Retrieval is the way to go.