The Limitations of Natural Language Processing and Generation Systems

Andrea HillCOMP 660

Definitions Natural language processing

(NLP) is a subfield of artificial intelligence and linguistics. It studies the problems inherent in the processing and manipulation of natural language, and, natural language understanding devoted to making computers "understand" statements written in human languages.- from Wikipedia: The Free Encyclopedia

Definitions Natural Language Generation

(NLG) is the natural language processing task of generating natural language from a machine representation system such as a knowledge base or a logical form.

Some people view NLG as the opposite of natural language understanding. - from Wikipedia: The Free Encyclopedia

Definitions Machine translation (MT) is a form

of translation where a computer program analyses the text in one language — the "source text" — and then attempts to produce another, equivalent text in another language — the target text — without human intervention.- from Wikipedia: The Free Encyclopedia

Hypothesis

While dramatic advances have been made in the area of natural language processing, the complimentary process of natural language generation is still in its infancy stage. The creation of unique, grammatical statements remains a human activity that can not yet be performed by computers in a meaningful way.

Experimentation Looked at four online translation

websites (translation2.paralink.com, babelfish.altavista.com, systransoft.com, online-translator.com )

Input a source text in English, translated it into a target language of French, Spanish, German or Russian, then back to English

Evaluated the services based on how accurate the meaning was retained, as well as the grammaticality of the output

Sample Results English: This is some sample text to see how well

these online translation tools work. I will see how well the text is translated to and from the following: French, German and Russian. Here goes!

German: Dieses ist etwas Beispieltext, zum wie gut man dieser on-line-Übersetzungshilfsmittel zu sehen, arbeitet. Ich sehe, daß wie gut der Text nach und von dem folgenden übersetzt wird: Französisch, Deutscher und Russe. Geht hier!

And back: This is something example text, to as to be seen well one these on-line translation aids, works. I see that like well the text to and by the following one translates: French, Germans and Russian. Goes here!

- babelfish.altavista.com

Sample Results English: This is some sample text to see how well

these online translation tools work. I will see how well the text is translated to and from the following: French, German and Russian. Here goes!

Russian: Это будет некоторый текст образца для того чтобы увидеть how well эти online пособия для перевода работать. Я увижу how well текст переведен to and from following: Франчуз, немец и русский. Здесь идет!

And back.. This wakes a certain text of model so that he would see how well these online benefits for the transfer it would work. 4 to uvizhu how well text is translated to and from following: Franchuz, German and Russian. Here it goes!

-Systransoft.com

Interpretation of Results

Translations back to source language were never exactly the same as the original text, but this is common with human-performed translations

The grammar of the final results was often incorrect

Conclusion

NLG systems are not yet robust enough to function without human intervention.

Even if we can build powerful systems, they will require ongoing modifications as language develops

Computers are great, but we as humans are irreplaceable!