Processing of Written Language

Prepared by: Ruel L. Montefolkaa.k.a : Zen0xide

Objectives:At the end of the lesson we would be able to;

1. understand the importance of the written

language in the 21st century education and its

meaning,

2. appreciate the valuable factors involve in

processing written language.

What is written language?

the representation of a language by means

of a writing system. Written language is an

invention in that it must be taught to

children; children will pick up spoken

language (oral or sign) by exposure

without being specifically taught.

Processing of Written Language

One of the most difficult topics is NLP (Natural Language Processing) : removal of the language barrier between people. This involves communicating with application programs or expert systems in the most natural and efficient format. Pattern recognition in speech and visual scene analysis are therefore significant. Success in natural language understanding has been slow in coming and achieved at great cost and effort.

The goal of natural language translation was

one of the first attempted by Artificial

Intelligence (AI) researchers, and failure to

reach it proved to be one of AI's greatest

disappointments. However, analysis of past

failures allowed progress has been made to

generalise this "translation" machine.

Ways of Communication of LanguagesThe majority of human linguistic

communication occurs as speech. Written language was only a recent invention and plays a less central role. However, processing written language (assuming an unambiguous representation) is easier than processing speech in general. For example, building a program that understands speech requires all facilities of a written language understand as well as enough knowledge to handle noise and ambiguities of audio signal.

Thus it is useful to split the problem into two subtasks:

1. Processing written text using;

a. Syntacticb. Lexicalc. Semantic knowledge of language

2. Processing spoken language using; all information needed above, and additional knowledge about phonology.

Steps in the Process

One pitfall in processing language is that it is tempting to define the language simply as a set of strings, without reference to understanding the context. However, in order to increase realism and accuracy, we must represent the language as a pair: (source language, target representation). The target representation would be chosen relevant to the situation. Hence it is possible to depict this as a mapping from the piece of language to some representation.

In overview, to achieve this we need to define precisely what the underlying task and target representation would look like.

1. Morphological Analysisa. Singleton words are analysed into their respective componentsb. Non-word tokens (e.g. punctuation) are categorised separately.

2. Syntactic Analysis

a. Structure holds linear sequences of related words

b. Word sequence is rejected if it violates the language rules for how words may be combined For example, an English analyser would reject: "Girl the walk computer do."

3. Semantic Analysis

a. Structures created by syntactic analyser are

assigned meanings

b. Mapping exists between syntactic structures

and objects in task domain.

c. Structures with no mappings may be

rejected (semantically anomalous). For

example, the sentence: "Colorless green

ideas sleep furiously" [Chomsky, 1957]

would be rejected

4. Discourse Integration a. Meaning of an individual sentence

could influence other sentences that precede or follow it.

For example, the word "it" in the sentence: "George needed it" depends on the preceding context; while the word "George" could influence the meaning of later sentences, such as "He always laughs".

5. Pragmatic Analysis

a. Structure representing what was said is interpreted to determine what is required to be done.

b. Application of a set of rules that characterise co-operative dialogues.

c. Translation from knowledge-based representation to a command executed by the system.

Complete separation of these phases is difficult. These steps all interact in some way, and can be processed sequentially or in parallel. However, if there is dependence from one phase to another, it is critical to process in an order which satisfies the overall performance.

Reference:

http://www.ling.fju.edu.tw/hearing/Processing

%20of%20Written%20Language.htm

https://www.google.com.ph/search?

q=semantics+meaning&biw=1318&bih=608&s

ource=lnms&tbm=isch&sa=X&ved=0CAYQ_A

UoAWoVChMIybzJhYWPyAIVjI6OCh1f1wdz