NEWSLETTER Vol. 2, No. 1 & Vol. 2, No.2 January-June 2011, July-December 2011

The Official Newsletter of

Society for Natural Language Technology Research, Kolkata

~ .. snLTR

A Society under the Department of IT Govt. of West Bengal

Contact Us

Working Office: Module 130, SDF Building, Salt Lake, Sector-V, Kolkata 700091 Phone No.: +91 33 65332223/23576454 Fax No. +91 33 23576454 Website: http://www .nltr.org

Registered Office : Department of Information Technology, Govt of West Bengal 4, Camac Street, 7th Floor Kolkata-700016

~-· 'i1C<t~'1t ~ <tSQ1<W>!>I

Content Page

Editorial 3

Summary for the previous works 3-6

Recent achievements 6-9

Progress report 10-11

External Projects 12

(C)SNLTR 2012

Contact Us

Society for Natural Language Technology Research (SNLTR) Working Office: Module 130, SDF Building, Salt Lake, Sector-V, Kolkata 700091

Phone No.: +91 33 65332223/23576454 Website: hcrp://www.nlrr.org

Registered Office: Department of Information Technology, Govr. of West Bengal 4, Camac Str<-et,7th Floor, Kolkata-700016

EDITORIAL

T he second issue of the official newsletter of SNLTR took a longer time to see the light of the day than we had expected. We are extremely sorry for that. This time the Newsletter is a combination of two issues, viz. Vol.I, No.2. and Vol.2, No.1. This issue presents reportage on our new achievements preceded by a summary of previous works and achievements. This also contains progress reports of the ongoing projects, director's message, news of the newly published books on Bangia language and an announcement section. Essentially, this newsletter is a reminder for the NLP tasking in Bangia language that is improving towards more technological accuracy and of course a social awareness for computing using Bangia enabling software. We are looking forward for a more persistent presence in the field of NLP and wishing to be present as a Bangia NLP group.

3

SUMMARY OF THE PREVIOUS WORKS

S NLTR runs since 2008 and over the three years it has been putting its enormous effort rowards research and development of language and technology. Primarily the activities and achievements ofSNLTR may be classified inro three categories, viz. Development of Technology and Tools, Development of Language and Linguistic Resources and IT-enabled tools for education of the less privileged. Of its targets, SNLTR has achieved many and is continuously marching ahead to achieve more as far as it gets the financial and human support.

The first aims at developing technology to enable use of computers using local languages. Keeping this aim in mind, SNLTR has developed a number of code-conversion software that can convert the electronic version of legacy data to the Unicode 5.0 format. In developing Bangia language technology, the well-known achievement of

SNLTR is Baishakhi Linux ~ ~), a bilingual version of the Linux Operating System. This operating system is open source and is based on Ubunru flavor ofLinux. Any desktop or laptop can have Baishakhi Linux as the operating system. The Baishakhi Linux operating system supports all the available features of office works, such as Document Preparation, Presentation Preparation, Spreadsheet Computation, E-mail facilities, Web browsing etc. Further, these operations can be carried out in a bilingual mode that is, both in Bangia as well as in English.

Not only the bilingual Linux-based

4

operating system, but SNLTR also has developed software which give opportunities to work on the Windows platform. Thus, SNLTR has developed a number of tools, available in a CD, which can be installed in any PC with Windows XP or Vista, to enable users carry out Office applications, such as document preparation, Powerpoint presentation preparation, Excel Spreadsheet computations, Ermailing, Web browsing, etc. in Bangia. The CD also contains various Keyboard Layouts. In order to facilitate data and document enrry in Bangia, a number of keyboard layouts has been designed and implemented to suit the needs of different people habituated with different typing practices. The keyboard layouts are Baishakhi Keyboard for easy phonetic typing, Baishakhi lnscript, Uni-Gitanjali, Top-type layout, Webellayour and Udbodhan layout. The salient feature of all these keyboards is that they are all Unicode 5.0 compatible and hence any document entered through any of these keyboards is acceptable across any standard platform and can be viewed through any Unicode compliant Bangia fonts. Apart from the layouts, there are Bangia Soft Keyboards, Font Installer, Bangia 6refox 3.0, Bangia Open Office 2.4 (Windows), Bangia Sahayika in the CD.

The second-type of acrivity of SNLTR focuses on making the literary resources available to the users in a standardized and computer-searchable form. Moreover, availability of a large and clean corpus of language resource in standardized form is a necessity in development of modern day technology, such as machine translators, question-answering systems etc.

As a maiden endeavour towards

fulfilling this goal SNLTR had digitized the entire Bangia literary works of Rabindranath Tagore and put the same into the official website of the Society () . This website contains all the original Bangia writings of Rabindranath- poems, stories, novels, dramas, essays, songs along with their notations. In addition to these, the sire also records other information related to Tagore and his works like Tagore's biographical information, information regarding his literary works, letters, criticisms, etc.

The key features of the site are summarized below-

0 All the works ofTagorc arc categorized into different genres, viz. poems, stories, novels, dramas, essays, songs, ere. This makes the task of a viewer much easy in finding out the desired work/ text.

0 It has a robust search engine "Anusandhan" which helps one to find our particular words or phrases.

o The fearure "Anveshan" enables its viewers to get the meta-information of a work­the dare of composition, the main cbaraaers, the category of the work, ere.

0 An alphabetical list of all his works is also available in the sire.

0 Notations of all the songs (Swaralipi) are available online through a link in the text of each song.

0 Any or all pares of this online version can be printed, emailed as per the requirement of the target audience.

This online publication stands as a fitting tribute to the celebration ofTagore's

5

I 50th Birth Centenary year and was released on the 2nd of February 2010.

The third activity aims at development of interactive, self taught, Computer Aided Instruction cools using multimodal technology for the children who need special support. Many of the schools in the state experience the paucity of teachers. Computer Based Teaching (CBT) methods have proved to have positive effects towards the education quality and retention, due to the interactive nature of the system that allows the students co learn in a participatOry mode and also present the courseware in a much more interesting manner through multimodal animations. Experiments have proved that this approach has also been able co attract back the drop outs of the rural schools to the classrooms.

Based on the above perspective, SNLTR has undertaken the task of developing Multimedia CBTs for school students on the basis of West Bengal Board of Secondary Education curriculum. Presently, the following modules are ready and can be deployed in the schools- English CBT for Class V, Class VI, Class VII, Class VIII, Class IX and Class X and Mathematics for Classes IX and X. For College Students, Science and Engineering Education at the undergraduare and graduate levels can well be supplemented with (in facr, they require to be supplemented with) educational software for mathematical modeling, chemical modeling and physics simulations. However, such commercially available software are very expensive and often beyond the reach of the colleges in the state. In order to address this problem, SNLTR has prepared a collection of equivalent open-source software and have tuned them

for use in the colleges. The set of tools christened "Saraswar" (~) is a collection of standard open source based educational application packages available over internet. This was aimed to implement at the degree colleges followed by training to rhe reachers of different subjects concerned. Saraswar includes 49 software packages from various disciplines.

SNLTR implemented this project ro five government women's colleges in West Bengal. The Society has selected some standard educational packages (Open Source based) from the open source repository and compiled them into one CD. These include mathematics, physics, chemistry, electronics, geography, computer science ere. Next, SNLTR demonstrated the applications to some eminent academicians of Indian Statistical Institute and received their recommendation letter in this regard. Further, SNLTR has arranged for the testing of Saraswat by ERTL (East) and this was successfully completed in due course and the society has obtained the Testing Certificate about the software packages from the STQC, ERTL (East).

The implementation process has rwo stages: Installation and Training. Saraswat was implemented to 5 governmem women's colleges namely,

a) Bethune College ,Kolkata

b) Lady Braboume College, Kolkara

c) Nisrarini College, Purulia

d) Balurghar Mabila Maha Vidyalaya, Daksin Dinajpur

e) PD Women's College, Jalpaiguri.

6

SNLTR desires to take the task further to deploy the tools in all colleges based on the government support.

RECENT ACHIEVEMENTS

01 Chiki Keyboard

S amali, a language of rhe Austro-Asiatic language family, is one of the scheduled languages enlisted in the eighth schedule of the Indian Constitution as of May 2007. The speech communities of Santali are dispersed among so many regions of the indian Territory, though it is mainly the communicative language of the tribal community of Santhals of the Chota Nagpur Plateau (comprising the states of Bihar, Chattisgarh, Jharkhand and Orissa). According to the Indian Census 2001 Santal population of the country is 2,280,540 (Ethnologue http://www.erhnologue. comlshow_language.asp?code=sat report said it was 5,960,000 counted in 1997). This language is found in Bihar (Bhagalpur, Munger districts), Jharkhand (Manbhum, Hazaribagh districts), Orissa (Balasore District), West Bengal (Birbhum, Bankura districts), Assam, Mizoram and Tripura. This language is also spoken in Bangladesh , Bhutan, Nepal. Erhnologue, owned by SIL International (Summer Instirure of Linguistics, inc, a non-profit organization committed to serving language communities worldwide as they build capacity for sustainable language development, founded in 1934), also found some alternative names like; Har, Hor, Samrali, Sandal, Sangtal, Santa!, Sanrhali, Santhiali, Satar, Senrali, Sonthal. The dialects surveyed by SIL are: Karmali (Khole),

Kamari-Santali, Lohari-Santali, Manjhi, Paharia. Pandit Raghunath Murmu is the originator of OJ Chiki script (1925) for San tali language. As this language is one with its own special characteristics, and has a rich literature since 15th century.

SNLTR, Kolkata is now presenting an OJ Chiki keyboard layout, which enable Santali writings with the OJ Chiki fonts in Unicode. It is definitely a course oflanguage­development in order to facilitate data and document entry in OJ Chiki. The salient feature of this keyboard is that it is Unicode 5.0 compatible and hence any document entered through this is suitable across any standard platform and can be visualized through any Unicode compliant 01 Chiki fonts.

Development of OCR for Bangia

Automatic recognition of document text is useful in office automation, postal mail sorting, tabular form processing, bank cheque processing, digiral library generation, reading aide for the bl ind etc. Documents can be mainly of cwo types: (a) Printed Document & (b) Handwritten Document. Printed documents are generated by various printing technology such as laser printing, inject printing, offset printing, intaglio printing, screen printing, flexographic printing system etc. Handwritten documents are made by manual writing on paper. Both kind of documents are either scanned or photographed in CCD camera and put in the computer. Then a software is used to recognize these documents. The recognition processes of both systems are different in their details; they can be broken down into

7

four fundamental stages: pre-processing, feature extraction, classification and post­processing.

Pre-processing is essential to prepare input images to be suitable for recognition srages. Examples of the pre-processing stages include the processes to identifY text regions within the documents, el i minate imperfections, and segment the text into isolated characters. The goal of the feature extraction stage is to obtain a compact description (a feature vector) that can be used to uniquely represent the character. Classification is the main decision making stage in which the extracted fearure vector is classified into one among several character categories. Post-processing is used ro correct the error in the classification stage by the use of language models and contextual informacion.

SNLTR has developed an OCR system for Bangia under the supervision of Prof. Bidyut B. Chaudhuri (Indian Statistical Instirute) that can recognize text in various fonts with 97% accuracy. This is a pioneering development on the subject.

Bang/a S11pport for Software and for other Operating System

(Bangia support for InDesign CS4 System)

Regular Adobe InDesign CS4 support Unicode, but it can't render Bangia font properly. Keeping this technological inability in mind, SNLTR Team write a java script (with the help of Peter Kahrel scripts) which enables a hidden composer named ''Adobe World-Ready Paragraph Composer". This composer is responsible for indic support in

Adobe lnDesign CS4. This script is mainly written for Bangia Language, but it also supports other Indian Languages like Hindi.

DIGITIZATION OF SARAT­RACHANASAMAGRA

The entire gamut of the literary works of the legendary writer in and after the Rabindra­era, Sri Sarat Chandra Charropadhyay, some of whose novels arc still the best sellers of Bangia Literature, is now available online (http://www.sarat-rachanabali.nl rr.orgl). This has become possible by a Bengal Engineering and Science University-team under the thematic supervision of SNLTR. One can now read the creations of Sarat Chandra, like his novels, short stories, dramas, essays, pieces which were written using other name, unpublished writings, etc. just upon a click. The project of digitization of Sarat Chandra's work was given to an IT-team working at Bengal Engineering and Science University, Howrah, under the guidance of Prof Surya Sarathi Barat and Dr. Indrajit Banerjee. Honorable Education Minister Sri Brarya Basu inaugurated the website on 15th Sepetember 20 II.

DIGmZATION IN UNICODE FORMATAND~BHO~GOF

BANKlM RACHANABALl

The project, "Digitization in Unicode Format and Web Hosting of Bankim Rachanabali" had been taken tO formalize one of the aims of SNLTR. SNLTR at its birth aimed to develop Language and Linguistic Resources. Previously it rook an opportunity to digitize

8

Rabindra-rachanabali motivated by the same inspiration. Mrer Rabindra-rachanabali, Sarar-rachanasamagra was digitized in Unicode and both are available online (vide supra).

Banlcim Chandra Chattopadhyay (27 June, 1838- 8 April, 1894) is considered to be one of the most celebrated Bengali novelists and pioneering essayists in India. He is one of the representative icons in cultural genealogy of Bangla tradition. SNLTR has digitized the entire collection of Bankirnchandra's literary works in Bangia in Unicode and made a website for this iconic literary resource to reach the common mass. Now it is waiting for official release. Like other literary repositories built by SNLTR, Bankim online will also enable readers (1) to access and download all works of Banlcim Chandra Chattopadhyay, (2) to search for different information about the creations, (3) co search for different lines and text segments across the texts and ( 4) to have access co the different commentaries about the works. A hosr of other Bangia literary classics are in the pipeline which will be published seriatim, such as Vivekananda Rachanabali, Sri Ma Kathamrira, Najrul Rachanasamagra, Sukanta Rachana-samagra, Upendrakishore Samagra, ere.

SWARALIPI EDITOR

While editors for writing musical notations are available for western scores, none such exists for the Indian Music System. The present work is developing an editing system that will enable the users to write the lndi.an musical notations in a computer. The project

funher aims at providing facility tO play the Swaralipi to facilitate music composition by the composers.

Project Name : Swaralipi - A System for Scripting Tagore's Musical Notes and transcoding

What's the challenge? -The Objective: To build a system for scripting Tagore's musical notes (Rabindra Sangeet Swaralipi) and transcoding it ro the MIDI (Musical Instrument Digital Interface) form for subsequent generation of music.

What do we want to do? The Goals : To build a desktop and/ or web application which will

o Take user input to create music sheets which satisfies structural and notational standards ofTagore's songs.

0 Generate instrumental musical audio by using appropriate notational symbols to MIDI algorithm. Instruments are ranging from wind, string to key instruments

0 Play the music in real time with time bar.

0 Generate MIDI flies of every song to build an archive that can be used as a web environment repository ofTagore's songs.

SPELL CHECKER

SNLTR has developed a spell-checking system for Bangia language and released a beta-version of that system online. Now a user can get help direccly from the website. The website is :

9

http://banglabanan.baishakhi.org/spellchec ker.htm. This project initially aimed at developing a robust spell checker for Bengali that will be suitable for use with editors and office applications.

CBT MATHEMATICS

A Report by Debojyoti Ba.m and Tarnal Sen.

SNLTR, being a society under DIT, is also committed to develop multimedia course materials for schools ofWest Bengal. In the first phase 'Learning English' lessons were developed for classes V, VI, IX and X which were highly appreciated by both students and teachers. Teachers remarked that not only did it help the students to overcome the fear of the English language; on hearing the audio renditions of the lessons students became habituated with the technique of conversing in English. Teachers were also happy to see the increasing interests of their students on this language through this content which was an area of weakness for them.

Subsequently SNLTR decided to develop multimedia lessons in Math for classes 9 and 10 as per the West Bengal Board of Secondary Education's new syllabus. One of the major aims of this content is to address multi-level, multi-grade learning in the classroom especially in Math. Students can also concentrate on the option of using the 'Olympiad Sums' to understand and solve problems with high degree of difficulty which will be incorporated in the next version. On the other hand average students in the class will benefit from the step-by-step animated

explanation of mathematical problems to clear their concepts. The lessons are structured in such a way that they are always in control of the reacher and can be easily inregrared in the class like any other Teaching-Learning materials. In addition examples of different problems are also provided in each chapter to enforce concepts and further learning.

In addition students can use these multimedia lessons to practice and use the step-by-step method for self learning at their own pace and time. Once they have understood the concept clearly they can further practice with the Harder Problems solution. This will help to clear the fear of Math in many of the students and help tO build the understanding of mathematical concepts.

IL&FS Education & Technology Services have provided the technical development support with subject matter expertise from Homi Bhabha Centre for Science Education. The developed content was mathematically validated by Dr. Parthasarathi Mukhopadhayay on behalf of West Bengal Board of Secondary Education.

Going forward negotiations are on with West Bengal Board of Secondary Education ro include the Math e-learning CDs as part of the Mathematics text book. In the near future the Math CD will be translated in Bengali and other languages as per requirement.

PROGRESS REPORTS

The Projects Undertaken by Proj: D.D.

10

D11tta rnaj11mdar (President of SNLTR and Professor Emerit11s of lSI and Emerit11s Scientist ofCSTR and Director of CSIT, Ko/kata)

0 Multilingual Speech and Language Technology Research Archive-

There is a huge repository of Bangia digital documents, which were prepared in non-standard fonts. To have the benefit of this large corpus of information and data for further use, SNLTR has developed a number of code-conversion software that can convert the electronic version oflegacy data ro the Unicode 5.0 format. Maintaining Unicode standard, this project aims to archive elecuonic language resources developed in Bangia, Sanskrit as well as in English along with a multilingual search engine. By now a huge amount of texts has been incorporated in this corpus. The technical support was extended by the Institute of Cybernetics Systems and Information Technology (ICSIT), Kolkara.

o Digitization of the cinema scripts of Saryajit Ray's film Goopy Gyne Bagha Byne -

As a pionering effort the screenplay of India's Oscar winning film director Saryajit Ray's ftlm Goopy Gyne Bagha Byne (GGBB) will be available online shordy. As the blueprint of the project goes the site will be a storehouse of Ray's creations as far as GGBB is concerned. Ray's scanned handwritten manuscripts along with proper annotations, frame by frame video clippings of the film ragged

with the scripts, music notations, streaming audio music ft!es, still photographs of the movie, the original Bengali story of the film by Upendrakishore Raychoudhury and its English translation, information regarding the making of the film (e.g., costumes, crew members, shooting of the film, etc.), sketches by Ray etc will all be available in the site. The making of the site is almost complete; at present the sire is under review for final release. SNLTR has been enrrusred by the Saryajir Ray Esrare for bringing the works of this genius to the art lovers, films critiques, researchers and commoners as a fitting tribute to one of the greatest pioneers of Indian ft!rn industry. The entire technical support was given by the Institute of Cybernetics Systems and Information Technology (ICSIT), Kolkata. Prof. Dwijesh Kumar Dutra Majumder is acting as the Principal Investigator, Sri Sandip Ray, Prof. Aditinath Sarkar and Dr. Prasenjit Majumder are acting as the Co-Pis of the project.

o Online publication of Swami Vivekananda's Complete Works-

In order to keep in line with irs commit­ment to make classical Bengali literature online to the common mass SNLTR undertook the prime responsibility of preparing a UNICODE 5.0 compliant on-line digital repository of the complete literary works of Swami Vivekananda. The digitisation process is complete. The site is under preparation now. Apart from the texrs this site will also house photo­graphs, scanned handwritten textS of

11

Swamiji and other necessary information of this spiritual revolutionist of India. The site will be made available for public access shortly which will be a part of the year-long celebration of Swarniji's 150th birth anniversary to be started from 12 January, 2012. The technical support was provided by the Institute of Cybernetics Systems and Informacion Technology (ICSIT), Kolkata.

SAHAJLINUX

August 11, 2011 reponed by Debmalya Sinha

Recent development in rural computing in India is enabling the rural people to use the benefitS of digital technology. These segmentS of nco-computer literates in rural India have mostly no prior experience with digital technology. The numerous sophisticated and complex functionalities provided by in­market User Interfaces incur high cognitive load and learning time for a novice user. The challenge is to make an intuitive User Interface exclusively for them which reduce the cognitive load for learning and using the digital technology. We have developed Sahaj Linux, a Linux based Gnome desktop wrapper exclusively for the users in rural India which provides the basic funccionalities like browsing the Internet, reading/writing text documents, viewing/editing images and playing audio/video ft!es in an intuitive way that reduces cognitive load to learn and use the Interface for the mentioned users . A major work is done to develop a new ftle browser named "Sahaj Browser" which has novel features like ability to open multiple folders in a unique tree view. The entire code

is written in C to provide maximum resource efficiency so mat it is operable on low end cheap hardwires for affordability. Currently research is going on user interaction studies and cognitive load analysis to fine rune the effectiveness of the design for maximum benefit of me target users. Our future plan is to make a version ofSahaj Linux for Linux based smart phones as well.

EXTERNAL PROJECTS

An Open Source Web Browser for dte Blind People

(Reported by Ritwika Ghosh, Tirthankar Dasgupta and Animesh Das)

W ith the advancement in technology and the enhancement of the World Wide Web, information of all genres has reached me doorstep of people all over the world. However, the visually handicapped people face tremendous difficulties accessing information through web. This difficulty arises due to a number of reasons like lack of user friendly web interfaces, limited means of accessing printed material, etc. Existing technologies like, screen reader, screen magnifier and TIS systems are not enough for web interfaces. Thus, in order to provide proper information access and to bridge the communication gap between the visually impaired and the sighted community, the need co build a dedicated web interrace for the blind people is significant.

The main aim of this project is to build an open source web browser for the blind community in India. The browser accepts multimodal input mechanisms and generates

12

output in speech or Braille. The system includes me development of an Open-Source Web Browser GUI. In order to support me navigation and usability of blind users, the browser GUI has been integrated wim Open Source Screen Readers: NVDA for W rndows and ORCA for Linux. The screen readers NVDA and ORCA provide audio feedback to any keyboard operation performed by a blind user. Presently, only two Indian languages namely Bangia and Hindi have been considered. However, it can provide voice feedback to any other Indian language characters provided the pre-recorded audio files of that language is available. Voice feedback for mouse movement/click operation has been provided. The browser also recognizes different mouse gestures as input, corresponding to some pre-defined browser operations. It also allows users to

customize and create new gestures. A Speaker Independent Automatic Speech Recognition (ASR) System has been integrated with me system as an alternate mode of input, to recognize pre-defined speech based commands to perform navigational operations. The ASR system recognizes English words/phrases spoken in Indian ascent.

The output of web pages can be presented in two modes: Speech and Braille. A Text-To-Speech (TIS) System integrated with the NVDA and ORCA screen readers have been used to represent the text of web pages in the form of speech. The screen reader GUI has been customized to add other TIS engines in any Indian Language. An alternate representation of the text of web pages is in the form of Braille. A Braille transliteration engine has been integrated to

the browser GUI. Apan from transliterating English text to Braille, the transliteration engine can also handle Indian Language scripts. Presently, the system supports conversion of English, Hindi, Bangia and Assamese texts to Braille.

Apart from the above mentioned modules, the browser also provides voice activated guided navigation for novice users. The Speech recognition system has been further enhanced to support Google search and obtaining results from Wikipedia using speech commands. Also, the browser system has been augmented with fearures that allow a blind person to access different features of

13

Gmail through voice commands.

Daisy

DAISY is an MHRD (Ministry of Human Resource and Development), Govt. oflndia sponsored project, which involves digitization of books in print to DAISY format as per certain. specifications. This project, "Creating Access1ble Study Material for Print Impaired Students" is being undertaken at liT Kharagpur. In the first phase of this project, SNLTR took up 35 books in Assarnese and Bangia for conversion. SNLTR has fulfilled the primary target of the project.

Web Browser for Blind People m Indian Institute of Technology, Kharagpur

A Project funded by DIT Govt. of India f Society for Natural language Technology • sr-. ... :r"' Research (SNLTR), Kolkata

0. Qo'ae ""Yta •Sa•nll'IIA ' L

71

1 ..... ~G-• I 1Dr¥'iS' • __ ._ ............... k. 1 1&1 alW.'¥ lkl

• 1 1 &'114oiiPS'WIII = I 't W ' ttAofl!l!ln 1 t 7' 1 o& ..... ................... 14

15

Edirors : Sibansu Mukhopadhyay and Rajib Chakraborty News Leccer published Biannual for SNLTR, Module 130 SDF Building, Sale Lake Scccor - V

Kolkaca 700 091 & Printed at Sankalpa'l2, 9830634064