IEEE TRANSACTIONS ON BROADCASTING, VOL. 43, NO. 4, DECEMBER 1997 383

Multimedia Information Broadcasting Using Digital TV Channels

L.Atzori, Student Member, IEEE, F.G.B.De Natale, Member, IEEE, M.Di Gregorio, Student Member, IEEE, and D.D.Giusto, Member, IEEE

Abstract-In this paper, the structure of a new system to broadcast multimedia information within a digital TV channel is outlined. The proposed scheme is based on the DSM-CC functions, and uses a newly developed protocol that allows to efficiently convey multimedia information. Such information has the same characteristics of HTML files used in WWW environments, as it may contain text, images, sounds and animation organized as an hypertext.

The transport structure used is the MPEG-2 Transport Stream, which is the most popular standardized platform for the development of new digital TV services. A prototype software was developed to encode the file system by means of DSM- CC operations and include it in a MPEG-2 compatible Transport Stream.

1. INTRODUCTION

efficient representation of digital image and

effort in the past two decades. The results achieved, and in particular the definition of a wide set of standards, have prepared for a large number of applications in several fields, including: advanced personal communications (teleconferencing/presence, videotelephony), remote operation (teleworking, distance learning), interactive digital services (multimedia applications, TV program production). In this framework, the growing availability of transmission links, the enormous progresses in digital signal processing and the development of the VLSI technology ’with application to image and video compression make visual communications more feasible than ever.

In the specific field of video broadcasting, the most signdicant progress since the introduction of TV, over 50 years ago, is the introduction of digital television

T video HE data has been the object of a great research

(DTV).

The authors are with the Department. of Electrical and Electronic Engineering, Faculty of Engineering, University of Cagliari, piazza d’Armi, Cagliari 09123 Italy

A preliminary version of this work was presented at the EUROPTO Conference on Digital Compression Technologies and Systems for Video Communications, Berlin (Germany), October 7- 9, 1996.

Publisher Item Identifier S 0018-93 16(97)09252-4

This evolution is mainly driven by two factors: the interests of technology providers (e.g., consumer electronics producers, pushed by the need for new market opportunities), and the interest of service providers (e.g., TV program producers and distributors, pushed by the increasing demand of advanced services and new TV program offers).

The advantages of the digital representation against the analog one is of course not limited to the increased signal quality and robustness to noise; digital broadcasting technology represents a fundamental revolution in television. In a near future, computers and Ws will be merged together into a new system, mean for entertainment and work at the same time. The same consumer equipment that decodes and shows entertainment programs (video/movie-on- demand, movie listing, games) would also become the basis for interactive and transactional services (teleshopping, bank operations, remote databases, teleworking), as well as a tool to access a number of socially useful operations (telemedicine, news-on- demand, and so on). In such a way, digital broadcasting is the driving force for the integration of diverse media and the extension of computing into our daily life. These new services, together with the hardwarelsoftware user interfaces, are the subject of both laboratory research and real-world pilot experiments, in the context of restructuring the global information system.

In this process, a fundamental role is played by efficient data representations, as a studio-quality TV signal requires a transmission channel of far greater bandwidth than the one of its analog counterpart. Only the use of efficient data compression techniques allows to reduce this to a rate convenient for cost- effective applications. The precise characteristics of such techniques depend on the available bandwidth, the application requirements, and the allowable complexity or cost of the processing equipment.

The introduction of digital broadcasting involves a very complex process of standardization that requires the cooperation between regions and countries with different infrastructures, technical backgrounds, and political and commercial interests. At the same time, this standardization process should implement the current state-of-the-art in order to furnish the best

0018-9316/97$10.00 0 1997 IEEE

384

available technical solution. In the field of rationalizing the huge amount of uncoordinated source/channel coding for digital video applications, researches and to create an European framework for the standardization process started in 1988 with t the development of digital TV broadcasting via constitution of the Moving Picture Expert Gro satellite, cable, or terrestrial transmitters. In Figure (MPEG) within the ISOAEC, with the initial objective 1, a DVB typical satellite transmission chain is of standardizing an audiohide0 coding algorithm depicted. targeted to digital storage media (CD-ROM). As a

2. MPEG-2 AND PRIVATE DATA TRANSMISSION result, two soirce coding standards (MPEG-1 and MPEG-2) have been so far defined, which satisfy the needs of a wide range of applications in the range from 500 Kbls to 20 Mb/s. A third standard (MPEG-4) is currently being defined, targeted t video coding (below 64 Kb/s). 13818) is the most widespread standard and allows the coding of video sequences a t different resolutions and qualities, enabling the implementation of HDTV (High Definition TV). Major standardization problems arise with the specification of the transmission chain (satellite, terrestrial and cable). At present, a consortium is operating in the USA (Grand Alliance,

, which has been established by seven American companies for standardizing the DTV system. An equivalent initiative started in Europe in 1993, that is the Digital Video Broadcasting @VB) group, aimed at

MPEG has defined the syntax for audiohide0 coding as a transport structure, in order to combine different bitstreams and to convey the synchronization of audiohide0 signals [I]. Two transport structures have been provided for the system coding layer, where only the Transport Stream (TS) is suitable for television broadcasting. Such a structure allows to multiplex in a single bitstream different MPEG audio and video signals, as well as every other type of digital stream anyway defined; these data are referred to as private data. This allows new elementary streams to be handled at the transport layer without modifying the hardware, and gives the opportunity to implement additional services inside the digital TV channel.

TVl

Program selection

385

HTML files and can include images of different kinds, as well as audiohideo sequences. The main characteristic of the WWW database is that it branches out in several local hypertexts, which are separate open systems. This feature can not be directly implemented in a broadcast application, for in this case every transmitted document must be a closed web, that is it must contain every link, as it is not foreseen a return channel for a complete interaction. The implementation of such a service, therefore, requires the whole web file system to be sent, taking care to rebuild, at the client side, a new path for every file in order to preserve its links.

The target service to be offered is a cyclic transmission of a set of file systems (data carousel), where the user can download the webs he is interested in. In order to provide the user with sufficient information to make this selection, the data carousel has to include, within the multimedia data, a directory mechanism that describes the Information of the carousel and provides a procedure to group the information.

A digital version of teletext has been already standardized by ETSI, through the definition of the syntax to insert data within the Transport Stream [2]. Furthermore, the sixth part of the ISOIIEC 13818 [3] describes a global server-network-client system, and defines the relevant bitstream syntax for the implementation of additional broadcast and interactive multimedia services.

This is the Digital Storage Media Command and Control @SM-CC) specification, concerning a set of protocols that provide the specific control functions and operations to manage an ISO/IEC 13818 bitstream (although the concepts and protocols do apply to more general situations).

The system developed and presented in this paper is aimed at conveying multimedia information within the MPEG-2 Transport Stream, by making use of the DSM-CC features and by introducing a service that is similar in some way to the World-Wide-Web 0 information system available on Internet.

Focus of the paper is on the definition of a protocol that rules the insertion, within the MPEG transport structure, of files linked each other to form a hypertext (analogous to a n hypertext mark-up language, HTML, documents).

A software simulation of coding-transmission- decoding process has been already developed and is described in the following: the simulation has been carried out through the creation of a client-server architecture managed by two processes running on remote UNIX workstations. Such processes are connected through a socket interface, which is used by the first process (server) to transmit on request the packetized information included into the transport stream, and by the second one (client) to receive the encoded data.

3. HWERTEXTUAL INFORMATION SYSTEM BROADCASTING

A hypertext consists of a group of different documents connected to each other via hyperlinks. From the structural viewpoint, a hypertext is comparable to a database where the user can j u m p from a record to another if they are connected either by a direct link (e.g., a common element) or by other adjacency relations (e.g., consecutive pages in a n electronic book).

Most of the hypertextual systems allow the generation of multimedia documents, which can include textual information as well as other types of data, like drawings, images, video, animation, and so on. Each of these objects, entirely or partly, can become a link within the hypertext [4].

For the purpose of our work, we refer to the WWW database structure, where each web is made up of

4. THE BROADCAST DSM-CC DOWNLOAD FUNCTION

The DSM-CC specification provides a set of protocols to implement different applications such as movie-on- demand, tele-shopping, news-on-demand, remote database access, and so on. To this end, several functions are defined, based on the exchange of messages between the network and the user or between two users. The main characteristic of this syntax is that it does not specify the underlying physical, data-link and transport layers of the global protocol stack. It is intended to provide a unified signalling layer over a wide variety of underlying network topologies.

A particular feature of DSM-CC is the download function, which is intended to provide a very lightweight and fast data or software server-to-client download, or network-to-client download. Furthermore, flow-controlled operation and broadcast download are supported, based on the same message set.

A complete download operation transfers an “image” data, which is made up of logically separated sections named modules. In order to meet the transmission constraints, such as bit-error rates and maximum transport packet size, each module is then divided into blocks, all with the same size. The overall operation is made up of six messages, which are divided into control messages and data messages: the former are used for setting up the transmission parameters, while the latter convey user information.

Since OUT interest is focused onto broadcast transmission, only two messages have to be used in

386

oad parameters are InfoRespoiise dessage that d server to set up the ction has been set up, the y the DowizloadDataBloclz

), block size, number of image, and specific

ach module. Moreover, the syntax d parameters relevant r the reception of this

message, the client is ready to receive and to parse the data messages.

block of the download image, and conveys some additional information (enc within the header of the message) regarding the tity of the module to which it belongs, the version of the da number.

The DowizloadDataBlock message cont

5. THE CODING LAYER STRUCTURE

The hypertext coding and transmission are implemented using three layers. The first layer has

oduced to convey specific information about , such as the name and the path. This

information is necessary for th ect rebuilding of the file system, so that every preserved. The second layer provides the fun ecessary to set up the connection and to transmit the files to the client. For this purpose the DSM-CC download message structure has been adopted in order to obtain

eam compatible with a generic DSM-CC parser. choice increases the application portability, at

the cost of a slight overhead (due to fields in the command syntax). The transport layer, which is the s Transport Stream. The following some design details of the above de To ease the comprehension of the proto some tables are reported, which make syntax description convention defined in the ISOIIEC- 13818 document.

A The file section

syntax of the file sec

look-up table (LUT), p -

section. syxtax (ISO/IE

nloadIitfoRespoizse synta 13818 syntax description coitvewtioii).

downloadld blocksize windowsize

downloadld blocksize windowsize ackPeriod t CDo wnload Win 4 t CDo wnloadSce 4 comp a tibilityL ength 2 numberOfModules for( i=O; i <

n um berOfModu lesi++) { module

value

-

module Size module Version modulelnfoLength compression- flag reserved

path-description-byte

additional fields were introduced for obeying the message syntax constraints. Furthermore, to the fields having values that are constant or not significant for the application, pre-defined values were assigned; as a matter of fact, these fields could not be removed from the message in order to be correctly parsed by a DSM- CC decoder.

The DownloadInfoResponse starts a download session by conveying some general information about the session and specific information related to each module. 'The downloadld is a 4-byte long field that identifies the download session. The field following the dowiiloadId one conveys the length of each block for the present download session; in our application, the adopted size was 4018 bytes, which is the maximum size allowed by the transport layer (each download message is encapsulated in packets with maximum dimension of 4 Kbytes). This choice has been done in order to minimize the overhead due to the transmission of the header.

Each module is associated to a n identifier, called the moduleld, which must be unique within one download session. Moreover, the size of the module and its version number must be transmitted. The former (moduZeSize field) allows the decoder to identify the end of the data messages, the latter (moduleVersioiL field) gives the client a tool to detect image coherency problems. Such problems may arise when the transmitter is updating the information, so that the module has a version that is not consistent with that of the image. In this case, the client can abort the download operation and restart it in a successive cycle.

Table III. Type-code assigninelits.

file type 1 code IIfile type I code html I 000000 I1 uostscriDt I 0001 11

001001

000100 AIFF-C 000101 proprietary from 001 100

0001 10 (user defined) to 11 11 11

Finally, the original message syntax gives the opportunity to introduce a user-defined modulehfo field, whose length is coded in the rnoduZeIiifoLength field. We used the inoduZe6ifo field to send specific information for each module file, that is the compression state (coinpressioiiflag) and the file type (type-code). The former is a l-bit field that is set to 1 when the file has been compressed (this could happen

387

for a HTML document, in order to reduce the bitrate required for the transmission). The latter is a 6-bit field that conveys the file type, represented by one of the codes reported in Table 111.

Finally, the message conveys a priuateData field that contains general private information about the image. We used this field to introduce the path-description, which provides the tree structure of the file system on a pre-order transversal method. The description given for each node is related to the directory name length (coded in 1 byte), the directory name, and the path code (also coded in 1 byte).

After the node descriptiqn, another directory-itame-length is expected if the last node is not a leaf, otherwise a back-code of zero value is introduced. The back-code indicates that the next node is located one step back in the path. The end of the description is given by the priuateDatalength. As a n example, Figure 2 shows an easy tree structure with the corresponding path description fields.

field directory-name-length directory-name path-code directory-name-length directory-name path-code back-code directory-name-length directory-name path-code

byte 1 3 1 1 5 1 1 1 4 1

Figure 2. A simple example of the 1 field.

value 3 "one" 1 5 "first" 2 0 4 "last" 3

:th Description

Once the client has received the whole DownloadInfoRespoiLse message, he is able to parse the data messages and to build the file system through the DowitloadDataBlock messages, whose syntax is shown in Table IV. In order to identify the module to which the block belongs, the module identity is transferrcd; if it is not equal to the moduleh! sent in the control message, an error is detected. Also the identity of the subsequent field (i.e., the inoduZeVersioii) must be the same of the control message. The blockNuinber is used for numbering the blocks of the module, starting from zero for each

388

blocks but the last, which may be shorter.

module Version

blockNumber

Table V: DSM-CC dowidoad message header forinat.

syntax dsmccDownload DataHeaderO

protocolDiscriminator (

dsmcc Type messageld

downloa reserved

messageLength for (adaptationLength>O) {

dapta fronHeader() 1

I

bytes value =F= 2 I OxlOOZ ’

The DSM-CC download proced further encapsulation of the me header, as illustrated in Table V. information about the type transmitted, as well as any adaptation data needed by the transport mechanism; this includes con access information required t o decode the data.

a t the message is , the dsrnccType

assume two 0x1002 and Ox

the download session integrity, and correlates the Dowi~loadDataBlock with the dow

ensure error

structure is used the messages

multiplexing, each elementary s into transport packets (TP) with fixed dimension (188 bytes). These packets consist of two parts, the packet header (4 bytes) and the payload containing the elementary stream data. Summarizing, significant fields are: - syitc-byte: it is an 8-bits field wit

‘01000111’ (Ox47), used for encoder and the decoder;

- packet-ideiLtity-iwi that uniquely ide carried within the TS packe

ed for a particular type o - uity-counter: it is a 4-

sequence numbering. I t is incremented with each Transport Stream packet! haviag the same PID

Two conditions have to be respected, as the first byte of each private section must be the first byte of a transport packet payload, and every TS packet

ntain data coming from one private section only n a TS, several elementary streams

together, each one identified by a PID. Moreover, elementary streams can be grouped together into programs (e.g., the elementary streams containing the audio and the video information of a movie). For this

reason, the decoder requires additional information to distinguish the streams included in a program, and to identify the list of prograins conveyed within the TS. This information is called Prograin Specific Inforination (PSI) and includes a Prograin Association Table (PAT), as well as a Prograin Map Table (PMT). The PAT lists all the prograins in a transport stream; its position in the stream can be easily located by the decoder, for the TS packets containing the PATS have PID equal to zero. For each prograin, the PAT gives the PID of the TS packets containing the PMT (each program has a specific PMT). Then each PMT gives specific information about the relevant program and specifies each elementary stream being part of the program.

Several webs may be conveyed in a TS, and each download session makes a prograin of only one elementary stream. Hence, different PID values are assigned to each download session.

389

generating a Transport Stream. The decoding process de-multiplexes the stream and rebuilds the file system of the received hyper-document. The two processes communicate by means of a stream socket in the Internet domain.

A The eizcodiizg algorithm

The algorithm developed for the simulation of a Transport Stream encoder essentially analyses the directory structure of the hypertext to be transmitted, and executes the multiplexing by inserting all the necessary information.

The file systems belonging to the webs to be multiplexed are placed in different "root" directories. The operation of multiplexing data coming from different webs is performed by the encoder through a cyclic scanning of data coming from different file systems, and by assigning equal priority to each web: the operation terminates when all the files have been sent.

At the end of one operation, the broadcasting cycle is restarted; this simulates a real application in which all the multimedia information is continuously refreshed, allowing for a real-time updating. This continuous updating of information is particularly useful in certain applications, such as stock data exchange and distributed databases. Furthermore, it allows the user to receive data in every moment, by decoding one of the refresh streams. A general layout of the implementation of the encoding process is presented in Figure 3.

6. SIMULATION OF HYPERTEXT TRANSMISSION

In order to simulate the transmission of a hypertext according to the ISO-13818 specifications, we used two UNIX workstations linked together by a LAN. It simulates a distributed application based on a client/server model, where the server implements the encoder that delivers transport packets in response to a client (decoder) calling for a connection.

To this end, both a software encoder and a prototype of the corresponding decoder have been implemented. The web data are encoded and multiplexed, by employing the DSM-CC download protocol, so

TS Transport

Stream

Figure 3. The coding and inultipleriiig system layout.

390

order to assure a hig s introduced, which pro oder; in particular, the

of webs to be multiplexed, belonging to each web, and associated to these webs.

i I readthestartandtheI

I I I 4

i,

788

Figure 4. The encoding scheme.

This file can be substituted every refreshing cycle, rogram execution, wi

a real-time updatin

ng the configuration file na eginning of each cycle. Fur

being transmitted to the decoder. This is

consequently to suspend data transmissio execution, by setting a particul

ure 4, the encoding s configuration files, the

program performs the of the PSI sections. To each web a proper PID is assigned, by starting from the number 16 and by coding the PMT sections without any stream descriptor. As a result, PAT and PMT tables are packetized and multiplexed a t the beginning of the Transport Stream. The following step consists in sending the Dowizload message, in a private section, for each download session.

, the transmission of the data file into blocks, he DowiiloadDataBlock messages. These

are first encap d after divided

first section) or 184 bytes the data within the trans

the multiplexing ort packet from a dlfferent web at each time.

B The decoding

The decoding algorithm accomplish multiplexing of the programs inser Transport Stream and rebuilds the structure of the hypertexts being transmitted, on a fact, it extracts the payload packets, and uses it to create a system a t the receiving platform.

Furthermore, we made the hypothesis that several webs are multiplexed together; owing to t h s , the decoder is able to re-create all the file systems needed to locate each file in the right path. The decoding software re-create these directories as well,

As previously mentioned, the PSI tables play a fundamental role in the delive e main information to parse the stream and to perform the stream de-multiplexing. As a consequence, both PATS (PID=O) and PMTs must be decoded sequentially a t the beginning of the reception cycle.

As shown in Figure 5, relevant to the decoding scheme, the first operation is the parsing of the control information sent using the PSI tables. In such way, the decoder knows how many elementary streams are multiplexed together, as well as the PIDs related to each web. Thus, the decoder sets up all the

39 1

program registers that are used for the mapping of blocks and packets, which are received and must be

correctly decode the module-file size, in order to avoid errors in decoding the data messages; these errors can

recorded.

m receive a transport packe

decode PAT packets

I

decode a PMT packet L-J

I receive a DIR packet I

information in program variables

4 packets

?-I receive a transport packe

containing web data blocks

compromise the overall download operation. The last part of the message conveys the

information necessary to rebuild the tree structure of the file system, by associating a byte code to each path. The tree structure is created once the overall control message has been received and before the reception of the files.

The final step consists in decoding the transport packets that ' carry the blocks containing the transmitted files. The first block of each file delivers also the file-header, which allows for its localization inside its own file system. If the path does not exist, then a n error occurs and the relevant file data are discarded.

All the information conveyed in a cycle can be identical to the preceding cycle. Unfortunately, there is no way to recognize any change of data before receiving them; hence, it is necessary to perform decoding and rebuilding of files every time.

7. CONCLUSIONS

In this paper a new protocol for broadcasting hypertextual data within the MPEG-2 Transport Stream was proposed. In particular, specific reference to the features of HTML webs was made, and a novel method to multiplex different hyper-documents in the same numerical stream was developed. To achieve this goal, the DSM-CC download protocol was used, defined by the ISO-13818-5 specification, which has been adapted to the specific application and to the transport layer characteristics. This work demonstrated, through a computer simulation of a real network application, the flexibility and the reliability of the proposed protocol, and the possibility of efficiently broadcasting interactive multimedia information in the framework of a DVB system.

Furthermore, this model may constitute a basis for creating other advanced multimedia services in the field of digital TV broadcasting.

ACKNOWLEDGMENTS

The authors would like to thank Mr. Paolo Dettori for useful discussions and suggestions.

REFERENCES Figure 5. The decoding scheme.

Afterwards, the Dow~tZoadI~tfoRespo~tse messages must be parsed. By means of these messages it is Dossible to determine the block size and the specific features related to each file. It is fundamental to ~ ... - "

1. ISO/IEC JTCl/SC29/WGll, ISO/IEC 13818-1: Information technology - Generic coding of moving pictures and associated audio - System, November 1994.

2. ETSI. "Digital broadcasting svstems for television:

392

Specification for conveying ITU-R system B Extension for Digital Storage Media Command teletext in digital video broadcasting (DVB) and Control," Dallas, November 1995. bitstreams," prETR 300472, November 1994. 4. N.Woodhcad, Hypertext and IT-. .

Information technology - Generic coding of moving Cliffs, 1991. pictures and associated audio information -

1 In, . 3. ISO/IEC JTClISCZ9NVG11, "ISO/IEC 13818-6: and Applications, Addison '. .

T,uigi Atzori (S'97) was born in Cagliari ( 1 1 11: I on 1971 He received the Laurea ::P', in Electronic Engineering from the

! ' . i i \ t5rsity of Cagliari, in 1996, receiving :iI . .I Telecom Italia award for his thesis \\-,rI, In 1994 he spent some months as a i i-ii:iig student a t the Technical i : I I \ c,rsity in Braunschweig (Germany)

." ])Besent he is a PhD student in i +,mmunications a t the Dept. of

Electrical and Electronic Engineering, University of Cagliari His research interests are in the field of imageivideo processing and transmission

Massimiliano D i Gregorio (S'94) was born in Cagliari (Italy), on 1972. He received the Laurea degree (summa cum laude) in Electronic Engineering in 1996 from the University of Cagliari; he also received a Telecom Italia award for his thesis. In 1993 he was a visiting student a t the Fermi Lab in Chicago. At present he is a Ph.D. student in Computer Science at the Dept. of Electrical and Electronic

Engineering of the University of Cagliari. His main research activity is in the area of remote sensing image classification, neural networks and image databases.

_1

Francesco G.B. De Natale (M'97) was born in Genoa (Italy) in 1964. He ,received

, . . the Laurea degree i o , Electronic .. . Engineering in 1990 and the Ph.D. degree

Y I . , , . in Telecommunications in 1994, both from the University of Genoa. In 1994-96 he was a Visiting Professor a t the University of Trento (Italy), teaching the courses on

'"4 "Pattern Recognition Techniques" and "Telecommunication Systems". Since 1996 he is an Assistant Professor a t DIEE (University .of Cagliari, Italy). His main

fields of interest concern . signal processing, with particular attention to analysis, classification and coding of image and video data.

' . " L

Dr. De Natale is a member of AEI.

Daniele D. Giusto (M'86) received the Laurea degree (M.Sc.) in Electronic Engineering in 1986, and the Dottorato di Ricerca degree (Fh.D.) in Telecommunications in 1990, both fiom the University of Genoa '(Italy). Since 1994, he is Assistant Professor with the Information and Communications Lab a t the Dept. of Electrical and Electronic Engineering, University of Cagliari, Italy; since 1995, he is also a Visiting Professor

of Image Processing and Transmission a t the School of Advanced Optics, Free University of Nuoro, Italy. In summer 1995 he was a Visiting Researcher a t the Institut fur Nachrichtentechnik, Technische Universitat Braunschweig, Germany. His research

.. : for the transmission. He has been involve(: : . . .:.. . , . .. European Commission within the RACE, ESPRIT-LTR. and MAST programmes.

In 1993, he received the Ottavio Bonazzi best paper award from AEI. Dr. Giusto is a member of EURASIP and AEI.

interests are in the area of ~':;:.*-~'''::':.::- :. , I - - -,,-.:ng and