ORIGINAL RESEARCH

QoS control framework for content satisfaction in ubiquitousmultimedia computing

K. L. Eddie Law • Sunny So

Received: 5 March 2011 / Accepted: 5 September 2011 / Published online: 19 October 2011

� Springer-Verlag 2011

Abstract Ubiquitous computing platform enables inter-

actions among users, applications, and services through

wireless devices. People always connect to the Internet

while moving. Although portable device can provide high-

speed connectivity, a user may move into area with weak

signal or thin bandwidth, which possibly hinder the

reception of satisfactory multimedia traffic. In this paper, a

ubiquitous computing platform is designed for quality

control of multimedia and data context through the Internet

to users. For the proposed platform to perform properly,

traffic content should be classified into real-time or non-

real-time, casual or critical, data or multimedia traffic type.

Indeed, a session may consist of different traffic classes.

Information for classifications may be identified through

the communications between content providers and clients,

or through direct communication between the platform and

providers or clients. It is the goal of the proposed Quality

of Service control framework which provides mechanisms

for delivering satisfactory content to subscribers, through

content adaptation and selections. The focus of this paper

will be on setting real-time performance bound regarding

multimedia traffic. Quality of images and videos can be

adapted to fit different performance constraints in net-

works. Furthermore, traffic becomes redundant and useless

when delivered information exceeds the delay tolerance of

human beings. In our proposed platform, modifications are

executed through agent technology inside the Internet.

Thorough experiments have been carried out, and the

impacts of real-time bounds are examined. Measured

results indicate that the proposed framework works and

meets the expectations of end-users.

Keywords Egress agent � Ingress agent �Monitoring capsules � Perceptual satisfaction � QoS control

framework � Ubiquitous multimedia computing

1 Introduction

With the advancement of the latest high-speed wireless and

smart phone technologies, people with portable devices can

easily stay connected with the Internet while traveling

(Satyanarayanan 2001). Mobile clients at different loca-

tions may encounter a wide range of networking conditions

during the course of their travelings. A few unpleasant

characteristics, that mobile users may commonly encoun-

tered, can be intermittent interrupts, low bandwidth, high

latency, or high expense. In general, a connection with one

or more of these properties is considered as having weak

connectivity. On the other hand, wired connections rarely

suffer any of these shortcomings and thus offering strong

connectivity.

With mobility, a person may have just moved into a

crowded area, the wireless accessing quality may be

changed abruptly. If the access bandwidth is drastically

reduced, it may take much longer time to download a data

file. Certainly, some people may not mind waiting longer

for downloading data files. But pauses or packet losses in

real-time multimedia communication, such as in voice or

video connection, may create unpleasant perception for

content recipient. Perceptual satisfaction is also called

Quality of Experience (QoE) (ITU-T Recommendation

K. L. E. Law (&)

Kirin Cloud Solutions, Hong Kong, China

e-mail: eddielaw@ieee.org

S. So

ViXS Systems, Toronto, Canada

e-mail: sunny.so@utoronto.ca

123

J Ambient Intell Human Comput (2012) 3:103–112

DOI 10.1007/s12652-011-0077-8

P.800 1996; Rahrer et al. 2006) which is used to identify

the overall performance of a multimedia system from the

point of view of a content subscriber.

How do we quantify the receiving multimedia quality?

Multiple factors can contribute to the final perceptual sat-

isfaction of a subscriber. These may include (1) the regu-

larity of connection breakdown, or loss of connection; (2)

the number of packet loss, or loss of video frames corre-

spondingly; (3) the variation of traffic latency, or the fre-

quent occurrences of jittery frozen video frames; and (4)

quality of reconstructed images or videos. Networking

conditions and receiving multimedia quality are inter-

related. If the connecting path between content provider

and subscriber is abundant in terms of bandwidth with a

short propagation delay, then in general, the subscriber

may not suffer any of aforementioned problems. But when

the available bandwidth is limited or insufficient for

delivering requested multimedia traffic in a timely fashion,

then any of these problems may arise. To an end-user, a

basic method to judge a multimedia communication ses-

sion is to quantify the receiving multimedia quality.

Commonly, the peak signal-to-noise ratio (PSNR) is

selected for providing quantitative evaluation of receiving

multimedia content. It may be a reasonable system per-

formance measure for visual perception if signal errors are

spread evenly across an image, but it may fail to identify

blocking and regional corruptions in images and videos.

Another possible choice can be root-mean-square error

measure RRMS, which is based on retinal response model

(details on the computation of RRMS can be found in

Appendix). In fact, either PSNR or RRMS can be used in the

proposed framework. We select RRMS for its frequency

responsive model in assessing receiving image quality. In

the proposed QoS framework, the RRMS is used to check the

accepted lower multimedia quality bounds of subscribers.

Hence, no real-time RRMS computation is needed in the

proposed framework; instead, it is used only in the initial

calibration operations of every new content subscriber.

Then next, how do we construct a ubiquitous computing

platform which can offer satisfactory QoE to a content

recipient? As of today, agent technology has found a wide

range of applications (Chen et al. 2010; Lee et al. 2007;

Pratistha et al. 2005; Jrad et al. 2005). Then, it can also be

used for serving different purposes and services on the

Internet. Agent entity can be constructed on top of existing

protocols to achieve different system goals. There are

examples such as transportation control (Chen et al. 2010),

and remote site supervision (Lee et al. 2007). In fact, with

the pervasiveness of the HyperText Transfer Protocol

(HTTP) (Fielding et al. 1999), web browsers can be used to

access different agents for control and management (Pra-

tistha et al. 2005). In this paper, an agent-based Quality of

Service (QoS) control framework is proposed for managing

network traffic context based on the latest networking con-

ditions for mobile clients. The focus of this paper is to

elaborate the functional details and protocol designs of the

proposed framework, and how it deals with different types of

multimedia content to meet the expectation of subscribers.

2 QoS control framework

For the networking infrastructure shown in Fig. 1, initially

the traffic communicating between a mobile client C and

service provider S is traveling over the Path 1 as shown.

Upon moving, the client C may have moved to another

location, and the communicating path has been switched to

Path 2. The associated networking parameters might have

completely changed, for example, the bottleneck link

bandwidth and propagation delay, etc. As a result, the

amount of data flow and traveling latency could be dif-

ferent, which then impact the perceptual quality of

receiving multimedia traffic.

Our proposed QoS control framework is called Multi-

media Agent Framework. The design goal is to adapt traffic

content to changing network constraints dynamically. The

framework is constructed based on agent technology. There

are a few basic components for the system to operate

properly. In Fig. 1, a functional connection is consisted of

at least a content provider S, a mobile client C, and two

agents, which sit at the edges on the Internet. Depending on

the direction of the traffic flow, the agents are generically

named the Ingress Agent (IA) and Egress Agent (EA). For

the depicted traffic flow from a server to a client, the agents

connecting to the source and destination are called ingress

agent and egress agent, respectively. Since wireless con-

nections are not always stable, a mobile client may

encounter different perceptual experiences while traveling.

For example, the access bandwidth between C and egress

agent EA2 is 54 Mbps for 802.11g, while the one between

C and EA1 can be 11 Mbps for 802.11b. And in this paper,

we assume that the bottleneck always occurs at the wireless

link between egress agent and client.

Fig. 1 An operating model of the multimedia agent framework

104 K. L. E. Law, S. So

123

To sustain a satisfactory user experience of a mobile

client under changing networking conditions, traffic con-

text can be adjusted accordingly (Byess et al. 2004; Lum

et al. 2002; Banavar et al. 2002; Noble 2000) through the

proposed QoS control framework. Apart from the ingress

and egress agents, the framework allows content alterna-

tion within the Internet upon permitted by both the content

service providers and subscribers. But at the current stage,

we focus on the basic operating model which functions

between the two agents. In future, this model can be

extended to the routers inside the Internet.

A traditional client–server transaction model is shown in

Fig. 2. A web page is being retrieved from a client. Sup-

pose that it is consisted of one HTML text document, and

one picture file. Two separate HTTP GET requests from

client can be used to retrieve the two files from the server.

But in multimedia agent framework, a high-level concep-

tual design feature is enabled through the two agents, as

shown in Fig. 2. Upon receiving the HTTP GET request

from client, the egress agent also delivers information

regarding the associated resource constraint of the wireless

link to the ingress agent. Then the ingress agent can rep-

resent the client to collect both files and examine if the

latest available resources are sufficient to receive both files

in perfect condition. If not, the ingress agent can adjust the

file context to meet the latest available QoS constraints for

the client.

To have the framework worked as expected, structural

control message flows between the egress and ingress

agents have been designed. And these control messages are

called ‘‘capsules’’ messages. In order to create a light-

weight and effective design, multiple operating phases are

introduced, which include: initialization, QoS negotiation,

and provision phases (Fig. 3). When a client moves from

one access point to another access point, packets may be

lost temporarily due to incorrect routing. This error can be

reduced if the path change indication can be saved within

the Internet. At the moment, this layer 3 operation is not

investigated in this paper. Our current focus is on the

changes of resource constraints, whereas multimedia

information may not be able to provide the expected

quality at the recipient.

For a newly moving-in mobile user, an egress agent may

not be aware of any existing active connections. It starts to

take notice, if this client sends a new request, retransmits a

request, or the agent receives an incoming redirected

messages from a server through the mobile IP protocol

(Law et al. 2010; Johnson et al. 2004; Perkins 2002). In

either of these cases, the egress agent should start an ini-

tialization phase. That is, the egress agent attempts to

establish and register a relationship with the corresponding

ingress agent. The next stage is to begin a negotiation

phase. At the current stage, the information being passed

between the agents is for QoS monitoring. This enables the

ingress agent to make decision on service class selection

with an estimated performance for a specific traffic type. In

(a)

(b)

Fig. 2 Network transaction models for multimedia traffic. a Traditional

client-server model, b conceptual model of multimedia agent framework

Fig. 3 Operating phases of the framework

QoS control framework for content satisfaction 105

123

future, information regarding service subscription should

be integrated. This indicates that the right of use of a ser-

vice class must also be verified if it is covered in its paid

services. Then afterward, based on measured QoS param-

eters, the ingress agent makes decision if any modifications

should be applied to the traffic, which should be delivered

with satisfaction expectation of the subscriber. The role of

QoS monitoring plays an important role in determining the

method of adaptations which should be carried out.

2.1 QoS monitoring

There are different factors that may affect the receiving

quality of multimedia or data traffic. It may belong to real-

time or non-real-time, casual or critical traffic type. In our

platform, the networking conditions should be estimated

for the agents to provide content adaptation, reliability, etc.

Monitoring capsules are sent between agents to estimate:

• end-to-end delay,

• bottleneck link bandwidth,

• conditions of the wireless access link,

• available CPU resources.

Upon sending a control capsule from an egress agent

towards the server direction, the ingress agent registers the

destination and sends measurement capsules regularly to

the egress agent. Data can be collected and updated along

the path towards the egress agent, if the routers along the

path are also equipped with the agent capabilities. These

routers are called transit agents in our framework. But if

routers do not carry these capabilities, then those regions

are not monitored. In our current assumption, the bottle-

necks are always at the access links.

2.1.1 Measuring round-trip delay

A timestamp is put in a capsule whenever it is sent. If a

transit agent understands the capsule, it copies the time-

stamp and returns a result capsule to the ingress agent. This

enables ingress agent to collect more information regarding

different hops along the path. When a result capsule from

the egress agent is collected, the ingress agent subtract the

returning time from the timestamp. Then the round-trip

time (RTT) can be collected. This RTT can be further

carried out through averaging mechanism for more robust

estimation. The framework can also offer packet retrans-

mission, then this RTT is used for generating the retrans-

mission timeout in the system.

2.1.2 Bandwidth measurement

Linux operating system keeps a cumulative byte counts

received and sent at each interface. The file /proc/net/

dev in Linux at each node keeps the real-time cumulative

byte counts of traffic sent and received on each network

interface. Suppose bj,i and bj?1,i are the reported byte

counts at the jth and (j ? 1)st measured instants, respec-

tively, of port interface i. Then, the tj,i is the corresponding

duration between two measuring instants. If Bj,i is the

average of bandwidth utilization of port interface i, we

have

Bj;i ¼bjþ1;i � bj;i

tj;i(bytes/s):

2.1.3 Computing power measurement

Another potential useful measurement is to find the avail-

able CPU resources in networks. Modifications and adap-

tations require CPU resources in order to change traffic

context efficiently. Measuring available CPU idle per-

centage can be found in file /proc/stat. A running

CPU can be in one of the four states: user, system, nice,

and idle. /proc/stat provides a real-time information

regarding the cumulative state counts of each of the four

states. The kernel increments one of the counters, based on

the state of the CPU, every 10 ms. Since we are interested

in the idle percentage, we can estimate the idle percentage

over a period as follow. Suppose cj and cj?1 are the idle

counts at the jth and (j?1)-st measured instants, respec-

tively. And the dj and dj?1 are the sum of all counts cor-

respondingly. If Ij,i is the average CPU idle percentage,

then we have

Ij ¼cjþ1 � cj

djþ1 � dj:

3 Testbed evaluation

A testbed consists of ten nodes has been set up to validate

the QoS control framework for multimedia traffic. Exper-

iments have been carried out to confirm if the proposed

QoS control framework can adapt traffic content to meet

the expectation of subscriber. One set of experiments is to

examine the delivery of web pages with real-time infor-

mation component. Another set of experiments is to adapt

video stream to meet the link bandwidth constraint in real-

time. For both experiments, classification of traffic types

have been preset for carrying out expected component

adaptation accordingly.

3.1 Real-time delivery of web pages

In this set of experiments, a web page consists of multiple

informative components is sent every 2.5 s. As listed in

Table 1, these components are pre-classified into three

ranking classes based on their relative importance. Rank 1

106 K. L. E. Law, S. So

123

information is considered the most important, and it

should be sent to the subscriber whenever possible. Then

there is a picture in the page. It belongs to rank 2 class,

and can be compressed with lower resolutions upon nee-

ded. The other rank 3 components in the HTML page are

for creating an appealing look of the page only. They are

not as important and can be dropped if resources are

running low.

During the experiments, the ingress agent computes the

desired adapted size, sADAPTED, after learning the limiting

bottleneck bandwidth through receiving the result capsules

from the associated egress agent. Measured data are aver-

aged through the last 100 samples. Certainly, wireless

access link bandwidth varies due to the mobility of the

user, which is simulated through changing the Linux traffic

control function. Different access bandwidths are used for

testings, which include 115 kbps, 1 Mbps, and 2.4 Mbps.

Furthermore, we have imposed different real-time con-

straints for the delivery of this web page information. The

patience limit of a user is interpreted as the real-time

delivery constraint. For the tests, the constraints with val-

ues of 1,000, 800, 600, and 400 ms are used in testbed.

This limit setting indicates that the user shall re-click or

reload the page when the time is reached. This is the goal

of the framework to deliver at least the rank 1 traffic to the

subscriber within this time constraint.

In Fig. 4a, the measured sADAPTED is shown under dif-

ferent user patience limits and bottleneck bandwidth. Fig-

ure 4b is obtained by dividing the measured sADAPTED with

the bottleneck link bandwidth, thereby giving the bottle-

neck link traversal duration values. When selective deliv-

ery is triggered, the bottleneck link traversal time remains

constant while the bandwidth is shrinking. This trend

continues until only the rank 1 component is delivered.

Thus, the bottleneck link traversal is the reciprocal of the

bottleneck bandwidth multiplied by the size of the rank 1

component. Figure 4c shows the resultant response time

across the networks. If there is no limit on the patience

threshold, the response time exponentially increases when

the bottleneck bandwidth linearly decreases. In fact, when

the bottleneck bandwidth is reduced to 100 kbps, the

response time surges to 12,016 ± 339 ms due to the

exponential increase in the backlog of data waiting to be

delivered to the client at the last-mile link.

The response time characteristics under various user

patience limits are similar. In general, the response time

increases exponentially when the bottleneck bandwidth

decreases. This trend continues until they reach their

respective adaptation thresholds. From then on, the

response time decreases linearly with the reduction of

bandwidth. When the bandwidth decreases even further,

Table 1 Sizes of various components in a web page: original picture

size is 25,958 bytes, maximally compressed size is 13,590 bytes

Rank Content Size (bytes)

1 Real-time data 6,600

2 Picture 13,590–25,958

3 Other data 7,900

(a)

(b)

(c)

Fig. 4 Web page delivery. a Adapting sizes of web pages, b link

traversal times, c response times

QoS control framework for content satisfaction 107

123

the response time reaches a point beyond which ranks 2

and 3 in-page components are dropped. The response time

stays flat at about 150 ms.

3.2 Real-time delivery of images

The next set of experiments is to delivery images across

networks. The bottleneck link is the access link of a sub-

scriber. The real-time constraint W is passed from the client

preference list to the server, and it can be discovered by the

agents in networks. They can detect the existent of real-

time constraint of a communication session, and the

delivery of images based on the meta-information. If there

are multiple good quality paths existing between the two

agents, the ingress agent can then have multiple choices in

carrying out appropriate operations inside the infrastruc-

ture. An exemplary operational details of using a single

path computation in the infrastructure is shown in Fig. 5.

The goal of the infrastructure is to fit the image delivery to

receiver within the duration of W seconds.

The image scaling operation starts when the ingress

agent receives the incoming Lena BMP images, as shown

in Fig. 6a. In the experiments, the ingress agent converts

them into JPEG images, forwards them along a path, and

the running times are measured against the time con-

straints. Regularly, the egress agent reports the measured

bottleneck link bandwidth to the ingress agent. Through the

measurements, the image compression ratio can be esti-

mated through operations in the infrastructure. Different

scaling parameter, k, can be set between 1 and 16 inclu-

sively, where 16 is the best quality and 1 is the worst. The

next question is to determine if the generated images with

selected compression ratio can meet the real-time and

bandwidth constraints. Then it leads to design of an initial

calibration process. The goal of the process is to find the

lowest acceptable bound of the compression ratio for a

subscriber.

With different compression ratios, sizes of resulting

compressed images are listed in Table 2. For example, the

visual qualities of some compressed images are shown

from Fig. 6b–j. Through the initial calibration process, a

Fig. 5 Real-time image scaling operations

Fig. 6 Visual quality of compressed images: a original in bitmap

format, b–j compressed in JPEG formats

108 K. L. E. Law, S. So

123

subscriber can actually select the worst quality image that

he or she can accept, and inform the content provider. With

this information, the infrastructure can then be aware of the

minimum QoE expectation of the subscriber. And from the

table, an ingress agent can pick a lowered but acceptable

bounding value of the compression scaling parameter to

meet the latest networking conditions, if they have just

turned worse. There are two parameters that should be

considered before carrying out compression operations.

The first one is to determine if the real-time constraint can

be met. The second is to determine if the image quality can

meet the quality expectation of a user. Both conditions

must be met; otherwise, the images are dropped at ingress

agent, because there are no good reasons to send an

unacceptable low quality images.

3.3 Real-time delivery of streaming content

Multimedia applications, such as IPTVs, and online con-

ference meetings, belong to the real-time streaming traffic

class with temporal relationship among sending informa-

tion. Typically, a streaming movie session may consist of

three traffic types: video, voice, and/or data such as the

subtitle. Actions are spread across multiple consecutive

video frames. Apart from the blocking effects in images,

video motion may not be reconstructed properly if some

motion vectors for decoding are lost. As a result, subscriber

could have trouble interpreting the video content. When a

video is transmitted across networks, some video frames

can be dropped. Then, long overdue frame group not dis-

playing at all causes viewer waiting for long delay, and the

content may appear out-of-order. Hence, when there are

more frames dropped, the reproducing video becomes jerky

and gives poor perceptual interpretation.

It is important that our proposed framework can offer

satisfactory experience of real-time video delivery. In the

experiments, video frames are extracted from a theatrical

screen advertisement promoting the 1955 Chevrolet mod-

els. The advertisement movie has long sequences of car

movements. Inter-frames smoothness can be observed

easily, and the perceived quality can be assessed easily.

For demonstration purpose, this movie clip does not

carry voice component which is actually replaced by sub-

title. Each frame in video is a 24-bit RGB bitmap image

with a size of 63,414 bytes and a visual dimension of

176 9 120 pixels. The sizes of the accompanying subtitles

range from 97 to 120 bytes. Compression ratio may differ

frame by frame depending on the networking conditions.

With the highest compression ratio, compressed frame

images have sizes ranging from 8,806 bytes (13.89% of the

originals) to 12,674 bytes (19.99%). Two frames are

extracted every second for creating one new accompanying

subtitle. Eighty-eight frames are extracted. At the server,

the video clip is replayed continuously in the experiments.

The relationship between compressed size and compression

scaling parameter, k, is encoded using meta-information.

Thus, the network infrastructure can efficiently choose the

closest adapted size, sA, with an appropriate compression

parameter k. Then the network nodes execute compression

operations according to k. The real-time delivery scheme

of the video experiment is based on a two-level ranking

levels: (1) subtitle is classified as rank 1, R1; (2) frame

image is classified as rank 2, R2; where rank 1 has higher

priority than rank 2 traffic.

For video communication, the ingress agent can have

choices on delivering frames across the networks.

Depending on the latest measured network conditions, the

agent can send frames unchanged, compressed, or simply

drop the frames. Certainly, frames sent may still get

dropped inside the networks due to congestion, and these

are called frame lost events.

Two snapshots of the movie are shown in Fig. 7a and b.

When the network delay is too long with a real-time delay

constraint being applied. Then frames are dropped at the

ingress agent to reduce the traffic load in testbed. For

example, as shown in Fig. 7c, the movie is stuck at frame

number 50, but the subtitle has arrived at frame 56. This

implies that the proposed pervasive infrastructure can

enforce certain minimal QoS quality. In this case, the

receiver is still able to see the latest subtitle section, or he

or she can listen to the voice part if the voice section exists.

An MPEG-1 viewer at the subscriber has been created to

compare a locally stored copy with the remotely retrieving

video. The viewer, as shown in Fig. 7d, also reports the

packet loss and CPU consumption.

For performance measurements, the link speed of the

simulated wireless access shrinks from 3 Mbps to

100 kbps. With different real-time constraints applied,

different operations on video frames have been carried out.

If bandwidth is abundant, frames are sent uncompressed, as

in all constraint cases with link speed of 3 Mbps. But when

the link rate reduces, and the real-time constraints is

Table 2 Visual quality and RRMS(C, R)

k Size (% of original) RRMS(C, R) (%) # in Fig. 6

1 24 5.026 (b)

2 36 3.985 (c)

3 45 3.376 (d)

4 51 2.984 (e)

5 55 2.607 (f)

6 63 2.393 (g)

10 77 1.806 (h)

13 89 1.562 (i)

16 100 1.409 (j)

QoS control framework for content satisfaction 109

123

Fig. 7 a Frame 50, b frame 56, c time at frame 56, picture at frame

50, d comparing viewer with traffic load monitoring

Fig. 8 Video stream modifications. a Real-time constraint of 400 ms,

b real-time constraint of 600 ms, c real-time constraint of 800 ms,

d real-time constraint of 1,000 ms

110 K. L. E. Law, S. So

123

shorter, then more frames are compressed, or dropped. It is

always important to observe that, for the cases reported in

Fig. 8, the amount of frame losses inside the networks are

small. This satisfies the goal of the proposed QoS control

framework for multimedia traffic, that is, to deliver as

much information as possible to the subscribers.

4 Conclusion

In this paper, an agent-based QoS control framework for

delivering satisfactory multimedia traffic across the Inter-

net has been designed. The framework is currently built on

top of existing networking protocols. The major compo-

nents in the platform consists of ingress and egress agents.

QoS monitoring capsules are regularly sent between agents

in order to enable the ingress agent to adapt the content

information, while meeting the requirements and expecta-

tions of subscribers and end-users. At the current phase,

communicating protocol designs between the two agents at

the edges of networks have been tested. Furthermore, dif-

ferent traffic types have been thoroughly experimented.

Through multimedia content classification, whether it is for

real-time or non-real-time, important or unimportant, traf-

fic can be sent to subscribers to meet their expected mul-

timedia quality in our framework. And in future, more

thorough investigation shall be carried out to enable routers

inside the Internet to assist and relieve the loads of the

ingress and egress agents.

Appendix

Peak signal-to-noise ratio (PSNR) is an easy-to-use error

measurement metric, and it is widely used for providing

quantitative evaluation of receiving multimedia quality. It is

a reasonable system performance measure for visual per-

ception if signal errors are spread evenly across an image.

However, upon observing the two rugby team pictures

shown in Fig. 9a and b, both suffer identical overall loss

conditions. The errors are spread across all pixels in Fig. 9a;

therefore, it is visually more appealing than the one shown

in Fig. 9b. These cases indicate the need of using subjective

evaluation upon examining the design of our framework.

One possible filter responsive function is called Visual

Differences Predictor (VDP) (Daly 1993), which can be

adopted to model the retinal sensitivity of human being.

The retinal response to luminance is a non-linear function,

and it is used to evaluate the quality of distorted images.

Algorithm derived for VDP attempts to indicate the prob-

ability of detecting a difference between the two images on

a pixel-by-pixel basis. In our testing, we have adopted a

simplified VDP version using a relative mean calculation.

These computations may not be needed to run at all times;

the image difference computations can be carried out

during quality calibrations or upon user requests.

The luminous response, l(i, j) shown in (1), is called

amplitude non-linearity value,

lði; jÞ ¼ Lði; jÞLði; jÞ þ c1ðLði; jÞÞb

ð1Þ

where L(i, j) is the luminance of the pixel (i, j) in

dimension cd/m2. Typically assigned values for the

parameters b and c1 are 0.63 and 12.6, respectively.

Spatial frequencies fl(u, v) can be obtained through the Fast

Fourier Transform of the luminance sensitivity spatial

pattern l(i, j). It gives the magnitude at horizontal spatial

frequency at u and vertical spatial frequency at v. Human

being is aware of changes in signal contrast. The contrast

sensitivity function (CS) is used to model the retinal

contrast sensitivity to a spatial frequency.

CSðu; vÞ ¼ ð0:008 z�1:5 þ 1Þ�0:2

� 1:42ffiffi

zp

e�0:3ffiffi

zp�

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

1þ 0:06e0:3ffiffi

zpp

:ð2Þ

where z = u2 ? v2.

The CS, in (2), gives a two-dimensional spatial fre-

quency plane (u, v). As shown in (3), the CS function filters

the fl(u, v) values and obtains the parameter Filtered

Magnitudes of the Amplitude Non-Linearity Value

(FMANLV), g(u, v). The g(u, v) measures the luminance

sensitivity filtered by the retinal response with respect to

the spatial frequency (u, v), which is

gðu; vÞ ¼ flðu; vÞCSðu; vÞ: ð3Þ

Calibration operations

There are no needs to run these computations in real-time.

In the proposed platform, the framework can initiate an

initial calibration phase between subscriber and content

provider. In other words, the ingress and egress agents can

subsequently set appropriate preferences for a subscriber.

Fig. 9 a Rugby picture 1, b rugby picture 2. 1% packet loss rate,

10 ms delay, 50 ls jitter, 500 kbps bandwidth

QoS control framework for content satisfaction 111

123

The multimedia content provider may send a reference

image R with a reference gR(u, v) value. Then it sends

other compressed testing image, C, with gC(u, v) at the

recipient under the assumption of certain network condi-

tions. Suppose this is the least accepted quality of a sub-

scriber, then the egress agent can compute a relative image

metric through normalizing the gC(u, v) of the test image

with respect to the gR(u, v) of reference image. The

resulting measure is called the relative root-mean square

(RMS) error of the reference content,

RRMSðC;RÞ ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

P

u;vðgCðu; vÞ � gRðu; vÞÞ2P

u;v gRðu; vÞ2

v

u

u

t � 100%: ð4Þ

A smaller RRMS(C, R) value indicates that the received

image is visually closer to the original one. This quality

bound of a streaming multimedia content can be set

through finding the largest accepted RRMS(C, R) value of a

content subscriber. This bounding measure is returned to

the ingress agent for determining appropriate compression

operations in future.

References

Banavar G, Bernstein A (2002) Software infrastructure and design

challenges for ubiquitous computing. Commun ACM 45(12):

92–96

Byers JW, Considine J, Mitzenmacher M, Rost S (2004) Informed

content delivery across adaptive overlay networks. IEEE/ACM

Trans Netw 12(5):767–780

Chen B, Cheng HH (2010) A review of the applications of agent

technology in traffic and transportation systems. IEEE Trans

Intell Transp Syst 11(2):485–497

Daly S (1993) The visible differences predictor: an algorithm for the

assessment of image fidelity. In: Watson AB (ed) Digital images

and human vision. MIT Press, Cambridge, pp 179–206

Fielding R et al (1999) Hypertext transfer protocol—HTTP/1.1,

IETF, RFC 2616

ITU-T Recommendation P. 800 (1996) Methods for subjective

determination of transmission quality. International Telecom-

munication Union (ITU-T)

Johnson D, Perkins C, Arkko J (2004) IP mobility support for IPv6,

IETF, RFC 3775

Jrad ZE-F, Benmammar B, Correa J, Krief F, Mbarek N (2005) A user

assistant for QoS negotiation in a dynamic environment using

agent technology. In: 2nd IFIP international conference on

wireless optical communications networks (WOCN),

pp 270–274

Law KLE, Lau C (2010) Mobility service agent. In: 3rd IEEE

workshop on wireless and internet services (WISe), in conjunc-

tion with 35th IEEE conference on local computer networks

(LCN)

Lee J-S, Hsu P-L (2007) Implementation of a remote hierarchical

supervision system using petri nets and agent technology. IEEE

Trans Syst Man Cybern Part C 37(1):77–85

Lum WY, Lau FCM (2002) A context-aware decision engine for

content adaptation. IEEE Pervasive Comput 1(3):41–49

Noble BD (2000) System support for mobile, adaptive applications.

IEEE Pers Commun 7(1):44–49

Perkins C (ed) (2002) IP mobility support for IPv4, IETF, RFC 3344

Pratistha IM, Zaslavsky A, Cuce S, Dick M (2005) Improving

operational efficiency of web services with mobile agent

technology. IEEE/WIC/ACM international conference on intel-

ligent agent technology, pp 725–731

Rahrer T, Fiandra R, Wright S (2006) Triple-play services quality of

experience (QoE) requirements. DSL Forum, Architecture &

Transport Working Group, technical report TR-126

Satyanarayanan M (2001) Pervasive computing: vision and chal-

lenges. IEEE Pers Commun 8(4):10–17

112 K. L. E. Law, S. So

123