Author
vinodprime
View
258
Download
0
Embed Size (px)
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
1/83
CHAPTER 1
INTRODUCTION
1.1 ABOUT THE PROJECT
The main aim of this project is to effectively manage the video blogs
(vlogs) and make them more conveniently accessible for users.
A video blog (vlog) is a blog which uses video as the primary content, often
accompanied by supporting text, image, and additional metadata to provide
context.Compared with general videos, vlogs have several unique
characteristics. A vlog often provides textual content as description of the
video. As a medium for communication, a vlog usually has some comment
entries conveying vlog viewers opinions. Some unique but useful
information of a vlog, such as submitting time, viewed times, comment entry
number, and popularity rating, can be easily obtained.
We proposed a novel vlog management model which is comprised of
automatic vlog annotation and user-oriented vlog search. For vlog
annotation, we extract informative keywords from both the target vlog itself
and relevant external resources; besides semantic annotation, and we
perform sentiment analysis on comments to obtain the overall evaluation.
For vlog search, we present saliency-based matching to simulate human
perception of similarity, and organize the results by personalized ranking
and category-based clustering.
1
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
2/83
1.2 LITERATURE SURVEY
Blogging has been a textual activity, but text is only one aspects of the
diverse skills which are needed in order to understand and manage different
aspects of modern communication. Broadband connections are likely to
stimulate a rapid increase in audio-visual services on the web, presumably
changing the future conditions for blogging. Videoblogs can facilitate
practices which promote media literacy and collaborative learning through
the making of collective documentaries. Videoblogs with wiki-like functions
promise to turn users into producing collectives rather than individual
consumers of audiovisual content.
Arguably textual blogging has an effect on journalism by making the
process of gathering information more dynamic, potentially involving the
public before the moment of print publication, or even as providers of
content. When it comes to audiovisual media the users begin to abandon
broadcasting in favour of broadband services . This development will
probably continue as new technologies transform the viewing experience
into a more personalized activities, potentially by-passing the traditional
broadcaster completely .
2
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
3/83
1.2.1 VIDEOBLOGS
The importance of video, being a very powerful medium, the increased
amount of video material on the web, and the possibilities offered byweblogs when it comes to collaboration sums up in videoblogs as one of
the most promising tools which may foster media literacy.
Blogs began as a textual genre of personal publishing, but the genre has
developed visual expressions, like phototblogs, and more recently adapting
sound and video. Most bloggers publish short posts, they write quite often
with an extensive use of hypertext-linking. Linking and commenting makes
blogging a kind of collaboration, where individual bloggers are writing in
context created through collective, but often unorganised effort.
Textual blogs have at least three characteristics, apart from usable and easy
accessible software, which have made them easy to use whether as a
"producer" (writer), a "consumer" (reader) or both: Textblogs are based on
non-temporal media which is easily controllable, they are easy to cite and
they are part of long textual tradition, "re-mediating" many of the features
known from diaries and journals. Even though there are several substantial
differences the easiest way to explain blogging often is to begin with
reffering to "an online diary".
When it comes to audiovisual blogs these are more difficult to explain:
Audio- and videoblogs are based on temporal media and there are no
established tradition which they are closely related to. Audioblogs can
3
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
4/83
hardly be compared to radio or recorded sound and videoblogs are not like
television or private filmmaking: In contrast to broadcasting blogs are
personal and at the same time they are shared by people outside the private
sphere.Production, presentation and distribution: In all these areas bloggingpromise to be close to the opposite of broadcasting. Looking for the sources
of a videoblogging language we therefore have to explore other aspects of
audiovisual culture.
1.2.1.1 SEVERAL VIDEOBLOG-TRADITIONS
First one have to come to terms with what characterizes a videoblog. We
have to distinguish between several significantly different technical
solutions which claim to be videoblogs, ranging from simply uploading
unedited videofiles, via play-lists to edited sequences, sometimes with
complex interactivity. It becomes difficult to define distinct genres, but in
general there seems to be some major traditions, which of course interfere
into a number of sub-genres. I make a distinction between "vogs", which arebased on pre-edited sequences with interactive features, video-"moblogs",
consisting of relatively short, autonomous video-clips, and playlists, which
are collections of references to video-files on different servers.
Some of the characteristics of vogs and "vogging" are formulated in a
manifesto written by Adrian Miles, inspired by the danish filmmaking
initiative Dogme 95. In this tradition videoblogs are personal publishing of
video, exploring the potential of linking, using technology which is easy
available. Vogs are made of edited sequences which normally include
interactive elements. They are typically made with different kinds of
4
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
5/83
software on the producers computer and posted to individual websites. The
other major tradition has emerged along with the introduction of mobile
input-devices with internet connection (smartphones, PDAs, camera
cellphones).
Blogging from mobile devices is particularly interesting in relation to
documentary filmmaking, trying to grasp moments in life provoked or
captured by the presence of a camera. A pioneer within the moblogging
tradition is Steve Mann who has experimented with wearable cameras,
posting images to the Web since 1994 (Mann 1997). Today most moblogs
are based on technology quite similar to textual blogs with posts containing
uploaded pictures or videoclips and additional text. Moblogs are often
hosted by professional service providers where a large number of blogs
share the same infrastructure. Videos on moblogs normally contain
individual videoclips, not edited sequences. Vogs and moblogs are quite
different, both regarding they way they are produced and the way they are
consumed.
Miles makes a good distinction between the two traditions by emphasizing
that "a vog is a video blog where video in a blog must be more than video in
a blog". The posts in vogs are edited and may offer quite complex
interactivity. Therefore those who produce vogs have to combine skills in
the use of software with the ability to manipulate moving images and addhypertextual interactivity. If we restrict vogs to video-content there are not
many voggers around, even including those posting cinematic 2D- and 3D-
animations the number is still relatively small.
5
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
6/83
Even though there are no major technical barriers some skills are required,
preventing most users from becoming "voggers". Moblogs on the other hand
are easy to use, in most cases it is just a matter of simply uploading video-
files to a dedicated webserver. However, this is not only a question of easeof use, possibly even more important is the time which the producer has to
invest in order to get his material on the web. When posting is not time-
consuming bloggers are encouraged to post often, an aspect which has made
a lot of text-bloggers and blog-readers become avid users. The same criteria
for success apply to videoblogs.
Playlists are perhaps not videoblogs, but an interesting genre because they
use technologies which may bridge the gap between vogs and moblogs.
Playlists adress individual files on different servers and may even provide a
level of interactivity without manipulating the content in these files. One
way to achive this is by using SMIL (Synchronized Multimedia Integration
Language) an established XML-dialect defined by The World Wide Web
Consortium in order to control different media distributed through the
internet. SMIL seems to be an ideal platform for distributing various content
as "movies" without moving or manipulating the original source-files
(videoclips, pictures and text). Since SMIL-files are based on an open format
(XML) stored as ASCII-text it is quite easy to make alternative versions of
playlists, taking advantage of server-side applications and the internets
transparent nature.
6
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
7/83
1.2.2 COLLECTIVE DOCUMENTARIES
Looking for existing genres which videoblogs re-mediates the closest we get
are some traditions known from documentary filmmaking. One of these isdiary-films, which are personal first-person narratives. Another tradition is
found-footage films, which often are based on old private material filmed by
others than the filmmaker himself. Found-footage films are part of a larger
tradition known as compilation film, using material from a variety of
sources, including archived material. Any kind of film and video have the
possibility of ending up as found footage: Your grandfathers Super-8movies, old commercials, parts of feature films, recorded televion etc. Quite
a few excellent filmmakers have made their first movies with material
found as leftovers in a studio or in a film school. William Wees discusses
three general ways in which found footage is most often used
1. Compilation : Film where the editor cuts together pieces of footage inorder to illustrate a point. The images are intended to represent reality
and is typical in television documentaries.
2. Collage : Film which use found footage to create metaphors, provoke self-
consciousness and encourage critical viewing. The viewer is able to read
images critically with attention to the metaphors.
3. Appropriation : Film where images are reused in order to be decorative.
Representation is about surface, rather than creation of secondary
7
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
8/83
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
9/83
parts of the production process while others remain almost the same as if the
process was done with an offline computer. After the video-material is
recorded the videoblogging process can be divided into five stages :
1. "Posting", 2. "Selecting", 3. "Editing", 4. "Storing" and 5. "Re-editing".
1. Posting
The success of blogging is partly a result of cheaper net connections and the
increasing number of computer literate people using the web.Videologs may
become a genre where "href tracks", "sprites" and "interactive" elements
enhance the users personal experience in ways that are unique to computer
mediated communication. The considerable downside is that advanced
interactive features have consequences for the amount of work which each
user have to put into his posts. When posting becomes a task the most
important advantage of blogging disappears and the media literacy-
potential will decrease. Following the moblog tradition, emphasizingsimplicity, posting should be as easy as possible. The users should be
encouraged to post short clips of unedited video-material which is
transformed into a unified video-format on the server, becoming a common
resource for future editing and citation. Comments is a way of posting
which is important when building an online community for at least two
major reasons: First comments is the easiest way to become a part of a
community without having to be among those who provide the content in the
first place. Secondly comments help maintaining a community by making
different members aware of each other. Frequent comments give those who
post an explicit confirmation of their publics presence in addition to some
9
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
10/83
substantial feedback. This is particularly important in photo- and video-
blogs because these are personal expressions, often even more vivid than
most textual blogs which in many cases might be considered as varieties of
content management.
2.selecting
A flexible system must allow a media-object to be assigned to multiple
categories, allowing hierarchy but without enforcing it. This will in most
cases be an excellent system usable for a large videoblogging environment,
perhaps even with features combining personal and global categories.
Combined with personal information, time and date of posting and possibly
geograpgical information, this provides metadata for searching or automatic
generation of play-lists which may be re-edited manually.The problem with
searching and search results is that the information is shown out of context.
In order to get an idea of the quality of a specific clip it might be helpful to
know how many who have used it in a sequence. If a lot of people have used
it you have an indication that it might be worth looking at. Sequences are infact the best way to view a clip as long as it provides both context and an
example of how the clip might be trimmed (where to begin and where to
end).
3. Editing
Both the edit and the link provides context to separated segments, but both
theory and practice addressing editing / linking tend to concentrate on the
segments. Editing techniques which have a duration, like dissolves and
wipes, are easier to identify, but they are almost never used unless the
director wants to call for the viewers attention. In an analogous manner
10
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
11/83
theory and practice in hypertext design emphasize the nodes in hypertext
should be designed in order to make navigation as intuitive as possible,
emphasizing the importance of a unified design in order not to confuse the
user. The editing interface has to display all the clips which the user haveselected during a session.The editing process result in a text-document
(SMIL), small enough to allow storage of an almost infinite number of
sequences, which can be made by combining a limited number of video-
clips.
4. Storing
Collective editing capabilities relies on storage which consume as little disk-
space and bandwidth as possible. Downloading video-clips in order to re-
edit a sequence and uploading a new version will not be effective. I would
like to propose an approach to videoblogs focusing on the simplicity known
from moblogs combined with an easy to use editing-interface which makes
it possible to combine clips from different logs into sequences and store
these as SMIL-documents (Synchronized Multimedia Integration Language).In order to make a flexible system all references to users, shots and
sequences are stored in a database, generating SMIL-documents "on the fly".
Because SMIL documents are plain text files which might be played in
Quicktime it becomes an easy task to generate customized movies using
server-side applications. The online editor must be capable of combining
clips into sequences and store these as SMIL-documents in the creators blog.
The SMIL-document include references to the video-clips, controlling order
and duration, positioning, and additional text.
11
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
12/83
5. Re-editing
Re-editing in a videoblogging environment means that any user can take a
sequence or a number of individual video-clips into the editor, make a re-
edit and store the result as a new sequence in his own blog. The idea ofmaking collective documentaries or fiction in this way is intriguing: Those
who are unhappy with a version, may comment on the original sequence or
just make their own version, possibly adding their own content.
1.2.2.2 CURRENT VIDEO BLOGS
Current video blogs are essentially text blogs with externally linked videosfor each entry. Though the fragments of video content form a cohesive diary,
theyre always introduced and navigated to via text. . Typical examples of
current video blogs are Miles teaches the theory and practice of hypermedia
and interactive video at RMIT University, Australia, and uses his blog to
demonstrate some of the ideas.His video blog differs because he includes
timed hyperlinks to other Web resources inside his videos, and haspostproduced speech tracks with timed transcriptions of his speech inside the
QuickTime video files. Its an exceptional video blog, requiring skills and
tools not usually available to the more common blogger. Although those
pioneering the creation of video blogs are finding ample room for
expression, and they obviously enjoy pushing the limits of current
technology, users cite a few problems with current video blogs
theres no way to add comments in video form;
video items cant be easily found via search engines;
video items cant be aggregated easily;
12
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
13/83
interesting clips cant be viewed on their own; instead the video must
be played back in its entirety; and as with most multimedia on the
Web, sufficiently high bandwidth for reasonable quality video isnt
widely available.
1.2.2.3 VIDEO BLOG SEARCH
The aim is to make video blogs as easily searchable by Web search engines
as normal Web pages. Web search engines increasingly support scanning of
RSS and Atom feeds, which allows more tightly coupled searches of actual
blog entries. In fact, the original blogging company Blogger was acquired by Google to bring about such developments. Although a multimedia
syndication language would be just as amenable to scanning and indexing by
search engines, it would unfortunately be able to index only the metadata
referring to a video blog entry, not the actual video content itself. Ideally, the
search engine should index the video content itself, by scanning for
embedded transcriptions and timed metadata or even by performing
automated analysis of the video content directly.
1.2.2.4 VIDEO BLOG COMMENTS
That readers can add comments to blog entries is very popular, allowing
friendly advice and spontaneous discussions to take place in remote corners
of the Web. Its easy to imagine how lively these discussions would be in
video formatif any viewer could easily provide feedback for others towatch and themselves respond to. The technology for such video forums is
already being explored in various areas of the telecommunications industry,
but usually in the context of developing vendor-specific applications for
13
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
14/83
limited numbers of users. In such environments, users are simply able to
point-andshoot to reply to commentary posted by others.
1.2.3 WORDNET: A LEXICAL DATABASE FOR ENGLISH
WordNet provides a more effective combination of traditional lexicographic
information and modern computing. WordNet is an online lexical database
designed for use under program control. English nouns, verbs, adjectives,
and adverbs are organized into sets of synonyms, each representing a
lexicalized concept. Semantic relations link the synonym sets.
1.2.3.1 LANGUAGE DEFINITIONS
We define the vocabulary of a language as a set W of pairs (f,s), where a
formfis a string over a finite alphabet, and a sense s is an element from a
given set of meanings. Forms can be utterances composed of a string of
phonemes or inscriptions composed of a string of characters. Each form with
a sense in a language is called a word in that language. A dictionary is an
alphabetical list of words. A word that has more than one sense ispolysemous; two words that share at least one sense in common are said to
besynonymous. A words usage is the set Cof linguistic contexts in which
the word can be used. The syntax of the language partitions Cintosyntactic
categories. Words that occur in the subset Nare nouns, words that occur in
the subset Vare verbs, and so on. Within each category of syntactic contexts
are further categories ofsemantic contextsthe set of contexts in which a
particularf can be used to express a particulars. The morphology of the
language is defined in terms of a set Mof relations between word forms. For
example, the morphology of English is partitioned into inflectional,
derivational, and compound morphological relations. Finally, the lexical
14
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
15/83
semantics of the language is defined in terms of a set Sof relations between
word senses. The semantic relations into which a word enters determine the
definition of that word.
In WordNet, a form is represented by a string of ASCII characters, and a
sense is represented by the set of (one or more) synonyms that have that
sense. WordNet contains more than 118,000 different word forms and more
than 90,000 different word senses, or more than 166,000 (f,s) pairs. words in
WordNet are polysemous; approximately 40% have one or more synonyms.
WordNet respects the syntactic categories noun,verb, adjective, and adverb
the so-called open-class words .For example, word forms like back,
right, or well are interpreted as nouns in some linguistic contexts, as
verbs in other contexts, and as adjectives or adverbs in other contexts; each
is entered separately into WordNet. It is assumed that the closed-class
categories of Englishsome 300 prepositions, pronouns, and determiners
play an important role in any parsing system; they are given no semantic
explication in WordNet. WordNet includes the following semantic relations:
Synonymy is WordNets basic relation, because WordNet uses sets of
synonyms (synsets) to represent word senses. Synonymy (syn same, onyma
name) is a symmetric relation between word forms.
Antonymy (opposing-name) is also a symmetric semantic relation between
word forms, especially important in organizing the meanings of adjectives
and adverbs. Hyponymy (sub-name) and its inverse, hypernymy (super-name), are
transitive relations between synsets. Because there is usually only one
hypernym, this semantic relation organizes the meanings of nouns into a
hierarchical structure.
15
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
16/83
Meronymy (part-name) and its inverse, holonymy (whole-name), are
complex semantic relations. WordNet distinguishes component parts,
substantiveparts, and memberparts.
Troponymy (manner-name) is for verbs what hyponymy is for nouns,although the resulting hierarchies are much shallower.
Entailmentrelations between verbs are also coded in WordNet.
An XWindows interface to WordNet allows a user to enter a word form and
to choose a pull-down menu for the appropriate syntactic category. The
menus provide access to the semantic relations that have been coded into
WordNet for that word.
1.2.3.2 CONTEXTUAL REPRESENTATIONS
In information retrieval, a query intended to elicit material relevant to one
sense of a polysemous word may elicit unwanted material relevant to other
senses of that word. For example, in computer-assisted instruction, a student
asking the meaning of a word should be given its meaning in that context,
not a list of alternative senses from which to pick. WordNet lists the
alternatives from which choices must be made. WordNet would be much
more useful if it incorporated the means for determining appropriate senses,
allowing the program to evaluate the contexts in which words are used. The
limits of a linguistic context can be defined arbitrarily, but we prefer to
define it in terms of sentences. That is to say, two words co-occur in the
same context if they occur in the same sentence. A semantic concordance isa textual corpus and a lexicon combined so that every substantive word in
the text is linked to its appropriate sense in the lexicon.
16
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
17/83
1.2.4 WORD ASSOCIATION NORMS, MUTUAL
INFORMATION, AND LEXICOGRAPHY
1.2.4.1 MEANING AND ASSOCIATION
It is common practice in linguistics to classify words not only on the basis oftheir meanings but also on the basis of their co-occurrence with other words.
Running through the whole Firthian tradition, for example, is the theme that
"You shall know a word by the company it keeps" .On the one hand, bank
co-occurs with words and expression such as money, notes, loan, account,
investment, clerk, official, manager, robbery, vaults, working in a, its
actions, First National, of England, and so forth. On the other hand, we findbank co-occurring with river, swim, boat, east (and of course West and
South, which have acquired special meanings of their own), on top of the,
and of the Rhine.
The search for increasingly delicate word classes is not new. In
lexicography, for example, it goes back at least to the "verb patterns"
described in Hornby's Advanced Learner'sDictionary (first edition 1948).
What is new is that facilities for the computational storage and analysis of
large bodies of natural language have developed significantly in recent
years, so that it is now becoming possible to test and apply informal
assertions of this kind in a more rigorous way, and to see what company our
words do keep.
1.2.4.2 WORD ASSOCIATION AND PSYCHOLINGUISTICS
Word association norms are well known to be an important factor in
psycholinguistic research, especially in the area of lexical retrieval.
Generally speaking, subjects respond quicker than normal to the word nurse
if it follows a highly associated word such as doctor. Some results and
17
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
18/83
implications are summarized from reaction-time experiments in which
subjects either (a) classified successive strings of letters as words and
nonwords, or (b) pronounced the strings.
1.2.4.3 PREPROCESSING WITH A PARSER
Hindle has found it helpful to preprocess the input with the Fidditch parser
to identify associations between verbs and arguments, and postulate
semantic classes for nouns on this basis. Hindle's method is able to find
some very interesting associations, demonstrate. After running his parser
over the 1988 AP corpus (44 million words), Hindle found N = 4,112,943
subject/verb/object (SVO) triples. The mutual information between a verb
and its object was computed from these 4 million triples by counting how
often the verb and its object werefound in the same triple and dividing by
chance. Thus, for example, disconnect/V and telephone/0 have a joint
probability of7/N. In this case, chance is 84/Nx 481/N because there are 84
SVO triples with the verb disconnect, and 481 SVO triples with the object
telephone. The mutual information is log z 7N/(84 481) = 9.48. Similarly,
the mutual information fordrink/Vbeer/O is 9.9 = log 2 29N/(660 195).
(drink/V and beer/O are found in 660 and 195 SVO triples, respectively;
they are found together in 29 of these triples). This application of Hindle's
parser illustrates a second example of preprocessing the input to highlight
certain constraints of interest. For measuring syntactic constraints, it may be
useful to include some part of speech information and to exclude much ofthe internal structure of noun phrases. For other purposes, it may be helpful
to tag itemsand/or phrases with semantic labels such as *person*, *place*,
*time*, *body part*, *bad*, and so on.
18
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
19/83
1.2.5 AUTOMATIC SEMANTIC ANNOTATION FOR VIDEO
BLOGS
Vlog annotation is essentially a multi-labelingprocess , as a vlog can usually
be annotated with multiple words. There exist many effective approaches formulti-label image/video annotation, and it has become a trend that the
annotation should be extracted not only from the target image/video itself,
but also from other images/videos which are relevant to it.
1.2.5.1 AUTOMATIC VLOG ANNOTATION
In our vlog annotation model, the annotation of a vlog consists of two parts:
the intrinsic annotation extracted from the text of the target vlog and the
expanded annotation from relevant external resources.
1.2.5.2 INTRINSIC ANNOTATION EXTRACTION
Since a vlog often has supporting text in itself, we can extract informative
keywords as its intrinsic annotation. The textual content in a vlog mainly
comprises the title, description, and comments, among which the title and
description are closely related to the semantics of the vlog video, while the
comments are often filled with irrelevant words and thus too noisy to be
used. As a result, only the title and description are kept for annotation
extraction. As the title indicates the main topic of the whole vlog, it is of the
greatest importance for understanding the semantics of the vlog. Therefore
we first extract annotation words from the title. After stop word removal,important words are reserved in the word set Wtitle. For the textual
description, we also remove the stop words beforehand. Then, using the
standard text processing technique such as tf-idf, we can acquire the
important words, and create another word set, Wdescription. Since in the
19
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
20/83
description not all the words are relevant to the semantics of the central
video, Wdescription can be rather noisy. Considering the fact that in an
article, keywords are usually used to reveal the main subject, or the title, we
assume that if an annotation word is a good one, it should be highlycorrelated with at least one word in Wtitle. Therefore, we delete from
Wdescription the words which have low correlation with all the words in
Wtitle.
1.2.5.3 CONTEXT-BASED ANNOTATION EXPANSION
External annotation candidate extraction
Inspired by the search-based annotation methods , we conduct annotation
expansion for the target vlog through a searchbased mode, where a labeled
database is indispensable. As we know, YouTube4 is one of the most
popular video sharing websites which has by far the biggest collection of
videos. Each video on YouTube is labeled by one or more tags. Therefore,
we use YouTube as our labeled database. Given a keyword query, the text-
based video search engine (powered by Google) in YouTube can return
rather good results, hence we can use YouTube search to find the
semantically related videos. For the target vlog, we submit each word w in
Wintrinsic as a query to YouTube searcher, and get the corresponding search
resultsRw (for simplicity, only the top-ranked 20 results are included). For
each result rinRw, we extract the videos representative frame fr(which is
usually the first frame of the video) and the corresponding tags. Then,among the semantically related videos, visually related ones are selected
through content-based similarity between the vlog video and the result
videos found on YouTube. We define the visual similarity between a result
20
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
21/83
video rand the vlog video v as the maximum image similarity between the
representative framefrofrand the keyframefv ofv:
After the above two search stages, we have obtained a batch of videos which
are relevant to the vlog both semantically and visually with regard to theintrinsic annotation word w. We then gather the tags of all the reserved
videos into a tag set T(w), which is adopted as the external annotation
candidates for the vlog. This process is applied for each intrinsic annotation
word w in Wintrinsic. Finally, we obtain the word set Wexternalfor external
annotation candidates:
Context-based annotation refinement
Although the videos used for annotation expansion are all semantically and
visually relevant to the target vlog, it dose not follow that all the tags of the
videos are also relevant to the vlog. In the process of annotation expansion,
we have to deal with the serious problem of semantic drift. Therefore, we
should refine the expanded annotation candidates and delete the irrelevant
words. We calculate the relevance between an annotation candidate c and
the vlog by comparing c with the words in Wintrinsic. As we know, when
comparing two words, we should consider not only the semantics in them
but also the specific contexts they are in. In this paper, we propose a novel
context histogram to depict the semantics of a word in a specific context. For
a word w, its context is substantially a set of words which confines its
specific semantics. We first calculate the one-to-one correlation between w
and each of the words in its context Wcontext. Then, we organize all thecorrelation values into a histogram and get the context histogram forw with
respect to Wcontext. The problem of context comparison is now reduced to
histogram comparison. Here we simply use histogram intersection as a
metric of the context histogram similarity.We perform the context-based
21
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
22/83
external annotation refinement as follows: For an intrinsic annotation word
w ofWintrinsic, we create its context histogram with respect to Wcontext=
Wintrinsic {w}; while for an annotation candidate c in Wexternal, we also
build its context histogram with respect to the same Wcontext= Wintrinsic {w}. In order to compare c with w, we calculate both their one-to-one word
correlation Simword and their contextual similarity Simcontext. The total
correlation between c and w is defined as:
Sim ( , ) Sim ( , total c w = word c w) + Simcontext(c,w) ,
where and are adjustable parameters. Only those annotation candidates
with high relevance to Wintrinsic are kept in Wexternal. After the
refinement, we merge Wintrinsic and Wexternal to get the final annotation
for the target vlog.
1.2.6 ANNOSEARCH: IMAGE AUTO-ANNOTATION BY SEARCH
A novel solution to image auto-annotation problem is rather than training a
concept model using supervised learning techniques as most previous works
do, we propose a data-driven approach leveraging the Web-scale image
dataset and search technology to learn relevant annotations. In an ideal case,
if a well annotated and unlimitedscale image database is available, then for
any query image, we can find its duplicates in this database and simply
propagate its annotation to the query image. In a more realistic case that the
image database is of limited scale, we can still find a group of very similar
images in terms of either global features or local features, extract salient
phrases from their descriptions, and select the most salient ones to annotate
the query image. Thus to by-pass the semantic gap, we can divide and
conquer the annotation problem in two steps:
1) find one accurate keyword for a query image;
22
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
23/83
2) given one keyword, find complementary annotations to describe the
details of this image.
The requirement in the first step is not as lacking in subtlety as it may first
seem. For example, in a desktop photo search, users usually provide alocation or event name in the folder name. Or, in a Web image search, we
can choose one of a Web images surrounding keywords as the query
keyword.
1.2.6.1 THE ANNOSEARCH SYSTEM
It contains three stages: the text-based search stage, the content-based search
stage and the annotation learning stage.
1.2.6.2 TEXT-BASED SEARCH
Jeon et al. recommend using high quality training data to learn prediction
models as it affects greatly the annotation performance. Hence in our
approach, we collected about 2.4 million high-quality Web images
associated with meaningful descriptions from online photo forums. Thesedescriptions capture the corresponding images contents to certain degrees.
1.2.6.3 CONTENT-BASED SEARCH
Because visual features are generally of high dimensional, similarity-
oriented search based on visual features is always a bottleneck for large-
scale image database retrieval on search efficiency. To overcome this
problem, we adopt a hash encoding algorithm to speed up this procedure.
23
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
24/83
CHAPTER 2
SYSTEM ANALYSIS
2.1 EXISTING SYSTEM
The traditional annotation models focus exclusively on the semantic
aspect, while the sentimental aspect is totally neglected. The most existing
vlog search methods employ the traditional text-based retrieval techniques
which mainly rely on the textual content of the vlogs.
2.2 PROPOSED SYSTEM
In proposed vlog management model when a user uploads a vlog to
the database, semantic annotation will run automatically using vlog text and
relevant external resources, and sentiment evaluation is obtained from vlog
comments. After that, the vlog will be stored in the database with the
corresponding annotation and evaluation. When a user submits a query to
the search engine, the vlog search module will access the vlog database to
obtain relevant vlogs by saliency-based matching; then, using user specified
ranking strategy and clustering, the results will be returned to the user in a
well-organized manner.
24
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
25/83
CHAPTER 3
REQUIREMENT SPECIFICATIONS
3.1 HARDWARE REQUIREMENTS
Hard Disk : 10GB and above
RAM : 256MB and above
Processor : Pentium IV
3.2 SOFTWARE REQUIREMENTS
Windows Operating System
JDK 1.6
Web Browser Internet Explorer
Glassfish Application server
Apache Tomcat Server
Oracle 10G
JMF
3.3 SOFTWARE DESCRIPTION
3.3.1 JAVA
It is a Platform Independent. Java is an object-oriented programming
language developed initially by James Gosling and colleagues at Sun
Microsystems. The language, initially called Oak (named after the oak trees
outside Gosling's office), was intended to replace C++, although the feature
set better resembles that of Objective C.
25
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
26/83
3.3.1.1 INTRODUCTION TO JAVA
Java has been around since 1991, developed by a small team of SunMicrosystems developers in a project originally called the Green project.
The intent of the project was to develop a platform-independent software
technology that would be used in the consumer electronics industry. The
language that the team created was originally called Oak.
The first implementation of Oak was in a PDA-type device called Star Seven
(*7) that consisted of the Oak language, an operating system calledGreenOS, a user interface, and hardware. The name *7 was derived from the
telephone sequence that was used in the team's office and that was dialed in
order to answer any ringing telephone from any other phone in the office.
Around the time the First Person project was floundering in consumer
electronics, a new craze was gaining momentum in America; the craze was
called "Web surfing." The World Wide Web, a name applied to the Internet'smillions of linked HTML documents was suddenly becoming popular for
use by the masses. The reason for this was the introduction of a graphical
Web browser called Mosaic, developed by ncSA. The browser simplified
Web browsing by combining text and graphics into a single interface to
eliminate the need for users to learn many confusing UNIX and DOS
commands. Navigating around the Web was much easier using Mosaic.
It has only been since 1994 that Oak technology has been applied to the
Web. In 1994, two Sun developers created the first version of Hot Java, and
then called Web Runner, which is a graphical browser for the Web that
exists today. The browser was coded entirely in the Oak language, by this
26
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
27/83
time called Java. Soon after, the Java compiler was rewritten in the Java
language from its original C code, thus proving that Java could be used
effectively as an application language. Sun introduced Java in May 1995 at
the Sun World 95 convention.
Web surfing has become an enormously popular practice among millions of
computer users. Until Java, however, the content of information on the
Internet has been a bland series of HTML documents. Web users are hungry
for applications that are interactive, that users can execute no matter what
hardware or software platform they are using, and that travel across
heterogeneous networks and do not spread viruses to their computers. Javacan create such applications.
3.3.1.2 WORKING OF JAVA
For those who are new to object-oriented programming, the concept of a
class will be new to you. Simplistically, a class is the definition for a
segment of code that can contain both data and functions.
When the interpreter executes a class, it looks for a particular method by the
name of main, which will sound familiar to C programmers. The main
method is passed as a parameter an array of strings (similar to the argv[] of
C), and is declared as a static method.
To output text from the program, we execute the println method of
System.out, which is javas output stream. UNIX users will appreciate thetheory behind such a stream, as it is actually standard output. For those who
are instead used to the Wintel platform, it will write the string passed to it to
the users program.
27
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
28/83
3.3.2. JAVA MEDIA FRAMEWORK
The Java Media Framework (JMF) is an application programming interface
(API) for incorporating time-based media into java application and applets.
This guide is intended for java programmers who want to incorporate time-
based into their application and for technology providers who are interested
in extending JMF and providing JMF plug-ins to support additional media
types and perform custom processing and rendering.
The JMF 1.0 API enables programmers to develop java program that
presented time-based media .The JMF2.0 API extends the framework to
provide support for capturing and storing media data, controlling the type of
processing that is performed during addition ,JMF2.0 defines a plug-in API
that enables advanced developers and technology providers to more easily
customize and extend JMF functionality.
JMF provides a unified architecture and managing protocol for managing
the acquisition, processing and delivery of time-based media data. JMF is
designed to support most std media content types, such as AIFF, AV, AVI,
GSM, MIDI , MPEG , quickTime, RMFand , WAV.
By exploiting the advantage of the java platform, JMF delivers the promise
of write once,Run Anywhere, to developers who want to use media such
as audio and video in their java programs. JMF provides a common cross
platform java API for accessing underlying media frame works. JMF
implementation can leverage the capabilities of the underlying OS, while
developers can easily create portable java programs that features time-based
media by writing to the JMF API.
28
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
29/83
With JMF , creating applets and applications that present, capture,
manipulate and store time-based media .The frame work enables advanced
developers and technology provides to perform custom processing of raw
media data and seamless extend JMF to support additional content types andformats, optimize handling of supported formats and create new presentation
mechanism.
HIGH-LEVEL ARCHITECTURE
Devices such as tape decks and VCRs provide a familiar model for
recording, processing and presenting time based media. When playing a
movie using a VCR, it provides the media stream to the VCR by inserting a
video tape. The VCR reads and interprets the data on tape and sends
appropriate signals to television and speakers.
JMF uses this same basic model .A data source encapsulates the media
stream much like a video tape and a player provides processing and control
mechanism similar to VCR. Playing and capturing audio and video
microphones, cameras, speakers and monitors.
FIG: 3.1 JMF BASIC MODEL
29
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
30/83
Data source and players are integral parts of JMFs high-level API for
managing the capture, presentation and processing of time-based media.
JMF also provides a lower-level API that supports the seamless integration
of custom processing components and extensions. This layering providesjava developers with an easy-to-use API for incorporating time-based media
into java programs while maintaining the flexibility and extensibility
required to support advanced media application future media technology.
To present time-based media such as audio or video with JMF, player can be
used. Playback can be controlled programmatically, or it can be able to
display a control-panel comp that enables the user to control play-back
interactively. If several media streams want to be play, a separate player is to
be used for each one, to play them in sync, Player object can be used to
control the operation of others.
PLAYER
A player process as an input stream of media data renders it at a precisetime. A Data Source is used to deliver the input media stream to the player.
The rendering destination depends on the type of media being presents.
FIG: 3.2 JMF PLAYER
30
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
31/83
A player does not provide any control over the processing that it performs
that it performs or how it renders the media data.
PROCESSORS
Processors can also be used to present media data. A processor is just a
specialized type of player that provides control over what processing is
performed on the input media stream. A processor supports all of the same
presentation controls as a player.
In addition to rending media data to presentation devices, a Processor can
output media data through a data source so that it can be presents by anotherplayer or processor, further manipulated by another processor, or delivered
to some other destination such a file.
FIG: 3.3 JMF PROCESSORS
EXTENSIBLITY:
JMF can be extended by Implementing customs plug-ins, media handlers
and data sources. By implementing of the JMF plug-in interfaces, can be
accessed directly and manipulated the media data associated with the
processor:
Implementing the Demultiplexer interface enables to control how
individual tracks are extracted from a multiplexed media stream.
31
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
32/83
Implementing the code interface enables to perform the processing
required to decode compressed media data, convert media data
from one format to another and encode raw media data into a
compressed for materials.
3.3.3 APACHE TOMCAT SERVER
Apache Tomcat version 6.0 implements the Servlet 2.5 and Java Server
Pages 2.1 specifications from the Java Community Process, and includes
many additional features that make it a useful platform for developing and
deploying web applications and web services.
3.3.3.1 TOMCAT ARCHITECTURE
Tomcat is a container that is made up of pluggable components that fit
together in a nested manner. Tomcat is configurable you can set such
settings to use specialized filters, change port numbers and IP address
bindings, security settings, etc. You should always change the default setting
when using in a production environment especially the security aspects.
3.3.3.2 TOMCAT DIRECTORY OVERVIEW
Directory Files Description
bin bootstrap.jar
commons-
daemon.jar
tomcatuli.jar
startup.bat
This directory hold some of the JAR files
that are required when starting Tomcat, it
also holds the startup files themselves, the
startup.bat used to start the Tomcat as a
daemon process, the catalina.sh can be used
32
http://www.jcp.org/http://www.jcp.org/8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
33/83
catalina.shon a commandline and to add additional
parameters to change Tomcat when starting.
conf
catalina.policy
contains security policy statements that are
implemented by the Java SecurityManager.
It replaces the java.policy file that comes
with the JVM, it prevents rogue code of JSPs
from executing damaging code that can
affect the container. It is only used once
when Tomcat is launched thus you need to
restart Tomcat if you change this file
catalina.properties
contains a list of Java packages that cannot
be overridden by executable Java code in
servlets or JSPs which could be a security
risk.
context.xml
this file is used by all Web applications, it
explains where the web.xml should be
accessed
logging.properties
this file details the logging within Tomcat,
two default configuration are setup a
ConsoleHandler and a FileHandler, you can
change the logging level using this file.
server.xml
this is the main configuration file in Tomcat,
it is used by the "digester" to build the
container on startup
tomcat-users.xml Used for security to allow access to the
33
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
34/83
Administration applications section, it is
used with the default UserDatabase Realm as
referenced in server.xml.
web.xml
The default web.xml file that is used by all
Web applications, it sets up the JSPServlet to
allow your applications to handle JSPs and a
default servlet to handle static resources and
HTML files. It also sets up default session
times outs, welcome files and MIME types.
lib number of JAR files
all the JAR files that the container uses are
located here, this includes Tomcat JAR's and
the servlet and JSP application programming
interfaces (API's). Place your own JAR files
here if they will be used across all your Web
applications.
logs number of log files
contains a number of logs files, these are
produced by JULI logging which will be
discussed in a later topic. The logs are
rotated each day, so you may need to clear
them down from time to time.
temp ? used for scratch files and temporary use
webapps Web app files this is were the Web application files reside,
including your own Web applications. This
is were you place your Web Application
aRchive (WAR) file, Tomcat will then
34
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
35/83
deploy the file. We will get into deploying
Web applications in another topic.
There are several default Web applicationthat come with Tomcat:
ROOT - The welcome screen that you
saw when you first installed Tomcat.
This is a special directory called "/",
this gets removed when you move into
production. From this web you canaccess all the below Web applications
docs - contains the Tomcat
documentation
examples - contains some JSP and
servlet examples
host-manager - allows you to manage
the hosts that run in your application,
use the /host-manager/html URL to
access
manager - allows you to manage your
applications in Tomcat, you can start,
stop, reload, deploy and undeployyour applications. Use /manager/html/
URL to access
work used for temporary working files, it is used
heavy during JSP compilation where the
35
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
36/83
JSPs are converted to a Java servlet and
accessed through this directory.
3.3.3.3 TOMCAT ARCHITECTURE OVERVIEW
Tomcat 6 consists of a nested hierarchy of components, containers are
components that can contain a collection of other components.
FIG 3.4 ARCHITECTURE OF TOMCAT SERVER
36
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
37/83
Server
The server is Tomcat, its an instance of the Web application
server, it owns a port that is used to shutdown the server (port
8005). You can setup multiple servers on one node providing
they use different ports. The server is an implementation of the
Serverinterface, it implements the StandardServerobject.
Service
Aservice groups a container (usually an engine) with a set of
Connectors. The service is responsible for accepting requests,
routing them to the specified Web application and specific
resources and then returning the result of the processing of the
request, they are the middle man between the clients web
browser and the container.
Connectors
Connectors connect the applications to clients. They receive the
incoming requests HTTP (port 8080) or AJP (port 8009) by
default from the clients.
The default connector is Coyote which implements HTTP 1.1.
Engine
The engine is the top-level container, it cannot be contained by
another container, thus this is the parent container for all the
containers beneath it. The engine is a request-processing
component that represents the Catalina Server Engine.
It examines the HTTP headers to determine the virtual host or
context to which requests should be passed. An engine may
contain Hosts representing a group of Web applications and
Contexts representing a single Web application i.e. a virtual host
Realm
The realm for an engine manages user authentication and
authorization. Resources uses roles to allow access, the realm
enforces the security polices. A realm applies across the whole
engine, however this can be overridden by using a realm at the
Host level or the Context level, it a object that can be superceded
by its children objects.
Valves are used to intercept a request and preprocess it. They are
similar to filter mechanism of the servlet specifications but are
37
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
38/83
3.3.3.4 CONNECTOR ARCHITECTURE
All connectors work on the same principle, they have an Apache module
end(mod_jk or mod_proxy) that loads just like any other Apache module.
On the Tomcat end, each Web application instance has a connector module
component written in Java. In Tomcat 6 this is with the
org.apache.catalina.Connectorclass. The constructor takes one of two
connector types, HTTP/1.1 or AJP/1.3. You call the constructor indirectly
via theserver.xmlfile using the connectorandprotocoltags. Depending on
what setup you have, different classes will be used.
Apache
Portable
Runtime
(APR) is
supported
HTTP/1.1:
org.apache.coyote.http11.Http11AprProcotol
AJP/1.3: org.apache.coyote.ajp.AjpAprProtocol
APR is not
supported
HTTP/1.1: org.apache.coyote.http11.Http11Procotol
AJP/1.3: org.apache.jk.server.JkCoyoteHandler
The Web server handles all the static content, but when it comes across
content intended for a servlet container, it passes it to the module in question
(mod_jk, mod_proxy), the web server knows what content to pass to the
Connector module because the directives in the Web servers configuration
specify this.
38
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
39/83
FIG 3.5 INTERACTION BETWEEN TOMCAT SERVER AND WEB
SERVER
The Apache JServ Protocol (AJP) uses a binary format for transmitting data
between the Web server and Tomcat, a network socket is used for all
communication. The AJP packet consist of a packet header and a payload,below is the structure of the packet.
39
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
40/83
As you can see, the binary packet starts with the sequence 0X1234, this is
followed by the packet size (2 bytes) and then the actual payload. On the
return path the packets are prefixed by AB (the ASCII codes for A and B),
the size of the packet and then the payload.
The HTTP protocol is exactly as the name implies it uses the HTTP protocol
to exchange messages. You can use HTTPS but you require a SSL certificate
and make a few changes to Tomcat's configuration.
3.3.3.5 LIFECYCLE
Tomcat starts and stops the components in the order that were started, thus
when starting the parent gets started first then the children get started,
stopping is the reserve order. This is done through theLifecycle interface:
LifecycleEventandLifecycleListener.
TheLifecycle interface has two key methods start() and stop(), all major
components usually contain aLifecycleSupportobject that manages all of
theLifecycleListenerobjects for that component, it is this object that
40
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
41/83
propagates and fires general events. The top-level component calls all of its
child's start() methods, the reverse is true when stopping. This method
allows to to stop/start Host components without affecting any other Hosts.
TheLifecycleListenerinterface can be added at any level in the Tomcat
container that can execute specific code when a particular event is fired. By
default there are three listeners configured at the server level, they are
configured in the server.xml or context.xml file at the specific level.
Configuration
The most important file in Tomcat is the server.xml file, when Tomcat starts
it uses a version of the Apache Commons Digester to read the file, the
digester is a utility that reads XML file and creates Java objects from a set of
rules. With what you have learned above you can see that the rules in the file
follows Tomcat architecture exactly.
3.3.3.6 WORKING WITH TOMCAT SERVER
Apache Tomcat is a famous Servlet container developed at Apache
Software Foundation. This software is released under under the Apache
Software License. Anyone can use it for the development as well as
deployment of the applications. Tomcat is the official reference of
implementation of java Servlets and java Server Pages. Tomcat is very easy
to install and configure. Anyone can learn it very fast and start using the
Tomcat server for the development and deployment of the web applications.
These days many web hosting companies are providing Tomcat support on
their server. So, if you develop the application in Java technology you can
41
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
42/83
get any host and then deploy it on the internet. Earlier it was a difficult task
to get a good host for hosting.
3.3.3.7 DEPLOYING SERVLETS ON TOMCAT SERVER
To deploy servlets on Tomcat Server, following steps are to be taken for
example given below.
1. Create web application
To develop an application using servlets or jsp, a directory structure is to
be maintained for the example given below.
Step1: Create a web application folder (servlet-examples) under tomcat
webapps directory. The path will be C:\apache tomcat\webapps\servlets-
examples.
Step2: Create a WEB-INF folder which should be created under servlets-
examples.
Step3: Create web.xml file and classes folder under the WEB_INF folder.
2. Compile the servlet Program- Create a servlet program and compile it
on the command Prompt .The procedure is not different from any java
program. The set of classes required for writing servlets is available in
servlet-api.jar which is put into CLASSPATH.
3. Copy the Servlet class(Hello) into classes folder, which is under WEB-
INF folder.
4. Edit web.xml to include servlet's name and url pattern.
42
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
43/83
Hello
Hello
Hello
/Hello
5. Run Tomcat Server and execute your servlet- To run the server Go
to C:\apache tomcat\bin\startup.bat and double click on it , the server will
start up. After assuring that the server is running successfully, you can run
your servlet. To execute your servlet, open your web browser and type the
url which you have mentioned in your web.xml. The url will be like this:
http://localhost:8080/servlets-examples/Hello
3.3.4. GLASSFISH
3.3.4.1 ABOUT GLASSFISH
The GlassFish open-source application server is based on the Java Platform,
Enterprise Edition (Java EE) reference implementation and is built for
mission-critical enterprise deployments.
43
http://localhost:8080/servlets-examples/Hellohttps://glassfish.dev.java.net/http://localhost:8080/servlets-examples/Hellohttps://glassfish.dev.java.net/8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
44/83
Sun GlassFish Enterprise Server enables customers to leverage the benefits
of open source with a subscription that provides support, training credits,
limited indemnification and more.
3.3.4.2 BENEFITS OF GLASSFISH
The Sun GlassFish Enterprise Server provides the foundation to develop
and deploy Java EE artifacts, including Web services. It provides value-
added services for management, monitoring, diagnostics, clustering,
transaction management, and high availability of mission-critical
services.
44
http://www.sun.com/service/applicationserversubscriptions/http://www.sun.com/service/applicationserversubscriptions/8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
45/83
CHAPTER 4
SYSTEM DESIGN
4.1 ARCHITECTURE
FIG 4.1 ARCHITECTURE DIAGRAM FOR THE SYSTEM
when a user uploads a vlog to the database, semantic annotation will run
automatically using vlog text and relevant external resources, and sentiment
evaluation is obtained from vlog comments. After that, the vlog will bestored in the database with the corresponding annotation and evaluation.
When a user submits a query to the search engine, the vlog search module
will access the vlog database to obtain relevant vlogs by saliency-based
matching; then, using user specified ranking strategy and clustering, the
results will be returned to the user in a well-organized manner.
45
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
46/83
4.2 SEQUENCE DIAGRAM
FIG 4.2 SEQUENCE DIAGRAM FOR THE SYSTEM
A sequence diagram shows an interaction between the system and its
environment, arranged in a time sequence. It shows the objects participating
in the interactions by their lifelines and the messages they exchange,
arranged in a time sequence as shown in figure. The sequence diagram is
very simple and has immediate visual appeal. It is an alternative way tounderstand the overall flow of the control.
46
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
47/83
4.3 USE CASE DIAGRAM
FIG 4.3 USECASE DIAGRAM FOR THE SYSTEM
The use case diagram shows the relationship between the actors and the use
cases within the system. The clients are the actors who uploads and searchfor the videos in the server. And the results are received by user from the
database.
47
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
48/83
4.4 ACTIVITY DIAGRAM
FIG 4.4 ACTIVITY DIAGRAM FOR THE SYSTEM
An activity diagram is used to provide a view of flows and what is going on
inside an use case or among several classes. Activity diagram is used to
represent a classs method implementation as shown in figure.
48
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
49/83
CHAPTER 5
MODULES
5.1 MODULES
Semantic annotation
Content Analysis
Sentiment evaluation
Saliency-based matching and Ranking
5.2 MODULE EXPLANATION:
Semantic annotation
Annotation is the process of extracting informative keywords from text of
the vlog.This is necessary because the words used in the vlog texts arearbitrary and non-standard.When a user uploads a video automatic semantic
vlog annotation is run. The textual content in a vlog mainly comprises of
title, description, and comments, among which the title and description are
closely related to the semantics of the vlog video. So we use the title and
description for annotation process. The title indicates the main topic of the
whole vlog. It is of the greatest importance for understanding the semantics
of the vlog. Therefore, we first extract annotation words from the title and
then we extract annotation words from the body of the textual content, i.e.,
the vlogs description. It is then stored in the database.
49
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
50/83
Content Analysis
In the content analysis the video is split into a number frames and stored in
the database in blob form. Each frame consists of objects. Different object
from the each frame is detected and stored in the database. Thus no twoframe will have same object and this makes the searching process efficient.
The process of frame splitting is done with the help of Framesplitter class.
Sentiment evaluation
The main purpose of the sentiment evaluation process is to extract
annotation words from the users comments. The users can decide whether a
vlog is worth viewing based on the existing comments. Traditional
annotation models focus solely on the semantic aspect while the sentiment
aspect is totally neglected. For opinioned texts such as comments in vlog
,extra informations can be obtained through sentiment analysis. The users
comments will be predominantly text based. Sentiment evaluation is used to
obtain overall evaluation of the vlog.
Saliency-based matching and Ranking
We propose a novel saliency-based similarity matching approach for vlog
search. This is done with the help of canny edge detection algorithm.
Saliencies are nothing but the edges of an image.Even if two images are
same the saliencies will not be the same.Thus when an image is given as
input query ,this algorithm aids is resulting the exact video. After the
relevant vlogs are obtained using saliency based matching different rankingstrategies are adopted. Finally the ranked vlogs are clustered according to
the category information to further facilitate users browsing.
50
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
51/83
5.3 DATA FLOW DIAGRAMS:
Level 0:
FIG 5.3.1 LEVEL 0 DFD FOR THE SYSTEM
This is the Level 0 DFD for the system.When the user visits the login page
,the server checks with the database. If he is a registered user then he can
access the home page if not he has to register to access the videos.
51
Data
Base
Home PageRegistration
Server
User
Login
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
52/83
Level 1:
FIG 5.3.2 LEVEL 1 DFD FOR THE SYSTEM
This is the Level 1 DFD for the system. When a user uploads a video
automatic semantic vlog annotation is run. Content analysis is performed
where the video is split into a number frames and stored in the database in
blob form.
52
User Upload Server
Data base
DataBase
Annotation
process
Content
Analysis
Stored in
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
53/83
Level 2:
FIG 5.3.3 LEVEL 2 DFD FOR THE SYSTEM
This is the Level 2 DFD for the system. When the user sends a request for
searching the video,The server checks in the database. Finally the results are
given to the user in a ranking manner.
53
User Search
Server
Data
BaseResults
Sends Request to
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
54/83
CHAPTER 6
TABLES
6.1 USERLOGIN
FIELD NAME DATA TYPE
Username Varchar 2
Password Varchar2
6.2 VIDEOUPLOAD
FIELD NAME DATA TYPE
Videoname Varchar2
Title Varchar2
Description Varchar2Comments Varchar2
Count Number
Path Varchar2
Video blob
54
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
55/83
CHAPTER 7
TESTING
7.1 SOURCE CODE TESTING
This examines the logic of the system. If we are getting the output that is
required by the user, then we can say that the logic is perfect.
7.2 MODULE LEVEL TESTING
In this the error will be found in each individual module, it encourages the
programmer to find and rectify the error without affecting other modules.7.3 UNIT TESTING
Unit testing is conducted to verify the functional performance of each
modular component of the software. Unit testing focuses on the smallest unit
of the software design (i.e.), the module.
7.4 INTEGRATION TESTING
Integration testing is a systematic technique for constructing the program
structure while at the same time conducting tests to uncover errors
associated with. Individual modules, which are highly prone to interface
errors, should not be assumed to work instantly when we put them together.
The problem of course, is putting them together- interfacing. There may
be the chances of data lost across on anothers sub functions, whencombined may not produce the desired major function; individually
acceptable impression may be magnified to unacceptable levels; global data
structures can present problems.
55
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
56/83
7.5 FUNCTIONAL TEST
Functional test cases involved exercising the code with nominal input values
for which the expected results are known, as well as boundary values and
special values, such as logically related inputs, files of identical elements,
and empty files.
Three types of tests in Functional test:
Performance Test
Stress Test
Structure Test
7.5.1 PERFORMANCE TEST
It determines the amount of execution time spent in various parts of the unit,
program throughput, and response time and device utilization by the
program unit.
7.5.2 STRESS TEST
Stress Test is those test designed to intentionally break the unit. A Great
deal can be learned about the strength and limitations of a program by
examining the manner in which a programmer in which a program unit
breaks.
7.5.3 STRUCTURE TEST
Structure Tests are concerned with exercising the internal logic of a program
and traversing particular execution paths. The way in which White-Box test
strategy was employed to ensure that the test cases could Guarantee that all
56
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
57/83
independent paths within a module have been have been exercised at least
once.
Exercise all logical decisions on their true or false sides.
Execute all loops at their boundaries and within their
operational bounds.
Exercise internal data structures to assure their validity.
Checking attributes for their correctness.
Handling end of file condition, I/O errors, buffer problems and
textual errors in output information.
7.6 WHITE BOX TESTING
This testing is also called as Glass box testing. In this testing, by knowing
the specific functions that a product has been design to perform test can be
conducted that demonstrate each function is fully operational at the same
time searching for errors in each function. It is a test case design method that
uses the control structure of the procedural design to derive test cases. Basis
path testing is a white box testing.
7.7 BLACK BOX TESTING
In this testing by knowing the internal operation of a product, test can be
conducted to ensure that all gears mesh, that is the internal operation
performs according to specification and all internal components have been
adequately exercised. It fundamentally focuses on the functionalrequirements of the software.
57
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
58/83
7.8 USER ACCEPTANCE TESTING
User acceptance of the system is key factor for the success of any system.
The system under consideration is tested for user acceptance by constantly
keeping in touch with prospective system and user at the time of developing
and making changes whenever required. This is done in regarding to the
following points.
Input screen design.
Output screen design.
58
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
59/83
CHAPTER 8
C
ONCLUSIONAND FUTURE ENHANCEMENT
8.1 CONCLUSION:
In a vlogs annotation, we extract informative keywords not only from the
textual content of the target vlog itself but also from external recourses
which are semantically and visually relevant to it; besides semantic
annotation, we obtain sentiment evaluation from comments as guidance for
vlog browsing In the user-oriented vlog search, we adopt saliency-basedmatching to make the search results more agreeable to users; we use
different ranking strategies are adopted according to the users specific
interest.
8.2 FUTURE ENHANCEMENTS
The proposed system results in a video given a frame of the video as input.It
can further be enhanced by giving any object in the frame as input resulting
in the exact video.
59
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
60/83
APPENDIX 1
SCREEN SHOTS
This is the home page for video blog where the user must signup to access
the videos.
60
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
61/83
This is the members login page.
61
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
62/83
This is the video upload page where the user uploads the video by giving
title, description and comment. While uploading the video frames will be
splitted and stored in database after automatic semantic annotation.
62
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
63/83
This is the page for searching the video. The user browses the image and
search for the video.
63
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
64/83
This the page for changing the password. If the user wishes to change his
password he can do so by providing a new password.
64
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
65/83
APPENDIX 2
SAMPLE CODING
MONTY TAG:
import java.util.*;import montytagger.JMontyTagger;public class montytag{
JMontyTagger mon=new JMontyTagger();public montytag(){}
public static void main(String[] args){
//new montytag("this video is super");String str="sachin&dravid";//new montytag(str);
}public Vector method(String strr){
Vector vv=new Vector();try{
String sr=mon.Tag(strr);Vector v=new Vector();StringTokenizer st=new StringTokenizer(sr);while(st.hasMoreElements()){
v.add(st.nextElement());}System.out.println(v);
for(int i=0;i
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
66/83
String t2=stt.nextToken();vv.add(t1);
}}System.out.println("Final vecu"+vv);
}catch (Exception e){
e.printStackTrace();}return vv;
}
}
VIDEO UPLOAD:import java.io.*;import java.sql.*;import javax.servlet.ServletException;import javax.servlet.http.HttpServlet;import javax.servlet.http.HttpServletRequest;import javax.servlet.http.HttpServletResponse;import java.util.*;
/*** servlet implementation class upload
*/public class upload extends HttpServlet {private static final long serialVersionUID = 1L;public Connection conn;public montytag mm=new montytag();Video_FrameSplitter vfs;
public void doPost(HttpServletRequest request,HttpServletResponse
response) throws ServletException, IOException{
try {
vfs=new Video_FrameSplitter();Properties p=new Properties();
//System.out.println(request.getRealPath("/"));FileInputStream fis=new FileInputStream("C:/Program
Files/Apache Software Foundation/Tomcat 6.0/webapps/VideoBlogfull/src/Database.properties");
p.load(fis);
66
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
67/83
String system=p.getProperty("system");String username=p.getProperty("username");String password=p.getProperty("password");String video=request.getParameter("Browse");System.out.println("video path "+video);
vfs.Video_SplitterMethod("d:/f.avi");/*String title=request.getParameter("Title");Vector vt = mm.method(title);String stit=vt.get(0).toString();String desc=request.getParameter("Description");Vector vd = mm.method(title);String sdes=vd.get(0).toString();String comm=request.getParameter("Comment");Vector vc = mm.method(title);String scomm=vc.get(0).toString();DriverManager.registerDriver( new
oracle.jdbc.driver.OracleDriver());conn =DriverManager.getConnection("jdbc:oracle:thin:@"+system+"",username,password);
//System.out.println("in database");*/} catch (Exception e) {
e.printStackTrace();}
}}
DBSTORE
import java.io.ByteArrayOutputStream;import java.io.File;import java.io.FileInputStream;import java.io.InputStream;import java.io.ObjectOutputStream;import java.sql.Connection;import java.sql.DriverManager;import java.sql.PreparedStatement;
import java.sql.Statement;import java.util.Vector;
public class DBstore {
67
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
68/83
public DBstore( String video,Vector vec,String tit,String des,Stringcomm)
{try{
DriverManager.registerDriver( neworacle.jdbc.driver.OracleDriver() );Connection conn =
DriverManager.getConnection( "jdbc:oracle:thin:@ramarathinam","system","redhat");
System.out.println(video+vec+tit+des+comm);System.exit(0);Statement stmt = conn .createStatement();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ObjectOutputStream objOstream = newObjectOutputStream(baos);objOstream.writeObject(vec);objOstream.flush();objOstream.close();
byte[] bArray = baos.toByteArray();
System.out.println("*** bArray = " + bArray);
PreparedStatement objStatement = conn.prepareStatement("insert into samp(video,frameobj) values (?,?)");
File newfile=new File("d:/f.avi");String filename="d:/f.avi";String finame=newfile.getName();
System.out.println(" Video File NAme & Path ::::::::::: "+filename);
InputStream fis=new FileInputStream(filename);System.out.println(" Video File Length :
"+newfile.length());
System.out.println(" File InputStream "+fis.available());
objStatement.setBinaryStream(1,fis,(int)newfile.length());objStatement.setBytes(2, bArray);
68
8/7/2019 A Novel Vlog Management Model of Automatic Vlog Annotation and Search With Context and Content Analysis
69/83
objStatement.execute();System.out.println("stored");}
catch(Exception e){e.printStackTrace();}}}
SEARCH IMAGE:
import java.io.*;import javax.servlet.*;import javax.servlet.http.HttpServlet;import javax.servlet.http.HttpServletRequest;import javax.servlet.http.HttpServletResponse;import java.sql.*;import java.util.*;import java.awt.image.BufferedImage;import javax.imageio.ImageIO;import javax.servlet.http.HttpSession;
public class Searchimg extends HttpServlet{public Connection conn;HttpSession hs;TreeMap tm1 ;TreeMap tm2 ;TreeMap tm3 ;TreeMap tm4 ;TreeMap tm5 ;TreeMap tm6;Vector v2 ;
Vector v3 ;Vector fp ;Vector trueval;Vector resv;
ConPixel cp=new ConPixel(); public void doPost(HttpServletRequest request,HttpServletResponse
response)throws ServletException, IOException{
try{
tm1=new TreeMap();tm2=new TreeMa