SeerSuite: Developing a scalable and reliable application framework forbuilding digital libraries by crawling the web

Pradeep B. TeregowdaPennsylvania State University

Isaac G. CouncillGoogle

Juan Pablo Fernandez R.Pennsylvania State University

Madian KhabsaPennsylvania State University

Shuyi ZhengPennsylvania State University

C. Lee GilesPennsylvania State University

Abstract

SeerSuite is a framework for scientific and academic dig-ital libraries and search engines built by crawling scien-tific and academic documents from the web with a fo-cus on providing reliable, robust services. In additionto full text indexing, SeerSuite supports autonomous ci-tation indexing and automatically links references in re-search articles to facilitate navigation, analysis and eval-uation. SeerSuite enables access to extensive document,citation, and author metadata by automatically extract-ing, storing and indexing metadata. SeerSuite also sup-ports MyCiteSeer, a personal portal that allows users tomonitor documents, store user queries, build documentportfolios, and interact with the document metadata. Wedescribe the design of SeerSuite and the deployment andusage of CiteSeerx as an instance of SeerSuite.

1 Introduction

Efficient and reliable access to the vast scientific andscholarly publications on the web requires advanced cita-tion index retrieval systems [21] such as CiteSeer [22, 4],CiteSeerx [5], Google Scholar [9], ACM Portal [1], etc.The SeerSuite application framework provides an uniqueadvanced and automatic citation index system that is us-able and comprehensive, and provides efficient access toscientific publications. To realize these goals our designfocuses on reliable, scalable and robust services.

A previous implementation, CiteSeer (maintained asof this date), was designed to support such services.However, CiteSeer was a research prototype and, as such,suffered serious limitations. SeerSuite was designed toprovide a framework that would replace CiteSeer, to pro-vide most of its functionality, but designed to be extensi-ble. SeerSuite improves on several aspects of the originalCiteSeer with features such as reliability, robustness andscalability. It does this by adopting a multi-tier architec-ture with a service orientation and a loose coupling of

modules.CiteSeerx, an instance of SeerSuite is one of the top

ranked resources on the web and indexes nearly one andhalf million documents. The collection spans computerand information science (CIS) and related areas such asmathematics, physics and statistics. CiteSeerx acquiresits documents primarily by automatically crawling au-thors web sites for academic and research documents.CiteSeerx daily receives approximately two million hitsand has more than two hundred thousand documentsdownloaded from its cache. The MyCiteSeer personalportal has over ten thousand registered users.

While the SeerSuite application framework has mostof the functionality of CiteSeer, SeerSuite represents acomplete redesign of CiteSeer. SeerSuite takes advan-tage of and includes state of the art machine learningmethods and uses open source applications and modules.The structure and architecture of SeerSuite is designed toenhance the ease of maintenance and to reduce the costof operations. SeerSuite is designed to run on off theshelf hardware infrastructure.

With CiteSeerx, SeerSuite focuses primarily on CIS.However, there are requests for similar focused servicesin other fields such as chemistry [33] and archaeol-ogy [37]. SeerSuite can be adopted to providing ser-vices similar to those provided by CiteSeerx in these ar-eas. SeerSuite is a part of the project ChemXSeer [3],a digital library search engine and collaboration servicefor chemistry. Though designed as a framework forCiteSeerx-like digital libraries and search engines, themodular and extensible framework allows for applica-tions that use SeerSuite components as stand alone ser-vices or as a part of other systems.

2 Background and Related Work

Domain specific repositories and digital library systemshave been very popular over the last decades with severalexamples such as arXiv [23] for physics and RePEc [15]

for economics. The Greenstone digital library [10] wasdeveloped with a similar goal. These repository systems,unlike CiteSeerx, depend on manual submission of doc-ument metadata, a tedious and resource expensive pro-cess. As such CiteSeerx which crawls authors web pagefor documents is in many ways a unique system, closestin design to Google Scholar.

The popularity of services provided by the origi-nal CiteSeer and limitations in its design and archi-tecture were the motivation behind the design of Seer-Suite [18, 30]. The design of SeerSuite was an incre-mental process involving users and researchers. User in-put was received in the form of feedback and feature re-quests. Exchange of ideas between researchers and usersacross the world through collaborations, reviews and dis-cussions have made a significant contribution to the de-sign. In addition to ensuring reliable scalable services,portability of the overall system and components wasidentified as an essential feature that would encourageadoption of SeerSuite elsewhere. During the process ofdesigning the architecture of SeerSuite, other academicrepository content management system (CMS) architec-tures such as Fedora [28] and DSpace [8] were studied.The Blackboard architecture pattern [34] had a strong in-fluence on the design of the metadata extraction system.The main obstacles in adopting existing repository andCMS systems were the levels of customization and theeffort required to meet SeerSuite design requirements.More specifically, implementation of the citation graphstructure with a focus on automatic metadata extractionand workflow requirements for maintaining and updatingcitation graph structures made these approaches cumber-some to use.

In addition to repository, search engine and digitallibrary architectures, advances in metadata extractionmethods [24, 25, 19, 27, 32] and the availability of opensource systems have influenced SeerSuite design. We be-gin our discussion of SeerSuite by describing the archi-tecture.

3 Architecture

In the context of SeerSuite reliability refers to the abilityof the framework instances to provide around the clockservice with minimal downtime, scalability to the abilityto support increasing number of user requests and doc-uments in the collection, and robustness to the ability ofthe instance to continue providing services while some ofthe underlying services are unavailable or resource con-strained.

An outline of SeerSuite architecture is shown in fig-ure 1. By adopting a loosely coupled approach for mod-ules and subsystem, we ensure that instances can bescaled and can provide robust service. We describe the

overall architecture in the web application, data storage,metadata extraction and ingestion, crawling and mainte-nance sections.

Figure 1: SeerSuite Architecture

3.1 Web applicationThe web application makes use of the model view con-troller architecture implemented with the Spring frame-work. The application presentation uses a mix of javaserver pages and javascript to generate the user inter-face. Design of the user interface allows the look andfeel to be modified by switching Cascading Style Sheets(CSS). The use of JavaServer Pages Standard Tag Li-brary (JSTL) supports template based construction. Theweb pages allow further interaction through the use ofjavascript frameworks. While the development environ-ment is mostly Linux, components are developed withportability as a focus. Adoption of the spring frameworkallows development to concentrate on application design.The framework supports the application by handling in-teractions between the user and database in a transparentmanner.

Servlets use Data Access Objects (DAO) with the sup-port of the framework to interact with databases and therepository. The index, database and repository enablethe data objects crawled from the web to be stored andaccessed through the web, using efficient programmaticinterfaces. The web application is deployed through aweb application archive file.

2

3.2 Data storage

The databases store metadata and relationships in theform of tables providing transaction support, wheretransactions include adding, updating or deleting objects.The database design partitions the data storage acrossthree main databases. This allows for growth in anyone component to be handled by further partitioning thedatabase horizontally.

The main database contains objects and is focused ontransactions and version tracking of objects central toSeerSuite and digital libraries implementations. Objectsstored in the main database include document metadata,author, citation, keywords, tags, and hub (URL). The ta-bles in the main database are linked together by the doc-ument they appear in and are identified by the documentobject id.

One of the most unique aspects of SeerSuite is thecitation graph. The nodes of this graph correspond todocuments or citations and the edges to the relationshipsamong them. This graph is stored in its own database.The citation relationship is stored in a graph table inthe form of ‘cited by’ and ‘cites’ fields. In additionthe database stores a canonical representation of cita-tion or document metadata which is basically the mostfrequent and correct of the citations, as determined bya Bayesian network. These records serve as a group-ing point for metadata collected about a particular docu-ment or citation. This database thus provides support forautonomous citation indexing and related features. Thecitation relationships are established by generating andmatching keys for the document metadata records dur-ing ingestion or updates. The use of triggers allows datain the graph database to be updated when transactionssuch as insertions, deletions and corrections occur in themain database. In addition to triggers, application logicand maintenance functions maintain the link between thegraph and the main database. MyCiteSeer personaliza-tion portal stores user information, queries and documentportfolios in a separate database. The link between theuser and the main database is through references in thetags in the MyCiteSeer database and the main databaseversion tracking tables. A conventional RDBMS withsupport for triggers is used to host these databases.

The repository system provides SeerSuite with theability, to provide cached copies of the documentscrawled. In addition to the cached copy, the repositorystores xml files containing extracted document metadata.These files serve as backup copies of the metadata storedin the databases. This is similar to the Fedora ObjectXML document representation [28]. The repository isorganized into a directory tree. The top of the tree con-sists of a root folder containing sub directories mappedaccording to the segments in the document identifier. The

final level contains metadata, text and cached files. Thedocument identifier structure ensures that the there are anominal number of sub folders under any folder in thetree structure.

An index provides a fast efficient method of storingand accessing text and metadata through the use of aninverted file. SeerSuite uses the Apache Solr an Indexapplication [16] supported by Apache Tomcat to providefull text and metadata indexing operations. The meta-data items are obtained from the database and the fulltext from the repository. The interaction of the applica-tion in the form of controllers is through the REST (Rep-resentational state transfer) [20] interface. This allowsany indexing application supporting REST API’s to beadopted by SeerSuite. In addition, this enables introduc-tion of newer feature sets in the index or new versions ofSolr without disruptions.

3.3 Metadata Extraction and IngestionMetadata extraction methods are built using Perl andC++. To enable interaction with the extraction services,a service oriented architecture is utilized using a Busi-ness Process Execution Language (BPEL) or, for con-venience, scripts. Each component of the extraction sys-tem individually contributes to the final document, in thiscase an xml file, which is then ingested. The feedback toindividual systems is manual adjusting of parameters orreplacing components. The system can be rewired to in-clude or exclude any extraction modules or applications.

The user can either batch process the incoming dataor process each item individually. Addition of metadatainto the system is controlled by the ingestion system,which interacts with main database using DAOs. Theingestion system ensures that essential metadata is cor-rectly and uniquely labeled, with the help of an objectidentifier and checksum based de-duplication. In addi-tion, the ingestion system makes use of listeners to sharenotification data such as alerts that inform users and pro-grams about objects of interest.

The ingestion process can itself be distributed acrossmachines, taking advantage of the Document ObjectIdentifier (DOI), database and shared repository services.The DOI is issued by on of our web services with its owndatabase and tracks document identifiers and the time oftheir issue.

3.4 CrawlerThe suite also includes a Heritrix [11] based crawler forharvesting documents from the web. The interface be-tween the ingestion system and the crawler is based onthe Java Messaging Service over ActiveMQ [2]. Due tothe modularity of the design, other crawlers can be used.

3

The ingestion system sends URL submissions messagescontaining a job identifier and url through the ingestionchannel, and the crawler responds with ’new content’messages pointing to the acquired resource and meta-data to the ingestion system. The crawler can use otheroptional channels to provide status messages to listen-ers. The crawler uses the Heritrix job submission sys-tem which consumes messages in the submissions chan-nel and processes these submissions.

3.5 Maintenance

Maintenance services support and enable associatedfunctionality for the ingestion and presentation systems.Maintenance systems are responsible for updating the in-dex, identifying metadata by inference, generating cita-tion and document statistics, charts generation, and ex-ternal data linking in the system. For convenience, com-mon maintenance processes are available through a com-mand line interface.

An evidence based inference system [19] utilizes aBayesian inference framework to determine canonicalmetadata for documents in the collection. The systembuilds an ideal citation record for a document inferringfrom the information provided by citations and the doc-ument metadata.

Citation graphs which accompany the document vieware generated from the graph database, by examining thecitation relationships and the distribution across years. Inaddition to enabling access to extensive document meta-data, SeerSuite allows documents in the collection to belinked to copies or bibliographic records in other collec-tions. SeerSuite provides components to map and link toother services such as DBLP [7] and CiteULike [6].

The configuration of an SeerSuite instance or appli-cation is controlled through properties and context files,through which information about the file system for therepository, database, index and system parameters can bespecified. The web application and the maintenance andindexing functions use similar configuration files. In ad-dition the SeerSuite distribution provides configurationfiles or examples of configurations for applications usedsuch as the Solr index and Tomcat.

4 Workflow

The outline of the process of adding documents to Seer-Suite instances is shown in Figure 2. Documents areharvested from the web using focused crawlers (step1). These documents are first converted from PDF orPostScript format to text with application tools such asPDFBox and TET or GhostScript for PostScript docu-ments in the step labelled 2.

In step 3, to prevent processing of documents such asresumes and non-academic documents part of the har-vested collection, SeerSuite uses a regular expressionbased filter on the converted text file. The converted, fil-tered text is processed using state of the art automaticmetadata extraction systems. These include the SupportVector Machine based header parser [24], which extractsmetadata such as titles, publication information, authorsand their affiliation, and abstract from the document. TheParsCit [27] citation extraction system extracts citationand context information from the document.

The ingestion system identifies unique documents andrequests a document identifier for the document. If thedocument is found to be a checksum duplicate based oncontent, URL mappings are updated to include alternateURL(s) in step 4. In the same step, the repository is up-dated with a complete set of files including the documentin the original format. The converted text files, citations,crawler and document metadata are placed in the relevantdocument directory under the repository tree. In additionto the individual metadata files, a file copy of the com-plete document metadata is stored in the form of an xmlfile. With updates and corrections, the xml files are up-dated and stored with a version tag. In the main databasepapers, authors citation and url mapping tables are up-dated, triggering updates to the graph database. Updatesto the graph database ensure that the citation relationshipof the incoming metadata to the data already existing inthe collection is accurately maintained.

Documents in the database have a time stamp indicat-ing time of update, helping the maintenance scripts per-form incremental updates of the index. Incremental up-dates are crucial in reducing the time required for mainte-nance. With step 5, new or updated metadata are indexed.The maintenance script scans the database for updatedand new documents and creates an in-memory indexabledocument, including the document metadata fields andcitation information gathered across the main and cita-tion graph databases. This document is then indexed bythe main Solr index over the REST interface with an up-date command. The maintenance system optimizes theindex after each update, using an built in feature in Solr.

Statistics provide users with a perspective of the col-lection from a citation, document and author ranking us-ing aggregated citation information. Statistics are gen-erated from the graph database, in the form of text files,which are then presented to the user through the statis-tics servlet interface. Maintenance scripts are manuallyscheduled or run by the administrator.

5 An Instance of SeerSuite: CiteSeerx

CiteSeerx serves as a flagship deployment of SeerSuite.It utilizes Apache Tomcat as the supporting platform

4

Figure 2: SeerSuite Workflow: Links represent actions and documents data

with MySQL as the RDBMS. The deployment spansmultiple machines with varied configurations. Compo-nents are distributed based on functionality on these ma-chines. A pair of level 4 load balancing servers directtraffic to a pair of webservers. The load balancers use aconnection based metric to determine to which server aparticular request will be directed. To ensure availabil-ity, the load balancers are configured as a high availabil-ity asymmetric cluster that uses open source linux highavailability software.

The web servers host the application on the ApacheTomcat platform with the Tomcat instances as part ofan Apache Tomcat TCP cluster with session informationshared across the cluster. The application processes theserequests and processes them with the help of the index,database or the repository.

In case of an search query, the application translatesthe query to a suitable format and dispatches the queryto the Solr indices. The results are processed and pre-sented to the user. CiteSeerx uses indices for documentand citation objects, table objects and disambiguated au-thor objects. Requests for metadata are handled by theMySQL database system containing the main graph andthe user databases. The repository is responsible for stor-ing cached documents and the text and metadata files.Requests for cached files are handled by the applica-tion with support from the repository stored on a storageserver. The repository system is shared with the inges-tion system and web servers, using the Global File Sys-tem within the cluster.

The processing system is maintained separately fromthe web application and data storage infrastructure. Thedocument processing systems are responsible for con-verting and extracting the metadata from converted textfiles. This operation is distributed across machines by di-viding the incoming data into distinct sets, each set beingprocessed by individual machines across the cluster.

The deployment is supported by a staging and devel-opment system, where new features are introduced andreviewed before being introduced in the production sys-tem. Major components in the system are backed up ei-ther using component level replication services and or byfile level backups. In addition to backup on site, off sitebackups are utilized to ensure redundancy.

CiteSeerx depends on operating system based securityand application security provided by firewalls plus intru-sion detection systems and the underlying framework.Application logs and Tomcat error and access loggingprovide audit trails for bug fixes and troubleshooting.

Infrastructure adopted by CiteSeerx has led to severalissues. Frequent freezes occur due to deadlocks involv-ing the shared file system. Hardware failures have led toloss of data across the repository database. In such casesback ups have helped restore services. The ability to re-cover from these unfortunate losses showcase CiteSeerx

robustness.

5.1 Focused Crawling

For the initial crawling seeds, CiteSeerx assimilated thecomplete collection of documents and their URLs fromCiteSeer with some exceptions. These documents wereingested into the system by utilizing the already existinginformation in the CiteSeer databases.

Though equipped with Heritrix, the new CiteSeerx

uses a customized focused crawler, which runs incre-mentally. The crawler is run on a daily basis and canfetch several thousand new documents every day. Thesystem maintains a list of parent URLs where documentswere previously found. The parent URL’s include aca-demic homepages containing lists of online publications.

CiteSeerx has nearly two hundred thousand uniqueURLs containing links to publications. The crawl pro-cess begins with the crawl scheduler, which selects a

5

Figure 3: CiteSeerx Deployment

thousand parent URLs based on an estimated likelihoodof these pages having new documents. The selectedURLs are fed to a seed crawler. The seed crawler re-visits these thousand URLs and also follow links upto two hops from each scheduled URL. By parsingfetched pages, the seed crawler will retrieve all documentlinks eligible for processing. These represent documentswhich will be later downloaded. If a document link isfound on a new URL, this URL will be added to the par-ent URL list. This enables the parent URL list to expand.The crawling system also maintains a list of documentURLs and a list of document checksum hash values forall previously crawled documents. These two lists helpavoid downloading duplicate documents. When the seedcrawler outputs a list of discovered document URLs, itfirst compares them to the maintained document URL listand filters out duplicate URLs. Therefore, documentswith same URLs will not be downloaded again. Onlynew document URLs will be fed to the document crawler.

The document crawler then simply fetches all docu-ments in PDF or PostScript from the input list. Afterthe documents are crawled, their checksum values arecalculated based on their content and compared to themaintained checksum list by the ingestion system. Thus,content based duplicates are removed. Finally, only newdocuments are ingested into CiteSeerx. User submis-sions to CiteSeerx are directly submitted to the crawlerseed listing. A user interface allows tracking of the doc-uments obtained from a submitted URL. This allows theuser to determine, whether a document has been ingestedinto CiteSeerx.

6 User Interface

A user interacts with SeerSuite through a number of webpages. The most common of these include results ofsearches, document details, and citation graphs. We de-scribe a set of pages, whose coverage spans most of thefunctionality provided by SeerSuite. The user interfacesare built using jsp, backed by the controller servlets. Anavigation panel allows the user quick access to the mainpages. The access control to pages across the applicationis based on the login system provided as part of MyCite-Seer component.

6.1 Search Interface

Figure 4: Document Search Results

The search interface in SeerSuite allows users to makequeries across document, author, and table metadata ei-ther by using the default query interface or an advancedsearch interface. Search results are generated as a re-sult of a user query. The results can be sorted accordingto the relevance, citations, and recency of the documentingestion. Since SeerSuite indexes document metadataacross entities such as title, authors and their affiliation,year of publication, publication venue, keywords and ab-stract, the user can make queries which span one or moredocument or citation entities through the advanced queryinterface.

Each search result includes the title, author names,publication information and a short snippet of the textcontaining the query terms. A javascript interface en-ables users to view the abstract of a document from amouse over the down arrow associated with each searchresult. The process and content of the search results are

6

unique and can be ranked based on citations, relevanceand recency.

SeerSuite applications use a Solr instance configuredto index metadata fields defined by SeerSuite across doc-uments in the collection. SeerSuite uses a simple rank-ing algorithm based on the default ranking algorithmprovided by Solr for ranking results. These results areboosted based on the location of where the results werefound and citations, e.g. a result containing the queryterm in the title or author names have higher rank. Newranking methods are readily implemented in Solr.

Relevance based searches are used by default in Seer-Suite. The ability to rank documents based on the num-ber of citations is an optional default.

SeerSuite uses author normalization for the metadataextracted from the documents to provide a comprehen-sive set of variations of the author names. The normal-ization allows authors to be searched for through namevariants.

In addition to viewing results on the search resultspage, the user can subscribe to search feeds, which up-date the user when new results are available for a partic-ular query.

6.2 Document SummaryThe document view in figure 5, provides the user a viewof metadata information aggregated for each document.This view contains extensive metadata about a individualdocument including - title, author names, venue of pub-lication, year of publication and the citations containedin the document. The tabbed interface allows the userto browse citation relationships and the metadata versioninformation. Link to the source of the document alongwith the option to download a cached copy are provided.In case the document is found at multiple URLs, the pagelists all the alternate document URLs. The citation link-ing information provided along with the document in-cludes links to documents cited by this document. Thedocuments cited by the document are ranked by how wellthey are cited in the collection. A graph illustrating thecitations to the document across years which can be use-ful for identifying trends and impact is also displayed.

The document summary page provides severalMyCiteSeer interaction points. Add to Collection, Cor-rect Errors and Monitor changes links allow the user toinsert documents to his collection, correct metadata andmonitor the document.

Corrections to the metadata involve several changes tothe document metadata and citation relationships. Seer-Suite examines these relationships, updating the citationgraphs as necessary. It either creates or updates the ci-tation cluster established for the current document af-ter the corrections are submitted. A versioning system

enables the administrator to track changes to documentmetadata. A user can view the modifications to the doc-ument through the versions tab, which displays all ver-sions of the document metadata and the attribution foreach update or correction.

The document view page contains data spanning mul-tiple databases and the generation of this page is resourceintensive. In addition to providing metadata, the docu-

Figure 5: Document Summary

ment also allows the user to download the bibtex of thearticle or collect the bibtex of the article in a meta cartfor download later. The page also provides several book-marking links and the ability to copy document data pro-vided on the page using browser plugins.

6.3 Citation Relationships

The citation graph generated and stored as part of the in-gestion process and updated as part of the maintenanceprocess allows SeerSuite to provide users with tools forcitation analysis. Citation based relationships such as ac-tive bibliography and co-citations are available for eachdocument through the related documents tab in the doc-ument summary page. This relationship provides valu-able information to users, allowing users to track docu-ments of their interest. Such analysis is helpful in ex-ploring topics and literature surveys. Active Bibliogra-phy provides links between documents citing the sameset of documents and is one way grouping of documents.Another method is by identifying Co-citations which arelinks between documents which cite the same documentsto a particular document. Figure 6 shows the Active Bib-liography of a document in CiteSeerx.

7

The citation graph is dynamic, changing as a resultof corrections and other metadata updates. SeerSuiteindexes the citation relationships with the document.Therefore, in addition to the database, rendering citationrelationships such as active bibliography and co-citationsrequires queries to the database followed by queries tothe index. The listing of each citation in the relation-ship is based on a similarity document measure imple-mented at the index. The links utilize the cluster ID,which is mapped to the document in case the document isavailable in the collection. In case where the documentis not available, it points to the index listing of docu-ments citing the citation entity. The citation relationshipsand ranking of authors, documents and citations are alsosummarized in a detailed year by year list in the statisticspages.

Figure 6: Active Bibliography

7 MyCiteSeer

SeerSuite aims to provide the user with services whichimprove the efficiency of the user in accessing informa-tion. MyCiteSeer plays a crucial role in providing andsupporting these services. MyCiteSeer allows users tostore queries, document portfolios, tag documents, andmonitor and track documents of interest. The portalspace is available after user registration and login. Webriefly describe the user interface of MyCiteSeer.

Figure 7 shows the index page for MyCiteSeer. Theindex page serves as the landing page, with the menuproviding links to other pages including the profile, col-lections tags and monitoring pages.

The profile page presents the user with interfaces

to update information stored as part of his profile onMyCiteSeer including the password for the account. Inthe case where API support has been enabled, the profilepage also allows the user to request an API key.

The collections page allows the user to view collec-tions of documents stored within the account. These col-lections are user defined sets of documents, aggregatedunder their profile for ease of access. The user can usea collection to download bibliographic data for all docu-ments in a collection.

Tags provide the user with a listing of tags defined bythe user and link to the documents tagged with that tag.The tag portal page allows the user to view and deletetags defined by the user and the documents these tags arelinked to.

The monitoring page allows users to track changes toa document in the SeerSuite collection. Any updates todocument metadata, including the citation graph linkedwith the document for documents in the monitored col-lection are sent to the user through e-mail registered withthe system.

In addition to providing the user with a portal, MyCite-Seer enables SeerSuite to utilize crowd sourcing or dis-tributed error correction [29] for corrections to documentmetadata. By assessing weights based on prior correc-tions, the evidence based system can detect maliciouschanges.

The application program interface component utilizesMyCiteSeer user data for generating the access key andcontrolling access to services provided. An user markedas an administrator has additional functionality availableto him through MyCiteSeer. A subset of configurationand administrative interfaces are made available throughan admin console.

The portal framework shares the structure and stor-age with the main application. While the servlets forMyCiteSeer are developed with SeerSuite in mind, theinteraction with SeerSuite applications can easily be ex-tended to other projects and services. The MyCiteSeercomponent interacts with SeerSuite ingestion and main-tenance modules, using listeners. The maintenance andingestion service provide notifications on objects beingupdated or processed through these listeners.

8 Other Interfaces

8.1 OAIThe Open Archives Initiative (OAI) provides an effi-cient protocol metadata dissemination framework fordata sources such as a SeerSuite. A low barrier mech-anism, OAI is particularly suitable for SeerSuite applica-tions, enabling instances that provide metadata sharing,publishing and archiving. SeerSuite supports interfaces

8

Figure 7: MyCiteSeerx Index Page

compliant to the OAI-Protocol for metadata harvesting(PMH) [14] previously established with CiteSeer [36].

In the earlier CiteSeer system, modified CGI scriptswere utilized for handling queries and generating com-pliant content for the OAI. In contrast, requests madeto the SeerSuite OAI interface are handled by servlets,which translate requests into data access calls. Theservlets assemble results in an OAI compliant manner,which is presented to the client. SeerSuite supports thefull complement of OAI-PMH version 2 verb requestsand provides content in the Dublin Core format. Thus,all document metadata is accessible to a client throughthe OAI interface.

8.2 APIApplication Programmable Interfaces (APIs) are cen-tral for programmatic access to the repository. ThroughREST [20] web services, SeerSuite supports the needs ofboth programmers aiming to access metadata from thedigital library and software agents looking to exchangeinformation between digital libraries. SeerSuite API is arevamped version of the previously developed CiteSeerAPI [35] which was SOAP/WSDL-based.

The goal of the SeerSuite API is to share metadatawith developers and software agents. Moving to a REST-based web service from a SOAP/WSDL-based one re-duces the size of requests and the responses exchangedbetween the client and the server. Hence, developers canretrieve information faster, and total network traffic is re-duced. The version of the API deployed in CiteSeerx

provides access to the papers, authors, citations, key-words, and citation contexts. The API caller may pro-

vide a SQL-like query to be executed as a filter on thematching set. The resource URI formats are shown intable 1. Objects are identified by document IDs (docid),author IDs (aid) and citation IDs (cid). SeerSuite APIcan output the results in both XML and JSON formatsdepending on the callers preference.

Type URI FormatPaper http://host/papers/[docid]Author http://host/authors/[aid]Citation http://host/authors/[cid]

Table 1: CiteSeerx API Resource URI Formats

SeerSuite adopts Jersey [12] as a library to build theRESTful web service, which in turn implements theJAX-RS [13] reference. SeerSuite requires users lookingto use the API to have a valid MyCiteSeer account. Ac-count information in MyCiteSeer is used to generate anApplication ID (appid) which has to be passed in everyHTTP request as a mechanism of authentication. Dailylimits for users are monitored and can be managed byadministrators for performance.

9 Federation of Services

CiteSeerx includes several unique services, which are notpart of the SeerSuite application framework. Provision-ing for these services is an unique aspect of the Seer-Suite framework. Many of these services have evolvedas a result of research and are still being developed. Thedeveloper builds and operates these services indepen-dently, sharing hosting infrastructure with the main ap-plication. Separate tables and databases and index oper-ations maybe provisioned for each service. In the follow-ing sections, we briefly discuss Table search and authordisambiguation search.

9.1 Table SearchTables in documents often contain important data notpresent elsewhere. Table search services are based onTableSeer developed as part of the project [31]. Tablesearch automatically extracts tables metadata, and in-dexes and ranks tables present in a SeerSuite collection.While table search shares components of the web appli-cation and shares the repository with SeerSuite, the indexand extraction components are independent of SeerSuite.

The SeerSuite interface utilizes the main applicationframework for interaction with the table index. Thequeries results from the index are again processed andpresented by the main application framework. Indepen-dent operation of the index from the main index allowsfor more efficient query processing and ranking of table

9

search results. The results utilize the SeerSuite file sys-tem infrastructure view the result of particular pages ofthe tables in cached documents. The ingestion, main-tenance and updates services for Table search are inde-pendent of SeerSuite, allowing for flexibility in researchand development. Some aspects of the table search in-gestion system require access to the document metadatasuch as title, author not extracted as part of Table ex-traction, which are acquired from the main SeerSuite in-stance metadata.

Table search has served as a template for the devel-opment of similar services such as algorithm and figuresearch, which are in development.

9.2 Author Disambiguation

Author disambiguation enables users to identify whetherrecords of publications in a SeerSuite collection refer tothe same person. The author disambiguation service pro-vided by SeerSuite is based on an efficient integrativeframework for solving the name disambiguation prob-lem. A blocking method retrieves candidate classes ofauthors with similar names and a clustering method, DB-SCAN, clusters papers by author. The distance met-ric between papers used in DBSCAN is calculated byan online active selection Support Vector Machine algo-rithm(LASVM) [26]. This system has been utilized inCiteSeerx. The disambiguation application identifies dis-tinct authors based on header information which includesauthor affiliation and co-authorship.

The implementation makes use of already existing au-thor object data in the main database, and generates clus-ter IDs for disambiguated authors that are stored in a sep-arate table and index. Results for disambiguated authorqueries are handled by main application framework byinteracting with the main database and the index. A pro-file page exists for each disambiguated author, with au-thor affiliation, impact, and publications garnered fromthe SeerSuite instance. The profile page also providesa link, if available, to the author homepage obtainedthrough a system HomePageSeer. An incremental algo-rithm replacing the offline batch algorithm currently usedis in development.

10 Usage

CiteSeerx receives nearly two million requests fromacross the globe. A significant portion of this traffic isas a result of document views, downloads and searches.An analysis of access logs is presented in this section.

The graph in figure 8 shows average hits per monthfor CiteSeerx during the year 2009. The search group-ing includes requests for document and author search.

Figure 8: CiteSeerx Traffic in 2009

The ’other’ grouping includes queries for citing docu-ments, legacy mappings (redirects from CiteSeer), OAI,and requests to author profile and static pages. Docu-ment related requests include downloads and summaryviews. The graph indicates a growth in the number ofhits, driven by downloads, views of citation relation-ships, and search. The number of document views havegrown by a lesser margin. During this time, the collec-tion of documents in CiteSeerx grew by 200,000 docu-ments with updates to document metadata through cor-rections.

The majority of the referrals to CiteSeerx are throughpages hosted on CiteSeerx (67 %). A number of users(29 %) arrive without a referrer,(i.e., users landing di-rectly on CiteSeerx or requests by crawlers). Redirectionfrom CiteSeer contribute (1 %); references from Google,Google Scholar (2 %,1 %), Yahoo, and Bing (all <1 %).

Figure 9: Country Profile

Figure 9 shows a division of traffic from differentcountries in 2009 identified using GeoIP. Among users

10

of CiteSeerx, nearly half of users are from the UnitedStates. Taiwan, Germany, China, India, UK, France,and Canada are other major sources of traffic. Trafficfrom two hundred and twenty countries are grouped un-der ’other’.

Along with valid accesses, CiteSeerx experiences a va-riety of attacks every day. These attacks involve access toforbidden areas, portscans, SQL injection attacks, doubledecoding, buffer overflow, and cross scripting attempts.

11 Collaboration and Distribution

SeerSuite has been developed in collaboration with sev-eral groups across the world. This collaboration includesexchange of ideas, development of modules and host-ing of services. Independent copies of CiteSeerx aremaintained by these research groups at University ofArkansas, National University of Singapore and KingSaud University.

Data collected as part of crawls, document metadata,anonymized user log data are available on request. Sev-eral research groups have already taken advantage ofthese datasets. The source code for SeerSuite has beenreleased under the Apache software license version 2 onsourceforge.

To improve adoption of SeerSuite based systems andprovide the user an efficient way to explore and deployinstances. A virtual appliance with SeerSuite has beenmade available for such a purpose. The appliance usesopen source components and contains a complete de-ployment of SeerSuite. This allows users to explore andoperate instances of SeerSuite, in the pre-installed VM.The use of the virtual appliance also benefits developersand testers.

User feedback and comments have helped improveboth user the experience and the troubleshooting bugs.SeerSuite enables other innovative and valuable tools tobe built using metadata extracted and published. Suchefforts include projects such as PaperCube [17] andJabRef.

12 Lessons Learned and Future Work

By adopting a multi-tier architecture, open source appli-cations and the use of software frameworks, SeerSuitehas improved upon the client server architecture initiallyadopted by CiteSeer. In addition this is an approach thathas provided reliable scalable services in CiteSeerx.

One of the lessons learned is in the provisioning ofinfrastructure. To allow SeerSuite instances to grow tolarge sizes without constraints on storage and computa-tion, virtualization and distributed computing need to beutilized and managed.

The number of requests for metadata and data from theCiteSeerx collection emphasizes the need for developingautomated methods data sharing such as API and OAIservices for SeerSuite instances. There have also beenrequests for a content based similarity and duplicate de-tection service, which is under development.

With the new design adding features in SeerSuite ismore straight forward. However, further work on sepa-rating components is required to support systems whichmake use of a smaller subset of services. These systemsmay not include the citation graph service. Among otherservices, MyCiteSeer should be developed as a standalone service. This will easily enable other services suchas HomePageSeer that take advantage of login based ac-cess. Development of MyCiteSeer as an independent ser-vice, shared across many Seer instances is under con-sideration. This would allow users to share informationacross projects and instances.

One of the continuing major challenges is new andhigh quality metadata extraction. Another is that mod-ules will need to be refactored to support massive crawlsusing parallelization at the module level.

A number of optimizations have been implemented inSeerSuite, prominent among these is the citation rela-tionships stored in the index. This allows citation rela-tionship queries which form document summary viewsto be more readily handled. Such an optimization has thedrawback in that updates to these relationships are onlyavailable to the user once there has been updates to theindex.

A federation of services model adopted for newer ser-vices benefits both users and developers. This modelgives users a peek into new features and researchers aneasy a way to include new services. From a developmentstandpoint, this is also useful since services can be testedand users can provide feedback.

Improvements to the User Interface will be required tosupport upcoming features. Ranking of results and im-proving relevance in search are active topics of research.Performance analysis tests for SeerSuite to identify pos-sible improvements and issues are being designed.

13 Summary

We have described the design, architecture, and deploy-ment of SeerSuite. We believe SeerSuite overcomesmany of the issues in an earlier system, CiteSeer, which,due to its design, limited growth and extensions to otherservices. SeerSuite is designed to take advantage of opensource applications, frameworks and state of the art com-ponents. It also allow users to readily build mashups andrelated applications. The use of loose coupling of mod-ules and federated services enables SeerSuite to easilyoffer new features and components.

11

The user interface allows search, document summaryviews, and citations clustering. We identify workflowsfor generating these pages. Various features such astagging, building collections and correcting documentswere identified for the MyCiteSeer portal. In addition toproviding services through the web user interface, Seer-Suite also provides services through the OAI/PMH andAPI interfaces.

The statistics and usage pattern for CiteSeerx, a Seer-Suite instance, provide information about growth in traf-fic and the profile of users based on the country of origin.We show that SeerSuite is a collaborative venture withopen source code and virtual appliances that encourageadoption and research. We believe that SeerSuite willcontinue to improve and support a wide variety of ser-vices and user needs while remaining scalable, reliableand robust.

14 Acknowledgments

We gratefully acknowledge partial support from the NSFand useful comments by Yves Petinot.

References[1] ACM Portal. http://portal.acm.org/portal.cfm.

[2] ActiveMQ. http://activemq.apache.org/.

[3] ChemxSeer. http://chemxseer.ist.psu.edu.

[4] CiteSeer. http://citeseer.ist.psu.edu.

[5] CiteSeerx. http://citeseerx.ist.psu.edu.

[6] CiteULike. http://www.citeulike.org/.

[7] DBLP. http://www.informatik.uni-trier.de/

˜ley/db/.

[8] Dspace. http://www.dspace.org/.

[9] Google Scholar. http://scholar.google.com/.

[10] Greenstone. http://www.greenstone.org/.

[11] Heritrix. http://crawler.archive.org/.

[12] Jersey API. https://jersey.dev.java.net/.

[13] JSR 311. https://jsr311.dev.java.net/nonav/releases/1.1/index.html.

[14] Open archives initiative - protocol for metadata harvest-ing v.2.0. http://www.openarchives.org/OAI/openarchivesprotocol.html.

[15] RePEc. http://repec.org/.

[16] Solr. http://lucene.apache.org/solr/.

[17] BERGSTROM, P. Papercube. http://papercube.peterbergstrom.com.

[18] COUNCILL, I. G., GILES, C. L., IORIO, E. D., GORI, M.,MAGGINI, M., AND PUCCI, A. Towards next generation cite-seer: A flexible architecture for digital library deployment. InResearch and Advanced Technology for Digital Libraries, ECDL2006 (2006), pp. 111–122.

[19] COUNCILL, I. G., LI, H., ZHUANG, Z., DEBNATH, S.,BOLELLI, L., LEE, W. C., SIVASUBRAMANIAM, A., ANDGILES, C. L. Learning metadata from the evidence in an on-linecitation matching scheme. In JCDL (2006), pp. 276–285.

[20] FIELDING, R. T. Architectural Styles and the Design of Network-based Software Architectures. PhD thesis, University of Califor-nia, Irvine, Irvine, California, 2000.

[21] GARFIELD, E. ”Science Citation Index”a new dimension in in-dexing. Science 144, 3619 (1984), 649 – 654.

[22] GILES, C. L., BOLLACKER, K. D., AND LAWRENCE, S. Cite-seer: An automatic citation indexing system. In Digital Libraries(1998), pp. 89–98.

[23] GINSPARG, P. Can Peer Review be better Focussed.http://people.ccmr.cornell.edu/˜ginsparg/blurb/pg02pr.html.

[24] HAN, H., GILES, C. L., MANAVOGLU, E., ZHA, H., ZHANG,Z., AND FOX, E. A. Automatic document metadata extractionusing support vector machines. In JCDL ’03: Proceedings of the3rd ACM/IEEE-CS joint conference on Digital libraries (2003),pp. 37–48.

[25] HAN, H., MANAVOGLU, E., ZHA, H., TSIOUTSIOULIKLIS, K.,GILES, C. L., AND ZHANG, X. Rule-based word clustering fordocument metadata extraction. In SAC (2005), pp. 1049–1053.

[26] HUANG, J., ERTEKIN, S., AND GILES, C. L. Efficient namedisambiguation for large scale databases. In The 10th EuropeanConference on Principles and Practice of Knowledge Discoveryin Databases (2006), pp. 536–544.

[27] ISAAC COUNCILL, C. L. G., AND KAN, M.-Y. Parscit: anopen-source crf reference string parsing package. In Proceedingsof the Sixth International Language Resources and Evaluation(LREC’08) (Marrakech, 2008), European Language ResourcesAssociation.

[28] KAHN, R., AND WILENSKY, R. A framework for distributeddigital object services. International Journal on Digital Libraries6, 2 (2006), 115–123.

[29] LAWRENCE, S., BOLLACKER, K., AND GILES, C. L. Dis-tributed error correction. In DL ’99: Proceedings of the fourthACM conference on Digital libraries (1999), p. 232.

[30] LI, H., COUNCILL, I., LEE, W.-C., AND GILES, C. L. Cite-seerx: an architecture and web service design for an academicdocument search engine. Poster Session 15th International WorldWide Web Conference (2006).

[31] LIU, Y., BAI, K., MITRA, P., AND GILES, C. L. Tableseer:automatic table metadata extraction and searching in digital li-braries. In JCDL (2007), pp. 91–100.

[32] MCCALLUM, A., FREITAG, D., AND PEREIRA, F. C. N. Maxi-mum entropy markov models for information extraction and seg-mentation. In ICML (2000), pp. 591–598.

[33] MITRA, P., GILES, C. L., SUN, B., AND LIU, Y. Chemxseer: adigital library and data repository for chemical kinetics. In CIMS’07: Proceedings of the ACM first workshop on CyberInfrastruc-ture: Information Management in eScience (2007), pp. 7–10.

[34] NII, H. P. Blackboard systems, part one: The blackboard modelof problem solving and the evolution of blackboard architectures.AI Magazine 7, 2 (1986), 38–53.

[35] PETINOT, Y., GILES, C. L., BHATNAGAR, V., TEREGOWDA,P. B., HAN, H., AND COUNCILL, I. Citeseer-api: towards seam-less resource location and interlinking for digital libraries. InCIKM (2004), pp. 553–561.

[36] PETINOT, Y., TEREGOWDA, P. B., HAN, H., GILES, C. L.,LAWRENCE, S., RANGASWAMY, A., AND PAL, N. ebizsearch:an oai-compliant digital library for ebusiness. In JCDL (2003),pp. 199–209.

[37] TAN, Q., MITRA, P., AND GILES, C. Metadata extraction andindexing for map search in web documents. In Proceeding of the17th ACM CIKM (2008), pp. 1367–1368.

12