Mohan Bang

Content Delivery System

Mohan Bang

Agenda: Enterprise Search – Content Delivery System

• Information Needs Drive Business Processes

• Why do companies invest in Search Technology?

• Search Engine Overview (objective, types, how they works)

• Enterprise Search vs Web Search

• Business Information Problem and Challenges

• Enterprise Search Engine (features, types, how they works, logical architecture)

• Implementation of a Search Strategy

• Different levels of the Search Service

• Enhancing Enterprise Application by integrating with Enterprise Search

• Taxonomy, Faceted Search / Guided Navigation, Ranking and Relevancy

• Various Enterprise Search Technologies in market

• Enterprise Search Solutions in the Manufacturing Supply Chain

• Search Engine Optimisation & Search Engine Marketing

• Conclusion & Summary

Mohan Bang

Information Needs Drive Business Processes

Find information needed to complete business tasks

Improve problem resolution capabilities for contact center

Find products you want to purchase more easily

Mohan Bang

Why do companies invest in Search Technology?

Business Benefit Value of Search

Increase Productivity Enable employees to more quickly find information needed to complete their business activities

Decrease Costs Empower customers and partners to support themselves and perform their own research

Increase Revenues Ensure customers can easily find products and services, driving higher sales and increasing customer retention

Enterprise Search – is critical to Organizational Success

Mohan Bang

• End users should be able to find the information (provides information on-demand)• Initiate more business through the application / web• User-friendly interface• Advertising Programs

Strong management of quality and quantity of information Interface matching with expectations of all users From Google like to Expert search A search that is contextual to the company, the location and the business A search that is versatile (synonymous, cleansing, navigation…) Response time accepted and ensured Functional upgradeability (related links…) and technical scalability A quality roadmap (reporting and monitoring leading to actions)

Objective of Search Engine

Mohan Bang

Web search (limited to public documents on the web) is generic search. One size fits all. Features serve the technology to better enable it to serve the masses. Search technology has to work for the broadest document set, those billions plus pages

Important to note: The Page Rank algorithm is a pre-query calculation. It is a value that is assigned as a result of the search engine’s indexing of the entire Web and the associated value has no relationship to the user’s information need.

Desktop Search (limited to private documents on the local machine)

Enterprise Search (no limitations on document type and location) is fundamentally about Information Access – delivering the right information, on demand, in the right context and at the right time, helping knowledge workers solve business problems and giving customers the information they need to take action. In today’s Internet world, search is the primary mechanism for information access – but information access goes beyond search for example in insightful solutions, where search is joined by mining and BI interfaces in a suite of technologies for delivering breakthrough business insight.

Three key challenges in effective information reuse and information management.

• Information Workers cannot find the desired information. Finding the information is challenging.• Some systems, such as Document Management Systems require that Information Workers classify

information. That does not happen!• These systems also require metadata model such as taxonomies. Unfortunately metadata in these

solutions are unmanageable!

Types of Search Engine

Mohan Bang

Search Engines:– Web spiders look for links, take web pages content and create/build a list of key search

words that enable online users to find the pages they are looking for and notes where they were found.

– Index built on own system of weighting (Ranking involves comparison of your web pages to others and looks for keywords. After this comparison, they are ranked).

– Web spiders encode data to save space, compressing the information. Data is then stored for users to access.

– Web Spiders cannot get into Flash / JavaScript / CSS and so a website needs web spider friendly pages. Flash / JavaScript / CSS should be in separate file. Web Spiders only get into HTML/XHTML.

Overview on Components of an Search Engine / Enterprise Search System

A search engine operates, in the following order: Collecting Data / Web crawling Analyzing Data Indexing Data Searching Data

How Search Engine works?

Mohan Bang

Enterprise Search vs Web Search

  Web Search Enterprise Search

Search corpus Every public webpage – the whole internet Public documents in the enterprise, departmental docs, plus local docs (My Documents)

Context Generic : Shopping or seeking news and information

Company-specific: Executing a role in a business process

Taxonomies / categories

Generic – Open Directory Project, Wikipedia, News, etc.

Domain Specific (customers, organization, products, technologies, processes)

Info Security Information is public Information is secure with role-based access controls

Search algorithms Keyword and Link-based (Page Rank) Keyword and tag-based

Perfect result Most popular content Highest quality content

More Differences • Publishers want their content to be found• Publishing model = “anyone, anywhere, any time”• Unlimited document set• No real standards or code, more like guidelines• No central authority• Spam• Commercialization• Has to work the same for everyone worldwide

• Publishers do not think about document discoverability• Controlled corpus of documents• Standards and practices in place• No spam• Users and authors generally share contextual understanding• Customized tagging or metadata• Can customize search technology to enterprise themes and concepts

Mohan Bang

Business Information Problem

What are the difficulties?• Storage: increasing• Access: faster• Business Data complexity: more – for example: Catalog Management System• Information Quantity: increasing exponentially

Why the users would stop using your search ?• Bad representation of the results• Interface not user-friendly• Response time• Not exhaustive

The lost users are extremely hard to get back !

Mohan Bang

Challenges

• Performance and scalability• Rich functions and features• Manageability• Flexibility• Easy maintenance• Quick issue and problem solving• Reduce total cost of ownership

Finding the information needed was difficult and time consuming. Consumer Web has raised expectations for Enterprise Search

Why did we require Enterprise Search?

Mohan Bang

“Enterprise Search offers a solution for searching, finding and presenting enterprise related information in the larger sense of the word”

Enterprise Search is the practice of identifying and enabling specific content across the enterprise to be indexed, searched and displayed to users

• All about finding: rich navigation; focus on quick find• Small targeted audience• Specialized and customized screens (use of taxonomies and classifications)• Use of identity (results customized to user)• Grouping: field collapsing, faceted search and clustering

What is Enterprise Search?

Mohan Bang

Information Sources and Types

- Wide range of sources: local and remote file systems, content repositories, e-mail, databases, internet, intranet and extranet

- Type not limited: any type ranging from structured to unstructured data, text and binary formats and compound formats (zip)

Usage

- Not limited to interactive use: automated business processes

Security

- Integrations with enterprise security infrastructure

User Interaction and personalization

- Identity enables more personalized search results

Ranking

- More control over ranking: personalized ranking (group)

Features: Enterprise Search

Mohan Bang

Data extraction and derivation

- Extract data using various techniques: Xpath, Xquery

- Derive data: using external knowledge models: RDBMS, Web Services

- Conditional extraction and derivation

Managing and monitoring

- On-the-fly management

- Real time monitoring

User Interfaces

Features: Enterprise Search

Mohan Bang

A Wide Array of Search Needs

The Enterprise Search Market (Information Access)

Types of Enterprise Search:Types of Enterprise Search:

Component Infrastructure Experience-DrivenSimple Utility Insightful

Basic, Unsecured Web Site Or

Departmental File System Search

Component Search Engine Or

Libraries

Scalable, Secure Enterprise Search With Broad Reach To Many Content

Sources

Tunable, Business Controlled Search With Application-

specific Interfaces, And Reporting

Search That Understands

Semantics, Can Answer Questions

Directly, And Can Pull Different Pieces Of Data Together Into

Meaningful Analyses.

OEM

Simple site search

SMB/basic intranet

Secure intranet search

Integration of enterprise content

Intranet portals

Platform for search applications

Self Service

e-Commerce

Contact Centers

Customer Care & Customer Insight

Quality Early Warning

Public Image Monitoring

Compliance & Legal Discovery

BI for the Masses

Mohan Bang

Mohan Bang

Logical Architecture: Enterprise Search Solution

Enterprise Search - Technology Stack

Co

nten

t Ind

exing

Qu

ery

Pro

cess

ing

Mohan Bang

Enterprise Search – Collection Process

Mohan Bang

Enterprise Search – Collection Process

Mohan Bang

Enterprise Search – Collection Process

Mohan Bang

Enterprise Search – Collection Process

Mohan Bang

Enterprise Search – Collection Process

Mohan Bang

Enterprise Search – Publication Process

Mohan Bang

Enterprise Search – Publication Process

Mohan Bang

Enterprise Search – Publication Process

Mohan Bang

Enterprise Search – Publication Process

Mohan Bang

Enterprise Search – Publication Process

Mohan Bang

Enterprise Search – Publication Process

Mohan Bang

Analysis

What is the status of my information repository ?

Quality Audit, sizing, volumes, dispersion, number of sources, variation, evolution..

What is the expectation of my business users ?

Architecture

What are the key functions ? What is the evolution path ? What will be the enterprise search engine ? What will be the interface tool(s) ? What is the sizing ?

Phased implementation

Scalability, Function Evolution..

Quality process

Usage Monitoring > Tuning of the search service + actions on information management

The implementation of a search strategy

Mohan Bang

Search criteria

• Google like : full text

• Synonyms, business dictionaries

• Source filtering

• Metadata (classification, filters, facet's) > depend on the produced information

• Taxonomy, …

Results

• list ordered by relevancy

• list with quick view

• contextual filters

• faceted navigation

The different levels of the search service

Mohan Bang

• One can perform Google-like search that scan your entire enterprise data in no time.

• Searching as easy as the web with Enterprise Application you quickly locate the information you need. The Enterprise Application allows you to perform Google-like search by simply typing in the search bar: For example, if you key in a customer name, the Enterprise Application will presents a list of objects which contain the name you have typed.

• The list is sorted by relevance. Categorize the search result to get a better overview of the objects that are related to the search criteria.

• Controlled access security is built in and ensures that only those with the appropriate authority can view protected information.

• Find what you need—quickly: The advanced search options in Enterprise Applications means you spend a lot less time searching and a lot more time doing.

Enhancing Enterprise Application by integrating with Enterprise Search

Mohan Bang

A taxonomy is a hierarchical topic structure to which information can be assigned through

the dual processes of classification (filing to a location) and categorisation (tagging with

relevant metadata). A taxonomy provides browsable navigation and supports filtered

searching

Taxonomy

Mohan Bang

Faceted search lets users refine or navigate a collection of information by using a number of discrete attributes – the so-called facets.

Faceted Search / Guided Navigation

Chevrolet is a facet, a way of categorizing the results

Corvette, Camaro, Chevelle are constraints, or facet values

The breadcrumb trail shows what constraints have already been applied.

Search Result List

The facet count or constraint count shows how many results match each value

Mohan Bang

Ranking

A Ranking allows you to control the order in which results are returned. For example, one approach is

to rank more relevant results by the number of keywords in a query that match in a record. Records

with a higher number of matching keywords display before records with a lower number of matching

search keywords.

Mohan Bang

Auto-Correct / Spell Check

Example of Auto-Correct / Spell Check

Mohan Bang

List ordered by relevancy

Example of List ordered by relevancy

Mohan Bang

Old Model Problem – Performance, Expert Search, Business Data Complexity, Infrastructure / Server Issue

Mohan Bang

Various Enterprise Search Technologies in market

• FAST Enterprise Search Platform, Microsoft Office SharePoint Server by Microsoft (acquires FAST)

• Google Search Appliance by Google

• Lucene/Solr by Apache

• Autonomy by Autonomy Corp. (acquires Ultraseek, Verity)

• OmniFind by IBM

• Endeca by Endeca Technologies, Inc.

Mohan Bang

New Model - Why Enterprise Search? Scalable Performance

Mohan Bang

Endeca in Banking Industry

Endeca is a Big Data Technology used to managed the data and for performance driven enterprise search. Below are the areas where Endeca can be used in Banking Industry for Quality Purposes. To improve the banking services, bank should have analytics with them.

We can develop enterprise application/web portal using endeca for performance driven enterprise search.

a) That will have the analytics on any kind of banking services as per the bank need

b) Handling of fraud cases

c) To display graph/chart for big data

d) Endeca use case in Bank - COMPLAINT ANALYTICS

Mohan Bang

Enabling Information Visibility to Critical Business Processes in the Manufacturing Supply Chain

With Historic data and Enterprise search, one can do sales forecast

Mohan Bang

Enterprise Search Solutions in the Manufacturing Supply Chain

Mohan Bang

Warranty generate issues across the value chain

Mohan Bang

Complex Relationships in Traceability & Investigation of a Warranty Data

Mohan Bang

Warranty Traceability & Insight

An custom Enterprise application integrated with Enterprise Search Engine for getting quick

results for search can be used to analyze trends related to vehicle, part and supplier quality,

based upon warranty claims submitted by its authorized service centers. Measures, such as

number of claims and claim cost, can be charted against dimension such as labor code

identifying the repair type, model year, etc. There are many dimensions by which the

warranty claim result set can be narrowed for analysis and charting.

Mohan Bang

Warranty Traceability & Insight

An automotive manufacturer analyzes detailed comments in warranty claims to detect failures and understand relationships among parts involved in failures. Catching an engineering flaw early reduces recall costs and litigation potential

Mohan Bang

Healthcare Industry

Healthcare providers rely on Enterprise Search across a broad range of services. An executive demands analysis of the cost of operational readiness, a physician makes life or death care decisions based on their patient's condition and medical research data.

Enterprise Search for Healthcare enable knowledge workers and constituents to discover new, untapped information to gain knowledge and improve decision making faster, more easily, and more accurately than ever before. These capabilities enable a wide range of business goals, including identifying and reducing waste, fraud, and abuse, improving quality of care, and measuring performance against expenditures, as well as delivering valuable externally facing services to the public.

Mohan Bang

Enterprise Search in Health Care Industry Sector

Few nations in the world with a completely free public healthcare system. Like any public service of that magnitude, many operational problems exist. Parts of the system were largely paper based and very little integration was available between existing IT systems between the government and local health care providers. For example:

No scheduling system existed that could allow health care providers to book patients into appointments with each other. For example, a patient requiring a visit to a heart specialist would need to find an available specialist on their own, often by waiting in long lines at multiple specialists and booking duplicate appointments until an early enough opening could be found.

Lack of information about patient history such as history past operations or medical conditions at any point of care.

The same patient was often recorded multiple times in a number of separate databases.

Inability for the government to do any medical resource capacity planning or react to shortages since important statistics such as births, deaths, medical operations and disease diagnoses were stored in un-integrated paper-based systems.

Inability to detect and prevent abuse of the medical system since IT systems that modeled health legislation and policy was not integrated with point of care systems.

Many local doctors had no information systems at all and patients were required to wait all day long due to a lack of scheduling.

Mohan Bang

Enterprise Search in Health Care Industry Sector

With these problems in mind, the building of a health care automation system is required. The designed system should handle all aspects one could imagine in a public healthcare information system including scheduling and inventory management of doctors and equipment, billing, disease tracking, reporting/auditing, regulatory compliance, and security access control.

Mohan Bang

Example - Endeca Selected by the National Cancer Institute to Power Search of Web Most Comprehensive Collection of Cancer Information

Endeca, the leading provider of Guided Navigation®, Search and Analysis solutions announced today that the National Cancer Institute (NCI), part of the U.S. National Institutes of Health, has selected and deployed the Endeca ProFindà search and navigation platform on its award-winning Web site, Cancer.gov (www.cancer.gov). Using the new search engine, Cancer.gov users can more quickly and easily access the largest and most comprehensive collection of research-based information on cancer types, treatment, prevention, and clinical trials, as well as research and funding information, from across all of NCI's Web sites.

NCI is the federal government's principal agency for cancer research. Cancer.gov is a primary vehicle for disseminating this information to NCI's broad range of constituents - cancer patients, their families, physicians, and researchers. And the search tool is an essential adjunct to Cancer.gov's award-winning site design and navigation aids. Recipient of the coveted 2004 FREDDIE Award (the International Health and Medical Media Awards) for best Web site, Cancer.gov provides immediate access to information and resources on cancer and helps people with cancer become better informed about their disease so they may more actively participate in their treatment and care.

Mohan Bang

Example - Endeca Selected by the National Cancer Institute to Power Search of Web Most Comprehensive Collection of Cancer InformationEndeca ProFind was selected after a thorough review of technologies from a number of leading vendors, and it replaced a legacy commercial search solution. The implementation of Endeca's search capabilities on Cancer.gov will follow a phased approach. The initial phase was deployed within two weeks of the conclusion of the selection process and provides accurate search services to the site's users. To ensure that Cancer.gov visitors have access to all relevant information, the new search engine indexes thousands of documents on the site plus highly relevant content culled from among tens of thousands of documents on NCI's other cancer-related Web sites. In the months to come, the software's versatile capabilities will be deployed to offer users advanced, state-of-the-art search functionality to provide enhanced access to NCI's online resources. The software will also be used to build search capability for other online NCI products.

Used on Web sites, intranets, extranets, and within corporate enterprises, Endeca ProFind is a next-generation information retrieval platform that combines highly advanced search with Guided Navigation and Content Spotlighting capabilities to help people quickly and easily find the information most relevant to their unique needs. Endeca' approach to information retrieval mimics the human discovery process by integrating the two most common means of finding information online - searching and browsing - allowing people to continually adapt and hone their search based on their own determination of relevancy. Endeca ProFind' robust and easily managed Content Spotlighting capabilities promote relevant content based on defined rules, such as user profile or workgroup, search terms, navigation options, and time of day or day of the week. This combination gives organizations the powerful and intuitive tools to maximize the value of internal and external information, regardless of source, structure or file type.

Mohan Bang

Conclusion & Summary

It helps you find your stuff…

• A tool to help your business users find what they are looking for• Allows for indexing, searching, displaying of business data from dedicated / distributed

sources with little or no code• Designed for Read-only access. Also Inserts/Updates/Deletes are possible but there are

limitations and it is not a replacement for a Data access layer

Powerful Full-Text Search High performance, scalable, availability Faceted Search Data caching User Friendly Administration Interface Search Relevancy and Filters Industry standard platforms Multi-language support Very fast access to information Dynamic Ranking

Mohan Bang

Get Traffic, Get Customers

• Search Engine Optimisation (SEO)• Search Engine Marketing (SEM)

• Online advertising– Search engines like Google (organic search results)– Search engine Advertising (paid-for search results)

Paid SearchPaid SearchResultsResults

(sponsored)(sponsored)

Organic Organic SearchSearchResultsResults

(natural)(natural)

Mohan Bang

Search engine optimisation (SEO) is the process of improving the volume or quality of traffic to a web site from search engines via "natural" or un-paid ("organic" or "algorithmic") search results as opposed to search engine marketing (SEM) which deals with paid inclusion links or pay-per click advertisements.

Search Engine Optimisation (SEO) involves the process of altering or “optimising” a website so that it does well in the organic, crawler-based listings of search engines.

• Is the process of optimising your website so that it will be found easier by search engines like Google.

• Aims to achieve higher “organic” search engine results• Involves some technical optimisation but mostly content

Search Engine Marketing (SEM) is the process of marketing a website via search engines, both organic listings, paid listings or both.

• SEO and SEM are interlinked.• Goal: high ranking of website pages in search engines.• SEO takes time.

Mohan Bang

Mohan Bang

SEO Fundamentals

• Long Tail SEO Strategy• Keywords - High-impact search terms used regularly by prospects trying to find your

products and services• Use of Long-tail keywords even more important• With many pages, you have the opportunity to target more specific keywords on deeper

internal pages Ex: Not ‘laptop’ but ‘small black laptop’

Mohan Bang

Thing That Matter in SEO

• Keyword Research• Theme of the Site• Crawl ability/Ease of Link ability• Depth of Site• Duplicate Content (block, redirect or canonical)• Page Load Time• Social Media (Facebook + Twitter Aren’t Enough)• Measure, Fine Tune, and Adjust Accordingly• Rewrite Dynamic URL – User Friendly URL• Build sitemap

Implementation

Tracking

Analysis

Mohan Bang

Using Site Structure to Help SEO

• Cascading Style Sheets • Proper Use of “H” Tags• Content in Div Tags• Navigation Techniques• On-Site Blog• Footers

Optimising Website Pages

Search-friendly domain names page titles meta keywords and descriptions body text website

Mohan Bang

Conclusion & Summary

Mohan Bang