Upload
jesse-barber
View
236
Download
3
Tags:
Embed Size (px)
Citation preview
Effective Search Deployment and Operations in SharePoint 2013Kristopher Loranger Mert SangarSr. PFE Sr. PFEMicrosoft Microsoft
SPC360
Search Topology Review
SharePoint 2013 Search Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
Crawl
Search Admin
DB
Link
Analytics Reporting
FAST Search Index
SharePointSP AppsDevicesNon-SP UX
HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS
Public API
Search topology components
Content Query
New-SPEnterpriseSearchTopology New-SPEnterpriseSearchCrawlComponent New-SPEnterpriseSearchIndexComponent … Set-SPEnterpriseSearchTopology
SharePoint Search TopologyVital (and large) part of your farm!
Scalable – content and query load
Fault-tolerant\HA
Appears as a SharePoint Service Application (SSA)
Manage using PowerShell
Status from UI/PowerShell
Main changes from SharePoint 2010… from SharePoint Server 2010 Search
Crawl Component
Query Component
SharePoint 2010 Search Service Application
Crawl Indexing Engine
Query Engine
Search Admin
Property Store (SQL)
Content
UXWFE
Main changes from SharePoint 2010… from FAST Search Server 2010 for SharePoint
FAST Content SSA
FAST Query SSA
FAST back-end components(managed separately)
Extensibility:• Sandbox• Entity
Extraction Indexin
g Engine
Query Engine
Content Pipeline
Analysis Engine
Query Pipeline
Search Admin
UXWFE
Content
Crawl
Main changes from SharePoint 2010… to SharePoint Server 2013 Search
Entire index on local disk
UXWFE
More details: SPC336 - Search Architecture and Sizing in SharePoint 2013
Link/query analysis &
recommendations
Separate crawl and indexing
Content
SearchAdmin
Crawl
ContentProcessing Index
QueryProcessing
AnalyticsProcessing
FAST Search Index
Property Store (SQL)
SharePoint 2013 Search Service Application
Where does Search live in the farm?Windows servicesSharePoint Search Host Controller service
Runtime/lifecycle control of search components (except crawler) hostcontrollerservice.exe
SharePoint Server Search serviceCrawl Component mssearch.exe mssdmn.exe
ProcessesNoderunner.exe
Runtime environment for search components (except crawler)
msseearch.exe
mssdmn.exe
CrawlComponen
t
noderunner.exe
Search Runtime Environment
hostcontrollerservice.exe
Host Controller
Sh
are
Poin
t A
pp
Serv
er
Admin entitiesSearch Service Instance: Provisioning of the search service on each boxSearch Service Application: SharePoint Configuration entity
Still there, but only Crawl Component
noderunner.exe
noderunner.exe
noderunner.exe
noderunner.exe
Admin Component
Query Processing Component
Content Processing Component
IndexComponent
Analytics ProcessingComponent
Search High-AvailabilityComponent HA mode of operation
Crawl Component Load-sharing, stateless in DB
Index ComponentPer partition Primary/Sec/Ter (dynamic selection)
Query load-sharing
Content/Query Processing components Load-sharing, stateless
Analytics Processing Component Distributed map/reduce, stateless in DB
Admin Component Primary/Secondary (dynamic selection)Hot standby
Host Controller (for search dictionary repository)
Primary/Secondary (cmdlet controlled)Lost primary –no newdictionaries deployed
> New-SPEnterpriseSearchTopology> New-SPEnterpriseSearchIndexComponent, New-
SPEnterpriseSearchQueryProcessingComponent, … > Set-SPEnterpriseSearchTopology
Set up your initial search topology
Prepare servers• HW/SW reqs!• Install SharePoint• Add servers to
farm• Start Search
Service Instance• Create SSA
Create a new topology object for the SSA
Add search components to the topology
Assign components to servers
Activate the new topology
This will replace the single-server topology with a scaled-out topology
Need more performance or capacity?Scale out your search farm
Create an edit copy of your active topologyNew-SPEnterpriseSearchTopology -CloneYou never modify the active topology object
Add before removeAdd componentsActivateGreen stateRemove components
Small search topologyOne index PartitionPrepared for 10 Mill indexed documents
Fault tolerance 2 physical servers – 4 VMs
Which components on same server?
Search admin db Link db
Analytics dbCrawl db
All SharePoint databases All SharePoint databases
Analytics
Replica ReplicaIndex partition 0
Query Processing Query Processing
Crawl
Admin
Content processing
Analytics
Crawl
Admin
Content processing
Application Server Application Server
Application Server Application Server
Redundant copies of all databases
VM01
VM03
VM02
VM04
Host server A
Host server D
Host server C
Host server B
Crawl
Content Processing
Index
Analytics Processing
Admin
Query Processing
More details: SPC336 - Search Architecture and Sizing in SharePoint 2013
Index PartitionReplica Replica
Query ProcessingQuery Processing
Demo
Deploying a Multinode Search Topology
Location of files on disk matters!Local disk for index files!Index operations are I/O demanding
At install timeYou can specify the data folder for all search components (including index location)
Per Index ComponentSeparate data folder for the index filesWhen creating a new index componentNew-SPEnterpriseSearchIndexComponent
-RootDirectory E:\Index
ULS logsPrevent disk contention/floodingSet-SPDiagnosticConfig -LogLocation F:\Logs
Note change from SP 2010 Set index location per SSI not supportedSet-SPEnterpriseSearchServiceInstance
-DefaultIndexLocation
Scale out by re-partitioning the indexWhen each partition exceeds 10 million itemsStateless partitioning using consistent hashingPause query and feedingStart with a symmetric index layout
Index
Index
Index
Index
Content Processing
Partitions
Index
Index
Replicas
Regional farm 2Regional farm 1
Index
Regional farm 3
Crawler
Crawler
Crawler Index
Index
Regional farm 2Regional farm 1
Index
Office 365 farm
Crawler
Crawler
Crawler Index
Index
Federation
Regional farm 2
Central Search Farm
Regional farm 1
Search service
CrawlerCrawler
Crawler
Crawler
Regional farm 3
Large enterprise: Multi-farm deploymentsCentral search farm Serving multiple content farmsEasier administrationBetter relevancyBetter end-user experience
Distributed deploymentSearch index on each farmSearch results using federationResult blocks in the search result
HybridOn premise and O365 – Federation Stepwise migration to the cloud
When to choose federation?Slow WAN between remote farmsRegulatory restrictionsMost users want search results from local sourcesHybrid (On premise and O365)
Keeping your search topology healthy
Search Diagnostic SourcesSystem/Component state
Central Admin SSA Search Administration (UI)PowerShell: Get-SPEnterpriseSearchStatus
Search health reports (UI)Content feeding performance, query rate/latency, 0 queries
Monitoring / telemetrySCOM management pack ULS logs and performance counters
Understanding component issuesGreen = Good!Yellow (degraded)
Transient component lifecycle states – Restarts and topology changesIndex replica not in sync
Red (failed / unknown)Manual intervention neededIndicates that server, or search component, is unreachablePossible HW/Network issues
Get-SPEnterpriseSearchStatus gives more details
Get-SPEnterpriseSearchStatusComponent state for each search component
Active / degraded / failed / unknown (not reachable)Extended info with additional data when yellow/red
Primary/secondary role for stateful HA components
Admin component, index component (per partition)
Status for analytics background jobsUsage analytics / Search analytics
Detailed diagnostic data for indexerIndex housekeeping (seeding, master merge, catch up)Diagnostic data for issue resolution
Get-SPSearchTopologyState.ps1One stop shop for search status
Consolidates data from multiple cmdlets and sources
All data from Get-SPEnterpriseSearchStatusCrawl status (crawling/idle/paused)# of indexed documentsHA status for topologyIndex size alertHost controller dictionary repository
Download: See TechNet Wiki
Demo
Health Status and Reports
Monitoring using SCOM
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
CrawlSearch Admin
DBLink
FAST Search Index
Query Service: Service availability query processing
QueryProcessing
Index: Missing partition
Index
FAST Search Index
Content Processing: Gatherer Content Processing connector
ContentProcessing
Search Gatherer: Disk Full Crawler
Crawl
Analytics analysis: failed to start warning – search analytics
AnalyticsProcessing
TechNet SCOM knowledge article - http://technet.microsoft.com/en-us/library/ee513082(v=office.15).aspx
QueryContent
WFE
API
Key day-to-day search operations
Search admin and Business users jobsDaily – AdminsCheck search topology UI, any component not green there is a problemMonitor performance reports, crawler health and query health
Weekly – AdminsVerify # of documents per partition and disk space on index, crawler and analytics componentsDR backupSecurity updates + Cumulative updates
Daily – Business userReview usage reports for trends
No Result Queries aka 0 hits
Top Queries by DayAbandoned Queries
Weekly – Business userFor a better search experience create
Query RulesBest betsAuthoritative Pages
Patching and updatesSequence is important!Define your update groups
1st half of components2nd half of componentsAdmin Component
Search admin db Link db
Analytics dbCrawl db
All SharePoint databases All SharePoint databases
Analytics
Replica ReplicaIndex partition 0
Query Processing Query Processing
Crawl
Admin
Content processing
Analytics
Crawl
Admin
Content processing
Application Server Application Server
Application Server Application Server
Group 1 Group 2 Group 3
SP1 and Cumulative UpdatesOffice SP1! Current best practice is to install
At the very least it is recommended to install March PU + Oct CU
Limit Type Old Value New Value
Crawl components Supported 2 / SSA; 1 / server Unlimited*
Crawl databases Boundary 5 per SSA 15 per SSA
Crawl log entries Supported 100 million Unlimited
Index components Supported 60 / SSA; 1 / server 60 / SSA; 4 / server
Index partitions Supported 20 / SSA 25 / SSA
Indexed items Supported 100 M / SSA10 M / indexer
10 M / indexer (250 M / SSA implied)
Search components Supported Not Documented 64 / SSA
Limit Type Value
Content processing components
Supported 1 / server
Content sources Supported 50 / SSA
Index replicas Supported 3 / index partition
Very important unchanged limits
What is your SLA? Cold – Hours to Days
Warm – Minutes to Hours
Hot – Minutes to Seconds
Pros: Cheap…. Maintenance and operational
Cons: Slow to recover but this may be ok for your business
Pros: Fairly inexpensive to recover Cons: expensive and time-consuming to maintain
Cons: Very expensive and time-consuming to maintain
Pros: Fastest recovery for critical apps
IaaS and Disaster Recovery
Cloud ServiceVirtual Network
Windows Azure
SQL DR1(A6)
SP DR1
(Large)
AD1(X-Small)
On Premises
Web servers
Application servers
SQL DR2(A6)
SP DR2
(Large)
SP DR4
(Large)
SP DR5
(Large)
SP DR3
(Large)
Directory servers
VPN Tunel
Queries, Crawler/Content
Replace a failed server (with HA)
Add replacement server to topology
Clone active topologyAdd componentsActivate topology
Wait until added components are green
For index replica: Seeding must be completed
Remove old server from topology
Clone, remove, activate
Before you remove server from the farm!
Verify Host Controller repository status
Needs action if primary host controller was on the failed server
Verify status of primary host controller – in case it was on the failed server> Get-SPEnterpriseSearchHostController
If primary host controller was on the failed server:> Set-SPEnterpriseSearchPrimaryHostController
Demo
Replace a failed server (with HA)
Backup and Restore
Everything but the index is in databases(just about)
Restore possible to a different deploymentMay use B/R for standby DR locationRestore location must have the same basic topology
Ballpark figuresMin: 8 minutes to backup 10K index: 3 nodes, 2GB of data, 6 minutes restoreMax: 8 hours to backup 80M index: 12 nodes, 2TB of data, 6 hours restoreYMMV - test
Backup and RestoreSharePoint CA UI
PowerShell Backup-SPFarm -Directory <BackupFolder> -BackupMethod {Full | Differential} -Item “<SearchServiceApplicationName>”
How does the standard backup work?Consistent across all DBs and indexIndex backup phase 1
Backup the index – indexing paused – feeding continues – content updates persisted to the journal
Index backup phase 2Backup the journal – content updates since phase 1 started – feeding paused
Continued operations during backupQueries continue to work throughout the backupCrawl is paused for ~10% of the overall backup time
Demo
Backup
Survival Guide (TechNet Wiki)http://social.technet.microsoft.com/wiki/contents/articles/14382.survival-guide-sharepoint-2013-and-enterprise-search-sp2013es.aspx
TechNet Documentationhttp://technet.microsoft.com/en-us/library/cc303422(v=office.15).aspx
Related Resources
TechNet contentSearch AdministrationAdminister search in SharePoint Server 2013
Search TopologyScale search for performance and availability in SharePoint Server 2013Technical diagrams for SharePoint 2013Manage the search topology in SharePoint Server 2013Get-SPEnterpriseSearchStatus
Backup/RestoreBack up Search service applications in SharePoint 2013Overview of SharePoint Foundation and the Volume Shadow Copy Service
MySPCSponsored by
connect. reimagine. transform.
Evaluate sessionson MySPC using yourlaptop or mobile device:myspc.sharepointconference.com
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.