SharePoint SpeedMetal [Admin 101](S5A-104)

Admin – 200

Chris McNultyKMA

Welcome to SharePoint Saturday—The Conference

Welcome to SharePoint Saturday—The Conference Welcome to SharePoint Saturday—The Conference

• Please turn off all electronic devices or set them to vibrate.• If you must take a phone call, please do so in the hall so as not

to disturb others.• Open wireless access is available at SSID: SPSTC2011• Feel free to “tweet and blog” during the session• Thanks to our Diamond and Platinum Sponsors:

Thank you for being a part of the first SharePoint Saturday conference

• Founded 1995– 29 employees– 4 partners, including 2 co-founders

• Principal lines of business:• Professional Services

– SharePoint Consulting– SharePoint Managed Services– Custom Application Development

• Software Product– Mekko Graphics advanced charting

software

• Roots in academia (MIT, Harvard, BU)

About KMA

Chris McNulty• KMA SharePoint Practice Lead/Manager• Working with SharePoint technologies since

2000/2001• 20 years consulting and financial services

technology (Santander, John Hancock, GMO, State Street)

• MBA in Investment Management from Boston College Carroll School of Management

• Write and speak often on Microsoft IW technologies (blogs & books)

• Microsoft MCSE/MCTS/MSA/MVTSP• Hiking, cooking, playing guitar, colonial

history, photography• My family: Hayley, three kids (16, 7, 4) and

my dog Stan

• Out Of Scope– Deep Dives (e.g. PowerShell, BI, Upgrade, SQL DBA)– Development– Customization– Design & Architecture– Power User (e.g. Library Customization, Designer Workflows,

etc.)– Office 365

• Rules– Move fast, PowerPoint is shared– Questions – time permitting during session– Any time after session – email etc.

Objectives

• The dilemma• Architecture, Design & Planning• Installation and Upgrade

– Post Installation Best Practices– Service/Feature Placement

• Support– Monitoring and Optimization– Backup– PowerShell– Development Functions– Optimization– Patching– SQL Maintenance

• Best Practices

Agenda

o You’re the new SharePoint Administrator!!!o But…o You’re still responsible for:

• Exchange• Active Directory• SQL• Desktop• Help Desk• Network/Firewall• Cooking & Cleaning• Etc.

Congratulations!

o SharePoint administration is often an ‘add-on’ for other IT professionals (SQL DBAs, AD Admins, Exchange Engineers)

o Time and focus are scarce resources!o Common pain points include

• Upgrades are complex and hard to monitor• Dispersed workforce, little control of browsers and

Office versions• Hard to understand and troubleshoot “behind the

scenes” performance and capacity planning• Best practices not always understood or compared to

system health• “All or nothing” administration means IT must be

engaged for all admin responsibilities, even search

The Dilemma

Microsoft SharePoint Server 2010 … the bright frontier

Eastern Long Island, July 4, 2010

Architecture and Design

• Typical Roles:• http services• Search query

• Scaling• Add servers to load balanced

cluster• Performance Optimization

• RAM• Easily virtualized

Server Farm – Web Front End

14

• Typical Roles:• Search index/crawl• Excel calculation• User profiles• Managed Metadata

• Scaling• Add search servers and

partitions• Move shared services to

dedicated servers• Performance Optimization

• CPU

Server Farm - Application Server

15

• Typical Roles:• Data storage• SQL Reporting

• Scaling• Add storage capacity

• Performance Optimization• Disk I/O

Server Farm - Database

16

• Typical Roles:• Small teams• Small pools of documents

• Considerations• Performance & fault tolerance

less of a concern• SQL & Web on same system• Search not a core function

Sizing - Single Server

• Typical Roles:• 100-10,000 users• 10,000 – 1MM documents

• Scenarios• Enterprise portal• Large scale collaboration• Broader applications platform• Larger external search pool• Mix and match internal external

front end servers on common content databases

Sizing - Medium Farm

• Typical Roles:• Large distributed

enterprise users (10000+)

• Large pools of documents (>1MM)

Sizing - Large Farm

• Farm level• Web applications

• Independent top level URLs

• Run inside IIS pools• Consume shared

services and admin from the farm or other farms

• Site collections • Security, branding,

database frontier• Contain single sites or

site hierarchies• Sites

• Group related SharePoint elements (lists, libraries, pages, web parts)

Top Level Logical Components

20

Web Applications

http://intranet

http://centadmin

Site Collections

Site Hierarchies

Single SItes (MySite)

SitesLists Liibraries Pages Web Parts

• High capacity!• Maximums

• 250,000 sites per site collection

• 5,000 site collections per content DB

• 200GB max content DB (single site collection)

• >200GB post SP1• 300 Content DBs per

web application• 30MM

documents/library• 2GB document size

22

Logical Components

Content Search

Initial Content Size XXX GB External Crawl Size YYY GB

Initial User Pool U

User Collab Size .25GB

n YR Growth Rate – Archive Rate

G%

End Content Size XXX (1+G)n = ECS End Search Size YYY (1+G)n = ESS

End User Collab Size .25 * U * (1+G)n = EUCS

Content DBs ECS + EUCS

Search DBs .05 * (ECS + EUCS + ESS)

Search Index Files .05 * (ECS + EUCS + ESS)

Disk Sizing

• Inputs: Size for SharePoint content and non-SharePoint content included in search

• For DBs, don’t forget to consider transaction logs, disk dumps (if used for backup) which can add 1-3X.

• In SAN or virtual environments, not all disk need be provisioned early

• Web Front End (WFE)– 8GB minimum– 12GB larger– 16GB max

• Application– 8-12GB

• SQL– 8-20GB

• HP Sizing Tool– http://

h71019.www7.hp.com/activeanswers/Secure/548230-0-0-0-121.html

• Don’t forget about the swap files (1-2X size of RAM)

Memory Sizing

• Classic– Zones mapped– Single URL for each

authentication method:

– e.g.• Default: Windows• Forms (LDAP, SQL,

ADAM)

– Generate an NTToken to represent SPUser

• Claims– Unified URL– Multiple sources

(Windows, FBA, SSO) combine to generate a single SAML token to represent SPUser

Authentication Architecture

Regular label-callout text

Multi-AuthenticationMixed Authentication

SharePointFarm

Web Application

Extended Web Application

Extended Web Application

Extended Web Application

Extended Web Application

Zone: Custom

Zone: Extranet

Zone: Intranet

Zone: Internet

Zone: DefaultWindows Authentication

FBAAuthentication

...

...

...

SharePointFarm

Web Application

Extended Web Application

Extended Web Application

Extended Web Application

Extended Web Application

Zone: Custom

Zone: Extranet

Zone: Intranet

Zone: Internet

Zone: DefaultWindows AuthenticationFBA Authentication

SAML Based AuthenticationFBA Authentication

Windows Authentication

...

...

27

• Traditional• Inexpensive• Simple• Only one firewall• External traffic

comes inside internal network

Internet Topology – Edge Firewall

WFE SQLEdge

Firewall AD

Internet

App

Internal Network

• More complex• Duplicative networks,

backup, AD• External traffic is

reserved• Larger server foot

print (exposure) in perimeter

• Internal users need domain trusts

• Internal users access site across firewall

Internet Topology – Perimeter

Router/Firewall

WFE SQLEdge

Firewall AD

Internet

App

Perimeter Network Internal Net

• Most complex• Intricate firewall rules• App, AD and search

roles optionally in perimeter

• Optional internal WFE or internal users always cross a firewall

• Crawl topologies important to avoid overtaxing the firewall

Internet Topology – Split Back to Back

Router/FirewallWFE SQL

EdgeFirewall AD

Internet

App

Perimeter Network Internal Network

• Multi-farm• SSA farm• Content publishing

Internet Topology – Enhanced Techniques

Internal• Active Directory• Exchange / File Shares• Index and integrate BCS data

External• LDAP• Mail Relay• Indexed search content

Other Systems• FAST (Search)• Project Server / TFS• BizTalk• LoB/Dynamics• Oracle (BCS)• Notes (Search)• Wikis and other indexed web sites

Common Integration Touchpoints

• SharePoint 2010 is a 64 bit only platform. Direct upgrades from 32 bit to 64 bit requires prep work.

• Windows Server 2008 or Windows Server 2008 R2 X64

• SQL Server 2005 x64 SP3 CU3 Or

• SQL Server 2008 x64 SP1 CU2Or

• SQL Server 2008 R2

Platform Basics

• 2007 Shared Services Provider has been broken up; each of its elements is now a Shared Service Application

• Mix and match them singly or in groups, to match farm’s needs.

• Crawl/index no longer a single server role

• In 2010, administration can be delegated– Key targets: Enterprise

search, metadata, user profiles

Shared Service Applications

http://globalweb http://itportal

Visio

Search

Excel Calc

Metadata

User Profiles

• Internet Explorer 7/8/9, Firefox and Safari are all supported.

• Some support for Chrome• IE6 is not supported• Most other browsers are still supported for

Internet configurations• Office 2010 includes optimizations for the new

platforms• Offline Access

– 2007: used Outlook 2007 and Groove– SharePoint Workspace 2010 integrates offline

documents and lists

Client/Browser Technology

• SharePoint 2010 provides a server version of Office applications – Office Web Access, or “OWA”.

• In part, this enables simultaneous multiuser editing of Office documents:– Excel in OWA, not client– Word/PowerPoint on

client only if file opened from a shared document library

– OneNote client or OWA

Office Web Applications

Installation and Upgrade

• Servers:– Windows 2008 R2 X64 Enterprise Edition– SQL Server 2008 R2 x64

• Service Accounts– spfarm (Farm acct; local admin on the SharePoint servers and either sa

or dbcreate, dbowner and security admin on the SQL server.)– svcsql (SQL Server service acct)– sppool (IIS pool acct)– spcrawl (Search accts)– spadmin Interactive admin (install account; local, site collection and

farm admin privileges)• Install as SPAdmin • Install Software Prerequisites - Checks for following elements:

– Application Server Role, Web Server (IIS) Role, Microsoft SQL Server 2008 Native Client, Hotfix for Microsoft Windows (KB976462), Windows Identity Foundation (KB974405), Microsoft Sync Framework Runtime v1.0 (x64), Microsoft Chart Controls for Microsoft .NET Framework 3.5, Microsoft Filter Pack 2.0, Microsoft SQL Server 2008 Analysis Services ADOMD.NET, Microsoft Server Speech Platform Runtime (x64), Microsoft Server Speech Recognition Language - TELE(en-US), SQL 2008 R2 Reporting Services SharePoint 2010 Add-in

Installation - Prerequisites

• Initial– Product Key– Type of installation - Always SERVER

FARM– Installation Type - Complete [Not

Single Server]– Accept default file locations – index

files will stay on C:\Program Files\Microsoft Office Servers\14.0\Data

– At end NO Wizard– Run OWA Setup– Then, WIZARD! The wizard starts,

and yes, it’s OK for IIS to reset during the wizard…

• Create a new farm– Set farm account– Pick configuration database,

Passphrase, CentralAdmin Port (Conventions)

– Final confirm and let the wizard run

Installation

• Pros– Easy– All SSAs Configured– Saves time and PowerShell

hand tooling of SSAs

• Cons– My Sites setup in same app and

DB as primary– Database Names are default,

GUID happy– Kills kittens (ask SharePoint

911!)

• What it does– Sets up service acct for SSAs

and other services (sppool)– Sets up a port 80 web app with

a My Sites Host sub-site collection in WSS_Content database

Installation – Farm Config Wizard?

• Three paths– In place

– Database upgrade

– Third party tools

• Process– Pre-upgrade checker

– Visual Upgrade

– Resumable upgrade

– Progress reports

– Parallel DB upgrades

Predictable Upgrade

• Additional Prepwork– Content pruning

– Database alignment • stsadm-o

mergecontentdbs

• DB Attach– Preinstall Required

Features

– Stsadm –o addcontentdb –databasename DBNAME –url URL –assignnewdatabaseid

– PowerShell Mount-ContentDatabase

– Test, test, test!

Upgrade Preparation

• SharePoint 2007 SP2 minimum, October 2009 CU best

– STSADM.exe –o preupgradecheck

• Documentation– All servers and components in the farm,

and whether the servers meet 64-bit hardware/OS requirements

– Alternate access mapping URLs– A list of all site definitions, site templates,

features, and language packs that are installed in the farm.

– Unsupported farm customizations (such as database schema modifications).

– Database or site orphans – Missing or invalid configuration settings in

the farm (missing Web.config file, invalid host names, invalid service accts).

– Whether the databases meet the requirements — for example, databases are set to read/write, and any databases stored in Windows Internal Database and larger than 4 GB.

Pre-Upgrade Check

Upgrade servers without changing the user interface

Switch-on new UI across site collections in a controlled manner

Pre-release screenshots, subject to change

Preview new UI

IT Pro Investments – Visual Upgrade

DEMO• Upgrade

(48)Copyright 2011 © Knowledge Management Associates, LLC. Twitter hashtag: #sptechcon

MonitoringMonitoring and Operations

• Developer Dashboard– Empower developers

and users

• Integrated Health Analyzer– Runs when necessary– Alerts anomalies – Fixes when it can

• Web Analytics– User usage– Resource usage

Proactive Issue Resolution

• Unified Logging• Out-of-the-box

reports• Richer Web Analytics• Open Schema• SCOM Integration• PLUS

– Developer Dashboard

– Health Analyzer

Logging, Monitoring, and AlertsKnow what is going on

ULS Logs

Windows Events

Page requests

Feature Logging

Health data

Logging DB

• Monitoring– SCOM– Central Admin

• Health Analyzer– Site Collection Web Analytics– Developer Dashboard

• stsadm –o setproperty –pn developer-dashboard –pv OnDemand

• (Get-SPFarm).PerformanceMonitor.DeveloperDashboardLevel = "OnDemand"

– Troubleshooting• Correlation ID – One GUID to rule them all!• ULS Logs, Event Logs, Performance Monitor

– OR• WSS_Logging DB

Monitoring – General

Developer Dashboard Improve customized solutions with the Developer Dashboard

Pre-release screenshots, subject to change

SQL Queries Performance

Memory Usage

Data-Request Trace

• Query Database Views Directly• Requires Timer Jobs Enabled

– Diagnostic Data Provider: Trace Log

– Diagnostic Data Provider: Event Log

• ULS Configuration Matters• Database will GROW!• Aggregates from ALL Servers• Sample:

– SELECT * FROM [WSS_Logging].[dbo].[ULSTraceLog] WHERE CorrelationID = '04377DAE-C2FD-4DBE-A57E-101B3005059E'

Monitoring – WSS_Logging

• Third Party Tools• Recycle Bin• Granular / Site Collection Backup (UI)

– *.bak file– Restore-SPSite

• Unattached Recovery– Browse unattached content database– Account needs DB permissions– Database need not be on the same server!– No more granular than list or library!– Browse Content

• Export Site or List

– Export as a CMP file– PowerShell restore

• PS: Import-SPWeb http://msshome2010 –Path C:\ListRecovery.cmp

• SQL Backup• SharePoint Backup (UI or script)

Backup/Recovery

@echo offecho ==================================================echo Backup the farmecho ==================================================@SET stsadm="C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\BIN\stsadm"rmdir /S /Q "\\spsql08\spbackup\farmold"ren "\\spsql08\spbackup\farm" "farmold" md "\\spsql08\spbackup\farm"%stsadm% -o backup -directory "\\spsql08\spbackup\farm" -backupmethod fullecho complete

STSADM Backup

# NOT NEEDED write-outputwrite-host ==================================================write-host Backup the farmwrite-host ==================================================Add-PSSnapIn Microsoft.SharePoint.PowershellRemove-Item -Path "C:\PSBackup\farmold“ -recurseRename-Item -Path "C:\PSBackup\farm" -NewName "farmold" New-Item -type directory -path C:\PSBackup\farmBackup-SPFarm -directory "C:\PSBackup\farm" -backupmethod full –verbose –percentage 5Write-host Backup complete

PowerShell Backup

• SharePoint Shell vs. Base Shell– Add-PSSnapin Microsoft.Sharepoint.Powershell

• Command -?– Get-Help Command– Get-Help Command –examples

• Pipe– Get-Command –Noun SP*– Get-Command – Noun SP* | Select Name– Get-Command – Noun SP* | Select Name | Out-File

Commands.txt

• Get-SPSite –limit all | Get-SPWeb –limit all | Select URL, webtemplate | Out-GridView– WindowsPowerShell Integrated Scripting Environment to

allow Out-GridView

PowerShell

• Visual Update a range of sites:

• Site Backup• Add MMS Term

Some Useful PowerShell Snippets

$webapp = Get-SPWebApplication http://sitenameforeach ($s in $webapp.sites){$s.VisualUpgradeWebs() }

$str = “SAMPLE”$site = new-object Microsoft.SharePoint.SPSite("http://MYSITE")$session = new-object Microsoft.SharePoint.Taxonomy.TaxonomySession($site)$termstore = $session.TermStores[“MYTERMSTORE"]

[…create group…][…create term set…]

$term = $termset.CreateTerm($str, 1033)

• Create and configure a library

Some Useful PowerShell Snippets II

#Load the Sharepoint .net Assembly[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint") #set the url of the site collection to a variable$siteurl = "http://msshome2010/"$subsitename = "Marketing"$newlibraryname = "NewLib"$newlibrarydesc = "NewLib Description" #create the new object passing the site collection URL, attach subsite$mysite=new-object Microsoft.SharePoint.SPSite($siteurl)$subsite = $mysite.openweb($subsitename) #make the new library - 101 is the generic for DocumentLibrary template$subsite.lists.add($newlibraryname ,$newlibrarydesc, 101)  #open the new library and break inheritance$mylib = $subsite.lists[$newlibraryname]$mylib.BreakRoleInheritance($false)

Development Support – Three Regions

Development

• often internal to developers• problem reproduction that

require advanced inspection tools (e.g. Visual Studio) are done here

• permissions can be looser, may have multiple environments for multiple developers

• sensitive data from production cannot be copied here without masking or customer signoff

• changes here can be deployed ad hoc

Staging/Test

• no Visual Studio, no MS Office• match/mirror production as

closely as possible; match hardware/system performance as closely as practical

• security permissions match production

• any sensitive data copied here stays under production-grade controls

• test accounts should be created in a separate OU if possible

• changes here can only be delivered and deployed from source control and according to production release methods

Production

• optimized hardware configurations

• highly secure• no use of user rotating

password accounts as service accounts

• changes here can only be delivered and deployed from source control and according to production release methods

DEMO• Monitoring

• Developer Dashboard• Health Analyzer

• PowerShell

(67)Copyright 2011 © Knowledge Management Associates, LLC. Twitter hashtag: #sptechcon

OptimizationOptimization

• Disk-based BLOB Caching– Local store for audio/video, PDF other frequent

read only files– Edit in Web.config (C:\Inetpub\wwwroot\wss\

virtualdirectories\...)– <BlobCache location="" path="\.(gif|jpg|jpeg|jpe|

jfif|bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled="false" />

• Location = Local Disk Location• maxSize = GB• Enabled = true

• Different from RBS/EBS!• Find Sean McDonough

Optimization

Patching – High Level Process

Patch

• SharePoint Foundation

Patch

• SharePoint Server

Deploy

• Run SharePoint Products and Technologies Wizard• (Or psconfig)

• Sequential Application to Central Admin, Application Server(s), Web Front End Servers

• Backups– Local Disk – easy but storage intensive– Agents – remote, requires extra software

• RBS Maintenance– BLOB Orphans

• Log Sizing– Full logged (default) generates huge t-logs– Simple doesn’t but prevents point in time restore

• Maintenance Plans

SQL Maintenance

Best Practices

• Users Receive “Cannot Connect to Configuration Database” Web Page– SharePoint farm account is locked out

• No one can upload anything but site is up– Database disk volume is full – check transaction logs,

backups– In virtualized environment, host file systems may be full

• I can’t find a document I think I should see; Someone can’t see a file I just uploaded– Security and permission variations– Document “movement” (a/k/a ECM) try search by name

or Document ID. Check ECM logs/audits– Confirm permissions, and make sure document is

checked in (Required properties may be missing)

Troubleshooting – Top Support Questions

• Repeated requests to re-enter Windows credentials– Add to Local intranet zone, add site, custom level,

automatic login with current user name and password (it’s the last thing in the item list)

– OR Trusted sites

• My workflow didn’t start– Recycle timer service– “FixSharePoint.exe” = IISReset & Timer Service Recycle

• I’m not seeing the right search results– Confirm that crawls are running and complete by checking

crawl logs; restart a full crawl if crawls finish OK

• I need a file back that I deleted– Recycle Bin Recovery– Use Backup & Restore

Troubleshooting – More Support Questions

Where Governance Begins?

Portal

Dept Sites

Project/Team Sites

Personal / My Sites

Hig

her

Vis

ibili

ty

Lighte

r govern

ance

Seven Deadly Sins for the SharePoint Professional

• No SQL maintenance plans• Default names for every database

(WSS_CONTENT_12345abc…)• No patching• One environment for everything• One acct for everything• Single server install with SQL Express• Runaway content database size

Seven Deadly Sins

Seven Virtues for the SharePoint Professional

• Security Applied via AD Groups and SharePoint Groups

• Review System Health• Test Restore and Recovery• Monthly Web Analytics Review – Usage,

Storage, Search• PowerShell instead of STSADM• Governance• Use ECM, MMS, Clients, Archiving and Training

to Keep Content in SharePoint, reduce accidental duplication and keep searching and browsing fresh

Seven SharePoint Virtues

o You’re the new SharePoint Administrator!!!

o And nowo You understand:

• Design and Architecture• Installation and Upgrade• Support and

Maintenance and Optimization

• PowerShell• Customizations• Troubleshooting• Best Practices• People from New York?

Congratulations!

• From Microsoft:– SharePoint 2010 site: http://sharepoint.microsoft.com– SharePoint Team Blog: http://blogs.msdn.com/sharepoint/default.aspx

– From KMA– Webinars, white papers and blogs at www.kma-llc.net

• From Me:– SharePoint Saturday the Conference

• Saturday 9:30am S1A-101 - A decade of SharePoint Adoption Best Practices• Saturday 11:30am S2B-104 - See Beyond The Numbers: Data Visualization & BI in SharePoint 2010• Saturday 3:00pm S4A-101 Playing Tag - Managed Metadata and Taxonomies in SharePoint 2010

– New Jersey SharePoint User Group September 14 (Business Intelligence)– SharePoint Saturday NH – September 24, 2011 (Business Intelligence)– KMWorld DC October 2011 (IT)

Resources

• Questions?• Evaluations• Contact Me• Prizes! Follow @kmallc

for the code word!

• Email cmcnulty@kma-llc.net• Blog http://blogs.kma-llc.net/microknowledge• Twitter: http://twitter.com/cmcnulty2000• LinkedIn:http://www.linkedin.com/in/cmcnulty

Thank you…

Welcome to SharePoint Saturday—The Conference

Thanks to Our Other Sponsors!Thanks to our Sponsors

Welcome to SharePoint Saturday—The Conference

Session EvaluationPlease complete and turn in your Session Evaluation Form so we can improve future events. Survey can

be filled out at:

http://app.fluidsurveys.com/s/spstc2011-Fri-S5A-104

Presenter: Chris McNultySession Name: SharePoint SpeedMetal [75 Minute Admin]Session No.: S5A-104