Upload
phungkiet
View
215
Download
0
Embed Size (px)
Citation preview
EDUCATION
SAS OverviewManu GuptaPatni Computer Systems
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 2
SNIA Legal Notice
• The material contained in this tutorial is copyrighted by the SNIA.
• Member companies and individuals may use this material in presentations and literature under the following conditions:– Any slide or slides used must be reproduced without
modification– The SNIA must be acknowledged as source of any
material used in the body of any document containing material from these presentations.
• This presentation is a project of the SNIA Education Committee.
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 3
Abstract
This presentation explores the various aspects of SAS Architecture and SAS Transport Protocols. It elaborates the SAS Network Topology with SAS device discovery process. It depicts how scalable storage systems can be build using SAS. This presentation is intended for Technical Architects, OEMs, Storage Protocol Stack Developers and IT Managers
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 4
SAS Standard Layers
SAS Port Layer
SSP Link Layer STP Link Layer SMP Link Layer
SSP Transport Layer
STP Transport Layer
SMP Transport Layer
SCSI Application Layer
ATA Application Layer
Management Layer
SAS PHY Layer
SAS Physical Layer
SAS Link Layer
Cables & Connectors
Encoding & Speed Negotiation
Connection Management
Port Management
Protocol Frame Definitions
SCSI Operations\ SAS Management\ ATA Operations
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 5
SAS Network Topology
Fan out Expander Device
Edge Expander Device Edge Expander Device
End Device
End Device
End Device
End Device
Initiator Target Target Target
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 6
SAS Terminology
• PHY• Physical Links• SAS Address• Port• SAS Device (Initiator & Target)• Expander Device
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 7
PHY Attributes
• Transmitter & Receiver
PHY PHY
Transmitter
Receiver
Transmitter
Receiver
Physical Link
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 8
PHY Attributes
• Physical Link– Set of four wires used as two differential signal pairs– Physical Link between two PHYs
• Full Duplex Operation– Transmitter and Receiver operates at same physical
link rate• SAS Address inherited from SAS Port• PHY Identifier unique within device• SAS uses 8b/10b encoding
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 9
SAS Address
• Each SAS Port has worldwide unique 64-bit SAS Address.
• 24-bit Company ID assigned by IEEE• 36-bit Vendor Specific ID assigned by organization
associated with Company ID
Byte/Bit 7 6 5 4 3 2 1 0 0 NAA (5h) 1 2 IEEE COMPANY ID 3 4 5 6 7
VENDOR SPECIFIC IDENTIFIER
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 10
SAS Port & SAS Device
• Port contains one or more PHYs– Group of PHYs having same SAS Address– Narrow Port
• If there is only one PHY in the Port– Wide Port
• If there are more than one PHY in the Port• SAS Device contains one or more ports• Connections are made between PHYs but
addressed to Ports
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 11
SAS Port & SAS Device
PHYPHY
Device Device
Narrow Port Narrow Port
PHYPHY
PHYPHY
Device Device
Wide Port Wide Port
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 12
Identification Sequence
• Each PHY transmits IDENTIFY address frame– SAS Address of Port– Device Type– PHY Identifier– Role/Capabilities
• Each PHY receives same set of information from peer PHY in IDENTIFY address frame
PHYPHY
PHY
DeviceDevice
Wide Port
Narrow Port
PHY
Device
Narrow Port
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 13
Expander Device
• Facilitates communication between SAS Devices
• Can have one or more PHYs• Each PHY has same SAS address• Expander ports are created based on attached
SAS devices• Each PHY in an expander had “Routing
Attributes”• Expanders may contain devices for enclosure
management
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 14
Routing Attributes
• Direct – routes connection requests to attached end devices
• Table – routes connection requests to attached expander devices using Route Table
• Subtractive – routes unresolved connection requests to attached expander device– An expander device can have at most one defined
port for subtractive routing
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 15
Routing Attributes
End Device
End Device
End Device
End Device
D
Expander Device
Expander Device
D D
D
SS
T T
End Device
End Device
D
D
Expander Device
T T
End Device
End Device
Expander Device
D
D
End Device
End Device
Expander Device
D D
D – Direct RoutingT – Table RoutingS – Subtractive Routing
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 16
Route Table
• An association between SAS Destination Address and expander PHY to forward connection request
PHY-based Route Table Expander-based Route Table
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 17
Expander Devices
• Fan-out Expander Devices– Expanders without subtractive routing PHYs– Uses table routing when connected to other expander devices
• Edge Expander Devices– All subtractive PHYs must be connected to PHYs with same
SAS address
• Edge Expander Device Set– Group of one or more edge expander devices– Table Routing is used to connect to Expander devices within
Expander device set– Subtractive Routing is used to connect to other expander device
set or Fan-out Expander Device
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 18
SAS Domain
• Contains SAS Devices and Expander Devices
SAS DomainSAS Domain
SAS Initiator Device
SAS Initiator Device SAS Initiator Device SAS Initiator Device
SAS Target Device SAS Target Device SAS Target Device SAS Target Device
Initiator Port
Initiator Port
Initiator Port
Initiator Port
Initiator Port
Initiator Port
Target Port
Target Port
Target Port
Target Port
Target Port
Target Port
Expander devices Expander devices
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 19
SAS Domain - Rules
Expander Device Y
Expander Device ZSAS Device A
SAS Device B
SAS Device C
No Multipaths
Expanded Device Y
Expanded Device ZSAS Device A
SAS Device B
SAS Device CExpander Device X
No Loops
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 20
Pathways & Connections
• Potential Pathway – set of physical links between SAS Initiator PHY and SAS Target PHY
SAS PHY
SAS PHY
Directly Attached
SAS PHY
SAS PHY
Attached through Expander
SAS PHY
SAS PHYExpander
PHYExpander
PHY
Expander PHY
Expander PHY
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 21
Pathways & Connections
• Pathway – set of physical link between SAS Initiator PHY and SAS Target PHY being used by connection
• Partial Pathway – set of physical links participating in connection request (not yet established)
• Connection – temporary association between SAS Initiator Port and SAS Target Port– Only one connection can be active on physical link at a given
time– Number of connection established by a SAS Port is limited to
no of PHYs in given SAS Port– SAS Initiator Wide Port can have multiple connection with
different SAS Target Ports– SAS Target Wide Port can have multiple connection with
different SAS Initiator Ports
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 22
Connection Example
SAS Target Device
PHY
SAS Target Port
B
SAS Target Port
PHY
PHY
A
C
PHY
SAS Target Port
D
SAS Initiator Device
SAS Initiator Port
PHY
PHY
C
D
SAS Initiator Port
PHY
PHYB
A
Expander Device
Expander Port
Expander PHY
Expander Port
Expander PHY
Expander PHY
Expander Port
Expander PHY
Expander PHY
Expander Port
Expander PHY
Expander PHY
Expander Port
Expander PHY
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 23
SAS Management Protocol -SMP
• SMP – Protocol used by SMP Initiator port to communicate with SMP target in SAS Domain
• SMP Request Frame –Initiator to Target
• SMP Response Frame –Target to Initiator
Byte/Bit 7 6 5 4 3 2 1 0
0 SMP FRAME TYPE (40h)
1 FUNCTION
2 RESERVED
3 REQUEST LENGTH
4 to (n-3) ADDITIONAL REQUEST BYTES
(n-3) to n CRC
CRC(n-3) to n
ADDITIONAL RESPONSE BYTES4 to (n-3)
RESPONSE LENGTH3
FUNCTION RESULT2
FUNCTION1
SMP FRAME TYPE (41h)0
01234567Byte/Bit
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 24
SMP FunctionsCode SMP Function
00h REPORT GENERAL
01h REPORT MANUFACTURER INFORMATION
02h READ GPIO REGISTER
03h REPORT SELF-CONFIGURATION STATUS
04h REPORT ZONE PERMISSION
10h DISCOVER
11h REPORT PHY ERROR LOG
12h REPORT PHY SATA
13h REPORT ROUTE INFORMATION
14h REPORT PHY EVENT INFORMATION
15h REPORT PHY BROADCAST COUNTS
16h DISCOVER LIST
17h REPORT EXPANDER ROUTE TABLE
80h CONFIGURE GENERAL
81h ENABLE DISABLE ZONING
82h WRITE GPIO REGISTER
85h ZONED BROADCAST
86h ZONE LOCK
87h ZONE ACTIVATE
88h ZONE UNLOCK
8Ah CONFIGURE ZONE PHY INFORMATION
8Bh CONFIGURE ZONE PERMISSION TABLE
90h CONFIGURE ROUTE INFORMATION
91h PHY CONTROL
92h PHY TEST FUNCTION
93h CONFIGURE PHY EVENT INFORMATION
C0h - FFh VENDOR SPECIFIC
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 25
SMP – Response Frame
Byte/Bit 7 6 5 4 3 2 1 0
0 SMP FRAME TYPE (41h)
1 FUNCTION (00h)
2 FUNCTION RESULT
3 RESPONSE LENGTH
…. …..
6 – 7 EXPANDER ROUTE INDEXES
8 ….
9 NUMBER OF PHYS
48 - 51 CRC
REPORT MANUFACTURER INFORMATION RESPONSE
REPORT GENERAL RESPONSE
CRC40 - 43
ROUTED SAS ADDRESS16 – 23
…..…..
PHY IDENTIFIER9
….8
EXPANDER ROUTE INDEX6 – 7
…..….
RESPONSE LENGTH3
FUNCTION RESULT2
FUNCTION (13h)1
SMP FRAME TYPE (41h)0
01234567Byte/Bit
CRC60 - 63
…......
PRODUCT IDENTIFICATION20 – 35
VENDOR IDENTIFICATION12 – 19
…..….
RESPONSE LENGTH3
FUNCTION RESULT2
FUNCTION (01h)1
SMP FRAME TYPE (41h)0
01234567Byte/Bit
REPORT ROUTE INFORMATION RESPONSE
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 26
Serial SCSI Protocol - SSP
• Full Duplex Protocol• All SSP Frames are
acknowledged by ACK/NAK
• ACK/NAK must arrive within 1ms
• There is only one frame definition in SSP
Byte/Bit 7 6 5 4 3 2 1 0
0 FRAME TYPE
1 – 3 HASHED DESTINATION SAS ADDRESS
4 RESERVED
5 – 7 HASHED SOURCE SAS ADDRESS
8 – 9 RESERVED
10 RESERVED RETRY RETRANSMIT CDP
11 RESERVED NO OF FILL BYTES
12 – 15 RESERVED
16 – 17 TAG
18 – 19 TARGET PORT TRANSFER TAG
20 – 23 DATA OFFSET
24 - m INFORMATION UNIT
m – (n-3) FILL BYTES
(n- 3) – n CRC
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 27
SSP
• Information Unit
CodeFrame Originator
Unit Size (Bytes)Interlock Requirement
01h DATA (READ/WRITE) INITIATOR OR TARGET 1 to 1024 NON-INTELOCKED
05h XFER_READY TARGET 12 INTELOCKED
06h COMMAND INITIATOR 28 to 284 INTELOCKED
07h RESPONSE TARGET 24 to 1024 INTELOCKED
16h TASK MANAGEMENT INITIATOR 28 INTELOCKED
F0h – FFh VENDOR SPECIFICS
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 28
SSP
Transmitter Receiver
Time Time
Interlocked Frame
Interlocked Frame
ACK / NAK
ACK / NAK
Interlocked Frame Transmission Sequence
Non-Interlocked Frame
Transmitter Receiver
Time Time
Non-Interlocked Frame
ACK / NAKACK / NAK
Non-Interlocked Frame
Non-Interlocked Frame
ACK / NAKACK / NAK
Non-Interlocked Frame Transmission Sequence
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 29
SSP
Send SCSI Write Command
SSP Initiator Port SSP Target Port
Time Time
Receive Data out
SCSI Write Command receivedCOMMAND Frame
XFER_READY Frame
WRITE DATA Frame
RESPONSE Frame
Data out received
Send command completeCommand complete received
Send Data Frame
Send SCSI Read Command
SSP Initiator Port SSP Target Port
Time Time
SCSI Read Command receivedCOMMAND Frame
READ Data Frame
RESPONSE Frame
Data in delivered
Send command completeCommand complete received
Send Data in
Write Command Sequence
Read Command Sequence
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 30
SATA & STP
• SATA devices are not aware of multiple hosts– Flow Control is Host-to-device
• STP Frame transmission is defined by SATA (ATA/ATAPI-7 V3)
• STP encapsulates SATA Frames with connection management
• SATA devices tunnel SATA frames over SAS for transport as if they are directly connected
• STP flow control is point-to-point• Expanders require SATA/STP bridge • Set of primitives are defined for STP
EDUCATION
SAS Overview© 2007 Storage Networking Industry Association. All Rights Reserved. 31
Q&A / Feedback• Please send any questions or comments on this presentation to
SNIA: [email protected]
Many thanks to the following individuals for their contributions to this tutorial.
SNIA Education Committee
Howard Goldstein