Upload
odanyboy
View
363
Download
3
Tags:
Embed Size (px)
Citation preview
© 2009 BMC Educational Services
Discovery Overview
Getting Data from the Estate
© 2010 BMC Educational Services
Outline
The Basic Philosophy First Scan Second Scan Optimization Bringing It All Together
Completed State Charts Summary
© 2009 BMC Educational Services
The Basic Philosophy
© 2010 BMC Educational Services
Basic Discovery Sequence (1)
We start a scan of a collection of IPs
We try to contact each one – we record success/failure
We try to establish what sort of device is on the IP from a combination of heuristics and direct access and record that
Discovery
Run
Discovery
Access
Device
Info
Discovery
Run
Discovery
Run
Discovery
Access
© 2010 BMC Educational Services
Basic Discovery Sequence (2)
If the device is a Host and we were able to log on we try and get more information about it and it’s list of interfaces
If we get the Host Information and Interface List then we have enough to try to infer a host, and we carry on with our standard set of discovery
Discovery
Run
Discovery
Access
Device
Info
Host
Info
Interface
List
Discovery
Run
Discovery
Access
Device
Info
Host
Info
Interface
List
Host
Processes
© 2010 BMC Educational Services
Basic Discovery Sequence (3)
Once we have finished standard discovery we can infer other hardware items like Network Interfaces and link the Host to global nodes like subnet
At this stage in the sequence we will start inferring software products, correlating virtual hosts with their physical hosts etc
Discovery
Run
Discovery
Access
Device
Info
Host
Info
Interface
List
Process
List
Host
Discovered
Network
Interfaces
NIC
Subnet
© 2010 BMC Educational Services
Basic Discovery Sequence (4)
Although our standard discovery has finished, we still have the ability to request more information if we need it
This could be the pattern to create software looking for further data or a platform related pattern looking for additional information to do with the Host
Additional discovery will be done via commands, files, registry queries, WMI queries, SNMP queries, SQL queries…
Discovery
Run
Discovery
Access
Device
Information
Host
Information
Interface
List
Process
List
Host
Oracle
Discovered
File
© 2009 BMC Educational Services
First Scan Details
© 2010 BMC Educational Services
We will build up the following Discovery Access state chart:
The DiscoveryAccess node contains 3 key summary attributes to record what happened during session establishment:
ShouldAccess/ SHOULDACCESS
Excluded
/ DONTACCESS
DeviceIdentified/ DEVICE
NoResponse
/ NODEVICE
HostFound/ HOST
UnsupportedDevice
/ NOTHOST
HostInferred/ GOTACCESS
NoAccess
/ NOACCESS
Opt1stScan
/ DONTACCESS
GoodAccess/ SHOULDACCESS
OptNotBestIP
/ DONTACCESS
OptAlreadyProcessing
/ DONTACCESS
Skipped
NoAccess
Success
NoResponse
Error
/ TRACEBACK
Error
OptRemote
Result attribute contents
/ DONTACCESS
Discovery Walk Through
result [ Success | NoAccess | Skipped | NoResponse ]
end_state [ From state diagram ]
reason [ Free text summary reason for lack of success ]
© 2010 BMC Educational Services
Is Access Allowed?
Check to see if we are allowed to access the endpoint
result = Skipped
end_state = Excluded
Excluded
/ DONTACCESS
© 2010 BMC Educational Services
Ping Response?
Pings the endpoint to see if anything responds
© 2010 BMC Educational Services
Check For Open Ports
If something responds then we see if any of the ports we can use are open
© 2010 BMC Educational Services
Ports
Ports we can use UNIX
22 SSH 23 Telnet 513 rlogin
Windows 135 RPC
SNMP 161 (UDP) SNMP
External OS Detection Only 4 (closed port for IP fingerprint) 80 HTTP
© 2010 BMC Educational Services
Dark Space
If we have not got a response at this point we regard this endpoint as Dark Space
end_state = NoResponse result = NoResponse
ShouldAccess/ SHOULDACCESS
NoResponse
/ NODEVICE
© 2010 BMC Educational Services
Credential Vault
We look in the vault for a credential that matches: The IP of the endpoint The service ports seen open on
the device (SSH, Telnet..)
© 2010 BMC Educational Services
Credential Selection
We try to establish a session with each one of the credentials that match
Used in the order they are defined in the vault (as seen on the credential page)
© 2010 BMC Educational Services
UNIX Access
We try the UNIX access methods first
Only tried if we found a UNIX port open
If we get a response we ask the device what it is
© 2010 BMC Educational Services
Windows Access
If the UNIX methods don’t get a result then we try the Windows methods
Only tried if we found a Windows port open
Slaves are used in the order defined
If we get a response we ask the device what it is
© 2010 BMC Educational Services
SNMP Access
If the UNIX and Windows methods don’t get a result then we try the SNMP methods
Only tried if we found the SNMP port open
If we get a response we ask the device what it is
© 2010 BMC Educational Services
Other Attempts
If no access methods have worked we try to determine what the device might be from external evidence SNMP SysDescr IP Stack Fingerprinting Telnet Banner HTTP HEAD
© 2010 BMC Educational Services
Host Classification
All the results so far go through the OS/Device classifier
If the device is a “Host” we will continue
Otherwise we skip this endpoint
end_state = UnsupportedDevice
result = skipped
ShouldAccess/ SHOULDACCESS
DeviceIdentified/ DEVICE
HostFound/ HOST
UnsupportedDevice
/ NOTHOST
© 2010 BMC Educational Services
Once We Determine a Host…
Now we know it is a Host we return to the session and ask for getHostInfo getInterfaceList
Both these are critical for running the Host Identification algorithm
© 2010 BMC Educational Services
Access Failure
If these methods fail to complete then discovery stops here
end_state = NoAccess result = NoAccess
ShouldAccess/ SHOULDACCESS
DeviceIdentified/ DEVICE
HostFound/ HOST
NoAccess
/ NOACCESS
© 2010 BMC Educational Services
Host Identity Algorithm
Host Algorithm uses strong identity properties to compute a Host.key OS, Kernel, MAC, IP, Serial,…
A new Host is created
end_state = GoodAccess result = Success
ShouldAccess/ SHOULDACCESS
DeviceIdentified/ DEVICE
HostFound/ HOST
HostInferred/ GOTACCESS
GoodAccess/ SHOULDACCESS
© 2010 BMC Educational Services
Credential Caching
We cache the successfully used credential and slave for use next time
© 2010 BMC Educational Services
Further Discovery
Standard discovery continues collecting Processes Packages Etc
TPL based discovery starts after Standard discovery
No further change to end_state
© 2009 BMC Educational Services
Second Scan Details
© 2010 BMC Educational Services
Is Access Allowed?
Check to see if we allowed to access the endpoint
result = Skipped end_state = Excluded, end_state =
OptNotBestIP or end_state = OptRemote
Excluded
/ DONTACCESS
OptNotBestIP
/ DONTACCESS
OptRemote
/ DONTACCESS
© 2010 BMC Educational Services
Cached Credential
We check to see if there are cached results from the previous access to this endpoint
© 2010 BMC Educational Services
Use Last Slave
We try the previous credential/slave to see if we make contact
This shortcuts establishing a session
© 2010 BMC Educational Services
Does This Cached Attempt Succeed?
If we do not re-establish a session we have to go back to the full analysis
If we do succeed we start at the OS / Device classifier
© 2010 BMC Educational Services
Back to the Standard Tasks…
OS/Device classifier
HostInfo/InterfaceList
Host Algorithm
© 2009 BMC Educational Services
Optimization Details
© 2010 BMC Educational Services
Optimize – Skipped Endpoints
In order to maximise throughput and reduce load on the targets there are a series of optimisations
These can result in skipped endpoints
© 2010 BMC Educational Services
Duplicate IP in Progress
To prevent collisions if an IP is already in progress duplicates are dropped
result = Skipped end_state = OptAlreadyProcessing
OptAlreadyProcessing
/ DONTACCESS
© 2010 BMC Educational Services
Best IP
Many Hosts have more than one active interface
Many endpoints in a range that relate to the same Host
No point scanning the same Host 12 times in one range
© 2010 BMC Educational Services
Best IP - Aims
Scan over a single endpoint Try and keep the single
endpoint chosen stable over time
Minimize network access
© 2010 BMC Educational Services
Best IP – 1st Scan Optimization
Kicks in when we try to update an existing Host
If the Host has already been updated by an endpoint reaching the GoodAccess end_state discovery stops
result = Skipped end_state = Opt1stScan
ShouldAccess/ SHOULDACCESS
DeviceIdentified/ DEVICE
HostFound/ HOST
HostInferred/ GOTACCESS
Opt1stScan
/ DONTACCESS
© 2010 BMC Educational Services
Multiple IP – 2nd Scan Optimization
Kicks in when we scan the same endpoint again
We check to see if it was optimised last time, if it was we assume it will be again
Discovery stops here
result = Skipped end_state = OptNotBestIP
OptNotBestIP
/ DONTACCESS
© 2010 BMC Educational Services
Multiple IP – 2nd Scan Optimization
We don’t assume the optimisation is correct forever
Every so often we will contact the Host again to confirm If it turns out the IP is new
Host full discovery will occur Otherwise you will get 1st Scan
Optimization again
ShouldAccess/ SHOULDACCESS
DeviceIdentified/ DEVICE
HostFound/ HOST
HostInferred/ GOTACCESS
Opt1stScan
/ DONTACCESS
© 2009 BMC Educational Services
Bringing It All Together
© 2010 BMC Educational Services
Errors
If an error occurs the result will be set Error
The end_state will be set to the last state reached, or Error if it occurs before any other state
Error
/ TRACEBACK
© 2010 BMC Educational Services
Complete State Chart
Bringing together all the states allows us to draw a complete state chart for standard discovery
ShouldAccess/ SHOULDACCESS
Excluded
/ DONTACCESS
DeviceIdentified/ DEVICE
NoResponse
/ NODEVICE
HostFound/ HOST
UnsupportedDevice
/ NOTHOST
HostInferred/ GOTACCESS
NoAccess
/ NOACCESS
Opt1stScan
/ DONTACCESS
GoodAccess/ SHOULDACCESS
OptNotBestIP
/ DONTACCESS
OptAlreadyProcessing
/ DONTACCESS
-end_state [from states above]-result [Success|NoAccess|Skipped|NoResponse]-reason [Full text or details]
DiscoveryAccess::.
Skipped
NoAccess
Success
NoResponse
Error
/ TRACEBACK
Error
OptRemote
Result attribute contents
If a traceback occurs before any end_state is set then end_state will be set to ‘Error’, otherwise end_state is left to record progress.
Result will always be set to ‘Error’
/ DONTACCESS
© 2010 BMC Educational Services
Complete State Chart – Additional Discovery
Discovery doesn’t stop once a session is established Patterns will cause further discovery as required
ShouldAccess/ SHOULDACCESS
Excluded
/ DONTACCESS
DeviceIdentified/ DEVICE
NoResponse
/ NODEVICE
HostFound/ HOST
UnsupportedDevice
/ NOTHOST
HostInferred/ GOTACCESS
NoAccess
/ NOACCESS
Opt1stScan
/ DONTACCESS
GoodAccess/ SHOULDACCESS
OptNotBestIP
/ DONTACCESS
OptAlreadyProcessing
/ DONTACCESS
-end_state [from states above]-result [Success|NoAccess|Skipped|NoResponse]-reason [Full text or details]
DiscoveryAccess::.
Skipped
NoAccess
Success
NoResponse
Error
/ TRACEBACK
Error
OptRemote
Result attribute contents
If a traceback occurs before any end_state is set then end_state will be set to ‘Error’, otherwise end_state is left to record progress.
Result will always be set to ‘Error’
Standard Discovery Additional Discovery
/ DONTACCESS
© 2010 BMC Educational Services
Complete State Chart – Sweep Scan
If the scan level is restricted to Sweep Scan then the DeviceIndentified state is regarded as a success
result = Success end_state = DeviceIdentified
ShouldAccess/ SHOULDACCESS
Excluded
/ DONTACCESS
DeviceIdentified/ DEVICE
NoResponse
/ NODEVICE
HostFound/ HOST
UnsupportedDevice
/ NOTHOST
HostInferred/ GOTACCESS
NoAccess
/ NOACCESS
Opt1stScan
/ DONTACCESS
GoodAccess/ SHOULDACCESS
OptNotBestIP
/ DONTACCESS
OptAlreadyProcessing
/ DONTACCESS
SCAN LEVEL = SWEEP SCAN
SCAN LEVEL = FULL DISCOVERY
-end_state [from states above]-result [Success|NoAccess|Skipped|NoResponse]-reason [Full text or details]
DiscoveryAccess::.
Skipped
NoAccess
Success
NoResponse
Error
/ TRACEBACK
Error
OptRemote
Result attribute contents
If a traceback occurs before any end_state is set then end_state will be set to ‘Error’, otherwise end_state is left to record progress.
Result will always be set to ‘Error’
Standard Discovery Additional Discovery
/ DONTACCESS
© 2010 BMC Educational Services
Summary
Atrium Discovery can be restricted from scanning sensitive/high risk endpoints
Atrium Discovery needs to be able to see network ports on the target to pick the right access methods and credentials
Atrium Discovery does full discovery of Host devices and basic discovery of other devices
Atrium Discovery caches successful access methods and credentials for faster future session establishment
Atrium Discovery optimizes it’s access to ensure that Hosts with multiple IPs are not repeatedly scanned
Atrium Discovery needs to succeed with getDeviceInfo, getHostInfo and getInterfaceList in order to create/update a Host node
© 2010 BMC Educational Services
Online Documentation: http://www.tideway.com/confluence/display/81/The+Discovery+Process
Tideway Foundation
Version 7.2
Documentation
Title
Further Resources
© 2009 BMC Educational Services
Discovery Overview State Charts