Upload
insync2011
View
898
Download
1
Embed Size (px)
DESCRIPTION
Citation preview
The most comprehensive Oracle applications & technology content under one roof The most comprehensive Oracle applications & technology content under one roof
Log files: A wealth of forensic evidence
Kevin Powe Integral Technology Solu6ons
The most comprehensive Oracle applications & technology content under one roof
More info at h:p://bit.ly/kapowelogs
The most comprehensive Oracle applications & technology content under one roof
Forensic process Log files Case files Tools
The most comprehensive Oracle applications & technology content under one roof
The Forensic Process
The most comprehensive Oracle applications & technology content under one roof
Step One: Secure The Scene
The most comprehensive Oracle applications & technology content under one roof
Operating System Evidence
netstat for network issues top or Windows Task Manager for CPU issues iostat or vmstat for I/O issues
The most comprehensive Oracle applications & technology content under one roof
Rolling Log Files
The most comprehensive Oracle applications & technology content under one roof
Cause
4-‐6PM 2-‐4PM
Symptoms
The most comprehensive Oracle applications & technology content under one roof
Step Two: Investigate The Scene
The most comprehensive Oracle applications & technology content under one roof
Don’t. Search. The. Log. Files.
The most comprehensive Oracle applications & technology content under one roof
‘Error’ versus ‘Warning’
‘Failing’ versus ‘Failed’
The most comprehensive Oracle applications & technology content under one roof
Step Three: Gather And Correlate Evidence
The most comprehensive Oracle applications & technology content under one roof
Step Four: Build A Hypothesis
The most comprehensive Oracle applications & technology content under one roof
1) Secure the scene 2) Investigate the scene 3) Gather and correlate evidence 4) Build a hypothesis
The most comprehensive Oracle applications & technology content under one roof
Forensic process Log files Case files Tools
The most comprehensive Oracle applications & technology content under one roof
AdminServer
managedServer2
managedServer1
WebLogic Server Domain
Java processes
The most comprehensive Oracle applications & technology content under one roof
HTTP Access Logs
The most comprehensive Oracle applications & technology content under one roof
192.168.5.6 -‐ -‐ [19/Nov/2010:13:34:49 +0800] "POST /AccountServices/ProxyServices/AccountServices HTTP/1.1" 200 29487
192.168.5.6 -‐ -‐ [19/Nov/2010:13:34:49 +0800] "POST /WarehousingServices/ProxyServices/RequestOrderDetails HTTP/1.1" 200 1167
192.168.5.6 -‐ -‐ [19/Nov/2010:13:34:49 +0800]
"POST /WarehousingServices/ProxyServices/RequestOrderDetails HTTP/1.1“
200 1167
Remote host
rfc931
authuser
date
request
status bytes
The most comprehensive Oracle applications & technology content under one roof
ELF = Extended Logging Format
The most comprehensive Oracle applications & technology content under one roof
Extended Logging Format Fields Common format fields date 6me bytes sc-‐status
Network fields c-‐ip s-‐ip c-‐dns s-‐dns
Request fields cs-‐method cs-‐uri cs-‐uri-‐stem cs-‐uri-‐query
The Good Stuff cs-‐comment 6me-‐taken custom
The most comprehensive Oracle applications & technology content under one roof
Server log files
The most comprehensive Oracle applications & technology content under one roof
The most comprehensive Oracle applications & technology content under one roof
####<2/08/2011 12:49:35 AM EST> <No6ce> <Server> <brother-‐eye> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-‐tuning)'> <<WLS Kernel>> <> <> <1312210175933> <BEA-‐002613> <Channel "Default" is now listening on 10.0.2.15:7001 for protocols iiop, t3, ldap, snmp, h:p.>
####<2/08/2011 12:49:35 AM EST> <No6ce> <WebLogicServer> <brother-‐eye> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-‐tuning)'> <<WLS Kernel>> <> <> <1312210175933> <BEA-‐000331> <Started WebLogic Admin Server "AdminServer" for domain "example1030Domain" running in Development Mode>
<2/08/2011 12:49:35 AM EST> <Nodce> <WebLogicServer> <brother-‐eye>
Timestamp Severity Subsystem Machine
<AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-‐tuning)'>
Server Thread ID
<<WLS Kernel>> <> <> <1312210175933> <BEA-‐002613> <Channel "Default" is
User Txn ID Diagn. Time (msecs) Message ID Text
The most comprehensive Oracle applications & technology content under one roof
Debug flags
The most comprehensive Oracle applications & technology content under one roof
HTTP: weblogic.servlet.DebugH:p SSL: default.DebugSSL JDBC: weblogic.jdbc.sql.DebugJDBCSQL
The most comprehensive Oracle applications & technology content under one roof
<4/08/2011 07:47:35 PM EST> <Warning> <netuix> <BEA-‐423420> <Redirect is executed in begin or refresh ac6on. Redirect url is /console/console.portal?_nfpb=true&_pageLabel=HomePage1.> Loaded index.jsp page Loaded index.jsp page Loaded index.jsp page <4/08/2011 23:20:34 PM EST> <Info> <Health> <brother-‐eye> <AdminServer> <weblogic.GCMonitor> <<anonymous>> <> <> <1311830434630> <BEA-‐310002> <86% of the total memory in the server is free>
TO <4/08/2011 07:53:38 PM EST> <No6ce> <WebLogicServer> <BEA-‐000365> <Server state changed to RUNNING> <4/08/2011 07:53:38 PM EST> <No6ce> <WebLogicServer> <BEA-‐000360> <Server started in RUNNING mode> <4/08/2011 07:53:49 PM EST> <Nodce> <Stdout> <BEA-‐000000> <Loaded index.jsp page> <4/08/2011 07:53:50 PM EST> <Nodce> <Stdout> <BEA-‐000000> <Loaded index.jsp page> <4/08/2011 07:53:51 PM EST> <Nodce> <Stdout> <BEA-‐000000> <Loaded index.jsp page> <4/08/2011 08:20:34 PM EST> <Info> <Health> <brother-‐eye> <AdminServer> <weblogic.GCMonitor> <<anonymous>> <> <> <1311830434630> <BEA-‐310002> <86% of the total memory in the server is free>
The most comprehensive Oracle applications & technology content under one roof
Oracle Service Bus tracing
The most comprehensive Oracle applications & technology content under one roof
JMS Message Logs
The most comprehensive Oracle applications & technology content under one roof
SOA Suite Diagnostic Logs
The most comprehensive Oracle applications & technology content under one roof
Forensic process Log files Case files Tools
The most comprehensive Oracle applications & technology content under one roof
Case File #1 An Unbalanced Load
The most comprehensive Oracle applications & technology content under one roof
Load balancer
Sun Reverse Proxy
Sun Reverse Proxy
WebLogic Server
WebLogic Server
The most comprehensive Oracle applications & technology content under one roof
cat access.log* | awk ‘{ print $x }’ | sort | uniq
(where x = posi-on of the cookie in the log file)
The most comprehensive Oracle applications & technology content under one roof
Case File #2 Fear Of Commitment
The most comprehensive Oracle applications & technology content under one roof
Oracle Service Bus
Tuxedo
The most comprehensive Oracle applications & technology content under one roof
Forensic process Log files Case files Tools
The most comprehensive Oracle applications & technology content under one roof
Tools Editors The Gun vi
Querying data find grep sed awk tail
Analysis Excel R Splunk
The most comprehensive Oracle applications & technology content under one roof
@kapowe
kevinpowe
kapowe