Debugging Android App using RxLoggerusing RxLogger

Subbu Jagadeesan

Sr. Engineering Manager, EMC

Bangalore Solution Center

AGENDA

RxLogger Overview

• What is it

• Features

• Configuration• Configuration

• RxLogger Output

Overview

What is RxLogger?• RxLogger is a collection of Android debug/diagnostic tools

• Logs

• System Dumps

• Performance Monitoring

• ANR Capture

• Resource Monitoring• Resource Monitoring

• When is it used?• Diagnose problems in the field

• Provides remote debug capability

Features

• Diagnostic tool for AndroidTM similar to eMscript on WM/CE• Familiar high level CSV log format with additional “drill down” logs

• Pluggable interface• Allows for 3rd party development and seamless add-ons

• Flexible configuration• Configurable through user interface or config xml file for ease of use in single • Configurable through user interface or config xml file for ease of use in single

or mass deployments

• Remote log retrieval• Capability to push logs to a remote FTP server – Remote Debugging

• UI available on device• Ability to view logs and configure on device

USER INTERFACE

• Simple interface to allow user control and configuration

• Notification icon to indicate when the tool is running

• Features

• Backup creates a zip file of current logs

• View displays logs on device

Output UI and log exampleApplication Not Responsive tracesXML configurationDetailed snapshots

Network traces (disabled by default)

Logcat “event” buffer

Logcat “main” buffer

Logcat “radio” buffer

High level resource logging

Logcat “system” buffer

Time Line Sub-System Information Events/Details

Main Output Log

Plugin Menus

• Main – System Monitor –Resource.csv

• PushPullClient – Push Log Files/Pull configuration

• LTS- Long Term Storage – Backup RxLogger files

• Snapshot – Detailed Process information

CONFIGURATION - mainUSER INTERFACE

• Configure settings for all available pluginsConfiguration stored in XML

Automatic restart of service when

new config file detected

PUSH PULL PLUGIN

• The Push Pull Plugin enables pushing of local log files to an FTP server.

• It can also be used to pull a file (such as a config.xml) from a server to the device.

Remote folders from a server to the device.

• Pushed files are organized on the FTP server by device serial number.

Remote folders

per device

RxLogger Output

Viewing Log Output

• Log files are generally reviewed in the following order• High Level Time line (Resourcex.csv)

- Identify outliers/anomalies

- Directs what details to look at next

• LogCat (developer trace messages)• LogCat (developer trace messages)

• General Detailed debug traces when looking for a specific issue

• Application not responsive (ANR)

• Points to why an application locked up

• Catch errors users don’t tell you about

• Various (plug in) snapshots

• Deep dive details on specific subsystem area

FILE STRUCTURELOG OUTPUT

Application Not Responsive traces

XML configuration

Detailed snapshots

Network traces (disabled by default)

Logcat “event” buffer

Logcat “main” bufferLogcat “main” buffer

Logcat “radio” buffer

High level resource logging

Logcat “system” buffer

Main CSV Log Features

• Date/Time

• Power

• Battery Capacity

• Battery Current

• Battery Voltage

• Battery Temp

• Screen State

• Screen Brightness

System Resources

• Network

• IP Address

• Gateway

• Total TX

• Total RX

• WAN

• Total Mobile TX

• Total Mobile RX

• Bluetooth

• BT Enabled

• BT Discoverable

• BT Connected

• Events

• Event Type

• HeartBeat

• ANR

Captured Parameters:

• System Resources

• Avg CPU%

• Current CPU%

• Free Physical Mem

• Free Storage Mem

• Process Count

• Foreground Package

• Foreground Activity

• Total Mobile RX

• WAN State

• WAN Type

• Signal Strength

• Wifi

• Wifi Enabled

• BSSID

• RSSI

• AP

• ANR

• Snapshot

• External Power

• Event Details

• SnapShot file

• ANR Details

Main Log SamplePower

CPU & Memory

Network WLAN LANNetwork WLAN LAN

BlueTooth Events and Details

CPU USAGE – Resource0.csvGraphing the CPU usage shows a sustained elevation

The foreground package change seems to correspond to the jump in CPU usage

ANR LOGS

• ANR or (Application Not Responsive) is an event triggered by the OS when an application’s main thread blocks for a long period of time.

• Typically the user will be presented with a dialog to “force close” the app or to wait.app or to wait.

• RxLogger detects these events and saves the call stack information for later analysis.

UNRESPONSIVE APP• Thread call stacks of every Dalvik thread in the system is logged in the

ANR log file

In this case an activity called “UnresponsiveActivity” called into a function

called “beUnresponsive” which then blocked in a sleep call.

LOGCATLOG OUTPUT

• Logcat is the built-in Android logging mechanism. It is a collection of circular memory buffers that log information from the system as well as user applications

• RxLogger records all four available Logcat buffers:

1. Main

• Mainly used for application/framework logs

• Good place to start to get a general idea of what the system is doing at a given time

2. System2. System

• Messages from system services (power manager, battery, package manager etc.)

3. Event

• Messages regarding the creating/destroying of activities and services

• Helpful for troubleshooting application/service crashes

4. Radio

• Message from the RIL and other cellular components

SNAPSHOTSLOG OUTPUT

• Snapshots are detailed logs that are recorded once every 30 minutes by default. They are intended to provide a second level analysis based on information obtained from the Resource.csv file. Information captured is configurable

• System Resources

• Per process CPU

• Per process memory

usage

• Process/thread list

• Network

• Interfaces

• Routing table

• Data connections

• File System

• Snapshot Configuration UI

• Process/thread list

• Power

• Battery history

• Wakelocks

• CPU states

• System Properties

• File System

• Partitions

• Free Space

• Usage Statistics

• Number of starts for

each package per day

Snapshot Sample

• Opening snapshot of interest shows top CPU usage by each process

Badappa is consuming > 40% of each core

Measurements from a TC55 - dual core processor

TCPDUMPLOG OUTPUT

TCPDump collects network packet traces on the device to be viewed with a tool like

Wireshark.

Helpful for debugging web apps or other network issues.

•RxLogger provides an SDK to support Plug-In development.

•Import the sdk (rxloggerplugininterface.jar) into the Android

Project and configure the Build Path ( Build Path -> Configure

build path) to add the jar file.

•The plug-in class should have RxLoggerPluginService from the

jar file as its super class.

RxLogger Plug-In Development

•Implement the required methods as shown

•getPluginName()

•pluginInit()

•pluginDeinit()

•pluginStart()

•pluginStop()

•Refer to the user guide for more details:

M O TO RO LA an d th e S tyl ized M Lo go a re re gis te re d in t he US P a ten t & Tra dem ark O ff i ce . A ll ot h er p rodu c t o r se rv ic e nam e s a re th e prop e rt y o f thei r re s pec tiv e o wne rs .

© M ot orol a, I nc . 201 1

C om pan y N a m e – S o lu t ion N am e w / S olu tio n V er sio n Jo h n S eim e r, C a thy W e i, M a tt W h it e M o to ro la G lo b al S o lu t io n Ce n te rs

R xL og g er U ser G u id e

v 1 .0

THANK YOUTHANK YOU

TROUBLESHOOTING METHODOLOGYResource.csvLevel 1

WHATIdentify the symptom

Look for red flags that may

point to an area of interest

Level 2WHOIdentify the offending

application/process

Hig

h

CP

U

Hig

h

Me

m

Sh

ort

Ba

tte

ry L

ife

Ne

two

rk

Ap

p

Cra

sh

Per Process

CPU dump in

Per Process

mem dump in

Wakelock

dump in

TCPDump

network

Per Process

CPU dump in application/process

Use event column to find

detailed log file around time

of red flag

Level 3WHEREIdentify the part of

the application

causing the problem

CPU dump in

Snapshot

mem dump in

Snapshot

dump in

Snapshot

network

traces

CPU dump in

Snapshot

PC/Connected Debug Tools

Eclipse debugger, method profiler, HPROF heap dump analyzer, Wireshark, etc.

RESOURCE CSV LOGLOG OUTPUT

• The Resource.csv file presents a timeline of device activity and vital statistics

• The default configuration will log once per minute plus asynchronous events (power

notifications, etc.)

FAULT FINDING EXAMPLES

Detecting High CPU usage – Resource0.csv• Graphing the CPU usage shows a sustained elevation

The foreground package change seems to correspond to the jump in CPU usage

CPU USAGEFAULT FINDING EXAMPLES

• Opening snapshot of interest shows top CPU usage by each process

badappa is consuming > 40% of each core

Measurements from a TC55 - dual core processor

MEMORY USAGEFAULT FINDING EXAMPLES

A steady decline of “Free Physical Mem” over a long period of time could be indicative of a

memory leak. In this case the memory keeps decreasing until the app is killed and then all the

“leaked” memory is reclaimed.

MEMORY USAGEFAULT FINDING EXAMPLES

Memory used by each process can be viewed individually in snapshot files for growth

UNRESPONSIVE APPFAULT FINDING EXAMPLES

• Browsing the event column of the Resource.csv file we can see an ANR situation was detected

Using this message we can find the log file associated with this event

(traces_com.example.unresponsiveapp.txt)

UNRESPONSIVE APPFAULT FINDING EXAMPLES

• Thread call stacks of every Dalvik thread in the system is logged in the ANR log file

In this case an activity called “UnresponsiveActivity” called into a function

called “beUnresponsive” which then blocked in a sleep call.

WAKELOCKSFAULT FINDING EXAMPLES

Note High current draw while the terminal

is in suspend – Screen State = 0

High current draw over time will impact

battery lifebattery life

WAKELOCKSFAULT FINDING EXAMPLES

• Looking at power section of snapshot shows all requested wake locks

PowerManagerService was holding a wake lock for over 45 minutes in the

PowerMaagerService encompasses all partial wakelocks requested by Apps so we need to look further

WAKELOCKSFAULT FINDING EXAMPLES

“WakeLockerBadWakeLock” is the com.example.badappd

OTHER TOOLS

• RxLogger can coexist and complement existing Android tools.

• mLog is a tool built in to the OS on some of our Android products. It collects some similar information however the focus is on a user initiated “export” of the logs after a problem occurs.

THANK YOUTHANK YOU