1

Detecting Hardware Trojans in Unspecified Functionality Using Mutation Testing

Nicole FernK.-T. Tim Cheng

UC Santa Barbara

2

Main Contributions

• Information leakage Trojan only modifying unspecified functionality

• Mutation testing based detection method

• Discovered vulnerabilities and verification holes in UART design with sophisticated testbench

3

Hardware Trojans

• Malicious circuitry inserted in the hardware design

• Can be inserted by any party with access to the design!

• Goals: leak information, induce faults, chip failure, gain root privileges, etc.

4

Trojan Classes

1. The logic functions of some design signals are altered, system specifications are violated

2. The Trojan leaks information through side-channels

3. The logic functions of only those design signals which have unspecified behavior are altered to add malicious functionality without violating system specifications

5

FIFO Example

What is the value of read_data when read_en is 0?

6

Threat Model

• Trojans can be inserted in the RTL and all subsequent design stages

• Our method analyzes RTL code, identifies Trojans which leak information from the circuit by hiding in unspecified functionality

• Behavior of circuit under Trojan activation condition is unspecified and unverified

– Trojan can be active frequently, yet go undetected!

7

Detection Methodology Overview

• Goal: design independent method to identify dangerous unspecified functionality

• Use mutation testing to uniformly sample possible design modifications (can think of as very simple Trojan modifications)

• Use additional information to determine if modification is “dangerous”

8

Mutation Testing

Basic Idea: If the testbench cannot detect an artificial error, testbench likely incapable of detecting a real error

DUTTests CheckerDetected

Undetected

Add more tests

Fix

9

Mutation Testing

• Used in software domain since the 1970’s to test program correctness– Can also identify security weaknesses 1,2

• Used in hardware domain for testbench qualification 3,4

• Long simulation runtime and manual effort required for mutant analysis are drawbacks

1. Jia and Harman. An analysis and survey of the development of mutation testing. TSE, 2011.2. Breech et al. An attack simulator for systematically testing program-based security mechanisms. ISSRE, 2006.3. Bombieri et al. Functional qualification of TLM verification. DATE, 2009.4. Lisherness et al. Mutation Analysis with Coverage Discounting. DATE, 2013.

10

Interpreting Undetected Faults

2 Classifications:1. Affect Poorly Tested Functionality• Ex. Interrupt line set to static 0

2. Redundant Fault: does not affect design functionality• Ex. for (int i = 0; i < 10; i++)• Ex. Adder output toggled only during

intermediate cycles

Coverage Discounting: automated analysis to identify Class 1 faults

1. Lisherness et al. Mutation Analysis with Coverage Discounting. DATE, 2013.

!=

11

Identifying Dangerous Faults• Attacker-observable signals: primary outputs,

software-visible registers, network interface, bus interface,…

• Information leakage possible if undetected fault causes change in attacker-observable signalsif (key) { code w/ fault; } else { original code; } key

0

1

0/11/0

Attacker-observable

signal

12

Dangerous Unspecified Functionality

Undetected Fault Classes:1. Affect Poorly Tested Functionality• Ex. Interrupt line set to static 0

2. Redundant Fault: does not affect design functionality• Ex. for (int i = 0; i < 10; i++)• Ex. Adder output toggled only during

intermediate cycles

!=

Undetected Fault Classes:1. Affect Poorly Tested Functionality• Ex. Interrupt line set to static 0

2. Affect attacker-observable signals, but not design functionality

3. Causes no change in any signal values (truly redundant)

Automated method to

identify only dangerous

Class 2 faults

13

Trojan Detection Methodology

DUTTests CheckerDetected

Undetected

Add more tests

Fix

Before Fault Injection• Record functional coverage• Record attacker-observable signals

Functional coverage differs?

Fault affects poorly tested specified functionality

Attacker-observable

signals differ?Fault affects dangerous unspecified functionality

Refine Specification

14

Poorly Tested Specified v. Dangerous Unspecified Functionality

Redundant

15

Methodology Applied to FIFO Example

What is the value of read_data when read_en is 0?

1) Fault is undetected2) Causes changes in

attacker-observable signal read_data!

||

16

Fault Ranking Mechanism

• Might be too expensive to analyze all faults classified as dangerous

• Ideal to identify and fix functionality related to the “most dangerous” faults first

• Quantities easy to measure for each fault:1. Number of attacker-observable bits differing 2. Total time attacker-observable signals differ 3. Number of distinct tests producing differences in

attacker-observable signals

17

UART Controller Case Study

• OpenCores IP, OVM testbench from EDA vendor• Verification Infrastructure: 80 tests, 846 coverpoints• Mutation Testing: 1183 total faults injected– 110 faults not detected• 32 caused differences in attacker-observable signals

– 4 discounted coverpoints

UART

wb_dat_o

int_o

wb_ack_o

baud_o

stx_pad_o

rts_pad_o

dtr_pad_o

Output signals going to main processor

Output signals for serial data transmission

38 attacker-observable bits

18

Wishbone Bus Trojan• Analyzed 3 most dangerous faults– All affect bus between UART and main processor– All affect output enable signal for data bus

• Information can be leaked on data bus if a valid read transaction is NOT occurring!

UART

wb_dat_o

int_o

wb_ack_o

baud_o

stx_pad_o

rts_pad_o

dtr_pad_o

19

Wishbone Bus TrojanThe 3 Undetected Faults:

Write enable is de-asserted (read transaction)

Slave is selected Valid bus cycle in progress

| |

|

Undetected Faults changing & to |

20

Wishbone Bus Trojan

Data can be leaked on wb_dat_o whenever all 4 conditions for a valid read transaction are not simultaneously met!

|

We will leak data when wbstate==01but not during a valid read transaction

21

Improving Bus Checker

• Data during invalid bus cycle, or output data during write cycles was not checked

• New assertion added: value of wb_dat_o cannot change unless design has been reset or read request is being acknowledged

• New check detects 3 faults and bus Trojan

Method able to highlight unspecified functionality in on-chip bus protocols allowing attacker to leak

information using the system bus

22

Interrupt Functionality

• 5 possible interrupt sources, 1 interrupt bit • Next most dangerous fault causes int_o to

become X for many cycles during 49 of 80 tests!• Attacker can spuriously change int_o to

encode information

UART

wb_dat_o

int_o

wb_ack_o

baud_o

stx_pad_o

rts_pad_o

dtr_pad_o

23

Why was this fault undetected?

• Interrupt Identification Register (IIR) reveals source of interrupt

• For all 5 events which cause interrupts, testbench checks that IIR is properly set, and that int_o is set within 10 cycles

• Due to a bug in the testbench if int_o is X this check is skipped!

24

Interrupt Functionality

Functional coverage did not change,

suggesting that the coverage model is

insufficient!

Method able to highlight verification hole relating to specified interrupt functionality

25

Conclusions

• Illustrated how unspecified functionality can be altered for malicious purposes

• Automated analysis methodology:1. Uncovered Trojan leaking information through

unspecified functionality in a standard bus protocol

2. Identified verification hole for interrupt functionality

26

Questions?

27

Payload violates Design Specification

• Relies on rare triggering conditions

• Detection likely if triggered during verification/testing

• Detection methods identify “almost unused logic”

• Ex. When specific 128-bit plaintext value appears, dump key bits to output

Payload operates completely within unspecified functionality

• Design behavior unspecified for activation conditions

• Likely not detected if triggered during testing

• Functionality Trojan modifies is unspecified and unverified

• Activation condition can occur frequently during testing (ex. read_en == 0)

Comparison of Trojans in Logic Domain

28

Coverage Discounting

DUTTests CheckerDetected

Undetected

Add more tests

No

Yes

Fix

CoverageChanged?