COMBINATORIAL TESTING ONID3V2 TAGS OF MP3 FILESZhiqiang Zhang1, Xiaojian Liu2, Jian Zhang1

1 Institute of Software, Chinese Academy of Sciences2 Institute of Automation, Shandong Academy of Sciences

Introduction• MP3 (MPEG-1 or MPEG-2 audio layer III) is an audio file

format developed by the Moving Pictures Experts Group (MPEG)• One of the most popular audio formats• Supported by almost all audio players

• Usually, an MP3 file comes with an ID3 tag, which stores audio information such as:• Title, artist, album, …

• ID3 has two unrelated versions:• ID3v1 & ID3v2

Overview• ID3v2 tag format• Test goals• Modeling• Experiments & Results• Conclusion

Combinatorial Testing• Considerably high fault coverage• Model the system under test (SUT) as a parameterized

black box• Usually use a covering array as the test set

• Small # of test cases

MP3 frames & ID3v2 tags• An MP3 file is built up from a sequence

of MP3 frames (MPEG frames)• A short segment of audio data• The first 11 or 12 bits are always set, can be

used for synchronization• Sometimes MP3 frames depend on each

other, and cannot be freely cut

• ID3v2 tag:• A metadata container for audio file

information• Located at the beginning of an MP3 file

ID3v2 tag

MP3 frames

ID3v2 tag format

• In our work, we consider a subset of the ID3v2 tag format• An ID3v2 header• Several ID3v2 frames

• ID3v2 header format:ID3v2/file identifier “ID3”ID3v2 version $03 00ID3v2 flags %abc00000ID3v2 size 4 * %0xxxxxxx

ID3v2 header

ID3v2 frame 1

ID3v2 frame 2

ID3v2 frame 3

…

Tag size

ID3v2 Frames• An ID3v2 frame consists of a frame header and the frame

content• Frame header

Frame ID $xx xx xx xx (four characters)Size $xx xx xx xxFlags $xx xx

• Text information framesText encoding $xxInformation <text string>

Frame header

Frame contentFrame size

Text Encodings

• Encodings commonly used in China:• $00: ASCII or GBK• $01: Unicode (UTF-16)

• Little-endian (LE)• Big-endian (BE)

• $02: Unicode BE• $03: UTF-8

Test Goals

• Usually, tag processing and audio playing are performed by separate modules of MP3 players

• Audio playing is often done by a decoder (out of the scope of our work)

• ID3v2 tag processing• Goal I: text information recognition and display

• Text encodings of some frames may influence the processing of other frames

• Goal II: robustness against bad header & frame sizes• Offset related• May cause buffer overflow or vulnerable read operations

Modeling (Test Goal I)• Six types of text information frames

• TIT2 (title), TPE1 (artist), TALB(album), TRCK (track number), TCON (genre) and TYER(year)

• The model:

TIT2 TPE1 TALB TRCK TCON TYER

NONE NONE NONE NONE NONE NONE

ASCII ASCII ASCII ASCII ASCII ASCII

GBK GBK GBK

ULE ULE ULE

UBE UBE UBE

UBE2 UBE2 UBE2

UTF-8 UTF-8 UTF-8

Modeling (Test Goal II)• The sizes indicated by the ID3v2 header & frame size

bytes might be faulty• May cause critical failures• For building each test case

1. Set the header and frame size bytes

2. Set the actual header and frame sizes

3. Build an ID3v2 frame

4. Build an ID3v2 tag by filling the content with frames

5. Attach MP3 frames

Modeling (Test Goal II)

• Building an ID3v2 frame • Building an ID3v2 tag

Frame header

$00 AAAAAA…

Frame size bytes

Actual frame size

ID3v2 header

ID3v2 frame

ID3v2 frame

ID3v2 frame

ID3v2 frame

…

Tag size bytes

Actual tag size

Modeling (Test Goal II)

• Building a complete MP3 file• Attach a sequence of MP3 frames after the ID3v2 tag

• In some cases, the actual tag size is not an integral multiple of the total frame size• To fill the remaining space: 4 filling modes

Filling Modes

Tag size

ID3v2 frame

MP3 frames

Tag size

None Safe Cut Overflow

Tag size

ID3v2 frame

MP3 frames

$00 00 00…

Tag size

ID3v2 frame

MP3 frames

ID3v2 frame

ID3v2 frame

ID3v2 frame

MP3 frames

Modeling (Test Goal II)

• TS: tag size indicated by the ID3v2 header size bytes• TSD: difference of the actual tag size compared with TS• FS: frame size indicated by the frame header size bytes• FSD: difference of the actual frame size compared with FS• FM: filling mode• ATCHMP3: whether MP3 frames are attached or not

TS TSD FS FSD FM ATCHMP3

0KB 0KB 0B 0B NONE YES

4KB 1KB 256B 64B SAFE NO

64KB -1KB 1KB -64B CUT

OVFL

Modeling (Test Goal II)• Normal constraints

• IF TS=0KB THEN TSD<>-1KB• IF FS=0KB THEN FSD<>-64B

• When TS=0KB and TSD=0KB, the actual tag size is 0KB. So no frames will be filled. Thus FS, FSD and FM are invalid parameters• Introduce a special parameter value ‘#’ for invalid parameters• IF TS=0KB AND TSD=0KBTHEN FS=# AND FSD=# AND FM=#

• IF TS<>0KB OR TSD<>0KB THENFS<># AND FSD<># AND FM<>#

Experiments• Test subjects:

• I: an on-vehicle leisure and entertainment system• II: a portable MP3 player

• Test generation:• Use Microsoft’s PICT• Test Goal I: 57 test cases (strength=2)• Test Goal II: 59 test cases (strength=3)

Experimental Results(Test Goal I, Subject I)

• 2 passed, 55 failed• Actually, only ASCII text can be displayed

Condition Results

ALL ASCII text is correctly displayed.

ALL UBE2 text is incorrectly displayed.*

TIT2=NONE Title is displayed as file name.

TPE1=NONE Artist is displayed as folder name.

TPE1{ASCII,UBE2} ANDTIT2{GBK,ULE,UBE,UTF-8}

Title is not displayed.*

TPE1{ASCII,UBE2} ANDTIT2{GBK,ULE,UBE,UTF-8}

Title is displayed as file name.*Artist is displayed as folder name.*

Experimental Results(Test Goal I, Subject II)

Condition Results

TIT2{UBE2,UTF-8} File name is displayed as the title.*

TIT2<>NONE ANDTPE1{UBE2,UTF-8}

File name is displayed as the title.*

OTHERWISE All information is correctly displayed.

• 33 passed, 24 failed• Problem when processing two encodings• Failures caused by 1-2 parameters

Experimental Results(Test Goal II, Subject I)

Condition Results

TS=0KB ANDTSD=1KB ANDFS<>0KB ANDATCHMP3=NO

The subject encounters a failure, and all audiofiles copied into the subject cannot be played.*

ATCHMP3=YES The file can be successfully played.

ATCHMP3=NO The file is skipped.

• 52 passed, 7 failed• Failures caused by 4 parameters

Experimental Results(Test Goal II, Subject II)

• All test cases passed• We occasionally found a bug

• When ATCHMP3=NO, the subject• Displays that the file is damaged for a few seconds• Proceeds to the next file.

• Just between this two moments, it is not possible to play back to the previous file

• Not Discovered by CT!!

Condition Results

ATCHMP3=YES The file can be successfully played.

ATCHMP3=NO The player shows that the file is damagedand proceeds to the next file.

Conclusion• Benefit from CT:

• Small test suite• Test Goal I: 57 out of 7323=2744 exhaustive test cases• Test Goal II: 59 out of 3442=648 exhaustive test cases (including test

cases not satisfying the constraints)

• Interaction faults discovered• Failures are caused by 1-4 parameters

Additional Results• Experiments are conducted on two PC audio players• PC audio player I perfectly passes all the tests • PC audio player II:

• Test Goal I: similar failures as subject I• Test Goal II:

• When all test cases are added to the player playlist, the player’s UI begin to respond slowly

• The player got stuck when playing 4 test cases

THANK YOU！