Person Re-identification usingCo-occurrence Attributes of Physical and

Adhered Human Characteristics

Masashi Nishiyama, Shota Nakano, Tappei Yotsumoto, Hiroki Yoshimura,Yoshio Iwai and Kazunori Sugahara

Graduate School of Engineering, Tottori University, JapanEmail: nishiyama@eecs.tottori-u.ac.jp

Abstract—We propose a novel method for extracting featuresfrom images of people using co-occurrence attributes, which arethen used for person re-identification. Existing methods extractfeatures based on simple attributes such as gender, age, hairstyle, or clothing. Our method instead extracts more informativefeatures using co-occurrence attributes, which are combinationsof physical and adhered human characteristics (e.g., a manwearing a suit, 20-something woman, or long hair and wearinga skirt). Our co-occurrence attributes were designed using priorknowledge of methods used by public websites that search forpeople. Our method first trains co-occurrence attribute classifiers.Given an input image of a person, we generate a feature by vec-torizing confidences estimated using the classifiers and computea distance between input and reference vectors with a metriclearning technique. Our experiments using a number of publiclyavailable datasets show that our method substantially improvedthe matching performance of the person re-identification results,when compared with existing methods. We also demonstratedhow to analyze the most important co-occurrence attributes.

I. INTRODUCTION

Person re-identification is an active topic in pattern recog-nition research and has many potential applications such aswatchlist monitoring and video surveillance. The problem isespecially difficult because we must allow for variable view-points, illuminations, and poses. To overcome these difficulties,researchers have developed various approaches for extractinginvariant features from images of people. These features havesignificant influences on the matching performance. In thispaper, we focus on extracting invariant features that representthe people in images.

There are currently two main approaches for extractingfeatures to be used in person re-identification applications.The first exploits low-level features [1]–[4] such as the dis-tributions of gradients and colors. The second exploits high-level representations [5]–[8] such as gender, age, clothing,and gait, which are called human attributes in the field ofsoft-biometrics. However, it is exceptionally difficult to accu-rately infer high-level representations. Instead of inferring eachattribute, researchers have recently started to use mid-levelsemantic attributes [9], [10] for person re-identification. Theseattributes directly represent elements of features extracted fromimages of people. However, existing methods for determininginvariant features are not sufficient for person re-identification,when compared with characteristics that are used when peopleidentify each other.

Physical  and  adhered    human  characteris1cs  

Person re-identification using

co-occurrence attributes

Physical  and  adhered    human  characteris1cs  

& &

Fig. 1. Person re-identification using co-occurrence attributes. We proposeusing combinations of physical and human characteristics to extract featuresfrom images of people.

Our key idea is to exploit our prior knowledge of thecharacteristics used by people when looking for others. Forinstance, public websites of open criminal investigations [11],[12] or missing persons [13], [14] use a combination of phys-ical and human characteristics such as gender and clothing.These combinations are considered to be informative clueswhen searching for people, and we expect that they wouldbe useful invariant features for person re-identification.

In this paper, we attempt to determine what kind of com-binations of physical and human characteristics are valuablefor person re-identification, as illustrated in Figure 1. We callthese combinations co-occurrence attributes. We propose anovel method for automatically extracting features based onthe confidences of co-occurrence attributes, to enhance theperformance of person re-identification methods. The extractedfeatures were developed using our prior knowledge of thecharacteristics used in public websites that identify or findpeople. The rest of this paper is organized as follows. Section IIcontains a brief summary of related work, and Section IIIdescribes the design of the co-occurrence attributes. Section IVdescribes the proposed method for extracting features usingco-occurrence attributes. Section V contains the results ofour experiments and analyses, which demonstrate the methodseffectiveness. Our concluding remarks are given in Section VI.

II. RELATED WORK

Existing methods exploit low-level features such as his-tograms of oriented gradients [2], color histograms [1], or theircombinations [3]. The learning-based approach in [4] selectedeffective low-level features from filter banks of gradients andcolors. Unfortunately, low-level features are often affected

2016 23rd International Conference on Pattern Recognition (ICPR)Cancún Center, Cancún, México, December 4-8, 2016

978-1-5090-4846-5/16/$31.00 ©2016 IEEE 2086

Co-­‐occurrence  (physical  &  adhered)  

Co-­‐occurrence  (adhered  &  adhered)  

Co-­‐occurrence  (physical  &  physical)  

Woman wearing a skirt

Man in his 20s

Wearing short sleeves and shorts

Fig. 2. Examples of co-occurrence attributes of physical and adhered humancharacteristics.

by the variability of viewpoints, illuminations, and poses.To extract invariant features, researchers have focused onhuman attributes [15]. We consider each characteristic (e.g.,gender, hair style, or clothing) as a single attribute. Somemethods [16], [17] improved the inference of single attributesusing a joint learning technique. Khamis et al. [6] proposedjointly optimizing the inference of single attributes and theidentification. Shi et al. [9] applied a domain shift betweenfashion and surveillance data to avoid complications due tolabeling processes. Layne et al. [10] developed a methodfor weighting single attributes to increase the identificationperformance. However, existing methods have not sufficientlyconsidered the combinations of physical and adhered humancharacteristics that are often used when people identify eachother.

III. DESIGN OF CO-OCCURRENCE ATTRIBUTES

A. Combinations of physical and human characteristics

We designed co-occurrence attributes that can be extractedfrom images of people. As described in [18], human attributescan be split into three intuitive types: physical, behavioral,and adhered human characteristics. Physical characteristics areperson-specific traits and do not significantly change over time(e.g., gender, age, hairstyle, and beard). Behavioral character-istics are temporal changes such as gesture or gait. Adheredhuman characteristics depend on a persons appearance and aredefined as things that are temporarily attached to a person (e.g.,clothing or sunglasses).

We used combinations of physical and adhered humancharacteristics because we are motivated by certain publicwebsites [11]–[14]. There are three ways to combine twocharacteristics, as illustrated in Figure 2: a combination ofphysical and adhered human characteristics (e.g., “ womanwearing a skirt”), a combination of physical characteristics(e.g., “man in his 20s”); and a combination of adhered humancharacteristics (e.g., “wearing short sleeves and shorts”).

B. Binary representation

Physical and adhered human characteristics use two typesof class labels: binary labels such as gender (male or female)and sunglasses (presence or absence); and multi-class labelssuch as age (e.g., 20s, 30s, or over 40) and tops (e.g., longsleeves, short sleeves, or suit jacket). When simply combiningtwo characteristics with L1 and L2 classes, the number ofclasses in the combination is L1L2. Our method uses a binary

Man wearing

long sleeves

Man wearing

short sleeves

Man wearing a suit

Woman wearing

long sleeves

Woman wearing

short sleeves

Woman wearing a suit

&

&

&

& &

&

Fig. 3. Binary labels for representing co-occurrence attributes of physical(gender) and adhered human (tops) characteristics.

label to represent a co-occurrence attribute by assuming thateach characteristic class is independent. For instance, whencombining gender (L1 = 2) and tops (L2 = 3), we obtainL1L2 = 6 binary labels for the co-occurrence attributes.As illustrated in Figure 3, binary labels are represented as“man wearing long sleeves”, “man wearing short sleeves”,“man wearing a suit”, “woman wearing long sleeves”, “womanwearing short sleeves”, and “woman wearing a suit” (andhave true or false attributes). In biometrics, [18] showedthat binary representations are intuitively easy for humansto understand, when compared with discrete and continuousrepresentations. By exploiting the binary representation of co-occurrence attributes, our aim is to easily analyze which co-occurrence attributes are useful for person re-identificationmethods.

IV. PERSON RE-IDENTIFICATION USING CO-OCCURRENCEATTRIBUTES

A. Overview of our method

We now briefly describe our method that uses co-occurrence attributes for person re-identification, which isillustrated in Figure 4. We train co-occurrence attribute clas-sifiers to infer whether an image of a person contains theattributes. In this training process, we use images that havebeen labeled with co-occurrence attributes. These images areassigned positive or negative labels following the binary rep-resentation described in Section III-B. Note that this designcomplicates the labeling process, because there are manycombinations of physical and adhered human characteristics.Thus, we automatically assigned co-occurrence attributes tothe training samples using combinations of the labels forsingle attributes. If we have L1L2 combinations of singleattributes, there is 1 positive label of a co-occurrence attributeand L1L2 − 1 negative labels. For instance, a positive labelin Figure 3 is “man wearing long sleeves”; the remaining arenegative.

Given an input image of a person, we compute the confi-dences of co-occurrence attributes using the trained classifiers.We generate a feature vector for person re-identification byvectorizing the confidences, and compute the distance betweenan input feature vector and a reference feature vector. Thedetails of each step are described below.

2087

Confidence  N  

Classifier  of    co-­‐occurrence  a1ribute  1

Classifier  of    co-­‐occurrence  a1ribute  2

Classifier  of    co-­‐occurrence  a1ribute  N  

…

Feature  vector

Feature  vector

Metric  learning  

Confidence  1

Confidence  2

…

2. Estimate confidences

… … False examples

True examples

& &

3. Compute a distance

1. Train classifiers

Fig. 4. Overview of our method that uses co-occurrence attributes for personre-identification.

B. Training classifiers of co-occurrence attributes

There may be much less positive attributes than negative.This is particularly the case when using co-occurrence at-tributes. If the number of positive and negative samples aresignificantly different, generic machine learning classificationalgorithms do not work well. To overcome the problemsassociated with imbalanced data, we can weight attributesaccording to inverse of the number of samples or align thenumber of samples. The alignment approach performed thebest in our preliminary experiments, so this is the method weused in the remainder of this paper.

Next, we extracted feature vectors from the images to trainthe classifiers of co-occurrence attributes using the ensemble oflocalized features (ELF) method [4]. We used a linear supportvector machine classifier SVMi (i ∈ 1, ..., N), where N is thenumber of co-occurrence attributes.

C. Extracting features for person re-identification

We describe how to compute the features used to matchimages of people. Given an image, we compute a signeddistance hi by applying a co-occurrence attribute classifier,that is,

hi = SVMi(e) , (1)

where e represents an ELF vector extracted from an imageof a person. An image is assigned a positive label when thesigned distance is positive, and vice versa. We regard a signeddistance as a confidence value, in the same manner as existingtechniques. Note that if we directly use the signed distances aselements of a feature vector, the ranges of possible values areimbalanced because they depend on their respective element.Thus, we apply a simple scaling technique to align the rangesof values. We estimate the confidence value xi = 2h′i−1 using

h′i =hi −minhti

maxhti −minhti, (2)

where max or min returns the maximum or minimum valuehti (t ∈ 1, ..., T ), and T is sum of the number of posi-tive and negative samples. The feature vector for person re-identification is x = [x1, x2, ..., xN ]T.

D. Computing the distance between feature vectors

We describe how to compute the distance between featurevectors extracted from images of people. If we use the all

elements of a feature vector, we believe that some elementscontribute to the identification and others do not. To increasethe matching performance, some methods give larger weightsto the important elements of a feature vector using a greedyalgorithm technique [10] or a metric learning technique [19].Our method used the large margin nearest neighbor (LMNN)method [20]. This technique generates a metric matrix M thatreduces the distance between feature vectors belonging to thesame person using the k-nearest neighbors, while lengthen-ing the distance between vectors from different people. Thedistance, d, between features xa,xb is

d2 = (xa − xb)TM(xa − xb) . (3)

Note that d is smaller when the feature vector of individual ais more similar to the feature vector of individual b.

V. EXPERIMENTAL ANALYSIS OF PERSONRE-IDENTIFICATION USING CO-OCCURRENCE ATTRIBUTES

We evaluated the effectiveness of our method using somecomputational experiments. The person re-identification resultsare reported in Section V-A and our analysis of a metricmatrix trained using LMNN is given in Section V-B. Wealso evaluated our method on a number of public datasets,as described in Section V-C.

A. Evaluation of basic performance

1) Experimental conditions: To evaluate the performanceof our person re-identification method, we used the PETADataset [21], which consists of 10 publicly available datasets:3DPeS [22], CAVIAR4REID [23], CUHK [24], GRID [25],i-LID [26], MIT [27], PRID [28], SARC3D [29], Town-Centre [30], and VIPeR [31]. In the PETA dataset, all theimages are labeled with single attributes. We selected 15single attributes, as shown in Table I. P represents physicalcharacteristics and A represents adhered human characteristics.In our experiments, we selected single attributes from [10]and public websites [11]–[14]. Note that we did not selectcolor attributes, because colors significantly vary betweensurveillance cameras (as described in [18]).

We also designed 96 co-occurrence attributes, as shown inTable II. P&A represents combinations of physical and ad-hered human characteristics, P&P represents combinations ofphysical characteristics, and A&A represents combinations ofadhered human characteristics. We removed 23 combinationsof single attributes (e.g., “wearing a suit jacket and shorts”)from the 119 combinations, because they did not commonlyoccur in practical cases. There were 59 P&A attributes, 22P&P attributes, and 15 A&A attributes (Table II). The numberof positive samples for each co-occurrence attribute was 48 orabove.

We split the PETA dataset into test and training sam-ples. We used images of people from VIPeR to evaluatethe performance of the person re-identification method. Werandomly selected 316 individuals from VIPeR for our testsample. The remaining 316 individuals were used to train ametric matrix for LMNN. We repeated the random selection10 times to generate different test sample sets. We usedtwo major indicators: cumulative match characteristic (CMC)curves dependent on the n-th rank matching rate, and the

2088

TABLE I. SINGLE ATTRIBUTES FOR EVALUATING AN EXISTINGMETHOD. WE USED 2 CLASSIFIERS FOR BINARY CHARACTERISTICS AND

13 CLASSIFIERS FOR MULTI-CLASS CHARACTERISTICS.

Binary label Multi-class label

Physical Gender Hairstylecharacteristics (Male/Female) (Short/Long/Bald)(P) Age

(16–30/31–45/46–60/Over 60)

Adhered Sunglasses Topshuman (Presence/Absence) (Short Sleeves/Long Sleeves/Suit)characteristics Bottoms(A) (Shorts/Skirt/Suit)

0

20

40

60

80

100

1 316

Matching  rate  (%

)

Rank

   CA    (91.6)

   SA    (85.1)

   ELF    (77.8)

Fig. 5. CMC curve evaluated on the VIPeR dataset. The number inparentheses represents the nAUC for each method.

normalized area under the CMC curve (nAUC). We comparedthe following methods.

• CA: Our method for extracting features from imagesusing the co-occurrence attributes described in Sec-tion IV.

• SA: An existing method [10] that extracts featuresusing singles attributes. Note that we used the singleattributes in Table I and LMNN instead of a greedyalgorithm.

• ELF: An existing method [4] that extracts low-levelfeatures using filter banks of gradients and colors.

2) Experimental results: Figure 5 shows the performanceof the person re-identification results based on the CMC curvesand nAUC. The plot shows the average values for the 10 setsof randomly generated test samples. We can clearly see thatour features based on the co-occurrence attributes are superiorto features using singles attributes and the low-level featuresbased on gradient and color. Overall, our method outperformedthe others, and improved the matching rate for rank n = 20by 20 points when applied to this difficult task.

To determine the most effective combinations of character-istics, we applied the method

• P1: without the combinations of physical and adheredhuman characteristics (P&A);

• P2: without the combinations of physical characteris-tics (P&P);

• P3: without the combinations of adhered human char-acteristics (A&A); and

• P4: using all the combinations in Table II.

TABLE II. CO-OCCURRENCE ATTRIBUTES FOR EVALUATING OURMETHOD.

Age 31–45 & Shorts (P&A) Male & Long Hair (P&P)

Suit Jacket & Long Sleeves (A&A) Age 46–60 & Short Sleeves (P&A)

Female &Age Above 60 (P&P) Male & Age16–30 (P&P)

Short Hair & Suit Pants (P&A) Bald & Short Sleeves (P&A)

Short Hair & Short Sleeves (P&A) Female & Skirt (P&A)

Sunglasses & Short Hair (P&A) Age31–45 & Shor tHair (P&P)

Age 46–60 & Suit Jacket (P&A) Male & Long Sleeves (P&A)

Female & Suit Pants (P&A) No Sunglasses & Shorts (A&A)

Sunglasses & Short Sleeves (A&A) Age 46–60 & Bald (P&P)

Long Sleeves & Suit Pants (A&A) No Sunglasses & Long Sleeves (A&A)

Age 31–45 & Long Hair (P&P) Female & Age 31–45 (P&P)

Female & Suit Jacket (P&A) Age 31–45 & Suit Pants (P&A)

Age 16–30 & Short Sleeves (P&A) Female & Long Sleeves (P&A)

Short Hair & Skirt (P&A) Male & Age Above 60 (P&P)

Male & Suit Jacket (P&A) Age 16–30 & Suit Jacket (P&A)

Age 46–60 & Long Hair (P&P) Male & Short Hair (P&P)

Female & Shorts (P&A) Shorts & Short Sleeves (A&A)

Age 46–60 & Long Sleeves (P&A) Sunglasses & Female (P&A)

Female & Age 46–60 (P&P) Long Hair & Shorts (P&A)

Age 16–30 & Suit Pants (P&A) Female & Short Hair (P&P)

No Sunglasses & Suit Pants (A&A) Long Hair & Suit Pants (P&A)

Female & Age 16–30 (P&P) Suit Jacket & Short Sleeves (A&A)

No Sunglasses & Suit Jacket (A&A) Age 31–45 & Long Sleeves (P&A)

Skirt & Long Sleeves (A&A) Sunglasses & Age 31–45 (P&A)

Age 16–30 & Shorts (P&A) Age 16–30 & Short Hair (P&P)

Sunglasses & Long Sleeves (A&A) Long Hair & Skirt (P&A)

Male & Short Sleeves (P&A) Age 46– 60 & Short Hair (P&P)

Long Hair & Long Sleeves (P&A) Age Above 60 & Short Sleeves (P&A)

Bald & Long Sleeves (P&A) Age 31–45 & Short Sleeves (P&A)

Male & Suit Pants (P&A) Female & Short Sleeves (P&A)

No Sunglasses & Age 46–60 (P&A) Female & Long Hair (P&P)

Sunglasses & Male (P&A) Suit Jacket & Suit Pants (A&A)

Age Above 60 & Bald (P&P) No Sunglasses & Female (P&A)

No Sunglasses & Skirt (A&A) Male & Age 31–45 (P&P)

Age Above 60 & Suit Jacket (P&A) Age 46–60 & Suit Pants (P&A)

Short Hair & Suit Jacket (P&A) Long Hair & Short Sleeves (P&A)

No Sunglasses & Age 31–45 (P&A) Sunglasses & Age 16–30 (P&A)

Age Above 60 & Long Hair (P&P) No Sunglasses & Age 16–30 (P&A)

Age 31-45 & Skirt (P&A) Long Hair & Suit Jacket (P&A)

No Sunglasses & Short Hair (P&A) Male & Age 46–60 (P&P)

Male & Bald (P&P) No Sunglasses & Short Sleeves (A&A)

Male & Shorts (P&A) Age 16–30 & Skirt (P&A)

Sunglasses & Long Hair (P&A) No Sunglasses & Age Above 60 (P&A)

No Sunglasses & Bald (P&A) Age 31–45 & Suit Jacket (P&A)

No Sunglasses & Long Hair (P&A) Age Above 60 & Long Sleeves (P&A)

Age 16–30 & Long Hair (P&P) No Sunglasses & Male (P&A)

Age 16–30 & Long Sleeves (P&A) Short Hair & Long Sleeves (P&A)

Shorts & Long Sleeves (A&A) Short Hair & Shorts (P&A)

Table III shows the matching rates and nAUCs for thesefour experiments. The combination of physical and adheredhuman characteristics increased the performance of the personre-identification method (the results for P1 were inferior tothe others). The method using all the combinations (P4)produced the best results. These results demonstrate thatthe co-occurrence attributes of physical and adhered humancharacteristics developed using our method can successfullyre-identify people in images.

2089

TABLE III. COMPARISON OF MATCHING RATES AND NAUC TOEVALUATE THE EFFECTIVENESS OF THE DIFFERENT COMBINATIONS.

Plan n = 1 n = 5 n = 10 n = 20 nAUC

P1 7.9 24.5 37.4 54.1 89.2

P2 9.9 29.5 42.1 58.6 90.9

P3 11.6 30.6 44.0 60.5 91.4

P4 11.3 32.0 45.6 63.9 91.6

No. eigenvalues of single attributes No. eigenvalues of co-occurrence attributes 1

0.5

1.0

0.0

0.5

1.0

0.0

0.5

1.0

0.0

0.5

1.0

0.0

Cum

ula

tive

contr

ibution r

atio

Cum

ula

tive

contr

ibution r

atio

Cum

ula

tive

weig

ht

ratio

Cum

ula

tive

weig

ht

ratio

No. elements of the 1st eigenvector of single attributes

No. elements of the 1st eigenvector of co-occurrence attributes

15 1 96

1 15 1 96

Fig. 6. Comparing the cumulative contribution ratio and the cumulativeweight ratio curves between co-occurrence and single attributes.

B. Analysis of the metric matrix of LMNN

1) Analysis algorithm: To determine the most valuableco-occurrence attributes, we investigated a metric matrix Mtrained using LMNN. The matrix M is positive-semidefiniteand is represented as

M =

N∑i=1

λiqiqiT , (4)

where λi is the i-th eigenvalue and qi is the i-th eigenvector.The eigenvectors corresponding to larger eigenvalues have themost important role when computing the distance betweenfeature vectors (Equation (3)). We compute the cumulativecontribution ratio Cm from λ1 to λm using

Cm =

m∑k=1

λk∑Ni=1 λi

. (5)

To determine the impact of the l-th element qi,l in eigenvec-tor qi when computing the distance between feature vectors,we compute the weight ratio wi,l, i.e.,

wi,l =|qi,l|∑Nj=1 |qi,j |

, (6)

and compute the cumulative weight ratio Wi,l from wi,1 towi,l, i.e.,

Wi,l =

l∑k=1

wi,k . (7)

Note that wi,k (k ∈ 1, ..., l) are in descending order.

TABLE IV. COMPARING THE IMPORTANCE OF ELEMENTS WHENCOMPUTING THE DISTANCE.

Co-occurrence attributes1st eigenvector

Age 31–45 & Shorts (P&A)

Suit Jacket & Long Sleeves (A&A)

Short Hair & Suit Pants (P&A)

Short Hair & Short Sleeves (P&A)

Age 46–60 & Suit Jacket (P&A)

Female & Suit Pants (P&A)

Long Sleeves & Suit Pants (A&A)

Age 31–45 & Long Hair (P&P)

Co-occurrence attributes2nd eigenvector

Short Hair & Shorts (P&A)

Female & Age 31–45 (P&P)

No Sunglasses & Age 31–45 (P&A)

Male & Age 46–60 (P&P)

Age 31–45 & Shorts (P&A)

Male & Suit Pants (P&A)

Age 16–30 & Long Hair (P&P)

Bald & Short Sleeves (P&A)

Co-occurrence attributes3rd eigenvector

Age 16–30 & Suit Jacket (P&A)

Female & Skirt (P&A)

No Sunglasses & Suit Jacket (A&A)

Male & Age Above 60 (P&P)

Female & Age Above 60 (P&P)

Age 31–45 & Short Sleeves (P&A)

Single attribute1st eigenvector

Short Hair (P)

2) Analysis results: We analyzed the matrix of co-occurrence attributes (MCA) and the matrix of single at-tributes (MSA). We investigated the elements of eigenvectorsunder the condition that the cumulative contribution ratiosand cumulative weight ratios of MCA and MSA were nearlyidentical. Both matrices were trained using images of 632individuals from the VIPeR dataset. We selected eigenvectorscorresponding to the eigenvalues when Cm = 0.4: the 1st, 2ndand 3rd eigenvectors of MCA and the 1st eigenvector of MSA.We selected elements of the eigenvectors for Wi,n = 0.2:eight elements of the 1st eigenvector of MCA, eight elementsof the 2nd eigenvector of MCA, six elements of the 3rdeigenvector of MCA, and one element of the 1st eigenvectorof MSA. Figure 6 shows the cumulative contribution ratio andthe cumulative weight ratio curves of MCA and MSA.

Table IV shows the informative elements of the eigenvec-tors of MCA and MSA. There were 12 co-occurrence attributescontaining age and seven co-occurrence attributes containinggender. Age and gender are indeed important for person re-identification, because these physical characteristics are alsoused in public websites [11]–[14]. There were six, three, and13 co-occurrence P&P, A&A, and P&A attributes in Table IV,respectively. Three co-occurrence attributes contained shorthair. These results show that an effective characteristic appearsin both co-occurrence and single attributes. We believe that ourmethod can effectively determine the characteristics used whenpeople recognize each other.

C. Evaluation using public datasets

We evaluated the performance of our method using the3DPeS, CAVIAR4REID, GRID, i-LID, PRID, SARC3D, andTownCentre datasets. We used test samples for person re-identification, training samples for LMNN, and training sam-ples for the attribute classifiers with the experimental con-ditions described in Section V-A1. Table V shows the n-th matching rates and nAUC for the person re-identificationresults. We can see that our method using co-occurrenceattributes is superior to the existing method [10] using single

2090

TABLE V. COMPARING MATCHING RATE AND NAUC OF THE PERSONRE-IDENTIFICATION RESULTS USING A NUMBER OF PUBLIC DATASETS.

Dataset Method n = 1 n = 5 n = 10 n = 20 nAUC

3DPeS CA 20.0 39.8 51.0 65.0 78.6SA 14.1 31.2 42.2 56.9 74.2

CAVIAR CA 24.7 55.8 72.3 90.1 81.74REID SA 15.6 40.7 59.2 80.2 72.9

GRID CA 18.9 46.4 65.1 80.3 89.9SA 10.1 30.9 45.2 62.5 81.9

i-LID CA 20.9 46 60.7 77.6 81.2SA 10.7 32.8 48.4 68.3 73.7

PRID CA 8.9 28.5 44.2 61.3 78.7SA 4.1 19.5 31.4 50.6 71.3

SARC3D CA 60.9 94.5 98.9 99.5 96.5SA 50.0 90.3 98.4 99.9 94.7

Town CA 32.3 49.9 57.8 67.6 82.5Centre SA 18.4 35.1 44.2 55.1 74.6

attributes, on the all datasets. This significant improvement inthe matching performance demonstrates the effectiveness ofthe co-occurrence attributes of combinations of physical andadhered human characteristics.

VI. CONCLUSION

We proposed a method for person re-identification thatuses co-occurrence attributes. To extract invariant features fromimages of people, we introduced a combination of physical andadhered human characteristics. The contributions of this workcan be summarized as follows.

• We designed co-occurrence attributes using physicaland adhered human characteristics, which were basedon prior knowledge of how people recognize eachother.

• We analyzed a metric matrix trained using a metriclearning technique to determine the most importantco-occurrence attributes.

In the future, we will extend our method by consideringcombinations of three or more characteristics. We also intendto combine low-level features with our method.

REFERENCES

[1] C. Nakajima, M. Pontil, B. Heisele, and T. Poggio, “Full-body personrecognition system,” Proceedings of Pattern recognition, vol. 36, no. 9,pp. 1997–2006, 2003.

[2] X. Wang, G. Doretto, T. Sebastian, J. Rittscher, and P. Tu, “Shape andappearance context modeling,” Proceedings of International Conferenceon Computer Vision, pp. 1–8, 2007.

[3] M. Farenzena, L. Bazzani, A. Perina, V. Murino, and M. Cristani,“Person re-identification by symmetry-driven accumulation of localfeatures,” Proceedings of Computer Vision and Pattern Recognition,pp. 2360–2367, 2010.

[4] D. Gray and H. Tao, “Viewpoint invariant pedestrian recognition with anensemble of localized features,” Proceedings of European Conferenceon Computer Vision, pp. 262–275, 2008.

[5] L. Bourdev, S. Maji, and J. Malik, “Describing people: A poselet-based approach to attribute classification,” Proceedings of InternationalConference on Computer Vision, pp. 869–878, 2011.

[6] S. Khamis, C. Kuo, V. Singh, V. Shet, and L. Davis, “Joint learningfor attribute-consistent person re-identification,” Proceedings of ECCVWorkshop on Visual Surveillance and Re-Identification, pp. 134–146,2014.

[7] A. Li, L. Liu, K. Wang, S. Liu, and S. Yan, “Clothing attributes assistedperson reidentification,” IEEE Transactions on Circuits and Systems forVideo Technology, pp. 134–146, 2015.

[8] J.W.Davis, “Visual categrization of children and adult walking styles,”Proceedings of ACM Multimedia, pp. 295–300, 2001.

[9] Z. Shi, T. Hospedales, and T. Xiang, “Transferring a semantic represen-tation for person re-identification and search,” Proceedings of the IEEEConference on Computer Vision and Pattern Recognition, pp. 4184–4193, 2015.

[10] R. Layne, T. Hospedales, and S. Gong, “Person re-identification,”chapter Attributes-Based Re-identification, pp. 93–117, 2014, Springer.

[11] “Wanted,” http://www.keishicho.metro.tokyo.jp/foreign/submenu.htm,Metropolitan Police Department.

[12] “Most wanted,” http://www.scotland.police.uk/whats-happening/most-wanted/, Police Scotland.

[13] “Kidnappings & missing persons,” https://www.fbi.gov/wanted/kidnap,Federal Bureau of Investigation.

[14] “Missing persons,” http://www.scotland.police.uk/whats-happening/missingpersons/james-barr, Police Scotland.

[15] N. Kumar, A. Berg, P. N. Belhumeur, and S. Nayar, “Describable visualattributes for face verification and image search,” IEEE Transactions onPattern Analysis and Machine Intelligence, vol. 33, no. 10, pp. 1962–1977, 2011.

[16] L. Bourdev, S. Maji, and J. Malik, “Describing people: A poselet-based approach to attribute classification,” Proceedings of InternationalConference on Computer Vision, pp. 1543–1550, 2011.

[17] J. Roth and X. Liu, “On the exploration of joint attribute learningfor person re-identification,” Proceedings of 12th Asian Conference onComputer Vision, pp. 673–688, 2014.

[18] A. Dantcheva, C. Velardo, A. Dfangelo, and J. Dugelay, “Bag of softbiometrics for person identification,” Multimedia Tools and Applica-tions, vol. 51, no. 2, pp. 739–777, 2011.

[19] S. Liao and S. Li, “Efficient psd constrained asymmetric metric learningfor person re-identification,” Proceedings of International Conferenceon Computer Vision, pp. 11–18, 2015.

[20] K. Weinberger, J. Blitzer, and L. Saul, “Distance metric learningfor large margin nearest neighbor classification,” Journal of MachineLearning, pp. 207–244, 2009.

[21] Y. Deng, P. Luo, C. Loy, and X. Tang, “Pedestrian attribute recognitionat far distance,” Proceedings of ACM Multimedia, pp. 3–7, 2014.

[22] D. Baltieri, R. Vezzani, and R. Cucchiara, “3dpes: 3d people datasetfor surveillance and forensics,” Proceedings of joint ACM workshop onHuman gesture and behavior understanding, pp. 59–64, 2011.

[23] D. Cheng, M. Cristani, M. Stoppa, L. Bazzani, and V. Murino, “Custompictorial structures for re-identification,” Proceedings of British MachineVision Conference, pp. 68.1–68.11, 2011.

[24] W. Ouyang and X. Wang, “A discriminative deep model for pedestriandetection with occlusion handling,” Proceedings of Computer Visionand Pattern Recognition, pp. 3258–3265, 2012.

[25] C. Loy, T. Xiang, and S. Gong, “Time-delayed correlation analysis formulti-camera activity understanding,” International Journal of Com-puter Vision, vol. 90, no. 1, pp. 106–129, 2010.

[26] T. Wang, S. Gong, X. Zhu, and S. Wang, “Person re-identificationby video ranking,” Proceedings of European Conference on ComputerVision, pp. 688–703, 2014.

[27] C. Papageorgiou and T. Poggio, “A trainable system for object detec-tion,” International Journal of Computer Vision, pp. 15–33, 2000.

[28] M. Hirzer, C. Beleznai, P. M. Roth, and H. Bischof, “Person re-identification by descriptive and discriminative classification,” 2011, pp.91–102.

[29] D. Baltieri, R. Vezzani, and R. Cucchiara, “Sarc3d: a new 3d bodymodel for people tracking and re-identification,” 2011, pp. 197–206.

[30] B. Benfold and I. Reid, “Stable multi-target tracking in real-timesurveillance video,” 2011, pp. 3457–3464.

[31] D. Gray, S. Brennan, and H. Tao, “Evaluating appearance models forrecognition, reacquisition, and tracking,” Proceedings of PerformanceEvaluation of Tracking and Surveillance, pp. 1–7, 2007.

2091