Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
THECURSEOFDIMENSIONALITYANDIMAGERECOGNITION
BRANDONEDWARDS
OUTLINE
• Imageclassification• Worst-casetestimages(adversarialexamples)• Defenseagainstadversarialattacks
• ’AdversarialSpheres’,JustinGilmer,LukeMetz,Fartash Faghri,SamSchoenholz,
Maithra Rughu,MartinWattenberg,IanGoodfellow(2018)ICLRPaper
• Relevancetoadversarialexamplesinimageclassification
IMAGECLASSIFICATION
IMAGECLASSIFICATION
ImageSource:“ImageNetClassificationwithDeepConvolutionalNeuralNetworks”,AlexKrizhevsky,IlyaSutskever,GeoffreyE.Hinton,2012
• ImageNet(ILSVRC):1000classes;training-1.2million,validation-50k,test-150k• ~83%successforgroundtruthinthetop5classes• Currenttop5performance>95%
MODELFUNCTION
ModelFunction𝑓:ℝ%& → ℝ( n=#pixels,m=#classes
ADVERSARIALEXAMPLES‘CLOSE’IMAGESTHATCLASSIFY‘INCORRECTLY’
DIGITALATTACK
ImageSource:‘ExplainingandHarnessingAdversarialExamples’,ICLR2015,Goodfellow,Shlens,Szegedy
• AttackaboveisonGoogLeNet (ImageNet)(>94%top5accuracy).• Theperturbationisclearlysmallbyhumanstandards.
• Digitaladversarialattackscenario:Phishingdetection
PHYSICALATTACKImageSource:“AccessorizetoaCrime:RealandStealthyAttacksonState-of-the-ArtFaceRecognition”,MahmoodSharif,Sruti Bhagavatula,Lujo Bauer,MichaelK.Reiter;CCS2016
• Attackagainstpre-trainedfacialrecognitionmodel• 88%ofimageswithglassesclassifiedasMilla Jovovich• Meanconfidencewas78%.
• Thisperturbationislarger,butwoulditberaisesuspicion?
DEFENSEAPPROACHESLearnthe(distributional)differencebetweenadversarialexamplesand‘natural’data.
• Preprocessing(removingperturbation)– JPEG,neuralnetworkde-noiser
• Detectionofadversarialexamples
Imposeconstraintsonmodelfunctiontolimitlocalchanges
• RegularizationorLipschitzconstraints
Consideradversarialexamplesduringtraining
• Adversarialtraining
Improvemodelinotherways
• Capsulenetworks?
ADVERSARIALSPHERESPAPER
ADVERSARIALSPHERESPAPER
’AdversarialSpheres’,JustinGilmer,LukeMetz,Fartash Faghri,SamSchoenholz, MaithraRughu,MartinWattenberg,IanGoodfellow (2018)ICLR
• Simpleclassificationtask
• Experimentalmodelresults
• Theoreticalresultsrelatingmodelaccuracyandproximityofadversarialexamples.
CLASSIFICATIONTASKANDEXPERIMENTALRESULTS
• Twospherescenteredattheorigininℝ)(R=1andR=1.3).• Trainanartificialneuralnetwork.• ModelInput:Apointinℝ)• ModelOutput:”probability”ofbeingclosertotheinnersphere
• ExperimentalFocusond=500• Trainonpointsuniformlysampledfrombothspheres• Testonpointsuniformlysampledfromtheinnersphere• Hightestaccuracy,butcloseadversarialexamplesremain
THEORETICALRESULT
• Non-zeroerrorimpliesarbitrarilycloseadversarialexamplesforlargeenoughdimensiond.
• Thm:Forlargeenoughdimensiond,theaveragedistancetoamiss-classificationontheinnerspherecanbeboundedabovebyafunctionofonlytheerrorrateofthemodel(ontheinnersphere)andd.Forafixednon-zeroerrorrate,thisfunctionisO(1/ 𝑑� ).
Thm:Forlargeenoughdimensiond,theaveragedistancetoamiss-classificationontheinnerspherecanbeboundedabovebyafunctionofonlytheerrorrateofthemodel(ontheinnersphere)andd.Forafixednon-zeroerrorrate,thisfunctionisO(1/ 𝑑� ).
SketchofProof
LetEbethesetofmisclassifiedpoints,so𝜇 𝐸 = 𝑞 > 0 [𝜇 𝑆8 = 1].
Let𝑑 𝐸 = 𝔼;~=>𝑑(𝑥, 𝐸) (averagedistancetoE).
Maximum𝑑 𝐸 occursfora”cap”(intersectionof𝑆8 withahalf-space)[Figiel et.al.1977].
Thm:Forlargeenoughdimensiond,theaveragedistancetoamiss-classificationontheinnerspherecanbeboundedabovebyafunctionofonlytheerrorrateofthemodel(ontheinnersphere)andd.Forafixednon-zeroerrorrate,thisfunctionisO(1/ 𝑑� ).
SketchofProof(Continued)
(larged)anycoordinateon𝑆8 hasdistribution:𝑁(0,1/𝑑).– [Poincaré\Lévy]
Asmallbandaroundan“equator”containsthemajorityofspherevolume.Thecapboundaryisthusclosetotheequator– wherethemajorityofpointslie.
ADDITIONALTHOUGHTS
Adversarialproblemisworse
Theasymptoticresultsignoretheadversarialexamplesthatmaybefoundofftheinnersphere.
ModelErrorasafunctionofNumberofTrainingPoints
Learnmodelsthatusestheradiusofpoints- perfectmodelsforALLdimensions.I.E.Domainspecificfeaturesmayprovidethelowerrordesired.
LESSONSFORIMAGECLASSIFICATION?
• Insightintocurrentdefenseideas• Caution:Adversarialexamplescouldlieonthedatadistribution.
• Confirmation:Lipschitzconstraintsandadversarialtrainingwouldhelphere
• Imagesmaybedifferent:shapeofindividualclassdistributions
• Couldbebetterinsomeways,worseinothers
• Ex:𝐵C×𝐼)FC forsmallk.Betterforin-distributionexamples,butmoresurfaceareacouldallowmoreoff-surface?
SUMMARY
• Thespherestoyproblemprovidesinsightrelatedtocurrentadversarialimagedefensetechniques.
• LargedimensionsCANleadtoverystricterrorrequirementsinordertoavoidclose‘adversarialexamples’.
• Domainspecificlowdimensionalfeaturecreationorotherconstraintscouldprovidethelowerrorneededto‘pushoff’adversarialexamplesfortheaveragetestpoint.
THANKYOU
NON-ZEROERRORIMPLIESARBITRARILYCLOSEADVERSARIALEXAMPLESFORLARGEENOUGHDIMENSIONS
Proof:
• LetEbethesetofmiss-classifiedpointsontheinnersphere,witherrorrate𝜇(𝐸) =q.ForcalculatinganupperboundwereplaceEwitha“cap”𝐸′ with𝜇 𝐸′ = 𝜇(𝐸).Withoutlossofgenerality,weassume:
E′ = {𝑥 ∈ 𝑆8: 𝑥K > 𝛽/ 𝑑� } forsome𝛽 > 0.
• Thenq = 𝜇 𝐸′ ≅ ℙ 𝑁 0, K)> Q
)�= ℙ 𝑁 0,1 > 𝛽 = 1 − Φ(𝛽) ,where
Φ isthestandardnormalcdf.
NON-ZEROERRORIMPLIESARBITRARILYCLOSEADVERSARIALEXAMPLESFORLARGEENOUGHDIMENSIONS
Proof(Continued):
• Thus𝛽 = ΦFK(1 − 𝑞).
• Notethat𝑑 𝐸′ : = 𝔼;~=>𝑑 𝑥, 𝐸′ ≤ 𝔼 max 2� Q)�− 𝑁 0, K
), 0
= 𝑂(Z[\(KF])
)�).
• Finally,forfixedqwehave𝑑 𝐸 ≤ 𝑑 𝐸^ = 𝑂( K)�).
TWOEXPERIMENTALMODELSPiecewiseLinearModel
• TwolinearlayernetworkwithReLU activations.Mini-batchstochasticgradientdescentwasusedwithbatchsize50.Batchnormalizationwasperformedatthetwohiddenlayers.
QuadraticModel(Ellipsoidal)
• Singlelayernetworkwith1000hiddenunits,andactivationfunction:𝜎 𝑥 = 𝑥`.• Arotationalchangeofvariablesontheinputspaceallowsthedecisionboundarytobe
expressedas∑ 𝛼c)cdK 𝑥c`.
• The{𝛼c} determinetheprincipalaxislengthsoftheellipsoiddecisionboundary.
• Themodelisaperfectclassifierifandonlyif𝛼c ∈ (1/𝑅`, 1) forall𝑖.
• Analyticalerrorestimates(usingtheCentralLimitTheorem)arecalculatedusingthe𝛼cvalues.
STRUCTUREDFEATUREDETECTION
• INPUT:Rawpixelvalues(2-Darray)structure.
• INTERMEDIATEFEATUREVARIABLES:
• LocalFeatures:Edges,Textures,…,Ears,Eyes,...
• GlobalFeatures:Face,Body,…
• Keyoperations:Convolutions, Down-Sampling,Up-Sampling,…
• Built-ininvariance:Shift,Scale,…
• FINALLAYERS:Usefeaturevaluestocomputeclassconfidencevalues.
ModelFunction𝑓:ℝ%& → ℝ( n=#pixels,m=# classes
TWOTRAININGMODES
• [Online](makesurematchestheirstatements)Uniformlysamplefromtheinnerandoutsphereforeachnewtrainingpoint.
• [Batch]UniformlysamplefromtheinnerandoutsphereforNpointseach.Iterateoverthese2Npointsrepeatedlyduringtraining.
FORLARGEDIMENSIONS,CLOSEADVERSARIALEXAMPLESAREFOUND.
• Piecewiselinearmodel(Onlinetraining– 25millionpointspersphere)withd=500.Noerrorwasobservedin10milliontestpoints.
• Note:Volumeofthismisclassifiedspaceontheinnersphereissmall!!!
• Note:d=60wastheobservedpointwheretheexperimentabovestartedtohaveadversarialexamples.
ALINEARINCREASEINADVERSARIALDISTANCEREQUIRESANEXPONENTIALDECREASEINERRORRATE
Experimentalmodelerror
rateestimates,𝑞 v.s.𝑑(𝐸).
Theupperbound𝑑(𝐸^) is
thesolidblackplot.
ImageSource
`AdversarialSpheres’ICLR2018
MOREINFOONQUADRATICNETWORK
• Singlelayernetworkwith1000hiddenunits,andactivationfunction:𝜎 𝑥 = 𝑥`.– Thedecisionboundarywillbeanellipsoid.
• Arotationalchangeofvariablesontheinputspaceallowsthedecisionboundarytobeexpressedas∑ 𝛼c)
cdK 𝑥c`.
• The{𝛼c} determinetheprincipalaxislengthsoftheellipsoiddecisionboundary.
• Themodelisaperfectclassifierifandonlyif𝛼c ∈ (1/𝑅`, 1) forall𝑖.
• Analyticalerrorestimates(usingtheCentralLimitTheorem)arecalculatedusingthe𝛼cvalues.
QUADRATICMODELOBSERVATIONS
• A rotationalchangeofvariablesontheinputspaceallowsthedecisionboundarytobeexpressedas∑ 𝛼c)
cdK 𝑥c`.
• Themodelisaperfectclassifierifandonlyif𝛼c ∈ (1/𝑅`, 1) forall𝑖.
Online:50milliontrainingpoints(samesetupfromReLU experiment)
All𝛼c wereinrangeforaperfectclassifier– ie perfectclassifier
QUADRATICMODELOBSERVATIONSBatch:batchsize1million
Noerrorsin20milliontestpoints
Adversarialexamplesarefound
394/500𝛼c areoutofrange.
Withhighprobability,the
effectsofthebad𝛼c cancel
eachotherout.
ImageSource:`AdversarialSpheres’ICLR2018