Veriﬁkacijasoftvera · Uvod Konkoliˇckoizvr ˇsavanjeiprincipidizajna Strategijeobilaskaputeva Literatura Simboliˇckoizvr ˇsavanjekrozprimer Istorija,alati,stabloizvrˇsavanja

Verifikacija softvera— Simbolicko izvrsavanje (prvi deo) —

Milena Vujosevic Janicic

www.matf.bg.ac.rs/~milena

Matematicki fakultet, Univerzitet u Beogradu

www.matf.bg.ac.rs/~milena

UvodKonkolicko izvrsavanje i principi dizajna

Strategije obilaska putevaLiteratura

Pregled

1 Uvod

2 Konkolicko izvrsavanje i principi dizajna

3 Strategije obilaska puteva

4 Literatura

Milena Vujosevic Janicic Verifikacija softvera



Simbolicko izvrsavanje kroz primerIstorija, alati, stablo izvrsavanjaPrimene simbolickog izvrsavanjaIzazovi simbolickog izvrsavanja

Pregled

1 UvodSimbolicko izvrsavanje kroz primerIstorija, alati, stablo izvrsavanjaPrimene simbolickog izvrsavanjaIzazovi simbolickog izvrsavanja



4 Literatura





Konkretno izvrsavanje





Simbolicko izvrsavanje











Neformalno...Izvrsavamo program sa simbolima, tj pratimo simbolicka stanjaumesto konkretnih ulazaIzvrsavamo puno putanja simultano, kada izvrsavanje nekeputanje divergira, pravimo nove putanje i dodajemo uslove nadsimbolickim vrednostimaKada izvrsavamo jednu putanju zapravo simuliramo veliki brojtestova s obzirom da razmatramo sve ulaze koji prolaze kroz tuistu putanju





Poreklo ideje...

Tehnika nastala jos 70tih godina proslog vekaNaucni rad koji je najvise citiran: James C. King. 1976.Symbolic execution and program testing. Commun. ACM 19,7 (July 1976), 385-394.DOI=http://dx.doi.org/10.1145/360248.360252https://yurichev.com/mirrors/king76symbolicexecution.pdf


DOI=http://dx.doi.org/10.1145/360248.360252

https://yurichev.com/mirrors/king76symbolicexecution.pdf

https://yurichev.com/mirrors/king76symbolicexecution.pdf




Poreklo ideje...

Ima i drugih slicnih radova iz tog periodaRobert S. Boyer, Bernard Elspas, and Karl N. Levitt. 1975.SELECT — a formal system for testing and debuggingprograms by symbolic execution. In Proceedings of theinternational conference on Reliable software. ACM, New York, NY,USA, 234-245. DOI=http://dx.doi.org/10.1145/800027.808445

Lori A. Clarke. 1976. A program testing system. In Proceedingsof the 1976 annual conference (ACM ’76). ACM, New York, NY,USA, 488-491. DOI=http://dx.doi.org/10.1145/800191.805647

Leon J. Osterweil and Lloyd D. Fosdick. 1976. Program testingtechniques using simulated execution. In Proceedings of the 4thsymposium on Simulation of computer systems (ANSS ’76), HaroldJoseph Highland (Ed.). IEEE Press, Piscataway, NJ, USA, 171-177.







Izazovi

Tek od 2005 — prakticna upotreba simbolickog izvrsavanjaU trenutku nastanka ideje, nije bilo jasno kako resiti osnovneprobleme koji su se odmah javili. Proboj su napravili alati

DART Godefroid and Sen, PLDI 2005 (uvodenje dinamickogizvrsavanja u simbolicko izvrsavanje)EXE Cadar, Ganesh, Pawlowski, Dill, and Engler, CCS 2006(STP: podrska za teoriju nizova)





Alati

AlatiKLEE (Stanford) — Open source, runs on top of LLVM, Hasfound lots of problems in open-source softwareSAGE — Microsoft internal tool, Symbolic execution to findbugs in file parsers - E.g., JPEG, DOCX, PPT, etc.Cloud9 parallel symbolic execution, also supports threadsPex symbolic execution for .NETCUTE (UC Berkeley) and jCUTE (symbolic execution for Java)Java PathFinder (NASA) - symbolic executionS2E (EPFL) — LLVM based platformSymDroid - symbolic execution on Dalvik BytecodeKleenet - testing interaction protocols for sensor network





Alati






Stablo izvrsavanjaIzvrsavanje programa nad simbolickim vrednostima.Simbolicka stanja preslikavaju promenljive u simbolickevrednosti.Uslov putanje (path condition) je formula (bez kvantifikatora)nad simbolickim ulazima koja sadrzi sve odluke koje su do teprilike doneteSve putanje programa formiraju stablo izvrsavanja





Simbolicko stablo izvrsavanja










Primer











Primena simbolickog izvrsavanjaPronalazenje gresaka

In Microsoft, 30% of bugs were discovered by symbolicexecution during the development of Windows 7, (these werebugs that other program analyses and blackbox testingtechniques missed)Symbolic execution is the key technique used in DARPAa

Cyber Grand Challengeb.

Generisanje test primeraOtkrivanje nedostiznih putanja

aDefense Advanced Research Projects AgencybA two-year competition seeking to create automatic systems for

vulnerability detection, exploitation, and patching in near real-time






Primena simbolickog izvrsavanja

Proving two code segments are equivalent (Code Hunthttps://www.codehunt.com/)Advanced applications:

Generating program invariantsProgram repairDebugging


https://www.codehunt.com/




Pronalazenje gresaka





Generisanje test primera





Nedostupne/nedostizne putanje





Nedostupne/nedostizne putanje





Teorija i praksa

Teorija i praksa simbolickog izvrsavanjaA symbolic execution of a program can generate – in theory – allpossible control flow paths that the program could take during itsconcrete executions on specific inputs. While modeling allpossible runs allows for very interesting analyses, it istypically unfeasible in practice, especially on real-worldsoftware. Indeed, complex applications are often built on top ofvery sophisticated software stacks. Implementing a symbolicexecution engine able to statically analyze the whole stack can berather challenging given the difficulty in accurately evaluating anypossible side effect during execution. Several problems arise in thiscontext, which can hardly be faced following the purely symbolicapproach.





Izazovi simbolickog izvrsavanja

Eksplozija stanjaState space explosion: how does symbolic execution deal withpath explosion? Language constructs such as loops mightexponentially increase the number of execution states. It isthus unlikely that a symbolic execution engine can exhaustivelyexplore all the possible states within a reasonable amount oftime.





Path Explosion






Modelovanje hipa i rezonovanje o pokazivacimaMemory: how does the symbolic engine handle pointers,arrays, or other complex objects? Code manipulating pointersand data structures may give rise not only to symbolic storeddata, but also to addresses being described by symbolicexpressions.






Modelovanje okoline (bibliotecke funkcije, sistemski pozivi)

Environment and third-party components: how does theengine handle interactions across the software stack? Calls tolibrary and system code can cause side-effects, e.g., thecreation of a file, that could later affect the execution andmust be accounted for. However, evaluating any possibleinteraction outcome may be unfeasible.






Efikasni resavaci — SMTConstraint solving: what can a constraint solver do in practice?SMT solvers can scale to complex combinations of constraintsover hundreds of variables. However, constructs such asnon-linear arithmetic pose a major obstacle to efficiency.





Unutrasnjost alata sa simbolicko izvrsavanje





Unutrasnjost alata sa simbolicko izvrsavanje






Binarni kodBinary code: what issues can arise when symbolicallyexecuting binary code? In some scenarios binary code is theonly available representation of a program. However, havingthe source code of an application can make symbolic executionsignificantly easier, as it can exploit high-level properties (e.g.,object shapes) that can be inferred statically by analyzing thesource code.






Odluke se prave u zavisnosti od konteksta upotrebeDepending on the specific context in which symbolic execution isused, different choices and assumptions are made to address thehighlighted questions. Although these choices typically affectsoundness or completeness, in several scenarios a partial explorationof the space of possible execution states may be sufficient toachieve the goal (e.g., identifying a crashing input for anapplication) within a limited time budget.




Dinamicko simbolicko izvrsavanjeSelektivno simbolicko izvrsavanjePrincipi dizajna

Pregled

1 Uvod

2 Konkolicko izvrsavanje i principi dizajnaDinamicko simbolicko izvrsavanjeSelektivno simbolicko izvrsavanjePrincipi dizajna


4 Literatura





Konkretno i simbolicko izvrsavanje

Prakticni problemiExhaustive exploration of external library calls may lead to anexponential explosion of states, preventing the analysis fromreaching interesting code portions.Calls to external third-party components may not be traceableby the executor.Symbolic engines continuously invoke SMT solvers during theanalysis. The time spent in constraint solving is one of themain performance barriers for an engine, and programs mayyield constraints that even powerful solvers cannot handle well.





Konkretno i simbolicko izvrsavanje

Konkolicko izvrsavanje (konkretno+simbolicko)

A fundamental idea to cope with the aforementioned issues and tomake symbolic execution feasible in practice is to mix concrete andsymbolic execution, called concolic execution.

Vrste konkolickog izvrsavanjaDinamicko simbolicko izvrsavanjeSelektivno simbolicko izvrsavanje





Dinamicko simbolicko izvrsavanje

Simbolicko izvrsavanje vodeno konkretnim vrednostimaIn addition to the symbolic store and the path constraints, theexecution engine maintains a concrete store. After choosing anarbitrary input to begin with, it executes the program bothconcretely and symbolically by simultaneously updating the twostores and the path constraints. Whenever the concrete executiontakes a branch, the symbolic execution is directed toward the samebranch and the constraints extracted from the branch condition areadded to the current set of path constraints. In short, thesymbolic execution is driven by a specific concrete execution.As a consequence, the symbolic engine does not need to invoke theconstraint solver to decide whether a branch condition is(un)satisfiable: this is directly tested by the concrete execution.





Dinamicko simbolicko izvrsavanje

Simbolicko izvrsavanje vodeno konkretnim vrednostimaIn order to explore different paths, the path conditions given by oneor more branches can be negated and the SMT solver invoked tofind a satisfying assignment for the new constraints, i.e., togenerate a new input. This strategy can be repeated as much asneeded to achieve the desired coverage.





Dynamic Symbolic Execution

One popular concolic execution approach, where concrete execution drivessymbolic execution. Consider the code:

Hakjoo Oh AAA615 2017 Fall, Lecture 9 November 24, 2017 22 / 49













































































Izbor uslova za negiranje?

Tehnike pretrageAlthough dynamic symbolic execution uses concrete inputs to drivethe symbolic execution toward a specific path, it still needs to picka branch to negate whenever a new path has to be explored. Noticealso that each concrete execution may add new branches that willhave to be visited. Since the set of non-taken branches across allperformed concrete executions can be very large, adopting effectivesearch heuristics can play a crucial role.

DART — DFS (depth-first search strategy)SAGE — generacijska pretraga...





Izbor pocetnih vrednosti

Uticaj na pretraguSince the state space is only partially explored, the initial inputplays a crucial role in the effectiveness of the overall approach. Theimportance of the first input is similar to what happens intraditional black-box fuzzing and, for this reason, symbolic enginessuch as SAGE are often referred to as white-box fuzzers.





Kako pratiti funkcijske pozive za koje kod nije dostupan?






Uticaj 4aConsider function foo in Figure 4a and suppose that bar is notsymbolically tracked by the concolic engine (e.g., it could beprovided by a third-party component, written in a differentlanguage, or analyzed following a black-box approach). Assumingthat x = 1 and y = 2 are randomly chosen as the initial inputparameters, the concolic engine executes bar (which returns a = 0)and skips the branch that would trigger the error statement. At thesame time, the symbolic execution tracks the path constraintαy ≥ 0 inside function foo. Notice that branch conditions infunction bar are not known to the engine.






Uticaj 4aTo explore the alternative path, the engine negates the pathconstraint of the branch in foo, generating inputs, such as x = 1and y = -4, that actually drive the concrete execution to thealternative path. With this approach, the engine can explore bothpaths in foo even if bar is not symbolically tracked.






Uticaj 4bA variant of the previous code is shown in Figure 4b, wherefunction qux – differently from foo – takes a single input parameterbut checks the result of bar in the branch condition. Although theengine can track the path constraint in the branch condition testedinside qux, there is not guarantee that an input able to drive theexecution toward the alternative path is generated: the relationshipbetween a and x is not known to the concolic engine, as bar is notsymbolically tracked. In this case, the engine could re-run the codeusing a different random input, but in the end it could fail toexplore one interesting path in foo.






Uticaj 4cA related issue is presented by Figure 4c. Function baz invokes theexternal function abs, which simply computes the absolute value ofa number. Choosing x = 1 as the initial concrete value, theconcrete execution does not trigger the error statement, but theconcolic engine tracks the path constraint αx ≥ 0 due to the branchin baz, trying to generate a new input by negating it. However thenew input, e.g., x = -1, does not trigger the error statement due tothe (untracked) side effects of abs. In this case, after generating anew input the engine detects a path divergence: a concreteexecution that does not follow the predicted path. Interestingly, inthis example no input could actually trigger the error, but theengine is not able to detect this property.





Posledice

Simbolicko izvrsavanje moze da ima lazne negativne rezultate

As shown by the example, false negatives (i.e., missed paths) andpath divergences are notable downsides of dynamic symbolicexecution. Dynamic symbolic execution trades soundness forperformance and implementation effort: false negatives arepossible, because some program executions - and therefore possibleerroneous behaviours - may be missed, leading to a complete, butunder-approximate form of program analysis.

Osnovni pojmovi — ispitujemo ispravnsotSoundness — sve greske ce biti pronadeneCompleteness — ako se za nesto tvrdi da je greska, to ce stvarno ibiti greska





Selektivno simbolicko izvrsavanje S2E

Drugaciji pristup mesanju simbolickog i konkretnog izvrsavanjaA different approach to mix symbolic and concrete execution isbased on the observation that one might want to explore only somecomponents of a software stack in full, not caring about others.Selective symbolic execution carefully interleaves concreteand symbolic execution, while keeping the overall explorationmeaningful.

Suppose a function A calls a function B and the execution modechanges at the call site.





Od konkretnog prema simbolickom i nazad

Konkretno izvsavanje A, simbolicko BThe arguments of B are made symbolic and B is exploredsymbolically in full. B is also executed concretely and its concreteresult is returned to A. After that, A resumes concretely.





Od simbolickog prema konkretnom i nazad

Simbolicko izvrsavanje A, konkretno BThe arguments of B are concretized, B is executed concretely, andexecution resumes symbolically in A.

This may impact both soundness and completeness of the analysis.





Completeness

Lazni pozitivni rezultatiDa bi bili sigurni da simbolicko izvrsavanje preskace sve putanjekoje nisu dostizne u skladu sa izvrsenom konkretizacijom, tj da nebi imali lazno pozitivne rezultate, potrebno je skupiti svaogranicenja putanje koja je izvrsena u skladu sa odgovarajucomkonkretizacijom, sve bocne efekte koje je B napravila kao ipovratnu vrednost koju B proizvodi.





Soundness

Lazno negativni rezultatiKonkretizacija moze da uzrokuje da se u simbolickom izvrsavanjupreskoce grane koje su dostizne nakon sto se vratimo u A, sto mozeda dovede do lazno negativnih rezultata. Da bi to izbegli, skupljenaogranicenja se markiraju kao soft ogranicenja: kada god nekagrana, nakon vracanja u A bude obelezena kao nedostizna zbog softulsova, izvrsavanje radi bektreking i bira na drugi nacin argumenteza B. Za vodenje nove konkretizacije argumenata za B, u tomslucaju se koriste uslovi grana koji su skupljeni za B i biraju se onekonkretne vrednosti koje omogucavaju drugacije konkretnoizvrsavanje kroz B.





Principi dizajna

Napredak, ponavljanje posla i ponovno koriscenje rezultataProgress: the executor should be able to proceed for anarbitrarily long time without exceeding the given resources.Memory consumption can be especially critical, due to thepotentially gargantuan number of distinct control flow paths.Work repetition: no execution work should be repeated,avoiding to restart a program several times from its verybeginning in order to analyze different paths that might have acommon prefix.Analysis reuse: analysis results from previous runs should bereused as much as possible. In particular, costly invocations tothe SMT solver on previously solved path constraints shouldbe avoided.





Principi dizajna

Kako izabrati prioritete u dizajnu?Online izvrsavanjeOffline izvrsavanjeHibridno izvrsavanje (kombinacija prethodna dva)





Izbori dizajna

Online izvrsavanjeSymbolic executors that attempt to execute multiple pathssimultaneously in a single run – also called online — clone theexecution state at each input-dependent branch (KLEE, AEG,S2E )These engines never re-execute previous instructions, thusavoiding work repetition.However, many active states need to be kept in memory andmemory consumption can be large, possibly hindering progress.





Izbori dizajna

Offline izvrsavanjeOn the other side, work can be largely repeated, since each runusually restarts the execution of the program from the verybeginning.In a typical implementation of offline executors, runs areconcrete and require an input seed: the program is firstexecuted concretely, a trace of instructions is recorded, and therecorded trace is then executed symbolically.





Izbori dizajna

Offline izvrsavanjeOn the other side, work can be largely repeated, since each runusually restarts the execution of the program from the verybeginning.In a typical implementation of offline executors, runs areconcrete and require an input seed: the program is firstexecuted concretely, a trace of instructions is recorded, and therecorded trace is then executed symbolically.




Naivni pristupi, random strategijaIzvrsavanje vodeno pokrivenoscu kodaRazne strategijeIzvrsavanje unazad

Pregled

1 Uvod


3 Strategije obilaska putevaNaivni pristupi, random strategijaIzvrsavanje vodeno pokrivenoscu kodaRazne strategijeIzvrsavanje unazad

4 Literatura





Strategije obilaska puteva

Heuristike vodene ciljevimaSince enumerating all paths of a program can be prohibitivelyexpensive, in many software engineering activities related to testingand debugging the search is prioritized by looking at the mostpromising paths first. Among several strategies for selecting thenext path to be explored, we now briefly overview some of the mosteffective ones. We remark that path selection heuristics are oftentailored to help the symbolic engine achieve specific goals (e.g.,overflow detection). Finding a universally optimal strategyremains an open problem.





Najcesce strategije: DFS i BFS

Naivne tehnike: DFS i BFSTehnike zasnovane na strukturi koda

DFSDepth-first search (DFS) — expands a path as much aspossible before backtracking to the deepest unexplored branch.DFS is often adopted when memory usage is at a premium,but is hampered by paths containing loops and recursive calls.





Najcesce strategije: DFS i BFS

BFSBreadth-first search (BFS)— expands all paths in parallel.In spite of the higher memory pressure and of the long timerequired to complete the exploration of specific paths, sometools resort to BFS, which allows the engine to quickly explorediverse paths detecting interesting behaviors early.On the other hand, if the ultimate goal requires to fullyterminate the exploration of one or more paths, BFS may takea very long time.





Random strategija

Random path selectionKako se sprovodi random pretraga?

Ideja 1: Izaberi sledecu putanju za istrazivanje randommetodomIdeja 2: Random metodom restartuj pretragu ukoliko se nistanovo ne desava vec neko vremeIdeja 3: Kada imamo da istrazimo dve jednako prioritetneputanje, izaberi sledecu random...

Problem sa reprodukovanjem, pesudo-random se koristi i cuvase seed





Random strategija

Random path selectionKako se sprovodi random pretraga?

Ideja 1: Izaberi sledecu putanju za istrazivanje randommetodomIdeja 2: Random metodom restartuj pretragu ukoliko se nistanovo ne desava vec neko vremeIdeja 3: Kada imamo da istrazimo dve jednako prioritetneputanje, izaberi sledecu random...

Problem sa reprodukovanjem, pesudo-random se koristi i cuvase seed





Random strategija

Random path selectionKLEE assigns probabilities to paths based on their length andon the branch arity: it favors paths that have been exploredfewer times, preventing starvation caused by loops and otherpath explosion factors.





Izvrsavanje vodeno pokrivenoscu koda

Maksimizovati pokrivenostIzaberi putanju koja ce najverovatnije da izvrsi neku novuinstrukciju.

Pokusaj da posetis instrukcije koje ranije nisu bile izvrsavane.Ukoliko takve putanje nema, izaberi onu putanju u kojoj suinstrukcije izvrsavane najmanji broj puta.

Dobra osobina: greske su cesto u delovima programa koji se retkoizvrsavaju, a ova strategija pokusava da dopre svuda.





Izvrsavanje vodeno pokrivenoscu koda — stanja

Coverage optimize search......, discussed in KLEE, computes for each state a weight, which islater used to randomly select states. The weight is obtained byconsidering how far the nearest uncovered instruction is, whethernew code was recently covered by the state, and the state’s callstack.





Izvrsavanje vodeno pokrivenoscu koda — putanje

Subpath-guided search ...... attempts to explore less traveled parts of a program by selectingthe subpath of the control flow graph that has been explored fewertimes. This is achieved by maintaining a frequency distribution ofexplored subpaths, where a subpath is defined as a consecutivesubsequence of length n from a complete path. Interestingly, thevalue n plays a crucial role with respect to the code coverageachieved by a symbolic engine using this heuristic and no specificvalue has been shown to be universally optimal.





Shortest-distance symbolic execution

Shortest-distance symbolic executionShortest-distance symbolic execution does not target coverage, butaims at identifying program inputs that trigger the executionof a specific point in a program. The heuristic is based however,as in coverage-based strategies, on a metric for evaluating theshortest distance to the target point. This is computed as thelength of the shortest path in the inter-procedural control-flowgraph, and paths with the shortest distance are prioritized by theengine.





Generacijska pretraga (SAGE)

Generacijska pretragaHibrid DFSa i izvrsavanja vodenog pokrivenoscu koda

Generacija 0: Izvrsi random putanju do krajaGeneracija 1: Uzmi sve putanje iz generacije 0, negiraj jedanuslov tako da vodi do novog prefiksa putanje, nadji resenje teputanje i onda je izvrsi...Generacija N: slicno, samo sto se grananje vrsi u odnosu naputanju generacije N-1 (za biranje putanje se koristi heuristikapokrivenosti koda)





Kombinovana strategija

Pretraga istovremeno sa razlicitim algoritmimaIzvrsavaj vise razlicitih algoritama istovremeno i alternirajizmedu njihZavisi od uslova koji su potrebni da se pronade greska u kodu,ponasa se kao najbolji od njih, sa konstantnim faktoromizgubljenog vremena i memorije sa svim ostalim algoritmimaMogu se koristiti razliciti algoritmi da bi se doslo do razlicitihdelova programa





Symbolic Backward Execution (SBE)

Izvrsavanje unazadSimbolicko izvrsavanje unazad je varijanta simbolickogizvrsavanja u kojem izvrsavanje pocinje od ciljne tacke prematacki ulaza u program, tj analiza se izvrsava u obrnutom smeru.Osnovni cilj ovog pristupa je da se napravi test primer kojiuzrokuje izvrsavanje specificne linije koda (obicno nekogassert-a ili throw naredbe)Ovo je takode veoma korisno za debagovanje ili regresionotestiranje





Izvrsavanje unazad (SBE)

Slicnost sa obicnim simbolickim izvrsavanjemKako izvrsavanje pocinje od ciljne linije koda, ogranicenjaputanje se skupljaju po grananjima unazad.Vise putanja se istrazuje u jednom trenutku i kao kod obicnogsimbolickog izvrsavanja, putanje se povremeno proveravaju dali su dostizne.Ako putanja nije dostizna, ona se odbacuje i radi sebacktracking.





Izvrsavanje unazad (CCSBE)

Call-chain backward symbolic execution (CCBSE)

Tehnika zapocinje utvrdivanjem validne putanje u okvirufunkcije gde je ciljna linija locirana.Kada se putanja pronade, pomeramo se na funkciju pozivaocaove funkcije i pokusavamo da rekonstruisemo validnu putanjuod njenog ulaza do poziva funkcije u kojoj je ciljna linija koda.Proces se rekurzivno nastavlja dok ne dodemo do main funkcijeOsnovna razlika izmedju SBE i CCSBE je sto se u okviru svakefunkcije za CCSBE izvrsava obicno simbolicko izvrsavanje dokse za SBE izvrsava unazad.





Izvrsavanje unazad

OgranicenjaDa bi izvrsavanje unazad moglo da se primeni, potrebno je dapostoji na raspolaganju inter-proceduralni CFG (control-flowgraph) koji obezbeduje tok kontrole za ceo program iomogucava da se odrede mesta poziva svih funkcija kojeucestvuju u istrazivanju.Nazalost, konstruisanje takvog grafa cesto je vrlo slozen posaou praksi.Dodatno, svaka funkcija moze biti pozvana sa vise mesta ukodu sto dodatno otezava (usporava) pretragu.




Pregled

1 Uvod



4 Literatura




Literatura

LiteraturaTkest je zasnovan na raduA Survey of Symbolic Execution Techniquesautori:Roberto Baldoni, Emilio Coppa, Daniele Cono D’Elia, CamilDemetrescu, and Irene Finocchihttps://arxiv.org/abs/1610.00502


https://arxiv.org/abs/1610.00502

Documents

Veriﬁkacijasoftvera · Uvod Konkoliˇckoizvr ˇsavanjeiprincipidizajna Strategijeobilaskaputeva Literatura Simboliˇckoizvr ˇsavanjekrozprimer Istorija,alati,stabloizvrˇsavanja