Click here to load reader

Sikuli: Using Screenshots for GUI Automation and Testing

  • View
    13.202

  • Download
    5

Embed Size (px)

DESCRIPTION

 

Text of Sikuli: Using Screenshots for GUI Automation and Testing

  • 1. SIKULI USING SCREENSHOTS FOR GUI AUTOMATION AND TESTING Tsung-Hsiang (Sean) Chang, Rob Miller MIT CSAIL Collaborator: Tom Yeh 1
  • 2. How to tell computers what to do? 2
  • 3. case $rtval in 1) rm -f /tmp/dirip.$$ ; return ;; 255) rm -f /tmp/dirip.$$ ; return ;; esac mfile=`cat /tmp/dirip.$$` if [ -z $mfile ] then mfile=`pwd`/* SHELL SCRIPT else grep "*" /tmp/dirip.$$ if [ $? -eq 1 ] then ! mfile=$mfile/* fi fi for i in $mfile do if [ -f $i ] then ! echo "$i Delete?" >> /tmp/finallist.$$ fi! done 3
  • 4. 4
  • 5. GUI AUTOMATION 5
  • 6. 2 3 1 6
  • 7. 7
  • 8. click click 7
  • 9. SIKULI SCRIPT 8
  • 10. SIKULI TEST 9
  • 11. OUTLINE Problem - How to refer to a GUI widget? Sikuli Script - examples and demos Sikuli Test - demo Test By Demonstration Support of testing practices Conclusions 10
  • 12. HOW TO REFER TO A GUI WIDGET? 11
  • 13. HOW TO REFER TO A GUI WIDGET? Abbot by pre-programmed references 11
  • 14. HOW TO REFER TO A GUI WIDGET? Abbot by pre-programmed references by absolute locations 11
  • 15. Use Screenshots! 12
  • 16. TEMPLATE MATCHING click( ) 13
  • 17. TEMPLATE MATCHING click( ) 13
  • 18. EXAMPLE 1: EMPTY TRASH CAN 2 3 1 14
  • 19. EX2: FACEBOOK STALKER 15
  • 20. EX2: FACEBOOK STALKER 15
  • 21. 16
  • 22. RESTRICT SEARCH REGION 17
  • 23. EX3: MUTE SPEAKERS 18
  • 24. EX3: MUTE SPEAKERS 18
  • 25. SPATIAL OPERATORS 19
  • 26. SPATIAL OPERATORS 19
  • 27. SPATIAL OPERATORS (2) 20
  • 28. EX4: SET IP ADDRESS 21
  • 29. VISUAL-EVENT-DRIVEN PROGRAMMING 22
  • 30. VISUAL-EVENT-DRIVEN PROGRAMMING 22
  • 31. SIKULI TEST 23
  • 32. VISUAL ASSERTION Verify appearance 24
  • 33. VISUAL ASSERTION Verify replacement 25
  • 34. TEST BY DEMONSTRATION detect changes click 26
  • 35. Demo 27
  • 36. GENERATE ACTION COMMANDS mouse mouse A A b b down up down up down up 28
  • 37. GENERATE ACTION COMMANDS low-level events mouse mouse A A b b down up down up down up 28
  • 38. GENERATE ACTION COMMANDS low-level events mouse mouse A A b b down up down up down up high-level commands click type Ab 28
  • 39. GENERATE ACTION COMMANDS low-level events mouse mouse down up A A b b click down up down up high-level commands click type Ab combine with screenshots 28
  • 40. GENERATE ASSERTIONS click 29
  • 41. GENERATE ASSERTIONS click 1. nd differences / manually hint 29
  • 42. GENERATE ASSERTIONS click 1. nd differences / manually hint 2. group together 29
  • 43. GENERATE ASSERTIONS click 1. nd differences / manually hint 2. group together 3. combine with assertExist 29
  • 44. GENERATE ASSERTIONS click 1. nd differences / manually hint 2. group together 3. combine with assertExist 29
  • 45. jUNIT INTEGRATION 30
  • 46. TEST-DRIVEN DEVELOPMENT 31
  • 47. READABILITY OF CODE 32
  • 48. PLATFORM INDEPENDENCE Works on ANY GUI that can be displayed on Windows/Linux/Mac Virtual machines Remote desktop Mobile simulators: Android, iPhone Web: Flash, HTML+Javascript 33
  • 49. 34
  • 50. 34
  • 51. LIMITATIONS Theme Variations Visibility Constraints 35
  • 52. DISCUSSION Sikuli programs are written against the user interface instead of an API UI: visible, familiar, always exists API: faster, probably more stable 36
  • 53. FUTURE WORK fast and accurate OCR on screen more robust against different skins 37
  • 54. Thanks! http://sikuli.csail.mit.edu or, just google sikuli New Sikuli 0.10 released. Follow us on twitter @sikuli Thanks to our funding sources: NSF and Quanta Computer (as a part of TParty project.) 38