Upload
bitbar
View
6.946
Download
2
Tags:
Embed Size (px)
Citation preview
webinar
How to Leverage Appium in Your Mobile App Testing
7 January 2014
Ville-Veikko HelppiTechnical Product [email protected]
Saad ChaudrySr. Software Engineer
Agenda• Different Ways to Automate Your App Testing• Appium and How It Compares Against Other
Test Automation Frameworks• Appium with Real Devices (@Testdroid Cloud)• 10 Tips for Professional Mobile App Testing• Demonstration• Q&A
webinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
2
Agenda• Different Ways to Automate Your App Testing• Appium and How It Compares Against Other
Test Automation Frameworks• Appium with Real Devices (@Testdroid Cloud)• 10 Tips for Professional Mobile App Testing• Demonstration• Q&A
webinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
3
Different Approaches To Testing webinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
4
Different Ways of Doing Test Automation
webinar
Automatic test exercisers Record and PlaybackHand written test scripts
Benefits:
Accurate, specific to your testing needs, plenty of options with frameworks, tools
Fast to create, accurate, not as sensitive to human-errors as hand-written tests, tools avail’ty
Fastest & extremely automated, excellent for smoke testing/quick testing, availability
Tradeoffs:
Takes a lot of time, ties resources to write test cases/scripts, error-prone (humans)
Compelling Recorder+Playback tools available for only few test automation frameworks
Not accurate as real test cases
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
5
Option #1: Cloud-Based Testingwebinar
Developers
Source code
Repository
Build failed
Polling
Notifying
Successful build
QA & Testers
Building
Error reporting
Test Cases
Application
Users / Consumers
Rele
ase
Com
mit
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
6
Option #2: Continuous Integrationwebinar
Developers
Source code
Repository
Build failed
Polling
Notifying
Successful build
QA & Testers
Building
Error reporting
Test Cases
Application
Users / Consumers
Rele
ase
Com
mit
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
7
Manualwebinar
Automation
Smaller coverage, More money burnt & time wasted, Error-prone
Large coverage, quickly
completed, Less money & time wasted, Exact
results.
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
8
Why Apps Fail?webinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
9
Why Real Devices are Must-to-Have?• Emulators cannot help you testing…– User Experience and Usability– Hardware– Software– Infrastructure
webinar
0 % = the percentage of your app users that use emulator to run your app!
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
10
How to Improve the App Quality• Automate generic things as much as you can• During the development your app changes –
the same must apply for testing!• Carefully select (testing) technology & partner• Use all 24 hours per day with test automation• Cloud-based platform is the only solution to
get you quickly covered in the global markets
webinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
11
Agenda• Different Ways to Automate Your App Testing• Appium and How It Compares Against Other
Test Automation Frameworks• Appium with Real Devices (@Testdroid Cloud)• 10 Tips for Professional Mobile App Testing• Demonstration• Q&A
webinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
12
Android Example: The Family Tree of Test Automation Frameworks
webinar
JUnit
Android Instrumentation Framework
Robotium Espresso
UI Automator
Calabash
Appium
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
13
Appium Introductionwebinar
• Uses Selenium Webdriver (W3C working draft) as a scripting framework– Standard for browser automation – libs for various languages
• Supports native Android, native iOS and mobile web:– Android via uiautomator (API level >=17) and Selendroid (API
level <17)– iOS via UI Automation– Mobile web as Selenium driver for Android and iOS
• You can write your Appium scripts on almost any programming language (Haskell/Go/Clojure/Java/Ruby)
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
14
Appium: How it works?webinar
• Appium is an HTTP server that creates and handles WebDriver sessions
• It starts an Appium server on the device that is listening commands from the main Appium process
• It receives json requests from client libraries over HTTP
• On Android Appium executes these commands as either uiautomator or Selendroid commands depending on the API level of the device
• Tests are driven from a Selenium script on Appium server
Device
Your app
Bootstrap.jar
Test server
UI Automator controller or
Selendroid driver
Selenium script
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
15
Appium: Test Lifecyclewebinar
1. Appium installs Bootstrap.jar as an instrumentation package which contains an uiautomator server and a tcp server
2. The commands from a Selenium script are executed on the main Appium server that relays the commands over TCP/IP to the tcp server running on the device
3. The UI Automation server converts the Selenium commands to uiautomator commands on the fly
4. If the device has API level <17 Appium installs a Selendroid server that converts the Selenium commands to Android Instrumentation (JUnit) commands
5. All test verification and result processing is happening on server side in the main Appium Server
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
16
Appium: Code Examplewebinar
# wait for hellosleep(3)textFields = driver.find_elements_by_tag_name('textField')assertEqual(textFields[0].get_attribute("value"), "Hello")
# click sign-in buttondriver.find_elements_by_name('Sign in')[0].click()
# find the text fields again, and enter username and passwordtextFields = driver.find_elements_by_tag_name('textField')textFields[0].send_keys("twitter_username")textFields[1].send_keys("passw0rd")
# click the Login button (the first button in the view) + sleepdriver.find_elements_by_tag_name('button')[0].click()sleep(3)
# click the first button with name "Compose” + type in the tweet message + press “Send”driver.find_elements_by_name('Compose')[0].click()driver.find_elements_by_tag_name('textField')[0].send_keys(”#Testdroid is awesome!")driver.find_elements_by_name('Send')[0].click()
# exitdriver.quit()
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
17
How Appium Compares to Others?webinar
Appium Robotium uiautomator Espresso Calabash
Android Yes Yes Yes Yes Yes
iOS Yes No No No Yes
Mobile web Yes (Android & iOS)
Yes (Android)
Limited to x.y clicks
No Yes (Android)
Scripting Language
Almost any Java Java Java Ruby
Test creation tools
Appium.app Testdroid Recorder
UI Automator viewer
Hierarchy Viewer
CLI
Supported API levels
All All 16 => 8, 10, 15-19 All
Community Active Contributors Google Google Pretty quiet
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
18
Agenda• Different Ways to Automate Your App Testing• Appium and How It Compares Against Other
Test Automation Frameworks• Appium with Real Devices (@Testdroid Cloud)• 10 Tips for Professional Mobile App Testing• Demonstration• Q&A
webinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
19
Appium @ Localhostwebinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
20
Test Script
Test Case
Desired Capabilities
{ “device”: “Android”, “app”: “/Users/user/ApiDemos.apk” “app-package”: “com.example.android.apis” “app-activity”: “.ApiDemos”}
Appium @ Localhost
Appium @ Localhostwebinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
21
Test Script
Test Case
Desired Capabilities
{ “app”: “com.bitbar.testdroid.BitbarIOSSample”}
Appium @ Localhost
Appium @ Localhostwebinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
22
Test Script
Test Case
Desired Capabilities
Appium @ Localhost
WebDriverhttp://localhost_4723/wd/hub Appium Server
4723
Device
Appium @ Localhostwebinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
Test Script
Test Case
Desired Capabilities
From Localhost to Testdroid Cloud
WebDriverhttp://localhost_4723/wd/hub
*Testdroid Caps
http://appium.testdroid.com/wd/hub
Appium @ Localhostwebinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
Test Script
Test Case
Desired Capabilities
From Localhost to Testdroid Cloud
WebDriver { “testdroid_username”: “[email protected]”, “testdroid_password”: “p4s$w0rd”, “testdroid_project”: “My First Project”, “testdroid_testrun”: “Test 1”, “testdroid_device”: “iPad Mini 7.0.4 A1432”, “testdroid_app”: “http://domain.com/app_v1.ipa” . . “app”: “com.bitbar.testdroid.BitbarIOSSample”}
Projects and Test Runs
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
25
webinar
Appium @ Testdroid Cloud
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
26
webinar
Appium @ Testdroid
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
27
webinar
Appium @ Testdroid
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
28
webinar
Appium @ Testdroid
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
29
webinar
Appium @ Testdroid
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
30
webinar
Appium @ Testdroid
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
31
webinar
Running tests on multiple devices
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
32
webinar
Projects and Test Runs
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
33
webinar
Projects and Test Runs
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
34
webinar
Appium Broker
5 Appium Ready
sessionid
Appium @ Localhostwebinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
35
Test Script
Behind the Scene
Device Cluster
Device 1 Device 2 Device 3
1 WebDriver Session Request@ http://appium.testdroid.com/wd/hub/
Desired Caps, .apk / .ipa
2 Configure project
3Wait for device to become available
4 Start Appium
Session Map
WebDriver Session response6
sessionid
7 Test Run
sessionid Proxy
Appium
Device 1
Agenda• Different Ways to Automate Your App Testing• Appium and How It Compares Against Other
Test Automation Frameworks• Appium with Real Devices (@Testdroid Cloud)• 10 Tips for Professional Mobile App Testing• Demonstration• Q&A
webinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
36
Tip #1: Test early, Test oftenwebinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
37
Tip #2: Plan What to Automatewebinar
People
Infrastructure
Tools
Training
Time
MAN
UAL
AUTO
MATED
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
38
Tip #3: Use only Real Deviceswebinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
39
Tip #4: Use Atomic Test Unitswebinar
Test iteration #1 Test iteration #2
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
40
Tip #5: Create Hermetic Testswebinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
41
Tip #6: Use All Possible Deviceswebinar
Testdroid Cloud’s 250+ Android devices = 93-95% global Android volumes!
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
42
Tip #7: Separate Apps and Testswebinar
Your App
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
43
Tip #8: Output Everything to Logswebinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
44
Tip #9: End-to-End Testingwebinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
45
Tip #10: Integrate Dev & Testingwebinar
Com
mit
Developers
Source code
Repository
Build failed
Polling
Notifying
Successful build
QA & Testers
Building
Error reporting
Test Cases
Application
Rele
ase
Users / Consumers
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
46
Agenda• Different Ways to Automate Your App Testing• Appium and How It Compares Against Other
Test Automation Frameworks• Appium with Real Devices (@Testdroid Cloud)• 10 Tips for Professional Mobile App Testing• Demonstration• Q&A
webinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
47
Agenda• Different Ways to Automate Your App Testing• Appium and How It Compares Against Other
Test Automation Frameworks• Appium with Real Devices (@Testdroid Cloud)• 10 Tips for Professional Mobile App Testing• Demonstration• Q&A
webinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
48
webinar
© Copyrights by Bitbar Technologies Ltd. 2014 All rights reserved.
49