Our Agenda1. WORKSHOP
Device lab requirements
Device parameters
Planning: location, equipment, software, storage
Budget
Device lab technician: role description and responsibilities
Where to buy devices and overview of issues
Lab Maintenance
Device usage in Agile development team
How to present device lab during negotiations with the clients
2. PRACTICE and Q&A
NO DEVICE LAB
Developers and testers are using their own devices
Application delivered to client without testing
Or Using device emulatorOr BYOD policy
External Test LabTYPES:1. Device Rental Labs2. Crowd solution (using private devices across the globe)3. Free device lab
PRO: no need to invest in device acquisition
CONS: 4. Cost:$15-25 per device per usage, $25-$30 per hour device+ tester5. Confidentiality: your app might be exposed to the market 6. Timeline: Sending out your testers at the end of test cycle7. No flexibility in OS update/Version8. Your target devices may not be available9. Booking in advance10. No way to reproduce bug on device11. Developers cannot investigate the bugs and validate features
Overview: Possible roadblocks
Do You Need Internal Test Lab1. What kind of mobile application are you developing?
Mobile Web? Games? Mobile apps?
2. How many projects do you have at the same time?
3. How many developers are in the company?
4. Are development and QA co-located?
5. Do you have a test department? How many testers?
6. Are you developing the app for Android/iOS market? Or for your clients?
7. Do you have contract obligations to test the app on mobile devices?
8. Agile or Waterfall?
9. When is Device test lab usage required? At the end of development cycle or for everyday usage?
Overview: various roadblocks
Business Focus1) Who are the users?
2) Who will pay (how to negotiate with the clients or get management
buy-in)?
3) How many devices do you need or could you afford?
4) Device selection/Device matrix
5) Device fragmentation (specifically, Android fragmentation)
6) Managing device inventory
7) What is the budget?
8) Device selection should be relevant for a year, preferably for 2
9) Return of investment: what is the value of your lab?
Who Are The Internal Users Of A Mobile Test Lab?
1. Development teams: How many?
2. Developers – how many?
3. Testers
4. Designers
5. Artists
6. Customer support / Publishing
7. Sales
8. User acceptance coordinators
Who: Target audience
Where to get info: from GD, Sales, Data analytics, GOOGLE
1. Market: countries
2. How secure the application is? Do you need to test hackers attacks?
3. Age and gender of the users
Overview: various roadblocks
And: Technical requirementsWhere to get info: from GD, Lead developers, game teams
1. IOS and Android? Android only? iOS only?
2. Any specific hardware requirements (i.e. gyroscope)
3. Screen requirements
4. Memory requirements
5. Graphic processor requirements
6. Other specific requirements
7. Phones and tablets? Or just phones? Or only tablets?
8. Pre-installed applications
9. Application to download
Overview: various roadblocks
Device Lab PlanningCentralized or Distributed?
Static lab: Round table for testing (devices are co-located, anyone who want to test should come there.
Agile world: Devices are assigned per team and per tester, with visibility for the whole team: anyone who needs it could borrow for 1 hour or so.
• Location• Storage Space• Tracking tool: I’m using Meraki• Cables, cleaning, power supply• Inventory • Bookkeeping/Accounting• Internal Policy• Disasters happen
Overview: Common mistakes
Budget
• List of devices
• Cost of devices
• Cost of tools and supplies
• Cost of device lab (storage, etc)
• Cost approval process
• Lab technician (salary)
• Who will pay
Overview: typical mistakes
More About Device Lab Budget• Per project
• Per tester
• Track device life
• Maintenance materials
• Plastic gloves
• Cables
• Power supply
• Charging stations
• Tape for marking cables and devices
• SIM cards
• Budget for testing mobile data
Device Lab TechnicianKeep inventory changes
Track lost devices
Plan and manage OS updates
Root devices if needed
Sideload client build
Charge all devices
Gather devices spread across office
Create device test list per app
Perform device testing per app
Perform compliance testing
What To Buy And How Many
List devices to cover the technical spec (low end, high end, phones and tablets)
How many teams do you have?
How many developers per team? Total?
How many testers per team? Total?
Each QA should have 1 Android and 1 IOS gadget (1 phone, 1 tablet)
Each team should have 1 high end, 1 low end (both iOS and Android)
Who managed provisioning profile? How many devices are already in?
Some samples of requests to address
Buying devices*** No contract
*** New or used
*** Beware of jailbroken devices
• Amazon
• E-bay
• Classifieds
• Cell phone providers
• Buying old devices from staff (how much do you want to pay)
Lab Maintenance• Inventory (locate each gadget bi-monthly)
• Mark each device, cable and charger
• Cable replacement
• Monitor chargers
• Buy sleeves
• Monitor upcoming OS changes
• Monitor new devices on the market
• Manage device assignments to team members
Negotiation
Presenting options with cost and risk
Cost of device rent per test cycle, availability
Outsource – cost per cycle
Developers and QA need devices during development cycle
In-house, including
Cost of device tester
Quantity of test cycles
List of devices per app requirements
*** Who will own devices at the end of development
*** Add 1/6 of device budget every 6 months to keep current
Workshop
• List of 30 Devices with larger market share (samples)
• Operation system diversity
• Practical exercise: reviewing application requirements and
selecting 10 top devices
• Practical exercise: Working on device lab budget
• Practical exercise: Managing device test lab
• Practical exercise: Device compatibility checklist