View
483
Download
0
Category
Preview:
Citation preview
Agenda
•Introduction to Presenters
•Problem Statement
•Our Solution
•Planned Improvements
•Jenkins Configuration and UT Output
•Key Takeaways
•Q & A
Seastar Labs Private Limited 2
Developing an IoT device requires multiple development environments.
• Firmware functions• Drivers• Operating System
IoT Device
• Device drivers on windows/mac• Software application on windows/mac to interact with the device• Monthly deliverables and showing demos to the client on current
progress• One team responsible for whole nine yards
Need automated tests for embedded code and host PC and Mac: Should run to qualify every change.
Seastar Labs Private Limited 4
USB
PC/MAC
The e2e solution stack involves multiple technologies
HTML/CSS
Java Script
Business Logic (C/C++)
Service(s) (C/C++)
Host OS
Embedded App (C/C++/Assembly)
Device Drivers
XYZ Board
Wire Protocol
Host can be PC/Mac/Linux
Device
Seastar Labs Private Limited 5
Chromium Embedded FW
Embedded Linux
Multi-OS development/build environment requires all tests to pass every where.
Source Control (SSOT)
Linux Windows Mac
Seastar Labs Private Limited 6
Multi OS development environment gives rise to interesting challenges.
• Device code is cross compiled for ‘xyz’ embedded platform using Linux x86 . But it can only run on ‘xyz’. How do we test ?
• Mac version of host software requires Mac OSX to build and test.
• Windows version of host software requires Win 7/8/8.1 to build and test.
• Most changes would trigger all 3 builds. How do we test ?
• Should we slow down the development pace of Host Software to wait for device software to be available?
• In production, host software needs device to be connected with which human interacts. Should we use same setup for integration tests ?
Seastar Labs Private Limited 7
Multiple stakeholders means we need to have a clear process for each stakeholder
• Develops code, • Writes Uts for
new/modified functionality
• Runs Uts and ensures they pass
• Checks in to source control after code review
• Update Jira
Commit triggers builds
• Waits for a successful build• Tests the build to verify the visible changes based on
Jira• QA releases the build to field users
QA/QE
Jenkins
Build successJira notification
Developer• Builds the source code (device and host)• UTs executed: simulate device• Build is broken and mails are triggered for dev.
o Issue with device code/UTso Issue with host code/UTso Issue with signing the binarieso Issue in publishing the build
• Build is successful mails are triggered for QA and internal users
Multiple Jenkins Builders
Seastar Labs Private Limited 9
Build failure
Keep code and unit tests platform neutral to get biggest bang for the buck.• Device code is also compiled for Linux x86. So that unit tests for device can be
run on Linux x86 machine.
• Use platform neutral libraries such as APR, Boost C++, Poco, thrift etc. to make sure that the application code behaves predictably on Win/Mac. Note: we still run all unit test cases on all platforms.
This speeds up the development process as we can use a single code
base which would compile differently on different OS
• Developed device simulator to mimic entire protocol between device and host, this way we didn’t have to wait for the device hardware and software to be ready
• Used Jenkins as the CI tool to ensure that all check ins are Unit Tested before the build is given for QA
• Use TUT C++ framework: Really light weight (1 hpp file) and platform neutral
• Use platform emulators to test assembly code during test run
Seastar Labs Private Limited 10
We are evaluating improvements in the testing part of the Jenkins builder
• We are evaluating use of software which will ensure that the unit tests will be run in the following condition• Latest firmware is installed on the device (This is being done manually now)• Unit tests would be run to ensure that we are actually interfacing with the
device‘s latest firmware
• Software's being considered include:• Robot Framework• Fitnesse• Slim
Seastar Labs Private Limited 12
Configuration settings that need to be made in the project to achieve running of UTs
Seastar Labs Private Limited 14
Unit test reports from Jenkins mentioning all test groups to ensure all classes are covered
Seastar Labs Private Limited 15
Build time impact to run each unit test in Jenkins is around 2.24 seconds
Seastar Labs Private Limited 16
With minimal effort we can ensure that embedded software can be built using CI
• Key Takeaways:• Use platform independent libraries such as APR, Boost C++, Poco framework,
thrift etc. to make sure that the application code behaves predictably on Linux/Win/Mac.
• Run all unit test cases on all platforms.
• Use light weight Unit test framework.
• Develop a device simulator to mimic entire protocol between device and host, this way you don’t have to wait for the device hardware and software to be ready.
• Use a CI tool like Jenkins to ensure that all check ins are Unit Tested before the build is given for testing.
• Developer is responsible for unit testing in one dev environment while Jenkins builders are responsible for all.
• Attention to build failures is crucial.
Seastar Labs Private Limited 18
Recommended