130
Scott Janousek - 2011 AIR WOLF Leveraging ActionScript to Control a Quadcopter EPISODE 2 “A Tale of Reverse and Forward Engineering”

AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Embed Size (px)

DESCRIPTION

AIR Wolf: Leveraging ActionScript to Control a Quadcopter EPISODE 2 “A Tale of Reverse and Forward Engineering” Presented by Scott Janousek at 360|Flex in April of 2011.

Citation preview

Page 1: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

AIR WOLFLeveraging ActionScript to Control a Quadcopter

EPISODE 2“A Tale of Reverse and Forward Engineering”

Page 2: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

DISCOVERY

EXPLORATION

TINKERING

EXPERIMENTATION

Page 3: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Agenda

INTRO

DEMO(s)

PROJECT AIR WOLF

OUTRO

Page 6: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Scott janousek

Founder, Hooken Mobile

Long Time (Flash) Mobile Advocate

Working on all kinds of devices & platforms

“Gadget Geek”

AR.Drone enthusiast

Page 7: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

contact

www.scottjanousek.com

@scottjanousek

www.hookenmobile.com

@hookenmobile

Page 8: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

AIR WOLFProjectgenesis

Page 9: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Growing up,I was a big fan of ...

Page 10: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Page 11: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

For those not familiar ....

Page 12: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011Season 4 intro

Page 13: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Always wanted to y ...

Page 14: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Page 15: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Options

Page 16: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

The real deal

Page 17: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

$$$ ... not to mentionThe whole “eyesight thing”

Page 18: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

hard core rc

Page 19: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Cool, but I never got into RC.

Page 20: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

mini rc TOY

Page 21: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011Too cheap and “plastic-y”

Page 22: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

How about makingsomething custom?

Page 24: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011http://blog.makezine.com/archive/2010/05/arduimu_quadcopter_drone.html

DIY

Page 25: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

“Hey Scott, did you hear about ...”

Page 26: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

ar drone

Page 27: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

AR.DRONE

Page 28: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Electric Powered “QUAD COPTER”

(or quadro-copter)

Page 31: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

“ok, ok, the commercialis kind of silly ... but, let’s

have a go at it ...”

Page 32: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

AR.droneDEMOS

Page 33: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

AR.DRONEfreeflight

(native ios app)

Page 34: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

native remote demo

Page 35: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

experiment

Page 36: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

bci experiment

Page 37: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

tracking experiment

Page 38: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

There are more seriousDrones out there besides

AR.Drone ...

Page 39: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

OTHERCOOL DRONEhardware

Page 40: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

hummingbirde.g. http://www.avinc.com/

Page 41: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011Humming Bird Drone - Goverment R&D

Page 42: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

civilian uses

Page 43: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

RECENT uav USAGE

CNN: “Flying Drone peers into Japan’s damaged reactors”

Page 44: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

echoquadrocopter

R&D

Page 45: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

ping pong

Page 46: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

playing piano

Page 47: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

AIR WOLF project

discussion

Page 48: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

AR.DRONEhardware

Page 49: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011 BTW: It costs about $300 USD

Page 51: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

more specs

Page 52: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

“GUTS”

Page 53: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

4 “props”

Page 54: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

underside

Acoustic Sensors

IndicatorLight

Belly Camera

Unpair

Reset

Page 55: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Cameras

Belly Cam Front Camera

Page 56: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

TROUBLE-SHOOTing

Page 57: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

basic flight

Page 58: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

pitch, yaw, roll

Page 59: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

nativeremotecontrol

apps

Page 60: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

native h.u.d.

Page 61: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Native mobile Apps

iOS

Android

Page 62: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

iphone - freeflight

Page 63: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

android

Page 64: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

desktop - LINUX & windows

Page 65: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

“Yes, I know native is better in this case, but ...

I wonder if I could

connect to AR.Drone viaACTIONSCRIPT ...”

Page 66: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

SOFTWARE

ar drone sdk vs. urbi

Page 67: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

“old school”

Page 68: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

AR DRONE SDK

Page 69: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

ardrone SDk

Windows, Linux, Android, iOS support

Page 70: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

discovery

Page 71: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Architecture

Page 72: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

control Flow

Page 73: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Areas

NavData

(Video) Stream

Cmds

Page 74: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

PLANNING

Page 75: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Cross Platform App

Desktop

Windows, Mac, Linux

Mobile

AIR for Android

Page 76: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Reverse Engineering

1. Download AR Drone SDK

2. Browse Wiki & Forums

3. Look at C Source Code provided

4. Figure out the open Protocol

Page 77: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

sdk 1.5 docs

Page 78: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

forums

Page 79: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

APPARATUS

Page 80: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

setup

Wi-!

Ad-hoc Wi-!

10.0.0.x physical network

10.0.0.x port 8080

10.0.0.1 port 808010.0.0.x port 8080

192.168.1.1 port 5556

192.168.1.2 port 5556

“bandaid” reset button

MacBook Pro

AR Drone

DROIDX (Android)

Airport Extreme

Page 81: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

network config

Page 82: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

internet sharing

Page 83: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

airport config

Page 84: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

ethernet

Page 85: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

DEVELOPMENT

Page 86: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

original PoC

Page 87: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

at commands

Page 88: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

basiccommands

altitude max2m AT*CONFIG=1,\"control:altitude_max\",\"2000\"Landing AT*REF=101,290718208Takeoff AT*REF=102,290717696Hovering AT*PCMD=201,1,0,0,0,0gaz 0.1 AT*PCMD=301,1,0,0,1036831949,0gaz -0.1 AT*PCMD=302,1,0,0,-1110651699,0pitch 0.1 AT*PCMD=303,1,1036831949,0,0,0pitch -0.1 AT*PCMD=304,1,-1110651699,0,0,0 yaw 0.1 AT*PCMD=305,1,0,0,0,1036831949yaw -0.1 AT*PCMD=306,1,0,0,0,-1110651699roll 0.1 AT*PCMD=307,1,0,1036831949,0,0roll -0.1 AT*PCMD=308,1,0,-1110651699,0,0pitch -30 deg AT*ANIM=401,0,1000pitch 30 deg AT*ANIM=402,1,1000

Page 89: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

AT syntax

AT*REF=<sequence>,<UI>AT*PCMD=<sequence>,<enable>,<pitch>,<roll>,<gaz>,<yaw> ("oat)0.05 = (int)1028443341 ("oat)-0.05 = (int)-1119040307 ("oat)0.1 = (int)1036831949 ("oat)-0.1 = (int)-1110651699 ("oat)0.2 = (int)1045220557 ("oat)-0.2 = (int)-1102263091 ("oat)0.5 = (int)1056964608 ("oat)-0.5 = (int)-1090519040AT*ANIM=<sequence>,<animation>,<duration>AT*CONFIG=<sequence>,\"<name>\",\"<value>\"

Page 90: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

at bitsUI_BIT:00010001010101000000000000000000 | | | | | || | ||||+--0: Button turn to left | | | | | || | |||+---1: Button altitude down (ah - ab) | | | | | || | ||+----2: Button turn to right | | | | | || | |+-----3: Button altitude up (ah - ab) | | | | | || | +------4: Button - z-axis (r1 - l1) | | | | | || +--------6: Button + z-axis (r1 - l1) | | | | | |+----------8: Button emergency reset all | | | | | +-----------9: Button Takeoff / Landing | | | | +-------------------18: y-axis trim +1 (Trim increase at +/- 1??/s) | | | +---------------------20: x-axis trim +1 (Trim increase at +/- 1??/s) | | +-----------------------22: z-axis trim +1 (Trim increase at +/- 1??/s) | +-------------------------24: x-axis +1 +-----------------------------28: y-axis +1

Page 91: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Proof of Concept

Page 92: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

desktop app

Page 93: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

“first flight”

Page 94: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

ardroneserver

Page 95: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

walkthrough

Page 96: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

android app

Page 97: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

android

Page 98: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

ardroneclient

Page 99: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

adobe max 2010

Video courtesy of @polygeek (Dan Florio)

Page 100: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

post-mortem

Page 101: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Challenges

Battery Life

Lack of protocol documentation

Indoor testing limits R&D

Network cong was time confusing

No Binary Sockets in AIR for Android !!!!

Network Latency

Buggy Firmware (pre 1.5)

tech challenges

Page 102: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

“new school”

Page 103: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

URBIUniversal Robot Body Interface

Page 104: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

URBI FORGE

Page 105: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

PSYKOKWAK

Page 106: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Urbi is an open source driver for multiple robotic systems.

A driver has been developed that allows Urbi to interface with the AR Drone.

So now you can control your drone with a laptop, joystick, keyboard, wii remote, etc.

Urbi even allows you to write scripts to have the bot perform actions.

Page 107: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

urbi server launch

Page 108: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

urbi client connection

Page 109: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

urbi launch script example

Page 110: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

connectingto urbi

Page 111: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

urbi script

Page 112: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

ar.drone urbicommands

Page 113: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

walkthrough

Page 114: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

demos

Page 115: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

motion tracking

control test

Page 116: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

experiment

Page 117: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

braincomputerinterface

(b.c.i)

Page 118: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

BCI hardware

Page 119: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

visualizer

Page 120: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

experiment

Page 121: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

AS3 API for Neurosky Mindset BCI

actionscript api

Page 122: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

gostai studio& gostai lab

Page 123: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Gostai Studio and Labs are IDEs that allow developers to interface with robotics, etc.

It can be used with URBI.

ULA files are markup files similar to MXML.

Drag and Drop components are available.

Page 124: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

gostai labs ide

Page 125: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

gostai ula

Page 126: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

outro

Page 127: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

take aways

AR Drone is an fun toy

AIR, Flex = Prototyping made easy

URBI & Gostai = great for hobby robotics

Experiment, tinker, & have fun

Air Wolf Movie? ... I hope so ... :)

Page 129: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

Thank you

Page 130: AIR Wolf: Leveraging ActionScript to Control a Quadcopter

Scott Janousek - 2011

@scottjanousek

http://www.scottjanousek.com