33

AirSim: A Powerful Simulator for Robotics Research Yulai Weng CIS 700/002: Topics in Safe Autonomy Department of Computer and Information Science School of Engineering and Applied Science University of Pennsylvania 02/12/2019 1

AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Download PDF Report

Upload
others
View
11
Download
0

Embed Size (px)

Citation preview

Page 1: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

AirSim: A Powerful Simulator for

Robotics Research

Yulai Weng

CIS 700/002: Topics in Safe Autonomy

Department of Computer and Information Science

School of Engineering and Applied Science

University of Pennsylvania

02/12/2019

1

Page 2: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

2

Page 3: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Successes of Machine Intelligence

Real World Flying Systems

Page 4: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Three Fundamental Challenges

4

Page 5: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Old Version of UAV Simulators

5

Page 6: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

AirSim: The Next Generation Simulator

6

Page 7: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

What Does Simulator Enables?

7

Page 8: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Why Use Unreal Engine?

8

Page 9: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

The Heart of the Vehicle

9

Page 10: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

How Simulator Works?

10

Page 11: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

What Does Physics Engine Do?

11

Page 12: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

The Physics Loop

12

Page 13: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

May the Force & Torque be with You

13

Page 14: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Physics: Dynamics

14

Page 15: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Simulating IMU

15

Page 16: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Simulating Barometer

16

Page 17: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Simulating Magnetometer

17

Page 18: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

The Architecture of AirSim

18

Page 19: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

AirSim is Open Source

19

Page 20: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Experiment: Simulation vs. Reality

20

Page 21: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

AirSim Has APIs

21

Page 22: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Make Drone Move in AirSim Using APIs

22

Page 23: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

23

Page 24: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

AirSim Extensibility

24

Page 25: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

AgIoT: Precision Agriculture

25

Page 26: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

AgIoT: Vision

26

Page 27: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

AgIoT: System Architecture

27

Page 28: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Autonomous 3D Scanning of

Large Structures

28

Page 29: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Expands AirSim AI Simulator to

Include Autonomous Car Research

29

Page 30: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

End-to-End Deep Learning for

Autonomous Driving

30

Page 31: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Deep Reinforcement Learning

for Autonomous Driving

31

Page 32: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

AirSim on Unity

32

Page 33: AirSim: A Powerful Simulator for Robotics Research · import cv2 import sys ('127.e.e.1') client - AirSimC1ient # get depth image = client. setlmageTypeForCamera(Ø, . Scene) result

Thanks !

33

Export Export ---- Import Import Import Bureau Of ... 43Nov2014.pdf · Export Export ---- Import Import Import Bureau Of IndiaBureau Of IndiaBureau Of India 2nd Floor, -166, shrenikpark,

Export Export ---- Import Import Import Bureau Of ... 43Nov2014.pdf · Export Export ---- Import Import Import Bureau Of IndiaBureau Of IndiaBureau Of India 2nd Floor, -166, shrenikpark,

Documents

Automated Clearing House (ACH) Client – Preparing Transactions · file that will be imported into ACH Client. (a) All files for import must contain the columns in Table 1 below

Automated Clearing House (ACH) Client – Preparing Transactions · file that will be imported into ACH Client. (a) All files for import must contain the columns in Table 1 below

Documents

VM Import/Export - User Guide - AWS Documentation · VM Import/Export User Guide Features of VM Import/Export What Is VM Import/Export? VM Import/Export enables you to import …

VM Import/Export - User Guide - AWS Documentation · VM Import/Export User Guide Features of VM Import/Export What Is VM Import/Export? VM Import/Export enables you to import …

Documents

E-Konek Pilipinas, Inc Import Assessment System.pdf · Pre-Requisites • Client Profile Registration System (CPRS) application • Active Customs Client Number (CCN) from CPRS •

Documents

917.277520 - Craftsman Import · 917.277520 - Craftsman Import

917.277520 - Craftsman Import · 917.277520 - Craftsman Import

Documents

Import | Ausgezeichnet IMPORT-VERSAND MIT FEDEXimages.fedex.com/downloads/downloadcenter/learn/2016-Master-Repo… · 02 i import | ausgezeichnet i import-versand mit fedex import-versand

Import | Ausgezeichnet IMPORT-VERSAND MIT FEDEXimages.fedex.com/downloads/downloadcenter/learn/2016-Master-Repo… · 02 i import | ausgezeichnet i import-versand mit fedex import-versand

Documents

airsim Documentation - Read the Docs

airsim Documentation - Read the Docs

Documents

FX CLIENT - Banca Intesa Beograd › upload › documents › elektronsk...Asseco SEE FX Client specification : export/import file formats 5 Transaction value date in UTC format (DDDD-MM-YYYYTHH:MM:SS)

FX CLIENT - Banca Intesa Beograd › upload › documents › elektronsk...Asseco SEE FX Client specification : export/import file formats 5 Transaction value date in UTC format (DDDD-MM-YYYYTHH:MM:SS)

Documents

download.microsoft.com · Web viewAs a best practice, we recommend that you do not import management pack files that you do not plan to use. How to Import Windows 8 Client Management

download.microsoft.com · Web viewAs a best practice, we recommend that you do not import management pack files that you do not plan to use. How to Import Windows 8 Client Management

Documents

Client Certificate Import Manuali.riken.jp/download/cert-import-en20171018.pdf · This is a user manual for importing the client certificate for VPN under the following terminal environments

Client Certificate Import Manuali.riken.jp/download/cert-import-en20171018.pdf · This is a user manual for importing the client certificate for VPN under the following terminal environments

Documents

AirSim-W: A Simulation Environment for Wildlife ...teamcore.usc.edu/papers/2018/bondi_camera_ready_airsim-w.pdf · can play a role in patrolling by deterring poaching ... with individual

AirSim-W: A Simulation Environment for Wildlife ...teamcore.usc.edu/papers/2018/bondi_camera_ready_airsim-w.pdf · can play a role in patrolling by deterring poaching ... with individual

Documents

· Import from East (2.1.1+2_1.2+2.1.3+2.1.4) Import from Russia Federation Import from Ukraine Import from Uzbekistan Import from Kazakhstan Import from Turkmenistan Import from

· Import from East (2.1.1+2_1.2+2.1.3+2.1.4) Import from Russia Federation Import from Ukraine Import from Uzbekistan Import from Kazakhstan Import from Turkmenistan Import from

Documents

ADAV aula 16 - ipp.pt · Because the client never needs to link against the DLL’s import library, the client has no load dependences on the DLL. This means: yThe client can execute

ADAV aula 16 - ipp.pt · Because the client never needs to link against the DLL’s import library, the client has no load dependences on the DLL. This means: yThe client can execute

Documents

CIM Import Common Information Model Import

CIM Import Common Information Model Import

Documents

Client Companion Guide - marketplace.bamboohr.com · Full import history of all data sync’d and a ticket management ... My client name is PlanSource ... would like employees PayPeriods

Client Companion Guide - marketplace.bamboohr.com · Full import history of all data sync’d and a ticket management ... My client name is PlanSource ... would like employees PayPeriods

Documents

PyCUDA - Elefantel = dot(x, y).get() ... import pycuda.driver as cuda import pycuda.autoinit import numpy from pycuda.curandom import XORWOWRandomNumberGenerator

PyCUDA - Elefantel = dot(x, y).get() ... import pycuda.driver as cuda import pycuda.autoinit import numpy from pycuda.curandom import XORWOWRandomNumberGenerator

Documents

Action Request System - cs.helsinki.fi · AR System Server Database Remedy Import Client Other API Client. 2 ... kentässä jne. Aktiivilinkille voi myös antaa ehdon joka tarkistetaan

Action Request System - cs.helsinki.fi · AR System Server Database Remedy Import Client Other API Client. 2 ... kentässä jne. Aktiivilinkille voi myös antaa ehdon joka tarkistetaan

Documents

1. Open Mail Client (Outlook Express.) - Indian Coast Guard · IMPORT OTHERS PUBLIC KEY ON THEIR EMAIL ID'S 1. Open Mail Client (Outlook Express.) 2. Go to ... Name Home Business

1. Open Mail Client (Outlook Express.) - Indian Coast Guard · IMPORT OTHERS PUBLIC KEY ON THEIR EMAIL ID'S 1. Open Mail Client (Outlook Express.) 2. Go to ... Name Home Business

Documents

airsim Documentation · airsim Documentation AirSim is a simulator for drones, cars and more, built onUnreal Engine. It is open-source, cross platform and supports hardware-in-loop

airsim Documentation · airsim Documentation AirSim is a simulator for drones, cars and more, built onUnreal Engine. It is open-source, cross platform and supports hardware-in-loop

Documents

LAMPIRAN A import java.util.*; import java.io.*; import java.net.*; import java.lang.*; import com.dalsemi.system.*; import com.dalsemi.shell.server.*; public class Utama { Sistem

LAMPIRAN A import java.util.*; import java.io.*; import java.net.*; import java.lang.*; import com.dalsemi.system.*; import com.dalsemi.shell.server.*; public class Utama { Sistem

Documents

interoperability.blob.core.windows.net · Web viewThe Content Deployment Remote Import Web Service Protocol enables a protocol client to remotely manage the import of a set of content,

interoperability.blob.core.windows.net · Web viewThe Content Deployment Remote Import Web Service Protocol enables a protocol client to remotely manage the import of a set of content,

Documents

INTRODUCTION · Web viewSAP Customer Data Cloud performs an import of Client’s existing database and will consult on the schema design. To achieve this import, Client must export

INTRODUCTION · Web viewSAP Customer Data Cloud performs an import of Client’s existing database and will consult on the schema design. To achieve this import, Client must export

Documents

Large Client-Base Trusted Platform Refined Platform · COMPREHENSIVE ASSESSMENT APPROACH Create & Manage •Cloud-Based Item Banking •Type, Copy/Paste, Import •Unlimited ques

Large Client-Base Trusted Platform Refined Platform · COMPREHENSIVE ASSESSMENT APPROACH Create & Manage •Cloud-Based Item Banking •Type, Copy/Paste, Import •Unlimited ques

Documents

[MS-UPSIMP]: User Profile Import Protocol Specification€¦ · The User Profile Import protocol enables the protocol client to import user profiles, member groups and membership

Documents

How-To... Import a Portal Public Key Into an ECC Client

How-To... Import a Portal Public Key Into an ECC Client

Documents

Guía de alojamiento del BCN AirSim Meeting 2013

Guía de alojamiento del BCN AirSim Meeting 2013

Documents

1 - Button Kontrolu Nasıl Yapılır? · MainActivity.java package com.example.uygulama1; import android.app.Activity; import android.os.Bundle; import android.view.Menu; import android.view.View;

1 - Button Kontrolu Nasıl Yapılır? · MainActivity.java package com.example.uygulama1; import android.app.Activity; import android.os.Bundle; import android.view.Menu; import android.view.View;

Documents

SVA Vias Centrales Anestesia Regional Manejo de vía … · AIRSIM ADVANCE COMBO Ref.: E9850 Ideal para el entrenamiento en: Intubación con fibrobroncocopio Intubación traqueal

SVA Vias Centrales Anestesia Regional Manejo de vía … · AIRSIM ADVANCE COMBO Ref.: E9850 Ideal para el entrenamiento en: Intubación con fibrobroncocopio Intubación traqueal

Documents

Welcome [] · Zülküf Küçüközer TriangleCanvas.java import javax.microedition.lcdui.Graphics; import javax.microedition.lcdui.game.*; import javax.microedition.m3g.*;

Welcome [] · Zülküf Küçüközer TriangleCanvas.java import javax.microedition.lcdui.Graphics; import javax.microedition.lcdui.game.*; import javax.microedition.m3g.*;

Documents

SafeNet Authentication Client 8.3 使用手冊 · 2. Import License 1. From the Start menu, select Programs > SafeNet > SafeNet Authentication Client > SafeNet Authentication Client

SafeNet Authentication Client 8.3 使用手冊 · 2. Import License 1. From the Start menu, select Programs > SafeNet > SafeNet Authentication Client > SafeNet Authentication Client

Documents