Upload
others
View
21
Download
1
Embed Size (px)
Citation preview
SECOND EDITION
Programming Interactivity
Joshua Noble
O'REILLY4
Beijing • Cambridge • Farnham • Köln • Sebastopol • Tokyo
Table of Contents
Preface xiii
1. Introducing Interaction Design 1 What This Book Is For 1 Programming for Interactivity 2
The Nature of Interaction 3 Messages and Interaction 5 Interfaces and Interaction 6 Languages of Interaction 7
Design and Interaction 15 Art and Interaction 16 Data Exchange and Exploration 17 Working Process 21
2. Programming Basics 23 Why You'll Read This Chapter More Than Once 24 The Nature of Code 24 Variables 25
Simple Types 26 Arrays 31 Casting 35 Operators 35
Control Statements 39 if/then 39 for Loop 40 while Loop 41 continue 42 break 42
Functions 43 Defining a Function 43 Passing Parameters to a Function 44
Some Suggestions on Writing Functions 45 Overloading Functions 46
Objects and Properties 48 Scope 51 Review 52
3. Processing 55 Downloading and Installing Processing 56 Exploring the Processing IDE 56 The Basics of a Processing Application 58
The setupO Method 58 The draw() Method 60
The Basics of Drawing with Processing 63 The rect(), ellipse(), and line() Methods 63 RGB Versus Hexadecimal 64 The fillO Method 66 The backgroundO Method 67 The lineO Method 67 The stroke () and stroke Weight () Methods 68 The curve () Method 68 The vertexO and curveVertexO Methods 69
Capturing Simple User Interaction 70 The mouseX and mouseY Variables 71 The mousePressedO Method 71 The mouseReleasedO and mouseDraggedO Methods 72 The keyPressed and key Variables 75
Importing Libraries 80 Downloading Libraries 80
Loading External Data into Processing 81 Loading and Displaying Images 81 Displaying Videos in the Processing Environment 83 Using the Movie Class 83 Reading and Writing Files 85
Running and Debugging Applications 87 Exporting Processing Applications 88 Conclusion 91 Review 91
4. Arduino 93 Starting with Arduino 94
Installing the IDE 95 Configuring the IDE 97
Touring an Arduino Board 99
iv | Table of Contents
The Controller 99 Uno Versus Mini Versus Mega 99
Touring the Arduino IDE 105 The Basics of an Arduino Application 108
The setup Statement 108 The loop Method 108
Features of the Arduino Language 110 Constants 112 Methods' 112 Arrays 114 Strings 114
How to Connect Things to Your Board 118 Resistors 120
Hello World 121 Debugging Your Application 124 Importing Libraries 127 Running Your Code 130
Running Your Board Without a USB Connection 130 Review 130
5. Programming Revisited 133 133 134 135 136 137 139 141 143 144 146 148 150 151 153 154 154 155 156 157 158
Object-Oriented Programming Classes
The Basics of a Class Class Rules
Public and Private Properties Inheritance Processing: Classes and Files C++: Classes and Files
.cpp and .h A Simple C++ Application
Pointers and References Pointer Reference When to Use Pointers When to Use References Some Rules for Pointers Some Rules for References Pointers and Arrays When Are You Going to Use This?
Review
Table of Contents | v
Your IDE and Computer Windows Mac OS X Linux
Taking Another Quick Tour of C++ Basic Variable Types Arrays Methods Classes and Objects in C++
Getting Started with oF Touring an oF Application
Methods Variables
Creating "Hello, World" Drawing in 2-D
Setting Drawing Modes Drawing Polygons
Displaying Video Files and Images Images Video
Compiling an oF Program Compiling in Xcode Compiling in Code::Blocks
Debugging an oF Application Using the cout Statement Using the GNU Debugger Using the Debugger in Xcode Using the Debugger in Code::Blocks
Importing Libraries ofxOpenCv ofxVectorGraphics ofxAssimpModelLoader ofxNetwork ofxOsc
Review
7. Physical Input Interacting with Physical Controls Thinking About Kinetics Getting Gear for This Chapter Controlling Controls
The Button As an Electrical Object
160 160 161 161 162 163 163 164 165 166 172 172 174 174 176 179 179 182 182 184 186 186 188 188 189 190 190 192 193 195 195 195 195 196 196
..' 199 199 200 201 202 202
vi I Table of Contents
The Button As an Interactive Object Reading the Value of a Button
Turning Knobs The Dial As an Interactive Object Potentiometers
Using Lights Wiring an LED
Detecting Touch and Vibration Reading a Piezo Sensor Getting Piezo Sensors
Detecting Proximity Proximity Through Capacitance
Detecting Motion PIR Motion Sensor
Reading Distance Reading Input from an Infrared Sensor
Understanding Binary Numbers Binary Numbers Bits and Bit Operations Why Do You Need to Know Any of This?
Communicating with Other Applications Sending Messages from the Arduino
openFrameworks Detecting Forces and Tilt Introducing I2C
Gyroscopes What's Next Review
8. Programming Graphics The Screen and Graphics Seeing Is Thinking, Looking Is Reading Math, Graphics, and Coordinate Systems Drawing Strategies
Use Loops to Draw Use Vectors to Draw Draw Only What You Need Use Sprites
Transformation Matrices Creating Motion
Shaping the Gaze Setting the Mood Creating Tweens
202 202 203 203 203 206 206 208 209 210 210 210 216 216 218 220 222 222 222 225 225 228 229 232 237 240 243 244
247 248 250 251 254 254 256 262 263 263 267 268 268 270
Table of Contents | vii
Using Vectors 276 Using Graphical Controls 285
ControlP5 Library 286 Event Handling 286
Importing and Exporting Graphics 288 Using PostScript in Processing 289 Using PostScript Files in oF 290
What's Next 294 Review 294
9. Bitmaps and Pixels 297 Using Pixels As Data 298 Using Pixels and Bitmaps As Input 300 Providing Feedback with Bitmaps 301 Looping Through Pixels 302
ofPixels 303 Manipulating Bitmaps 306
Manipulating Color Bytes 309 Using Convolution in Full Color 310
Analyzing Bitmaps in oF 311 Analyzing Color 312 Analyzing Brightness 314 Detecting Motion 315
Using Edge Detection 321 Using Pixel Data 328 Using Textures 331
Textures in oF 332 Textures in Processing 335
Saving a Bitmap 338 What's Next 339 Review 339
10. Sound and Audio 341 Sound As Feedback 342 Sound and Interaction 345 How Sound Works on a Computer 347 Audio in Processing 350
Instantiating the Minim Library 350 Generating Sounds with Minim 352 Filtering Sounds with Minim 356
Sound in openFrameworks 362 openFrameworks and the FMOD Ex Library 364 Maximilian 371
viii | Table of Contents
Physical Manipulation of Sound with Arduino 381 A Quick Note on PWM 382 Creating Interactions with Sound 385 Further Resources 385 Review 386
11. Arduino and Feedback 389 Using Motors 390
DC Motors 391 Stepper Motors 394 Motor Shields 396
Smart Feedback 397 Using Servos 399
Connecting a Servo 400 Communicating with the Servo 400 Wiring a Servo 401
Using Household Currents 405 Working with Appliances 408 Introducing the LilyPad Board 414 Using Vibration 416
Nano, Fio, and Mini 419 Using an LED Matrix 419
Using the LEDControl Library 419 Using the SPI Protocol 422 Serial LED Matrix 423 Using LCDs 425 Using Solenoids for Movement 429 What's Next 432 Review 432
12. Protocols and Communication 435 Communicating over Networks 436 Using XML 438 Understanding Networks and the Internet 441
Network Organization 441 Network Identification 442 Network Data Flow 443
Handling Network Communication in Processing 443 Client Class 444 Server Class 445 Sharing Data Across Applications 448
Understanding Protocols in Networking 453 Using the ofxNetwork Add-on 454
Table of Contents | ix
Creating Networks with the Arduino 469 Initializing the Ethernet Library 470 Creating a Client Connection 470 Creating a Server Connection 472
Wireless Internet on the Arduino 475 Communicating with Bluetooth 479
Using Bluetooth in Processing 479 Using the bluetoothDesktop Library 480
Communicating Using MIDI 482 Review 486
13. Graphicsand OpenGL 489 What Does 3-D Have to Do with Interaction? 489 Understanding 3-D 490 What Is OpenGL? 491 Working with 3-D in Processing 492
OpenGL in Processing 493 Lighting in Processing 494 Controlling the Viewer's Perspective 496
Making Custom Shapes in Processing 500 Using Coordinates and Transforms in Processing 503
Transformations 506 3-D in openFrameworks 506
Drawing in 3-D 508 Transformations in openFrameworks 509
Lighting in OpenGL 509 Blending Modes in OpenGL 511 Creating 3-D Objects in oF 515 Using Textures and Shading in Processing 519 Using Another Way of Shading 520
What Does GLSL Look Like? 520 Vertex Shaders 521 Geometry Shader 522 Fragment Shader 522 Variables Inside Shaders 523
Using ofShader 524 Using Shaders in Processing 530 What to Do Next 531 Review 532
14. Motion and Gestures 535 Computer Vision 536
Interfaces Without Controls 537
x | Table of Contents
Example CV Projects 538 OpenCV 539 Using Blobs and Tracking 539
Starting with ofxOpenCV 540 Detecting Features with oF 545 Using OpenCV in Processing 549 Feature Tracking in Processing 554 Using Blob Tracking with Physics 559 Exploring Further in OpenCV 565 Detecting Gestures 566
Using ezGestures in Processing 567 Using Gestures in oF 570
Capturing iOS gestures with oF 574 Touch with oF 577
Tuio 577 reacTIVision 578 CCV 578
What's Next 578 Using the Microsoft Kinect 579
Processing 579 openFrameworks 580
Review 580
15. Movement and Location 583 Using Movement As and In Interaction 583 Using Software-Based Serial Ports 585 Understanding and Using GPS 587 Storing Data 599 Logging GPS Data to an Arduino 602
Using the breadcrumbs Library 602 Implementing Hardware-Based Logging 603
Sending GPS Data 606 Getting Location on a Mobile Device 608
On the iPhone 608 On an Android Phone 609
What to Do Next 610 Review 610
16. Spaces and Environments 613 Using Architecture and Space 613 Sensing Environmental Data 614 Using an XBee with Arduino 615
Creating a Simple Test 618
Table of Contents | xi
Configuring the XBee Module Addressing in the XBee XBee Library for Processing
Placing Objects in 2-D Using the X10 Protocol Setting Up an RFID Sensor Reading Heat and Humidity Determine Position of an Object What's Next Review
17. Further Resources What's Next?
Software Tools Construction Processes Artificial Intelligence Physics Hardware Platforms
Bibliography Interaction Design Programming Hardware Art
Conclusion 673
Index 675
620 622 623 627 634 637 643 648 655 655
657 657 657 660 661 662 668 670 670 671 672 672
xii | Table of Contents