Introducing Computer Programming with Multimedia Barb Ericson Institute for Computing Education Georgia Institute of Technology [email protected]

Introducing Computer Introducing Computer Programming with Programming with

MultimediaMultimediaBarb EricsonBarb Ericson

Institute for Computing EducationInstitute for Computing Education

Georgia Institute of TechnologyGeorgia Institute of [email protected]@cc.gatech.edu

Computer Science CrisisComputer Science Crisis

• 40% drop in majors since 2001 Nationwide– Up to 60% drop in some places

• Percentage of women is down to about 12%– From a high of 30% in the 1980s

• Never attracted many African Americans, Hispanics, Native Americans

Why the Decline?Why the Decline?

• Dot com crash in 2001– Loss of jobs

• AP Exam switched from C++ to Java– Many schools quit teaching AP Computer Science

• Visual Basic changed– In .net to be object-oriented

• Perceptions of Computer Science– boring, antisocial, not creative, and just programming

Looming ShortageLooming Shortage

• Only a few jobs have gone overseas– About 2-3%

• We have recovered from the dot com crash– There are now more jobs in this area than in

2001

• Computing jobs are some of the fastest growing– US Dept of Labor predicts 45.93% growth

Reversing the TrendReversing the Trend

• We need to get more students interested in computer science– Not drive them away

• We need to appeal to a diverse group of students– Not just geeks

• All students need to have some understanding of computer science– Our society relies on technology

Our SolutionOur Solution

• Use multimedia projects to introduce computing concepts– Pictures, sounds, movies, text

• Use open-ended projects to encourage creativity– Create an image collage, create a sound

collage, create a movie

• Make computing social– Encourage students to show off their work

Choice of LanguagesChoice of Languages

• Python– Used by many companies: IL&M, Google, Nextel, etc.– Easier syntax than many languages– Can use to teach functional, procedural, or object-

oriented programming– Used as a scripting or extension language for many

applications– Growing in popularity

• Java– Popular language used for AP CS

Digitizing PicturesDigitizing Pictures

• We digitize pictures into lots of little dots

• Enough dots and it looks like a continuous whole to our eye

• Each picture element is referred to as a pixel– Each pixel knows its color– It also knows where it is in

its picture

Encoding Color: RGBEncoding Color: RGB

• In RGB, each color has three component colors:– Red, Green, and Blue

• In most computer-based models of RGB, a single byte (8 bits) is used for each– So a complete RGB color

is 24 bits, 8 bits of each– 0,0,0 is black– 255,255,255 is white– 255,0,0 is red

Make and Show a PictureMake and Show a Picture

Use pickAFile function To select a picture file(ends in .jpg)

Use makePicture(file) to create the picture

Use show(picture) todisplay the picture

Use MediaToolsUse MediaTools

• Click on MediaTools• Select the picture to

explore• Move the cursor while

holding down the left mouse button– To see x,y,r,g,b

• Click on zoom– To zoom in or out

Creating a Python FunctionCreating a Python Function

Define a function

Load the function

Try the function

Rules for Python FunctionsRules for Python Functions

• Use the keyword def to define a function– Followed by the

function name

• Use parentheses after the name– Name parameters

• End with a colon• Indent the body of the

function

def pickAndShow(): myFile = pickAFile() myPicture =

makePicture(myFile) show(myPicture)

Using a LoopUsing a Loop

def decreaseRed(picture):

for p in getPixels(picture):

value = getRed(p)

setRed(p,value * 0.5)

More Picture FunctionsMore Picture Functionsdef greyscale(picture): for p in getPixels(picture): redness=getRed(p) greenness=getGreen(p) blueness=getBlue(p) luminance=(redness+blueness+greenness)/3 setColor(p,

makeColor(luminance,luminance,luminance))

def negate(picture): for px in getPixels(picture): red=getRed(px) green=getGreen(px) blue=getBlue(px) negColor=makeColor(255-red,255-green,255-

blue) setColor(px,negColor)

How Sound Works:How Sound Works:

• Sounds are waves of air pressure– Sound comes in cycles– The frequency of a

wave is the number of cycles per second (cps), or Hertz

– The amplitude is the maximum height of the wave

Volume and PitchVolume and Pitch

• Our perception of volume is related (logarithmically) to changes in amplitude– If the amplitude doubles, it’s about a 3 decibel (dB)

change.

• Our perception of pitch is related (logarithmically) to changes in frequency– Higher frequencies are perceived as higher pitches– We can hear between 5 Hz and 20,000 Hz (20 kHz)– A above middle C is 440 Hz

Digitizing SoundDigitizing Sound

• We can estimate the sound curve– Analog-to-digital

conversion (ADC) will give us the amplitude at an instant as a number: a sample

– How many samples do we need?

• 2 x max frequency• CD quality is 44,100

samples per second

Digitizing sound in the computerDigitizing sound in the computer

• Each sample is stored as a number – 16 bits, 216 = 65,536 combinations– But we want both positive and negative values

• To indicate compressions and rarefactions.

• Each sample can be between -32,768 and 32,767

Exploring a SoundExploring a Sound

Use MediaTools to explore a Sound

Working with SoundsWorking with Sounds

• We’ll use getSamples to get all the sample objects out of a sound– Use getSample to get the value from a Sample obj

• We can also get the value at any index with getSampleValueAt

• Sounds also know their length (getLength) and their sampling rate (getSamplingRate)

• Save a sound to a file using writeSoundTo(sound,”file.wav”)

Decreasing the volumeDecreasing the volume

def decreaseVolume(sound): for sample in getSamples(sound): value = getSample(sample) setSample(sample,value * 0.5)

This works just like decreaseRed, but decreasing the amplitude (volume) instead of the red color.

Multimedia Java CS1Multimedia Java CS1

• We use DrJava– A free and easy development environment– With an interactions pane

• We have created Java classes– For the students to use and modify

• World• Turtle • Picture• Sound• FileChooser

• Introduction to Java– Math operators, printing results, data types, casting,

relational operators, Strings, variables

• Introduction to Programming– Creating and naming objects

• Using a turtle and a world

– Creating new Turtle methods• Draw simple shapes• Using parameters

Java SyllabusJava Syllabus

Syllabus - ContinuedSyllabus - Continued

• Modifying Pictures using Loops– One-dimensional

arrays– Use for-each, while,

and for loops to– Increase/decrease

colors, fake a sunset, lighten and darken, create a negative, and grayscale

Create and Show a PictureCreate and Show a Picture

Java Negate MethodJava Negate Method

/** * Method to negate the picture */public void negate(){ Pixel[] pixelArray = this.getPixels(); Pixel pixelObj = null; int redValue, blueValue, greenValue = 0;

// loop through all the pixels for (int i = 0; i < pixelArray.length; i++)

Java Negate Method - ContJava Negate Method - Cont

{ // get the current pixel pixelObj = pixelArray[i];

// get the values redValue = pixelObj.getRed(); greenValue = pixelObj.getGreen(); blueValue = pixelObj.getBlue();

// set the pixel's color pixelObj.setColor( new Color(255 - redValue, 255 - greenValue, 255 - blueValue)); } }

Testing NegateTesting Negate


• Modifying Pixels in a Matrix– Two-dimensional

arrays– Nested loops– Copying, mirroring,

blending, rotation, scaling


• Conditionally Modifying Pixels– Replacing a color,

reducing red-eye, edge detection, sepia-toned, posterize, highlight extremes, blurring, background subtractions, chromakey


• Drawing on Pictures– Using existing Java

classes • Inheritance• Interfaces

– Drawing simple shapes, drawing text, general copy, general scale, shearing, gradient paint, general blending, clipping


• Modifying all Samples in a Sound– 1D arrays– Loops– Conditional execution– Change volume,

normalizing a sound (make it as loud as possible), force to extremes


• Modifying Samples using Ranges– Loops– Clipping, splicing,

reversing, mirroring


• Combining and Creating Sounds– Class and private

methods– Composing

sounds, blending sounds, changing frequencies, and creating echoes

– MP3 and MIDI


• Creating Classes– Defining a class– Overloading constructors– Creating and initializing an

array– Creating getters and setters– Creating a main method– Javadoc comments– Using Inheritance

ConfusedTurtleTurtle

Syllabus - ContinuedSyllabus - Continued• Creating and Modifying

Text– Reading from and writing

to files• Handling Exceptions

– Modifying programs– Getting text from networks– Creating random

sentences– Using text to shift between

media

Syllabus - ContinuedSyllabus - Continued• Making Text for the

Web– Throwing exceptions,

HashMap, Generics, and Iterators

– Create a web page from a directory

– Create a web page from other web pages

– Creating a web page from a database


• Encoding, Manipulating, and Creating Movies– Frame-based

animations with simple shapes and text

– Special effects – fade out, fake sunset, and chromakey


• Speed– What makes programs

fast?• Compilers and Interpreters• Searching• Algorithms that can’t be

written

– What makes computers fast?

• Clock rates, Storage, Display


• Javascript– Syntax– User Interfaces– Multimedia

Multimedia CS2 in JavaMultimedia CS2 in Java

• Driving question: “How did the wildebeests stampede in The Lion King?”

CS2 IdeasCS2 Ideas• Teach data structures using multimedia

– Stacks and queues for simulations– Linked lists of pictures

• Order in linked list is order in scene• Order in linked list is the layer the picture is in• Circular linked list for animation

– Linked lists of sounds• Weave and repeat musical phrases

– Trees of pictures• Scene graph

– Trees of sounds• Different traversal gives different result

Examples of Student WorkExamples of Student Work

Soup-Audio Collage

Canon- LinkedList of (MIDI) Music

ResultsResults

• Python CS1 is used for non-majors at Georgia Tech– Business students went from 49% to 88% pass rate– Similar results at Gainesville and U.Ill-Chicago

• Java CS1 is used in Georgia High Schools– Both in an introductory course and in Advanced

Placement courses– Also used at Dennison and Duke

• Java CS2 is used at Georgia Tech– Students are transferring into computer science

Why Media Computation?Why Media Computation?• Works on nearly all computers

– Unlike Alice which requires newer computers

• Low cost– Unlike LEGO Mindstorms or other robots

• Students can see (and hear) if their programs work• Computer Science concepts are introduced to solve

problems– Looping instead of changing thousands of pixels by hand

• Open-ended assignments allow students to be creative– Students can use their own media

• Students can publicly share their media artifacts with others

What’s on the CDWhat’s on the CD

• CS1-Python materials– Chapters from the Media Computation book in Python– Course slides– Free Jython Environment for Students (JES)– Instructor Resources

• CS1-Java materials– Chapters from the Media Computation book in Java– Course Slides– Java Classes

What’s on the CD - ContinuedWhat’s on the CD - Continued

• CS2-Java materials– Course notes– Course slides– Java Classes

• DrJava– Free Development environment for Java

• Mediasources– Royalty free pictures and sounds

Additional ResourcesAdditional Resources

• Web Site for Media Computation– http://coweb.cc.gatech.edu/mediaComp-plan

• CS1 Python Book– Introduction to Computing and Programming in

Python, A Multimedia Approach by Mark Guzdial, ISBN: 0131176552

• CS1 Java Book– Introduction to Computing and Programming with

Java, A Multimedia Approach by Mark Guzdial and Barbara Ericson, ISBN: 0131496980

• CS2 Java Book– Coming in 2007

Funding SourcesFunding Sources

• National Science Foundation

• Georgia Tech's College of Computing

• Georgia’s Department of Education

• Al West Fund

• Toyota Foundation

Documents

Introducing Computer Programming with Multimedia Barb Ericson Institute for Computing Education Georgia Institute of Technology [email protected]