Bee Conjuring andSwarm Orchestration

Optimizing Collaborative Innovation

April 12, 2006Ned GulleyThe MathWorks, Inc.

1. Conjuring a crowd

2. Keeping them busy

3. Social issues

4. Future directions

1. Conjuring a crowd

2. Keeping them busy

3. Social issues

4. Future directions

Problem design

Micro-rewards

War stories

Next big thing

Contest features

• Entries are automatically scored, ranked, and displayed immediately

• Code, author, and score are visible at all times

• Anyone can modify anyone else's code and resubmit it as their own

How it worksrules

standings

standings

1 karim2 carol3 eric

karim.fcn f(x)...

view entry

new entry

ned.fcn f(x)...1 ned

2 karim3 carol4 eric

What makes a good problem?

• Low cost of admission

• Plenty of headroom

Blockbuster

What to reward?

• A good answer

• Gotten quickly

Better

Time

Wednesday Monday

BetterBetterAnswer

Faster Result

1000!

Social issues

• hacking

• tweaking

• obfuscation

• acknowledgment

• scoring inequities

• tweak bombs

• clone bombs

Obfuscation

function A = domove(A,mv) N(:) = false; findNeighbors(mv(1),mv(2),A(mv(1),mv(2))); anyN = any(N); for j = 1:size(A,2) if anyN(j) …

Obfuscation

zzz1018=zzz1018+zzz1032;for zzz1069=2:zzz1014if zzz1048(zzz1018),zzz1041(zzz1018)=zzz1018;zzz1053=zzz1018-zzz1032;if zzz1048(zzz1053)==zzz1048(zzz1018)zzz1041(zzz1053)=zzz1018;end…

Two can play at this game. It's sad that this has sunk to this level. Can't we return to the spirit of the contest?

Alan Chalker says…

…"spirit of collaboration" vs "spirit of competition"?! This is a competition right? Otherwise, anyone up for a round of cum-ba-ya?

Dr. Seuss says…

It is a competition, but with certain rules and goals. One of the goals is collaboration and improving other codes. The darkness and twilight zones are added to give people time to develop their own algorithms without being tempted too much for just tweaking others codes.

Stijn Helsen says…

Titles as conversation

• Scrambled Eggs• Rotten Eggs• I didn't start the fire• don't get obfuscated ... follow the light• You Call This Collaboration? Give

Me A Break

Titles as conversation

• Tweak Tweak Tweak• tweakfest • tweak-o-rama• slippery ground (tweak me)• Seems like everyone's just tweaking• I said to myself not to do this kind of

things....• Poopdeck Pappy gives in to peer

pressure

Titles as conversation

• I wish I knew how this works

Player types

• Spectators, Lurkers

• Drop-ins, Visitors

• Tweakers, Top-grazers

• Algorithm Artists

• Churners, Integrators

• Warriors, Obfuscators

Player types

1. 2. 3. 4. …

Tweaker (many)

Churner (few)

Drop-in (many)

First use of

conv

First leader

with conv

Features we’ve added

• Leaderboard • Lots of realtime stats • Mini-contests • Mid-contest analysis • Darkness and twilight • Newsgroup discussion • Contest blog • File differencing

Future directions

What’s the goal?

• Fun == Optimizing innovation.

How do we optimize?

• Modulate diffusion

• Modulate testing

• Listen to the contestants!

Putting the bees to work

• Contest to create shipping code

• Rent-a-swarm

• Create your own contest

The Big Lesson: Focus on the small

• Approachable problem

• Low start-up cost

• Acknowledge effort with micro-rewards

• Many small prizes

Similarity Distance

Contest EvolutionVirus Evolution