Multiplying 10-digit numbers using Flickr:The power of recognition memory

Andrew Drucker∗

1 The recognition method

In this informal article, I’ll describe the “recognition method”—a simple,powerful technique for memorization and mental calculation. Compared totraditional memorization techniques, which use elaborate encoding and visu-alization processes [1], the recognition method is easy to learn and requiresrelatively little effort.

There is a catch: to apply the method, you need continuous access to alarge collection of interesting, unfamiliar photographs. (This is easy to setup on a home computer, using the photo-sharing website Flickr.com, and I’llexplain how.) Now, some might object that this is cheating, since the photosact as aids to memory. However, the method doesn’t alter or rearrange thephotographs during the process, so their role as aids is very limited comparedto pen and paper.

The basic idea for the method is not new; it comes from the psychologicalliterature [2], and has been applied by computer scientists to the designof computer password systems [3, 4]. However, it seems not to have beenexplored before as a systematic mnemonic device, or as a tool for mentalcalculations.

∗MIT CSAIL, Cambridge, MA, 02139. Supported by a DARPA YFA grant of ScottAaronson. Email: andy.drucker@gmail.com

c© 2011 Andrew Drucker

1

The method works : using it, I was able to mentally multiply two random10-digit numbers, by the usual grade-school algorithm, on my first attempt! Ihave a normal, untrained memory, and the task would have been impossibleby a direct approach. (I can’t claim I was speedy: I worked slowly andcarefully, using about 7 hours plus rest breaks. I practiced twice with 5-digitnumbers beforehand.)

I chose multiplication as a widely-familiar problem that many people cansolve with pen and paper. However, the recognition method can be appliedmuch more broadly. Although I haven’t attempted this, it seems that themethod could be readily used to mentally simulate general models of com-putation, such as Turing machines [5]. Of course, human memory is finite,and this puts a bound on the size of mental computations we can perform;but the recognition method suggests that our brains’ capacity for precisecomputation is much greater than we might naively expect.1

My main goal here is to expose more people to the amazing power of recogni-tion memory, and to describe a concrete application (mental multiplication)that readers can try at home. In the last part of the article, I’ll also specu-late about the ultimate power of the “recognition method,” and share someideas about a wacky new kind of memory competition based on recognitionmemory—a competition that could astound, entertain, and perhaps evenyield new insights into the human mind.

Recognition tasks

Suppose I have a collection of pairs of images. From each pair, I select oneat random to show to you. Afterwards, I choose a random pair, show youboth images, and ask you to identify the one you’ve seen before. (Note thatwe are testing recognition, as opposed to a more active recall of features fromthe image.)

1Conventional wisdom suggests that human brains are inferior to computers in at leasttwo respects. First, we’re slow and error-prone when doing basic logical operations; second,we have a feeble capacity for storing “meaningless” numerical data in working memory.

The recognition method is a powerful “hack” to partly overcome this second weakness.Can similarly powerful hacks be found to improve our speed and reliability at logic? I findthis question fascinating.

2

It turns out that ordinary people are incredibly good at this task. In one ofthe most widely-cited studies on recognition memory, Standing [2] showedparticipants an epic 10,000 photographs over the course of 5 days, with 5seconds’ exposure per image. He then tested their familiarity, essentially asdescribed above. The participants showed an 83% success rate, suggestingthat they had become familiar with about 6,600 images during their ordeal.Other volunteers, trained on a smaller collection of 1,000 images selected forvividness, had a 94% success rate.

As you’d expect, recognition memory works best when the images beinglearned are interesting and sufficiently distinct. Also, similar training pro-cesses based on recognizing word combinations have somewhat less impres-sive success rates. A number of studies suggest that our visual recognitionmemory is particularly strong [6].

Exploiting recognition

In recent years, Standing’s and related studies helped inspire an interestingline of work in an unexpected area: computer security. By familiarizing anindividual with a randomly chosen subset of images from a fixed collection,a computer program can create a form of knowledge held by that individualalone. This knowledge can then be used to reliably identify that person. Inother words, recognition memory can serve as a basis for password systems(see [3] for a survey). For example, in the commercially-available Passfacessystem [4], a user’s “password” is a set of familiar faces.

Passfaces and related systems use, either implicitly or explicitly, a “recogni-tion method” that allows numerical data to be encoded into a user’s famil-iarity with certain images. The basic idea is as follows. Say that we wantto memorize 5 numbers, each in the range 0-9. Let’s call these numbersn0, n1, . . . , n4. (It’s most convenient to index starting from 0.)

We acquire 50 previously-unseen photos, and label them with numeric indices00 to 49. We think of the photos 00, 01, . . . , 09 as forming a “memory cell”which is to “store” our value n0. Similarly, the photos 10, 11, . . . , 19 form acell which will store n1, and so on. To store n0, we first inspect its value. Forconcreteness, suppose n0 = 4; then we look at the photo 04, and familiarizeourselves with it:

3

Figure 1: Photo 04.

Now n0 is “stored,” and we go on to store n1. If n1 = 7 (say) then we lookat photo 17. We store the other values similarly.

Later on, when we want to retrieve n0, we just look at photos 00, 01, . . . , 09:

Figure 2: Photos 00-09. One was seen before.

By recognizing the image 04 as familiar, we infer that n0 = 4.

Note that, while retrieving n0, we may become familiar with some of the otherimages in the range 00, . . . , 09. This may degrade or destroy our “encoding”of n0. Now in practice, I’ve found that this is usually not a major problem:if we study the desired image intently during the encoding process, and passquickly over the others during retrievals, we can reuse an encoding multipletimes. But there is also a more reliable fix: we can make multiple copies ofour encoding into auxiliary cells, and use each copy only once. (The priceis that a larger number of photos and cells are required.) Happily, in themental-multiplication procedure I’ll describe, each stored value only needsto be retrieved once.

4

2 Mental calculations with the recognition method

We’re now ready to see how to multiply large numbers with the recognitionmethod.

Our approach is based on the usual multiplication algorithm taught in Amer-ican schools. I won’t review the method in any detail, but here is a quick,worked example. In Phase 1, we multiply the top number by each of thedigits of the bottom number:

Figure 3: After Phase 1.

For example, the middle row is obtained as 1308 = 327 × 4. In Phase 2, wethen pad with zeros and add up the numbers from Phase 1, as follows:

Figure 4: After Phase 2.

(I haven’t shown the “carry”-digits in this example.)

5

Now let’s see how to perform the same multiplication using the recognitionmethod. The following important diagram will be our guide:

Figure 5: A visual guide to our method.

Here we’ve taken all the positions in which information would be stored by thepen-and-paper method, and assigned these positions “indices.” The indicesare coordinates, using the top-right storage position as our reference. Forexample, the index “12 ” is 1 step down, and 2 steps left from the top-rightposition.

As you might have guessed, each index will become a “storage cell” of thekind described earlier. We begin with a collection of 360 images, labeled 000to 359, which will correspond to cells 00 through 35 . To mentally multiplythese numbers with the recognition method, we follow the ordinary mul-tiplication method exactly. The only difference is that, whenever we wouldordinarily write a number into a position, we will instead encode that numberinto our recognition memory, by the method described earlier. For example,to “write” a 0 into the position corresponding to cell 12 , we would just fa-miliarize ourself with image 120. Similarly, whenever we would ordinarilyread a number from a position, we will instead look at the ten images of thecorresponding cell, and see which one is familiar.

Let’s walk through the first few steps of the procedure applied to the product

×327246 , with reference to Fig. 5. To fill cell 00 , we mentally compute the

product 6 × 7 = 42. We store the 2 in cell 00 by looking at image 002.

6

We then carry the 4. I use my fingers to store such carry-digits, allowingmyself this most human of memory aids. (It’s best to have a one-handedsign for any number between 0 and 10; you can use the Chinese method, forexample [7].)

We go on to compute 6 × 2 = 12, plus 4 equals 16. So we store the 6 in cell01 by looking at image 016, and we carry the 1. In this way, we carry outall the multiplications in Phase 1.

In Phase 2, it’s time to retrieve the stored values and add them up, columnby column, to fill out the final row. For example, in the rightmost column,there is only a single term to be added, which can be found in cell 00 . Sowe just look up that value and transfer it into the bottom-row cell 30 . Tofill cell 31 , we add up the contents of cells 01 and 11 . As it happens, weend up with a 1 to carry into the next column. We proceed in this way untilwe’ve filled the bottom row; this is our final answer, and can be read off onedigit at a time by inspecting the cells.

That’s the scheme in a nutshell. But how to set up the needed photo library?And, is it really possible to carry out a large multiplication? If so, what tricksare helpful? These are the questions I’ll address next.

Getting a Photo Library

Setting up a photo library breaks down into two tasks: first, assembling alarge number of images (hopefully interesting and memorable ones); second,renaming them in numerical order to organize them into “cells.”

To get my images, you can use the photo-sharing site Flickr.com, wherethousands of new photos are uploaded daily. Step one is to get a free Flickrmembership. Next, a free program called Bulkr (Win/Mac/Linux) allows youto bulk-download images from the site. Bulkr will only download a photowhose owner’s permissions/copyright settings on Flickr allows downloads;my understanding is that it’s legitimate, legal software.

7

Figure 6: The Bulkr interface.

Bulkr has an “Explore” feature that brings up the most “interesting” photosuploaded on any given day, as voted by Flickr users. I paid to upgrade to thefull version of Bulkr, to access more of these photos and download hundredsat a time.

To rename the files with numerical indices, I used the Automator tool on myMac. It has a function called “Make Finder Item Names Sequential” thatdoes the trick nicely. For Windows, a program called Bulk Rename Utilityseems to do what we want, and for Unix-type systems, there are a variety ofoptions.

Making the scheme work

If you’ve read this far, perhaps you’re even curious enough to try the method.If so, here are some tips to maximize your chances of success.

8

1. Be wary of not-so-memorable images. I’ve found that, by and large, thephotos I download from Bulkr are of good quality and suitable for the task.But there are some pitfalls. Landscape photos can blur together in memory,as can photographs from a specific niche—for instance, there’s a ludicrousamount of Lego photography on Flickr.

The simplest way to address this problem is to spend more time familiarizingyourself with photos you consider less memorable. A more careful failsafe isto build a “backup” collection of images, for use when you hit a dud photo.I used this technique for my 10-digit multiplication task.

2. Familiarize yourself well with the cell-index system you’ll be using. It’sespecially important to be able to easily determine into which cell a particularvalue is to be stored.

If you find it confusing to locate cells, it might make matters easier to createa different folder on your desktop for each cell, and arrange them according totheir use in the multiplication procedure. If my cell-indexing scheme doesn’twork for you, invent your own.

3. Make sure you can remember what you need to remember. I don’t meanremembering the images you’ve seen—you’ll find that this part is surprisinglyeasy. The challenge is to keep track of where you are in the algorithm,while also holding on to carry-digits and (in Phase 2) the running column-sums.

I found that with 5-digit numbers to multiply, this was possible with reason-able concentration. For my 10-digit challenge, I used a simple observationto reduce the task to four 5-digit multiplications, plus a final addition (allwithout pen and paper). For example, we can write

1111122222 × 3333344444 = (11111 × 33333) × 1010

+ (11111 × 44444 + 22222 × 33333) × 105

+ 22222 × 44444.

If you find the approach I’ve sketched places too much demand on yourworking memory, it’s possible to enlist additional image-cells to help keeptrack of carry-digits, running sums, and your place in the algorithm.

9

Putting it to the test

On a sunny California day in June 2011, I grabbed some numbers from atable of random digits. In 7 hours of work, I used the recognition method tocalculate

9883603368 × 4288997768 = 42390752785149282624.

I have no special abilities or memory training, and what I did was not reallya feat. I firmly believe that an average person who knows how to multiplycould do the same thing with a little training (and do it much faster, withpractice). Also, as I’ve said, the technique is hardly limited to arithmetic—you can choose your own wild application. As one possibility: for thosefamiliar with cellular automata (CA), I would love to see someone mentallysimulate a long run of Conway’s Game of Life, or some other interesting CArule, with this method.

3 Toward new memory feats

The recognition method can enable ordinary people to use their memoryin startling ways. I hope readers will be inspired to see for themselves.But I admit, I’m also very curious what the real memory fiends out therecould do with the technique. Memory competitions are a thriving concerntoday; for an entertaining account (which helped inspire my own project),I recommend a recent book by Joshua Foer [1]. As an example of whatthe very best mnemonists (skilled memorizers) can do: recently Wang Fengof China became famous in his country by memorizing 480 random digitsin 5 minutes, easily a new world record [8]. Interestingly, most of the topmnemonists seem to have no innate, savant-like abilities; their achievementsare the product of technique and training.

Now suppose we allowed competitors to view the digits, encoded as imagesby the recognition method, and allowed them to decode the digits from mem-ory using the same collection of images. How much faster could they take ininformation? I believe the answer is much faster, and I’d love to see this con-firmed. It’d also be interesting to know whether current mental-arithmeticrecords could be broken with the aid of image collections. (Of course, this

10

would be a distinct category of competition. I should mention that mental-arithmetic champions can already do 10-digit multiplications in a matter ofminutes, without image aids.)

Such contests could have a side benefit to psychology: by allowing competi-tors to choose their own image collections, we could see a drive to identify themost memorable imagery available. This could help scientists to better un-derstand what features are likely to make an image memorable. Now, there’sbeen significant scholarly study of image memorability, some of which em-ploys familiarity-testing very similar to the recognition method—see [6, 9, 10].However, so far this work seems to have been done with ordinary test sub-jects, not trained mnemonists; and the images used have been supplied by theresearchers, not the subjects. I believe that injecting an element of competi-tion into this research could be a fruitful (and entertaining) direction.

The CAMRA game

For the task of memorizing raw data, I’d like to propose another new categoryof competition—one that generalizes the recognition method, and could beeven more interesting than the competition I suggested above. I’ll call thisnew proposal the CAMRA game, short for Computer-Aided Memorizationand RetrievAl.

In this competition, a contestant gets to bring along two computer programsof her choosing; we’ll call these Trainer and Retriever. To be concrete, let’sassume the task is to memorize 100 random digits, in as little time as possible.(What I mean by “memorize” is a bit unusual, as you’ll see.)

To set things up, the referees provide 100 randomly chosen digits as input toTrainer. Next, the game works as follows:

Stage 1 (“Training”): The buzzer sounds, and timing begins. The con-testant interacts with Trainer until she feels ready for the next stage. (Theidea is that Trainer is teaching her the random digits, perhaps in an alteredform.)

Stage 2 (“Retrieval”): The contestant moves to a second computer, whereshe interacts with Retriever. This second program has not been given access

11

to the random digits, and cannot communicate with Trainer. The decodingstage ends when Retriever outputs a sequence of 100 digits.

The contestant is successful if Retriever ’s output exactly matches the randominput digits to Trainer. Her time used could be measured, either as the timeused for both stages, or just the time used in the Training stage.

So what’s the point? In this game, the human contestant acts as the only“bridge” between Trainer, which knows the random digits, and Retriever,which does not. Thus if she can get Retriever to output the correct digits,she can meaningfully claim to have “memorized” those digits—albeit in aform that might be very indirect.

I believe that holding competitions for the CAMRA game could have sig-nificant scientific value. Here’s why. When mnemonists memorize randomdata (without computer assistance), they typically perform all kinds of imag-inative visualizations to transform the random data they want to learn intomeaningful, memorable ideas. The trouble is, we can’t directly see any ofthis; to learn about mnemonists’ techniques, we are largely dependent ontheir verbal accounts.

One way to think about the Trainer program’s role in the CAMRA game isas a partial externalization of the process by which information is made mem-orable. The hope is that, by studying the kinds of Trainer programs thatskilled contestants develop, we might get a more direct glimpse into howthis meaning-making activity works. (Of course, the distinctive strengthsand weaknesses of computers will also strongly shape the way the game isplayed.) Also, since the Trainer program should help make random datamore meaningful, the human contestants should be able to spend more timeabsorbing data (rather than imaginatively transforming/encoding it). Thusthe CAMRA game could be one way to “pry apart” the activities of en-coding and absorbing information, helping us to study these activities sepa-rately.

But I’ll admit—mostly, I hope mnemonists try CAMRA because I think itwould be fun to watch, and because I’m curious what skilled play wouldlook like. One possible strategy for the CAMRA game is to make Trainerand Retriever share a collection of memorable images, and implement therecognition method as I’ve described it. But this could be problematic: if a

12

contestant uses the recognition method at blazing speed, she will very likelymake a few mistakes. A certain fraction of mistakes can actually be corrected,however, with a little more sophistication. The basic idea is that Trainer canuse the recognition method to teach the contestant, not the random digitsthemselves, but an encoding of those digits under an error-correcting code—a powerful mathematical tool that’s widely used in electronic communica-tions.

Beyond these preliminary ideas, I’d love to see what new innovations contes-tants might bring to the task. An especially interesting feature of such com-petitions is their potential for collaboration: a contestant might develop herTrainer/Retriever programs with the aid of coders and psychologists.

A purely-mental recognition method?

Dedicated mnemonists might also try to take the recognition method in adifferent direction: it is at least conceivable that the basic technique could beused for memorization, even without access to an image library! How couldthis work? The hope is that with training, there could be a way to mentally“translate” sequences of numbers into meaningful, memorable mental images,in such a way that even very similar sequences would translate to sharplydistinct images. With this ability, one would simulate the recognition methodas described earlier, with a “virtual” collection of images.

This idea is not without precedents. Designers of computer password sys-tems have proposed algorithms to automatically translate random numbersinto memorable images, in order to apply the recognition method for userauthentication [11, 3]. It seems that these algorithms would be very difficultto simulate mentally, however.

Classical mnemonic techniques are also built around the idea of translatingnumbers into memorable images, which mnemonists then try to actively re-call [1]. Could some of these visualization practices could be adapted foruse with the recognition method? If so, could such a method have advan-tages over existing mnemonic techniques? I consider this an interesting openquestion.

13

Conclusion

Human recognition memory is a marvel, a great gift we all share. I wouldlike to exhort readers to see for themselves by trying the recognition method:Learn π’s digits with imagery! Play the CAMRA game! Go forth and (men-tally) multiply!

But ultimately, while I hope that the proposals in this article get exploredfurther, they’re just a few possibilities among many. I hope readers will beinspired (whether as researchers, as competitors, or as curious individuals)to find more exciting new ways to explore the power of human memory.

Acknowledgments

I’m grateful to Scott Aaronson, Luis Miguel Dickson, and John Ervin forhelpful comments—with extra thanks to Scott for encouraging me to “actu-ally do it.”

References

[1] Joshua Foer. Moonwalking with Einstein: The Art and Science of Re-membering Everything. Penguin Press HC, 2011.

[2] Lionel Standing. Learning 10000 pictures. Quarterly Journal of Exper-imental Psychology, 25:207–222, May 1973.

[3] Xiaoyuan Suo, Ying Zhu, and G. Scott Owen. Graphical passwords: Asurvey. In 21st IEEE Annual Computer Security Applications Confer-ence (ACSAC), pages 463–472, 2005.

[4] Passfaces (website). http://www.realuser.com/.

[5] Turing machine (wikipedia article). http://en.wikipedia.org/wiki/Turing_machine.

[6] W. Howard Levie. Picture recognition memory: A review of researchand theory. Journal of Visual/Verbal Languaging, 8:6–45, 1988.

14

[7] Brendan Connal. How to count to ten on one hand(in Chinese). http://www.instructables.com/id/

HOW-TO-COUNT-TO-TEN-ON-ONE-HAND-in-Chinese/.

[8] Memory achievements (webpage). http://www.

worldmemorychampionships.com/MemoryAchievements.asp.

[9] Phillip Isola, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. Whatmakes an image memorable? In 24rd IEEE Conference on ComputerVision and Pattern Recognition, pages 145–152, 2011.

[10] What makes an image memorable? (popular summary). http://

web.mit.edu/newsoffice/2011/memorable-images-0524.html. MITnews office, 2011.

[11] Adrian Perrig and Dawn Song. Hash visualization: a new technique toimprove real-world security. In In International Workshop on Crypto-graphic Techniques and E-Commerce, pages 131–138, 1999.

15