ALGORITHM-DRIVEN DESIGN

WILL ROBOTS REPLACE DESIGNERS?

YURY VETROV

MAIL.RU GROUP

WILL ROBOTS REPLACE DESIGNERS?In 2016, the technological foundations of these tools became easily accessible, and the design community got interested in algorithms, neural networks and artificial intelligence (AI).

Now is the time to rethink the modern role of the designer.

THE GRID CMSIt chooses templates and retouches and crops photos all by itself.

It runs A/B tests to choose the most suitable patterns.

OR IT ISN’T?The product was in private beta until recently, so we were able to judge it only by its publications and ads.

The Designer News community found real-world examples of websites created with The Grid, and they had a mixed reaction –people criticized the design and code quality. Many skeptics opened a champagne bottle on that day.

FULLY REPLACING A DESIGNER WITH AN ALGORITHM? UHM, A-HA, YES…The idea was praised by The Grid and some technologists –it sounds futuristic, but the whole point is wrong.

Product designers help to translate a raw product idea into a well-thought-out user interface, with solid interaction principles and a sound IA and visual style, while helping a company to achieve its business goals and strengthen its brand.

CREATIVE COLLABORATION

WITH ALGORITHMSDesigners work “in pair” with algorithms

to solve product tasks

JUGGLERSDesigners have learned to juggle many tools and skills to near perfection. As a result, a new term emerged, “product designer.”

HEAVYISH…However, balancing so many skills is hard – you can’t dedicate enough time to every aspect of product work. Of course, a recent boon of new design tools has shortened the time we need to create deliverables and has expanded our capabilities. However, it’s still not enough. There is still too much routine, and new responsibilities eat up all of the time we’ve saved.

We need to automate and simplify our work processes even more.

*

*

CONSTRUCTING A UI

SIMPLE PUBLISHING TOOLSPublishing tools such as Medium, Readymag and Squarespace have already simplified the author’s work – countless high-quality templates will give the author a pretty design without having to pay for a designer.

There is an opportunity to make these templates smarter, so that the barrier to entry gets even lower.

WIX ADVANCED DESIGN INTELLIGENCEA semi-automated website constructor that enables

non-professionals to create a website. Sounds like The Grid?

FIREDROPThe idea is similar to The Grid. You provide the content, then a virtual

assistant helps you to create a layout and choose a visual style.

PAIRED DESIGN WITH A COMPUTERSurely, as in the case of The Grid, rejecting designers from the creative process leads to clichéd and mediocre results (even if it improves overall quality).

However, if we consider this process more like “paired design” with a computer, then we can offload many routine tasks.

FREEDOM FROM ROUTINEOf course, we can’t create a revolutionary product in this way, but we could free some time to create one. Moreover, many everyday tasks are utilitarian and don’t require a revolution.

If a company is mature enough and has a design system, then algorithms could make it more powerful.

INTERPOLATIONFlorian Schulz shows how you can use the idea of interpolation

to create many states of components.

MY INTEREST:AN AUTOMATED MAGAZINE LAYOUT (2012)Existing content had a poor semantic structure, and updating it by hand was too expensive. A special script would parse an article. Then, depending on its content (the number and size of paragraphs, the number of photos and their formats, the presence of inserts with quotes and tables, etc.), the script would choose the most suitable pattern to present this part of the article. The script also tried to mix patterns, so that the final design had variety.

DUPLOFlipboard launched a very similar model a few years ago.

VOX MEDIA: CHORUS CMSThe algorithm creates harmonious homepage layouts using a pattern library, then selects the “best” layout.

MORE FLEXIBLE AND PERFORMANTIt’s more efficient than picking the best links by hand,

as proven by recommendation engines such as Relap.io.

PREPARING ASSETS AND CONTENT

ROUTINE :(Creating cookie-cutter graphic assets in many variations is one of the most boring parts of a designer’s work.

It takes so much time and is demotivating, when designers could be spending this time on more valuable product work.

AUTOMATIC COLOR MATCHYandex.Launcher uses an algorithm

to automatically set up colors for app cards, based on app icons.

…ACCORDING TO BACKGROUNDAnother example is

changing text color according to the background color.

EMPLASIZING EMOTIONHighlighting eyes in a photo for news articles

in an experiment from Berg studio for The Guardian.

PARAMETRIC TYPOGRAPHYInterpolation of typefaces from several key points.

YANDEX.MARKET COLLECTIONS (RU)A marketer picks a title and an image, then the generator proposes an endless number of variations (they all conform to design guidelines).

РОБО-РЕДАКТОР ENGADGETThey nurtured a robot apprentice

to write simple news articles about new gadgets. Whew!

ENGADGET ROBO EDITORThey nurtured a robot apprentice

to write simple news articles about new gadgets. Whew!

STYLE TRANSFER: PRISMATruly dark magic happens in neural networks.

Prisma app stylizes photos to look like works of famous artists.

STYLE TRANSFER: ARTISTOOur product can process video

in a similar way (even streaming video).

WILL NEURAL NETWORKS MAKE ILLUSTRATORS OBSOLETE?I doubt it will for those artists with a solid and unique style.

But it will lower the barrier to entry when you need decent illustrations for an article or website but don’t need a unique approach. No more boring stock photos!

WOLFF OLINS: OIA live identity which reacts to sound. You just can’t create crazy

stuff like this without some creative collaboration with algorithms.

PERSONALIZING UXFor a narrow audience segment

or even specific users

A FAMILIAR EXAMPLEWe see it every day in Facebook newsfeeds, Google search results, Netflix and Spotify recommendations, and many other products.

Besides the fact that it relieves the burden of filtering information from users, the users’ connection to the brand becomes more emotional when the product seems to care so much about them.

SPOTIFY DISCOVER WEEKLYGiles Colborne: the only element of classic UX design here is the track list, whereas the distinctive work is done by a recommendation system.

DECISION MAKING SUPPORTAirbnb learned how to answer the question, “What will the booked

price of a listing be on any given day in the future?” so that its hosts could set competitive prices.

GOOGLE NOW, SIRI, ETC.For example, they automatically propose

a way home from work using location history data.

PERSONALIZED TEXTPersado optimizes ads for a particular user.

They also experiment with UI.

MUTATIVE DESIGNLiam Spradlin describes a concept of mutative design: adaptive interfaces that consider many variables to fit particular users.

AN EXOSKELETON FOR DESIGNERS

TOOLSIf we look back to the middle of the last century, computers were envisioned as a way to extend human capabilities.

Roelof Pieters and Samim Winiger have analyzed computing history and the idea of augmentation of human ability in detail. They see three levels of maturity for design tools.

1.INCREASE POSSIBILITIESFirst-generation systems mimic analogue tools with digital means.

2.GET RID OF ROUTINEThe second generation is assisted creation systems, where humans and machines negotiate the creative process through tight action-feedback loops.

3.CO-AUTHORSHIPThe third generation is assisted creation systems 3.0, which negotiate the creative process in fine-grained conversations, augment creative capabilities and accelerate the acquisition of skills from novice to expert.

A PROPER WAY TO COLLABORATEAlgorithm-driven design should be something like an exoskeleton for product designers – increasing the number and depth of decisions we can get through.

How might designers and computers collaborate?

1

PROBLEM SPACE SOLUTION SPACE

EVALUATIONCROSS-PRODUCT INTEGRATION

PRODUCTION

2 3

4 5

ANALYSIS TOOLS

RESEARCHAnalysis of implicitly expressed information about users that can be studied with qualitative research is hard to automate.

However, exploring the usage patterns of users of existing products is a suitable task.

MACHINE LEARNINGJon Bruner gives a good example: An algorithm starts with a description of the desired outcome – an airline’s timetable that is optimized for fuel savings and passenger convenience. It adds in the various constraints: the number of planes the airline owns, the airports it operates in, and the number of seats on each plane.

It loads details on thousands of flights from an existing timetable; the timetable gradually improves over many iterations.

A CURATORIn this scenario, humans curate an algorithm and can add or remove limitations and variables.

The results can be tested and refined with experiments on real users. With a constant feedback loop, the algorithm improves the UX, too.

MACHINE LEARNING FOR DESIGNERSThe complexity of this work suggests that analysts will be doing it, but designers should be aware of the basic principles of machine learning.

SYNTHESIS TOOLS

AUTODESK DREAMCATCHERIt’s based on the idea of generative design, which has been used in

performance, industrial design, fashion and architecture for many years now. It made a lot of noise and prompted several publications from UX gurus.

PARAMETRIC DESIGNMany of you know Zaha Hadid Architects;

its office calls this approach “parametric design.”

LOGOJOYIt replaces freelancers for a simple logo design. Logojoy generates

endless ideas and shows an example of a corporate style based on it.

A GENERATIVE APPROACHIt’s not yet established in digital product design, because it doesn’t help to solve utilitarian tasks.

Of course, the work of architects and industrial designers has enough limitations and specificities of its own, but user interfaces aren’t static – their usage patterns, content and features change over time, often many times.

The working process of digital product designers could potentially look like this:

1. An algorithm generates many variations of a design using predefined rules and patterns.

2. The results are filtered based on design quality and task requirements.

3. Designers and managers choose the most interesting and adequate variations, polishing them if needed.

4. A design system runs A/B tests for one or several variations, and then humans choose the most effective of them.

FILTERING CONCEPTSIt’s yet unknown how can we filter a huge number of concepts in digital product design, in which usage scenarios are so varied. If algorithms could also help to filter generated objects, our job would be even more productive and creative.

CREATIVE AIRoelof and Samim launched a website on this topic,

it lists many interesting concepts.

RENEThe experimental tool Rene by Jon Gold, who worked at The Grid, is an

example of this approach in action. Gold taught a computer to make meaningful typographic decisions.

BRUTE-FORCE DESIGNWhile Jon jokingly calls this approach “brute-force design” and “multiplicative design,” he emphasizes the importance of a professional being in control.

Notably, he left The Grid team last year.

DO TOOLS ALREADY EXIST?

ADOBE PHOTOSHOPThe company constantly adds “smart” features.

Photoshop learned to complete a missing part of a photo.

DESIGNSCAPEIt automatically refines a design layout for you.

It can also propose an entirely new composition.

ADOBE SENSEIA platform that uses machine learning; it will be the foundation

for future “smart” features in Adobe’s products.

JOHN MCCARTHY

He coined the term “artificial intelligence” and famously said,

“As soon as it works,no one calls it AI anymore.”

Look at experimental ideas and tools –they could become

a part of the digital product designer’s day-to-day toolkit.

STYLITCreates a 3D model out of sketch.

ANIMATION AUTOCOMPLETEMicrosoft learned to autocomplete

illustrations and animations.

PATCHY GLIMPSES OF THE FUTURERight now, it’s more about individual companies building custom solutions for their own tasks.

One of the best approaches is to integrate these algorithms into a company’s design system.

OLIVER ROEDER

The algorithmic software is written by humans, after all, using theories thought up by humans, using a computer built by humans, using specifications written by humans, using materials gathered by

humans, in a company staffed by humans, using tools built by humans, and so on. Computer art is human art

— a subset, rather than a distinction.

CONCLUSIONS

This is a story of a beautiful future, but we should remember the limits

of algorithms — they’re built on rules defined by humans, even if the rules are being

supercharged now with machine learning.

The power of the designer is that they can make and break rules;

so, in a year from now, we might define “beautiful” as something totally different.

Remove the routine of preparing assets and content, which is more or less mechanical work.

Broaden creative exploration, where a computer makes combinations of variables, while the designer filters results to find the best variations.

Optimize a user interface for narrow audience segments or even specific users.

Quickly adapt a design to various platforms and devices, though in a primitive way.

Experiment with different parts of a user interface or particular patterns – ideally, automatically.

We can only talk about a company’s custom solutions in the context of the company’s own tasks. The work requires constant investment into development, support and enhancement.

As The Grid’s CMS shows, a tool alone can’t do miracles. Without a designer at the helm, its results will usually be mediocre. On the other hand, that’s true of most professional tools.

Breaking past existing styles and solutions becomes harder. Algorithm-driven design is based on existing patterns and rules.

Copying another designer’s work becomes easier if a generative design tool can dig through Dribbble.

LET A COMPUTER PLAY WITH THE FONTSDigital products are getting more and more complex: We need to support more platforms, tweak usage scenarios for more user segments, and hypothesize more. Rather than hire more and more designers, offload routine tasks to a computer.

P.S.The revolution is already happening, so why don’t we lead it?

THANKS!YURY VETROV

www.jvetrau.com twitter.com/jvetrau

BONUS: A COLLECTION WEBSITE

http://algorithms.design/