GPkit: A Human-Centered Approach to ConvexOptimization in Engineering Design

Edward BurnellMassachusetts Institute of

TechnologyCambridge, MA, USA

eburn@mit.edu

Nicole B. DamenUniversity of NebraskaOmaha, Nebraska, USAndamen@unomaha.edu

Warren HoburgNational Aeronautics and

Space AdministrationHouston, TX, USAwhoburg@mit.edu

ABSTRACTWe present GPkit, a Python toolkit for Geometric and Signo-mial Programming that prioritizes explainability and incremen-tal complexity. GPkit was designed through an ethnographicapproach in the firms, classrooms, and research labs where itbecame part of the fabric of daily engineering work. Organi-zations have approached GPkit both in ways which centralizedesign work and in ways which distribute it, usecases whichemerged from and inspired new toolkit features. This two-way flow between mathematical structure and practitionerknowledge resulted in several novel contributions to the for-mulation and interpretation of convex programs and to ourunderstanding of early-stage engineering design. For example,dual solutions (often considered incidental) can be more valu-able to a design process than the “optimal design” itself, andwe present novel algorithms and design methods based on thisinsight.

Author Keywordsconvex optimization; human-centered design; design models;geometric programming; modeling languages; toolkits

CCS Concepts•Human-centered computing → User studies; Usabilitytesting; Collaborative interaction; User interface toolkits;Open source software; Field studies; •Theory of computa-tion → Convex optimization; •Computing methodologies→ Optimization algorithms;

INTRODUCTIONCentral to the work of many engineers is a "design model" thatquantifies parameters of their designs and implements inter-actions amongst these parameters. Common types of designmodel include parameterized CAD assemblies which constructa shape from geometric constraints [35], spreadsheets whichcalculate performance [42, 51], and “design codes”, smallpieces of software which take in a desired performance andput out a design that achieves it [38].

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.CHI ’20, April 25–30, 2020, Honolulu, HI, USA.© 2020 Association for Computing Machinery.ACM ISBN 978-1-4503-6708-0/20/04 ...$15.00.http://dx.doi.org/10.1145/3313831.3376412

In engineering organizations design models often serve asloci for understanding what will be built, while encoding (andsometimes concealing) decisions on why [35, 57]. As such,design models are an important arena for intra-organizationaldesign politics: it is often within and around these models thatdesign participants’ perspectives clash and coalesce [6, 35].Any design model used by an organization has its outsiders andinsiders, spectators and maintainers, and formal and informalpower structures [26, 35].

Design models are subject to the agency and affordances ofthe “material” (e.g. Solidworks, Excel, FORTRAN) fromwhich they are made [6, 34, 39, 54]. The influence of a designmodel’s material is felt especially in a a design process’ earlieststages, where the work is predominantly conceptual and lacksphysical prototypes to reference [21, 57]. The effects of early-stage design model materials are typically examined throughexperimental outcomes [10, 66, 67], but some materials havenot been studied in this way.

“Mathematical programs” are a type of design code, commonin aerospace engineering, whose output achieves a desired per-formance while striving to minimize a chosen “cost” parameter[43]. Their research focuses on formulations, algorithms, andperformance on benchmark models; published observations ofeffects on design and organizational outcomes are rare. Thisis doubly true for convex mathematical programs. Recentwork (most notably by Boyd et al. and other contributors onopen-source modeling toolkits [13, 18, 62] and solvers [15,44]) has made convex programs more broadly accessible, butthere are still few engineering organizations that use convexprograms as a design model material.

This work sought to make convex programs more useful in en-gineering organizations and to study their effects as a materialfor early-stage design models. It did so by embracing a par-ticipatory human-centered framework focused on validatingworkers’ knowledge [26] and using connections between thatknowledge and the mathematics of optimization to developGPkit, a toolkit for convex Geometric Programs. GPkit wasused by many engineers and researchers during its develop-ment [1, 2, 11, 14, 16, 17, 27, 28, 32, 33, 36, 46, 48, 49, 53,60, 64, 68], which resulted in features that are:

• technically and conceptually simple but with important user-experience benefits (e.g. using physical units to type-checksymbolic equations),

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 1

• technically simple but conceptually complex (e.g. an object-oriented syntax for representing a set of “all potentiallydesirable airplane wings”),

• conceptually simple but technically complex (e.g. an algo-rithm for efficiently computing optimal performance trade-offs with bounded error), and

• some both conceptually and technically complex (e.g. analgorithm for efficiently approximating a set of “all poten-tially desirable airplanes” with bounded error).

Organizations approached these features in ways both central-ized (one firm-wide design model with a few developers) anddistributed (design models developed independently by dozensof engineers from shared resources). This work’s main contri-bution is our field observations of these outcomes alongsidethe toolkit features they emerged from and inspired.

BACKGROUNDThis section contextualizes GPkit by comparing it to alter-native software, design process concepts, and optimizationtechniques.

UsecasesSeveral in-depth comparisons between GPkit and alternativedesign model materials have been published by Ozturk, Bur-ton, and Hoburg for an aerospace audience [11, 48, 29]. Tosummarize, in an aerospace design setting, GPkit’s potentialalternatives are programs written

1. by individual engineers to explore a conceptual design space(often in spreadsheets or scripting languages such as MAT-LAB or Python),

2. by subsystem teams to validate their decisions before adesign review (often using a scripting language to wrapanalysis codes implemented in C or FORTRAN), and

3. by system engineering teams to guide tradeoffs betweensubsystems (often using a library such as openMDAO [24]to connect black-box subsystem models).

GPkit has focused particularly on usecase (2), seeing theagency of design models in the rhetorical arena of a designreview as important and underexplored.

Design ProcessFrameworks for early stages of product and system design,such as Pahl and Beitz’ [50] approach to engineering designand Ulrich, Eppinger and Yang’s process for product designand development [63], are often based on the notion of aniterative and progressive design specification.

Following them we consider the formation of a design modelas an iterative design process (Figure 1). First, the model is re-alized as a (Concept) in the (Modeler)’s mind, who formulatesit as a (GPkit) model. Next, it is translated into the low-levelrepresentation needed by a numerical (Solver), and the convexoptimization problem (or sequence of problems) is solved.GPkit then parses the returned solution and presents it back tothe modeler. If the solution has any apparent errors then themodeler will seek to fix these errors in either their concept or

Modeler Concept GPkit Solver

formulation generationrefinement

repairpresentation

parsing

Figure 1. Schematic representation of the design modeling process.Dashed lines represent actions taken after a solution has been generated.

its formulation (the “repair” arrow). However, even withoutapparent errors the modeler is not finished, and will seek torefine their concept after having reflected upon its results (the“refinement” arrow) [59]. Either way the cycles continues foras long as the design model is used.

This iterative perspective emphasizes the importance of ex-plainability [58] (presenting results with as much context andcausality as possible) and incremental complexity (encour-aging models to be made more complex one step at a time).Both principles are also important within an engineering or-ganization, where repairs and refinements emerge throughcollaborative explanation and reformulation.

OptimizationMathematical programs can be described as finding minima ofa “cost” function f (x) over a particular domain. For example,the minimum of f (x) = x2 differs if the domain is “all realvalues of x” (Figure 2a) or “all real values of x greater than0.5” (Figure 2b). This can be written generally as

minimizex

f(x)

subject to x ∈ S(1)

where S is that optimization’s “feasible set” [5]. S can be un-derstood as the space of all potentially desirable answers: it ispossible that any x in S might be an optimum, and the purposeof optimization is to return one which is. Each dimension ofx is a “free variable” in the sense that it is not known beforeoptimization, even though it may be heavily restricted by S.

This formulation affords both f (x) and S equal importance,but it can be useful to move all complexity into the latter. Forany optimization, a new free variable can be introduced as thecost (y in Figure 2c) and the original (potentially complex)cost function can be added to S as the inequality f (x) ≤ y,forming a new feasible set S∗ = S∩ ( f (x) ≤ y). where thenew “constraint” must also be met by all potentially desirableanswers. This transformation (called the “epigraph problemform” [8]) allows us to consider the set of potentially desirableanswers as the defining property of a mathematical program,fitting the engineering design intuition of “what a design spacelooks like depends on what you’re going for”.

Convex OptimizationAn mathematical program is “convex” when S∗ is convex, de-fined geometrically as “all points on any line between twopoints in S∗ are themselves in S∗”. If S∗ is described by con-straints, its convexity can also be derived from theirs: if eachconstraint represents a convex set, S∗ is also convex. Certainforms of constraint have been established as convex, and much

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 2

f(x) = x2

f(x*=0) = 0 S = { x ≥ 0.5 }

f(x,y) = y

A (left) a simple optimization: a cost function (dots), feasible set (solid) and optimal point (star)B (right) a constraint which restricts the feasible set

C an optimization equivalent to (B) but with a new constrained variable representing the cost function

S = { x ≥ 0.5, y ≥ x2 }

Figure 2. Visual representations of optimization.

research in convex optimization has been done on develop-ing new forms or applying existing ones to engineering andoperations problems [8].

Convexity can thus be considered a limitation on the types ofconstraints in a mathematical program. In return it providesbenefits and guarantees for optimization algorithms. For ex-ample, it is straightforward to show that convex programs haveonly global optima. This is important because most algorithmsfor non-convex programs cannot make global claims abouttheir solutions: instead of returning a point with the lowestpossible cost, they often return a local optimum (an answerwhose cost is not lowest in the entire set, just in a finite pieceof it). Being able to guarantee that each design returned isa best-possible realization of their convex design model isimmensely useful to those seeking to convince other designparticipants of their perspective.

Convexity also scales to large problems more readily thannon-convex optimization. It is possible to solve convex opti-mization problems with thousands of free variables and con-straints on a personal computer in seconds, while methods fornon-convex optimization may struggle to solve in hours formerely dozens of free variables [8].

Strong DualityAnother property associated with convexity is strong Lagrangeduality, by which a “primal program” can be transformed intoa “dual program” whose optima correspond exactly. Dualprograms optimize over free variables which each correspondto a primal constraint, and solving them gives the derivativeof the optimal cost with respect to those constraints. Thatis, if the cost (as in Figure 2c) is equal to the variable y andevery constraint in the primal is of the form gi(x)≤ ci (whereeach ci is a “fixed” variable not being optimized over), thedual solutions are equal to dy

dci. This interpretation of a dual

solution’s values as trade-offs between cis and the overall costhas led them to be called “shadow prices” [8].

Geometric ProgramsA Geometric Program is a type of convex program with strongLagrange duality whose objectives and constraints are con-structed of posynomials, which are expressions defined by

p(x) =M

∑i=1

ci

N

∏j=1

xai, jj (2)

where each x j is a free variable whose domain is real numbersgreater than 0, each ai, j is a fixed real number, and each c isa fixed real number greater than 0. Geometric Programs aremathematical programs of the form

minimizex

p0(x0,x1..,x j)

subject to p1(x0,x1..,x j)≤ 1,..,

pk(x0,x1..,x j)≤ 1

(3)

where p0, ..pk are posynomials [7]. These posynomial con-straints are not directly convex; rather, ln(p(ez)) is convexin z, and x = ez can be calculated after solving. GeometricPrograms also be expressed in terms of exponential cone con-straints [2].

Modeling LanguagesFor over 50 years Geometric Programs have been recognizedas particularly suited to engineering equations [65]. The rel-atively recent introduction of optimization toolkits that donot require a background in numerical methods [23, 40] hasmade Geometric Programs accessible to a broader community.These toolkits, often called “modeling languages”, abstractaway the solvers behind convex optimization with syntaxesin which constraints can be written more “naturally”. [18]Such a syntax reduces situations where the model received bythe solver is not as the modeler intended [22], lowering theexpertise barrier [23] for using convex optimization, allowingengineers to utilize the results of optimization researchers [62].Unlike the assignment operator of a conventional program-ming language (such as z = 0.5) which directly assigns theresults of a calculation to a variable, constraint specificationsin a modeling language are represented only symbolically untilthe model is passed to a solver, at which point all free variablesare simultaneously converged to an optimal point.

The Catastrophe of Delayed User FeedbackThat free variables’ values are not available in the code wherethey are declared or used but only after solving presents a userexperience catastrophe [56]. When users make typos whichrender a constraint syntactically valid but substantively incor-rect, they will not know of the error until the entire model issolved, and even then won’t know which constraint causedit. Attempts to find the offending constraint via “printf de-bugging” [52] (placing print statements to isolate the error)falter because users have been abstracted away from (and maynot have debugging experience with) the solver code in whichthose variables become numeric.

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 3

Field Site Users GPkit Design Models Use TypeClass project with (under)graduate students 16 Electric “air taxi” service MixedAerospace R&D firm of 100-500 employees 12 Costing and sizing for airplane projects DistributedUniversity researchers 10 Passenger jets, solar planes, other vehicles DistributedIndustry-government-university initiative 10 Hybrid propulsion topologies MixedTransportation startup of 150-250 employees 7 Pre-production system engineering CentralizedClass project with (under)graduate students 3 Gas-powered high-altitude surveillance drone Centralized

Table 1. Overview of field sites. The “Users” column is an approximation of active weekly users during the period of study.

Receiving a nonsensical “solution” after a simple typo andnot being able to find its cause is the primary way we’veseen optimization toolkits lose new users. Even for thosepatient enough to dig in and find the error it takes time torecover perceptions of the toolkit’s dependability. Like thestatic checks of a compiled programming language [4], anyerrors that can be raised during constraint specification, beforesolving, make repair iterations faster and more fluid.

Accessing Dual Solutions and Shadow PricesModeling languages for convex optimization typically makedual solutions accessible to users through either a vector ofvalues or directly via the relevant constraint [18, 23], butsurprisingly tend not to not suggest or encourage their useduring model development.

From the perspective of the design process above, the dualsolution can be incredibly useful. The relative importance ofeach constraint to a particular solution’s optimality is fullyexplained by the dual solution’s “shadow prices”. Constraintswith a shadow price of zero could be removed without chang-ing the solution at all, while those with surprisingly large orsmall shadow prices are likely erroneous, providing an oppor-tunity for triage to speed repair iterations. In a “refinement”iteration, shadow prices can be interpreted as “model risk”, orthe dependence of a design model’s conclusions on each ofits assumptions. Interpreted in this way, a list of constraintsranked by the magnitude of their shadow price helps prioritizepotential refinements.

In geometric programs, shadow prices can also be used todetermine derivative of the cost with respect to each fixedvariable (those which are not free to be changed during opti-mization) [29]. These can present even clearer feedback to adesign participant than shadow prices: if a potentially com-plex design parameter such as a propeller’s efficiency has beenmodeled as a constant, but the cost is extremely dependent onthis parameter, it may be worth developing a more complexmodel to refine the risk of that assumption being incorrect.

Disciplined Convex Programming (DCP)Disciplined Convex Programming [23] is both a method forconstructing certificates of a set’s convexity, and a syntax forformulating convex constraints. DCP works by creating atree of operators with known conditions for convexity andchecking if this tree can be formulated as convex. This allowsnon-convex constraints to raise errors during their creation,providing pre-solve feedback to users on the mathematicalreasons a constraint is invalid. As discussed above, this kindof immediate feedback is scarce in a modeling language, mak-

ing the “discipline” of Disciplined Convex Programming apowerful aid to a model’s development process.

METHODSOne of the best methods for gaining insight into user inter-actions is to observe users as they interact with systems ofinterest, but as noted by Olsen “simple usability testing is notadequate for evaluating complex systems” [45]. Exploring theconsequences of convex design models would benefit mostfrom expert users who are familiar enough with the toolkitto place a wide range of stringent demands on them [55].However due to their novelty the vast majority of workers arenovice users whose interactions will contain a mixture of gen-uine errors arising from intentional behavior, and false errorsarising during familiarization. These learning behaviors ofnovice users are insightful for system design and developmentbut muddy the waters when exploring consequences of convexdesign models for engineering organizations.

Additionally, while the agency [34] of a design model’s mate-rial is most apparent in people’s interactions with each other,i.e. organizational practices, [20], these are difficult to create ina more controlled environment. In a lab study participants arelikely to be strangers, lack experience in collaborative designmodeling, and have a limited period of interaction. Some par-ticipant groups in Breneman’s excellent study of collaborativesatellite design show the scattered focus and poor outcomesthat can occur in such contexts [3]. Similarly, lab experimentswith convex optimization have shown with individual noviceusers that the underlying mathematics of a design model canaffect design outcomes [10], but have struggled to show anychange in this effect with pairs of novice users [37].

These considerations steered work on GPkit to proceed by re-cruiting users, working with them to understand the impact ofGPkit on their design models, and then observing throughoutcontinued engagement how this new design model material af-fected organizations and designs. Observations that informedGPkit were carried out across various research groups, class-rooms, and industrial firms (see Table 1).

With users at each of these sites, the first author took field notesthrough in-person and digital observations [19], conductedinformal usability tests, and worked with them to form sharedunderstandings of their practice. This was done to gain insightinto what they wanted to express when they made designmodels, the ways in which they expressed those with eachmaterial, and why those were their methods and goals.

GPkit’s development process also provided insight throughmaterial dialogues between the first author and users. Toolkitfeatures often began as a way of asking a user “is this (syntax,

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 4

algorithm) what you meant when you said that about yourwork?”. In using that feature the user questioned often foundan expressivity different from what they or the first authorhad expected, and communicated this in how they used andreflected on it [59]. This created a space in which each couldshare knowledge and build concepts for design work and con-vex programs that were interdependent, requiring the previousconceptions of each to change simultaneously [26]. Featureswhich emerged from such spaces often felt spontaneous, eachparty feeling it wasn’t really their idea, and were often thefeatures most obviously responsible for organizational change.

RESULTS

Organizational OutcomesAll field sites (see Table 1) used GPkit to validate decisionspresented in design reviews. In one case this was its exclusiveuse: two researchers were funded to construct a GPkit modelby a firm hoping to convincingly validate its existing design.Field sites were split between using it in ways which central-ized design work, in ways which distributed it, or both. Thedifferences between sites reveal how GPkit may change engi-neers’ interactions with their design models and each other,particularly by increasing the amount of direct feedback onmodeling decisions.

Distributed UsecaseAt the aerospace research and development firm, GPkit wasprimarily contrasted with an in-house MATLAB toolkit. Mostdesign models made with either were developed by an in-dividual modifying copies of others’ code to represent theirparticular design prompt, testing their ideas for design oppor-tunities, and then validating their conclusions in a series ofteam meetings. The firm trained dozens of conceptual designengineers to develop GPkit models and interactive demon-strations in hopes of creating a horizontal culture of sharingmodels and using them across new projects. They also de-veloped a toolchain for interactive visualizations of GPkitmodels in presentations, with one designer describing his pri-mary usecase as “leading an audience through novel designspaces”. Through these presentations GPkit’s introduction hasincreased group discussion of detailed design model decisions.Overall GPkit complements the MATLAB toolkit, which hasa great deal of organizational trust but is less amenable thanGPkit to extremely novel design spaces and interactive useduring a presentation.

The increased discussion of detailed design model decisionswas echoed when GPkit was introduced to academic contexts,where design presentations are typically larger, longer, andless frequent than at the aerospace firm. Academic use ofGPkit has been also been distributed and propagated in similarways, though with a wider variety of toolkits in competitionor complement.

Centralized UsecaseGPkit has also lent itself to more centralized approaches, asshown by a transportation startup’s use. There a small “sys-tems engineering” team developed a GPkit design model, thenestablished biweekly meetings with each subsystem team tobuild consensus on a particular subsystem’s representation in

that model. Updates to the system model and its current re-sults results were then disseminated across teams on a monthlybasis to build organizational consensus. The firm developedan internal website to interactively run the system model andencourage all design participants to explore its design space.Development of this system design model helped bridged acommunication gap between higher-level managers and sub-system engineers that had been startling at our first site visit.At that time, subsystem engineers had said they felt unheardwhen they suggested practical but less glamorous or novel tech-nologies, while managers said that too many engineers didn’trealize the necessity of riskier technologies for their productcategory. System engineers, making models for the managersbut eating lunch with the subsystem engineers, felt caught inbetween. The structure of the GPkit design model’s develop-ment assured subsystem engineers that they could express themanufacturing and maintenance costs of risky technologies,while also reassuring managers that they could express themarket and sales costs of conventional ones.

The first use of GPkit in an aerospace design classroom alsocentralized the work of design modeling. Several studentstook on the role of system engineers and built consensus withtheir peers to deviate from the requested solar-power archi-tecture to a gas-powered one (see Observation D below formore details). Some of these students then reused their GPkitdesign code for manufacturing in the next semester’s “build”class, through which the gas-powered surveillance drone pro-posed the previous semester was built and successfully flown(video at youtu.be/HMu3x5WxpeM). Teaching staff expressedsurprise that the final build adhered so closely to the GPkitmodel’s weight estimates.

Mixed UsecasesOther field sites mixed the centralized and decentralized use-cases. In the design class after the gas-powered drone, GP-kit adoption started with distributed use but became fairlycentralized by the end of the semester. In the government-industry-academia initiative, some participants collaborated ona centralized design model while others developed individualmodels. During a mid-cycle progress review presentation, gov-ernment officials expressed a preference for distributed mod-els’ dual-solution-based scaling laws over centralized models’complex comparisons of propulsor topologies.

Toolkit Features and ObservationsThis section discusses exemplary GPkit features and observa-tions accompanying their development.

Observation A: Variable Declaration CommentsComments that users added to their code were used to gaininsight into their understanding of the design model. Onerepeat observation was of the comments added to variabledeclarations to provide additional information about thatvariable, for example “S = Variable() # [ft**2] wingsurface area”.

Feature A: Variable MetadataWhile some modeling frameworks do not capture a variable’sname as a code object [13], GPkit brought these variablemetadata directly into the code: “S = Variable(‘S’,

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 5

units=‘ft**2’, label=‘wing surface area’)”.Then, “paving the desire paths”[41], we used this informationwherever possible.

Adding variable names and labels allowed GPkit to provide adefault solution presentation that could be understood withoutreferencing the original code (note S, its units, and its label inFigure 3), helping users more quickly see errors and givingthem something which could be shown immediately to otherdesign participants.

Incorporating variables’ units and the corresponding unit al-gebra enabled errors to be raised whenever users attempted toadd variables or create a constraint whose units were incompat-ible. As previously working models stopped solving, we heardsome complaints. But unit checking provided immediate errorfeedback on constraints, and even the last holdouts came toactively appreciate it when it saved them from typos. Unitsalso reduced metrication errors, removing the risk of changinga variable’s unit. Units were still fairly new when the gas-powered drone mentioned above was proposed, and studentsspent an hour checking that the automatic unit conversionswere being done correctly as they prepared to present to theirclient; later users rarely thought about such concerns.

Observation B: Modeling GravityDuring development of a (different) solar plane, gravity wasmodeled as a fixed variable g (instead of a united constant),and at “+4” was the most sensitive fixed variable. As a bitof a joke (and to get it to stop showing up in fixed-variablesensitivity tables), researchers modeled the slight difference ingravity at high altitude; to their surprise this lead to changesin their design.

Feature B: Sensitivity TablesAs part of the default presentation of results GPkit summarizesthe dual solution (returned by the solver automatically), show-ing the “sensitivities” of fixed variables and constraints (note Aand the fuel-burn constraints in Figure 3). As with the shadowprices of strong Lagrange duality, sensitivities correspond tod log(y)d log(ci)

, where y is the cost and ci is a fixed variable whosevalue is known before solving. This means that sensitivitiescorrespond to a local power-law fit: in the situation above, thecost was locally proportional to g4, so decreasing gravity byε% decreased the cost by exactly (4× ε)% (for sufficientlysmall epsilon; sensitivities are only exactly true exactly at asolution). By showing these sensitivities to users, GPkit helpsthem repair erroneously large or small fixed-variable depen-dencies (which might not be clear in the primal solution) toprioritize refinements of their design model.

Observation C: Plotting AlongSolution tables are the atomic result of convex programs. Thefeatures mentioned above increase their usefulness for present-ing to other design participants, but numerical plots showingmultiple solutions at once are still more commonly used forthat purpose, in part because of their rhetorical weight [47, 31].The vast majority of user generated solutions are made specif-ically for plots comparing trade-offs between performanceparameters, and most such visualizations use a conservativenumber of manually specified points on a rectangular grid in

Cost----1.091 [lbf]

Free Variables--------------

| AircraftW : 144.1 [lbf] weight

| Aircraft.WingS : 44.14 [ft**2] surface areaW : 44.14 [lbf] weightc : 1.279 [ft] mean chord

| Mission.FlightSegment.AircraftPWburn : [ 0.274 0.272 ] [lbf] fuel burnWfuel : [ 1.09 0.272 ] [lbf] fuel weight

| Mission.FlightSegment.AircraftP.WingAeroD : [ 2.74 2.72 ] [lbf] drag force

Sensitivities-------------

| Aircraft.FuselageW : +0.97 weight

| Aircraft.WingA : -0.67 aspect ratio

rho : +0.43 areal density

Tightest Constraints--------------------

| Aircraft+1.4 : .W >= .Fuselage.W + .Wing.W

| Mission+1 : Wfuel[0] >= Wfuel[1] + Wburn[0]

+0.75 : Wfuel[1] >= Wfuel[2] + Wburn[1]+0.5 : Wfuel[2] >= Wfuel[3] + Wburn[2]

| Aircraft.Wing+0.43 : .W >= S*.rho

Figure 3. Example of a results table in GPkit.

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 6

A Initial solutions and sensitivities B Initial bounds

D Additional solutions decrease errorC Minimum-error estimate

Figure 4. Visual explanation of the Autosweep algorithm.

an attempt to avoid inaccuracy and perceptible discontinuitiesbetween solution points.

Feature C: AutosweepFor plots in which the independent axes are fixed variablesand the only dependent axis is cost we can solve for it withoptimal efficiency through the use of dual solutions. Becauseof convexity, variable sensitivities at corners of the grid de-fine hyperplanes which lower-bound the cost inside, and thehyperplanes constructible from those corners upper bound it.Any solved points are exact, so these upper and lower boundsintersect at those corners. Figure 4 illustrates this in two di-mensions with one independent fixed variable as the horizontalaxis and the dotted parabola representing the (unknown by thealgorithm) optimal cost. Averaging upper and lower boundsresults in an estimate whose inaccuracy is equal to half thevertical distance between upper and lower bounds. If this isn’taccurate enough, the estimate can be optimally improved bysolving at points of maximum vertical distance, iterating untilthe desired accuracy is achieved. Note that any straight orplanar portions of the cost function are completely describedby just their boundary points. This algorithm needs very fewsolutions to describe predictable portions of the design space.Between its improved solution placement and the ability forusers to directly specify their desired accuracy this algorithmis faster and more precise than naive gridding. Because of itsiterative nature it can also be used as an online algorithm forreal-time design space exploration.

Observation D: Shifting Costs and ConstraintsAs mentioned in Centralized Usecase above, GPkit was usedin an aerospace design classroom where the “client” requesteda solar-powered surveillance drone that could loiter for aslong as possible at high altitude above a target location. Stu-dents debated this specification, realizing the airplanes theirmodel returned were large enough to present transportation

and deployment difficulties. Could it make sense to considerarchitectures other than solar power? After some consterna-tion at going against the client’s instructions, they developed amodel for propulsion by a gasoline engine and changed theircost function, solving instead for the lightest gas-poweredplane that could loiter for at least 6 days.

To the surprise of teaching staff an optimized gas-poweredarchitecture was both capable of 6 days endurance and muchmore amenable to manufacturing. The underlying designmodel stayed mostly the same, changing only its cost functionand propulsion and fuel models, which helped the client un-derstand the tradeoffs and fully buy in to the new architecture.

Feature D: Modularity and ConvexityWhat came out of this observation was not just GPkit syntaxbut an appreciation for the synergy between convex modelingand engineering practice.

By asking users about how they used cost functions, welearned that their value was not in returning the “best result”,but rather in getting a better understanding of the set of po-tentially desirable designs by collapsing it along a particularaxis. We observed engineers optimizing for multiple perfor-mance parameters, each generating a “paragon” optimal bythat metric [49]. Comparison of these paragons gave a sense ofpossibilities for their design [48]. Sometimes even a singularand quantifiable goal (such as the expected profit of a product),was thanklessly complex to represent, leading engineers to apreference for discussing paragons. While many mathematicalprograms’ structure or method of solving is built around thecost function, in convex design models we have often foundthat all performance parameters are viable cost functions. As aresult, after the events above occurred we standardized settingcost functions as the last pre-solve step.

We also found that because constraints can represent equationsdescribing a design’s function as well as those describing itsform, constraints and entire models (e.g. an aircraft’s wingspar) are often reusable across projects.

Observation E: Where Constraints “Belong”We repeatedly heard GPkit users refer to where constraints“belonged”. This was done explicitly in “#TODO” commentsapologizing for a constraint’s misplacement, in “#NOTE”comments justifying its presence, and in design discussions(“does the fuel burn constraint belong here [with the fuelmodel] or there [with the airplane drag constraints]?”). Itwas also done implicitly, with “belonging” represented bythe clustering of constraints into named categories. Con-straints were generally described as “belonging” either tocategories arising from a hierarchy of the design’s physi-cal layout (wing, fuselage) or from the disciplinary hier-archy (aerodynamics_eqs, structural_eqs) common innon-convex aerospace design codes.

Feature E.1: Named Object-Oriented Constraint SetsWe found that such categories could be well represented asa tree of custom objects; for example, an Airplane instancecontaining Wing and Fuselage instances. Although most usershad not written object-oriented code prior to GPkit, adop-tion of this feature was surprisingly rapid; the object-oriented

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 7

Figure 5. Sankey diagram displaying a model’s constraint tree (left is towards the root) and the sensitivity of each named constraint set object (thickeris more sensitive). Discontinuous increases in thickness at a particular object indicate the sensitivity of the unnamed constraints inside it.

framework seemed to fit their experience and intuition, andintroduction of a hierarchy lowered the number of variablesavailable in each constraint’s and made the interlinking ofsubsystems more repairable and explainable. With this newlayer of abstraction, engineers felt more comfortable repairing,refining and explaining design models spanning multiple files,and began creating “stub models” with fixed variables and noconstraints to be refined later if the model was sensitive to oneof those variables [48].

Feature E.2: Constraint Set Boundedness VerificationThese objects (Wing, Fuselage) provided another way to checkconstraint’s correctness before solving, by asking users tospecify which variables they expected to be upper or lowerunbounded by instances of that object class. For example, aWingStructure instance might be expected to not put a lowerbound on b (representing the wing’s length), because a van-ishingly short wing structure would be perfect in its weight-lessness. However WingStructure ought to upper bound b,because a too-long wing would bend and snap. If a WingStruc-ture instance imposed a lower bound or lacked an upper boundon b, GPkit would warn the user and suggest they either repairtheir model or refine their expectations for WingStructure. Be-cause variables in a model are typically both upper and lowerbounded (if they’re not the model is generally infeasible), weexplained these bounds checks to users by relating them to ajigsaw puzzle where objects’ upper-bounded “tabs” and lower-bounded “notches” fit together to form a design model withoutany unmatched tabs or notches.

Observation F: “An Engine With a Stick Attached”A researcher in the industry-government-university initiativecame to us with a frustrated certainty that the jet engine com-ponent of their design model was “too brittle”: “The plane isbeing entirely designed around the engine! [...] changing theplane model doesn’t change the optimal engine at all; changingthe optimal engine model completely changes the plane. It’slike an engine with a stick attached.” Part of their frustration

was that, while they had become certain of this by solvingfor various points, they did not know how to convince theircolleague who was developing the engine model.

Feature F: Sankey Diagrams of the Dual SolutionBy summing all constraint sensitivities in a named constraintset we calculate the effect changing those variables simultane-ously would have on cost. The diagram in Figure 5 shows boththe hierarchy of a model (see Feature E.1) and the relative sen-sitivities of each named constraint set. For the engineer who’dcome to us, seeing this exact diagram felt like a vindication;here was clear indication of their design model’s sensitivity tothe engine to the exclusion of the rest of the airplane. Afterfurther finding that the basic lift constraint in the Airplaneobject was responsible for most of the rest of its sensitivity,their initial heated description of their design model as “anengine with a [lift-providing] stick attached” seemed quite apt.

We also developed Sankey diagrams for variables, showingthe portion of a variable’s total sensitivity coming from eachconstraint. Free variables always have zero sensitivity (ifthey’d be better at a higher or lower value, that would bydefinition be their optimal value), but seeing how positiveand negative sensitivities across constraints sum to that zeroexplains which constraints are responsible for that variable’svalue. This is particularly helpful if a free variable’s optimalvalue is either unexpected (repair cycle) or part of a designdebate (refinement cycle).

Observation G: Shared ToolingEarly in the gas-powered-drone class project, the model founda curious way of flying; accelerating as it increased in alti-tude, but unexpectedly keeping the same speed even as it spentfuel and became lighter. The culprit turned out to be onevariable, the Reynolds number (a non-dimensional characteri-zation of aerodynamics), which had been incorrectly specifiedas a scalar rather than a vector variable. The model was thus,correctly, trying to find an optimal single Reynolds number it

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 8

could maintain across all modes of flight, making the plane’sairspeed a function only of its altitude. Years later, the idea ofthis plane striving to achieve a perfectly consistent Reynoldsnumber is still considered extremely comedic by some of theoriginal teaching staff.

Feature G: Vectorization EnvironmentsTo make such mistakes less likely, we built upon Tony Tao’swork [61] exploring simultaneous optimization of a fleet of air-planes. Instead of designing one plane for multiple flight states(e.g. climbing, cruising, landing), he modeled one set of manu-facturing tools shared by multiple airplanes, each with multipleflightstates, thus adding another dimension to a typical “multi-point” design problem. In GPkit we introduced vectorizationenvironments to add a dimension to variables in any subtree ofthe constraint hierarchy. This made adding such abstractionsstraightforward; what had been one airplane and flight statecould, with a few lines of code, become multiple flight stateswhich shared an airplane, and with another few lines multipleplanes with shared tooling. As with object-oriented constraintsets, the introduction of this method encouraged users to makesignificantly larger models. Vectorization made adding newmodes of flight, new missions, and new structural loadingcases much simpler, though it required care in model construc-tion to ensure variables were partitioned to permit the desiredvectorization. On one project, the Wing model had to be splitinto WingStructure (which stayed in Airplane) and WingAero(which went into a new AirplanePerformance object) so that asingle Airplane could have a vectorized AirplanePerformancefor each of its flight states.

Observation H: Flight EnvelopesWhile the concept of a feasible set originates in optimization,we have found it quite intuitive to engineers. A graduatestudent using GPkit once explained to us that they did notwish to specify an objective function because they did notwant to solve for a particular point. Rather, they wanted tosolve for all designs which fit their requirements. Instead ofbuilding a design model to ask “at what altitude and speeddoes a single-bladed helicopter fly best?”, they wanted to ask“at what altitudes and speeds can a single-bladed helicopter beflown at all?”.

Feature H: Design Space ApproximationBecause of this request, GPkit has an efficient design spaceapproximation algorithm, similar to that for autosweep. Byselecting a simplex in the given dimensionality (a trianglein 2D; see Figure 6), we can solve from each corner for thenearest edge of the feasible set. Because the feasible set mustbe convex, the simplex formed by these feasible points isan inner approximation of the feasible set, while the areacontained by the perpendicular tangents of those points is anouter one. As with autosweep, averaging the two forms anestimate with minimum inaccuracy, reducable as desired byadditional solves at the most inaccurate points. If solving apoint outwards returns a certificate of dual infeasibility thedesign space is unbounded in that direction, and because ofconvexity that direction will be unbounded for all points.

A Initial nearest-feasible solutions B Initial bounds

D Additional solutions decrease error

C Minimum-error estimate

Figure 6. Visual representation of design space approximation.

DISCUSSIONIn this section we will explore how this work fits within theHCI literature, but first we should discuss its limitations [12].Convex programs are fundamentally limited as a materialfor design models, and this is a limited study of a particulartoolkit in a few field sites. While our methods produced awealth of information and insights that could otherwise nothave been obtained they are sensitive to selection biases, andour ethnographic methods were not rigorously planned but of-ten emerged from necessity or opportunity. To alleviate issuesof physical distance the we made an effort to provide up-to-date documentation and discussions online, but not everyonewas equally able or willing to communicate their experiencesback to the first author, which marginalized them in the de-velopment process. Further many of the above features (unitanalysis, variable metadata, sensitivity analysis) are present inother programming languages or engineering design toolkits,though as far as we know they are novel to convex optimizationtoolkits.

GPkit as a ToolkitToolkits have been defined by Greenberg as providing a “vo-cabulary and set of building blocks” which “give people a‘language’ to think about these new interfaces, which in turnallows them to concentrate on creative designs.” [25]. Ledoet al. [12] extended this to define toolkits as “generative plat-forms designed to create new interactive artifacts, provide easyaccess to complex algorithms, enable fast prototyping of soft-ware and hardware interfaces, or enable creative explorationof design spaces”. We have done several of these with GPkit,giving engineers the ability to rapidly build and improve inter-active explorations of convex design spaces. GPkit is thus an“artifact contribution”, where “new knowledge is embeddedin and manifested by artifacts and the supporting materialsthat describe them”, as demonstrated via the case studies andobservations above [12].

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 9

GPkit as a User Interface SystemThis work also follows in the footsteps of other user interfacesystems [45]. GPkit falls under each of the three limitationsof usability experiments, being a tool for expert users doingnon-standard tasks over a time period of months or years. Aswith previous papers using this framework [30] we considerGPkit in Olsen’s framework for User Interface Systems [45]:it introduces a new task to a preexisting group of users (en-gineering designers) in a preexisting situation (early stagedesign), but (unlike in Olsen), GPkit also seeks to change thatsituation, changing the ways in which computers are used inthese organizations as part of the its development.

Value Added by UI Systems ArchitectureGPkit sets up “paths of least resistance” [45, 9] for explainingdesign codes; this results in models that can be more amenableto iterative improvement than alternatives, allowing users toderive related solutions with reduced development viscosityand build on common infrastructure. Convex programs havebeen inaccessible to most engineering organizations, but bymaking them accessible, GPkit helps increase the scale of theirdesign models from dozens of free variables to thousands. GP-kit thus lowers the threshold and raises the ceiling for designcodes. The “Moving Targets” present in any use-motivatedwork [9] have been a fuel for the development of GPkit; wehave encouraged our users to change their goals and expecta-tions as they use it so they might adopt previously untenableorganizational practices.

CONCLUSIONMuch can be gained by integrating user experience and themathematics of convexity. Our original goal was to understandand develop convex optimization as a material for engineeringdesign models, so we adopted an explicitly human-centeredframework and attempted to align practitioners’ knowledge oftheir work with the way they used GPkit. We found convexdesign models to be powerful tools that can contain the hopesand concerns of many design participants. In this paper wehave presented several of the novel algorithms (such as de-sign space approximation and autosweep) and visualizations(such as Sankey sensitivity diagrams) that resulted from thisapproach. We have also developed insights into the user ex-perience of design models and the methods by which usersiteratively repair and refine them. The utility and ease of useof GPkit has led it to be woven into the fabric of several engi-neering organizations, changing their practices of engineeringdesign. By drawing from a breadth of research areas andpractitioner experiences, it has also opened up new avenuesfor HCI research in design models, convex optimization, andorganizational outcomes.

ACKNOWLEDGEMENTSThis study was supported in part by an award by the MIT-Sensetime Fund and the National Science Foundation undergrant #1854833. Any opinions, findings, and conclusions orrecommendations expressed in this material are those of theauthors and do not necessarily reflect the views of the funders.

EB: Without the creative ferment of the Convex Engineer-ing Group this work could never have happened; cheers toArthur Brown, Michael Burton, Lochie Ferrier, Cody Karcher,Philippe Kirschen, Berk Ozturk, and Ali Saab. Shouts out toDavid Anderson, who started with me on this research direc-tion while we were undergraduates, and to Maria Yang, whogave me a place from which to start again. To Ambika Kamath,whose reminder that “There’s room for solidity and for ambi-tion” kept me going. Thanks also to those who handled earlydrafts, some already listed but including also Dr. Hong-enChen and Katy Gero, whose questions and critiques carried usover the finish line. A particularly special thanks goes to themembers of my family (Nevin, Michael, and Nevin) who withpatient exasperation tolerated my lack of foresight as I finishedthis paper’s first submission amidst Yellowstone campsites.

Finally, we would like to acknowledge our kindly critical re-viewers, whose efforts in understanding our oddball first sub-mission and suggesting improvements made a stark differencein its quality.

REFERENCES[1] Sara Achour and Martin Rinard. 2018. Time Dilation

and Contraction for Programmable Analog Devices withJaunt. In ACM SIGPLAN Notices, Vol. 53. ACM,229–242.

[2] Akshay Agrawal, Steven Diamond, and Stephen Boyd.2019. Disciplined geometric programming.Optimization Letters (2019), 1–16.

[3] Jesse Austin-Breneman, Tomonori Honda, and Maria CYang. 2012. A study of student design team behaviors incomplex system design. Journal of Mechanical Design134, 12 (2012), 124504.

[4] Nathaniel Ayewah, William Pugh, David Hovemeyer,J David Morgenthaler, and John Penix. 2008. Usingstatic analysis to find bugs. IEEE software 25, 5 (2008),22–29.

[5] Brian Beavis and Ian Dobbs. 1990. Optimisation andstability theory for economic analysis. Cambridgeuniversity press.

[6] Daniel Beunza and David Stark. 2004. Tools of the trade:the socio-technology of arbitrage in a Wall Street tradingroom. Industrial and corporate change 13, 2 (2004),369–400.

[7] Stephen Boyd, Seung-Jean Kim, Lieven Vandenberghe,and Arash Hassibi. 2007. A tutorial on geometricprogramming. Optimization and Engineering (2007).

[8] Stephen Boyd and Lieven Vandenberghe. 2004. ConvexOptimization. Cambridge University Press, New York,NY, USA.

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 10

[9] Randy Pausch Brad Myers, Scott E Hudson. 2000. Past,present, and future of user interface software tools. 7(2000), 3–28. Issue 1. DOI:http://dx.doi.org/10.1145/344949.344959

[10] Edward Burnell, Michael Stern, Ana Flooks, andMaria C Yang. 2017. Integrating Design andOptimization Tools: A Designer Centered Study. InASME 2017 International Design Engineering TechnicalConferences and Computers and Information inEngineering Conference. American Society ofMechanical Engineers Digital Collection.

[11] Michael J Burton and Warren W Hoburg. 2017.Solar-Electric and Gas Powered, Long-Endurance UAVSizing via Geometric Programming. In 18thAIAA/ISSMO Multidisciplinary Analysis andOptimization Conference. 4147.

[12] Jo Vermeulen Nicolai Marquardt Lora OehlbergDavid Ledo, Steven Houben and Saul Greenberg. 2018.Evaluation strategies for HCI toolkit research. In ACMSIGCHI Conference on Human Factors in ComputingSystems. ACM, 18.

[13] Steven Diamond and Stephen Boyd. 2016. CVXPY: APython-embedded modeling language for convexoptimization. Journal of Machine Learning Research 17,83 (2016), 1–5.

[14] Luis Diez, George-Pantelimon Popescu, and RamónAgüero. 2016. A geometric programming solution forthe mutual-interference model in hetnets. IEEECommunications Letters 20, 9 (2016), 1876–1879.

[15] Alexander Domahidi, Eric Chu, and Stephen Boyd.2013. ECOS: An SOCP solver for embedded systems. In2013 European Control Conference (ECC). IEEE,3071–3076.

[16] Aidan Dowdle and Marija Ilic. 2017. Interconnectedstate-space modeling for turboelectric aircraft. In 2017North American Power Symposium (NAPS). IEEE, 1–6.

[17] Aidan P Dowdle, David K Hall, and Jeffrey H Lang.2018. Electric Propulsion Architecture Assessment viaSignomial Programming. In 2018 AIAA/IEEE ElectricAircraft Technologies Symposium (EATS). IEEE, 1–21.

[18] Iain Dunning, Joey Huchette, and Miles Lubin. 2015.JuMP: A modeling language for mathematicaloptimization. arXiv preprint arXiv:1508.01982 (2015).

[19] Robert M Emerson, Rachel I Fretz, and Linda L Shaw.2011. Writing ethnographic fieldnotes. University ofChicago Press.

[20] Martha S Feldman and Wanda J Orlikowski. 2011.Theorizing practice and practicing theory. Organizationscience 22, 5 (2011), 1240–1253.

[21] Sebastian K Fixson and Tucker J Marion. 2012.Back-loading: A potential side effect of employingdigital design tools in new product development. Journalof Product Innovation Management 29 (2012), 140–156.

[22] Robert Fourer, David M Gay, and Brian W Kernighan.1987. AMPL: A mathematical programming language.Citeseer.

[23] Michael Grant, Stephen Boyd, and Yinyu Ye. 2006.Disciplined convex programming. In Globaloptimization. Springer, 155–210.

[24] Justin Gray, Kenneth Moore, and Bret Naylor. 2010.OpenMDAO: An open source framework formultidisciplinary analysis and optimization. In 13thAIAA/ISSMO Multidisciplinary Analysis OptimizationConference. 9101.

[25] S. Greenberg. 2007. Toolkits and interface creativity.Multimedia Tools and Applications) 32 (2007), 139–159. Issue 2. DOI:http://dx.doi.org/10.1007/s11042-006-0062-y

[26] Davydd J Greenwood and Morten Levin. 2006.Introduction to action research: Social research forsocial change. SAGE publications.

[27] David K Hall, Aidan Dowdle, Jonas Gonzalez, LaurenTrollinger, and William Thalheimer. 2018. Assessmentof a Boundary Layer Ingesting Turboelectric AircraftConfiguration using Signomial Programming. In 2018Aviation Technology, Integration, and OperationsConference. 3973.

[28] Monowar Hasan, Sibin Mohan, Rodolfo Pellizzoni, andRakesh B Bobba. 2018. A design-space exploration forallocating security tasks in multicore real-time systems.In 2018 Design, Automation & Test in EuropeConference & Exhibition (DATE). IEEE, 225–230.

[29] Warren Woodrow Hoburg. 2013. Aircraft designoptimization as a geometric program. Ph.D. Dissertation.University of California, Berkeley.

[30] & Marquardt N. Houben, S. 2015. Watchconnect: Atoolkit for prototyping smartwatch-centric cross-deviceapplications. In ACM Conference on Human Factors inComputing Systems. ACM, 1247–1256.

[31] Sarah Kaplan. 2011. Strategy and PowerPoint: Aninquiry into the epistemic culture and machinery ofstrategy making. Organization Science 22, 2 (2011),320–346.

[32] Pasha Khosravi, Yitao Liang, YooJung Choi, and GuyVan den Broeck. 2019. What to Expect of Classifiers?Reasoning about Logistic Regression with MissingFeatures. arXiv preprint arXiv:1903.01620 (2019).

[33] Philippe G Kirschen, Edward Burnell, and WarrenHoburg. 2016. Signomial programming models foraircraft design. In 54th AIAA Aerospace SciencesMeeting. 2003.

[34] Bruno Latour. 1992. Where are the missing masses, thesociology of mundane artefacts. Bijerker, WE & Law,J.(1992). Eds., Shaping Technology/Building Society:Studies in Sociotechnical Change. MIT Press,Cambridge (1992), 255–258.

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 11

[35] Paul M Leonardi. 2011. When flexible routines meetflexible technologies: Affordance, constraint, and theimbrication of human and material agencies. MISquarterly 35, 1 (2011), 147–167.

[36] Ramon Leyva. 2016. Optimal sizing of Cuk convertersvia Geometric Programming. In IECON 2016-42ndAnnual Conference of the IEEE Industrial ElectronicsSociety. IEEE, 2480–2485.

[37] Eunice Lin. 2017. Collaboration in design optimization.Master’s thesis. Massachusetts Institute of Technology.

[38] Gloria Mark. 2002. Extreme collaboration. Commun.ACM 45, 6 (2002), 89–93.

[39] Nolwenn Maudet, Germán Leiva, MichelBeaudouin-Lafon, and Wendy Mackay. 2017. DesignBreakdowns: Designer-Developer Gaps in Representingand Interpreting Interactive Systems. In Proceedings ofthe 2017 ACM Conference on Computer SupportedCooperative Work and Social Computing. ACM,630–641.

[40] M Mutacipc, K Koh, S Kim, and S Boyd. 2008. Amatlab toolbox for geometric programming. (2008).

[41] Carl Myhill. 2004. Commercial success by looking fordesire lines. In Asia-Pacific Conference on ComputerHuman Interaction. Springer, 293–304.

[42] Bonnie A Nardi and James R Miller. 1991. Twinklinglights and nested loops: distributed problem solving andspreadsheet development. International Journal ofMan-Machine Studies 34, 2 (1991), 161–184.

[43] Jorge Nocedal and Stephen J Wright. 2006. Numericaloptimization. Springer Science+ Business Media.

[44] Brendan O’Donoghue, Eric Chu, Neal Parikh, andStephen Boyd. 2016. Conic optimization via operatorsplitting and homogeneous self-dual embedding.Journal of Optimization Theory and Applications 169, 3(2016), 1042–1068.

[45] Dan R. Olsen. 2007. Evaluating User Interface SystemsResearch. Proceedings of the 20th Annual ACMSymposium on User Interface Software and Technology(2007), 251–258. DOI:http://dx.doi.org/10.1145/1294211.1294256

[46] Max MJ Opgenoord, Brian S Cohen, and Warren WHoburg. 2017. Comparison of Algorithms for IncludingEquality Constraints in Signomial Programming.Aerospace Computational Design Lab., MassachusettsInst. of Technology, TR-17-1, Cambridge, MA (2017).

[47] Thomas Østerlie, Petter G Almklov, and Vidar Hepsø.2012. Dual materiality and knowing in petroleumproduction. Information and organization 22, 2 (2012),85–105.

[48] Berk Öztürk. 2018. Conceptual engineering design andoptimization methodologies using geometricprogramming. Master’s thesis. Massachusetts Instituteof Technology.

[49] Berk Ozturk and Ali Saab. 2019. Optimal AircraftDesign Deicions under Uncertainty via RobustSignomial Programming. In AIAA Aviation 2019 Forum.3351.

[50] Gerhard Pahl and Wolfgang Beitz. 2013. Engineeringdesign: a systematic approach. Springer Science &Business Media.

[51] Kevin LG Parkin, Joel C Sercel, Michael J Liu, andDaniel P Thunnissen. 2003. Icemaker™: an excel-basedenvironment for collaborative design. (2003).

[52] Michael Perscheid, Benjamin Siegmund, MarcelTaeumel, and Robert Hirschfeld. 2017. Studying theadvancement in debugging practice of professionalsoftware developers. Software Quality Journal 25, 1(2017), 83–110.

[53] Ryan Pilgrim. 2017. Source Coding Optimization forDistributed Average Consensus. arXiv preprintarXiv:1710.01816 (2017).

[54] Alex Preda. 2006. Socio-technical agency in financialmarkets: The case of the stock ticker. Social Studies ofScience 36, 5 (2006), 753–782.

[55] Prashanth Rajivan, Pablo Moriano, Timothy Kelley, andL Jean Camp. 2017. Factors in an end user securityexpertise instrument. Information & Computer Security25, 2 (2017), 190–205.

[56] A Rizzo, O Parlangeli, E Marchigiani, and S Bagnara.1996. The management of human errors in user-centereddesign. ACM SIGCHI Bulletin 28, 3 (1996), 114–118.

[57] BF Robertson and DF Radcliffe. 2009. Impact of CADtools on creative problem solving in engineering design.Computer-Aided Design 41, 3 (2009), 136–146.

[58] Wojciech Samek, Thomas Wiegand, and Klaus-RobertMüller. 2017. Explainable artificial intelligence:Understanding, visualizing and interpreting deeplearning models. arXiv preprint arXiv:1708.08296(2017).

[59] D Schon. 1983. The reflective practitioner. Howprofessionals think in action. Temple Smith, London.

[60] Junnan Shan, Mario R Casu, Jordi Cortadella, LucianoLavagno, and Mihai T Lazarescu. 2019. Exact andHeuristic Allocation of Multi-kernel Applications toMulti-FPGA Platforms. In Proceedings of the 56thAnnual Design Automation Conference 2019. ACM, 3.

[61] Tony Shuo Tao. 2018. Design, optimization, andperformance of an adaptable aircraft manufacturingarchitecture. Ph.D. Dissertation. Massachusetts Instituteof Technology.

[62] Madeleine Udell, Karanveer Mohan, David Zeng, JennyHong, Steven Diamond, and Stephen Boyd. 2014.Convex optimization in Julia. In High PerformanceTechnical Computing in Dynamic Languages(HPTCDL), 2014 First Workshop for. IEEE, 18–28.

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 12

[63] Eppinger Steven D Ulrich, Karl T and Maria C Yang.2011. Product Design and Development. (2011).

[64] Bo Wu, Murat Cubuktepe, Suda Bharadwaj, and UfukTopcu. 2019. Reward-Based Deception with CognitiveBias. arXiv preprint arXiv:1904.11454 (2019).

[65] Chin Chang Wu. 1976. Design and modeling of solarsea power plants by geometric programming.Carnegie-Mellon Univ., Pittsburgh, PA (USA).

[66] Maria C Yang. 2005. A study of prototypes, designactivity, and design outcome. Design Studies 26, 6(2005), 649–669.

[67] Maria C Yang. 2009. Observations on conceptgeneration and sketching in engineering design.Research in Engineering Design 20, 1 (2009), 1–11.

[68] Martin A York, Warren W Hoburg, and Mark Drela.2017. Turbofan engine sizing and tradeoff analysis viasignomial programming. Journal of Aircraft 55, 3(2017), 988–1003.

CHI 2020 Paper CHI 2020, April 25–30, 2020, Honolulu, HI, USA

Paper 285 Page 13