What is an Agent? - NC State Computer Science · 2008-03-31 · What is an optimal or correct resource allocation c Munindar P. Singh, CSC 513, Spring 2008 p.242 ... Given two horses

Module 7: Rationality

Basis for understanding interactions amongautonomous partiesMany questions reduce to resourceallocationWhat is an optimal or correct resourceallocation

c©Munindar P. Singh, CSC 513, Spring 2008 p.242

What is an Agent?

Abstraction to support autonomy andheterogeneity

In general, an agent is an activecomputational entity that

Carries a persistent (i.e., long-lived)identityPerceives, reasons about, and initiatesactivities in its environmentAdapts its behavior based on others’behaviorCommunicates (with other agents)

Example: an agent in a market or supply chain


http://www.csc.ncsu.edu/faculty/mpsingh/

http://courses.ncsu.edu/csc513/lec/001/



Negotiation

The key to adaptive, cooperative behaviorThe essence of businessInvolves several nontechnical considerationsThe computer science role is to providerepresentations and algorithms to facilitatenegotiation


Explicit Negotiation

Achieving agreement among a small set ofagents, e.g., to construct a supply chain

Based on “dialog” movesSuch as propose, counter-propose,support, accept, reject, dismiss, retractComposed into protocols allowing valid“conversations”

Presupposes a common abstraction of theproblem, and a common content language

Relate the moves to resource allocations,contracts, and post-contractual events






Deals: Joint Plans Among Agents

Possible outcomes of negotiationUtility of a deal for an agent is its benefitminus its costNegotiation set: set of deals (joint plans) withpositive utility for each agent

Conflict: the negotiation set is emptyCompromise: each agent prefers to workalone, but will agree to a negotiated dealCooperation: all deals in the negotiationset are preferred by both agents overachieving their goals alone


Simple Negotiation Protocol

Each party, in turn, proposes a joint planThey proceed as long as they agreeAny conflicts are resolved randomly (e.g.,“flip a coin” to decide which agent wouldsatisfy its goal)

Example: two friends each want coffee; one ofthem (selected via a coin toss) will fetch coffeefor both






Mechanism Design

Mechanism: a set of rules of an environmentunder which agents operate

Honor systemsHonor systems with social censure (as apenalty)AuctionsPaying taxes (voluntary, but with selectiveaudits and severe penalties for violators)

How do the above compare?Mechanism design: Creating a mechanismto obtain desired system-level properties,e.g., participating agents interactproductively and fairly


Example Mechanism: Puzzle

Given two horses to be raced for a mileOwner of horse proved faster wins a reward

Each owner is or hires a jockeyThe horses are raced against each otherThe winner of the race wins

Owner of horse proved slower wins a rewardMight consider rewarding the loser of arace, but such a race won’t terminatebecause each rider will want to go slowerthan the other






Economic AbstractionsA well-established approach to capture complexsystems of autonomous agents (people orsoftware)

Incomplete by themselvesSupport achieving optimal resourceallocationsProvide a basis for achieving somecontractual behaviors, especially in helping

An individual agent decide what to doAgents negotiate


How Can Trade Work?Whether barter or using money

Why would rational agents voluntarilyparticipate?Both cannot possibly gain; or can they?Consider the following. Would you trade

A dollar bill for another dollar bill?A US dollar for x Euros?Money for a bottle of drinking water?A bottle of drinking water for money?

It comes down to your valuations: differences invaluations make trade possible






Markets Introduced

Compare stock with specific real-estateCan be

PublicPrivate: part of restricted exchanges

Can restrict kinds of goods tradedEndogenous: NASDAQExogenous: eBay, where physical goodsare traded outside the scope of themarket

Offer some form of nonrepudiation


Centrality of Prices

A price is a scalar: easy to compareThe computational state of a market isdescribed completely by current prices forthe various goodsCommunications are between eachparticipant and the market, and only in termsof pricesParticipants reason about others and choosestrategies entirely in terms of prices being bid






Functions of a Market

Provides this information to participantsTakes requests (buy, sell bids) fromparticipants, enforcing rules such as bidincrements and time limitsDecides outcome based on messages fromparticipants, considering rules such asreserve prices, . . .


Achieving Equilibrium

When supply equals demandAt equilibrium, the market has computed theallocation of resources

Dictates the activities and consumptionsof the agents

Under certain conditions, a simultaneousequilibrium of supply and demand across allgoods exists

That is, the market “clears”Reachable via distributed biddingPareto optimal: you cannot make theallocation better for one agent withoutmaking it worse for another






Pareto Optimality

Allocation: how resources are allocated todifferent partiesThink of a vector of allocations, onedimension for each participantAn allocation is Pareto optimal ifimprovements along any dimension must beaccompanied by a reduction along anotherdimension


Auctions in Markets

Computational mechanism to manage supplyand demand: support dynamic pricing

Exchange common object (money) for goodsAscending (English) vs. Descending(Dutch)Silent (auctioneer names a price; bids aresilent) vs. outcry (bids name prices;auctioneer listens)Hidden identity or notCombinatorial: involve bundles or sets ofgoods






English Auction

Buyers bid for an itemPrices start low and increaseHighest bidder gets the object and pays theprice bidVariations:

Minimum bid incrementReserve price (no sale if too low)Limited time


Dutch Auction

Price “clock” or counter starts high and windsdownFirst to stop the clock wins and pays theprice on the clockIn other words, the highest bidder wins andpays the price bid






Winner’s Curse: 1

If you just won an English or a Dutch auctionYou just paid $x for somethingHow much can you sell it for?Obviously, you will be able to sell it for . . .

Not quite a curse if inherently valuable, butperhaps could have obtained the item for less


Winner’s Curse: 2

Sealed bid; no resaleA group of mutually independent peopleestimate the values of different goods andbid accordinglyAssume that the group is smart

The average is about right as an estimateof the true value

The winner bid the maximum






Fish Market Auction

Imagined scenario is based on a Spanish fishmarket

Auctioneer calls out pricesIf two or more bidders

repeat with higher priceIf no bidders

repeat with lower price


Suckers’ Auction

Consider two bidders bidding for $1 currencyBid in increments of 10c|Highest bidder winsBoth bidders pay (i.e., loser also pays)Once you are in, can you get out?

The myopically rational strategy is to bidThe outcome is not pleasant






Sealed Bid First-Price AuctionAlso known as tenders: bidding to buy

One-shot bidding without knowing what otherbids are being placedUsed by governments and large companiesto give out certain large contracts (lowestprice quote for stated task or procurement)

All bids are gatheredAuctioneer decides outcomes based ongiven rules (e.g., highest bidder wins andpays the price it bid)


Vickrey Auction

Second-price sealed bid auctionHighest bidder wins, but pays the secondhighest price






Pricing

General theme: allocate resources to thosewho value them the mostKinds of pricing

Fixed: slowly changing, based on variouscriteriaDynamic: rapidly changing, based onactual demand and supply


Fixed Pricing: Example Criteria

Flexibility: (restrict rerouting or refundabilityin air travel)Urgency: (convenience store vs. warehouse)Customer preferences (coupons:price-sensitive customers like them; otherspay full price)DemographicsArtificial (Paris Metro, Delhi “Deluxe” buses)Predicted demand (New York subway, phonerates)






Dynamic Pricing Motivation

Under certain assumptions, markets ensurea resource allocation where all the goodsthat can be sold (for the market price) aresold and all the goods that can be bought(for the market price) are boughtFixed pricing leaves some revenue that otherparties exploit (e.g., in secondary marketssuch as black markets as in football ticketscalping)Participants seek to maximize individualutility; those who value something more willpay more for it


Continuous Double AuctionAs in stock markets, a way to accomplishdynamic pricing

Multiple sellers and buyers, potentially withmultiple sell and buy bids eachBuy bids are like upper boundsSell bids are like lower boundsClears continually:

The moment a buyer and seller agree ona price, the deal is done and the matchingbids are taken out of the marketPossibly, a moment later a better pricemay come along, but it will too late then






Auction Management: Bidding

Bidding rules to govern, e.g.,Whose turn it isWhat the minimum acceptable bid is, e.g.,incrementsWhat the reserve price is, if any

Compare these for outcry, silent, sealed bid, andcontinuous auctions


Auction Management: Information

What information is revealed to participants?Bid value (not in sealed bid auctions)Bidder identity (not in sealed bid auctions orstock exchanges)Winning bid or current high bidWinnerHow often, e.g., once per auction, once perhour, any time, and so on






Auction Management: Clearing

Bids are cleared when they are executed andtaken out of the market

What defines a deal: how are bids matched?What prices? If uniform, then matching isnot relevantWho?

How often?Until when?


Mth and (M+1)st Price Auctions: 1

L = M+N single-unit sealed bids, notcontinuously cleared

M sell bidsN buy bids

Mth price clearing rulePrice = Mth highest among all L bidsEnglish: first price; M=1

Seller’s reserve price is the sole sell bid(assume minimum value, if no explicitreserve price)







(M+1)st price clearing rulePrice = (M+1)st highest among all L bidsVickrey: second price; M=1



The Mth and (M+1)st prices delimit theequilibrium price range, where supply anddemand are balanced

Above Mth price: no demand from somebuyersBelow (M+1)st price: no supply from somesellers

Restrict to M=1






Kinds of Valuations

How do agents place values of goods?Independent (and private): Agents valuegoods in a manner that is unaffected byothers

Consume or use: cakeCommon: Agents value goods entirely basedon others’ valuations, leading to symmetricvaluations

Resale: treasury billsCorrelated: Combination of above

Automobile or house


Concepts About Matching: 1

Buy and sell bids can be matched in variousways, which support different properties:

Equilibrium prices: those at which supplyequals demand, also known as market priceIndividually rational: each agent is no worseoff participating than otherwiseEfficient: No further gains possible fromtrade (agents who value goods most getthem): i.e., Pareto optimal






Concepts About Matching: 2

Uniform price: Multiple units, ifsimultaneously matched, are traded at thesame priceDiscriminatory: Trading price for each pair ofbidders can be differentIncentive compatible: Agents optimize theirexpected utility by bidding their truevaluations


Incentive Compatibility

Incentives are such it is rational to tell the truthRamification: Agents can ignore subtlestrategies and others’ decisions: hencesimpler demands for knowing others’preferences and reasoning about themBasic approach: payoff depends not ondecisions (bids) by selfExample: Vickrey (second-price sealed bid)auctions for independent private valuations

Underbid: likelier to lose, but price paidon winning is unaffected by bidOverbid: likelier to win, but may pay more






Economic Rationality

Space of alternatives or outcomesEach agent has some ordinal (i.e., sorted)preferences over the alternatives, capturedby a binary relation, �

� is a strict orderingAsymmetric, Transitive (impliesirreflexive)

� is not totalAnother binary relation, ∼, capturesindifference


LotteriesProbability distributions over outcomes oralternatives (add up to 1)

In essence, define potential outcomesFlip a coin for a dollar: [0.5: $1; 0.5: –$1]Buy a $10 ticket to win a car in a raffle:[0.0001: car−$10; 0.9999: −$10]Four choices:[p : A; q : B; r : C; 1 − p − q − r : D]






Using Lotteries

Infer (rational) agents’ preferences based ontheir behavior with respect to the lotteries

What odds will a specific person accept?For example, [0.01: car−$10; 0.99: −$10]


Properties of Lotteries

Substitutability of indifferent outcomesIf A ∼ B, then[p : A; (1−p) : C] ∼ [p : B; (1−p) : C]

Monotonicity (for preferred outcomes)If A � B and p > q, then[p : A; (1−p) : B] � [q : A; (1−q) : B]

Decomposibility (flatten out a lottery)Compound lotteries reduce to simplerones[p : [q : A; 1 − q : B]; 1 − p : C] = [pq :A; p − pq : B; 1 − p : C]






Expected Payoff

Expresses the value of a lottery as a scalar(i.e., in monetary terms)Expected payoff is sum of utilities weightedby probability

Calculate for [0.0001: car−$10; 0.9999:−$10] where the car is worth $25,010


Completeness of Preferences

Same as indifference being an equivalencerelation

Given outcomes A and B� means nonstrict preferenceEither A � B or B � A

That is, A ∼ B if and only if A � B andB � A

Thus, ∼ is an equivalence relationReflexivity: A ∼ ASymmetry: A ∼ B implies B ∼ ATransitivity: (A ∼ B and B ∼ C) impliesA ∼ C






Continuity of Preferences

A � B � C implies that there is aprobability p, such that

[p : A; 1 − p : C] ∼ B

Consider A, B, and C to be ice-cream,yogurt, and cookies, respectively

Informally, this means we can pricealternatives in terms of each otherIs this reasonable in real life? Why or whynot?


Utility Functions

One per agentMap each alternative (outcome) to a scalar(real number), i.e., money

U : {alternatives} 7→ RFor agents with irreflexive, transitive,complete, continuous preferences, there is autility function U such that

U(A) > U(B) implies A � B

U(A) = U(B) implies A ∼ B

U([p : A; 1 − p : C]) =p × U(A) + (1 − p) × U(C) (weightedsum of utilities)






Risk: 1According to the above, two lotteries with thesame expected payoff would have equalutilityIn practice, risk makes a big difference

RafflesInsuranceBusiness actions with unpredictableoutcomes


Risk: 2

Showing utilities instead of outcomesConsider two lotteries

L1 = [1 : x]

L2 = [p : y; 1 − p : z]

Where x = py + (1 − p)z. That is, L1and L2 have the same expected payoff

An agent’s preferences reflect its attitude torisk

Averse: U(L1) > U(L2)

Neutral: U(L1) = U(L2)

Seeking: U(L1) < U(L2)






Beyond Simple Utility

Other factors besides expected payoff andrisk are relevant in real life:

Total deal value: $10 discount for a t-shirtvs. for a carCurrent wealth: 1st million vs. 10th millionAltruism or lack thereof

Leads to social choice theory


Sharing Resources

Consider two scenarios for sharing—onlyrequirement is that the parties agree on the split

Splitting a dollar: relative sizes are obvious.Should splits consider the relative wealth ofthe splitters? Should splits consider the taxrates of the splitters?Sharing a cake: relative sizes and otherattributes (e.g., amount of icing) canvary—several cake-cutting algorithms exist






Simplifying Assumptions

Participants are risk neutralWilling to trade money for any of theirresources at a price independent of howmuch money they already have

Participants know their valuations, which areindependent and private


Pareto Optimality

A distribution of resources where no agentcan be made better off without makinganother agent worse offExample: A has goods g and values g at $1;B values g at $3

It is Pareto optimal for B to buy g at aprice between $1 and $3, say $2.50A’s gain: $2.50–$1 = $1.50B’s gain: $3–$2.50 = $0.50

No further gains can be made from trade






Allocations

How do we find Pareto optimal allocations ingeneral?

Private valuationsNo central control

Design mechanisms that are efficient andwhere participants have an incentive to bidtheir private values

Buyers and sellers are symmetrical: mayneed to flip a coin


Vickrey Incentive Compatibility: 1

That is, buy bids equal private valuationsConsider a single sellerConsider two buyer agents A1 and A2, withprivate valuations v1 and v2, bidding b1 andb2

If b1 > b2, A1 wins and pays b2

A1’s utility in that case is v1 − b2: couldbe positive or negative

If b1 < b2, A1 loses the auction: utility = 0(assuming no bidding costs)







Utility to bidder A1: (v1 − b2)

If (v1 − b2) > 0 (i.e., v1 > b2)Then A1 benefits by maximizingProb(b1 > b2)

Underbid: likelier to lose, but would paythe same price if it wins

Else A1 benefits by minimizingProb(b1 > b2)

Overbid: likelier to win, but may paymore than the valuation

Thus, setting the bid equal to valuation is thebest strategy



If A1 wins, what A1 pays depends on bids byother agentsA1 should try to

Win when it would benefit by winningLose when it would suffer by winning

How do the above ideas apply when a buyer isbidding for multiple units of the same item?






Mth and (M+1)st Price Auctions

Vickrey = (M+1)st price, with one unit for saleFor single-unit buyers, (M+1)st price inducestruthfulnessFor multiunit buyers, NO!

A buyer may artificially lower some bids tolower the price for other bids


Dominant Strategies

One which yields a greater payoff for the agentthan any of its other strategies (regardless ofwhat others bid)

Under Vickrey auctions, the dominantstrategy for a buyer is bidding according toits true valueUnder first-price auctions, the dominantstrategy for a seller is to bid its true value






Multiunit AuctionsMultiunit bids are divisible when notnecessarily the whole set needs to bebought or soldWhen multiunit bids are divisible,

Treat multiunit bids as multiple copies ofsingle-unit bidsIf indivisible, e.g., sets of two or four tires,then treat as bundled goods


Desirable Properties of MarketsEfficient: the one values it most gets it

If seller’s valuation < buyer’s valuation,they trade

TruthfulRational to bid true valuation for bothsellers and buyers

Individually rationalNo participant is worse off for participating

Budget balanced, i.e., no subsidy from themarket: Σpayment = Σrevenue

Seller receives what the buyer paysCan all of the above be satisfied?






Impossibility Result

Given a sealed buy bid b and a sealed sell bid s(Meyerson & Satterthwaite)

Valuations of each from overlappingdistributionsUltimately buyer pays pb and seller gets ps

For truthfulness, pb = s and ps = bBut the deal happens only if b > s, elseirrationalThus buyer pays less than the sellerreceives, i.e., a deficit!

That is, subsidize or relax another requirement


McAfee’s Dual Price Auction: 1Let p be a price in the equilibrium range

That is, Mth to (M+1)st

Let’s choose the midpoint to be specificOmits the lowest buyer at or above Mth andthe highest seller at or below (M+1)st

Which of the above properties does the dualprice auction violates?






McAfee’s Dual Price Auction: 2Individually rationalPromotes truthfulnessBudget balancedInefficient

Discards the lowest valued matchNot good if it is the only one


Kinds of Prices

Uniform: all matches at the same priceMth, (M+1)st, Dual

Discriminatory: prices vary for each matchChronological: prices = earlier (or later)bidEach match at one of the bid prices

Buyer’sSeller’s






Auction Example: 1

A1: (sell $1) at time 1A2: (buy $3) at time 2A3: (sell $2) at time 3A4: (buy $4) at time 4Consider (A1 → A4), (A3 → A2),(A3 → A4), (A1 → A2)


Auction Example: Uniform

Mth: (A1 → A4: $3), (A3 → A2: $3),(A3 → A4: $3), (A1 → A2: $3)(M+1)st: (A1 → A4: $2), (A3 → A2: $2),(A3 → A4: $2), (A1 → A2: $2)Dual: (A1 → A4: $2.5), (A3 → A2: NA),(A3 → A4: NA), (A1 → A2: NA)






Auction Example: Discriminatory

Earliest: (A1 → A4: $1), (A3 → A2: $3),(A3 → A4: $2), (A1 → A2: $1)Latest: (A1 → A4: $4), (A3 → A2: $2),(A3 → A4: $4), (A1 → A2: $3)Buyers: (A1 → A4: $4), (A3 → A2: $3),(A3 → A4: $4), (A1 → A2: $3)Sellers: (A1 → A4: $1), (A3 → A2: $2),(A3 → A4: $2), (A1 → A2: $1)


Continuous Double Auctions (CDAs)

As in stock marketsBid quote: what a seller needs to offer toform a matchAsk quote: what a buyer needs to offer toform a matchThe bid-ask spread represents the differencebetween the buyers and the sellers






Mth and (M+1)st as Bid-Ask

Generalize English, Vickrey, double auctions

The Mth price generalizes the ask quoteThe (M+1)st price generalizes the bid quoteWhen the standing bids overlap

Beating the ask price either matches anunmatched seller, or displaces a matchedbuyer


Mth and (M+1)st Winning Bid

A buy bid b wins if either of the followingholds

Either b > pask

Or b = pask and pask > pbid

If b = pask = pbid, then it is not clear if b iswinningSymmetric result for the seller






Module 8: Summary and Directions

Collective concept map


Key Ideas

Information system interoperationArchitecture conceptuallyImportance of metadataXML technologiesElements of rational resource allocation






Business Environments

Theme of this course: How is computer sciencedifferent for open environments?

AutonomyMessaging, not APIsMarkets

HeterogeneityCapturing structure of informationTransforming structures

DynamismPartially addressed through above

Support flexibility and arms-length relationships


Course: Service-Oriented Computing

Takes the ideas of this course closer to theirnatural conclusionsFor autonomous interacting computations

Basic standards that build on XMLDescriptions through richerrepresentations of meaningEngagement of parties in extendedtransactions and processesCollaboration among partiesSelecting the right parties

How to develop and maintain flexible,arms-length relationships






Documents

What is an Agent? - NC State Computer Science · 2008-03-31 · What is an optimal or correct resource allocation c Munindar P. Singh, CSC 513, Spring 2008 p.242 ... Given two horses