Sets and Counting

JA RitceyEE 416

Revised Oct 5 2009

Please elaborate with your own illustrations!

Elementary Set Theory

• Universe – superset containing any element of interest. This is the complement of the empty set.

• Subset – collection of none, some, or all elements• Set operations – complement, union, intersection• Cardinality – the number of elements in a set, size• Cardinality can be finite, countable, or uncountable

• Venn diagrams are useful pictorial representations• Set operations can be carried out in MATLAB!

Set Operations

• U = { u_1, … , u_N } with card(U) = N• Count by placing elements of U in one-to-one

correspondence with the natural numbers

• You should be able to carry out set operations by taking unions, intersections, and complements.

• You should be able to handle both lists and geometric regions.

Disjoint or Mutually Exclusive Sets

• 2 sets a disjoint or mutually exclusive if they have no common elements

• disjoint means the intersection is the empty set

• Disjoint sets are simpler than those with overlap

• A collection of sets is mutually disjoint if no two intersect. Together they partition the union.

• Partition – a collection of disjoint subsets that is complete and together yields the entirety

Partitions

• A set S is partitioned by a collection of disjoint subsets that together encompass all of S

• The days of the week are partitioned into work days and weekend

• The numbers {1,2,3, …} are partitioned into odd and even

• Partitions are used to when something is held invariant over each subset

Sensor Nodes Partitioned by Radius from Acces Point

Cartesian Products

• Given 2 sets A = { a_i } B = { b_j } create the • Ordered pair c = (a_i, b_j )

• C = AxB is called the Cartesian product• |C| = |AxB| = |A|x|B| . This is called the

multplication principle in counting (combinatorics)

• Select one from column A and 1 from column B

• This can be extended to ordered J-tuples• C= A_1 x … x A_J the J-fold product• A vector can be created this way

Power Sets

• Power Set Pwr(S) is the set of all subsets of any S

• If a set S is finite, say |S| = N, then• |Pwr(S)| = 2^N• To see this equate each member of Pwr(S) with

the binary string (b_1, b_2, … , b_N ).• The b_i in (0,1) label whether element s_i is in

the particular subset.• There are 2^N binary strings of length N

• If the cardinality, or size, of A is countable, the power set has uncountable cardinality

Properties of the Power Set• Pwr(S) has some closure properties• A collection of sets C is closed under set operations:• (1) If A in C, A^c in C (closed under complements)• (2) If A_1, …A_N in C, their union is in C

• Often this last property is extended to countable unions. If A_1, A_2,… in C, their union is in C

• Collections of sets with this property are called algebras and sigma-algebras.

• When S is uncountable Pwr(S) is too big to assign probabilities to everything – a subtle point

Generating some small power sets

• function powerset(n);• %function

powerset(n);

• n1=n-1;• for knt = 0:2^n1-1• C= bitget(knt,n1:-

1:1);• disp([knt C])• end

0 0 0 0 0 1 0 0 0 1 2 0 0 1 0 3 0 0 1 1 4 0 1 0 0 5 0 1 0 1 6 0 1 1 0 7 0 1 1 1 8 1 0 0 0 9 1 0 0 1 10 1 0 1 0 11 1 0 1 1 12 1 1 0 0 13 1 1 0 1 14 1 1 1 0 15 1 1 1 1

Counting Finite Sets

• We have already done some counting, using the Multiplications Principle

• Cartesian product card |C| = |AxB| = |A| |B|• All pairs must be possible• The size of A and B can be different, but finite

• There are 52 = 4x13 cards in a deck• There are 100 = 2x50 US Senators

• Lets call a set S an N-set when |S|=N

Ordered Selection w/ Repetition

• Given a base N-set S• In how many ways can we select k objects• Assume an ordered selected With

replacement. So that repetition is possible

• We can do this is N^k ways by the multiplication principle

• Just equate the selection of an object with an ordered k-tuple The elements are independently selected and each drawn from an N-set (repetitions)

Tabular View

• Repetition• With Without• • W/ Order N^k N!/(N-k)!

• WO/ Order (N+k-1)!/ (N-1)! k! N!/(N-k)!k!• (N+k-1 choose k) (N choose

k)

• It is easier to remember these, when they are considered as a group

Ordered Selection W/Replacement

• Each Selection is a row• Possible Choices along

columns

• 7^12 = 7 x7 …x7

• Cartesian Product

• Illustrates the Multiplication Principle

1 2 3 4 5 6 7

2

3

4

5

6

7

8

9

10

11

12

Examples

• Cards = 4 suits x 13 face values = 52 cards

• You can place r balls into n cells in n^r ways• n^r = n x n … x n

• Harder: How many ways can r flags be put on n flagpoles?

• Ans: n x (n+1) x (n+2) x … x (n+r-1)• Why: The second flag can be put above or

below the first. So the number of choices increases

Ordered Selection wo/ Repetition

• Given a base N-set S• In how many ways can we select k objects• Assume an ordered selected without

replacement. So that repetition is not possible• We can do this is (N)_k = N(N-1) …(N-k+1)

ways, again by the multiplication principle• Just equate the selection of an object with an

ordered k-tuple The elements are each drawn from sets of decreasing size

• When k=N, we can select in N! ways

Tabular View

• Repetition• With Without• • W/ Order N^k N!/(N-k)!

• WO/ Order (N+k-1)!/ (N-1)! k! N!/(N-k)!k!• (N+k-1 choose k) (N choose

k)

• It is easier to remember these, when they are considered as a group

Factorial Interpretation

• The factorial n! = n(n-1)(n-2) …(3)(2)(1) 0!=1• It gets very big very fast,• n! asy sqrt(2pi)n^(n+1/2)exp(-n)

• Then (N)(N-1)…(N-k+1) = N! / (N-k)!

• Best ways to compute the factorial is through the Gamma Function (gamma.m) More later.

• You can permute (reorder) an N-tuple in N! ways

Factorial Growth

Stirling’s Formula

Matlab M-File• functioncompareGamma(M)

;

• m = [1:M]';half=1/2;• fact = gamma(m+1);• plot( m, fact, 'b*-');grid• title('m! = Gamma(m+1) -

Grows very fast');• xlabel('m');ylabel('G(m)');

• %try logs to compress the dynamic range

• figure(2)• subplot(121);• stir =• sqrt(2*pi).*m.^(m+half).*exp(-

m);• semilogy( m, fact,

'b*-',m,stir,'ro');grid• subplot(122);• RE = abs(stir-fact)./fact;• semilogy(m,RE,'r*');grid• title('Relative Error');• figure(1)

Unordered selection wo/ Repetition

• What if we select without regard to the order.• Solve this by reconsidering ordered selection

wo/ replacement of k objects from N-set• Each such selection resulted a k-tuple that

indexed the selection a_i is that object selected on the i^th turn

• But a k-tuple can be reordered (permuted) in k! ways

• Therefore we need to reduce our result to• N(N-1)…(N-k+1)/k! = N! / (N-k)! K! • Combinations - binomial coefficient ( N choose

k )

Tabular View

• Repetition• With Without• • W/ Order N^k N!/(N-k)!

• WO/ Order (N+k-1)!/ (N-1)! k! N!/(N-k)!k!• (N+k-1 choose k) (N choose

k)

• It is easier to remember these, when they are considered as a group

Computing Binomial Coefficient

• { n choose k } = n! / (n-k)! k!• But n! = gamma(n+1) = G(n+1) (gamma.m)

• { n choose k } = G(n+1)/G(n+1-k) G(k+1)

• Even better to use loggamma = log(Gamma(m))

• { n choose k } = • exp( gammln(n+1) -gammaln(n+1-k) -

gammaln(k+1) )

More Binomial Coefficient

• binomialcoef(n) Think (1+x)^n. The coefficients:

• 1 1• 1 2 1• 1 3 3 1• 1 3 5 3 1• 1 4 9 9 4 1• 1 6 15 19 15 6 1

• Computation – recursion, approximate, logGamma

Matlab Code

• function binomialcoef(n);• %generates an array of binomial coefficients

• k = [0:n];• lgGn = gammaln(n+1);• lgGnk = gammaln(n+1-k);• lgGk = gammaln(k+1);• • bico = exp( lgGn -lgGnk -lgGk );• bico = floor(bico);• fprintf('%5.0f', bico);

Unordered w/ Repetition

• This is the most complicated situation so far• Thinks for a long time, then come up with a• Clever way to represent a selection bars & balls• 4 objects choose 2 o||o| select object 1 and 3• Another choice is |||oo select object 4 twice

• To select k objects from N-set • Use N-1 bars and k balls string of |||oo• But these can be drawn in ( N+k-1 choose k)

ways• (N+k-1)!/(N-1)!k!

Tabular View

• Repetition• With Without• • W/ Order N^k N!/(N-k)!

• WO/ Order (N+k-1)!/ (N-1)! k! N!/(N-k)!k!• (N+k-1 choose k) (N choose

k)

• It is easier to remember these, when they are considered as a group