IERG 3050 Week 6 Generating Random Variates

IERG 3050 Week 6Generating Random Variates

Bolei ZhouDepartment of Information Engineering

The Chinese University of Hong Kong

Announcement

• Pick up your graded homework 1• After class today• Go to TA’s tutorials or office hour or by appointment at SHB 702

• This week’s tutorial will go through homework 1 and homework 2 briefly• Work on your homework 2 (no need to hand in)• Prepare for Quiz 1 next Wed (Oct.16)

Outline• Generating random variates from U(0, 1) (a.k.a. generating

random numbers)• Linear congruential generators (LCGs)• Testing pseudo-random number generators (PRNGs)

• Generating random variates from arbitrary distributions • Inverse transform method• Acceptance/Rejection method• Composition method• Convolution method

• Reading: Chapters 7 and 8

□ Acknowledgement: Prof. Minghua Chen, Prof. Rosana Chan, Prof. Angela Zhang, Prof. Jianwei Huang, and Prof. Pascal Vontobel for contributing to the slides

3

Be careful about the saying

4

Random-Number Generation

5

Criteria for PRNGs

6

Example code

import randomrandom.seed(3)random.random()random.random()

Arithmetic way to generate ‘Random’ numbers

• First arithmetic generator by Von Neumann and Metropolis in 1940s• Midsquare method:

1. Let Z0 be a four-digit positive integer such as 71822. Square Z0 to make it eight digit number3. take the middle four digits as next four digit Z1, U1 be the 0.Z1

4. Repeat

Problem with the Midsquare method

• It has a strong tendency to degenerate rapidly to zero and stay there forever• Better solution for sequential arithmetic random number generator

given Z0

Zi = f(Zi-1)Ui = g(Zi)

Linear Congruential Generators (LCGs)

10

Example 1

11

Example 2

12

LCGs

13

Period is determined by the parameters

Period of LCGs

15

m is usually very large, say 109 or more (a billion possible values)

Full-Period Theorem

16

Desired Properties of LCG

17

LCG with c = 0: Multiplicative LCG

18

Integer Overflow

19

b = 4

Some “Good” LCGs (with One Exception)

20

Parameters in common use

Testing Random Number Generators

22

Uniformity Test

23

Uniformity Test: Example

24

Higher-Dimensional Uniformity Test (Serial Test)

25

Runs-Up Test

26

It is a test of independence only, not for uniformity

Runs-Up Test

27

Runs-Up Test

28

Runs-Up Test

Correlation Test

Outline• Generating random variates from U(0, 1) (a.k.a.

generating random numbers)• Linear congruential generators (LCGs)• Testing pseudo-random number generators (PRNGs)

• Generating random variates from arbitrary distributions

31

Generating random variates from arbitrary distributions

32

Inverse-Transform Method

33

Proof of Inverse-Transform Method

34

Intuition of Inverse-Transform Method

View f(x) as the slope function of F(x)

Intuition of Inverse-Transform Method

View f(x) as the slope function of F(x)

Example

37

Exercise

38

Another Example

39

Inverse Transform Method for Discrete Random Variates

40

Acceptance/Rejection Method

41

Motivating Example 1

42

Motivating Example 1

43

Motivating Example 2

44

Motivating Example 2

45

Motivating Example 2

46

Motivating Example 2

47

(a)

Motivating Example 2

48

Acceptance/Rejection Method

49

Acceptance/Rejection Method

50

Acceptance/Rejection Method

51

Composition Method

52

Composition Method

53

Example 1

54

Example of combining three generation techniques

55

Textbook. Example 8.7

Convolution Method

56

Convolution Method

57

Example

58

Typical Continuous Distributions: Uniform

Typical Continuous Distributions: Exponential

Typical Continuous Distributions: m-Erlang

Typical Continuous Distributions: Gamma

Typical Continuous Distributions: Normal distribution

Typical Continuous Distributions: Lognormal

Typical Continuous Distributions: Beta

Typical Discrete Distribution: Bernoulli

Typical Discrete Distribution: Binomial

Typical Discrete Distribution: Geometric

Other Methods

• See Chapter 8 of the book by Averill M. Law on additional, more sophisticated and/or more specialized ways to generate random variates for commonly used distributions (gamma, lognormal,etc)• Code example: The standard random function in pythonhttps://github.com/python/cpython/blob/3.7/Lib/random.py

69

Next Week

• Confidence intervals and hypothesis testing• Required reading: Chapter 4