Upload
danny-manno
View
238
Download
0
Embed Size (px)
Citation preview
8/2/2019 W11 Linear Regression
1/29
Centre for Computer Technology
ICT114Mathematics for
Computing
Week 11
Linear Regression
8/2/2019 W11 Linear Regression
2/29
March 20, 2012March 20, 2012 Copyright Box Hill Institute
ObjectivesObjectives
Review week 10Review week 10
Curve FittingCurve Fitting
RegressionRegressionDependent/Independent variableDependent/Independent variable
Method of Least SquaresMethod of Least Squares
Least Squares lineLeast Squares line
8/2/2019 W11 Linear Regression
3/29
March 20, 2012March 20, 2012 Copyright Box Hill Institute
Numerical DifferentiationNumerical Differentiation
Newtons Forward Difference FormulaNewtons Forward Difference Formula
ff//
(x)= (1/h) [ (x)= (1/h) [ 11
//22 2 +2 +
++11
//33 33
11//44
44 +.]+.]
Newtons Backward Difference FormulaNewtons Backward Difference Formulaff//(x)= (1/h) [(x)= (1/h) [ ++11//22
22++11//3333
++ 11//4444+..]+..]
8/2/2019 W11 Linear Regression
4/29
March 20, 2012March 20, 2012 Copyright Box Hill Institute
Numerical IntegrationNumerical Integration
Trapezoid RuleTrapezoid Rule
= (= (hh//22) [ f(a) [ f(a00) + 2 f(a) + 2 f(a11) + 2f(a) + 2f(a22) +.) +.
+ 2 f(a+ 2 f(a
n-1n-1) + f(a) + f(a
nn)])]
Simpsons One Third Rule (the number ofSimpsons One Third Rule (the number ofintervals have to be even)intervals have to be even)
= (h/3)[ ( f(a= (h/3)[ ( f(a00) + f (a) + f (ann) )) )
+ 4 (f(a+ 4 (f(a11) + f (a) + f (a33) + f(a) + f(a55)+ f(a)+ f(an-1n-1) )) )
+ 2 (f(a+ 2 (f(a22) + f (a) + f (a44) + f(a) + f(a66) + .. f(a) + .. f(an-2n-2) ) ]) ) ]
8/2/2019 W11 Linear Regression
5/29
Centre for Computer Technology
Curve FittingCurve Fitting
8/2/2019 W11 Linear Regression
6/29
March 20, 2012March 20, 2012 Copyright Box Hill Institute
Curve Fitting (1)Curve Fitting (1)
A relation between two variables isA relation between two variables is
expressed mathematically by an equationexpressed mathematically by an equation
connecting both.connecting both.For example if x and y are the height andFor example if x and y are the height and
weight of an individualweight of an individual
Then a sample of n individuals will haveThen a sample of n individuals will haveheight xheight x11, x, x22, .., x, .., xnn
weight yweight y11, y, y22, .., y, .., ynn
8/2/2019 W11 Linear Regression
7/29
March 20, 2012March 20, 2012 Copyright Box Hill Institute
Curve Fitting (2)Curve Fitting (2)
Plotting the points (xPlotting the points (x11, y, y11), (x), (x22,y,y22), .(x), .(xnn,y,ynn))
on a rectangular coordinate system willon a rectangular coordinate system will
result in aresult in a scatter diagram.scatter diagram.The data in the scatter diagram can beThe data in the scatter diagram can be
generallygenerally approximated by a smoothapproximated by a smooth
curve.curve.The resulting curve is called theThe resulting curve is called the
approximating curveapproximating curve..
8/2/2019 W11 Linear Regression
8/29
March 20, 2012March 20, 2012 Copyright Box Hill Institute
8/2/2019 W11 Linear Regression
9/29
March 20, 2012March 20, 2012 Copyright Box Hill Institute
Curve Fitting (3)Curve Fitting (3)
If the data is approximated byIf the data is approximated bystraight line there is a linear relationshipstraight line there is a linear relationship
between the variablesbetween the variableselse, there is a nonlinear relationshipelse, there is a nonlinear relationship
Finding equations toFinding equations to approximating curvesapproximating curvesthat fit the given set of data is called curvethat fit the given set of data is called curve
fitting.fitting.
8/2/2019 W11 Linear Regression
10/29
March 20, 2012March 20, 2012 Copyright Box Hill Institute
8/2/2019 W11 Linear Regression
11/29
March 20, 2012March 20, 2012 Copyright Box Hill Institute
Curve Fitting (4)Curve Fitting (4)
The general equation of a straight isThe general equation of a straight is
y = a +bx (linear relation)y = a +bx (linear relation)
The general equation for a parabola or aThe general equation for a parabola or a
quadratic equation isquadratic equation is
y = a + bx + cxy = a + bx + cx22 (nonlinear relation)(nonlinear relation)
8/2/2019 W11 Linear Regression
12/29
March 20, 2012March 20, 2012 Copyright Box Hill Institute
8/2/2019 W11 Linear Regression
13/29
Centre for Computer Technology
Regression/Regression/Method of Least SquaresMethod of Least Squares
8/2/2019 W11 Linear Regression
14/29
March 20, 2012March 20, 2012 Copyright Box Hill Institute
RegressionRegression
In curve fitting we need to determine oneIn curve fitting we need to determine one
of the variables (the dependent variable)of the variables (the dependent variable)
from the other (the independent variable)from the other (the independent variable)
This process of estimation is calledThis process of estimation is called
regressionregression
8/2/2019 W11 Linear Regression
15/29
March 20, 2012March 20, 2012
Copyright Box Hill Institute
Dependent/Independent VariableDependent/Independent Variable
y = a + bxy = a + bx
determine the variables?determine the variables?
the corresponding equation is called thethe corresponding equation is called the
regression equation of y on xregression equation of y on xthe corresponding curve is called thethe corresponding curve is called theregression curve of y on xregression curve of y on x
8/2/2019 W11 Linear Regression
16/29
March 20, 2012March 20, 2012
Copyright Box Hill Institute
Method of Least Squares (1)Method of Least Squares (1)
It is a method to find the best fitting curveIt is a method to find the best fitting curve
for the data points (scatter diagram)for the data points (scatter diagram)
Let (xLet (x11, y, y11), (x), (x22,y,y22), .(x), .(xnn,y,ynn) be the data) be the datapoints.points.
LetLet C be the best fitting curveC be the best fitting curve
For a given value of x, xFor a given value of x, x11 there will be athere will be adifference between the corresponding ydifference between the corresponding y
and the value determined by C.and the value determined by C.
8/2/2019 W11 Linear Regression
17/29
March 20, 2012March 20, 2012
Copyright Box Hill Institute
8/2/2019 W11 Linear Regression
18/29
March 20, 2012March 20, 2012Copyright Box Hill Institute
Method of Least Squares (2)Method of Least Squares (2)
For the data points (xFor the data points (x11, y, y11), (x), (x22,y,y22), .), .
(x(xnn,y,ynn). Let the corresponding). Let the corresponding deviationsdeviations
be dbe d11, d, d22,, d,, dnn..The measure of the curve is provided byThe measure of the curve is provided by
D = dD = d1122 + d+ d22
22++d++dnn22
IfIfD is small the fit is goodD is small the fit is good
IfIfD is large the fit is badD is large the fit is bad
8/2/2019 W11 Linear Regression
19/29
March 20, 2012March 20, 2012Copyright Box Hill Institute
Method of Least Squares (3)Method of Least Squares (3)
The curve having this property is said to fit theThe curve having this property is said to fit the
data indata in least squares senseleast squares sense
The curve is called aThe curve is called a least squares regressionleast squares regression
curve or a least squares curvecurve or a least squares curve
Dependent VarDependent Var yy
Independent VarIndependent Var xx
OffsetOffset verticalvertical
8/2/2019 W11 Linear Regression
20/29
March 20, 2012March 20, 2012Copyright Box Hill Institute
Least Squares LineLeast Squares Line
For the data points (xFor the data points (x11, y, y11), (x), (x22,y,y22), .), .
(x(xnn,y,ynn))
LetLet y = a+bxy = a+bx is the least squares lineis the least squares linea and b are determined by solving thea and b are determined by solving the
(normal) equations(normal) equations
y = an + b xy = an + b x
xy = a x + b xxy = a x + b x22
8/2/2019 W11 Linear Regression
21/29
March 20, 2012March 20, 2012Copyright Box Hill Institute
Least Squares Line (y on x)Least Squares Line (y on x)
( y) (x( y) (x22) ( x) (xy)) ( x) (xy)
a = ---------------------------------------------a = ---------------------------------------------
n xn x22 (x) (x)22
n xy ( x)( y)n xy ( x)( y)
b = ----------------------------------------------b = ----------------------------------------------
n xn x22 (x) (x)22
8/2/2019 W11 Linear Regression
22/29
March 20, 2012March 20, 2012Copyright Box Hill Institute
ExampleExample
The table below shows the respective heights xThe table below shows the respective heights x
and y of a sample of 12 fathers and their oldestand y of a sample of 12 fathers and their oldest
sons. Find the least squares regression line of ysons. Find the least squares regression line of y
on x.on x.
Father (x) 65 63 67 64 68 62 70 66 68 67 69 71
Son (y) 68 66 68 65 69 66 68 65 71 67 68 70
8/2/2019 W11 Linear Regression
23/29
March 20, 2012March 20, 2012
Copyright Box Hill Institute
x y x2 xy y2
65
6367
64
68
62
70
66
68
67
69
71
68
6668
65
69
66
68
65
71
67
68
70
4225
39694489
4096
4624
3844
4900
4356
4624
4489
4761
5041
4420
41584556
4160
4692
4092
4760
4290
4828
4489
4692
4970
4624
43564624
4225
4761
4356
4624
4225
5041
4489
4624
4900
x = 800 y = 811 x2 = 53418 xy = 54107 y2 = 54849
8/2/2019 W11 Linear Regression
24/29
March 20, 2012March 20, 2012
Copyright Box Hill Institute
the normal equations arethe normal equations are
12a + 800b = 81112a + 800b = 811
800a + 53418b = 54107800a + 53418b = 54107
solving the above equations we getsolving the above equations we get
a = 35.82, b = 0.4776a = 35.82, b = 0.4776y = 35.82 + 0.4776xy = 35.82 + 0.4776x is the equation for theis the equation for the
regression lineregression line
8/2/2019 W11 Linear Regression
25/29
March 20, 2012March 20, 2012
Copyright Box Hill Institute
QuestionQuestion
The table below shows theThe table below shows the Consider the
variation of the bulk modulus of Silicon Carbide
as a function of temperature. Find the leastFind the least
squares regression line of y (G) on x (T).squares regression line of y (G) on x (T).
8/2/2019 W11 Linear Regression
26/29
March 20, 2012March 20, 2012
Copyright Box Hill Institute
SummarySummary
Curve FittingCurve Fitting
A relation between two variables isA relation between two variables is
expressed mathematically by an equationexpressed mathematically by an equationconnecting both.connecting both.
Finding equations approximating curvesFinding equations approximating curves
that fit the given set of data is called curvethat fit the given set of data is called curvefitting.fitting.
8/2/2019 W11 Linear Regression
27/29
March 20, 2012March 20, 2012
Copyright Box Hill Institute
SummarySummary
Method of Least SquaresMethod of Least SquaresFor the data points (xFor the data points (x11, y, y11), (x), (x22,y,y22), .), .
(x(xnn,y,ynn)) If y = a+bx is the least squares lineIf y = a+bx is the least squares linea and b are determined by solving thea and b are determined by solving the
(normal) equations(normal) equations y = an + b xy = an + b x
xy = a x + b xxy = a x + b x22
8/2/2019 W11 Linear Regression
28/29
March 20, 2012March 20, 2012
Copyright Box Hill Institute
SummarySummary
Method of Least SquaresMethod of Least Squares
( y) (x( y) (x22) ( x) (xy)) ( x) (xy)
a = ---------------------------------------------a = ---------------------------------------------
n xn x22 (x) (x)22
n xy ( x)( y)n xy ( x)( y)b = ----------------------------------------------b = ----------------------------------------------
n xn x22 (x) (x)22
8/2/2019 W11 Linear Regression
29/29
March 20, 2012March 20, 2012
Copyright Box Hill Institute
ReferencesReferences H L Verma and C W Gross : Introduction toH L Verma and C W Gross : Introduction to
Quantitative Methods,John WileyQuantitative Methods,John Wiley JB Scarborough : Numerical MathematicalJB Scarborough : Numerical Mathematical
Analysis, Jon Hopkins Hall, New JerseyAnalysis, Jon Hopkins Hall, New Jersey Gerald W. Recktenwald, Numerical MethodsGerald W. Recktenwald, Numerical Methodswith MATLAB, Implementation and Application,with MATLAB, Implementation and Application,Prentice HallPrentice Hall
Murray Spiegel, John Schiller, Alu Srinivasan,Murray Spiegel, John Schiller, Alu Srinivasan,Probability and Statistics, Schaums easyProbability and Statistics, Schaums easyOutlinesOutlines
http://mathworld.wolfram.comhttp://mathworld.wolfram.com