Skip to content

Machine LearningModule 3 · Guide and practice

Module 3 · Fit a line and know what it means

Multiply inputs by weights. Add a baseline. Check the misses. Check the uncertainty. That is the path through Module 3.

Read each small idea, then cover the answer and try its calculation. Jump to Q1: matrices, Q2: averages, Q3: fit a line, or Q5: test slopes. Keep the formula sheet nearby.

Browse the original slides above each topic; click a slide to enlarge it. Practice-sheet previews include the original solutions.

Linear Algebra — original slide 2 of 4
Original slides · Linear AlgebraOpen PDF ↗
Slide 2 of 4
  • Scalar: one number.
  • Vector: an ordered list, usually a column.
  • Matrix: a rectangular grid.
  • ARm×nA\in\mathbb R^{m\times n}: matrix AA has mm rows, nn columns, and real entries.
  • AijA_{ij}: the entry in row ii, column jj.
  • Transpose ATA^\mathsf T: swap rows and columns.

For example, [2,1]T[2,1]^\mathsf T is the column with 2 above 1.

Shape trick: inner dimensions match; outer dimensions survive.

(m×n)(n×p)(m×p).(m\times n)(n\times p)\longrightarrow(m\times p).

Multiply matching entries, then add:

(AB)ij=k=1nAikBkj.(AB)_{ij}=\sum_{k=1}^{n}A_{ik}B_{kj}.

kk runs along row ii of AA and down column jj of BB. The sum sign \sum means “add these terms.”

Invented example: use each row of AA with input column xx.

A=[1234],x=[21].A=\begin{bmatrix}1&2\\3&4\end{bmatrix}, \qquad x=\begin{bmatrix}2\\1\end{bmatrix}. Ax=[1(2)+2(1)3(2)+4(1)]=[410].Ax=\begin{bmatrix}1(2)+2(1)\\3(2)+4(1)\end{bmatrix} =\begin{bmatrix}4\\10\end{bmatrix}.

Inner product = number; outer product = grid

Section titled “Inner product = number; outer product = grid”

For column vectors u,vRnu,v\in\mathbb R^n:

uTv=k=1nukvk,(uvT)ij=uivj.u^\mathsf Tv=\sum_{k=1}^{n}u_kv_k, \qquad (uv^\mathsf T)_{ij}=u_iv_j.

Invented values: u=[2,1]Tu=[2,1]^\mathsf T, v=[3,4]Tv=[3,4]^\mathsf T.

  • Inner: 2(3)+1(4)=102(3)+1(4)=10.
  • Outer: every entry of uu multiplies every entry of vv:
uvT=[2(3)2(4)1(3)1(4)]=[6834].uv^\mathsf T=\begin{bmatrix}2(3)&2(4)\\1(3)&1(4)\end{bmatrix} =\begin{bmatrix}6&8\\3&4\end{bmatrix}.

Other matrix rules to recognize

  • Identity II: ones on the diagonal, zeros elsewhere; AI=IA=AAI=IA=A when sizes match.
  • Order matters: usually ABBAAB\ne BA.
  • Grouping works: (AB)C=A(BC)(AB)C=A(BC).
  • Distribution works: A(B+C)=AB+ACA(B+C)=AB+AC.
  • Transpose twice: (AT)T=A(A^\mathsf T)^\mathsf T=A.
  • Transpose a sum: (A+B)T=AT+BT(A+B)^\mathsf T=A^\mathsf T+B^\mathsf T.
  • Transpose a product: (AB)T=BTAT(AB)^\mathsf T=B^\mathsf TA^\mathsf T — reverse the order.

View the original practice sheet (includes solutions)
Module 3 and 4 Practice Problems - Solutions — original slide 1 of 3
Original practice sheet · Module 3 and 4 Practice Problems - SolutionsOpen PDF ↗
Slide 1 of 3

Given: calculate these three products. Rule: matrix multiplication.

(a)[1001][1234]\text{(a)}\quad \begin{bmatrix}1&0\\0&1\end{bmatrix} \begin{bmatrix}1&2\\3&4\end{bmatrix} (b)[9632]2(c)[1110]3\text{(b)}\quad \begin{bmatrix}9&6\\3&2\end{bmatrix}^{2} \qquad \text{(c)}\quad \begin{bmatrix}1&1\\1&0\end{bmatrix}^{3}

Hint: A2=AAA^2=AA and A3=(AA)AA^3=(AA)A; these powers do not square or cube entries individually.

Work through all three products

(a) Use row times column. The four entries are

1(1)+0(3)=1,1(2)+0(4)=2,0(1)+1(3)=3,0(2)+1(4)=4.\begin{aligned} 1(1)+0(3)&=1,\\ 1(2)+0(4)&=2,\\ 0(1)+1(3)&=3,\\ 0(2)+1(4)&=4. \end{aligned}

So the answer is [1234]\begin{bmatrix}1&2\\3&4\end{bmatrix}, as expected when multiplying by the identity.

(b) Use the rows and columns of the given matrix twice.

9(9)+6(3)=99,9(6)+6(2)=66,3(9)+2(3)=33,3(6)+2(2)=22.\begin{aligned} 9(9)+6(3)&=99,\\ 9(6)+6(2)&=66,\\ 3(9)+2(3)&=33,\\ 3(6)+2(2)&=22. \end{aligned}

Thus A2=[99663322]A^2=\begin{bmatrix}99&66\\33&22\end{bmatrix}.

(c) Call the given matrix FF. First multiply FF by itself:

F2=[1+11+01+01+0]=[2111].F^2=\begin{bmatrix}1+1&1+0\\1+0&1+0\end{bmatrix} =\begin{bmatrix}2&1\\1&1\end{bmatrix}.

Then multiply those calculated entries by the original FF:

(F3)11=2(1)+1(1)=3,(F3)12=2(1)+1(0)=2,(F3)21=1(1)+1(1)=2,(F3)22=1(1)+1(0)=1.\begin{aligned} (F^3)_{11}&=2(1)+1(1)=3,\\ (F^3)_{12}&=2(1)+1(0)=2,\\ (F^3)_{21}&=1(1)+1(1)=2,\\ (F^3)_{22}&=1(1)+1(0)=1. \end{aligned}

Thus F3=[3221]F^3=\begin{bmatrix}3&2\\2&1\end{bmatrix}.

Eigenvector = an arrow that stays on its line

Section titled “Eigenvector = an arrow that stays on its line”
Eigenvalues and Eigenvectors — original slide 3 of 4
Original slides · Eigenvalues and EigenvectorsOpen PDF ↗
Slide 3 of 4

A matrix can turn an arrow. An eigenvector is a nonzero arrow that the matrix only scales:

Mv=λv,v0.Mv=\lambda v,\qquad v\ne0.
  • MM: a square matrix.
  • vv: the eigenvector.
  • λ\lambda: the eigenvalue, or scaling factor.
  • Negative λ\lambda: reverse the arrow. Zero λ\lambda: collapse it to zero.

Check trick: multiply, then compare.

For invented M=[3002]M=\begin{bmatrix}3&0\\0&2\end{bmatrix} and v=[1,0]Tv=[1,0]^\mathsf T:

Mv=[3(1)+0(0)0(1)+2(0)]=[30]=3v.Mv=\begin{bmatrix}3(1)+0(0)\\0(1)+2(0)\end{bmatrix} =\begin{bmatrix}3\\0\end{bmatrix}=3v.

So vv is an eigenvector and λ=3\lambda=3.

Symmetric matrix: M=MTM=M^\mathsf T. A real symmetric n×nn\times n matrix has real eigenvalues and admits nn mutually perpendicular eigenvectors. Perpendicular means viTvj=0v_i^\mathsf Tv_j=0 for different chosen vectors; general real matrices can have complex eigenvalues/eigenvectors.

Invented self-check: with the same MM, is v=[1,1]Tv=[1,1]^\mathsf T an eigenvector? Rule.

Check your answer

Mv=[3,2]TMv=[3,2]^\mathsf T. A single multiplier cannot turn [1,1]T[1,1]^\mathsf T into [3,2]T[3,2]^\mathsf T: the first entry needs 3 and the second needs 2. So this vv is not an eigenvector.

Linear Regression — original slide 2 of 7
Original slides · Linear RegressionOpen PDF ↗
Slide 2 of 7

In a line, a coefficient tells you how much to add per input unit:

Y=β0+β1X+ε.Y=\beta_0+\beta_1X+\varepsilon.
  • XX: input; YY: observed output.
  • β1\beta_1: slope, the multiplier on XX.
  • β0\beta_0: intercept, the starting amount.
  • ε\varepsilon: noise, the part the rule does not explain.

The intercept is the prediction when inputs equal zero. We do not know nature’s coefficients; fitting data gives estimates, marked with hats:

y^i=β^0+β^1xi.\widehat y_i=\widehat\beta_0+\widehat\beta_1x_i.

ii identifies one observation. Uppercase X,YX,Y are random variables; lowercase xi,yix_i,y_i are recorded values. The prediction leaves out unknown noise.

Invented fitted rule: y^=10+3x\widehat y=10+3x. At x=4x=4:

  1. Input contribution: 3(4)=123(4)=12.
  2. Add baseline: 10+12=2210+12=22.

One extra xx unit adds 3; at x=0x=0, the prediction is 10.

With pp inputs, add one contribution per input:

Y=β0+j=1pβjXj+ε.Y=\beta_0+\sum_{j=1}^{p}\beta_jX_j+\varepsilon.

jj selects an input. βj\beta_j is its per-unit effect holding the other inputs fixed. pp excludes the intercept.

The data-generating story is “draw inputs, then an output given those inputs”:

fX,Y(x,y)=fYX(yx)fX(x).f_{X,Y}(x,y)=f_{Y\mid X}(y\mid x)f_X(x).

fXf_X is the input density; fYXf_{Y\mid X} is the output density given an input; fX,Yf_{X,Y} is their joint density. The linear rule and noise distribution describe that conditional output.

Least squares = smallest total squared miss

Section titled “Least squares = smallest total squared miss”
Linear Regression — original slide 6 of 7
Original slides · Linear RegressionOpen PDF ↗
Slide 6 of 7

Residual = observed − predicted. For observation ii:

ei=yiy^i,RSS=i=1nei2.e_i=y_i-\widehat y_i, \qquad \mathrm{RSS}=\sum_{i=1}^{n}e_i^2.

nn counts observations. RSS means residual sum of squares. Least squares chooses coefficients that make RSS smallest.

Why square? Invented misses +2+2 and 2-2 should both count: 22+(2)2=82^2+(-2)^2=8, rather than canceling.

Slope recipe = center, multiply, add, divide

Section titled “Slope recipe = center, multiply, add, divide”

For observed pairs (xi,yi)(x_i,y_i), first calculate the means:

xˉ=1ni=1nxi,yˉ=1ni=1nyi.\bar x=\frac1n\sum_{i=1}^n x_i, \qquad\bar y=\frac1n\sum_{i=1}^n y_i.

Then subtract those means — center the values — and build two totals:

Sxy=i=1n(xixˉ)(yiyˉ),S_{xy}=\sum_{i=1}^{n}(x_i-\bar x)(y_i-\bar y), Sxx=i=1n(xixˉ)2.S_{xx}=\sum_{i=1}^{n}(x_i-\bar x)^2.
  • SxyS_{xy}: how inputs and outputs move together.
  • SxxS_{xx}: how much the inputs spread out.

Divide to get the slope; use the means to position the line:

β^1=SxySxx,β^0=yˉβ^1xˉ.\widehat\beta_1=\frac{S_{xy}}{S_{xx}}, \qquad \widehat\beta_0=\bar y-\widehat\beta_1\bar x.

Require Sxx>0S_{xx}>0. Identical inputs cannot tell you the slope.

Memory cue: the intercept puts the line through its center, (xˉ,yˉ)(\bar x,\bar y).

Practice 2: why the line crosses the averages

Section titled “Practice 2: why the line crosses the averages”
View the original practice sheet (includes solutions)
Module 3 and 4 Practice Problems - Solutions — original slide 1 of 3
Original practice sheet · Module 3 and 4 Practice Problems - SolutionsOpen PDF ↗
Slide 1 of 3

Given: the least-squares formulas above. Show that the fitted line passes through (xˉ,yˉ)(\bar x,\bar y). Rule: simple regression.

Hint: substitute x=xˉx=\bar x into the prediction; replace the intercept with its formula.

Work through the cancellation

The prediction rule is y^(x)=β^0+β^1x\widehat y(x)=\widehat\beta_0+\widehat\beta_1x. At the average input, use x=xˉx=\bar x and β^0=yˉβ^1xˉ\widehat\beta_0=\bar y-\widehat\beta_1\bar x:

y^(xˉ)=(yˉβ^1xˉ)+β^1xˉ=yˉ.\begin{aligned} \widehat y(\bar x) &=(\bar y-\widehat\beta_1\bar x)+\widehat\beta_1\bar x\\ &=\bar y. \end{aligned}

The two slope terms cancel. The slope itself does not become zero. This property applies to ordinary least squares with an intercept.

Hypothesis Testing — original slide 7 of 9
Original slides · Hypothesis TestingOpen PDF ↗
Slide 7 of 9

Before fitting a line, predict the sample mean yˉ\bar y for everyone. Its total squared miss is total sum of squares:

TSS=i=1n(yiyˉ)2.\mathrm{TSS}=\sum_{i=1}^{n}(y_i-\bar y)^2.

RSS grades the fitted model. TSS grades the average-only baseline.

R2=1RSSTSS.R^2=1-\frac{\mathrm{RSS}}{\mathrm{TSS}}.

For example, R2=0.94R^2=0.94 means 94% less squared error than guessing the mean. It is not the percentage of individually correct predictions.

On training data, least squares with an intercept gives 0R210\le R^2\le1, provided TSS >0>0.

Residual standard error estimates noise size in output units.

RSE=RSSnp1.\mathrm{RSE}=\sqrt{\frac{\mathrm{RSS}}{n-p-1}}.
  • nn: observations.
  • pp: predictors, excluding the intercept.
  • np1n-p-1: observations minus fitted coefficients, called residual degrees of freedom.

Require n>p+1n>p+1 and linearly independent model columns: no column is an exact combination of others.

View the original practice sheet (includes solutions)
Module 3 and 4 Practice Problems - Solutions — original slide 1 of 3
Original practice sheet · Module 3 and 4 Practice Problems - SolutionsOpen PDF ↗
Slide 1 of 3

Given: Y=β0+β1X+εY=\beta_0+\beta_1X+\varepsilon with normal noise of mean zero and variance σ2\sigma^2. The three observed pairs are (1,2)(-1,-2), (0,0)(0,0), and (1,5)(1,5).

Find β^0\widehat\beta_0, β^1\widehat\beta_1, RSS, RSE, TSS, and R2R^2. Fitting formulas · Scoring formulas.

Hint: follow this order: averages → centered totals → coefficients → predictions → squared residuals → scores. Normality is not needed to do the least-squares arithmetic.

Work through the complete calculation

1. Calculate the averages from the three given observations.

xˉ=1+0+13=0,yˉ=2+0+53=1.\begin{aligned} \bar x&=\frac{-1+0+1}{3}=0,\\ \bar y&=\frac{-2+0+5}{3}=1. \end{aligned}

2. Calculate the slope using centered totals. With calculated means xˉ=0\bar x=0, yˉ=1\bar y=1:

Sxx=(1)2+02+12=1+0+1=2,Sxy=(1)(3)+(0)(1)+(1)(4)=3+0+4=7.\begin{aligned} S_{xx}&=(-1)^2+0^2+1^2\\ &=1+0+1=2,\\ S_{xy}&=(-1)(-3)+(0)(-1)+(1)(4)\\ &=3+0+4=7. \end{aligned}

The centered outputs 3,1,4-3,-1,4 come from subtracting yˉ=1\bar y=1 from the given outputs 2,0,5-2,0,5. Therefore

β^1=SxySxx=72=3.5.\widehat\beta_1=\frac{S_{xy}}{S_{xx}}=\frac72=3.5.

3. Calculate the intercept. Use yˉ=1\bar y=1, xˉ=0\bar x=0, and the calculated slope 3.53.5:

β^0=yˉβ^1xˉ=13.5(0)=1.\widehat\beta_0=\bar y-\widehat\beta_1\bar x =1-3.5(0)=1.

The fitted rule is y^=1+3.5x\widehat y=1+3.5x.

4. Predict, subtract, and square. Apply y^=1+3.5x\widehat y=1+3.5x at each given input; calculate e=yy^e=y-\widehat y:

xxyyy^\widehat yeee2e^2
1-12-21+3.5(1)=2.51+3.5(-1)=-2.50.50.50.250.25
00001+3.5(0)=11+3.5(0)=11-111
11551+3.5(1)=4.51+3.5(1)=4.50.50.50.250.25

Add the calculated squared residuals:

RSS=0.25+1+0.25=1.5.\mathrm{RSS}=0.25+1+0.25=1.5.

5. Estimate noise size. There are n=3n=3 observations and p=1p=1 predictor. Use calculated RSS =1.5=1.5:

RSE=1.5311=1.51.225.\mathrm{RSE}=\sqrt{\frac{1.5}{3-1-1}} =\sqrt{1.5}\approx1.225.

6. Calculate the mean-only baseline. Use given outputs 2,0,5-2,0,5 and calculated yˉ=1\bar y=1:

TSS=(21)2+(01)2+(51)2=9+1+16=26.\begin{aligned} \mathrm{TSS}&=(-2-1)^2+(0-1)^2+(5-1)^2\\ &=9+1+16=26. \end{aligned}

7. Calculate the relative improvement. With RSS =1.5=1.5 and TSS =26=26:

R2=11.5260.94230.94.R^2=1-\frac{1.5}{26} \approx0.9423\approx0.94.

Two quick checks: the calculated residuals sum to 0.51+0.5=00.5-1+0.5=0, and RSS =1.526==1.5\le26= TSS. Both hold for training-set least squares with an intercept.

Linear Regression — original slide 5 of 7
Original slides · Linear RegressionOpen PDF ↗
Slide 5 of 7

The design matrix MM stores the recipe’s inputs:

  • One row per observation.
  • One column per predictor.
  • An initial column of ones to add the intercept every time.

yy holds observed outputs; β\beta holds coefficients β0,,βp\beta_0,\ldots,\beta_p. Predictions are Mβ^M\widehat\beta.

β^=argminθyMθ2.\widehat\beta=\operatorname*{argmin}_{\theta} \lVert y-M\theta\rVert^2.

θ\theta is a trial coefficient vector. Argmin = coefficients giving the smallest error. The squared length r2=iri2\lVert r\rVert^2=\sum_i r_i^2 adds squared entries of residual vector rr.

With linearly independent columns, the reusable solution is

β^=(MTM)1MTy.\widehat\beta=(M^\mathsf TM)^{-1}M^\mathsf Ty.

An inverse undoes a matrix: A1A=IA^{-1}A=I. The displayed inverse exists when no design column is an exact combination of others.

Same Q3 data, using the matrix formula

Given pairs: (1,2),(0,0),(1,5)(-1,-2),(0,0),(1,5). Put ones beside their inputs:

M=[111011],y=[205].M=\begin{bmatrix}1&-1\\1&0\\1&1\end{bmatrix}, \qquad y=\begin{bmatrix}-2\\0\\5\end{bmatrix}.

Take column dot products to form MTMM^\mathsf TM:

  • Ones with ones: 12+12+12=31^2+1^2+1^2=3.
  • Ones with inputs: 1+0+1=0-1+0+1=0.
  • Inputs with inputs: (1)2+02+12=2(-1)^2+0^2+1^2=2.
MTM=[3002].M^\mathsf TM=\begin{bmatrix}3&0\\0&2\end{bmatrix}.

Multiply columns by outputs to form MTyM^\mathsf Ty:

MTy=[2+0+5(1)(2)+0(0)+1(5)]=[37].M^\mathsf Ty= \begin{bmatrix}-2+0+5\\(-1)(-2)+0(0)+1(5)\end{bmatrix} =\begin{bmatrix}3\\7\end{bmatrix}.

The diagonal matrix’s inverse has entries 1/3,1/21/3,1/2. Substitute into the fitting rule:

β^=[(1/3)(3)(1/2)(7)]=[13.5].\widehat\beta= \begin{bmatrix}(1/3)(3)\\(1/2)(7)\end{bmatrix} =\begin{bmatrix}1\\3.5\end{bmatrix}.

Same intercept, 1; same slope, 3.5.

Standard error = how much the estimate bounces

Section titled “Standard error = how much the estimate bounces”
Coefficient Uncertainty — original slide 3 of 6
Original slides · Coefficient UncertaintyOpen PDF ↗
Slide 3 of 6

Collect a fresh dataset and refit. Noise changes, so the estimated weights change.

  • True coefficient: fixed but unknown.
  • Coefficient estimate: varies between samples.
  • Standard error (SE): estimated standard deviation of those estimates.

Smaller SE = a more precise estimate.

Invented self-check: two fits estimate the same slope, 2. Fit A has SE 0.20.2; Fit B has SE 0.80.8. Which is more precise? Rule.

Check your answer

Fit A. Its relative SE is 0.2/0.8=1/40.2/0.8=1/4: its estimated repeated-sample spread is one quarter as large.

  • RSE: scale of output misses.
  • Coefficient SE: uncertainty in a fitted multiplier or intercept.

Assumptions for these uncertainty formulas: correct linear model; zero-mean noise given inputs; independent errors of common variance σ2\sigma^2; linearly independent design columns.

For one predictor, define input spread Sxx=i(xixˉ)2S_{xx}=\sum_i(x_i-\bar x)^2. If noise standard deviation σ\sigma is known:

SD(β^1)=σSxx,\operatorname{SD}(\widehat\beta_1)=\frac{\sigma}{\sqrt{S_{xx}}}, SD(β^0)=σ1n+xˉ2Sxx.\operatorname{SD}(\widehat\beta_0) =\sigma\sqrt{\frac1n+\frac{\bar x^2}{S_{xx}}}.

Here SD means standard deviation. If variance σ2\sigma^2 is supplied, take its square root to get σ\sigma.

Unknown noise? Use RSS → RSE → slope SE:

SE(β^1)=RSESxx.\operatorname{SE}(\widehat\beta_1) =\frac{\mathrm{RSE}}{\sqrt{S_{xx}}}.

Replace σ\sigma with RSE in the intercept formula too. Supplied SE? Use it directly.

More noise increases uncertainty. More input spread reduces slope uncertainty — widely separated points reveal the tilt more clearly.

Invented calculation: n=5n=5, p=1p=1, xˉ=0\bar x=0, Sxx=10S_{xx}=10, RSS =3=3.

  1. Estimate noise: RSE=3/(511)=1\mathrm{RSE}=\sqrt{3/(5-1-1)}=1.
  2. Slope SE: 1/100.3161/\sqrt{10}\approx0.316.
  3. Intercept SE: 11/5+02/100.4471\sqrt{1/5+0^2/10}\approx0.447.

Invented self-check: a fit reports RSE =8=8 and slope SE =0.3=0.3. Which number describes uncertainty in the slope? Rule.

Check your answer

0.30.3 describes estimated slope spread across repeated samples. The RSE of 8 describes estimated noise size in response units.

Unbiased = centered on the true coefficient

Section titled “Unbiased = centered on the true coefficient”

Under the assumptions above, conditional on the observed design MM:

E[β^M]=β,\mathbb E[\widehat\beta\mid M]=\beta, Cov(β^M)=σ2(MTM)1.\operatorname{Cov}(\widehat\beta\mid M) =\sigma^2(M^\mathsf TM)^{-1}.
  • E\mathbb E: average over repeated fits. Unbiased means this average hits the true coefficient.
  • Cov\operatorname{Cov}: covariance matrix, describing how estimates vary together.
  • Diagonal entries: coefficient variances. Take square roots for standard deviations.

Unbiased does not mean every estimate is close. SE tells you about spread around that center.

Confidence interval = estimate ± uncertainty margin

Section titled “Confidence interval = estimate ± uncertainty margin”
Confidence Intervals — original slide 3 of 6
Original slides · Confidence IntervalsOpen PDF ↗
Slide 3 of 6

An approximately normal estimate gives the quick approximately 95% interval:

β^j±2SE(β^j).\widehat\beta_j\pm2\operatorname{SE}(\widehat\beta_j).

jj selects the coefficient. Calculate margin = 2×SE2\times\mathrm{SE}, then subtract/add it.

Q5(c) supplied values: estimate 2, SE 1/21/2.

  1. Margin: 2(1/2)=12(1/2)=1.
  2. Lower endpoint: 21=12-1=1.
  3. Upper endpoint: 2+1=32+1=3.
  4. Interval: [1,3][1,3].

Meaning: about 95% of intervals made this way across repeated samples contain the fixed true coefficient. This particular interval is not guaranteed to contain it.

Approximation: 2 rounds the normal cutoff 1.96. With normal errors and estimated noise, exact regression intervals use a Student tt cutoff with np1n-p-1 degrees of freedom. The practice uses the stated ±2SE\pm2\mathrm{SE} rule.

Hypothesis Testing — original slide 3 of 9
Original slides · Hypothesis TestingOpen PDF ↗
Slide 3 of 9

A gap matters more when the estimate is precise. Measure the gap in standard-error units:

t=β^jbSE(β^j).t=\frac{\widehat\beta_j-b}{\operatorname{SE}(\widehat\beta_j)}.
  • β^j\widehat\beta_j: estimate.
  • bb: proposed true value.
  • SE: uncertainty in the estimate.
  • t|t|: distance in SEs; ignore direction for a two-sided test.

Approximate decision rule

  • t>2|t|>2: reject the proposed value.
  • t2|t|\le2: fail to reject; insufficient evidence against it.

Same trick, two views: the proposed value is over 2 SEs away exactly when it is outside the interval. Endpoints count as inside in these practice questions.

The formal hypotheses are H0:βj=bH_0:\beta_j=b (null) and Ha:βjbH_a:\beta_j\ne b (alternative). To test for a linear relationship, test slope b=0b=0.

A nonzero fitted slope can arise from noise even when the true slope is zero. Rejection supports an association under the model; it does not establish causation.

Significance α0.05\alpha\approx0.05: the long-run rate of rejecting a true null, under the assumptions. It is not the probability the null is true. Failing to reject does not prove it true.

Invented self-check: estimate 1.21.2, SE 0.40.4, proposed slope b=0b=0. Reject under the approximate rule? Rule.

Check your answer

Gap: 1.20=1.21.2-0=1.2. Divide by SE: t=1.2/0.4=3t=1.2/0.4=3. Since t>2|t|>2, reject. Interval check: margin 2(0.4)=0.82(0.4)=0.8, giving [1.20.8,1.2+0.8]=[0.4,2][1.2-0.8,1.2+0.8]=[0.4,2], which excludes 0.

View the original practice sheet (includes solutions)
Module 3 and 4 Practice Problems - Solutions — original slide 3 of 3
Original practice sheet · Module 3 and 4 Practice Problems - SolutionsOpen PDF ↗
Slide 3 of 3

Given: Y=β0+β1X+εY=\beta_0+\beta_1X+\varepsilon. Test the two separate claims H1:β1=1H_1:\beta_1=1 and H2:β1=2H_2:\beta_1=2 in each case below. Rule: confidence intervals and tests.

CaseGiven β^1\widehat\beta_1Given SE
(a)3-311
(b)1/21/222
(c)221/21/2

What this asks: could the true slope be 1? Could it be 2? Test each value separately. Here H1H_1 and H2H_2 are names for two separate claims, not a null/alternative pair.

Hint: two claims, two distances. Calculate (estimate − claim) ÷ SE separately for each claim. Reject only past 2 in magnitude. The interval provides a second check.

Work through the three decisions

(a) Estimate =3=-3, SE =1=1. Use the gap divided by SE for each claim:

b=1:t=311=4,b=2:t=321=5.\begin{aligned} b=1:\quad t&=\frac{-3-1}{1}=-4,\\ b=2:\quad t&=\frac{-3-2}{1}=-5. \end{aligned}

The claims are 4 and 5 SEs away. Both distances exceed 2, so reject H1H_1 and reject H2H_2. Check with the interval: its margin is 2(1)=22(1)=2,

[32,3+2]=[5,1].[-3-2,\,-3+2]=[-5,-1].

Both 1 and 2 are outside that range.

(b) Estimate =1/2=1/2, SE =2=2. Divide each gap by the supplied SE, 2:

b=1:t=1/212=0.25,b=2:t=1/222=0.75.\begin{aligned} b=1:\quad t&=\frac{1/2-1}{2}=-0.25,\\ b=2:\quad t&=\frac{1/2-2}{2}=-0.75. \end{aligned}

The claims are only 0.25 and 0.75 SEs away. Fail to reject either claim. The estimate is too uncertain to rule either out. The interval check uses margin 2(2)=42(2)=4:

[1/24,1/2+4]=[7/2,9/2].[1/2-4,\,1/2+4]=[-7/2,9/2].

Both 1 and 2 are inside. This does not establish that both values are true; it means the data do not distinguish them well enough.

(c) Estimate =2=2, SE =1/2=1/2. Divide each gap by the supplied SE, 1/21/2:

b=1:t=211/2=2,b=2:t=221/2=0.\begin{aligned} b=1:\quad t&=\frac{2-1}{1/2}=2,\\ b=2:\quad t&=\frac{2-2}{1/2}=0. \end{aligned}

The first claim is exactly 2 SEs away; the second matches our estimate. Fail to reject either under the stated rule, which rejects only distances greater than 2. The interval has margin 2(1/2)=12(1/2)=1:

[21,2+1]=[1,3].[2-1,\,2+1]=[1,3].

The value 1 is on the boundary and 2 is inside. The boundary decision depends on the chosen approximation.

What to do when a new problem changes the numbers

Section titled “What to do when a new problem changes the numbers”
  • Matrix: inner shapes match; outer shapes survive. Row times column.
  • Eigenvector: multiply; check for one common scaling factor.
  • Line: center → multiply → add → divide; intercept puts it through the averages.
  • Fit: RSS grades the line; TSS grades the mean; R² compares; RSE measures noise size.
  • Uncertainty: SE measures estimate spread; tt is gap divided by SE.

Next: Module 4 turns categories, interactions, and curves into inputs for the same weighted recipe.

Definition

Read the full glossary entry →

Module 3 formulas

Open the formula sheet →