Intro to prediction and Ordinary Least Squares

STAT 20: Introduction to Probability and Statistics

Warmup
10:00

For each of the following scenarios identify:

  • the target variable \(Y\) and its type
  • (time permitting, the features \(X_j\), their types, and how many they are (\(q\)))
  • whether it’s a regression or a classification task

Scenario A

  • Amazon sets the price for a specific book to optimize the sales, dependent on the book’s weight, genre, and whether it is hardback or paperback.

Careful!! This is a Causality question. “What would happen if I were to set the price of the book at some value? E.g. what are the expected sales when price=$ 30? Vs. price = $ 35? Etc.”

Scenario B

  • Ebay guesses the final sale price for a specific book, based on the initial bid price the seller has set, the book’s weight, genre, and whether it is hardback or paperback.

This is a straight-up Prediction task! It’s a Regression because sale price should be modeled as a continuous variable.

Scenario C

  • A Waymo engineer is working on a model to determine whether the self-driving car is seeing a real car or a reflection of a car.

This is a Prediction task. It’s a Classification task, because whether the car is real should be modeled as a discrete (Bernoulli) variable.

Scenario D

  • A marine biologist is trying to determine the nose-to-tail length of the typical manatee in the waters off the coast of Florida, broken down by sex and age groups.

You fell for it again! This is not a Prediction task. It’s a Generalization task, because they’re estimating various population parameters. These are hidden, non observable quantities.

Scenario E

  • The Weather Channel predicts tomorrow’s temperature based on historical data, using yesterday’s temperature and pressure, and tomorrow’s day of the year.

This is a Prediction task. It’s a Regression task, because temperature is modeled as a continuous variable.

Scenario F

  • The UCB Admissions office has to predict whether a student will accept its admission offer or not based on the student’s high school and their intended major.

This is a Prediction task. It’s a Classification task, because whether the student accepts the offer is should be modeled as a binary (Bernoulli) variable.

Scenario G

  • A philologist examines a 2nd century BCE papyrus from Egypt, full of holes, and written in Coptic. They make a guess about the missing letters in the holes, based on the surrounding words, the dating, the geographic origin of the papyrus.

This is a Prediction task! Althout this thing was written 2200 years ago, we are still “predicting” the value of an unseen random variable (the missing letter). It’s a Classification task, because a letter from the Coptic alphabet is a nominal categorical variable.

Which is better?

  • Large MSE or small MSE?

Small. MSE measures error (squared)!

Concept question
01:00

  • Imagine the following sample (\(n=3\)) with true values of \(Y\) and the predictions from some model \(\hat{Y}_{\mbox{model}}\). The units of \(Y\) are kilograms.

  • Calculate the sample MSE of the model!

Y Ŷ
1 2.3 2.8
2 −1.6 −1.2
3 4.3 3.6

\(\mbox{MSE} = \frac1{3} \left((-0.5 \, kg)^2 +(-0.4 \, kg)^2 +(0.7 \, kg)^2 \right) = 0.3 \, kg^2\)

Concept Question

  • Which of these two features \(X_j\) will yield the better prediction \(\hat{Y}\), based on the sample we collected and plotted below?

\(X_1\) on the left seems more strongly associated with \(Y\) than \(X_2\) is.

Concept Question

  • Here is a function f.
f <- function(x, y) {
  y*(x + 3) 
}

What will the following line of code return?

f(3,5)

Linear models

# A tibble: 4 × 5
  name      hours cuteness food_eaten is_indoor_cat
  <chr>     <dbl>    <dbl>      <dbl> <lgl>        
1 luca         10      8          218 FALSE        
2 jane         14     10          300 FALSE        
3 gargantua    16     10          220 TRUE         
4 behemot       8      8.5        195 FALSE        
m1 <- lm(formula = hours ~ cuteness + food_eaten + is_indoor_cat, 
         data = cats)
      (Intercept)          cuteness        food_eaten is_indoor_catTRUE 
     -28.89210392        6.01776698       -0.05395603       -3.24503100 

How many hours does the model predict Gargantua will sleep each day? Write out the linear equation of the model from the model output to help you. Hint: as an example, you can read 5e+01 as \(\, 5 \cdot 10^1\).

Break

05:00

Worksheet: Method of Least Squares

40:00