Lab: A Matter of Taste

STAT 20: Introduction to Probability and Statistics

Suggested timeline - [10 min] Slides and making teams - [45 min] Working on the protocol

General Advice - The students have seen nothing about experimental design yet in class. The idea is to have them use their own mental models to solve this problem, then later review and revise them. That’s to say: coach them only to be precise in their protocol, not to be using best practices in experimental design. Errors made here are great learning opportunities later. - The first phase of the lab can all be done with paper and pencil, so you can ask that students stow their laptops. - It might be helpful to guide students to start at the end, what they expect their data and plots to look like, then work back towards the hypotheses and claims. - As written, most null hypotheses will suggest the use of a difference in proportions as the test statistic.

Making Claims with Data

• So far: Pose question -> observe data -> analyze data -> make claim.

• e.g. Checking election results:
  1. Did fraud occur?
  2. Observe vote counts by city.
  3. Test of hypothesis that votes follow Benford’s law.
  4. Votes don’t follow Benford’s law well; but we wouldn’t expect them to in a fair election.

Making Claims with Data

• This week:
  1. Identify possible claim
  2. Imagine analysis that would show it.
  3. Plan data collection to allow that analysis.
  4. Carry out data collection + analysis.
  5. Check claim.

This week students will work backwards: start with a claim that they think may be true, then decide how to collect data in a manner that will be most effective at determining whether or not the claim is true.

But first

Please get into groups of 3 (or 4). Take turns introducing yourselves.

A Matter of Taste

Your challenge: Determine whether you can affect one your teammates’ perceptions of bubble water by manipulating their experience of tasting.

Each team will have access to

• 50 minutes (half of next class)
• 2 cans of soda water, each one from a different flavor
• small paper cups
• saltine crackers
• other materials welcome

Group Members

Write to the right of each role the first and last name of the group member filling that role (if there are groups of 4, two can serve as facilitators).

• Taster/Subject: Undergoes the experiment, tastes soda water, provides data.__________
• Materials: Handles the materials (water et. al) for the experiment.________________
• Facilitator(s): Carries out the experiment, keeps track of time, records data.__________

The main reason for the roles here is to dedicate just one person to handling the materials for health safety.

Question and Hypotheses

In one sentence, write the causal question you intend to test. It should be composed of a response variable having to do with perception of bubble water and whether it is affected by a given intervention / explanatory variable. Also identify your unit of observation.

Prompt them to think carefully about their wording here in the question. How ambitious and general do they want to be?

It is also very common for people to suggest non-causal questions. It will help to underline the idea of intervention on this slide.

Hypotheses

Write down the corresponding null hypothesis that you will gather evidence against, the alternative hypothesis, and the significance level, \(\alpha\), that you intend to use. Also identify your test statistic.

Protocol

Record below the step-by-step protocol that you will use to collect data to bear on the claim above. Be precise. You should be able to hand this off to another group and have them conduct your experiment.

Here they must be very precise. We’re spoken about reproducible science in class, so use that language to encourage careful work. They must be able to pass this protocol off to another group, and they should be able to carry out the intended experiment with no ambiguity.

Data

In the space below, sketch out an empty data frame that identifies the important characteristics of your data. Create the data frame such that the number of rows reflect your sample size, \(n\), and the number of columns represent the variables that you intend to collect. Label the columns with the names of those variables and note how you are defining your unit of observation.

This is intended to be a blank table with n rows and at least 1 column. They may plan to record either several extraneous variables or not enough to draw the conclusions they intend. As before, don’t correct them. They may discover mid-experiment that they did this part wrong, and that’s great.

Graphics

Sketch a plot that you will use to use to determine how consistent your data are with your original claim. Please label the axes. Since you do not yet have data, create two sketches of what the plot might look like in two scenarios. Below each plot, write the ggplot2 code that will create that graphic. Note: these are plots of data, not null distributions of statistics.

Scenario 1: Data supports \(H_0\)             Scenario 2: Data supports \(H_A\)

Encourage them to check that their viz is possible given their data frame and their data frame is possible give their protocol, and that their viz bears directly on their hypotheses.

Things to remember

• Be precise in your protocol.
• Be sure your claim corresponds to your protocol corresponds to your data corresponds to your plots.
• Important: Part I must be fully completed and shown to instructor/tutor before beginning part II.

50:00

STOP HERE ON DAY 1 OF THIS LAB!! The rest of the slides are to be covered once they have done the experiment on Day 2.

Part II

The Essential Question of Our Time

Which is the best flavor?

This purely for fun. Some people have strong opinions about this.

Please:

1. Conduct your experiment
2. Record your data
3. Complete and submit part II as a group

If you don’t finish in the allotted time, finish up after class.

60:00

Could your taster distinguish between flavors?

1. Yes, and it was statistically significant (p < .05).
   
2. Yes, but it was not statistically significant (p > .05).
   
3. Yes, but we haven’t conducted the test yet.
4. No.
   
5. Not sure, we haven’t computed the difference in proportions yet.
6. We tested a different claim.
   

The goal of this question is to allow all groups to concisely share their results with the class. It also opens up the conversation about how there were many different ways to carry out this experiment. Two groups might have come to different conclusions because their tasters really had different abilities, or because their claims and experiments were slightly different, or just due to random chance.

You may want to project 1 or 2 protocols and ask the groups to talk through how they uses the principles of experimental design.