Data Wrangling

Whipping your data into shape.

  • TODO

A closer look at the code

filter(msleep, vore == "carni")

Most of this should make sense to you. Filter is a function. The first argument is msleep, a dataframe. The last part is the weird one. Why to we write vore == "carni" to tell filter to give us the carnivores only?

Small but important: single equals = vs double equals ==

The code above uses two equals signs together: vore == "carni". Why not just one equals, vore = "carni"?

The reason is that single = and double == do fundamentally different things in R (and most coding languages):

  • A single = is a command. It says “make these things equal.” Some examples:
    • Variable assignment: x = 4 says “MAKE x equal to 4.” We are setting a variable x to the value 4. Though we usually do variable assignment with an arrow (x <- 4).
    • Named arguments to functions: when we write mean(numbers, na.rm = TRUE), the na.rm = TRUE part says “MAKE the na.rm argument equal to TRUE” (recall: this is how we remove missing values, NA, from a list before trying to average them).
  • A double == is a question. It asks, “are these things equal?” Some examples:
    • x == 2 asks “IS x equal to 2?” The answer will be TRUE or FALSE depending on x’s value.
    • If you have a vector, using == will ask the question of every element in the vector. See example code below
numbers <- c(1, 2, 3)
numbers == 2
[1] FALSE  TRUE FALSE

Since numbers is a vector of three different numbers, numbers == 2 will ask, three times, if a number is equal to 2. We’ll get three answers back in a logical vector: FALSE TRUE FALSE (only the second number in numbers is equal to 2).

Back to our code one more time:

filter(msleep, vore == "carni")

The vore == "carni" part asks, for each row, is the vore column equal to the text carni? We have 83 animals in our dataset, so we’ll get back a vector of 83 TRUE and FALSE answers. filter uses these TRUE and FALSE values to decide which rows to keep.

To demonstrate, consider the code below, which seems insane but totally works (remember, T and F are valid shorthand for TRUE and FALSE)

filter(msleep, c(T, F, F, F, F, F, T, F, T, F,
                 F, F, F, F, F, F, F, T, F, F,
                 F, F, F, F, F, F, F, T, F, F,
                 T, T, F, F, F, F, T, F, F, F,
                 F, F, F, F, T, F, T, F, F, F,
                 T, T, T, F, F, F, F, F, T, T,
                 F, F, F, F, F, F, F, F, F, F,
                 F, F, F, F, F, F, F, F, F, T,
                 T, T, T))
name genus vore order conservation sleep_total sleep_rem sleep_cycle awake brainwt bodywt
Cheetah Acinonyx carni Carnivora lc 12.1 NA NA 11.90 NA 50.000
Northern fur seal Callorhinus carni Carnivora vu 8.7 1.4 0.3833333 15.30 NA 20.490
Dog Canis carni Carnivora domesticated 10.1 2.9 0.3333333 13.90 0.0700 14.000
Long-nosed armadillo Dasypus carni Cingulata lc 17.4 3.1 0.3833333 6.60 0.0108 3.500
Domestic cat Felis carni Carnivora domesticated 12.5 3.2 0.4166667 11.50 0.0256 3.300
Pilot whale Globicephalus carni Cetacea cd 2.7 0.1 NA 21.35 NA 800.000
Gray seal Haliochoerus carni Carnivora lc 6.2 1.5 NA 17.80 0.3250 85.000
Thick-tailed opposum Lutreolina carni Didelphimorphia lc 19.4 6.6 NA 4.60 NA 0.370
Slow loris Nyctibeus carni Primates NA 11.0 NA NA 13.00 0.0125 1.400
Northern grasshopper mouse Onychomys carni Rodentia lc 14.5 NA NA 9.50 NA 0.028

The first value in the vector is TRUE. So we keep the first row (the Cheetah). The second values is FALSE, so we don’t keep that one (Owl Monkey). And so forth. We are left with just the carnivores.

Obviously hand-specifying which rows to keep and drop is very tedious, so we use expressions like vore == "carni". This little side question was just to show you what these expressions are doing: vore == "carni" produces a vector of TRUE and FALSE values, indicating to filter which rows to keep.

This specific set of notes contains references to many functions from the tidyverse library such as mutate(), select(), arrange(), summarize(). We delve more into some of these functions here. Each one of these are code cells are editable so after running them to see the output, play with them by modifying the code and understanding how these functions work – and how they break!

mutate()

This function allows you to create a new column in a dataframe. In typical tidyverse fashion, the first argument is a dataframe. The second argument names and defines how that new column is created. Above, we saw:

Here, the first argument, arbuthnot, is piped to mutate() and the second argument, total = boys + girls, creates a new column named total by adding together the columns boys and girls. You can use mutate() to create multiple columns at the same time:

Note that switching the order of the two new columns created above such that girl_proportion = girls / total comes before total = boys + girls will produce an error because total is used before it is created.

select()

This function is defined above as “selecting a subset of the columns of a data frame.” You’ve seen how to use select() to select or “grab” certain columns, but you can also use select() to omit certain columns. The last block of code can be rewritten to produce the same output by placing a minus sign, -, in front of the columns to omit:

arrange()

This function arranges the rows of a data frame according to some logical ordering of a column. This ordering is straightforward for numeric columns; the smallest numbers are placed first and ascend to the larger ones. That is, unless you use desc() (which stands for descending).

But what if you pass a column of characters to arrange()? Let’s take a look:

When arranged by species, Adelie penguins come first, followed by Chinstrap, then Gentoo. The penguins aren’t arranged in any specific order within a species, but we can change that by passing another column to arrange(). Passing additional columns to arrange() will systematically break ties. The below code arranges the data frame first by species (alphabetically) and then breaks ties by (ascending) bill length:

summarize()

This function summarizes a data frame into a single row. We can summarize a data frame by taking means or calculating the number of rows as above. We can also do other calculations like taking a median or calculating the variance of a column:

However, if summarize() is preceded by group_by(), then it will output multiple rows according to groups specified by group_by():

This syntax looks a lot like the syntax used for mutate()! Like in mutate(), we name and define new columns: new_column = formula. The difference is that summarize() returns a brand new data frame that does not contain the columns of the original data frame where mutate() returns a data frame with all columns of the original data frame in addition to the newly defined ones.