lab_r.Rmd

---
title: "Intro to R"
subtitle: "Workshop on ggplot"
author: "`r paste0('<b>Lokesh Mano</b> • ',format(Sys.time(), '%d-%b-%Y'))`"
output:
  bookdown::html_document2:
          toc: true
          toc_float: true
          toc_depth: 4
          number_sections: true
          theme: flatly
          highlight: tango
          df_print: default
          code_folding: "none"
          self_contained: false
          keep_md: false
          encoding: 'UTF-8'
          css: "assets/lab.css"
---

```{r, include=FALSE}
hooks = knitr::knit_hooks$get()
hook_foldable = function(type) {
  force(type)
  function(x, options) {
    res = hooks[[type]](x, options)
    
    if (isFALSE(options[[paste0("fold.", type)]])) return(res)
    
    paste0(
      "<details><summary>", type, "</summary>\n\n",
      res,
      "\n\n</details>"
    )
  }
}
knitr::knit_hooks$set(
  output = hook_foldable("output"),
  plot = hook_foldable("plot")
)
```

```{r,child="assets/header-lab.Rmd"}
```

```{r,include=FALSE}
# data handling
library(dplyr)
library(tidyverse)
library(kableExtra)
#library(stringr)

# plotting
library(ggplot2)

#library(biomaRt) # annotation
#library(DESeq2) # rna-seq
#library(edgeR) # rna-seq
```

R is a programming language for statistical computing, and data wrangling. It is open-source, widely used in data science, has a wide range of functions and algorithms for graphing and data analyses.

We will use R in this course for generating plots from different biological data that are of higher quality and standard for publications. To do this, you will brush-up your memory on some important aspects of R that are important for this course below:

**Before starting with the lab sessions of the entire course, you must have downloaded all the necessary files from [here](lab_download.html) and make sure that the directory tree looks similar to the one displayed in that page**

In that case, you can proceed with the exercise now and remember to have fun :)

# Input/Output

Input and output of data and images is an important aspect with data analysis.

## Text

Data can come in a variety of formats which needs to be read into R and converted to an R data type.

Text files are the most commonly used input. Text files can be read in using the function `read.table`. We have a sample file to use: **iris.txt**.

```{r}
dfr <- read.table("data/metadata_raw.csv",sep = ";", header=TRUE,stringsAsFactors=F)
```

This reads in a tab-delimited text file with a header. The argument `sep='\t'` is set by default to specify that the delimiter is a tab. `stringsAsFactors=F` setting ensures that character columns are not automatically converted to factors.

It's always a good idea to check the data after import.

```{r}
head(dfr)
```

```{r}
str(dfr)
```

Check `?read.table` for other wrapper functions to read in text files.

Let's filter this data.frame and create a new data set.

```{r}
dfr1 <- dfr[dfr$Time == "t0",]
```

And we can write this as a text file.

```{r,eval=FALSE}
write.table(dfr1,"iris-setosa.txt",sep="\t",row.names=F,quote=F)
```

`sep="\t"` sets the delimiter to tab. `row.names=F` denotes that rownames should not be written. `quote=F` specifies that doubles must not be placed around strings.

# Data-Frames

You have probably learnt about `data.frame` in your previous course and this is the most important data structure for generating plots using `ggplot`. So, below you have some quick exercises on how to work with Data-Frames.

Vectors positions can be accessed using `[]`. R follows 1-based indexing, meaning that the indexing starts at 1.

Data-frame or matrix positions can be accessed using `[]` specifying row and column like `[row,column]`.

```{r}
dfr <- data.frame(x = 1:3, y = c("a", "b", "c"))
dfr
dfr[1,]
dfr[,1]
dfr[2,2]
```


The function `cbind()` is used to join two data-frames column-wise.

```{r}
dfr1 <- data.frame(x = 1:3, y = c("a", "b", "c"))
dfr2 <- data.frame(p = 4:6, q = c("d", "e", "f"))
dfr1
dfr2

cbind(dfr1,dfr2)
```

Similarly, `rbind()` is used to join two data-frames row-wise.

```{r}
dfr1 <- data.frame(x = 1:3, y = c("a", "b", "c"))
dfr2 <- data.frame(x = 4:6, y = c("d", "e", "f"))
dfr1
dfr2

rbind(dfr1,dfr2)
```

Two data-frames can be merged based on a shared column using the `merge()` function.

```{r}
dfr1 <- data.frame(x = 1:4, p = c("a", "b", "c","d"))
dfr2 <- data.frame(x = 3:6, q = c("l", "m", "n","o"))
dfr1
dfr2

merge(dfr1,dfr2,by="x")
merge(dfr1,dfr2,by="x",all.x=T)
merge(dfr1,dfr2,by="x",all.y=T)
merge(dfr1,dfr2,by="x",all=T)
```

# Data formats 

In terms of the R (and other similar programming languages), the data can be viewed or stored in two main formats! They are called `wide` and `long` formats. Below you will see what exactly they stand for and why is it important for plotting in ggplot.

## Wide format

A quick preview: 

**Counts Table**

```{r, echo=FALSE}
gc <- read.table("data/counts_raw.txt", header = T, row.names = 1, sep = "\t")
kable(gc[c(1:6),c(1:4)]) %>%
  kable_styling(bootstrap_options = "striped", full_width = F)
```

And we usually have our metadata related to the samples in another table like below:

**Metadata Table**

```{r, echo=FALSE}
md <- read.table("data/metadata_raw.csv", header = T, sep = ";")
kable(md[c(1:4),]) %>%
  kable_styling(bootstrap_options = "striped", full_width = F)
```

* Wide format data is called “wide” because it typically has a lot of columns which stretch widely across the page or your computer screen.
* Most of us are familiar with looking at wide format data
  + It is convenient and we are more used to looking at data this way in our Excel sheets. 
  + It often lets you see more of the data, at one time, on your screen

## Long format

Below is glimpse how the long format of the same data look like:

```{r echo=FALSE}
samples <- colnames(gc[,c(1:4)])
gc[c(1:6),c(1:4)] %>%
  rownames_to_column(var = "Gene") %>%
  gather(Samples, count, -Gene) %>% 
  head(10) %>%
  kable() %>%
  kable_styling(bootstrap_options = "striped", full_width = F)
```

Or to be even more complete and precise:

```{r echo=FALSE}
samples <- colnames(gc[,c(1:4)])
gc[c(1:6),c(1:4)] %>%
  rownames_to_column(var = "Gene") %>%
  gather(Sample_ID, count, -Gene) %>% 
  full_join(md[c(1:4),], by = "Sample_ID") %>% 
  select(Sample_ID, everything()) %>% 
  select(-c(Gene,count), c(Gene,count)) %>%
  head(10) %>%
  kable() %>%
  kable_styling("striped", full_width = F)
```

* Long format data is typically less familiar to most humans
  + It seems awfully hard to get a good look at all (or most) of it
  + It seems like it would require more storage on your hard disk
  + It seems like it would be harder to enter data in a long format

## Which is better?

* Well, there are some contexts where putting things in wide format is computationally efficient because you can treat data in a matrix format and to efficient matrix calculations on it.
* However, adding data to wide format data sets is very hard:
  1. It is very difficult to conceive of analytic schemes that apply generally across all wide-format data sets.
  2. Many tools in R want data in long format **like ggplot**
  3. The long format for data corresponds to the relational model for storing data, which is the model used in most modern data bases like the SQL family of data base systems.
  
* A more technical treatment of wide versus long data requires some terminology:
  - <span style="color:blue">Identifier variables</span> are often categorical things that cross-classify observations into categories.
  - <span style="color:red">Measured variables</span> are the names given to properties or characteristics that you can go out and measure.
  - <span style="color:orange">Values</span> are the values that you measure are record for any particular measured variable.

* In any particular data set, what you might want to call an <span style="color:blue">Identifier variables</span> versus a <span style="color:red">Measured variables</span> can not always be entirely clear.
  - Other people might choose to define things differently.
* However, to my mind, it is less important to be able to precisely recognize these three entities in every possible situation (where there might be some fuzziness about which is which)
* And it is more important to understand how <span style="color:blue">Identifier variables</span>, <span style="color:red">Measured variables</span>, and <span style="color:orange">Values</span> interact and behave when we are transforming data between wide and long formats.


# Conversion between formats

As for the biological data analysis, to be able to use tools such as **ggplot**, in simple terms we should learn to convert our data:

As per our previous examples: We should learn to convert 

**From this format**

```{r, echo=FALSE}
gc <- read.table("data/counts_raw.txt", header = T, row.names = 1, sep = "\t")
kable(gc[c(1:6),c(1:4)]) %>%
  kable_styling(bootstrap_options = "striped", full_width = F) %>%
  row_spec(1:6, color = "orange") %>%
  column_spec(1, color = "red") %>%
  row_spec(0, bold = T, color = "blue")
```

**To this format**

```{r echo=FALSE}
samples <- colnames(gc[,c(1:4)])
gc[c(1:6),c(1:4)] %>%
  rownames_to_column(var = "Gene") %>%
  gather(Sample_ID, count, -Gene) %>% 
  full_join(md[c(1:4),], by = "Sample_ID") %>% 
  select(Sample_ID, everything()) %>% 
  select(-c(Gene,count), c(Gene,count)) %>%
  head(10) %>%
  kable() %>%
  kable_styling("striped", full_width = F) %>%
  column_spec(1:5, color = "blue") %>%
  column_spec(6, color = "red")%>%
  column_spec(7, color = "orange")
```

Here we will only cover the conversion from `wide` to `long`, as this is more relevant to us. For the other way around, one can look into `spread()` from the `tidyr` package.

## Using reshape2

By using the `melt()` function from the **reshape2** package we can convert the wide-formatted data into long-formatted data! Here, to combine the metadata table to the gene counts table, we will also use the `merge()` function like we did before!

```{r}
library(reshape2)
gc <- read.table("data/counts_raw.txt", header = T, row.names = 1, sep = "\t")
md <- read.table("data/metadata_raw.csv", header = T, sep = ";")
rownames(md) <- md$Sample_ID

#merging gene counts table with metadata
merged_data_wide <- merge(md, t(gc), by = 0)
#removing redundant columns
merged_data_wide$Row.names <- NULL

merged_data_long <- melt(merged_data_wide, id.vars = c("Sample_ID","Sample_Name","Time","Replicate","Cell"), variable.name = "Gene", value.name = "count")  
head(merged_data_long)
```

## Using tidyr

If you are more familiar with using `tidyverse` or `tidyr` packages, you can combine tables by `join()` and then use `gather()` to make long formatted data in the same command. This is a powerful and more cleaner way of dealing with data in R.

```{r, warning=FALSE, message=FALSE}
library(tidyverse)
gc_long <- gc %>%
  rownames_to_column(var = "Gene") %>%
  gather(Sample_ID, count, -Gene) %>% 
  full_join(md, by = "Sample_ID") %>% 
  select(Sample_ID, everything()) %>% 
  select(-c(Gene,count), c(Gene,count)) 
gc_long %>%
  head(10)
```

# Exercise

<i class="fas fa-clipboard-list"></i> Task &nbsp; Here in this exercise we have used the `counts_raw.txt` file, you can try to make similar R objects for each of the other three counts (`counts_filtered.txt`, `counts_vst.txt` and `counts_deseq2.txt`) in `long format`. So for example you would have `gc_filt`, `gc_vst` and `gc_deseq2` R objects in the end.

<i class="fas fa-lightbulb"></i> Tip &nbsp; Remember to take a look at how these files are formatted, before you import!


# Acknowledgements

Much of the data format explanations and exercises were obtained from the GitHub of [Eric C. Anderson](https://github.com/eriqande) 

# Getting help

- Use `?function` to get function documentation
- Use `??bla` to search for a function
- Use `args(function)` to get the arguments to a function
- Go to the package CRAN page/webpage for vignettes
- [R Cookbook](http://www.cookbook-r.com/): General purpose reference.
- [Quick R](https://www.statmethods.net/): General purpose reference.
- [Stackoverflow](https://stackoverflow.com/): Online community to find solutions to your problems.

# Session info

```{r, fold.output=FALSE, fold.plot=FALSE}
sessionInfo()
```

__End of document__