---
title: Using dittoSeq to visualize (sc)RNAseq data
author:
- name: Daniel Bunis
  affiliation: Bakar Computational Health Sciences Institute, University of California San Francisco, San
  email: daniel.bunis@ucsf.edu
date: "March 2nd, 2020"
output:
  BiocStyle::html_document:
    toc_float: true
package: dittoSeq
bibliography: ref.bib
vignette: >
  %\VignetteIndexEntry{Annotating scRNA-seq data}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}    
---

```{r, echo=FALSE, results="hide", message=FALSE}
knitr::opts_chunk$set(error=FALSE, message=FALSE, warning=FALSE,
    dev="jpeg", dpi = 72, fig.width = 4.5, fig.height = 3.5)
library(BiocStyle)
```

# Introduction

dittoSeq is a tool built to enable analysis and visualization of single-cell
and bulk RNA-sequencing data by novice, experienced, and color blind coders.
Thus, it provides many useful visualizations, which all utilize red-green
color blindness-optimized colors by default, and which allow sufficient
customizations, via discrete inputs, for out-of-the-box creation of
publication-ready figures.

For single-cell data, dittoSeq works directly with data pre-processed in other
popular packages (Seurat, scater, scran, ...). For bulk RNAseq data,
dittoSeq's import functions will convert bulk RNAseq data of various different
structures into a set structure that dittoSeq helper and visualization
functions can work with. So ultimately, dittoSeq includes universal plotting
and helper functions for working with (sc)RNAseq data processed and stored in
these formats:

Single-Cell:

- Seurat (versions 2 & 3)
- SingleCellExperiment

Bulk:

- SummarizedExperiment (the general Bioconductor Seq-data storage system)
- DESeqDataSet (DESeq2 package output)
- DGEList (edgeR package output)

For bulk data, or if your data is currently not analyzed, or simply not in one
of these structures, you can still pull it in to the SingleCellExperiment
structure that dittoSeq works with using the `importDittoBulk` function.

## Color blindness friendliness:

The default colors of this package are red-green color blindness friendly. To
make it so, I used the suggested colors from [@wong_points_2011] and adapted
them slightly by appending darker and lighter versions to create a 24 color
vector. All plotting functions use these colors, stored in `dittoColors()`, by
default.

Additionally:

- Shapes displayed in the legends are generally enlarged as this can be almost
as helpful as the actual color choice for colorblind individuals.
- When sensible, dittoSeq funcitons have a shape.by input for having groups
displayed through shapes rather than color. (But note: even as a red-green
color impaired individual myself writing this vignette, I recommend using color
and I generally only use shapes for showing additional groupings.)
- dittoDimPlots can be generated with letters overlaid (set do.letter = TRUE)
- The `Simulate` function allows a cone-typical individual to see what their
dittoSeq plots might look like to a colorblind individual.

## Disclaimer

Code used here for dataset processing and normalization should not be seen as
a suggestion of the proper methods for performing such steps. dittoSeq is a
visualization tool, and my focus while developing this vignette has been simply
creating values required for providing visualization examples.

# Installation

dittoSeq is available through Bioconductor.

```{r, eval=FALSE}
# Install BiocManager if needed
if (!requireNamespace("BiocManager", quietly = TRUE))
    install.packages("BiocManager")

# Install dittoSeq
BiocManager::install("dittoSeq")
```

## Some setup for this vignette that can be ignored

Here, we will need to do some prep as the dataset we will use from
@baron_single-cell_2016 is not normalized nor dimensionality reduced.

```{r}
library(dittoSeq)
library(scRNAseq)
library(SingleCellExperiment)
library(Seurat)
# Download data
sce <- BaronPancreasData()
# Trim to only 5 of the celltypes for simplicity of vignette
sce <- sce[,meta("label",sce) %in% c(
    "acinar", "endothelial", "gamma", "delta", "ductal")]
```

Now that we have a single-cell dataset loaded, we are ready to go.  All
functions work for either Seurat or SCE encapsulated single-cell data.

But to make full use of dittoSeq, we should reaally have this data
log-normalized, and dimensionality reductions and clustering run.

```{r}
# Make Seurat and grab metadata
seurat <- CreateSeuratObject(counts(sce))
seurat <- AddMetaData(seurat, sce$label, col.name = "celltype")
seurat <- AddMetaData(seurat, sce$donor, col.name = "Sample")
seurat <- AddMetaData(seurat,
                      PercentageFeatureSet(seurat, pattern = "^MT"),
                      col.name = "percent.mt")
# Basic Seurat workflow (possibly outdated, but fine for this vignette)
seurat <- NormalizeData(seurat, verbose = FALSE)
seurat <- FindVariableFeatures(object = seurat, verbose = FALSE)
seurat <- ScaleData(object = seurat, verbose = FALSE)
seurat <- RunPCA(object = seurat, verbose = FALSE)
seurat <- RunTSNE(object = seurat)
seurat <- FindNeighbors(object = seurat, verbose = FALSE)
seurat <- FindClusters(object = seurat, verbose = FALSE)
```

```{r}
# Grab PCA, TSNE, clustering, log-norm data, and metadata to sce

# sce <- as.SingleCellExperiment(seurat)
# At the time this vignette was made, the above function gave warnings

# So... manual method
sce <- addDimReduction(
  sce, embeddings = Embeddings(seurat, reduction = "pca"), name = "PCA")
sce <- addDimReduction(
  sce, embeddings = Embeddings(seurat, reduction = "tsne"), name = "TSNE")
sce$idents <- seurat$seurat_clusters
assay(sce, "logcounts") <- GetAssayData(seurat)
sce$percent.mt <- seurat$percent.mt
sce$celltype <- seurat$celltype
sce$Sample <- seurat$Sample
```

Now that we have a single-cell dataset loaded and analyzed in Seurat, let's
convert it to an SCE for examples purposes.

All functions will work the same for either the Seurat or SCE version.

# Getting started

## Single-cell RNAseq data

dittoSeq works directly with Seurat and SingleCellExperiment objects.  Nothing
special is needed. Just load in your data if it isn't already loaded, then go!

```{r}
dittoDimPlot(seurat, "Sample")
dittoPlot(seurat, "ENO1", group.by = "celltype")
dittoBarPlot(sce, "celltype", group.by = "Sample")
```

## Bulk RNAseq data

Bulk RNAseq data is handled by dittoSeq using the SingleCellExperiment
structure (as of version 0.99). This structure is essentially very similar
to the Bioconductor standard SummarizedExperiment, just with room added for
storing calculated dimensionality reductions.

```{r}
# First, lets make some mock expression and conditions data
exp <- matrix(rpois(20000, 5), ncol=20)
colnames(exp) <- paste0("sample", seq_len(ncol(exp)))
rownames(exp) <- paste0("gene", seq_len(nrow(exp)))
logexp <- logexp <- log2(exp + 1)

conditions <- factor(rep(1:4, 5))
sex <- c(rep("M", 9), rep("F", 11))
```

### Standard bulk data import workflow

Importing bulk data can be accomplished with just the `importDittoBulk()`
function, but metadata and dimensionality reductions can also be added after.

```{r}
# Import
myRNA <- importDittoBulk(
    # x can be a DGEList, a DESeqDataSet, a SummarizedExperiment,
    #   or a list of data matrices
    x = list(counts = exp,
             logcounts = logexp),
    # Optional inputs:
    #   For adding metadata
    metadata = data.frame(conditions = conditions,
                          sex = sex),
    #   For adding dimensionality reductions
    reductions = list(pca = matrix(rnorm(20000), nrow=20)))

# Add metadata (metadata can alternatively be added in this way)
myRNA$conditions <- conditions
myRNA$sex <- sex

# Add dimensionality reductions (can alternatively be added this way)
# (We aren't actually calculating PCA here.)
myRNA <- addDimReduction(
    object = myRNA,
    embeddings = matrix(rnorm(20000), nrow=20),
    name = "pca",
    key = "PC")
```

Additional details:

By default, provided metadata is added to (or replaces) any metadata already
inside of x, but the `combine_metadata` input can additionally be used to
turn retention of previous metadata slots off.

When providing expression data as a list of a single or multiple
matrices, it is recommended that matrices containing raw feature counts data
be named `counts`, and log-normalized counts data be named `logcounts`.

DGEList note: The import function attempts to pull in all information stored in
common DGEList slots (\$counts, \$samples, \$genes, \$AveLogCPM,
\$common.dispersion, \$trended.dispersion, \$tagwise.dispersion, and \$offset),
but any other slots are ignored.

```{r}
# Now making plots operates the exact same way as for single-cell data
dittoDimPlot(myRNA, "sex", size = 3, do.ellipse = TRUE)
dittoBarPlot(myRNA, "sex", group.by = "conditions")
dittoBoxPlot(myRNA, "gene1", group.by = "sex")
dittoHeatmap(myRNA, getGenes(myRNA)[1:10],
    annot.by = c("conditions", "sex"))
```

# Helper Functions

dittoSeq's helper functions make it easy to determine the metadata, gene, and
dimensionality reduction options for plotting.

## Metadata

```{r}
# Retrieve all metadata slot names
getMetas(seurat)
# Query for the presence of a metadata slot
isMeta("nCount_RNA", seurat)
# Retrieve metadata values:
meta("celltype", seurat)[1:10]
# Retrieve unique values of a metadata
metaLevels("celltype", seurat)
```

## Genes/Features

```{r}
# Retrieve all gene names
getGenes(seurat)[1:10]
# Query for the presence of a gene(s)
isGene("CD3E", seurat)
isGene(c("CD3E","ENO1","INS","non-gene"), seurat, return.values = TRUE)
# Retrieve gene expression values:
gene("ENO1", seurat)[1:10]
```

## Reductions

```{r}
# Retrieve all dimensionality reductions
getReductions(seurat)
```

These are what can be provided to `reduction.use` for `dittoDimPlot()`.

## Bulk versus single-cell

Because dittoSeq utilizes the SingleCellExperiment structure to handle bulk
RNAseq data, there is a getter and setter for the internal metadata which tells
dittoSeq functions which resolution of data a target SCE holds.

```{r}
# Getter
isBulk(sce)
isBulk(myRNA)

# Setter
mock_bulk <- setBulk(sce) # to bulk
mock_sc <- setBulk(myRNA, set = FALSE) # to single-cell
```

NOTE: for any non-SCE objects, `isBulk()` will always return FALSE

# Visualizations

There are many different types of dittoSeq visualizations. Each has intuitive
defaults which allow creation of immediately useable plots. Each also has many
additional tweaks avaiolable through discrete input that can help ensure you
can create publication-quality plots out-of-the-box.

## dittoDimPlot & dittoScatterPlot

These show cells/samples data overlayed on a scatter plot, with the axes of
`dittoScatterPlot()` being gene expression or metadata data and with the axes
of `dittoDimPlot()` being dimensionality reductions like tsne, pca, umap or
similar.

```{r, results = "hold"}
dittoDimPlot(seurat, "celltype")
dittoDimPlot(sce, "ENO1")
```

```{r, results = "hold"}
dittoScatterPlot(
    object = sce,
    x.var = "ENO1", y.var = "INS",
    color.var = "celltype", shape.by = "Sample",
    size = 3)
dittoScatterPlot(
    object = seurat,
    x.var = "nCount_RNA", y.var = "nFeature_RNA",
    color.var = "percent.mt",
    size = 3)
```

### Many additional dittoDimPlot features

dittoDimPlot has various additional features which can be overlayed on top.
Adding each is controlled by an input that starts with `add.` or `do.` such as
`do.label` or `add.trajectory.lineages`. Additional inputs that apply to these
features will then start with the XXXX part that comes after `add.XXXX` or
`do.XXXX`, as exemplified below.

```{r}
dittoDimPlot(seurat, "ident",
             do.label = TRUE, labels.repel = FALSE)
dittoDimPlot(seurat, "ident",
             add.trajectory.lineages = list(
                 c("3","4","11","8","2","10"),
                 c("3","9","6","12"),
                 c("3","9","7", "1"),
                 c("3","9","7","5")),
             trajectory.cluster.meta = "ident")
```

## dittoPlot (and dittoRidgePlot + dittoBoxPlot wrappers)

These display *continuous* cells/samples' data on a y-axis (or x-axis for
ridgeplots) grouped on the x-axis by sample, age, condition, or any discrete
grouping metadata. Data can be represented with violin plots, box plots,
individual points for each cell/sample, and/or ridge plots. The `plots` input
controls which data representations are used.  The `group.by` input controls
how the data are grouped in the x-axis.  And the `color.by` input controls the
color that fills in violin, box, and ridge plots.

`dittoPlot()` is the main function, but `dittoRidgePlot()` and
`dittoBoxPlot()` are wrappers which essentially just adjust the default for
the `plots` input from c("jitter", "vlnplot") to c("ridgeplot") or
c("boxplot","jitter"), respectively.

```{r, results = "hold"}
dittoPlot(seurat, "ENO1", group.by = "celltype",
    plots = c("vlnplot", "jitter"))
dittoRidgePlot(sce, "ENO1", group.by = "celltype")
dittoBoxPlot(seurat, "ENO1", group.by = "celltype")
```

Tweaks to the individual data representation types can be made with discrete
inputs, all of which start with the representation types' name.  For
example...

```{r}
dittoPlot(seurat, "ENO1", group.by = "celltype",
    plots = c("vlnplot", "jitter", "boxplot"),
    # change the color and size of jitter points
    jitter.color = "blue", jitter.size = 0.5,
    # change the outline color and width, and remove the fill of boxplots
    boxplot.color = "white", boxplot.width = 0.1,
    boxplot.fill = FALSE,
    # change how the violinplot widths are normalized across groups
    vlnplot.scaling = "count"
    )
```

## dittoBarPlot

This function displays *discrete* cells/samples' data on a y-axis, grouped on
the x-axis by sample, age, condition, or any discrete grouping metadata. Data
can be represented as percentages or counts, and this is controlled by the
`scale` input.

```{r, results = "hold"}
dittoBarPlot(seurat, "celltype", group.by = "Sample")
dittoBarPlot(seurat, "ident", group.by = "Sample",
    scale = "count")
```

## dittoHeatmap

This function is essentially a wrapper for generating heatmaps with pheatmap,
but with the same automatic, user-friendly, data extraction, (subsetting,) and
metadata integrations common to other dittoSeq functions.

For large, many cell, single-cell datasets, it can be necessary to turn off
clustering by cells in generating the heatmap because the process is very
memory intensive. As an alternative, dittoHeatmap offers the ability to order
columns in functional ways using the `order.by` input. This input will default
to the first annotation provided to `annot.by` for single cell datasets, but
can also be controlled separately.

```{r, results = "hold"}
# Pick Genes
genes <- c("SST", "REG1A", "PPY", "INS", "CELA3A", "PRSS2", "CTRB1",
    "CPA1", "CTRB2" , "REG3A", "REG1B", "PRSS1", "GCG", "CPB1",
    "SPINK1", "CELA3B", "CLPS", "OLFM4", "ACTG1", "FTL")

# Annotating and ordering cells by some meaningful feature(s):
dittoHeatmap(seurat, genes,
    annot.by = c("celltype", "Sample"))
dittoHeatmap(seurat, genes,
    annot.by = c("celltype", "Sample"),
    order.by = "Sample")
```

`scaled.to.max = TRUE` will normalize all expression data to the max expression
of each gene [0,1], which is often useful for zero-enriched single-cell data.

`show_colnames`/`show_rownames` control whether cell/gene names will be
shown. (`show.colnames` default is TRUE for bulk, and FALSE for single-cell.)

```{r}
# Add annotations
dittoHeatmap(seurat, genes,
    annot.by = c("celltype", "Sample"),
    scaled.to.max = TRUE,
    show_colnames = FALSE,
    show_rownames = FALSE)
```

A subset of the supplied genes can be given to the `highlight.genes` input to
have names shown for just these genes.

```{r}
# Highlight certain genes
dittoHeatmap(seurat, genes,
    annot.by = c("celltype", "Sample"),
    highlight.genes = genes[1:3])
```

Additional tweaks can be added through other built in inputs or by providing
additional inputs that get passed along to pheatmap (see `?pheatmap`).

## Multi-Plotters

These create either multiple plots or create plots that summarize data for
multiple variables all in one plot.  They make it easier to create sumarzies
for many genes or many celltypes without the need for writing loops.

Some setup for these, let's roughly pick out the markers of delta cells in
this dataset

```{r}
# Idents(seurat) <- "celltype"
# delta.marker.table <- FindMarkers(seurat, ident.1 = "delta")
# delta.genes <- rownames(delta.marker.table)[1:20]
# Idents(seurat) <- "seurat_clusters"

delta.genes <- c("SST", "RBP4", "PCSK1", "CPE", "GPX3",
    "NLRP1", "PPP1R1A", "PCP4", "CHGB", "DHRS2", "LEPR", 
    "PTPRN", "BEX1", "SCGN", "PCSK1N", "SCG5", "UCHL1",
    "CHGA", "GAD2", "SEC11C")
```

### multi_dittoPlot & dittoPlot_VarsAcrossGroups

`multi_dittoPlot()` creates dittoPlots for multiple genes or metadata, one
plot each.

`dittoPlotVarsAcrossGroups()` creates a dittoPlot-like representation where
instead of representing samples/cells as in typical dittoPlots, each data
point instead represents the average expression, across each x-grouping, of a
gene (or value of a metadata).

```{r}
multi_dittoPlot(seurat, delta.genes[1:6], group.by = "celltype",
    vlnplot.lineweight = 0.2, jitter.size = 0.3)
dittoPlotVarsAcrossGroups(seurat, delta.genes, group.by = "celltype",
    main = "Delta-cell Markers")
```

### multi_dittoDimPlot & multi_dittoDimPlotVaryCells

`multi_dittoDimPlot()` creates dittoDimPlots for multiple genes or metadata,
one plot each.

`multi_dittoDimPlotVaryCells()` creates dittoDimPlots for a single gene or
metadata, but where distinct cells are highlighted in each plot. The
`vary.cells.meta` input sets the discrete metadata to be used for breaking up
cells/samples over distinct plots. This can be useful for
checking/highlighting when a gene may be differentially expressed within
multiple cell types or accross all samples.

- The output of `multi_dittoDimPlotVaryCells()` is similar to that of
faceting using dittoDimPlot's `split.by` input, but with added capability of
showing an "AllCells" plot as well, or of outputing the individual plots for
making manually customized plot arrangements when `data.out = TRUE`.

```{r, results = "hold"}
multi_dittoDimPlot(seurat, delta.genes[1:6])
multi_dittoDimPlotVaryCells(seurat, delta.genes[1],
    vary.cells.meta = "celltype")
multi_dittoDimPlotVaryCells(seurat, "celltype",
    vary.cells.meta = "celltype")
```

# Customization via Simple Inputs 

**Many adjustments can be made with simple additional inputs**.  Here, we go
through a few that are consistent across most dittoSeq functions, but there
are many more.  Be sure to check the function documentation (e.g.
`?dittoDimPlot`) to explore more!

## Subsetting to certain cells/samples

The cells/samples shown in a given plot can be adjusted with the `cells.use`
input. This can be provided as either a list of cells' / samples' names to
include, or as a logical vector that states whether each cell / sample should
be included.

```{r}
# Original
dittoBarPlot(seurat, "celltype", group.by = "Sample", scale = "count")

# String method, first 10 cells
dittoBarPlot(seurat, "celltype", group.by = "Sample", scale = "count",
    cells.use = colnames(seurat)[1:10])

# Logical method, only acinar cells
dittoBarPlot(seurat, "celltype", group.by = "Sample", scale = "count",
    cells.use = meta("celltype", seurat) == "acinar")
```

## Faceting with split.by

dittoPlot, dittoDimPlot, and dittoScatterPlots can be split into separate
plots for distinct groups of cells with the `split.by` input.

```{r}
dittoDimPlot(seurat, "celltype", split.by = "Sample")
dittoDimPlot(seurat, "ENO1", split.by = c("Sample", "celltype"))
```

## All titles are adjustable.

Relevant inputs are generally `main`, `sub`, `xlab`, `ylab`, `x.labels`, and
`legend.title`.

```{r}
dittoBarPlot(seurat, "celltype", group.by = "Sample",
    main = "Encounters",
    sub = "By Type",
    xlab = NULL, # NULL = remove
    ylab = "Generation 1",
    x.labels = c("Ash", "Misty", "Jessie", "James"),
    legend.title = "Types",
    var.labels.rename = c("Fire", "Water", "Grass", "Electric", "Psychic"),
    x.labels.rotate = FALSE)
```

As exemplified above, in some functions, the displayed data can be renamed too.

## Colors can be adjusted easily.

Colors are normally set with `color.panel` or `max.color` and `min.color`.
When color.panel is used (discrete data), an additional input called `colors`
sets the order in which those are actually used to make swapping around colors
easy when nearby clusters appear too similar in tSNE/umap plots!

```{r, results="hold"}
# original - discrete
dittoDimPlot(seurat, "celltype")
# swapped colors
dittoDimPlot(seurat, "celltype",
    colors = 5:1)
# different colors
dittoDimPlot(seurat, "celltype",
    color.panel = c("red", "orange", "purple", "yellow", "skyblue"))
```

```{r, results="hold"}
# original - expression
dittoDimPlot(seurat, "INS")
# different colors
dittoDimPlot(seurat, "INS",
    max.color = "red", min.color = "gray90")
```

## Underlying data can be output.

Simply add  `data.out = TRUE` to any of the individual plotters and a
representation of the underlying data will be output.

```{r}
dittoBarPlot(seurat, "celltype", group.by = "Sample",
    data.out = TRUE)
```

For dittoHeatmap, a list of all the arguments that would be supplied to
pheatmap are output.  This allows users to make their own tweaks to how the
expression matrix is represented before plotting, or even to use a different
heatmap creator from pheatmap altogether.

```{r}
dittoHeatmap(seurat, c("SST","CPE","GPX3"), cells.use = colnames(seurat)[1:5],
    data.out = TRUE)
```

## plotly hovering can be added.

Any dittoSeq function that normally outputs a ggplot (dittoDimPlot, dittoPlot,
dittoBarPlot, dittoPlotVarsAcrossGroups) can be supplied `do.hover = TRUE` to
have it be converted into a plotly object that will display additional data
about each data point when the user hovers their cursor on top.

Generally, a second input, `hover.data`, is used to tell dittoSeq qhat extra
data to display.  This input takes in a vector of gene or metadata names (or
"ident" for seurat object clustering) in the order you wish for them to be
displayed.

```{r, eval = FALSE}
# These can be finicky to render in knitting, but still, example code:
dittoDimPlot(seurat, "INS",
    do.hover = TRUE,
    hover.data = c("celltype", "Sample", "ENO1", "ident", "nCount_RNA"))
dittoPlot(seurat, "INS", group.by = "celltype", plots = c("vlnplot", "jitter"),
    do.hover = TRUE,
    hover.data = c("celltype", "Sample", "ENO1", "ident", "nCount_RNA"))
```

When the types of underlying data possible to be shown are constrained because
the plot pieces represent summary data (dittoBarPlot and
dittoPlotVarsAcrossGroups), just `do.hover` is enough: 

```{r, eval = FALSE}
# These can be finicky to render in knitting, but still, example code:
dittoBarPlot(seurat, "celltype", group.by = "Sample",
    do.hover = TRUE)
dittoPlotVarsAcrossGroups(seurat, delta.genes, group.by = "celltype",
    do.hover = TRUE)
```

# Session information

```{r}
sessionInfo()
```

# References
