# Handling Expression Matrices with ImmuneSpaceR

#### Renan Sauteraud

#### 2018-08-30

This vignette shows detailed examples for the `getGEMatrix()` method

# 1 Create connections

As explained into the introductory vignette, datasets must be downloaded from `ImmuneSpaceConnection` objects. We must first instantiate a connection to the study or studies of interest. Throughout this vignette, we will use two connections, one to a single study, and one to to all available data.

    library(ImmuneSpaceR)
    sdy269 <- CreateConnection("SDY269")
    all <- CreateConnection("")

# 2 List the expression matrices

Now that the connections have been instantiated, we can start downloading from them. But we need to figure out which processed matrices are available within our chosen studies.

On the ImmuneSpace portal, in the study of interest or at the project level, the Gene expression matrices table will show the available runs.

Printing the connections will, among other information, list the datasets availables. The `listDatasets` method will only display the downloadable data. looking for. With `output = "expression"`, the datasets wont be printed.

    sdy269$listDatasets()

    ## datasets
    ##  demographics
    ##  elispot
    ##  hai
    ##  fcs_analyzed_result
    ##  cohort_membership
    ##  pcr
    ##  elisa
    ##  gene_expression_files
    ##  fcs_sample_files
    ## Expression Matrices
    ##  TIV_2008
    ##  LAIV_2008

Using `output = "expression"`, we can remove the datasets from the output.

    all$listDatasets(output = "expression")

    ## Expression Matrices
    ##  SDY300_dendriticCell_dcMonoFlu2011
    ##  SDY300_otherCell_dcMonoFlu2011
    ##  SDY67_HealthyAdults
    ##  SDY67_Batch2_HealthyAdult
    ##  SDY1328_healthy_adults
    ##  SDY1324_nonBCG_Vaccinated
    ##  SDY1324_Latent_TB
    ##  SDY1324_BCG_Vaccinated
    ##  SDY1293_Vaccinated
    ##  SDY1293_Infectivity_Control
    ##  SDY1289_Montreal_adult
    ##  SDY1289_Lausanne_cohort
    ##  SDY1289_Emory_adult
    ##  SDY1276_Validation_Cohort
    ##  SDY1276_Discovery
    ##  SDY1264_Trial_2
    ##  SDY1264_Trial_1
    ##  SDY1260_MPSV4_Cohort
    ##  SDY1260_MCV4_Cohort
    ##  SDY984_Young_cohort
    ##  SDY984_elderly_cohort
    ##  SDY888_denvNeg_endemicAreaTrue
    ##  SDY888_DenvNeg_endemicAreaFalse
    ##  SDY888_DenvPos_endemicAreaTrue
    ##  SDY690_WholeBlood_EnergixB
    ##  SDY690_PBMC_EnergixB
    ##  SDY667_HIPC_PSOR_PPP
    ##  SDY597_Cellsubs_Invitro
    ##  SDY387_NCH
    ##  SDY372_JDM_2012_2013
    ##  SDY368_NCH_2013_2014
    ##  SDY364_NCH_2012_2013
    ##  SDY315_GroupC
    ##  SDY315_GroupB
    ##  SDY315_GroupA
    ##  SDY312_GroupC_Cohort
    ##  SDY312_GroupB_Cohort
    ##  SDY312_GroupA_Cohort
    ##  SDY305_TIVGroup
    ##  SDY305_IDTIVGroup
    ##  SDY299_HEPISLAV_Cohort
    ##  SDY112_GroupC
    ##  SDY112_GroupB
    ##  SDY112_GroupA
    ##  SDY34_InfluenzaVaccine
    ##  SDY34_HealthyControls
    ##  SDY28_Dryvax_Recipients
    ##  SDY180_group2_saline
    ##  SDY180_group2_pneunomax
    ##  SDY180_group2_fluzone
    ##  SDY180_group1_saline
    ##  SDY180_group1_pneunomax
    ##  SDY180_group1_fluzone
    ##  SDY301_AIRFV
    ##  SDY296_AIRFV
    ##  SDY162_Macrophage_VLplus
    ##  SDY162_PBMC_VLplus
    ##  SDY162_Macrophage_VLminus
    ##  SDY162_PBMC_VLminus
    ##  SDY522_LAIV_Vaccine
    ##  SDY212_Cohort2_Older
    ##  SDY212_Cohort1_Young
    ##  SDY400_PBMC_Young
    ##  SDY400_PBMC_Older
    ##  SDY404_PBMC_Young
    ##  SDY404_PBMC_Older
    ##  SDY63_PBMC_Older
    ##  SDY63_PBMC_Young
    ##  SDY144_TIV2011
    ##  TIV_older
    ##  TIV_young
    ##  pH1N1_2009
    ##  TIV_2010
    ##  TIV_2007
    ##  TIV_2008
    ##  LAIV_2008

Naturally, `all` contains every processed matrices available on ImmuneSpace as it combines all available studies.

# 3 Download

## 3.1 By run name

The `getGEMatrix` method will accept any of the run names listed in the connection.

    TIV_2008 <- sdy269$getGEMatrix("TIV_2008")

    ## Downloading matrix..

    ## Constructing ExpressionSet

    TIV_2011 <- all$getGEMatrix(matrixName = "SDY144_TIV2011")

    ## Downloading matrix..
    ## Constructing ExpressionSet

The matrices are returned as `ExpressionSet` where the phenoData slot contains basic demographic information and the featureData slot shows a mapping of probe to official gene symbols.

    TIV_2008

    ## ExpressionSet (storageMode: lockedEnvironment)
    ## assayData: 16442 features, 80 samples
    ##   element names: exprs
    ## protocolData: none
    ## phenoData
    ##   sampleNames: BS586205 BS586250 ... BS586207 (80 total)
    ##   varLabels: study_time_collected study_time_collected_unit ...
    ##     biosample_accession (5 total)
    ##   varMetadata: labelDescription
    ## featureData
    ##   featureNames: DDR1 RFC2 ... NUS1P3 (16442 total)
    ##   fvarLabels: FeatureId gene_symbol
    ##   fvarMetadata: labelDescription
    ## experimentData: use 'experimentData(object)'
    ## Annotation:

## 3.2 By cohort

The `cohort` argument can be used in place of the run name (`x`). Likewise, the list of valid cohorts can be found in the Gene expression matrices table.

    LAIV_2008 <- sdy269$getGEMatrix(cohort = "LAIV group 2008")

    ## Downloading matrix..

    ## Constructing ExpressionSet

Note that when cohort is used, `x` is ignored.

# 4 Summarized matrices

By default, the returned `ExpressionSet`s have probe names as features (or rows). However, multiple probes often match the same gene and merging experiments from different arrays is impossible at feature level. When they are available, the `summary` argument allows to return the matrices with gene symbols instead of probes. You should use `currAnno` set to `TRUE` to use the latest official gene symbols mapped for each probe, but you can also set this to `FALSE` to retrieve the original mappings from when the matrix was created.

    TIV_2008_sum <- sdy269$getGEMatrix("TIV_2008", outputType = "summary", annotation = "latest")

    ## returning TIV_2008_sum_eset from cache

Probes that do not map to a unique gene are removed and expression is averaged by gene.

    TIV_2008_sum

    ## ExpressionSet (storageMode: lockedEnvironment)
    ## assayData: 16442 features, 80 samples
    ##   element names: exprs
    ## protocolData: none
    ## phenoData
    ##   sampleNames: BS586205 BS586250 ... BS586207 (80 total)
    ##   varLabels: study_time_collected study_time_collected_unit ...
    ##     biosample_accession (5 total)
    ##   varMetadata: labelDescription
    ## featureData
    ##   featureNames: DDR1 RFC2 ... NUS1P3 (16442 total)
    ##   fvarLabels: FeatureId gene_symbol
    ##   fvarMetadata: labelDescription
    ## experimentData: use 'experimentData(object)'
    ## Annotation:

# 5 Combining matrices

In order to faciliate analysis across experiments and studies, when multiple runs or cohorts are specified, `getGEMatrix` will attempt to combine the selected expression matrices into a single `ExpressionSet`.

To avoid returning an empty object, it is usually recommended to use the summarized version of the matrices, thus combining by genes. This is almost always necessary when combining data from multiple studies.

    # Within a study
    em269 <- sdy269$getGEMatrix(c("TIV_2008", "LAIV_2008"))

    ## returning summary matrix from cache
    ## returning summary matrix from cache

    ## returning latest annotation from cache
    ## returning latest annotation from cache

    ## Constructing ExpressionSet
    ## Constructing ExpressionSet

    ## Combining ExpressionSets

    # Combining across studies
    TIV_seasons <- all$getGEMatrix(c("TIV_2008", "SDY144_TIV2011"),
                                   outputType = "summary",
                                   annotation = "latest")

    ## Downloading matrix..

    ## returning summary matrix from cache

    ## returning latest annotation from cache

    ## Constructing ExpressionSet
    ## Constructing ExpressionSet

    ## Combining ExpressionSets

# 6 Caching

As explained in the introductory, the `ImmuneSpaceConnection` class is a `R6` class. It means its objects have fields accessed by reference. As a consequence, they can be modified without making a copy of the entire object. ImmuneSpaceR uses this feature to store downloaded datasets and expression matrices. Subsequent calls to `getGEMatrix` with the same input will be faster.

See `?R6::R6Class` for more information about R6 class system.

We can see a list of already downloaded runs and feature sets the `cache` field. This is not intended to be used for data manipulation and only displayed here to explain what gets cached.

    names(sdy269$cache)

    ## [1] "GE_matrices"        "TIV_2008_sum"       "featureset_18"
    ## [4] "TIV_2008_sum_eset"  "LAIV_2008_sum"      "LAIV_2008_sum_eset"

If, for any reason, a specific marix needs to be redownloaded, the `reload` argument will clear the cache for that specific `getGEMatrix` call and download the file and metadata again.

    TIV_2008 <- sdy269$getGEMatrix("TIV_2008", reload = TRUE)

    ## returning summary matrix from cache

    ## returning latest annotation from cache

    ## Constructing ExpressionSet

Finally, it is possible to clear every cached expression matrix (and dataset).

    sdy269$clearCache()

Again, the `cache` field should never be modified manually. When in doubt, simply reload the expression matrix.

# 7 Session info

    sessionInfo()

    ## R version 3.5.1 Patched (2018-07-24 r75005)
    ## Platform: x86_64-w64-mingw32/x64 (64-bit)
    ## Running under: Windows Server 2012 R2 x64 (build 9600)
    ##
    ## Matrix products: default
    ##
    ## locale:
    ## [1] LC_COLLATE=C
    ## [2] LC_CTYPE=English_United States.1252
    ## [3] LC_MONETARY=English_United States.1252
    ## [4] LC_NUMERIC=C
    ## [5] LC_TIME=English_United States.1252
    ##
    ## attached base packages:
    ## [1] stats     graphics  grDevices utils     datasets  methods   base
    ##
    ## other attached packages:
    ## [1] Rlabkey_2.2.2      jsonlite_1.5       httr_1.3.1
    ## [4] ImmuneSpaceR_1.8.2 rmarkdown_1.10     knitr_1.20
    ##
    ## loaded via a namespace (and not attached):
    ##  [1] Biobase_2.40.0        viridis_0.5.1         tidyr_0.8.1
    ##  [4] viridisLite_0.3.0     foreach_1.4.4         gtools_3.8.1
    ##  [7] assertthat_0.2.0      stats4_3.5.1          yaml_2.2.0
    ## [10] robustbase_0.93-2     pillar_1.3.0          backports_1.1.2
    ## [13] lattice_0.20-35       glue_1.3.0            digest_0.6.16
    ## [16] RColorBrewer_1.1-2    colorspace_1.3-2      preprocessCore_1.42.0
    ## [19] htmltools_0.3.6       plyr_1.8.4            pkgconfig_2.0.2
    ## [22] pheatmap_1.0.10       purrr_0.2.5           mvtnorm_1.0-8
    ## [25] scales_1.0.0          webshot_0.5.0         gdata_2.18.0
    ## [28] whisker_0.3-2         tibble_1.4.2          ggplot2_3.0.0
    ## [31] nnet_7.3-12           BiocGenerics_0.26.0   lazyeval_0.2.1
    ## [34] magrittr_1.5          crayon_1.3.4          mclust_5.4.1
    ## [37] heatmaply_0.15.2      evaluate_0.11         MASS_7.3-50
    ## [40] gplots_3.0.1          class_7.3-14          registry_0.5
    ## [43] tools_3.5.1           data.table_1.11.4     trimcluster_0.1-2.1
    ## [46] stringr_1.3.1         plotly_4.8.0          kernlab_0.9-27
    ## [49] munsell_0.5.0         cluster_2.0.7-1       fpc_2.1-11.1
    ## [52] bindrcpp_0.2.2        compiler_3.5.1        caTools_1.17.1.1
    ## [55] rlang_0.2.2           grid_3.5.1            iterators_1.0.10
    ## [58] rjson_0.2.20          htmlwidgets_1.2       labeling_0.3
    ## [61] bitops_1.0-6          codetools_0.2-15      gtable_0.2.0
    ## [64] flexmix_2.3-14        curl_3.2              reshape2_1.4.3
    ## [67] TSP_1.1-6             R6_2.2.2              seriation_1.2-3
    ## [70] gridExtra_2.3         prabclus_2.2-6        dplyr_0.7.6
    ## [73] bindr_0.1.1           rprojroot_1.3-2       KernSmooth_2.23-15
    ## [76] dendextend_1.8.0      modeltools_0.2-22     stringi_1.2.4
    ## [79] parallel_3.5.1        Rcpp_0.12.18          gclus_1.3.1
    ## [82] DEoptimR_1.0-8        tidyselect_0.2.4      diptest_0.75-7