the most useful R function of the week: unnest from tidyr

There are many great functions in CRAN and BioConductor, and certainly saying that unnest from the tidyr package is the best is a big exaggeration. However this function solved a big problem in data formatting that made me waste a lot of time in the past, that I was surprised no one had implemented a function for it yet.

Imagine we have a dataframe like the following:

The first column contains the Entrez of each gene. This columns is fine, as it contains only one value per row, and it is easy to query or join with other dataframes. The second column, however, contains a comma-separated list of gene names, all associated to the same Entrez IDs. This column is a mess to deal with, because we need to use grepl to query it, and we can’t join it with other dataframes as long as it is in this form.

The unnest function from tidyr allows to convert this data frame in a “tidier” form, containing one row for each combination gene symbol and alias:

This code makes use of the %>% and some functions from the dplyr package, but it is still R!

Having the dataframe in this long form makes it a lot easier to deal with it. For example, let’s imagine that somebody asks us to get the Entrez IDs for the list of gene symbols DER7 and DEC. We would just have to do a simple subset on the dataframe:

This is just a silly example, which may have been solved with some application of apply and grepl, but in the real world there are a lot of more complex applications for it. For example, here is some code I used to split Blat output into one line per exon (or blat alignment block):

Continue reading

a formula interface for GeneOntology analysis

clusterProfiler is a nice R library for doing GO and KEGG enrichment analysis. It has a simple interface and it can produce some clear plots using the ggplot engine. Today I contributed a formula interface to clusterProfiler, making it easier to do enrichment of multiple groups of genes.

Let’s imagine you have a dataframe in which one column contains a list of Entrez Ids, while the other columns encode some grouping variables:

The new formula interface allows to do a GO analysis on each of the groups. For example, we can group them  by the column “group”, and compare the classification of the two groups:

clusterProfiler example

In this case group A is enriched in membrane and extracellular region, while group B is only enriched in membrane genes. The groupGO function used here doesn’t provide p-values – we should have used enrichGO instead. I guess that the 3 Entrez ids in group A correspond to 5 genes in Gene Ontology, so that’s why the plots shows a total of 5 in group A. See clusterProfiler’s documentation for better examples.

The formula interface allows also multiple grouping. For example:

Of course this example is not much interesting, since it is only 6 randomly chosen genes. However with bigger datasets the formula interface can be much more powerful.

Now that I think of it, it would be better if compareCluster would return a dataframe with multiple columns, instead of merging them into a single column called Cluster. This would be make it possible to plot the results using facets or something more fancy. It would be something like this:

However this would probably require introduce some retro-incompatibility in the library, and it is not a big deal as the Cluster column can be easily split using the separate function from tidyr.

Tidy data and VCF

I think that 2014 has been the year in which my R programming style has changed the most. This is because a lot of innovative and nice libraries have been released, like dplyr, magrittr and tidyr. I started in January as a ddply enthusiast, and now instead my code is full %>% instructions and dplyr functions.

If you missed these libraries, a good starting point is the article “Principles of Tidy Dataset“, in which the author Hadley Wickham suggests some best practices for organising a dataset in a “tidy” format before doing any analysis. These practices will be already familiar to you if you have experience with the reshape/reshape2 packages, and if you used ggplot2 in the past. However, it is good to read a good summary as in the article.

Inspired by this article, I wrote a post on Biostar to discuss how a popular format in bioinformatics – the VCF – in a tidy format. Here is the link to the discussion.

The VCF in a tidy format would like more or less as below. On one hand, it would be a bit too redundant, and many columns would be replicated multiple times, making the file more sensible to typos introduced by the users and occupying more disk space. On the other hand, it would be easier to read, more flexible and able to accommodate other informations, like the population of each individual or more info about the genotype quality.

 

 

https://www.biostars.org/p/123018/

farewell to Barcelona

I am posting this a bit late (since I already moved 6 months ago); anyway, the news is that I left my lab in Barcelona, and moved to London!

prbb from avda aiguader
The institute where I did my PhD: the PRBB in Barcelona. On the other side of the building there is the beach.

I am satisfied about my time in Barcelona, where I did my master thesis and my PhD on network theory applied to human population genetics. However, it was time to move and try new experiences.

a picture taken from the 4th floor of the PRBB
a picture taken from the 4th floor of the PRBB

Apart from the change of city, I also changed my field of work, as I am now working on cancer genetics. My new group is a young group recently moved from Italy to London, famous for research on the systems-level properties of cancer genes , for a database called Network of Cancer Genes, and involved in a consortium for the sequencing of hepatocellular carcinoma. I will keep you informed of the proceedings!

my first PyPI package: vcf2networks

My first Python package is in PyPI!! I guess that now I can officially call myself a python programmer.

VCF2Networks is a python script to calculate genotype networks from population genetics data. Genotype networks are a method used in systems biology to study the “innovability” of a given phenotype, by representing all the genotypes associated with the phenotype as a graph, and studying some properties of this graph, such as the average path length and the average degree. For more info, you can look at the slides of the “Origins of Evolutionary Innovations” book club in this blog. The script in VCF2Networks allows to take any dataset of genotypes stored in the VCF format, and calculate many of these properties.

In principle, I am planning to submit an application note about the script to a bioinformatics-oriented journal. So, if you have some little time to lent me, and you want to test it, any feedback will be very useful for me. At the moment, the major issue is to simplify the installation, because this package depends on numpy and python-igraph, and these two modules require some terrible C libraries that must be installed separately. If you are aware of any way to distribute a binary package of a python module that depends on C libraries, your suggestion will be really welcome.

The presentation of my PhD defence

That’s it! Last week I defended my PhD thesis!! I have gone through it, and survived to tell!

I don’t feel very different from before, apart from being relieved :-). Now the future is possibly more difficult than before, because I have to look for a job position and finish a lot of things.

While I was preparing the slideshow, I realized that there are not many examples of presentations for a PhD defence online. This is bad, because you need all forms of help to prepare this presentation.The PhD defence is the last thing that you do as a PhD student, so you want to do it perfectly. It is also the moment when you describe many years of your work to the your colleagues and family. Thus, it is bad that there are few examples of slideshows for PhD defence online.

Here is the presentation that I have prepared for my defence. I hope that it will be useful to other people as an example for their defences.

I think that, for this type of presentation, the first slide to make is the “summary of the talk” slide, like the “Topics” slide I have. Usually I don’t like to have such summary slides in my presentation, but for the Thesis defence it is very important, because it gives you a feeling of security when you present. Having a well defined structure allows you to know when you can stop to drink some water or to check if everybody is following, and to know exactly what to say in each slide of the talk.

my poster featured in the “Better Posters” blog!

My ECCB2012 poster has been featured in the Better Posters blog. Check the article here: http://betterposters.blogspot.com.es/2013/10/invitating-interaction.html

I am glad because betterposters is one of my favorite blogs. It’s a blog about designing and improving posters for scientific conferences, and it contain many tips and examples of how scientific posters can be improved.

The poster featured there is the poster of the “Post-its”, which I briefly described in the article of the “best practices“.

DSC00132
Here are some other comments and tips from my experience of using post-its to get feedback during a conference:

Continue reading

Two books on Human Evolution and on the concept of Race, that should be read together

I have decided that, from time to time, I will post some book recommendation here on this blog. This is the first of this series, dedicated to a pair of books on the evolution of the human genome and the concepts of races / human populations.

fatal invention    10000 years explosion

Continue reading