RecSys 2016 – Part II. – Gravity @ RecSys 2016

by Balázs Hidasi


This is the second of three posts on the RecSys 2016 conference. Last week I wrote about the conference in general. This week I’ll discuss what Gravity contributed to the conference.

Since the conference was overseas, the Gravity contingent at RecSys was small this year, compared to last year. Only our CEO (Domonkos Tikk) and myself as the Head of Research represented Gravity. However, the contributions of Gravity were plentiful. We organized a workshop, had a long and two PPF papers, as well as a poster.

DLRS 2016

We co-organized the 1st Deep Learning for Recommender Systems (DLRS) workshop with Alexandros Karatzoglou from Telefonica Research and researchers from Israel (IBM Research & Ben Gurion University). The idea of organizing the workshop came to us naturally, since we were early adopters of deep learning in the RecSys community: we launched joint research collaboration with Alex last summer in this topic and have been working in this area ever since. We wanted to spread the word about deep learning in this community and encourage research, so we proposed to have this workshop. We were joined by another group who had a similar idea. The contribution of co-organizers showed significant variance, but this is pretty much what you can expect with seven organizers. 🙂

DLRS RecSys 2016 - Dep Learning For Recommender Systems - Workshop @ RecSys 2016
DLRS RecSys 2016 – Deep Learning For Recommender Systems

We didn’t know the level of interest in the community beforehand, so we decided to play it safe and advertised it as a half day workshop. It turned out that there is a huge interest in this topic: we received 13 research papers, of which we accepted 7. Our workshop was fully booked shortly after the registration was opened. For 70 places, we had approx. twice as many registrants, even though it was clear that the workshop was full and people could only get on the waiting list. It is a shame that we couldn’t upgrade to a larger room and thus we had to turn away half of our (potential) audience, but apparently this was the largest room they had at the workshop venue.

Other than this, the workshop was a roaring success with a strong program. Sander Dieleman from Google Deepmind gave a fantastic keynote about music recommendation using deep learning, presenting his now classic work on feature extraction from audio for collaborative filtering, as well as very recent research on WaveNets.

Sander Dieleman from Google Deep Mind presenting at RecSys 2016, DSLR workshop organized by Gravity R&D
Sander Dieleman from Google Deep Mind presenting at RecSys 2016, DSLR workshop organized by Gravity R&D

The presentations from the authors of accepted papers were also great. We (in agreement with the reviewers) especially liked the paper “Recurrent Coevolutionary Latent Feature Processes for Continuous-Time User-Item Interactions” by Hanjun Dai, Yichen Wang, Rakshit Trivedi, and Le Song. As the title suggests, the paper deals with user and item feature vectors that (co-)evolve through time. The evolution of users and items is a very important problem in many practical applications as taste of the users and even the audience of items can change over time and the recommender should follow that. In their paper the authors composed the feature vectors from four parts: the first models the temporal drift, the second the self-evolution, the third captures the user-item coevolution, and the fourth incorporates the interactions. The (co-)evolving is captured by Recurrent Neural Networks. We didn’t give out actual best paper awards this year, but this paper received the unofficial best paper award of the workshop.

The other papers were also great. We had papers on embedding, word2vec, autoencoders, RNNs, and more. It was interesting to see how different research groups used these techniques for a variety of tasks. Some used them to increase recommendation accuracy, while others were more interested in analyzing the semantics of the item space and look at the representations more closely.

You can browse the workshop program on our website and since all of the papers are on, you can read the papers that interest you the most. Links to the papers are also on the program page. Slides of some of the presentations are also available.

Enhancing session based recommendations with images

I presented a long paper at the main conference that also dealt with deep learning. This is the second paper of the earlier mentioned research collaboration. Besides me, Alexandros, and Domonkos, this paper was also co-authored by Massimo Quadrana.

In this paper we used image (e.g. thumbnail) and text (e.g. title, description) features to improve session based recommendations. We built on the top of our previous work – GRU4Rec – in which we adapted RNNs to the session-based recommendation task. The importance of this problem lies in that these information sources (image, text) significantly influence whether a user clicks on the recommended item or not, since this is the information they see on the screen. (The decision of the user also depends on their internal state and preconceptions, but we can’t really model those directly.)

Balazs Hidasi presenting at RecSys 2016 - Session Based Recommendations with Deep Learning

Extracting the features from these sources is the easy part: we used a pretrained GoogLeNet on thumbnails from an online video service; and TF-IDF weighting on uni- and bigrams of the title and short description of products on a classified site. The hard part is using this information in the network. If we use the content features only, recommendations tend to get too general. If we use it along with the IDs, then the ID input will dominate the network, because initially the ID is a stronger indicator. Thus we gain nothing from using the content features.

Our solution was to propose a new architecture, coined parallel RNN (p-RNN) in which each information source is processed by a separate subnet. The subnets have their own session models in their hidden states and the output is computed on the concatenation of these models. In and of itself this only gives slight improvements, because the subnets are too loosely coupled and both subnets learn somewhat similar things about the session, wasting some of the network’s capacity. Therefore we also proposed alternative training methods, in which the networks are forced to complement each other better. We achieve this by training only one subnet at a time and fixing the others. The three training strategies differ in the frequency of switching between the subnets.

Balazs Hidasi presenting at RecSys 2016 - Session Based Recommendations with Deep Learning

The proposed network achieves +15% in MRR compared to the original GRU4Rec and it can even gain +5-7% when increasing the number of hidden units and epochs in the original network has diminishing returns.

The paper was well received by the community; there was a lot of interest in it and attention around it. Several people came up to me after my presentation and praised the work, which I was really happy about.

If you are interested in the details, you can read the paper or flip through the slides. A video of my presentation will also be available sometime in the future.

Setting the trends

Here at Gravity, we pride ourselves on having been in the frontline of recommender systems research for a long time. Therefore it was obvious that we would contribute to Past Present Future (PPF) papers. PPF was a special track of the 10th RecSys conference. PPF papers discussed the legacy of the last ten years in the field of recommender systems as well as contemplated on where the field is heading or where should it head. One of the goals of PPF papers was to set future directions for the community and as such was of great interest. Gravity employees contributed to two of the PPF papers presented at the conference.

I and Domonkos contributed to Roberto Pagano’s “The Contextual Turn: from Context-Aware to Context-Driven Recommender Systems” paper. The paper discusses how recommenders are evolving from being context-aware to context-driven. Context-aware recommenders consider contextual information as side or auxiliary information to the user-item interactions, and the latter is considered to be the main information source. On the other hand, context-driven recommenders mainly focus on the recommendations’ surrounding or situation. This is in line with the observation that two people often behave more similarly to each other in the same situation than to themselves in different situations. In many practical recommender systems, the user is either irrelevant or hard to identify. Relevant products or content are determined by the context of the recommendations and/or can be derived from the most recent history of the user (i.e. the actual session). Our recent research on session-based recommendations and the General Factorization Framework lines up pretty well with this idea, so it was natural for us to join Roberto on this paper. I think it turned out to be a really good paper and I recommend reading it. I’m also looking forward to Roberto’s future work and Ph.D. thesis which will deal with this topic in more detail.

The lead author of the other paper was one of our colleagues, Tamás Motajcsek, and the paper was also co-authored by Domonkos. The paper is the result of the Revolution Workshop, a think-tank organized under the egis of the CrowdRec project. At the workshop, 20+ researches, practitioners, business people, and enthusiasts from and outside of the recommender systems community contemplated on what recommender systems can and should become in the long run. The conclusion of the workshop was summarized in the paper titled “Algorithms Aside: Recommendation As The Lens Of Life”.  Given its focus on the long term evolution of the field, the paper is less research focused, but rather emphasizes the nature of how organic recommendations work. It is an interesting read if you are open to a different point of view and not put off by the lack of algorithms.

Benchmarking recommenders

One of our colleagues, András Serény, co-authored a poster on Idomaar. Idomaar is multi-dimensional benchmarking framework for evaluating recommender systems from different perspectives. It is developed in the CrowdRec project.

Summary & what’s next?

I’m satisfied with what Gravity contributed to the conference this year. As someone, who coordinated the organization and put in (by far) the most hours, I’m especially proud of the success of the DLRS workshop. As a result, we plan to have a DLRS 2017 next year. The details are still up in the air, but it might be a longer event. We’ll see.

I’m also really proud of the good reception of my paper. It reassures me that it was a good idea to start working on deep learning early (i.e. a year before it broke into the RecSys community), because we are now able to go beyond trivial architectures and create something that is really interesting to a lot of people.

I’m convinced that exploring how deep learning can benefit recommender systems is the way to move forward for recommender algorithms in the next few years. Therefore it is good to see that the community is open to this topic and also appreciates my work in the field. We will continue working on this topic and hopefully show you something interesting again at RecSys 2017.


Balázs Hidasi is the Head of Data Mining and Research in Gravity R&D. He is responsible for coordinating his team’s and conducting his own research on advanced recommender algorithms. His areas of expertise include deep learning, context-aware recommender systems, tensor- and matrix factorization. Balázs also coordinates and consults for data mining projects within the company. He has a PhD in computer science from the Budapest University of Technology.

Share this post: