RecSys 2016 – Part II. – Gravity @ RecSys 2016

by Balázs Hidasi


This is the second of three posts on the RecSys 2016 conference. Last week I wrote about the conference in general. This week I’ll discuss what Gravity contributed to the conference.

Since the conference was overseas, the Gravity contingent at RecSys was small this year, compared to last year. Only our CEO (Domonkos Tikk) and myself as the Head of Research represented Gravity. However, the contributions of Gravity were plentiful. We organized a workshop, had a long and two PPF papers, as well as a poster.

DLRS 2016

We co-organized the 1st Deep Learning for Recommender Systems (DLRS) workshop with Alexandros Karatzoglou from Telefonica Research and researchers from Israel (IBM Research & Ben Gurion University). The idea of organizing the workshop came to us naturally, since we were early adopters of deep learning in the RecSys community: we launched joint research collaboration with Alex last summer in this topic and have been working in this area ever since. We wanted to spread the word about deep learning in this community and encourage research, so we proposed to have this workshop. We were joined by another group who had a similar idea. The contribution of co-organizers showed significant variance, but this is pretty much what you can expect with seven organizers. 🙂


Read More

RecSys 2016 – Part I. – The conference from a research perspective

by Balázs Hidasi


The 10th conference in the ACM RecSys conference series was held in Boston between September 15 and 19. Upon returning from the conference (and a few days of vacation dedicated to discovering Boston), I decided to post about my experiences.

This is the first in a series of three blog posts on RecSys 2016. In this post, I write about the conference from a research perspective and discuss the popular research directions of the conference and the field in general. The next post will discuss Gravity’s contribution, which includes the organization of the Deep Learning for Recommender Systems workshop, presenting a long paper, and more. The final post will conclude this series with my best paper picks from RecSys 2016. Check back next week for the second blog post!

General thoughts

Last year I was kind of disappointed with the technical quality of RecSys and hoped that it was due to everyone working on new exciting research directions and wanted to roll the old stuff out before moving on. It felt like the calm before the storm and you could feel that the community had been already working on novel research projects, but only a few was ready for the public. The only question was whether these exciting topics will be discussed this year?


Read More

Tiki Boosts Revenues and Increases Customer Engagement by Partnering With Gravity

We have recently partnered with, one of the most prominent players of Vietnam’s booming eCommerce sector. During the initial testing period, our solution significantly outperformed competitors.

Today’s well-informed consumers tend to favor sites with the lowest prices and are rarely loyal to any particular eCommerce brand. Vietnam has a vibrant marketing scene, but most companies are focusing their marketing efforts on new customer acquisition. When Tiki decided to look for a recommendation solution provider, their goal was to prioritize user engagement and generate more revenues by providing a personalized experience. Gravity’s solution has proven to be highly effective in facilitating the upselling and cross-selling of Tiki products and improving customer retention.

During the pilot period, Gravity’s recommendations resulted in an additional $13.15 GMV per 1000 recommendations and a 6% average conversion rate.

“Our cooperation with Tiki is a major milestone in the company’s expansion in the SEA region. We’ve recently opened our office in Vietnam and hired Ngô Kỳ Lam, an experienced e-commerce specialist as our Country Manager in order to establish fruitful partnerships, and provide maximum support to our present and future clients in the region.” – Marton Vertes – Business Development Manager, Gravity R&D

“Gravity has been outperforming other similar solutions in every important aspect and KPI: Revenue, AOV, CTR and response time. This is why, after evaluating the results, we have decided to move forward with Gravity as our chosen recommender system provider.” – Hung Tran Viet – Product Manager, Tiki is one of the largest eCommerce sites in Vietnam, a country that is showing enormous growth in the online retail, and IT sectors in general. Similar to Amazon, Tiki started out by selling books online. Through the years, they’ve significantly expanded the scope of their operations. Currently, there are over 300,000 listings on the site in over twelve product categories and their traffic is growing substantially month over month.

Gravity offers scalable solutions for Enterprise clients with unique needs and larger traffic, as well as for small and medium-sized businesses through its turnkey SaaS recommendation engine, Yusp (

Read More

The logo of Yusp, the SME Recommendation as a Service product of Gravity R&D.

The Yusp Open Beta kicks off!

We recently launched our brand new SME solution called Yusp which is an easy to administer, SaaS based, out-of-the-box recommendation engine.

By installing Yusp, you opt in to harness the analytic power of Gravity’s proprietary, patented recommender algorithms matching in sophistication to those used by Amazon and other industry top players.

As a part of our Open Beta program, we provide free installation and configuration support to all new registrants, in addition to the 30-day free trial!

What does Yusp do?

Recommendations served by Yusp run through the same servers and are generated by the exact same algorithms as our enterprise clients’. Our algorithms have proven, immense capabilities to drive sales, increase conversions and significantly boost customer satisfaction and engagement on your site.

Even better, to get Yusp to work you need not perform any technical tour de force. Simply paste a code snippet in the header section of your site, alongside your Google Analytics tracking code and everything else is done over our super-user-friendly graphic interface.

Import your stock automatically

Simply paste the URL of one of your product pages in the appropriate field and Yusp automatically scans and imports the items in your stock.

A screenshot from the Dashboard interface of the Yusp recommendation engine.
Importing your product catalog into Yusp is fast and intuitive.

Tracking users

Our system tracks and records every detail of how users interact with your site and enriches this data with contextual parameters such as location, time, device, and referrer.

Personalized product recommendations

Building on these insights, our system fills the multi-device-friendly recommender boxes on your site with the items most relevant to each user in each context.

Drive sales and improve user experience

By showing the right product at the right time to the right users, Yusp can significantly improve your sales metrics and conversion rates by creating personalized user journeys for each and every one of your visitors to maximize customer engagement. Moreover, you can track the performance of the recommendations through our analytics dashboard.

A screenshot from the Dashboard interface of the Yusp recommendation engine.
You can easily track the performance of your recommendations through our analytics Dashboard.


Still skeptical? Don’t take our word for it, sign up for the Yusp Open Beta program and see for yourself. On top of the 30-day free trial, you get free, dedicated installation and configuration support so you can maximize the benefits Yusp brings to your business!

Yusp free trial banner.

Read More

Reflections on RecSys 2015

As is the norm in each yearly RecSys conference, there were several strong papers. But I found that while many of the papers were impressive from an engineering and practical use standpoint, they did not fare as well from a scientific research standpoint. More specifically, I’m singling out articles that simply take an area of research, add in some additional data sources, and then integrate this additional data into existing formulas to elicit some improved accuracy.


We’ve come quite far since the days of RMSE

When the Netflix Prize Competition was still active from 2006 to 2009, there was just one massive dataset (of 100 million ratings) and one target: the root-mean-square error (RMSE). During that time, the research was focused and papers were very comparable to each other. We’ve since come a long way from papers published around the years of the Netflix Prize, it has been determined that algorithms have varying levels of effectiveness depending on which dataset it’s being used against. It turns out that RMSE isn’t a good choice when your purpose is to generate relevant recommendations.

At Gravity, we found that using RMSE is not effective back in 2009 when we were building a public demo based on the Netflix Prize datasheet. Further explanation about how we reached this conclusion can be found in the first section of our RecSys presentation: Neighbor methods vs matrix factorization – case studies of real-life recommendations.

These days, there are plenty of datasets, and many different evaluation metrics are also available. To further drive the complexity of the current state of the RecSys community, researchers often add an additional data source to create even more complex algorithms. Over time, research topics are becoming more diverse, and research papers are no longer comparable.

For Gravity’s customers, item-to-item recommendations (people who viewed this item also viewed) are in higher demand than personalized recommendations. However, it’s really hard to find papers on the topic of item-to-item recommendation.

However, there was one paper from this year’s conference that I did find of interest and which I will explain below.


Top-N Recommendation for Shared Accounts

A paper that stood out to me this year, and that I would identify as my favorite would be from Koen Verstrepen and Bart Goethals: Top-N Recommendation for Shared Accounts. I observed their approach in this paper to be the following:

  • Consider a user who viewed Nitems.
  • Use a typical item-neighbor method to assign a score to each recommendable item based on what the user has viewed previously.
  • Create 2N-1 temporary users, each with a different subset of the original user’s viewing history.
  • Generate prediction scores for each of those temporary users, using the item-neighbor method.
  • For each temporary user, divide the scores by the temporary user’s history length, or a power of that number (e.g. take the square root of temporary user’s history length).
  • When calculating the prediction for item i for the original user, take the maximum score for item i over each temporary user, that will be the final score for item i for the original user.
  • Order items by the computed prediction scores

They show that this can be done in O(Nlog(N))time instead of O(2N). This approach (taking the maximum score over each temporary user) has another nice property: it can provide explanations, i.e. the root cause why item i was recommended to the original user. Consider for example, that for item i, the maximum score was generated by a temporary user who viewed items i1, i2 and i3. Then for item i, the recommender algorithm can say that it was recommended because the original user viewed items i1, i2 and i3.

This paper was really interesting because it focused on algorithmic methods, featured a simple yet fast solution, and they show how this method helps when multiple users are using the same account (e.g. a household watching TV), without knowing the number of persons in the household or knowing which person of the household viewed which item. They also propose an elegant way to generate diverse recommendations:


  • First, take the highest scored item. It will also have some explanatory items (see above)
  • Second, take the highest scored item from the rest, but consider only those items that have at least one explanatory item that is not amongst the highest scored item’s explanatory items
  • Third, take the highest scored item from the rest, but consider only those items that have at least one explanatory item that is not amongst the above items’ explanatory items
  • And so on


They also show that their method’s accuracy is comparable to the original neighbor method which they operate on, and is also capable of giving good recommendations when multiple people are sharing the same account. In my opinion, this method is a nice way to give users recommendations with the following 3 properties: diverse, accurate and easily explainable, all at the same time. I really enjoyed this paper as it was able to provide new enhancements to a method-familiy (item-based neighbor methods) that has been studied for so many years.


Closing thoughts

This year’s RecSys was a well organized conference, there were some really strong papers, as usual. But overall, I felt that it lacked the spirit of the old years, when every conference would bring about the announcement of several new breakthroughs in research. There used to be plenty of algorithmic papers every year, and everybody was always curious how research would develop into the future. Now that we already have all those breakthroughs, this area is maturing, and it’s become difficult to make big discoveries. This year’s many papers containing engineering work also indicates the less research oriented direction the conference is now taking.


In the future, I’d like to see more emphasis and research placed on the following topics:


  • Correlating offline and online measures (e.g. Recall vs. CTR). There was a paper this year about this topic, hopefully there will be many more papers in the upcoming years.
  • Correlating short-term and long-term online measures (e.g. predicting long-term site-income increase from short-term CTR increase). Simple example: if you make a customer buy twice as much water as usual, then this customer may skip buying water next time.
  • Item-2-item recommendations: this is a frequent topic in need of more research.
  • Matrix factorization methods that deal with really large and sparse matrices (e.g. 50M items x 100M user, 3 events per user). The problem here is that you have to increase the number of latent factors, otherwise totally unrelated items might become similar.
  • Content-based filtering methods that are able to find the most relevant items in real-time, even when there are 200M items. Currently, there are approximate solutions (e.g. Locality-Sensitive Hashing), which provide a trade-off between accuracy and running time, but if you need good accuracy, you would be better off running the naive approach.
  • AutoML is an interesting new direction, i.e. instead of manually choosing the best algorithms and manually tuning the hyperparameters, the aim is to have this process done automatically. Perhaps the RecSys community should make some step in this direction, e.g. a RecSys challenge with 20 different RecSys problems simultaneously would be something new and challenging.


Here’s to wishing for more breakthroughs and excitement in the future of the RecSys community!

István Pilászy is the Head of Core Development as well as one of the founders at Gravity R&D.

Read More