\section{Suggesting Improvement to a Recommender System}


How can the measure be used to suggest improvement to a personalized recommender system? There are several ways that the measure can be used to suggest improvement. One way is to use it as a measure of the overall personalization of a recommender system and to use that measure, on the basis of the score, to either reduce or increase personalization. This can be achieved by for example introducing other features in addition to the ones that we have.

Another use of the measure is to use it at different levels of personalization, for example demographic or geographic,  and use that as a measure of the potential for personalization. This is like what the potential for personalization has done with users, but this time at another granularity. This measure is a quick way of knowing the potential for personalization at either an aggregate level or on a pair-of-users level. This can also be used to identify where we need to wok more, to introduce more personalization. For example, if we consider two cities, or two states, and they seem to be diverging more and more, we can zoom on those cities and do more personalization. Similarly, we can leave some pairs of users unaffected if the level of separation as indicated by the PullPush score is good. 

  The best way to use the method to improve a personalized recommender system, however,  is to use it as an objective function and experiment with different features with the aim of lowering the PullPush score  to $mathit{0}$. This is a much more useful objective function than CTR, or any other measure of personalization so far.  And finally, another perspective to bring on this measure, is assuming that we can use it to avoid the 'maximum filter bubble' that would organically occur if user cohorts' interests are indeed pushing the served content apart. Arguably, the system should always push back on the potential desire to be apart, and as such the 'maximum personalization score' should never be reached.