The Rating System
| Tier | Adjective | Low | Mid | High | Description |
|---|---|---|---|---|---|
| F | Bad | 0.0-0.9 | 1.0-1.9 | 2.0-2.9 | This should rarely be assigned, most releases have at least some redeeming qualities. |
| E | Mediocre | 3.0-3.6 | 3.7-4.2 | 4.3-4.9 | Bad, but there is far worse music. |
| D | Decent | 5.0-5.2 | 5.3-5.6 | 5.7-5.9 | This is music you would not want to listen to if given a choice, but you cannot really call it bad either, and the musicians involved are actually trying to make this sound good. |
| C | Good | 6.0-6.2 | 6.3-6.6 | 6.7-6.9 | Good, but not great. This music probably will not make it onto your daily playlist too often, but you would not mind listening to it occasionally |
| B | Great | 7.0-7.2 | 7.3-7.6 | 7.7-7.9 | This is playlist material. You really enjoy listening to this, but you are not doing so regularly because there is other music which is still better. |
| A | Awesome | 8.0-8.2 | 8.3-8.6 | 8.7-8.9 | This is where the bulk of your favorite music hangs out. It is what you listen to the most - there are better releases still, but these are masterpieces and these are not that numerous. And there is a lot more music on the lower tiers, but you do not listen to that a lot. |
| S | Supreme | 9.0-9.4 | 9.5-9.8 | 9.9-10 | The best of the best. If you know a lot of releases then you could have dozens of releases in the masterpiece tier, but most people only have a few. These should be releases that you consider to be (almost) flawless exemplars of their genres/styles and which in your opinion everyone should know and at least respect as landmark releases. |
Here you can choose how rating values will be displayed on the website. Choose the mode(s) that work best for you.
| Tier | Adjective | Value | Compact | Standard | |
|---|---|---|---|---|---|
| F | Bad | 0.0 | ☆☆☆☆☆☆☆ | ||
| F | Bad | 1.0 | ★☆☆☆☆☆☆ | ||
| F | Bad | 2.0 | ★☆☆☆☆☆☆ | ||
| E | Mediocre | 3.0 | ★★☆☆☆☆☆ | ||
| E | Mediocre | 3.7 | ★★☆☆☆☆☆ | ||
| E | Mediocre | 4.3 | ★★☆☆☆☆☆ | ||
| D | Decent | 5.0 | ★★★☆☆☆☆ | ||
| D | Decent | 5.3 | ★★★☆☆☆☆ | ||
| D | Decent | 5.7 | ★★★☆☆☆☆ | ||
| C | Good | 6.0 | ★★★★☆☆☆ | ||
| C | Good | 6.3 | ★★★★☆☆☆ | ||
| C | Good | 6.7 | ★★★★☆☆☆ | ||
| B | Great | 7.0 | ★★★★★☆☆ | ||
| B | Great | 7.3 | ★★★★★☆☆ | ||
| B | Great | 7.7 | ★★★★★☆☆ | ||
| A | Awesome | 8.0 | ★★★★★★☆ | ||
| A | Awesome | 8.3 | ★★★★★★☆ | ||
| A | Awesome | 8.7 | ★★★★★★☆ | ||
| S | Supreme | 9.0 | ★★★★★★★ | ||
| S | Supreme | 9.5 | ★★★★★★★ | ||
| S | Supreme | 9.9 | ★★★★★★★ |
How Are Ratings Aggregated?
There are many possible ways to calculate an aggregate rating from a list of individual ratings, which is what we obviously need to do for tracks and releases. The simplest is to compute an average value. But we think that this does not lead to the best results. Instead we compute the median value, or in other words, the typical value that is assigned by most users.
There are two additional things we do. First, there is the trust rank. Each user has a trust rank value, ranging from 1 to 10. New users start with 4, and more seasoned users - either by request or when they get recognized by admins - get trust ranks of 6 or 8. This trust rank is used as a weight in all computations. We also have machine accounts which collect data from other websites - those all have a trust rank of 1, because we regard those merely as a means to get more data into the system. To get back to the topic of rating aggregation, when we compute the median value the trust rank is always taken into account.
The other thing we do when calculating the mean is that we use a layered approach. We first divide all the tiers into three groups (low, mid and high) and see where most (weighted) rating values are. Within this group we then do the same for the tiers - we select the tier with the most "votes". Then within this tier, we select the correct sub-tier using the same mechanism. Finally, within the selected sub-tier we calculate the weighted median value, which we then choose as the aggregated value.
How Are Items Ranked?
When you browse the releases ordered by overall rank, the system needs to sort the releases in a way which takes into account the aggregated rating as well as the number of ratings. The simplest way to do that would be to sort by the product of these values. In practice we found that this boosts releases with a lot of ratings too much, so we chose to use a logarithmic value of the number of ratings instead, lessening the boosting effect somewhat.
So do not be surprised when in the list of ranked releases (or tracks) the aggregate rating of the releases appears to be unsorted, since items with lower aggregate ratings can outrank more highly rated items if the number of ratings is substantially higher.
