VibeMetrix Relevance, Ranking and the VBR

So as my first post on the VibeMetrix blog, I’m happy to announce our new ranking and relevance system is live! I’m going to post a few technical details later in the post so if you’re not interested you can skip that part but here’s the upshot of our new system: hopefully results in the ‘relevance’ sort mode will show higher quality results!

Originally it was our intention to use pre-existing ranking systems. As we reviewed what was available, there wasn’t any one metric or value that really encapsulated all the different factors that we wanted to surface. We also found that a lot of the ranking or metric systems were very opaque, and subject to high degrees of variability. Reading opinion pieces online, there also seemed to be some amount of general dissatisfaction with the ranking offerings currently available for blogs.

Our new system is based on algorithms we’ve developed. At the core of the new system is something that we’ve internally been calling the VBR, or VibeMetrix Blog Rank. It was designed to address our particular needs but maybe, as our algorithm improves, others could also use it as a reference point.

The VBR

So what is the VBR and what does it look like? We drew inspiration from some other pre-existing systems, like Technorati’s authority numbers and Google’s PageRank. We use several factors in computing the VBR, but currently the heaviest component is, similar to Technorati, how many unique blogs link to this particular blog. We roll up these factors and arrive at a single VBR score that ranges from 0 to 10, 0 being least ranked and 10 being highest rank. It’s not an absolute ranking but rather a measure of how good a source we believe that blog is. In this way it’s similar to Technorati’s authority but normalized to a score like Google’s PageRank.

Here’s the typical distribution of the VBR based on a cross section of the blogs we currently have in the system.^[*] As you can see, it’s a fairly sharp distribution. The VBR flattens the natural “top heavy” nature of blog linking a bit, to make an effort to normalize the “wide middle” of blog behavior.

It’s generally much harder to have a blog in the 7-10 VBR range than in the 3-6 VBR range.

To give a sense of the VBR, here are some popular blogs and their current VBRs:

Politics and political blogs are ranking very highly at the moment which is to be expected with all of the election time coverage in the blogs that’s occurred as of late.

Better Search Results

The most immediate effect of the VBR is to improve the results from our own search engine. To increase the relevance of the results, we use the VBR as a factor in determining the results of a search as well as the order in which those results are displayed. There are currently three things that contribute to what gets displayed and in what order, quality of the match, freshness of the result, and the VBR of the source of the item.

The first component, search quality, is a fairly complicated score but it reflects several things: where the terms you searched for appear in the post, how well do “phrases” match, as well as the frequency of the terms and their relative uniqueness in the search index. So a simple example is that if I search for “McCain”, posts with “McCain” in the post title will generally have a higher match quality score than if it appeared elsewhere in the post.

The second weighting used to provide search results is how fresh the item is. Since VibeMetrix is all about conversations, we thought that the items that have been posted within the last 24 hours should generally rank higher than others. Conversations on-line have a relatively short half-life, so it’s good to get in there early while people are engaged. We use a simple exponential decay function spread over 24 hours to apply a “freshness” score. This means that posts within the last hour will have a lot of weight added, posts within 12 hours, less so, and posts that occurred after 24 hours have very little additional weight applied.

This function looks like this:

The last factor applied is the VBR score. We’re adjusting the various contributions of each of these weighting factors, but here’s a graphical representation of a typical search “Social Media Marketing” using the ‘relevance’ sorting option.

You can see that the top item had a reasonably good match quality, the blog was moderately ranked in terms of it’s VBR, and it had a very fresh post. These all contributed to moving it up in the search result set.

Generally speaking, obscure terms that aren’t posted about as often will have their VBR and match quality be more important in terms of the search results weight whereas common terms that are posted about frequently will have freshness factor in much more prominently.

Fore example, here is a search for “Obama” in the same system.

There’s a lot of recent posts mentioning Obama within the last 24 hours, so ‘freshness’ contributes a lot more to the relevance of the posts.

The future of the VBR

This VBR factor is intended to be a work in progress and we’re hoping that people interested in blog ranking and the general “shape” of the conversation on-line will help us refine this as we go forward. We’re also hoping that after some initial development and work, that VBR numbers will remain relatively stable, with any major algorithmic changes being announced here. A common complaint I’ve heard is that other scores tend to bounce around a lot without warning, something we hope to avoid.

We’re considering lots of ways of extending the VBR score. Some future factors under consideration for applying to the VBR are how often the blog posts, how many outbound links there are to other domains, and most interestingly, what is the link pattern of those blogs. For example, is the conversation better served by high VBR blogs that point to lower VBR blogs or is a lot of high level to high level linking more useful. Since VibeMetrix is a blog/conversation management tool, we believe being relevant also means connecting to others, and we hope to bring more interesting analysis to who links to who.

Once we’ve gotten it to the point where we want it, we will provide a free tool that will let you interrogate our system to get the VBR score for your blog. We also intend to create tools for subscribed users that will allow you to get measurements of the movement of the VBR score. In other words, you’ll be able to use it as a measurement / reference point to help determine how well your social media marketing campaigns are progressing.

Overall, the VBR is a pretty central part of our technology and our plans going forward so we’re very interested in your thoughts, ideas, and feedback on it.

Webinars

As CTO of Grazr/VibeMetrix, I will occasionally share what I think are relevant or interesting technical details on the VibeMetrix blog regarding the work we’re doing. If there’s a particular technical question you want addressed here, don’t hesitate to let us know. In addition to the occasional posts, I’ll also be running an “Ask the developers” webinar running every Thursday starting on October 30th, you can sign up for it on our webinar information page.

Tagged as: ranking, Search, technology, vibemetrix