Facebook Removing Relevance Score, Replacing with 3 New Metrics 😱

April 16, 2019
Posted in Paid Social
16 Apr 2019
Paid Social
April 16, 2019 Ryan Sigler

Facebook Removing Relevance Score, Replacing with 3 New Metrics 😱

On April 30 2019, Facebook removed the all-important Relevance Score from its advertising platform.

Should we all freak out!?

No!

Here’s the exact text from Facebook’s announcement about the change:

“We’re replacing the Relevance Score metric and replacing that with a new set of ad relevance diagnostics that are clearer and more actionable. Quality Ranking, Engagement Rate Ranking, and Conversion Rate Ranking can help you better diagnose whether the ads you ran were relevant to the audience you reached. The metrics are best used together, and we’ve provided additional guidance to help you interpret and act on different results. Ad relevance diagnostics will be introduced gradually and will reflect your ad’s performance in ad auctions that ran on or after March 19, 2019.”

So what do we think of the change? Frankly, we welcome it!

Relevance score was introduced way back in 2015 as a reflection of ad performance, or a way for advertisers to gauge how well their targeting, creative, and results fared compared to other advertisers. While Facebook of course never revealed the relevance score algorithm, it was comprised of the positive and negative feedback an ad got on the platform.

Positive feedback likely consisted of things like engagement metrics (video views, likes, positive comments, shares) and conversion metrics (CTR, results, time on site, bounce rate, etc).

Negative feedback likely consisted of things like people hiding an ad, low CTRs, or even negative comments.

The new relevance score replacements seem to break out the above indicators into 3 MECE categories:

Quality Ranking

Facebook defines the new Quality Ranking metric as follows:

“Quality ranking explains your ad’s perceived quality compared to ads competing for the same audience. We measure ad quality through feedback from people viewing or hiding the ad and assessments of clickbait, engagement bait and other poor user experiences.“

Here are the Quality Rankings you’ll see on your ads and what they mean:

  • Above Average (Top 45% of ads)
  • Average (Bottom 55% of Ads)
  • Below Average (Bottom 35% of ads)
  • Below Average (Bottom 20% of ads)
  • Below Average (Bottom 10% of ads)

While we assume that the new Quality Ranking metrics have always been included in the current Relevance Scores, it is nice to see Facebook taking a more direct approach in guiding advertisers toward better user experiences. We also wonder how they assess clickbait or engagement bait. Surely there are certain buzzwords they look for, but do they also take website metrics like bounce rate or time on site into account? This could be something to be cognizant of when designing the user experience on your website.

Engagement Rate Ranking

Facebook defines Engagement Rate Ranking as follows:

“Engagement rate ranking explains how your ad’s expected engagement rate compared to ads competing for the same audience. The expected engagement rate calculates the likelihood that a person will click, react to, comment on, share or expand an ad. Engagement-baiting (For example, asking for likes, comments, and so on) will not improve your ad’s performance.”

The rankings for Engagement Rate follow the same structure as the Quality Ranking metrics above.

As with the Quality Ranking metric, we assume these indicators have always been part of Relevance Score but it’s nice to see them separate as a way to guide advertisers on what to improve in their campaigns. Looks like the biggest factors here will be CTR and engagement rate (likes, comments, and shares).

Conversion Rate Ranking

The final new metric is Conversion Rate Ranking. Facebook defines it as:

“…how your ad’s expected conversion rate compares to ads with the same optimization goal competing for the same audience. The expected conversion rate calculates the likelihood that a person who viewed your ad will complete your optimization goal. For example, the optimization goal for a campaign with the Video views optimization goal would be 10-second video views.”

The rankings for this goal are the same as the other two new metrics.

Like the others, we’ve known that this one has been part of Relevance Score but it will be helpful to see it split out as a separate component here. The most interesting aspect of this is that it completely depends on your optimization goal and the conversion action you are seeking.

This could push more advertisers to optimize toward conversion events higher up the funnel. For example, if you have a high-end retailer with expensive products but you are optimizing for Purchases, you may end up with a very low Conversion Rate Ranking. Does that impact your CPMs to the point that it would make more sense to optimize for Content Views or even Link Clicks?

Also how does that impact advertisers that optimize for Custom Conversions? Facebook will not display a Conversion Rank Rating for ad sets optimizing for Custom Conversions, so does that effectively equal an “Average” score?

With any major changes to the platform like this, we have many questions and we’re excited to start diving into stats when they become available. Facebook has provided one final gift to us advertisers with this announcement in the form of a handy chart showing recommend actions based on the various rankings you see on your ads. Take a look and bookmark this link!

Need help navigating Facebook’s new metrics? Our Facebook Ads Management Services can help. Contact us today to get a proposal.