We want TrustRadius to be your trusted source for evaluating software. As part of that mission, we aim to offer maximum transparency into how our site operates and how we ensure the integrity of our data. This FAQ covers all of our methodologies behind all of our quantitative data: scores, visuals, rankings and awards.
All of these practices follow our core guiding principles, and help us ensure the information on our site is useful, reliable and worthy of your trust.
Here are the questions covered in this FAQ:
What is trScore?
trScore is an algorithm that calculates a product’s scores based on a weighted average of reviews and ratings, rather than a simple average.
The problem: Simple averages can be misleading
Most review sites use simple averages to determine a product’s score or overall star rating. At first, TrustRadius did as well, until we noticed a couple of problems with simple averages:
- Over time, simple averages don’t always tell the most up-to-date story. Products can change quickly, and for products that have a large review base with a significant number of older reviews, a score based on a simple average won’t reflect current customer sentiment.
- Simple averages allow vendors to artificially inflate their product’s score by inviting only known promoters to write a review. Happy customers are valid users and reviewers, but when they are over-represented in a product’s review base, a score based on a simple average will be biased.
The solution: A weighted average called trScore
Our trScore algorithm weights ratings and reviews differently in the following ways, to provide a more reliable score than a simple average:
- More recent reviews and ratings are weighted more heavily than older ones.
- Reviews from a random, representative sample of users are weighted more heavily than biased samples.
- Ratings given as part of an in-depth review are weighted more heavily than single-click star ratings, since they are more considered and include more context.
All average scores on TrustRadius are calculated using the trScore algorithm, including the overall Likelihood to Recommend score, scores for attributes like usability and support, and scores for specific features like dashboarding and reporting.
The trScore algorithm allows us to present the most accurate and objective picture of customer sentiment to buyers, as well as a level playing field for vendors.
What are TrustMaps?
TrustMaps are two-dimensional charts that help you compare software products in a particular category based on user feedback. You might liken TrustMaps to analyst market quadrants, with some key differences:
- TrustMaps are based purely on user data rather than analyst investigation.
- TrustMaps are dynamically updated as new data comes in.
- TrustMaps can be filtered based on company size.
- TrustMaps are backed by in-depth, quality review content, so you can drill down to understand the context behind the numbers.
- A product’s inclusion on the map does not depend on company size, revenue, popularity or status as a TrustRadius customer. Any product that fits the category and has the required number of ratings and reviews (to ensure sufficient data) will appear on the map.
- Unlike other review sites, once a product clears the threshold for inclusion, its position on the map is not tied to the number of reviews it has on TrustRadius.
What data goes into a TrustMap?
Products are plotted on the TrustMap based on two dimensions: the overall star rating (trScore) on a 10-point scale, and research frequency by prospective buyers measured by pageviews on TrustRadius.
The Y-axis indicates customer satisfaction, and is supported by the hundreds of thousands of user ratings and reviews on TrustRadius.
The X-axis indicates market share and product momentum. Each month more than 1 million technology buyers research software products on TrustRadius to inform their decisions. Thus the amount of website traffic a product receives relative to its competitors is a good proxy for buyer interest.
There is a third dimension to the TrustMaps as well – the data can be filtered based on company size, to show how products compare in different market segments. This is important because a small business and an enterprise have very different needs; i.e., a software product might be highly rated by enterprise users but too complex or overkill for small businesses. You can filter TrustMaps so that they reflect the most relevant feedback for you.
How does a product get on the TrustMap?
A few factors go into this, all of which are geared toward ensuring TrustMaps are useful resources for you:
- Category - A product must fit the category. In addition to external research, TrustRadius researchers will often speak with vendors and experts to arrive at a meaningful set of criteria for a category. The category overview defines these criteria. In some cases, a software product fits multiple categories and can show up on multiple TrustMaps.
- Data - A product must meet the data threshold to appear on the TrustMap. In many categories, the threshold is a least 10 ratings and reviews overall, and at least 5 in a particular company-size segment. In categories where TrustRadius has collected far more data, the threshold is higher. Additionally, to be on a particular company size segment, at least 15% of a product's ratings and reviews must come from users at companies of that size, to show sufficient segment focus.
- Products per Category - A TrustMap with one or two products is not very meaningful, so not all categories have TrustMaps (yet!). There must be a sufficient number of products that meet the threshold for the map to be published.
What is Customer Verified?
Products with the Customer Verified badge have at least 10 reviews from the past 12 months. This standard was selected based on buyer feedback about how many recent reviews they usually need to get the information they need to make a more confident decision.
Customer Verified status can be earned or lost at any time. When a product is Customer Verified, that means you'll find recent, in-depth feedback from real users about the product.
What is Top Rated?
In most categories, there are plenty of options out there. Top Rated awards help distinguish products in a category that have excellent customer satisfaction ratings. TrustRadius issues Top Rated awards each year in every category on our site.
Top Rated awards are based entirely on reviews. There is no paid placement or analyst opinion.
To qualify for a Top Rated award, a product must meet the following three criteria for recency, rating and relevance at the time the awards are issued:
- Recency: Product must have 10 or more recent reviews (from the past year)
- Rating: Product must have a trScore of 7.5 or higher
- Relevance: Product must earn at least .5% of the site traffic in the category
Where do Top Rated badges appear?
Top Rated badges appear on a product’s page on our site, and in many places where that product is displayed elsewhere on our site, such as when it’s being compared to another product.
The full list of Top Rated products for a category appear on the category’s page under the “Top Rated” tab. Products that have this category as their primary category appear first on the list, sorted by highest trScore. This is followed by products that are included in the category, but it is not their main category, sorted by highest trScore.
The top 5 highest rated Top Rated products also appear on the main category page.
How can vendors use the Top Rated badge?
Vendors who earn the badge can use it, for free, on their own sites, in a press release, on a blog, in email, or on social media. The award allows vendors to share social proof with their audiences based on the authentic voice of their customers.
What are Best of Awards?
TrustRadius recognizes products that have the highest % of positive Key Insights statistics for Best Feature Set, Best Value for Price, and Best Relationship in their primary categories.
Best of Awards are based entirely on reviews. There is no paid placement or analyst opinion.
To qualify for a Best of Award, a product must meet the following criteria:
- Source 10 reviews during the selected timeframe.
- Products will be selected based on Key Insight statistics found in review content: Best Value, Best Feature Set, and Best Relationship per category.
- To be considered the product must be updated and accessed through our vendor portal after July 1st.
- Best Relationship looks at Key Insight data in the product reviews for Would Buy Again, Implementation Expectations, and Sales and Marketing Promises.
- As needed, additional vetting via textual review analysis will be performed by the TrustRadius research team.
How can vendors use the Best of Award badge?
Vendors who earn the badge can use it, for free, on their own sites, in a press release, on a blog, in email, or on social media in various media forms if desired. The award allows vendors to share the social proof with their audiences based on the authentic voice of their customers.
What is TRUE?
The TRUE program recognizes vendors who are Transparent, Responsive, Unbiased and Ethical in sourcing and using customer reviews.
We’re going beyond rejecting inauthentic reviews, to showcase companies that are committed to truly helping you make the right decision. When you read reviews for a TRUE-certified vendor, you can know the vendor has committed to transparency and takes their customer feedback seriously. This means you can make an eyes-wide-open decision about the product—and that once you’re a customer, your voice will be heard.
TRUE vendors must certify that they will be transparent, responsive, unbiased and ethical in managing their review programs. Any vendor can apply to earn the badge for the calendar year. There is no fee to be a part of this program.
TRUE vendors must:
- Provide equal opportunity for product users to safely share honest feedback in reviews (ie, they don’t cherry-pick customers to invite to write reviews)
- Disclose how they source reviews and how incentives are used
- Read all published reviews and respond when necessary
- Represent review content accurately in sales and marketing materials
- Disclose the use of intent data for marketing purposes
The TRUE program highlights vendors who are inviting ALL of their customers to write reviews, rather than cherry-picking the happiest ones.
If you are a vendor who is interested in applying for TRUE, you can learn more here.
How are products and reviews sorted?
As a website with tens of thousands of products, organized into hundreds of categories, with hundreds of thousands of ratings and reviews, how items are sorted is important.
Our guiding principles for sorting are:
- Our site should be as useful to you as possible
- Sorting logic should be objective and transparent
- Sorting can’t be artificially influenced by vendors
Products in a category
The default sort order of products in a category is by number of reviews in descending order -- Products with the most reviews are displayed first. Since the most widely used products are often the most reviewed, this tends to be a useful sort-order for buyers, and has the added benefit of being easy to understand.
You can also change the sort order based on your needs: date, highest overall rating, and lowest overall rating are other options.
Top Rated products
Products that have a category as their primary category appear first on the list, sorted by highest trScore. This is followed by products that are included in the category, but it is not their main category, sorted by highest trScore. Top Rated awards are issued once a year, and products that earn the award keep it all year. Scores are dynamic (updated daily based on new reviews), and therefore the sorting of Top Rated lists will change as scores change.
The top 5 highest rated Top Rated products appear on the category page as well, and are sorted with the same methodology.
Reviews on a product page
The default sort order of reviews on a product page is called “Relevance.” It’s a simple algorithm that prioritizes reviews based on recency, attribution (whether the reviewer chose to be public or anonymous), and level of detail or quality of the review. The “Relevance” algorithm is designed to put the most helpful reviews at the top of the page, by prioritizing the factors that our readers have told us matter most.
You can also change the sort order based on your needs: date, highest overall rating, and lowest overall rating are other options.
Alternatives and suggested comparisons
Two of the main reasons buyers visit TrustRadius are to discover alternative products and to compare products. Every product page on TrustRadius offers a list of alternatives and suggested comparisons.
These products are determined based on site traffic patterns on TrustRadius and Google search patterns (e.g. searches for Product X vs. Product Y). Products that are most often compared are suggested as alternatives and comparisons on each other’s pages.
