The Linguix team has recently conducted benchmarking tests that evaluated the performance of the grammar checking software, Linguix and Grammarly. 

For these purposes, we used two datasets containing 1515 sentences in English with both grammar mistakes and correct sentences. To be precise, there were several thousands of grammar mistakes to detect in total. 

Below are the performance comparison results of premium versions of both Linguix and Grammarly.

Linguix vs. Grammarly: which one detected more mistakes

When we analyzed performance based on single tokens (i.e. words with mistakes), Linguix found almost as many mistakes (1554) as Grammarly (1571).

When it comes to token spans (i.e. incorrect phrases), Linguix identified 1275 mistakes, while Grammarly found 1361.

We first measured the quality of grammar checking for our custom dataset. In this case, Linguix achieved an overall accuracy of 72%, while Grammarly demonstrated 76% accuracy:

Then we took a publicly available dataset (BEA). This time Linguix showed an accuracy of 61%, while Grammarly’s accuracy level was just about 45%:

Linguix’s progress

We are constantly working on improving our grammar-checking model quality. For the period starting from March 2020 till August 2022, the quality of identifying both words with mistakes and incorrect phrases has significantly increased:

According to our analysis, the difference between grammar-checking quality when using Linguix and Grammarly is almost unnoticeable.