News & Features

Share:

Before the USTA Clay Court Championships began, Tennis Recruiting put out heat maps forecasting the chances players had to make runs through the tournament. We made a number of predictions based on that forecast - speculating on which seeds had the toughest paths to the semifinals, which players had the easiest roads, and estimating the number of upsets we would see.

Now that the matches are played, it's time to check our work. How well did our forecast predict the matches? We have worked up a post-tournament analysis of all the main and consolation draw matches for the Clays, which you can access by clicking here. Let's take a look...

Rolling the Dice

Our forecasts are probabilistic in nature. When our charts project Player A as the favorite over Player B, you might think our forecast is accurate only if Player A wins. But our forecast actually expects Player B to win some fraction of the time. Upsets like these - where a lower-rated player defeats a higher-rated player - are supposed to happen.

As another example, consider predicting ten matches between players with very similar ratings. The favorites would all have a slightly better than 50% chance of winning their matches. We will always predict that the player with the higher rating will win, but at the same time we expect to be wrong about half of the time. We would expect there to be about five upsets.

For these reasons, the right question to ask is, "How accurate are the probabilities?" Across an aggregate set of matches, how do our probabilities for expected upsets match up with the reality of actual upsets? Now that the tournament is over, we can take a look.

Of the 2,616 matches played to completion in the tournament, the lower-rated player won 596 times, meaning that 22.8% of the matches were upsets. How did our forecast do? Our forecast projected 561 upsets, so our estimate of 21.42% upsets was within 1.4% of the actual number. At the broad level, our forecast looks to have done a good job with the Clays.

Inside the Numbers

The broad numbers look good, but were our predictions accurate at a finer level? For example, did players with a 40% chance of winning produce upsets more often than players with only a 20% chance?

To evaluate the finer level, we can break down the upsets into five groups: 0-10% win probability, 10-20%, 20-30%, 30-40%, and 40-50%. If our forecast is accurate, we would expect underdogs in the 0-10% group to win about 5% of the time, underdogs in the 10-20% group to win about 15% of the time, and so on.

Again, the numbers work out pretty well. Notice that the expected upsets fall about in the middle of each group - which is no big surprise. More interestingly, you can also see that the actual number of upsets fall within those same group ranges. All of our estimates were within 4.1% of the actual upsets.

Also note that the actual number of upsets in each group is increasing as the percentages get closer to 50%-50% - as you would expect. In other words, our system does a good job knowing which match-ups are more evenly matched (a coin toss) and which ones will be less competitive.

Seeds and Upsets

There were 672 matches in the main draw with at least one seed participating, and we predicted 139 upsets where the higher-seeded player would lose to a lower seed or unseeded player.

Looking across all age divisions, the actual number of upsets was 152, so our prediction came up 11 upsets short - within 1.9% of the actual.

Here is the breakdown by age division:

Scattershooting Across Predictions

We used our forecast to make four concrete predictions for the tournament in our pre-tournament article. Let's revisit those predictions and see how we did...

i) Players with the best chance of advancing to the semifinals are Alexa Noel, Jenson Brooksby, Sara Daavettila, Brandon Nakashima, Samantha Martinelli, and Patrick Kypson.

These predictions were a mixed bag. Three of our favorites - Noel, Brooksby, and Martinelli - went on to reach the finals, with Noel and Martinelli bringing home USTA Gold Balls. Our other three favorites went down earlier than expected.

ii) The draw that is toughest to pick is the Boys 18s.

The semifinals of the Boys 18s featured top seed and eventual champion Sam Riffice as well as No. 9 seed Vasil Kirkov and No. 17 seeds Alexandre Rotsaert and Nathan Perrone. Rotsaert in particular went on a tear - knocking off two higher-rated and higher-seeded players in the quarters and semis to reach the final before falling to Riffice.

Four of these five seeds suffered a main-draw loss prior to the semifinals. Kiger was able to reach the quarterfinals before falling to Danny Thomas, while Genesen, Fahey, and Joshi all lost their fourth-round matches. Only Mossmer was able to beat our projections - reaching the semifinals of the Girls 16s before losing to second-seeded Samantha Martinelli.

iv) Two draws with interesting first-round matchups are the Boys 14s and Boys 16s. We predict a whopping fifteen upsets in the first round of both of those tournaments. In the Boys 14s alone, there are eleven first-round matches between closely-ranked players where we believe the outcome is close to a coin toss.

The first days of the Boys 16s and 14s was even wilder than our projections anticipated. Sixteen of the 64 first-round matches in the 16s resulted in upsets. The 14s took it a step further with seventeen first-round upsets - including upsets in seven of the eleven matches between closely-ranked players that we highlighted.

Broadening the Perspective

The above analysis all focuses on the USTA Clay Court Nationals, but we actually practiced our method on a number of other international, national, and sectional events. You can click here to see similar analysis for thirteen different tournaments. For each of these tournaments, we used player ratings immediately prior to the tournament start as the basis of our predictions, and the analysis shows how well the predictions held up.

In these studies, we also show analysis for the USTA PPR rankings, Universal Tennis Ratings, and the International Tennis Federation where appropriate. As you can see, the Tennis Recruiting system compares favorably with other junior tennis ranking and rating systems - although all systems do a respectable job.

We could spend a lot more time exploring these other tournaments and other aggregate data, but we will leave that for future articles.

That's a Wrap...

This article concludes our coverage of Clay Court Championship Week. I hope that you have enjoyed the efforts of all our great content contributors. We have enjoyed offering you expanded coverage this year.

Make sure to check out any article you might have missed - and get ready for our coverage of the Hard Court Championships that start next weekend.

• Jul 20 - Clay Court Championship Week
• Jul 21 - Kalieva Upsets Top Seed in Three Sets in 12s
• Jul 21 - Saud Alhogbani Wins USTA Boys 12 Clays
• Jul 22 - William Grant Takes Control of Boys 14s
• Jul 22 - Martinelli Takes 16s Title in Virginia Beach
• Jul 23 - Noel Sweeps Through Draw to Claim Girls 14 Clays
• Jul 23 - Danny Thomas Wins Boys 16s in South Florida
• Jul 24 - Ellyse Hamlin Wins Singles, Doubles in Girls 18s
• Jul 24 - Riffice Pulls Through to Claim Gold in Boys 18s
• Jul 25 - Revisiting the Clay Court Forecast

Leave a Comment

More Ranking Articles

19-Sep-2017
Comparing Rating Algorithms
During the U.S. Open we posted information on social media about matchups involving American players - previewing matches by displaying rating and ranking differences and highlighting wins by underdogs. You might have noticed that the various rating systems often agreed on their favorites. But what about the close matches where they disagreed? Let's take a look ...

13-Jul-2016
Behind the Tennis Recruiting Rankings
This week marks the 569th consecutive week that Tennis Recruiting has put out graduation-based rankings of American junior boys and girls. Rankings are front and center on Tennis Recruiting, and people often ask us how our rankings work. Today we describe our ranking system ...

19-May-2016
What Is An Upset?
Tennis Recruiting is a website that rates and ranks junior tennis players. One of the questions we get most often from our users is, "What exactly is an upset?" There are many possible definitions of an upset - this article explores the question and puts forward an answer.

 

Ranking Article Index |

Subscribe