Home Boys' Lists Girls' Lists Men's Teams Women's Teams News Photos Contributors Links Help Sign-UpOnline Store
Latest News | Categories | Authors | News Archives

News & Features

Clay Court Championship Week
Revisiting the Clay Court Forecast

Before the USTA Clay Court Championships began, Tennis Recruiting put out heat maps forecasting the chances players had to make runs through the tournament. We made a number of predictions based on that forecast - speculating on which seeds had the toughest paths to the semifinals, which players had the easiest roads, and estimating the number of upsets we would see.

Now that the matches are played, it's time to check our work. How well did our forecast predict the matches? We have worked up a post-tournament analysis of all the main and consolation draw matches for the Clays, which you can access by clicking here. Let's take a look...


Rolling the Dice

Our forecasts are probabilistic in nature. When our charts project Player A as the favorite over Player B, you might think our forecast is accurate only if Player A wins. But our forecast actually expects Player B to win some fraction of the time. Upsets like these - where a lower-rated player defeats a higher-rated player - are supposed to happen.

As another example, consider predicting ten matches between players with very similar ratings. The favorites would all have a slightly better than 50% chance of winning their matches. We will always predict that the player with the higher rating will win, but at the same time we expect to be wrong about half of the time. We would expect there to be about five upsets.

For these reasons, the right question to ask is, "How accurate are the probabilities?" Across an aggregate set of matches, how do our probabilities for expected upsets match up with the reality of actual upsets? Now that the tournament is over, we can take a look.

Matches Matches Expected Upsets Actual Upsets Difference
Main Draw 1272 244 (19.2%) 255 (20.0%) -11 (-0.8%)
All 2616 561 (21.4%) 596 (22.8%) -36 (-1.4%)

Of the 2,616 matches played to completion in the tournament, the lower-rated player won 596 times, meaning that 22.8% of the matches were upsets. How did our forecast do? Our forecast projected 561 upsets, so our estimate of 21.42% upsets was within 1.4% of the actual number. At the broad level, our forecast looks to have done a good job with the Clays.


Inside the Numbers

The broad numbers look good, but were our predictions accurate at a finer level? For example, did players with a 40% chance of winning produce upsets more often than players with only a 20% chance?

To evaluate the finer level, we can break down the upsets into five groups: 0-10% win probability, 10-20%, 20-30%, 30-40%, and 40-50%. If our forecast is accurate, we would expect underdogs in the 0-10% group to win about 5% of the time, underdogs in the 10-20% group to win about 15% of the time, and so on.

Group (Win %) Matches Expected Upsets Actual Upsets Difference
0% to 10% 789 34 (4.4%) 30 (3.8%) +4 (+0.6%)
10% to 20% 539 79 (14.7%) 90 (16.7%) -11 (-2.0%)
20% to 30% 457 114 (25.0%) 133 (29.1%) -19 (-4.1%)
30% to 40% 406 142 (35.0%) 141 (34.7%) +1 (+0.3%)
40% to 50% 425 191 (44.9%) 202 (47.5%) -11 (-2.6%)
All Matches 2616 561 (21.4%) 596 (22.8%) -36 (-1.4%)

Again, the numbers work out pretty well. Notice that the expected upsets fall about in the middle of each group - which is no big surprise. More interestingly, you can also see that the actual number of upsets fall within those same group ranges. All of our estimates were within 4.1% of the actual upsets.

Also note that the actual number of upsets in each group is increasing as the percentages get closer to 50%-50% - as you would expect. In other words, our system does a good job knowing which match-ups are more evenly matched (a coin toss) and which ones will be less competitive.


Seeds and Upsets

There were 672 matches in the main draw with at least one seed participating, and we predicted 139 upsets where the higher-seeded player would lose to a lower seed or unseeded player.

Looking across all age divisions, the actual number of upsets was 152, so our prediction came up 11 upsets short - within 1.9% of the actual.

Here is the breakdown by age division:

Age Division Matches Expected Upsets Actual Upsets Difference
Boys 18s 86 17 19 -2
Boys 16s 74 20 25 -5
Boys 14s 83 19 21 -2
Boys 12s 84 18 21 -3
Girls 18s 85 19 16 +3
Girls 16s 81 15 16 -1
Girls 14s 92 17 17 0
Girls 12s 87 14 17 -3
All Divisions 672 139 152 -13

Scattershooting Across Predictions

We used our forecast to make four concrete predictions for the tournament in our pre-tournament article. Let's revisit those predictions and see how we did...


i) Players with the best chance of advancing to the semifinals are Alexa Noel, Jenson Brooksby, Sara Daavettila, Brandon Nakashima, Samantha Martinelli, and Patrick Kypson.

These predictions were a mixed bag. Three of our favorites - Noel, Brooksby, and Martinelli - went on to reach the finals, with Noel and Martinelli bringing home USTA Gold Balls. Our other three favorites went down earlier than expected.


ii) The draw that is toughest to pick is the Boys 18s.

The semifinals of the Boys 18s featured top seed and eventual champion Sam Riffice as well as No. 9 seed Vasil Kirkov and No. 17 seeds Alexandre Rotsaert and Nathan Perrone. Rotsaert in particular went on a tear - knocking off two higher-rated and higher-seeded players in the quarters and semis to reach the final before falling to Riffice.


iii) Seeds with the toughest road to the semifinals are Mac Kiger, William Genesen, Katharine Fahey, Nicole Mossmer, and Abhijeet Joshi.

Four of these five seeds suffered a main-draw loss prior to the semifinals. Kiger was able to reach the quarterfinals before falling to Danny Thomas, while Genesen, Fahey, and Joshi all lost their fourth-round matches. Only Mossmer was able to beat our projections - reaching the semifinals of the Girls 16s before losing to second-seeded Samantha Martinelli.


iv) Two draws with interesting first-round matchups are the Boys 14s and Boys 16s. We predict a whopping fifteen upsets in the first round of both of those tournaments. In the Boys 14s alone, there are eleven first-round matches between closely-ranked players where we believe the outcome is close to a coin toss.

The first days of the Boys 16s and 14s was even wilder than our projections anticipated. Sixteen of the 64 first-round matches in the 16s resulted in upsets. The 14s took it a step further with seventeen first-round upsets - including upsets in seven of the eleven matches between closely-ranked players that we highlighted.


Broadening the Perspective

The above analysis all focuses on the USTA Clay Court Nationals, but we actually practiced our method on a number of other international, national, and sectional events. You can click here to see similar analysis for thirteen different tournaments. For each of these tournaments, we used player ratings immediately prior to the tournament start as the basis of our predictions, and the analysis shows how well the predictions held up.

In these studies, we also show analysis for the USTA PPR rankings, Universal Tennis Ratings, and the International Tennis Federation where appropriate. As you can see, the Tennis Recruiting system compares favorably with other junior tennis ranking and rating systems - although all systems do a respectable job.

We could spend a lot more time exploring these other tournaments and other aggregate data, but we will leave that for future articles.


That's a Wrap...

This article concludes our coverage of Clay Court Championship Week. I hope that you have enjoyed the efforts of all our great content contributors. We have enjoyed offering you expanded coverage this year.

Make sure to check out any article you might have missed - and get ready for our coverage of the Hard Court Championships that start next weekend.


Leave a Comment


More Ranking Articles

Comparing Rating Algorithms
During the U.S. Open we posted information on social media about matchups involving American players - previewing matches by displaying rating and ranking differences and highlighting wins by underdogs. You might have noticed that the various rating systems often agreed on their favorites. But what about the close matches where they disagreed? Let's take a look ...

Behind the Tennis Recruiting Rankings
This week marks the 569th consecutive week that Tennis Recruiting has put out graduation-based rankings of American junior boys and girls. Rankings are front and center on Tennis Recruiting, and people often ask us how our rankings work. Today we describe our ranking system ...

What Is An Upset?
Tennis Recruiting is a website that rates and ranks junior tennis players. One of the questions we get most often from our users is, "What exactly is an upset?" There are many possible definitions of an upset - this article explores the question and puts forward an answer.

 Ranking Article Index |   Subscribe

Nike Air Zoom Vapor Pro Tennis Shoe Review! 🔥 🔥
To purchase this product or for a much more detailed written review, Click Here!
The lifeblood of Tennis Recruiting is its rankings - and our team has been ranking tennis players for more than two decades. Learn about tennis rankings in general - as well as our best-of-breed ranking system.
Recent Tournament Forecasts
Date   Tournament Predicted
12/27 Winter Nationals 75.0%
9/02 U.S. Open 68.4%
8/04 USTA National Champs. 75.2%
7/16 USTA Clay Court Champs. 76.0%
7/02 Wimbledon 72.6%
Click here for the complete list of all of our Tournament Forecasts.
adidas Tennis x Parley | Play for the Ocean
Play for the oceans in the adidas Tennis x Parley range. Our first ever collection for the court created entirely from reclaimed or intercepted ocean plastic.
1/27 Kuzuhara Saves Match Point, Reaches Semifinals of Australian Open Junior Championships; Six Americans Advance to Round of 16 at Les Petits As; Texas…
1/26 Colak, Kuzuhara and Hovde Reach Quarterfinals at Australian Open Junior Championships; Collins Joins Keys in Women's Semifinals; Three More…
1/24 Hovde, Colak and Rahman Advance to Third Round at Australian Open Junior Championships; Collins Through to Quarterfinals in Melbourne; Americans Earn…
1/24 Kuzuhara, Lopez Advance at Australian Open Junior Championships; Keys and Pegula Reach Quarterfinals in Melbourne; Nine Americans Begin Competition at…
1/23 Zheng Wins J1 Title in Colombia; Filin Sweeps Singles and Doubles at J4 in Costa Rica; Five US Juniors Advance to Round 2 at Australian Open; Collins,…
Colette Lewis has covered topflight junior events as a freelance journalist for over a decade. Read her weekly column, follow her on Twitter, and and find more of her daily commentary at ZooTennis.

Page updated on Monday, December 06, 2021
Contact our web team with any corrections