Benchmark Comparisons

When looking at a comparison infographic, sometimes it’s difficult to determine the standard for comparing two or more things. One solution is to apply a benchmark to the chart, but there are many ways to go about it.

Benchmark comparisons in the news

As someone who has a professional interest in the natural sciences, it saddens me to look at the news and discover that not everyone in the United States trusts scientists to do the right thing. More often than not, the media cites misinformation and alleged government propaganda as sources for such distrust. However, I do sometimes wonder what opinion people from other countries have of scientists. Recently, I read a Pew Research Center report about the scientific community’s relationship to the general public, and I was surprised that people in countries like India and the Netherlands have far more trust in their nation’s scientists than the American public has in ours.

On the other end of the spectrum, I found myself wondering why nations like Japan and South Korea are skeptical of scientists. They are among the world’s leaders in technological advances, yet the people don’t seem to trust scientists to do the right thing. When compared to median values, the United States exceeds the benchmark, if only by a slight margin, which gives me hope that the vocal minority that distrusts scientists will one day see their error and learn to trust science.

Another way of showing a benchmark is by comparing positive and negative values to determine favorable perspectives. With the United States presidential election fast approaching, the public is more divided than ever on whether or not this year’s elections will be run well. It is likely that a divide in partisan politics and factors like the current COVID-19 pandemic and ongoing police brutality protests are responsible for this immense discourse.

Based on Pew Research Center’s data, I estimate that conservative-leaning voters believe that the election will be handled well. On the other hand, liberal-leaning voters believe that this year’s election will run into a series of blunders or be subject to manipulative interference. As someone who wants to see Donald Trump and his alleged enablers voted out for the sake of the rest of the nation, I am hoping for the best but preparing for the worst.

What about comparisons related to topics I enjoy?

Even when I am not catching up on the news, I still enjoy looking at comparisons for the things I love. For example, I’ve been playing Square Enix’s Final Fantasy XIV Online (shorthand: “FFXIV”) for about a year, and have a strong interest in joining the small community of players who spend most of their time on high-end or difficult pieces of content. When a new piece of content comes out, one of the first places I go to look for information is FF Logs. Their tools analyze combat data and give players a sense of their overall performance and where they can improve. They also have statistical charts that show performance averages for each of the game’s 17 combat classes (jobs). For example, one of the most difficult “raid dungeons” currently in the game goes by the name of “Eden’s Verse: Refulgence.” (shorthand: “E8”). Raids in FFXIV typically have two modes of difficulty, normal and “savage,” with the exception of specially-themed “ultimate” raids. The community typically pays attention to statistics and ranking for savage raids. In the first few weeks of release, Eden’s Verse: Refulgence (Savage) (shorthand: E8S) proved to be a difficult challenge for most raid groups.

Combat jobs in FFXIV are divided into three major roles: tanks, healers, and damage per second (DPS). The box plot chart above shows an aggregate ranking for job performance in E8S over the course of two weeks across all percentiles. DPS jobs typically put out far greater damage than tanks or healers, with magic-based DPS jobs having the highest damage output. The only exception to that rule is the samurai job, which is typically described as being a selfish DPS job that brings little party support to a fight as it deals heavy hits at melee range.

Viewed from a different perspective– say, comparing the maximum and minimum percentiles for E8S damage– certain trends seem to appear among the 17 jobs, and it reflects in how well the player performs in the fight. The best players in the world generally fare much better and clear the raid faster than players who barely have a grasp of their jobs or exhibit poor performance.

Compared to others who play the same jobs as me (color-highlighted in the chart above), I would say that I am somewhere between the two extremes. However, in the case of the job I play the most, red mage, I tend to lean toward the lower end of the spectrum. I recall running a four-person dungeon once with some of my friends a few months before the COVID-19 pandemic exploded. One of them, who went in with a tank job, had a third-party tool open to analyze everyone’s damage output in real-time (which is against the developer’s terms of service, but Square Enix doesn’t crack down on third-party tools if players don’t discuss them publicly in-game), and once the program crunched the numbers, he uploaded the data to FF Logs and sent me a link to the log over Discord. As I looked it over, he laughed in voice chat and likely shook his head at how low my damage output was. Another one of my friends, a healer who did significantly better than me, pointed out that at least I knew my spellcasting priority rotation as a red mage and the random black mage who got matched with our three-player party did less damage than me. I didn’t care that my damage output was sub-par, though. I was happy just to run content with my friends.