vault backup: 2025-08-07 15:12:16

2025-08-07 15:12:16 -04:00
parent 8cc5b7ca70
commit baca4a962d
8 changed files with 239 additions and 129 deletions
@@ -0,0 +1,126 @@
+---
+id:
+aliases: []
+tags: []
+---
+# Calibration Questions
+
+## Examples
+
+### Boolean
+
+> The melting point of tin is higher than the melting point of aluminum.
+
+> In English, the word "quality" is more frequently used that the word "speed".
+
+reductive (used more frequently where?)
+
+> Any male pig is referred to as a hog.
+
+reductive (referred to by whom?)
+
+> California's giant sequoia trees are named for an early 19th century leader of the Cherokee Indians.
+
+reductive
+
+> The Model T was the first car produced by Henry Ford.
+
+reductive (Henry Ford didn't produce cars)
+
+> When rolling 2 dice, a roll of 7 is more likely than a 3.
+
+facile
+
+> No one has ever been reported to have been hit by any object that fell from space.
+
+reductive (reported by whom?)
+
+> Sir Christopher Wren was a British anthropologist.
+
+> Pakistan does not border Russia.
+
+unnecessary negative form, otherwise good.
+
+> The Navy won the first Army-Navy football game.
+
+should specify the official event name, otherwise good.
+
+> The paperback version of the book "The Da Vinci Code", as of July 2007, still ranks in the top 500 bestselling books on Amazon.
+
+obtuse phrasing, dated topic, otherwise good
+
+> Italian has more words than any other language.
+
+reductive (what is a word? what dialect?)
+
+> The month of August is named after a Greek god.
+
+borderline facile, reductive
+
+> The deepest ocean trench is deeper than the Grand Canyon.
+
+facile
+
+> Abraham Lincoln was the first president born in a log cabin.
+
+deceptive phrasing
+
+> As of July of 2007, more people search Google for  "Harry Potter" than "Hillary Clinton" (according to GoogleTrends).
+
+obtuse phrasing, dated topic, otherwise good
+
+> The population of Alabama is higher than the population of Arizona.
+
+borderline facile, deceptive phrasing
+
+> No category 5 hurricane hit the US in 2004.
+
+> The UK is among the top 10 largest economies in the world (by GDP).
+
+> The movie Forest Gump has grossed more to date than E.T. The Extra Terrestrial.
+
+obtuse phrasing, dated topic, otherwise good
+
+### Interval
+
+> What percentage of bronze is typically made of copper?
+
+reductive
+
+> How many countries have at least one McDonald's?
+
+As of when?
+
+> How many employees did eBay have in the first quarter of 2006
+> What was the population of Miami (within the city limits, not the entire metropolitan area) in 1990?
+> How many casualties did the French suffer in the Battle of Waterloo?
+> What is the range in miles of a Minuteman Missile?
+> What is the percentage of IT jobs in the US were unfilled in 1997?
+> The Supremes' (with Diana Ross) song "Stop! In the Name of Love" was how long? (minutes, seconds)
+> How many undergraduates attended Cambridge in 1990?
+> If you could jump 50 feet straight up into the air, how many seconds would you be airborne before you landed?
+> How many gallons are in a bushel (they are both measures of volume)?
+> How many sovereign rulers has England had in the last thousand years?
+> If the air temperature was 5 degrees below zero (Fahrenheit) and the wind speed was 15 mph, what would the temperature adjusted for wind-chill be?
+> Average cost of testing in software development is what percentage of total project costs?
+> On average, if a software development project was projected to take 17 months, it actually takes how many months?
+> How many meters tall is the Sears Tower?
+> How many gold medals did Jesse Owens win at the 1936 Berlin Olympics?
+> In 2005, the average combined MPG for all US cars and light trucks on the road was how much?
+> The average house in the United States uses how many gallons of water per day?
+> What was the average price in the United States of a house sold in 2001?
+
+## Writing Good Calibration Questions
+
+A good calibration question should not feel like it could be a "trick" question.
+
+Definitions/terminology are _always_ contentious,
+questions based on them always feel deceptive.
+
+Interval "questions" should describe the quantity
+rather than phrase it as a question.
+
+## Strategy for Answering Calibration Questions
+
+Confidence should never be less than probability of picking randomly
+(50% for true)