On Legislative Scorecards

To rate or not to rate, that is the question

Every political figure in Idaho claims to be conservative, even Democrats deploy that magic word. Yet the question before voters is: how can you tell if your lawmaker is truly upholding the conservative values you want to see promoted in the Legislature?

Wayne Hoffman and the Idaho Freedom Foundation (IFF) created the Idaho Freedom Index to answer that question. IFF staff rate as many bills as possible using objective metrics drawn from the U.S. Constitution, the Idaho Constitution, and traditionally conservative principles such as limited government. As lawmakers vote on those bills, they generate a score that voters can use to evaluate how conservative they really are.

This IFF white paper explains the somewhat complex math behind legislator scores.

The Freedom Index has drawn criticism from both inside and outside the Legislature, with some accusing it of being biased or not representative of what actually happens in the Capitol. While I do not believe the Index is perfect — more on that below — I think it provides a reasonably accurate big-picture view of where lawmakers stand on conservative principles.

The Idaho Freedom Index is not the only scorecard out there. The Idaho Association of Commerce and Industry (IACI) publishes its own ratings of bills and lawmakers based on the priorities of its member businesses. IACI only scored 13 bills last session, resulting in a very small sample size. However, even that was enough to show a negative correlation with the Freedom Index:

National groups like the American Conservative Union, Club for Growth, and the Institute for Legislative Analysis also publish scorecards of state legislators, but they typically select a small number of bills after the session. Nevertheless, these conservative scorecards often correlate positively with the Freedom Index.

Former state senator Steven Thayn recently wrote a guest editorial which took a harsh stance on the Freedom Index. He suggested that IFF “intimidates” legislators with the scorecard, telling them “how to vote or else.” While I consider Thayn a valued friend and political thinker, I disagree with this take. Having worked at IFF, I’ve seen behind the curtain. While IFF staff take positions on bills — both for and against — and communicate those positions to lawmakers with the understanding that their scores will change based on how they vote, this is no different from any other organization lobbying legislators to vote one way or another. Some might see that as intimidation, but it is simply advocacy.

IFF expanded its metrics for the Idaho Freedom Index this year to 14 questions, now including one regarding family values:

Does it promote the breakdown of the traditional family or the deconstruction of societal norms? Examples include promoting or incentivizing degeneracy, violating parental rights, and compromising the innocence of children. Conversely, does it protect or uphold the structure, tenets, and traditional values of Western society?

I believe this is a positive change. Previously, bill ratings sometimes penalized efforts to protect traditional values if they added regulations to the free market—such as age verification for pornographic websites. This small change will likely result in higher ratings for many conservative lawmakers.

Between that change and a general sense that this year’s Legislature is more conservative than in previous years, we should see improvements in overall Freedom Index scores. Although the House has only voted on 15 rated bills thus far, the average score for House Republicans currently sits at 75.4%, compared to 59.5% at the end of last year. It’s a good start.

The Freedom Index does have some limitations. While I believe the metrics are objective, they are still applied by subjective human beings. There is also a temptation to stack metrics on certain bills, whether to rate them highly positive or highly negative. Consider the difference in ratings between House Bill 1 (+5), a personal bill filed by Rep. Clint Hostetler, and House Bill 93 (+1), sponsored by Rep. Wendy Horman.

H1 received five metrics, all positive. H93, on the other hand, was evaluated using ten metrics — four positive, three negative, and three neutral. Some of the negative metrics on H93, such as the claim that it redistributes wealth, could also be applied to H1, since it, too, proposes refundable tax credits.

EDIT: I had a conversation with IFF’s education director Samuel Lair, who was responsible for the ratings on H1 and H93. He maintains that he evaluated each bill according to the same standard, and I accept that, and did not mean to impugn his academic integrity. I maintain that it’s impossible for IFF as an organization to be perfectly objective, as its ratings are the result of more than one person, each with his own subjective viewpoints. The definition of “redistribution” that I am using is the same that was used by a previous IFF employee in 2017 when rating H67, which eliminated sales taxes on groceries as well as the grocery tax credit. However, Lair is using a stricter definition of the word in his evaluation of both H1 and H93.

Nevertheless, because IFF rates 300–400 bills each session, minor discrepancies tend to even out due to sheer volume. While the difference between legislators scoring 70% and 90% might not clearly indicate who is “more conservative,” the difference between someone scoring 80% and someone scoring 30% is stark.

There is a deeper question, however, about the utility of legislative scorecards. Wayne Hoffman explained that IFF created the Freedom Index to give voters a metric for judging their lawmakers. A side effect, however, is that lawmakers might be tempted to chase high scores by aligning their votes as closely as possible with the ratings.

Conservative thinker Will Chamberlain tweeted last December that report cards “actively incentivize losing”:

If you vote for the least bad realistic option, knowing that if it fails the resulting bill will be worse…the report cards punish you

I think there is something to this observation, especially regarding spending bills. A few years ago, the House rejected a Medicaid budget bill, prompting JFAC to cut $150 million and return a revised version to the floor. Despite the significant cut, IFF rated the new bill exactly the same: -4. Representatives who voted against the first bill but wanted to support the second, perhaps to validate JFAC’s budget reduction, would still be penalized in the Spending Index. On the other hand, those who voted against both risked appearing completely intransigent, potentially undermining their ability to negotiate future bills. This is not to say the bill was rated improperly, as too much spending is too much spending. However, it reveals a structural weakness with the concept as a whole.

Even so, scorecards like the Idaho Freedom Index remain valuable tools for voters. They provide useful data points — even the IACI scorecard offers insights into our elected representatives. However, they are not the end-all, be-all. Scorecards don’t measure how effectively a legislator drafts and carries bills or negotiates with fellow lawmakers in both chambers. They don’t reveal who works the hardest or who operates behind the scenes to advance good legislation and block bad bills.

Voters are sometimes called upon to decide whether they prefer a legislator with a 95% score who votes “correctly” but does little else, or one with a 75% score who actively drafts and shepherds good bills through the legislative process.

By the same token, IFF analyses are valuable tools for lawmakers, but they shouldn’t dictate every vote. Legislators should resist the urge to chase high scores and instead vote according to the Constitution, the will of their constituents, and the dictates of their conscience. The people who write IFF’s analyses are intelligent and experienced, and lawmakers should take their insights into account. However, in the end, legislators are accountable to their constituents — not to any third party.

Other groups, such as the Mountain States Policy Center, approach policy differently. They produce research papers and collaborate with legislators but do not rate bills or score lawmakers. While it may be tempting to choose sides, I believe both organizations provide valuable services to the political process. The more perspectives our lawmakers consider, the better their legislation — and their votes — will be.

In the end, scorecards are a useful tool for both voters and lawmakers, but they should be viewed as just one piece of the larger picture. Use them, but I suggest you resist the urge to make them the sole measure of your elected representatives.