White House College Scorecard Suggestions

The White House asked for some feedback on their proposed "scorecard" for higher education cost and value which is intended "to make it easier for students and their families to identify and choose high-quality, affordable colleges that provide good value." Below are their questions and my (quick, off the top of my head, answering-an-online-survey level of analysis) responses.


What information is absolutely critical in helping students and their families choose a college:

You shouldn't be asking this question here. It's a researchable, empirical question. First, on what basis DO people decide? Then, to what degree do they have the appropriate information to do so?

As someone who studies things like this, I don't think the info presented here as it is here presented will provide much added value or better decisions. In terms of presenting information, probably better to summarize in simpler terms: "On metric one, college X is above/at/below average for it's sector." But then don't just stop there -- we also need to global comparison because people don't get how the sectors vary.

Note that costs are in fact a distribution and presenting average after grants still leaves family very much in the dark if they've no way to know where they're likely to fall on the distribution.

Graduation rates does not suffer from this problem.

Percent of loan repayment is too crude to be useful. It's useful for a banker who may want to finance loans for a student at a given school, but very unclear how this number helps student/family shopping for a college.

Average loan amount is useful.

As important as earning potential is, it's a really stupid number here. Just do a tiny bit of due diligence and you'll see screamingly wide variations across majors, careers, and even within majors. Lawyers, for example, have a certain average starting salary, to be sure, but really big range of variation. Frankly I think putting a single number or even a distribution of incomes next to the name of a school would be nothing but phony quantification. Either that or have a really big footnote explaining statistical significance of differences in means.

What other information would be helpful:


Rather than average loan amount and discount rate what would be useful would be ratios. Tell me (1) list price cost of attendance is X; (2) distribution of discounts is ... and (3) range of debt at graduation is ...

Interesting that you don't really have any room for general comments on doing this at all. You are going to end up diverting an incredible amount of resources toward a project that will in all likelihood produce at best some only moderately useful numbers with huge error bars on them. You will feed into the illusion that choice produces improvement (can you cite any actual evidence?). And you will do absolutely nothing that actually lowers or controls costs, increases graduation rates or lowers indebtedness. In short, not a drop of innovation here. Lots of window dressing, but very little that deserves the name policy.

I'm left wondering why this administration is so confident that "better than the alternative" will continue to be a reason people like me support you.

Does the scorecard cause you to think about things you might not have otherwise considered when choosing a college:

Not in the slightest. It makes me think that whoever made it up has never actually been through the process. It reads more like it is informed by a need to respond to conservative activists who are trying to make hay about higher education. As an Obama supporter and contributor I have to admit it's really a little bit embarrassing to read this as part of the administration's policy proposal. If you can't do better than this, I wonder how bad it would really be to have a republican in the WH as well as in control of congress.


How should this version be modified for 2-year colleges:

Look, it's pretty obvious that there are two issues with two year colleges: (1) to what degree does it lead to successful and timely completion of a four year degree, OR (2) to what degree does it yield serious, usable job training.

So, a start would be to provide rate of students who seek admission to four year who actually graduate from a four year. But really easy to get garbage data on this if you don't set up the categories and the tracking really smart.

On the job side, again, gonna be really serious data quality problems that will likely as not make the information worthless (mostly because you are going to see massive variation from program to program WITHIN schools). That said, let's start with simple "how many people are working in a full-time non-temporary job in or related to the field of their AA degree within X years?"

How should comparison groups for colleges be made? What are important things to consider in grouping institutions together that serve similar students:

Catch-22 here. You are asking people to choose -- if you separate it out too well, the really important thing gets lost: we want people to better understand what the different "rungs" represent. One of the big crimes in higher education is that crappy institutions with minimal value added get to promise people a college degree. And if you only compare within groups each one gets to, in a sense, set the standards. What you need is a tool that more clearly lets people see the payoff differences between the tiers (to the degree there are some).

A most important thing that you'll probably leave out is the effect of what you bring to college on the college outcomes. Huge naivete in college assessment world that the college output has only to do with what the college did. Gigantic effects of origins still at work in higher education. Just be sure your new tool doesn't simply do more to perpetuate the myth.

What search and comparison features would you like the online tool to have:

Something that shows schools in context and behind that groups in context (where does this school sit within its group and where does its group sit in the larger picture).

What should we call this tool? Would a different name better explain the service being provided:

One name would be "Republican Higher Education Policy as Adopted by Obama Administration."