07 July 2011

Lending Experiments

Via Arnold Kling:
John Caskey | Payday Lending: New Research and the Big Question [PDF]

In an ideal experiment, one would randomly grant payday loans to a group of applicants and randomly deny the loans as well as close substitutes to a similar group of applicants. One would then track indicators of financial stress over time across the two groups.
This is true not just of payday lending, but all lending.

Predicting who is a good credit risk for a mortgage is devilishly complicated by the fact that your training set is highly skewed. You have for more examples of people who got loans and paid them back than you do people who defaulted. We could have much better models of credit risk if there were as many of the latter as the former, but who's going to give bad credit risks money just for the sake of better data analysis and machine learning?  And of course you have no way of knowing the difference between people who were turned down and would have paid and people who were turned down and would have defaulted, so there is no way of differentiating the true negatives from the false negatives.

2 comments:

  1. > who's going to give bad credit risks money just for the sake of better data analysis and machine learning?

    D.E. Shaw.

    Around the time that I was there, the firm was routinely doing multi-million dollar experiments to see if correlations they THOUGHT existed in the data REALLY existed in an actively traded market.

    ReplyDelete
  2. I stand corrected. My hat is off to them for being smart enough and ballsy enough to invest money in gaining knowledge so directly.

    ReplyDelete