Archive for October, 2007

The Future Of Data and Analysis

Monday, October 29th, 2007

Technology is really about information, and our ability to interact and store information has been making some major changes for the past several decades.

Those of us who are researchers have been blessed in the past several years with an explosion of available information for examining nearly every topic we want to look at.

But at the same time, there has been a shift away from objective analysis toward studies completed by organizations with a vested interest in the outcome.

This has resulted, as noted at Overcoming Bias, in a crisis of faith in the use of statistics. No one believes any numbers that are released because they figure that the people who did the analysis have a hidden agenda. This is the most demoralizing aspect of being an analyst.

The problem is essentially that the analysis process is still closed. We don’t trust the numbers because we don’t know the process that went into obtaining them. Until the process of analysis is opened to a public that can understand and take part in a dialog about the methods, we can continue to write analytical work off as a waste of money and resources because someone else will always have a study saying the opposite.

We need the democratization of data analysis.

And I have high hopes for the future. One of my ideas has been to create an online data warehouse and analysis website, and a few groups such as Swivel and Many Eyes are doing just that.

The services are still fledgling, but in a fairly short time I think we’ll see the ability to perform analysis via the web, and to upload and share data. Hopefully this will result in the standardization of data sets and eliminate the monkey work of downloading a text file and formating it in excel.

I cry when I think how many people have done the very same thing with the very same data.

For a time there will be a lot of confusion, attacks, and some really bad work, but ultimately a community of devoted analysts performing essentially open and peer reviewed work will develop. Because it’s been through trial by fire from all sides of the issue, it will be the most objective approach to messy data analysis that we have ever obtained.

The new analytical hotshots will be the people who have weight with communities at Swivel and Many Eyes and others and are known for doing quality work. These people will engage in very public analysis and will be extremely valuable to everyone wanting analysis that can be seen as objective.

Hopefully that means that shoddy and lazy work will slip to the sides and we can regain confidence in numbers that we see.

I guess it could go the other way. That people don’t want objective analysis and we’ll shun those whose results we can’t know ahead of time. We’ll see even more extreme partisanship.

But that way lies certain death.

-zot, pretending to be a futurist.

Support The Decision Strategist.

Popularity: 56%

Vote for this post on: These icons link to social bookmarking sites where readers can share and discover new web pages. Digg del.icio.us Furl Reddit StumbleUpon Technorati DZone

Decision Making Roundup

Monday, October 29th, 2007

Some interesting decision making articles:

More later.

-zot.

Support The Decision Strategist.

Popularity: 27%

Vote for this post on: These icons link to social bookmarking sites where readers can share and discover new web pages. Digg del.icio.us Furl Reddit StumbleUpon Technorati DZone

The Dangers of Habit and The Difficulty of Creating Change

Saturday, October 27th, 2007

I spend a lot of my more introspective moments thinking about the interaction between decision making and habitual behavior. We tend to think that we face a decision from a conscious ‘free will’ standpoint, but the evidence suggests that our decision making process are much more like water running down a mountain: as time goes on we create stream beds and eventually rivers, and it gets harder and harder to act non-habitually.

This explains why some people, particularly as they get older, have only a few topics they revisit in most interactions. In my own life I know people who tirelessly revisit:

  • The non-existence of god
  • The conspiracy of the elite
  • Coming judgment day
  • Inequality and injustice
  • Desiring wealth

In a way these are things that the person is passionate about, and so are a good thing. But at what point are we simply rehashing old conversations without learning anything new?

I think non-habitual actions as a human being are one of the hardest things to achieve. It is something akin to the zen idea of being fully present in the moment, but still fighting the biological impulse to react to a situation in the same way as before.

Of course, some of these reactions are essential for survival and useful for everyday life. But in how many situations have we adopted a behavior that works but is badly suboptimal, or worse, a behavior that is destructive?

Attempting to change ones actions is one of the most difficult things we can do. At times I worry that it’s a futile effort. Even the concept of non-habitual action has problems.

As I get older, and as I see older people who have been stuck on an idea for several years only become more invested in it, I fear that it is an unavoidable trait of humanity that we eventually sink into a more or less automated life.

But there are a few role models that give me hope. Older men and women who are still learning new things and still excited about trying something different. I guess I’d be happy with just being able to change my more destructive habitual behaviors and reactions.

Sometime soon I’ll do through the painful process of looking for habitual choices and interactions that I want to change. Already I know there are topics I’ve been stuck on for several years. Maybe my friends can help point them out to me.

-zot

Support The Decision Strategist.

Popularity: 33%

Vote for this post on: These icons link to social bookmarking sites where readers can share and discover new web pages. Digg del.icio.us Furl Reddit StumbleUpon Technorati DZone

Sell Your House Or Let The Bank Foreclose?

Wednesday, October 24th, 2007

Disclaimer: I am not a financial professional and this should not be taken as financial advice. I don’t own a house either. I am only discussing what I would do in a situation in which housing prices are dropping, supply is at an all time high, and I was facing a mortgage rate reset.

A while ago I talked recently about the financial and emotional aspects of buying a house, but in today’s market, it’s probably better to consider the decision to sell your house. Specifically, the decision to sell your house if you are having financial problems.

The situation that a lot of people are facing right now is one in which they face reseting mortgage rates that will increase their monthly mortgage payment above what they are capable of paying. The chart below is from an IMF report (pdf link) on world credit that I found through Dr Housing Bubble.

Mortgage Reset

Not all of that is going to default, but if these numbers are any indication, a whole lot of people will be wondering what to do when they get their new higher mortgage bill.

It’s a tough situation. If housing prices were still rising, you could sell your house, pay off your mortgage, and be free and clear.

But housing prices are not rising, they are falling; in some cases drastically. There are really only two options:

Option 1: Sell you house

If someone is in danger of defaulting, they can sell their house. In today’s market they probably can’t sell it quickly or for as much as they paid for it. Since the purchase is recent, they probably have very little equity built up. Selling at a lower price means they are still going to owe the bank the difference between what they paid on the house and what they were able to sell for. That doesn’t include all the commissions and cuts that have to be paid, so it will actually be slightly more than that.

In other words, they are in the same position as they would be if they had rented, only they’ve been making higher monthly payments and still owe a sizable chunk due to the difference in prices. On the other hand, their credit isn’t absolutely terrible, since they avoided foreclosure.

Option 2: Cut and run

Instead, this hypothetical person could just stop making payments and let the bank foreclose on the house. Credit score would go down the toilet for several years, but they’d walk away debt free.

In fact, the result of this option isn’t that different from the first option: they are just exchanging the leftover debt from the difference between the buy and sell prices for a terrible credit score.

So then the question is how much debt is it worth incurring to avoid a bad credit score?

Maybe one of the personal finance bloggers could answer this question. I’m inclined to say that no amount of debt is worth it, and so the second option is an obvious better choice.

Sunk costs at play

Though there is some suggestion of people walking away from their houses (maybe from patrick.net?) , sunk costs are coming into play here in a big way. People will consider all the money they’ve already put into the house, and won’t want to lose that. In fact, the sunk cost effect is much stronger in this situation because of the emotions involved in the idea of owning your own house. The effect will be strong enough to keep most people from giving up on their house until they are forced to.

But if you’re facing certain foreclosure, whether now or a couple of years, and you won’t be able to sell your house for more than you bought it, it makes more sense to me to stop paying now (why sink more money into a lost cause?) and walk away.

qualifier: I don’t really know how the foreclosure process works. Are you still liable for whatever value the mortgage has that the bank can’t recover? If so then it doesn’t seem like it matters much what you do.

-zot

Support The Decision Strategist.

Popularity: 83%

Vote for this post on: These icons link to social bookmarking sites where readers can share and discover new web pages. Digg del.icio.us Furl Reddit StumbleUpon Technorati DZone