Why does elevation matter when choosing fuel octane?

I live at about 6,000ft elevation and everyone says that the higher octane only improves performance at sea level. Why is that? When I go visit San Diego this summer should I put 91 in??? Even though I usually use 85?