You were right to slap our hands, and here's why...
Tomatoes are surprisingly delicate, we've discovered! Harold McGee in his book On Food and Cooking explains that refrigerating tomatoes damages the membranes inside the fruit walls, causing the tomato to lose flavor and develop the mealy texture we associate with mid-January grocery store tomatoes.
So the best place to store tomatoes is, indeed, on the counter top at room temperature. They actually continue to develop flavor until maturation peaks a few days after picking.
If you (like us!) have been keeping your fresh tomatoes in the fridge, try letting them sit out at room temperature for a full day before eating them. McGee says that some of the enzymes will re-activate and boost the flavor back up.
The moral of the story is this: tomatoes are meant to be eaten, not hoarded. And preferably, they should be eaten right away, immediately, as soon as you're home from the farmer's market, and without delay!