I have two devices that work pretty much the way you want, I think.
Both are android devices- one is a Galaxy Tab, one is a rooted Nook Color. On the desktop of each, I have buttons generated from Home Buddy (an Android client for Vera) and I just hit the button to activate a scene. Works really well.
For the money, the Nook Color is a perfect device for this kind of thing. Although it doesn’t “wake up” when you pick it up, there’s a hardware button smack on the front. You can also disable the stupid swipe to unlock, so it’s almost exactly like a touchscreen remote control should be.
In fact, it’s even better than your usage model- with buttons on the desktop, you don’t even have to load the application before controlling the scenes.
Best Home Automation shopping experience. Shop at Ezlo!