I'd say a good place for a lunch date is somewhere casual and comfortable. Maybe take the lunch a little later so that it can be slower and not so hectic.

I'm go to a vegetarian/vegan cafe for lunch that has couches and a very laid back feel to the place. Of course, I'm not going on a date, but I am getting together with a friend that I want to be able to talk with and really spend time with... not just shoveling food in my face for my lunch hour.