requesting a timeout to say grace?

I'm mostly good at asking whomever I'm eating with for a timeout so I can say a silent grace over my food -- when I'm in a one-on-one situation, anyway. I've mostly stopped telling the other person that they can go ahead and eat, because people always reply by insisting that no they'll wait. I'm not good at timeout-requesting when I'm with a group, though -- which is problematic because I'm really bad at tuning out noise around me. I went to a luncheon today, and when I arrived, a couple people were already talking, and I sat down and bowed my head and folded my hands in front of my face and the longer I sat there trying to say grace, the more I wanted to either pick up my food and move to a quieter part of the room to say grace or literally ask them to stop talking for a minute.

So the question I posed to facebook: Is it gauche to, in a secular setting, ask the people eating with you to stop talking so you can say a silent grace over your food?