Show of Hands

Is it rude for an atheist/agnostic/etc to not participate in dinner prayer (or grace) while in a Christian's home?