Are you sure he’s talking about words such as “gonna”? What he means by casual chatter might be dfiffernt from what we’re talking about here. As an editor, when I tell a writer he has too much casual chatter, I mean he has too much dialogue that doesn’t move the story forward.
Dialogue really serves three purposes: 1. Move the story forward. 2. Reveal character. 3. Add verisimilitude.
But even when purposes two and three are being served, the first must be served, as well. In this sense, casual chatter is something that needs eliminated completely.