Is "Folks" a racial slur?

I just listened to an old You Tube video of a "debate" between Dr. Lemont Hill and Bill O'Reilly. Torture as I'm sure you guessed. However, both men slung around the terms "white folks" and "black folks" to the point that we could have held a drinking game and been pounded under the table.

But it got me to thinking. Is using the term "folks" a slur? Sure, we use it in both directions, but it seems a terribly antiquated Southern colloquialism and a rather untoward vestige of a time when (especially) black people were diminutized (ok! So I made that word up) by patronizing an entire community. "Folks" was a word used in polite company to describe a people that aren't as smart as you. When people say "folks", I think of under educated, blue collar, margainalized members of our communities. "Folks" are homespun yokels, which is just a stone's throw from redneck, hillbilly, and white trash. I won't even enter the realm of synonyms that might be used to describe members of the black community if you get my drift.

So I ask. Is "folks" the polite man's racial slur?