Why is it hate speech?

In Colorado, a group of White Supremacists have adopted a mile of highway and had a sign placed up to say so. The state tried to block the sign but the Anti-Defamation League told them they couldn't so the state relented.

According to the White Supremacists, "We want to let them know that we're here and we do good things," Unit leader Neal Land told FOX 31 News. "We're upstanding citizens, try to be good people, and try to portray ourselves that way."
But, according to the ADL, "Courts around the country have allowed white supremacists to sponsor highway signs," says Anti-Defamation League Director Bruce DeBoskey. So although the Anti-Defamation League couldn't be more opposed to the Nazi movement, it advised the state to put the application through.

"To have our freedom we have to have all kinds of speech, and this is a case where hate speech is protected," DeBoskey said"

So why is it hate speech? Is it any worse, or is it as bad, as what Elijah Muhammad, the Messenger of Allah, head of a stern, demanding, disciplined black-supremacist religious sect called "the Moslems stands for?

In a Time Magazine article the group had this to say at a rally in Manhattan, "Every white man knows his time is up," snapped the frail-looking Negro in the embroidered pillbox to 5,500 Negroes packed into Manhattan's St. Nicholas Arena one hot afternoon last week. "I am here to teach you how to be free. Yes, free from the white man's yoke. We want unity of all darker peoples on the earth. Then we will be masters of the United States, and we are going to treat the white man the way he should be treated."

And I guess, this is not hate speech?