The automatic assumption that he's a slave completely boggles my mind, and the fact that 'clean looking' should be 'scruffy and rugged' is unbelievable. There were slaves pre-Civil War that were very well put together with great upbringing. Frederick Douglas, for example, was a social reformer, orator, writer, and statesman.
he doesnt have to be a slave...most of the wild west culture we know of today was set after the civil war ended and thus was when slaves were free men ...or he could have been the son of a wealthy slave owner who then decided to buy him free or he earned his freedom. history for teh win