to be stumped -

how about

wrong side of the tracks -

rule of thumb