The criticism of selfishness, or the closely related term of disparagement, 'arrogance', is generally aimed at a person who does not need to cultivate the goodwill of those that criticise them. Hence, social association is usually between those of similar economic means, where there is sufficient need for such cultivation as to preserve social calm. By contrast, when there is disparity, the richer party will tend to neglect the poorer's interests, resulting in justified resentment issuing in accusations of selfishness (and, or, arrogance).
In any section of the wealth gradient, I would hazard the guess that there is more variation in wealth from richest to poorest as you move up the wealth curve, with the result that accusations of selfishness are more common than amongst sections lower down the curve. Hence increasing degrees of solidarity are found as you move down the income gradient. Poorer people aren't intrinsically nicer; they just have to be.