Labor markets in least developed countries are characterised by small wage sectors and low productivity and wages. Using household level data for many countries in Sub-Saharan Africa, we document that they also show a greater level of wage dispersion. This is in stark contrast with the positive correlation between income mean and income inequality for the same countries. We propose a labor search and matching framework with entry costs and firm heterogeneity that delivers endogenously the negative correlation between (i) wage dispersion and size of the wage sector and (ii) wage dispersion and wage mean. We also show that this model can reconcile the differences between wage and income inequality by accounting for labor reallocations between wage and self-employment sectors. We focus on three channels to explain these phenomena in Sub-Saharan Africa: entry costs (e.g. regulations, financial constraints to starting a business), differences in countries’ underlying productivity distribution (e.g. due to lower capital intensity, or poor infrastructure) and labor market frictions. A numerical simulation shows that the model does a good job in reproducing the main stylised facts and reveals how these different constraints interact to reduce labor market performance.