Social science research demonstrates that dispersal policies and restrictions on the freedom of residence have inhibited refugees’ socio-economic integration. The dominant explanation is that such policies prevent refugees from moving to places where they can employ their skills most fruitfully. However, previous studies of refugees’ actual residential choices in Europe provide little evidence that refugees move to places with good employment prospects. The combination of negative effects of residence restrictions and emerging evidence of disadvantaging secondary migration forms what we call the ‘refugee mobility puzzle’. In this study, we address this puzzle and ask: What attracts refugees to deprived areas, and can their seemingly unfortunate residential choices be understood as moves to labor market opportunity after all? Empirically, we draw on the IAB-BAMF-SOEP Survey of Refugees, track the location of more than 2,500 refugee respondents, and estimate discrete choice models across all German counties and postcodes. Our results confirm the existence of the refugee-mobility puzzle and complicate recent critiques of dispersal policies and restrictions by suggesting that refugees’ need for affordable housing and their desire to be close to (co-ethnic) friends and family may turn into an unintended lock-in factor in the mid- and long-run.