In recent years there has been growing public and political opposition against the principle of free movement of labor within the European Union (EU). Concerns are often based on the belief that immigrants hurt residents' labor market opportunities and they are particularly pronounced toward the mobility of nationals from countries that joined the EU since 2004. In this paper, we provide the first evaluation of the labor market effects of an increase in immigration on neighboring markets that resulted from the EU Eastern enlargement of 2004. Our empirical strategy exploits the fact that municipalities closer to the border received larger shares of immigrant workers after 2004 due to lower commuting costs. Relying on social security data on the universe of workers in Austrian municipalities within commuting distance to the new EU Member States from 1997 to 2015, we first show that the share of nationals from the new EU Member States among all employees increased by a factor of four over our observation period and that this increase is larger in municipalities closer to the border. Second, comparing changes over time in labor markets closer to the border to those further away within regions, we observe for subgroups of resident workers that their employment decreases relatively faster in municipalities closer to the border after 2004. This negative effect tends to be more pronounced in blue-collar occupations and for non-Austrian workers.