Populations of tree species with a wide geographic range, such as silver birch (Betula pendula Roth), show genetic specialization to native environments, while maintaining high phenotypical plasticity. Accordingly, assessment of local specialization is essential for adaptive management. The aim of the study was to detect geographic patterns of local adaptation of growth and stem quality based on two open-pollinated progeny trials in Latvia testing local material. Two provenance regions differing by continentality were distinguished, which also differed in genetic control of growth traits, likely originating from the post-glacial recolonization of vegetation and subsequent natural adaptation. Heritability of the traits was estimated for each of the distinguished regions, indicating differing patterns of genetic adaptation and potential for future selection. Trees from the more continental inland showed superior growth and possessed higher heritability. The coastal provenance region showed slower growth and intermediate heritability of the respective traits. Moderate to high heritability for stem quality traits was estimated irrespectively of region. Overall, better growth and higher heritability suggests that anthropogenic selection within the best inland provenances may constitute better performing and adaptable breeding population compared to the coastal one. Still, overlapping phenotypical variation and heritability of quality traits implies improved stemwood quality for plywood regardless of the provenance region. High adaptive capacity of silver birch genotypes suggests ability to cope with climatic changes, highlighting its potential for climate-smart forestry.