Author summary The central nervous system (CNS) is one of the most common extra-respiratory tract sites of infection for influenza A viruses. In ferrets—an animal model used to study the pathogenesis of influenza—highly pathogenic avian influenza H5N1 virus can enter the CNS via the olfactory nerve, resulting in the development of a severe meningo-encephalitis. In the present work, we evaluated the evolutionary dynamics of the virus populations entering and spreading throughout the CNS. We show that once inside the CNS, H5N1 viruses can acquire mutations that increase the polymerase activity in vitro. In vivo, the virus bearing these mutations retained its capacity to infect the CNS but showed reduced spread to other anatomical sites. Analysis of virus populations revealed that infection from the nasal turbinate to the olfactory bulb did not present a genetic bottleneck, suggesting a diffusive passage of viruses from the nasal cavity to the CNS. Inside the CNS, specifically in the brainstem, we found signs of positive selection. These findings support the idea that H5N1 viruses can invade the CNS efficiently via the olfactory nerve, and have the potential to adapt to the CNS.