Handeliodendron bodinieri (H. Lév.) Rehder is a rare, endangered, and therefore, protected tree species native to China. However, there are serious limitations to the effective protection of the species, including a low seed germination-rate and difficult storage due to a high seed oil-content. Here, we evaluated the feasibility of ultra-dry seed storage and its effects on seedling growth. We used the silica gel method to prepare ultra-dry seeds with different moisture contents to find an optimal moisture content range (2.54%–4.77%). Ultra-dry treatment improved storability of H. bodinieri seeds. Furthermore, seeds with a moisture content of 4.77% stored at room temperature, and seeds with a moisture content of 3.97% stored at 4 °C yielded the best results. Priming with an appropriate concentration of polyethylene glycol had a certain repairing effect on ultra-dry stored seeds and improved seed vigor, with a two-day priming treatment with 20% polyethylene glycol having the best effect. Finally, compared with sand storage at 4 °C, ultra-dry storage promoted seedling growth and root development; furthermore, it alleviated storage damage to H. bodinieri seeds, promoted soluble sugar and soluble protein accumulation, and increased seedling nitrogen, phosphorus, and potassium uptake. Therefore, ultra-dry storage can be effectively used to preserve H. bodinieri seeds. Specifically, low-temperature storage of ultra-dry seeds with a moisture content of 3.97% enhanced H. bodinieri seed vigor, and seedling growth and development.