No, not all bird bones are hollow. Birds have a mix of pneumatic (air-filled, "hollow") bones and marrow-filled bones, and the exact pattern varies significantly by species and even by individual bone within the same skeleton.
Pneumatic bones are hollow or partially hollow because they connect to the bird's respiratory system via air sacs, which extend into the bone cavities through small openings called pneumatic foramina. This is most common in the skull, vertebrae, sternum (breastbone), pelvis, and the large wing bones (humerus) of many flying birds, especially large soaring species like albatrosses and vultures, where the weight savings aid flight. The hollow bones typically have internal strut-like reinforcements (trabeculae) that maintain strength despite being lightweight, similar in principle to engineering trusses.
However, many bones remain filled with traditional bone marrow, particularly the smaller bones of the wingtips, feet, and lower legs. Marrow is essential for producing blood cells, so birds cannot make every bone hollow without compromising this vital function. Diving birds like penguins and loons actually have notably dense, less pneumatized (or even solid) bones, which helps reduce buoyancy and makes it easier to dive and swim underwater. Flightless birds such as ostriches and emus also show variation — some have pneumatized bones in certain areas despite not flying.
So the degree of bone hollowness in birds correlates more with lifestyle (flying, diving, running) than with being a bird per se. It's a common misconception, popularized by simplified explanations of flight adaptations, that a bird's entire skeleton is hollow and lightweight; in reality it's a strategic, species-specific mix of hollow pneumatic bones and solid marrow-containing bones.