In this paper, moisture absorption/desorption and hygroscopic swelling of silicone/phosphor composite film and epoxy mold compound (EMC) used in chip-on-board (COB) LED packaging is studied with in-situ characterization. Material properties, including diffusivity, activation energy, saturated moisture concentration and the coefficient of hygroscopic swelling, are extracted from the experimental data. For silicone/phosphor film, the moisture diffusion can be described by the conventional Fickian diffusion model. However, the saturated moisture concentration increases with temperatures under the same relative humidity condition. The dependence of saturated moisture concentration on temperature is characterized by an Arrhenius-like equation, and the net absorption heat is found to be 8–11 kJ/mol. Furthermore, the silicone/phosphor composite film shows reversible diffusion process during absorption and desorption cycle, which confirms the Fickian behavior of moisture absorption. Moisture diffusion in EMC, on the other hand, exhibits non-Fickian diffusion, and thus, a dual stage diffusion model is adopted to fit experimental data. Moreover, EMC shows partially non-reversible moisture absorption during absorption and desorption cycle, which is consistent with the dual-stage model description. In hygroscopic swelling testing, silicone/phosphor film shows a permanent swelling during moisture absorption despite that the moisture absorption is completely reversible. EMC shows partially reversible swelling after moisture is removed, which indicates the residual moisture in material. Further study is needed to understand the mechanism of permanent hygroscopic swelling for silicone/phosphor composite during moisture absorption and desorption.