Understanding directional spectra of infragravity (IG) waves composed of free and bound components is required due to their impacts on various coastal processes (e.g., coastal inundation and morphological change). However, conventional reconstruction methods of directional spectra relying on linear wave theory are not applicable to IG waves in intermediate water depths (20–30 m) due to the presence of bound waves. Herein, a novel method is proposed to reconstruct directional spectra of IG waves in intermediate depth based on weakly nonlinear wave theory. This method corrects cross-spectra among observed wave signals by taking account of the nonlinearity of bound waves in order to reconstruct directional spectra of free IG waves. Numerical experiments using synthetic data representing various directional distributions show that the proposed method reconstructs free IG wave directional spectra more accurately than the conventional method. The method is subsequently applied to observations of severe sea-states at two field sites. At these sites, free IG waves are not isotropic and have clear peak directions. Numerical modeling of the wave fields shows that these peak directions correspond to the reflection of IG waves from the shore and/or coastal structures. Additionally, the validity of the underlying weakly nonlinear wave theory of the present method is assessed by a newly proposed method employing bispectral analysis. The bound wave response generally agrees with the theory at the field sites but deviates slightly for energetic sea states. The applicability of the present method on a sloping bottom is further discussed by an analytical solution.