We present JWST NIRCam imaging targeting 13 z ~ 3 infrared-luminous (L_IR ∼ 5 × 10¹²L_⊙) galaxies from the ALESS survey with uniquely deep, high-resolution (0 . ″ 08-0 . ″ 16) Atacama Large Millimeter/submillimeter Array 870 μm imaging. The 2.0-4.4 μm (observed frame) NIRCam imaging reveals the rest-frame near-infrared stellar emission in these submillimeter-selected galaxies at the same (sub)kiloparsec resolution as the 870 μm dust continuum. The newly revealed stellar morphologies show striking similarities with the dust continuum morphologies at 870 μm, with the centers and position angles agreeing for most sources, clearly illustrating that the spatial offsets reported previously between the 870 μm and Hubble Space Telescope morphologies were due to strong differential dust obscuration. The F444W sizes are 78% ± 21% larger than those measured at 870 μm, in contrast to recent results from hydrodynamical simulations that predict larger 870 μm sizes. We report evidence for significant dust obscuration in F444W for the highest-redshift sources, emphasizing the importance of longer-wavelength MIRI imaging. The majority of the sources show evidence that they are undergoing mergers/interactions, including tidal tails/plumes—some of which are also detected at 870 μm. We find a clear correlation between NIRCam colors and 870 μm surface brightness on ∼1 kpc scales, indicating that the galaxies are primarily red due to dust—not stellar age—and we show that the dust structure on ∼kpc scales is broadly similar to that in nearby galaxies. Finally, we find no strong stellar bars in the rest-frame near-infrared, suggesting the extended bar-like features seen at 870 μm are highly obscured and/or gas-dominated structures that are likely early precursors to significant bulge growth.