The filler in the asphalt mixture is essential since it plays a significant role in toughening and stiffening the asphalt. Changes in filler type can lead the asphalt mixtures to perform satisfactorily during their design life or degrade rapidly when traffic and environmental effects are considered. This study aims to assess the impact of filler types such as limestone dust (LS) and hydrated lime (HL) on Marshall characteristics and moisture damage in asphalt mixtures. Three different percentages of HL were employed in this study to partially replace the LS mineral filler: 1.5, 2.0, and 2.5% by aggregate weight. Furthermore, a control mixture was created with 7% LS by overall aggregate weight for the wearing course layer. The Marshall method was used to obtain the optimal asphalt content and the asphalt mixes' volumetric properties. The optimum asphalt content was used to prepare the asphalt concrete mixes, which were then tested for moisture damage resistance using the indirect tensile strength (ITS) and the index of retained strength (IRS). The findings demonstrate that resistance to moisture damage can be significantly enhanced by partially substituting HL for the LS  filler. This was verified by the fact that the optimum increase in the tensile strength ratio (TSR) was 7.29% at 2.5% of HL, and at the same HL percent, the greatest rise in the IRS was 9.81% compared with the control mix.