HfO2 synthesized by atomic layer deposition (ALD) can be used as a passivation material for photodetectors. This paper shows a significant reduction of density of interface traps at the Si/SiO2 interface using ALD HfO2. This is explained by a chemical passivation effect due to presence of hydrogen from water used in the ALD process. Furthermore, ALD HfO2 layers appear negatively charged which generate an additional field effect passivation. The impact of the SiO2 underlayer is also discussed by comparing a chemical silicon oxide to a standard thermal silicon oxide. It is shown that chemical silicon oxide can act as a reservoir of hydrogen atoms which helps to reduce the density of defects close to the Si/SiO2 interface. This result demonstrates the importance of the surface preparation before the ALD of HfO2 in the passivation scheme. Finally, this work shows the correlation between negatively charged defects and Si–O–Hf bonds at the SiO2/HfO2 interface. A passivation stack composed of chemical oxide permits to reach both a low density of interface traps (∼1.0 × $10^{11}$ $cm^{−2}$$eV^{−1}$) and a negative charge density (∼−1.0 × $10^{11}$ $cm^{−2}$). This stack provides both chemical and field effect passivation of the silicon surface.