We have investigated electromigration process at metal nanojunctions by introducing a novel spectroscopic approach. When the junction voltage exceeded certain critical values, conductance showed successive drops by one quantum conductance, corresponding to one-by-one removal of metal atoms. The observed critical voltages agree with the activation energies for surface diffusion of metal atoms. This fact clearly indicates that the elementary process of electromigration is the self-diffusion of metal atoms driven by microscopic kinetic energy transfer from a single conduction electron to a single metal atom. This new finding can be applied to reproducible formation of atomic-spacing gaps for single molecule junctions and also to the failure-tolerant interconnects for VLSIs.