Based upon stress analysis that indicated that surface roughness causes higher contact pressure on contact surfaces, the question arises about whether the initial surface roughness formed by rail grinding may speed up the onset of rail rolling contact fatigue (RCF). In order to clarify whether initial surface roughness has an adverse effect on rail RCF, the authors carried out experiments by means of a twin-disc rolling contact machine and then investigated the experimental results from several perspectives, such as surface roughness, plastic flow, as well as the hardness and axis density of crystals beneath the rail surface. This paper describes the details of the experiments, the variation of roughness on contact surfaces that accompanies repeated rolling contact and the influence of the initial surface roughness on RCF.