The conventional two-step fractional motion estimation algorithm has been broadly adopted in the literature due to its high encoding performance. However it induces a huge computational complexity as well as a long latency. In this paper, we propose a fast fractional motion estimation algorithm. Based on high correlation between the motion vector of a block and its up-layer, as well as relationship of integer candidates, a one-step algorithm is proposed which not only reduces the computational complexity by eliminating unnecessary fractional-pels, but also saves hardware cost. Experimental results show that the proposed design can save 32% gate count, and reduce the latency by 39% compared with previous designs, while nearly maintaining the coding performance.