In this paper, we propose an efficient scalable and flexible arithmetic unit which executes word-based multiplication and squaring for RSA arithmetic and addition, multiplication and inversion in GF(2^m) for Elliptic Curve Cryptography(ECC) arithmetic operation. The Scalable Word-based Montgomery Modular arithmetic unit (SWMM) can process RSA with modified Finely Integrated Operand Scanning (FIOS) Montgomery algorithm. Also, the proposed flexible hybrid Galois Field Arithmetic Unit (GFAU) is beneficial in cost-effectively implementing ECC operation extensions. The proposed processor is synthesized by employing Synopsys Design Analyzer with a Hynix 0.25 standard cell library at the worst case of 2.3V, 100°C. The synthesis results show that the gate count of processor is below 60,000.