### Abstract:

Public-key cryptosystems have been instrumental in changing the method of secret communication between two parties. The RSA algorithm is widely used in such communications. The ongoing research on the security aspects of the RSA algorithm is based mainly on the underlying problem of the prime factorization of large integers. Factoring algorithms have improved over the past few decades but no polynomial-time algorithm has as yet been developed. The purpose of this study is to assign a suitable complexity class to the RSA algorithm by analyzing its encryption and decryption processes.<br><br> In order to analyze the RSA algorithm we discuss the basis on which algorithms are classified as tractable (they execute in polynomial time) or intractable (having exponential or worse execution times). The computational complexity of a given algorithmic problem is reflected by using the big-oh notation. An abstract model of computation such as a Turing machine (deterministic or non-deterministic) is used to identify a complexity class for such an algorithmic problem. The classes P and NP represent the sets of decision problems that can either be solved deterministically (P) or non-deterministically (NP) in polynomial execution time. Within the class NP there are NP-complete problems that are assumed to be the hardest (in an algorithmic sense) of NP problems.<br><br> Determining the complexity class of the RSA algorithm has to take into account the defined complexity classes as well as the properties of the so-called one-way functions. Since factoring is a complex mathematical problem and the RSA function may be a trapdoor (one-way) function, it seems as if a special complexity class may be required for the RSA algorithm. We show that, at present, there is no valid reason to claim that the RSA algorithm is NP-complete. Our conclusion is that with the theory available and the knowledge at our disposal the only suitable class is (F)NP, which represents the set of function problems in NP.