Design and Analysis of Fault Tolerance Mechanisms for Big Data Transfers

초록

Increased growth of the data and the need to move the data between data centers, demands, an efficient data transfer tool which can, not only transfer the data at higher rates but also handle the faults occurred during the transfer. Absence of fault tolerance mechanisms, would need to retransmit the whole data, in case of any error during the transfer. Hence, fault tolerance is an important aspect of big data transfer tools. In this paper, we have considered LADS data transfer tool, which proved to be superior to the existing data transfer tools with respect to the speed of the transfer. However, absence of fault tolerance implementation in LADS might result in data retransmission and congestion issues upon errors. In this paper, we design and analyze fault tolerance mechanisms which can be used with LADS data transfer tool. We have proposed three different fault tolerance mechanisms, File logging, Transaction logging and Universal logging. Also, we have analyzed the space and performance overhead of these fault tolerance mechanisms on LADS data transfer tool.