April, 20th - April, 26th

Our solution was further fine tuned to consider more possible scenarios. A particular scenario which had to be addressed concerned the possibility that a network traffic peak could potentially be of such magnitude that the upload time exceedes its desired time slot, overlapping the next time slot. To address this issue, we consider that if a new job arrives and an upload is still in progress, the new job should be burst to the cloud. The reason for this decision has to do with the fact that if an upload is still in progress, the job that will result out of this upload will very likely require more containers than those that will be free if the new job is launched locally.

Furthermore, we realized that the prediction algorithm for the HDFS upload time needs to consider possible fluctuations in the upload speed, since the network in our network laboratory is shared. This feature is now implemented and functional.

After a meeting with Professor Ricardo Morla, we concluded that the writing of the section regarding our solution in the dissertation should begin, which will give us enough time to conduct enough tests to correclty evaluate our solution. We also discussed possible future features that could be added to our solution to enhance it. Of particular interest is the idea of turning this solution into a tool for Network Administrators, providing useful information regarding their clusters and how they can be used to analyze network traffic using Hadoop.

NEXT OBJECTIVES: Start the writing of dissertation.