Data Synchronization for Iitbombayx Platform Related Work and Background Architecture and Design Implementation Conclusion and Future Work
Total Page:16
File Type:pdf, Size:1020Kb
Data Synchronization for IITBombayX Platform Related Work and Background Architecture And Design Implementation Conclusion and Future Work Data Synchronization for IITBombayX Platform Alpesh Rathore 113050057 Under the guidance of Prof. D.B. Phatak Computer Science Department, IIT Bombay 1 / 65 Data Synchronization for IITBombayX Platform Related Work and Background File Synchronization Techniques Architecture And Design Tools for File Synchronization Implementation IITBombayX Platform Conclusion and Future Work 1 Data Synchronization for IITBombayX Platform File Synchronization Techniques Tools for File Synchronization IITBombayX Platform 2 Related Work and Background Related Work Background 3 Architecture And Design File Synchronization Admin Utility File Synchronization Admin Web Application Android Application Architecture Use Case 4 Implementation File Synchronization Web Application Android Application Implementation Technologies Used 5 Conclusion and Future Work 2 / 65 Data Synchronization for IITBombayX Platform Related Work and Background File Synchronization Techniques Architecture And Design Tools for File Synchronization Implementation IITBombayX Platform Conclusion and Future Work Outline 1 Data Synchronization for IITBombayX Platform File Synchronization Techniques Tools for File Synchronization IITBombayX Platform 2 Related Work and Background Related Work Background 3 Architecture And Design File Synchronization Admin Utility File Synchronization Admin Web Application Android Application Architecture Use Case 4 Implementation File Synchronization Web Application Android Application Implementation Technologies Used 5 Conclusion and Future Work 2 / 65 Data Synchronization for IITBombayX Platform Related Work and Background File Synchronization Techniques Architecture And Design Tools for File Synchronization Implementation IITBombayX Platform Conclusion and Future Work Problem Statement Design a web-application based solution for full-fledged File Synchronization over a distributed set of servers in various colleges so that Faculty or students or other members of IITBombayX course can contribute to the course content which can be verified and synched to central server from where it becomes available to members of the course across teh globe, along with additional facilities, like, viewing history, periodic synchroinzation, notification, etc. 3 / 65 Data Synchronization for IITBombayX Platform Related Work and Background File Synchronization Techniques Architecture And Design Tools for File Synchronization Implementation IITBombayX Platform Conclusion and Future Work File Synchronization Keeping two different locations on same or different storing devices synchronized with each other. Mirroring: Keeping one location synced with another so that if some changes are made on one location, they are reflected on the other end, but NOT vice versa. Two Way: Both of synced locations are exactly identical every time. Refs: [1, 2, 3] 4 / 65 Data Synchronization for IITBombayX Platform Related Work and Background File Synchronization Techniques Architecture And Design Tools for File Synchronization Implementation IITBombayX Platform Conclusion and Future Work File Synchronization Techniques Delta Differencing: Transfer the change, not whole content as in FTP. Security and encryption Compression Multiple locations syncing Progress, failures, successes, performance, bandwidth utilization, etc. 5 / 65 Data Synchronization for IITBombayX Platform Related Work and Background File Synchronization Techniques Architecture And Design Tools for File Synchronization Implementation IITBombayX Platform Conclusion and Future Work Tools for File Synchronization There are some file synchronization tools which are available. Unison: Unison is open source (under GNU license) and works over both windows and Linux platforms. It allows synchronization from both directions, i.e., two-way synchronizations and key features like star schema for synchronizing multiple machines. RSync: RSync is more commonly used file synchronization tool which is also available with IBM AIX distribution. RSync was initially launched for Linux platform but now has been ported for Windows as well. It is a very flexibly configurable utility software with an array of options to configure it to work according to one’s requirements. RSync has also got some tools which are built on top of it to provide GUI for the same. But most of them are not stable. Refs: [1, 2, 3] 6 / 65 Data Synchronization for IITBombayX Platform Related Work and Background File Synchronization Techniques Architecture And Design Tools for File Synchronization Implementation IITBombayX Platform Conclusion and Future Work IITBombayX Platform 1 Open source platform for bringing students and faculties on a common platform. 2 Faculties offer courses from different universities on the platform. 3 Students can then register for various courses offered. 4 Students get through the course content, appear in tests, quizzes, etc. and get assessed. Refs: [4] 7 / 65 Data Synchronization for IITBombayX Platform Related Work and Background File Synchronization Techniques Architecture And Design Tools for File Synchronization Implementation IITBombayX Platform Conclusion and Future Work IITBombayX Platform 1 CMS: Content Management System or IITBombayX Studio. CMS helps in managing the content of a course. It provides facilities to add, delete or edit courseware. 2 LMS: Learning Management System. Students interact to IITBombayX through LMS, which provides facilities like quiz, comments, feedback, submission, and other interactions. It is also responsible for displaying contents on the web page. 3 Configuration: provides for configuration management when setting up new IITBombayX platform. It uses Ansible for configuration management which is an open source platform for configuring and managing machines. 4 XBlock: provides for creating courseware for a course. Courseware follows a hierarchical structure. A courseware may be considered as composed of various components, where each component may be as simple as a paragraph, input form, video ,etc, and as complex as a section, chapter or complete course. 5 edx-ora2: takes care of any assessment related activities. Refs: [4] 8 / 65 Data Synchronization for IITBombayX Platform Related Work and Background File Synchronization Techniques Architecture And Design Tools for File Synchronization Implementation IITBombayX Platform Conclusion and Future Work IITBombayX Platform 1 CS Comments Service: CS Comments Service module facilitates nested comments and voting. 2 XQueue: provides checking interface for LMS, so that when students makes any submission through LMS, the submission goes to XQueue which makes the submission assessed and graded by external service and sends the assessment back to LMS. 3 XServer: Responsible for taking code submissions taken by LMS and running the code using courseware checkers. 4 notifier: Sends daily feeds from forums to students registered on the forums. 5 Analytics Dashboard: Displays meta data about activities on their courses, like, enrollments, performance of students, etc. 6 Analytics Pipeline: Analyzes data from tracking logs and IITBombayX databases and provides analyzed information to outside world through edx-analytics-data-api. Refs: [4] 9 / 65 Data Synchronization for IITBombayX Platform Related Work and Background Related Work Architecture And Design Background Implementation Conclusion and Future Work Outline 1 Data Synchronization for IITBombayX Platform File Synchronization Techniques Tools for File Synchronization IITBombayX Platform 2 Related Work and Background Related Work Background 3 Architecture And Design File Synchronization Admin Utility File Synchronization Admin Web Application Android Application Architecture Use Case 4 Implementation File Synchronization Web Application Android Application Implementation Technologies Used 5 Conclusion and Future Work 10 / 65 Data Synchronization for IITBombayX Platform Related Work and Background Related Work Architecture And Design Background Implementation Conclusion and Future Work Related Work Lucky Backup [5]: Lucky Backup is a free desktop application which runs on top of RSync and gives following features[6]: 1 Backup It helps in keeping backup of the data on some remote machine, so that whenever any files are added, deleted, or modified, all the files and changes get backed up on the remote machine. 2 (Snapshots) User can take snapshot of the directory being backed up or synced and store the snapshot. Such snapshots can later be recovered and directory comes back to the same state as at the time of taking snapshot. 3 Sync User can sync multiple pairs of directories so that whenever there is a change in the synced file, those changes get reflected back on other locations. 4 Exclude Option To exclude one or more files based upon names or pattern of names. 5 Simulation This is a very powerful functionality, where if use is not sure of the outcome of running the RSync command, they can run a simulation of the command they are going to execute. Utility will produce similar results as real RSync command but there won’t be any changes on either end of synchronization. Once user is sure, they can go forward with executing the command. 11 / 65 Data Synchronization for IITBombayX Platform Related Work and Background Related Work Architecture And Design Background Implementation Conclusion and Future Work Related Work FlyBack [7]: FlyBack was initially built on top of RSync but is now created from scratch.