Project-Team GRAND-LARGE
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
ACM SIGACT News Distributed Computing Column 28
ACM SIGACT News Distributed Computing Column 28 Idit Keidar Dept. of Electrical Engineering, Technion Haifa, 32000, Israel [email protected] Sergio Rajsbaum, who edited this column for seven years and established it as a relevant and popular venue, is stepping down. This issue is my first step in the big shoes he vacated. I would like to take this opportunity to thank Sergio for providing us with seven years’ worth of interesting columns. In producing these columns, Sergio has enjoyed the support of the community at-large and obtained material from many authors, who greatly contributed to the column’s success. I hope to enjoy a similar level of support; I warmly welcome your feedback and suggestions for material to include in this column! The main two conferences in the area of principles of distributed computing, PODC and DISC, took place this summer. This issue is centered around these conferences, and more broadly, distributed computing research as reflected therein, ranging from reviews of this year’s instantiations, through influential papers in past instantiations, to examining PODC’s place within the realm of computer science. I begin with a short review of PODC’07, and highlight some “hot” trends that have taken root in PODC, as reflected in this year’s program. Some of the forthcoming columns will be dedicated to these up-and- coming research topics. This is followed by a review of this year’s DISC, by Edward (Eddie) Bortnikov. For some perspective on long-running trends in the field, I next include the announcement of this year’s Edsger W. -
Distributed Deep Learning in Open Collaborations
Distributed Deep Learning In Open Collaborations Michael Diskin∗y~ Alexey Bukhtiyarov∗| Max Ryabinin∗y~ Lucile Saulnierz Quentin Lhoestz Anton Sinitsiny~ Dmitry Popovy~ Dmitry Pyrkin~ Maxim Kashirin~ Alexander Borzunovy~ Albert Villanova del Moralz Denis Mazur| Ilia Kobelevy| Yacine Jernitez Thomas Wolfz Gennady Pekhimenko♦♠ y Yandex, Russia z Hugging Face, USA ~ HSE University, Russia | Moscow Institute of Physics and Technology, Russia } University of Toronto, Canada ♠ Vector Institute, Canada Abstract Modern deep learning applications require increasingly more compute to train state-of-the-art models. To address this demand, large corporations and institutions use dedicated High-Performance Computing clusters, whose construction and maintenance are both environmentally costly and well beyond the budget of most organizations. As a result, some research directions become the exclusive domain of a few large industrial and even fewer academic actors. To alleviate this disparity, smaller groups may pool their computational resources and run collaborative experiments that benefit all participants. This paradigm, known as grid- or volunteer computing, has seen successful applications in numerous scientific areas. However, using this approach for machine learning is difficult due to high latency, asymmetric bandwidth, and several challenges unique to volunteer computing. In this work, we carefully analyze these constraints and propose a novel algorithmic framework designed specifically for collaborative training. We demonstrate the effectiveness of our approach for SwAV and ALBERT pretraining in realistic conditions and achieve performance comparable to traditional setups at a fraction of the cost. arXiv:2106.10207v1 [cs.LG] 18 Jun 2021 Finally, we provide a detailed report of successful collaborative language model pretraining with 40 participants. 1 Introduction The deep learning community is becoming increasingly more reliant on transfer learning. -
A Study on Asynchronous Randomized Consensus Title Algorithms for Byzantine Fault Tolerant Replication
A Study on Asynchronous Randomized Consensus Title Algorithms for Byzantine Fault Tolerant Replication Author(s) 中村, 純哉 Citation Issue Date Text Version ETD URL https://doi.org/10.18910/34568 DOI 10.18910/34568 rights Note Osaka University Knowledge Archive : OUKA https://ir.library.osaka-u.ac.jp/ Osaka University A Study on Asynchronous Randomized Consensus Algorithms for Byzantine Fault Tolerant Replication Submitted to Graduate School of Information Science and Technology Osaka University January 2014 Junya NAKAMURA iii List of Major Publications Journal Papers 1. Junya Nakamura, Tadashi Araragi, Toshimitsu Masuzawa, and Shigeru Masuyama, \A method of parallelizing consensuses for accelerating byzantine fault tolerance," IEICE Trans- actions on Information and Systems, vol. E97-D, no. 1, 2014. (to appear). 2. Junya Nakamura, Tadashi Araragi, Shigeru Masuyama, and Toshimitsu Masuzawa, “Effi- cient randomized byzantine fault-tolerant replication based on special valued coin tossing," IEICE Transactions on Information and Systems, vol. E97-D, no. 2, 2014. (to appear). Conference Papers 3. Junya Nakamura, Tadashi Araragi, and Shigeru Masuyama, \Asynchronous byzantine request- set agreement algorithm for replication," in Proceedings of the 1st AAAC Annual Meeting, p. 35, 2008. 4. Junya Nakamura, Tadashi Araragi, and Shigeru Masuyama, \Acceleration of byzantine fault tolerance by parallelizing consensuses," in Proceedings of the 10th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT '09, pp. 80{ 87, Dec. 2009. Technical Reports 5. Junya Nakamura, Tadashi Araragi, and Shigeru Masuyama, \Byzantine agreement on the order of processing received requests is solvable deterministically in asynchronous systems," in IEICE Technical Report, vol. 106 of COMP2006-35, pp. 33{40, Oct. -
Auto-Stabilisation : Solution Elégante Pour Lutter Contre Les Fautes Sylvie
Auto-stabilisation : Solution Elégante pour Lutter contre Les Fautes Sylvie Delaët Rapport scientifique présenté en vue de l’obtention de l’Habilitation à Diriger les Recherches. Soutenu le 8 novembre 2013 devant le jury composé de : Alain Denise, Professeur, Université Paris Sud, France Shlomi Dolev, Professeur, Ben Gurion University of the Negev, Israël Pierre Fraigniaud, Directeur de recherche CNRS, France Ted Herman, Professeur, University of Iowa, Etats Unis d’Amérique Nicolas Thiéry, Professeur, l’Université Paris Sud, France Franck Petit, Professeur, Université Paris Pierre et Marie Curie, France Après avis des rapporteurs : Ted Herman, Professeur, University of Iowa, Etats Unis d’Amérique Rachid Guerraoui, Professeur, Ecole Polytechnique Fédérale de Lausanne, Suisse Franck Petit, Université Paris Pierre et Marie Curie, France ii Sommaire Remerciements iii 1 Ma vision de ce document 1 1.1 Organisation du document . .1 2 Un peu de cuisine(s) 3 2.1 L'auberge espagnole de l'algorithmique r´epartie. .4 2.2 Exemples de probl`emes`ar´esoudreet quelques solutions . .8 3 Comprendre le monde 15 3.1 Comprendre l'auto-stabilisation . 15 3.2 Comprendre l'algorithmique r´epartie . 30 3.3 Comprendre mes contributions . 34 4 Construire l'avenir 43 4.1 Perspectives d'am´eliorationdes techniques . 43 4.2 Perspective d'ouverture `ala diff´erence . 45 4.3 Perspective de diffusion plus large . 47 4.4 Perspective de l'ordinateur auto-stabilisant . 48 A Agrafage 51 A.1 Cerner les probl`emes . 51 A.2 D´emasquerles planqu´es . 59 A.3 Recenser les pr´esents . 67 A.4 Obtenir l'auto-stabilisation gratuitement . -
Developing P2P Protocols Across NAT Girish Venkatachalam
Developing P2P Protocols across NAT Girish Venkatachalam Abstract Hole punching is a possible solution to solving the NAT problem for P2P protocols. Network address translators (NATs) are something every software engineer has heard of, not to mention networking professionals. NAT has become as ubiquitous as the Cisco router in networking terms. Fundamentally, a NAT device allows multiple machines to communicate with the Internet using a single globally unique IP address, effectively solving the scarce IPv4 address space problem. Though not a long-term solution, as originally envisaged in 1994, for better or worse, NAT technology is here to stay, even when IPv6 addresses become common. This is partly because IPv6 has to coexist with IPv4, and one of the ways to achieve that is by using NAT technology. This article is not so much a description of how a NAT works. There already is an excellent article on this subject by Geoff Huston (see the on-line Resources). It is quite comprehensive, though plenty of other resources are available on the Internet as well. This article discusses a possible solution to solving the NAT problem for P2P protocols. What Is Wrong with NAT? NAT breaks the Internet more than it makes it. I may sound harsh here, but ask any peer-to-peer application developer, especially the VoIP folks, and they will tell you why. For instance, you never can do Web hosting behind a NAT device. At least, not without sufficient tweaking. Not only that, you cannot run any service such as FTP or rsync or any public service through a NAT device. -
Population Protocols: Expressiveness, Succinctness and Automatic Verification
Fakultat¨ fur¨ Informatik Technische Universitat¨ Munchen¨ Population Protocols: Expressiveness, Succinctness and Automatic Verification. Stefan Jaax Vollst¨andigerAbdruck der von der Fakult¨atf¨urInformatik der Technischen Universit¨at M¨unchen zur Erlangung des akademischen Grades eines Doktor der Naturwissenschaften (Dr. rer. nat.) genehmigten Dissertation. Vorsitzender: Prof. Dr. Helmut Seidl Pr¨ufendeder Dissertation: 1. Prof. Dr. Francisco Javier Esparza Estaun 2. Prof. Rupak Majumdar, Ph.D., Technische Universit¨atKaiserslautern Die Dissertation wurde am 03.03.2020 bei der Technischen Universit¨atM¨unchen eingereicht und durch die Fakult¨atf¨urInformatik am 04.06.2020 angenommen. Abstract Population protocols (Angluin et al., PODC, 2004) are a model of distributed computa- tion in which identical, finite-state, passively mobile agents interact in pairs to achieve a common goal. In the basic model of population protocols, agents compute number predicates by reaching a stable consensus. It is well known that population protocols compute precisely the semilinear predicates, or, equivalently, the predicates definable in Presburger arithmetic, the first-order theory of the natural numbers equipped with addition and the standard linear order. This thesis investigates three fundamental questions of the theory of population pro- tocols: Space complexity, verification complexity, and expressiveness of reasonable ex- tensions. Space Complexity. We show that every quantifier-free Presburger predicate ' aug- mented with remainder predicates is computable by a population protocol with poly(j'j) states, where j'j denotes the size of ' in binary encoding. Further, the protocol can be constructed in polynomial time. This is a major improvement to the previously known construction, which requires 2poly(j'j) states. As a special case, we consider predicates of the form 'c(x) = x ≥ c, where c is a positive integer constant. -
Population Protocols Are Fast
Population Protocols Are Fast Adrian Kosowski1 and Przemysław Uznanski´ 2 1Inria Paris, France 2 ETH Zürich, Switzerland Abstract A population protocol describes a set of state change rules for a population of n indistinguish- able finite-state agents (automata), undergoing random pairwise interactions. Within this very basic framework, it is possible to resolve a number of fundamental tasks in distributed computing, includ- ing: leader election, aggregate and threshold functions on the population, such as majority compu- tation, and plurality consensus. For the first time, we show that solutions to all of these problems can be obtained quickly using finite-state protocols. For any input, the designed finite-state proto- cols converge under a fair random scheduler to an output which is correct with high probability in expected O(poly log n) parallel time. In the same setting, we also show protocols which always reach a valid solution, in expected parallel time O(nε), where the number of states of the interacting automata depends only on the choice of ε > 0. The stated time bounds hold for any semi-linear predicate computable in the population protocol framework. The key ingredient of our result is the decentralized design of a hierarchy of phase-clocks, which tick at different rates, with the rates of adjacent clocks separated by a factor of Θ(log n). The construction of this clock hierarchy relies on a new protocol composition technique, combined with an adapted analysis of a self-organizing process of oscillatory dynamics. This clock hierarchy is used to provide nested synchronization primitives, which allow us to view the population in a global manner and design protocols using a high-level imperative programming language with a (limited) capacity for loops and branching instructions. -
Technology Stack for Decentralized Mobile Services
Technology Stack for Decentralized Mobile Services Matouš Skála Technology Stack for Decentralized Mobile Services by Matouš Skála to obtain the degree of Master of Science at the Delft University of Technology, to be defended publicly on Monday August 31, 2020 at 3:00 PM. Student number: 4893964 Project duration: November 15, 2019 – August 31, 2020 Thesis committee: Dr.ir. J.A. Pouwelse, TU Delft, supervisor Dr. J.S. Rellermeyer, TU Delft Dr. N. Yorke-Smith, TU Delft An electronic version of this thesis is available at http://repository.tudelft.nl/. Preface When I was choosing my thesis topic, I originally came up with an idea of designing a decen- tralized social network. After realizing how ambitious that goal was, I later decided to focus on more fundamental issues first and create a library that would allow for building any de- centralized applications, running purely on an overlay network consisting of smartphones. Rather than reinventing the wheel, I took inspiration from an existing networking library de- veloped at TU Delft over the last decade and created its wire-compatible implementation in Kotlin. Interestingly, in the end, I have even implemented a trivial social network to demon- strate the usage of the library, returning back to the original idea. I would like to thank my supervisor Johan Pouwelse for an endless stream of fresh ideas and valuable feedback, and to PhD students of the Delft Blockchain Lab for numerous coffee meetings and for serving me as a walking documentation of the existing codebase. Matouš Skála Prague, -
Technical Report Sce-12-04 Nat Traversal in Peer-To
TECHNICAL REPORT SCE-12-04 DEPARTMENT OF SYSTEMS AND COMPUTER ENGINEERING CARLETON UNIVERSITY NAT TRAVERSAL IN PEER -TO -PEER ARCHITECTURE Marc-André Poulin¹, Lucas Rioux Maldague¹, Alexandre Daigle¹, François Gagnon² 1 Cégep de Sainte-Foy, Canada [email protected] , [email protected] , [email protected] 2 Carleton University, Canada [email protected] Abstract. Peer-to-peer networks are well known for file sharing between multiple computers. They establish virtual tunnels between computers to transfer data, but NATs makes it harder. A NAT, Network Address Translation , is a process which transforms private IP addresses, such as 192.168.2.1, into public addresses, such as 203.0.113.40. The idea is that multiple private addresses can hide behind a single public address and thus virtually enlarge the number of allocable public IP addresses. When an application in the local network establishes a connection to Internet, the packet passes through the NAT which adjusts the IP header and maps an external port to the computer which sent the request. When packets are received from the Internet by the NAT, they are forwarded to the internal host which is mapped to the port on which the packet was received, or dropped if no mapping exists. In this paper, we will introduce you to NAT and P2P, we will discuss the numerous ways NATs use to translate private IP addresses into public ones, we will discuss known techniques used to fix the problem and we will also present how popular peer-to-peer programs bypass NATs. This paper is written so anybody with a reasonable knowledge of networking would grasp the essentials. -
A Framework for Multi- Channel Telecommunication Applied to Telecare Application
Master Thesis A Framework for Multi- Channel Telecommunication Applied to TeleCare Application Author: Yifan Ruan Supervisor: Danny Weyns Examiner: Danny Weyns Date: 2015-6-4 Course Code: 5DV00E, 30 credits Subject: Software Technology Level: Master Department of Computer Science Abstract This thesis is deriving from a telemedicine project "TeleCare" of constructing soft- ware for remote medical diagnosis between the doctor and the patient. The software has to fix the problem of managing local and remote media information. This thesis presents a telecommunication framework for synchronizing multiple media chan- nels, following research methodology, from problem description, iterative and incre- mental development to prototype finalization. For the framework, I have described the framework requirements and corresponding architecture design and implemen- tation. From the evaluation result of "TeleCare" software developed above it, I can conclude the framework has reached the problem. Keywords: framework, communication, synchronization, channel Preface Nowadays, the population of elderly people is growing fast around the world. So It is a nice idea to provide better social care with assistive technologies, such as "TeleCare" soft- ware. With "TeleCare", the patient and the doctor can communicate with each other like face-to-face meeting. In addition, with some state-of-the-art device, touchscreen monitor, they can express themselves through drawing on the screen. For example, the doctor can draw an arrow representing the patient should raise his arm to a certain height, and the patient can follow the instruction when seeing it on his screen. Image a scenario that if an elderly person lives far away from the hospital, one day he broke his leg, it was inconve- nient for him to go to hospital for reexamination again and again. -
ISBN # 1-60132-514-2; American Council on Science & Education / CSCE 2021
ISBN # 1-60132-514-2; American Council on Science & Education / CSCE 2021 CSCI 2021 BOOK of ABSTRACTS The 2021 World Congress in Computer Science, Computer Engineering, and Applied Computing CSCE 2021 https://www.american-cse.org/csce2021/ July 26-29, 2021 Luxor Hotel (MGM Property), 3900 Las Vegas Blvd. South, Las Vegas, 89109, USA Table of Contents Keynote Addresses .................................................................................................................... 2 Int'l Conf. on Applied Cognitive Computing (ACC) ...................................................................... 3 Int'l Conf. on Bioinformatics & Computational Biology (BIOCOMP) ............................................ 6 Int'l Conf. on Biomedical Engineering & Sciences (BIOENG) ................................................... 12 Int'l Conf. on Scientific Computing (CSC) .................................................................................. 14 SESSION: Military & Defense Modeling and Simulation ............................................................ 27 Int'l Conf. on e-Learning, e-Business, EIS & e-Government (EEE) ............................................ 28 SESSION: Agile IT Service Practices for the cloud ................................................................... 34 Int'l Conf. on Embedded Systems, CPS & Applications (ESCS) ................................................ 37 Int'l Conf. on Foundations of Computer Science (FCS) ............................................................. 39 Int'l Conf. on Frontiers -
Versatility and Efficiency in Self-Stabilizing Distributed Systems
Versatility and Efficiency in Self-Stabilizing Distributed Systems Stéphane Devismes To cite this version: Stéphane Devismes. Versatility and Efficiency in Self-Stabilizing Distributed Systems. Distributed, Parallel, and Cluster Computing [cs.DC]. Université Grenoble Alpes, 2020. tel-03080444v4 HAL Id: tel-03080444 https://hal.archives-ouvertes.fr/tel-03080444v4 Submitted on 21 Sep 2021 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. HABILITATION À DIRIGER DES RECHERCHES Spécialité : Informatique Arrêté ministériel : 23 Novembre 1988 Présentée par Stéphane Devismes Habilitation préparée au sein du laboratoire VERIMAG et de l’École Doctorale Mathématiques, Sciences et Technologies de l’Information, Informatique Versatility and Efficiency in Self-Stabilizing Distributed Systems Généralité et Efficacité dans les Systèmes Distribués Autostabilisants Habilitation soutenue publiquement le 17 décembre 2020, devant le jury composé de : Denis TRYSTRAM Professeur, Grenoble INP, Président Toshimitsu MASUZAWA Professeur, Osaka University, Rapporteur Achour MOSTÉFAOUI