Introducing Distributed Computing to Programming Competitions

Distributed Tasks: Introducing Distributed Computing to Programming Competitions

Adam KARCZMARZ1, Jakub ŁĄCKI2, Adam POLAK3 Jakub RADOSZEWSKI1,4, Jakub O. WOJTASZCZYK5 1Faculty of Mathematics, Informatics and Mechanics, University of Warsaw ul. Banacha 2, 02-097 Warsaw, Poland 2Department of Computer, Control, and Management Engineering Antonio Ruberti at Sapienza University of Rome via Ariosto 25, 00185 Roma, Italy 3Department of Theoretical Computer Science, Faculty of Mathematics and Computer Science, Jagiellonian University ul. Łojasiewicza 6, 30-348 Kraków, Poland 4King’s College London Strand, London WC2R 2LS 5Google Warsaw Emilii Plater 53, 00-113 Warsaw, Poland e-mail: [email protected], [email protected], [email protected], [email protected], [email protected]

Abstract. In this paper we present distributed tasks, a new task type that can be used at programming competitions. In such tasks, a contestant is supposed to write a program which is then simultaneously executed on multiple computing nodes (machines). The instances of the program may communicate and use the joint computing power to solve the task presented to the contestant. We show a framework for running a contest with distributed tasks, that we believe to be accessible to contestants with no previous experience in distributed computing. Moreover, we give examples of distributed tasks that have been used in the last two editions of a Polish programming contest, Algorithmic Engagements, together with their intended solutions. Finally, we discuss the challenges of grading and preparing distributed tasks. Keywords: programming contests, distributed tasks.

1. Introduction

When looking at major programming competitions, it is easy to notice that a large number of them are very similar in design. To name a few, the IOI, the ACM ICPC, Google Code Jam, TopCoder’s algorithmic track, Facebook Hacker Cup, the CodeForces competitions and many more, are all focused on small, self-contained tasks, with automated 178 A. Karczmarz et al. judging based on testing a program on a judge-provided set of testcases and mostly algorithmic in nature. This model seems attractive both to the organizers and the participants. The organizers appreciate the very clear, non-subjective mechanism of judging and the automation of judging, which means the competition can scale out easily to a larger number of competitors. The participants also appreciate the automated judging (which means fast results) and the objective judgment criteria (which makes the competition more fair); additionally the entry barrier to such contests is pretty low, as the introductory-level tasks can be very simple. If we consider programming competitions a way of educating future computer sci- entists and professional programmers, the model has its strengths, but also weaknesses. Some of these weaknesses come from the very nature of the small, self-contained tasks (which are a large part of the model’s attractiveness): the participants do not learn about code maintainability and extensibility, they also do not learn anything about larger-scale system design. These weaknesses seem to be intrinsic to the model itself; and competitions that abandon the model moving to larger or less clear-cut tasks are frequently less successful (for example, TopCoder’s Marathon Match track has over 10 times less reg- istered participants than the Algorithm track). However, there are also areas of programming expertise that the existing competitions do not teach, which are less intrinsically tied to the model of these competitions. In particular, the focus on algorithmic problems is not crucial to the model of the competition itself. Indeed, multiple authors already explored extending this classical model of competitions to other fields of computer science, for example visualization and precom- putation (Kulczyński et al., 2011), online algorithms (Komm, 2011), computer graphics and cryptography (Forišek, 2013).

1.1. Distributed Programming

Distributed (and cloud) computing has gained importance quickly in the recent years. The computing giants of today – Google, Amazon, Facebook and others – do not operate on the enormous mainframes that dominated computing in the past, but on networked farms of smaller servers. This is a model of computing that is not inherently in conflict with the algorithmic programming contest model, but is not taught by the dominant competitions of today; students that gained most of their programming skills through programming competitions will be totally unfamiliar with even the basic paradigms of distributed computing like the MapReduce framework (Dean and Ghemawat, 2008) or the CAP theorem (Fox and Brewer, 1999). In this article we present a framework for running a contest focusing on distributed algorithms, developed by engineers in Google’s Warsaw office in collaboration with the University of Warsaw, and show sample problems used in the Algorithmic Engage- ments (Potyczki Algorytmiczne in Polish) contest ran by the University of Warsaw. The same framework is used at the recently introduced Google’s Distributed Code Jam competition. Distributed Tasks: Introducing Distributed Computing to ... 179

1.2. Designing a Distributed Programming Contest

The primary focus of our design was simplicity from the contestant’s point of view. The introduction of a new programming paradigm, likely unfamiliar to most participants, is clearly a challenge for the contestants, and we aimed at making the transition as smooth as possible. Thus, the basic interface is similar to a programming contest like IOI. The participant submits a single program, which is compiled and executed by the framework. The same program is run on every node (computer) available to the participant. Obviously, the nodes need to be able to communicate in order to collaborate in computing. We decided on a protocol based on simple message-passing (“send this array of bytes to this node”). The message passing methods are available to the program through a library that is common for all problems, and provided by the framework. We also considered using an RPC-style interface. This, however, is more complex on an API level. A standard approach to RPC is that the programmer has to declare an interface (which will contain the remotely-callable methods). The server will just implement the methods of this interface. On the client side, the infrastructure needs to provide a way to generate a “stub” connected to some particular server; this stub will automagically have the method calls autogenerated. The language would have to provide some way to anno- tate that the interface is to be treated as a “remote” interface, increasing the complexity of the infrastructure implementation, and meaning more “magic” happening under the hood, which – in our perception – decreases the comprehensibility of the system. For instance, the “stub implementation” would need to make a choice of deep versus shallow copying of the arguments, each choice potentially leading to confusion for the participants. Let us now present the functions of our library, together with their declarations in C++. First, the library provides a function that returns the number of nodes  on which the solution is running, and the index (in the range [0  − 1]) of the node on which the calling process is running. int NumberOfNodes(); int MyNodeId();

The library maintains in each node a message buffer for each of the  nodes, which represents messages that are to be sent to this node. Messages are added to the buffer through the Put-methods.

// Append “value” to the message that is being prepared for // the node with id “target”. The “Int” in PutInt is // interpreted as 32 bits, regardless of whether the actual // int type will be 32 or 64 bits. void PutChar(int target, char value); void PutInt(int target, int value); void PutLL(int target, long long value); 180 A. Karczmarz et al. There is also a method that sends the message that was accumulated in the buffer for a given node and clears this buffer. This method is non-blocking, that is, it does not wait for the receiver to call Receive, but returns immediately after sending the message. void Send(int target);

The following function is used for receiving messages. int Receive(int source);

The library has a receiving buffer for each remote node. When you call Receive and retrieve a message from a remote node, the buffer tied to this remote node is over- written. You can then retrieve individual parts of the message through the Get-methods. This method is blocking – if there is no message to receive, it will wait for the message to arrive. You can call Receive(-1) to retrieve a message from any source, or set source to a number from [0  − 1] to retrieve a message from a particular source. Receive returns the number of the node which sent the message, which is equal to source, unless source is -1. Finally, for reading the buffer of incoming messages, the following three methods are provided.

// The “Int” in GetInt is interpreted as 32 bits, regardless // of whether the actual int type is 32 or 64 bits. char GetChar(int source); int GetInt(int source); long long GetLL(int source);

Each of these methods returns and consumes one item from the buffer of the ap- propriate node. You must call these methods in the order in which the elements were appended to the message (so, for instance, if the message was created with PutChar, PutChar, PutLL, you must call GetChar, GetChar, GetLL in this order). If you call them in a different order, or you call a Get-method after consuming all the contents of the buffer, the behaviour is undefined. The serialization we decided to use is very basic, compared to models like Java’s serialization mechanisms, Python’s “pickle” or even Google’s protocol buffer language. Again, we preferred to err on the side of simplicity, to minimize the entry barrier – this simple language turns out to be easy enough for the simple concurrency required to solve our problems, and is more straightforward to understand, both in terms of “how big will be the serialized message”, and “what is actually serialized” (this, again, is the deep vs shallow copy question). For correctness of execution, we chose what seemed the most natural model. The backend guarantees failure-less execution on all nodes, and requires all the instances of the program to execute correctly, within the specified time and memory limits. Note that the time used by the program is measured from the moment when the instances start until Distributed Tasks: Introducing Distributed Computing to ... 181 all instances have finished execution. This assumption is justified with the following -ex ample involving two machines. The first one spends second of CPU time and then sends a message to the second one. The second machine first waits for a message from the first machine and then uses one second of CPU time. The total time used by the solution is then roughly two seconds, although each instance used only one second of CPU time. In most of programming competitions, the programs access the input data by reading from standard input. (TopCoder’s Algorithm contest breaks out of this by providing the input as an argument to the function the contestant is supposed to write.) How- ever, a large fraction of the interesting distributed problems admit a solution that runs in () time, where  is the size of the input, and  is the number of nodes available to the contestant. This implies, in particular, that no node can afford to read the whole input (since that alone would take () time). Standard input is only accessible in a linear fashion, so it is not feasible for providing very large input data. The approach we chose is what is typically done for interactive tasks on competitions like IOI (see, e.g., Chávez, 2015) – each problem with large inputs defines a set of input-access methods that are available to the program, similarly to the message-passing interface. These methods are guaranteed to return the same values on all nodes (so, each node has access to the same view of the input). Additionally, we provide upper bounds on the execution time of a single call to the input-providing methods. Output is much simpler to handle (as it is often much smaller), so we went with the standard programming contest practice of expecting the output to be provided via standard output. We expect exactly one node to produce the output, while the others should not produce anything. This is a somewhat arbitrary decision (we could equally well have all the nodes output the exact same data to the standard output), however, as many solutions in practice have some sort of a “master” node that aggregates the work of the other nodes, it is convenient to the contestant to have one node output the result of the computation, and for the infrastructure not to prescribe which of the nodes it is. As for the number of nodes we run the contestants’ solutions on, we chose  = 100. This is large enough that a speedup by a factor of  is big enough to offset the extra time needed for inter-node communication, and yet small enough that providing that many nodes for judging is actually feasible. The final issue that needs to be considered is the amount of data sent during the communication between the nodes, both in terms of the number of individual messages sent, and the size of those messages. Obviously, the limits here will be dependent on the infrastructure we run the contest on. Benchmarks on our framework show that a single message will take roughly 3–5ms (split between the processing time in the sender, the actual network latency and the processing time on the receiver). This number will be constant for messages from negligible to tens of kilobytes, and start growing linearly when the message size goes into hundreds of kilobytes. While in the end, to fine-tune a solution the contestant will need to understand these patterns (for this purpose we provided the results of a few benchmarks to the contestants), we wanted the basic usecase not to require dealing with the calculations. To this end, we introduced an upper limit on the total number (around 1000) and size (a few megabytes) of messages a single node can send within the time limit, which roughly 182 A. Karczmarz et al. corresponds to the total throughput it could achieve if it spent all its time on communication. This is a clear and concise way of informing the participants that if their solution stays well within those bounds, its performance should not be significantly affected by the communication overhead.

1.3. Distributed Contest Judging Infrastructure

From a research perspective, the most interesting challenge in preparing such a distributed programming contest is defining the exact programming model from the contestants’ point of view. However, when doing it in practice, there is also a considerable amount of engineering effort involved in providing the infrastructure to run such a contest. In the process of preparing problems the most interesting new challenge is writing a library that provides the input data. While in a standard programming contest the input is just a text file, and can be generated offline in an arbitrary fashion and almost arbitrary time, in the distributed contests the requirements are much more strict. The input- providing library needs to satisfy the following requirements: ●● Access to an arbitrary element. ●● Consistency across nodes, and across accesses. ●● Access times on the order of 100ns. ●● Ability to serve data with total size on the order of 10GB or more. The requirements stem from the input model we chose. In particular, if we want to allow  ()-time solutions, with runtimes on the order of 1–5 seconds, we need the input read by one node to be on the order of at least 107 items (which means 109 items in total – if each item is, say, two 64-bit integers, we get a total of 16GB), and we need the node to be able to read these 107 items within 1 second (so that the input reading does not dominate the computation). The access times coupled with the data size mean it is infeasible to pregenerate all the input data – 10GB is too much to conveniently store in memory, while disk access (and even SSD access) is too slow. Thus, the input data is generated on the fly. This requires using pseudo-random generators that generate a sequence of numbers, and provide consistent and fast access to each element of the sequence (e.g., the CityHash family of functions). In the judging system, the challenge is the scale. Judging a single testcase for a single solution requires, typically, 100 virtual machines. So, as an example, a deployment of 900 virtual machines on Google Compute Engine were used as the backend for the Online Round of Google’s Distributed Code Jam. We think it is interesting that cloud computing, which is making distributed computing important as a topic of programming competitions, is also making distributed programming competitions much easier to or- ganize: instead of buying physical hardware to support such a competition, it is easy to rent virtual machines and pay by the minute. However, the real challenge in setting up a new competition type is in finding attractive problems: ones that challenge the contestants’ creativity and problem-solving skill, Distributed Tasks: Introducing Distributed Computing to ... 183 without requiring significant domain knowledge (in this case – knowledge of distributed programming paradigms), and that are fun, but not tedious, to implement, once you have the correct set of ideas. In the rest of this paper, we provide examples of problems that – in our opinion – satisfy these requirements. We start with a simple task which lets us demonstrate the basics of the framework from the contestant’s point of view. The remaining tasks were presented during the Al- gorithmic Engagements contest; their authors are: Jakub Łącki, Jakub Wojtaszczyk (task “Workshop”); Jakub Łącki (task “Assistant”); Adam Karczmarz (task “Sabotage”).

2. Sample Task “Divisors”

In this task we are to count the number of divisors of a given positive integer  ≤ 1018.

Input In this task the input data is provided via the standard input. The only line of the standard input contains . Output Your program should print exactly one line to the standard output containing one integer: the number of divisors of .

2.1. Solution

The easiest (sequential) solution that we can come up with is to check all candidates for a divisor  up to  and their counterparts of the form  . A sample C++ code follows.  int main() { long long n; int divisors_num = 0; cin >> n; for (long long d = 1; d * d <= n; ++d) { if (n % d == 0) { ++divisors_num; if (n / d != d) ++divisors_num; } } cout << divisors_num << endl; }

Probably this is not the fastest sequential solution for this problem. We will, however, focus on how to speed it up by performing the computations using  nodes (machines). 184 A. Karczmarz et al.

A natural idea is to partition the set of possible divisors and let each of the machines look through each of the parts. One way of performing this partition is as follows: machine 0 gets to check the candidates 1 1 +  1 + 2     machine 1 gets to check the candidates 2 2 +  2 + 2     etc. To make it work, it suffices to change the for-loop from the above code to: for (long long d = 1 + MyNodeId(); d * d <= n; d += NumberOfNodes()) {

In the end we need to aggregate the partial results. Let us select any of the machines (say, machine 0) as an aggregator. We just need to add code to be executed on each of the machines that sends the partial results from machines with positive numbers to the machine number 0.

if (MyNodeId() > 0) { PutInt(0, divisors_num); Send(0); } else { // MyNodeId == 0 for (int node = 1; node < NumberOfNodes(); ++node) { Receive(node); divisors_num += GetInt(node); } cout << divisors_num << endl; } }

The final solution works in  ( ) time, i.e., this is the maximum of the time complexities of the instances running on each of the machines.

3. Task “Workshop” (2014)

During an algorithmic workshop students are sitting in a circle and solving problems. Whenever someone comes up with a solution to a problem, he or she shares the idea with his or her two neighbours (which takes exactly one minute), and then they pass it to their neighbours (which also takes exactly one minute), and so on. If Johny comes up with a brilliant solution at quarter to twelve, what time will Chris hear about it? How many minutes does it take for a solution to reach from Kate to Tom? That is the kind of queries your program has to answer. Input The input data is provided by an interactive library. Your program can call six functions from the library: Distributed Tasks: Introducing Distributed Computing to ... 185

●●int NumberOfStudents(): returns the number  of students participating in the workshop (3 ≤  ≤ 109). The students are numbered with consecutive integers

from 1 to . ●●int FirstNeighbour(int i): returns the number of the first neighbour of the -th student (1 ≤  ≤ ). The students have a hard time distinguishing left from

right, therefore they prefer to call their neighbours in the order of their numbers. That is, the first neighbourof a given student always has a number smaller than his of her second neighbour. ●●int SecondNeighbour(int i): returns the number of the second neighbour of the -th student (1 ≤  ≤ ).

●●int NumberOfQueries(): returns the number  of queries your program has to answer (0 ≤  ≤ 200). The queries are numbered with consecutive integers from

1 to . ●●int QueryFrom(int i): for the -th query (1 ≤  ≤ ), returns the number of the

student who came up with a solution. ●●int QueryTo(int i): for the -th query (1 ≤  ≤ ), returns the number of the

student willing to know when he or she is going to hear the solution. Output Your program should print exactly  lines to the standard output. The -th line should contain the answer to the -th query, i.e., the number of minutes it takes for a solution to reach from one student to the other.

3.1. Solution

In the problem a cycle is specified using an oracle which for a given vertex returns its two neighbours. The neighbours are returned in an order that is not necessarily consistent with the order of the vertices on the cycle. A number of queries are given, each consisting of two vertices, and the task is to compute for each query the length of the shortest path between the two vertices. The first step of the model solution is to select a subset of vertices, which we call checkpoints. The checkpoints include all the vertices which are part of any query and additionally some number of randomly selected vertices. Later we discuss how to choose this number. After the checkpoints have been selected, each checkpoint is randomly assigned to some machine (node). All the random choices are made with a deterministic pseudorandom number generator so that all machines select the same checkpoints without needing to communicate. In the second step each machine processes the checkpoints assigned to it. Starting from a checkpoint the process running on the machine traverses the cycle in both direc- tions until it reaches (at both ends) any other checkpoints (they might be assigned to a different machine). While traversing the cycle, the process counts the number of visited vertices. Finally, it sends to the first machine a list of statements of the following form: the distance between the checkpoints  and  equals  and there is no other checkpoint between them. 186 A. Karczmarz et al.

The third step is run only on the first machine. First, it receives messages from all the other machines. Using information from the messages, the machine computes the order of checkpoints on the cycle and the distance between each two neighboring checkpoints. After linear-time processing of this information, it is easy to answer each query in constant time. We are left with the problem of choosing the number of checkpoints. Recall that  1 denotes the number of machines. Each machine processes a random fraction of  of the  checkpoints which makes the expected running time of each machine  (  ). However, from a theoretical point of view, this kind of a statement is worthless. Consider an imagi- nary situation in which we need to perform computations that take £( ) total time, and we pick a random machine to perform all of it. Then each machine spends £( ) time 1 with probability  , which gives exactly £( ) expected time for each machine, just like in the case of our solution. At the same time, we are interested in the running time of the slowest machine, which is clearly still £( ) . For this reason, we study the performance of our solution experimentally. Luck- ily, the running time depends mostly on the size of the input data, not on its structure. Our simulations show that with  randomly selected checkpoints, the longest running machine uses  ( lg) time, compared to  (  ) expected time, which is consistent   with a theoretical analysis in (David and Nagaraja, 2003, p. 135). It is possible to reduce the variation between machines by increasing the number of checkpoints. In practice, our solution which always selects 10 000 checkpoints is about 15–2 times faster than the one that selects  checkpoints.

3.2. Tests

It seems that virtually any nontrivial test is sufficient to distinguish solutions based on an incorrect algorithm. However, a bit more care is required to distinguish solutions that are correct but may be too slow – e.g. a variant of the model solution that selects as the checkpoints only the queries endpoints and the vertices 1 2      instead of randomly selected vertices. To make such solutions exceed the time limit we need to pay attention to keep large contiguous fragments of the cycle without any query vertex and leave a large fragment of the initial 1 2      cycle around the vertex 1 unaltered. The remaining part of the cycle is permuted either by performing a circular shift on the binary representations of vertex numbers or by xor-ing them with some fixed number. This method allows  (1)-time calculation of vertex neighbours and produces a cycle looking sufficiently random to make it difficult to come up with a clever incorrect solution exploiting this particular structure of the test.

4. Task “Assistant” (2014)

The life of an assistant is not easy. Not only did the professor order him to write a terribly long review, she also requested some corrections today. Distributed Tasks: Introducing Distributed Computing to ... 187

Pushing keys of a keyboard is very tiring, so the primary goal of the assistant is to push keys as few times as possible, while correcting the review. The keyboard that he is using with a single click allows him to delete a character in the review, change a character to a different one or insert one character anywhere in the review. To make things worse, the assistant has a very peculiar sense of esthetics. He likes letters from the beginning of the alphabet (like a, b or c), but is disgusted by the letters from the end of the alphabet (in particular, y and z). Each time he presses a key and changes a letter that comes earlier in the alphabet to a letter that comes later (for example, m to p), he suffers an esthetic shock, which is devastating for him. Because of that, the secondary goal of the assistant is to minimize the number of such changes. Input The first line of the standard input contains two integers  and  (1 ≤   ≤ 100 000), that specify the lengths of the first and the second version of the review. The following two lines contain the two versions themselves. Each review consists only of lowercase letters. Output Your program should output a single line containing the minimal number of keyboard presses that the assistant has to perform in order to correct the review, followed by the minimal number of esthetic shocks that he will suffer.

4.1. Solution

The problem considered in this task is a variant of the well-known edit distance problem. Our solution will refer to the classical dynamic programming approach to this problem, which has been described in a number of textbooks (see, e.g., Cormen et al., 2009). The solution that we obtain can be easily extended with minimizing the number of esthetic shocks. In short words, it suffices to, instead of storing only the edit distances, store integer pairs that describe the edit distance and the number of esthetic shocks, and compare them lexicographically.

Denote by 1      the characters in the first version of the review and by 1      the characters in the second version. Our goal is to compute a two dimen- sional   matrix , where  ( ) contains the minimum number of changes that £ the assistant has to perform in order to change 1      into 1      . Just like in the edit distance problem, the values  (1 ·) and  (· 1) can be computed in a straightforward way, whereas for 2 ≤  ≤  and 2 ≤  ≤ ,  ( ) can be computed in constant time, given  ( − 1 ),  (  − 1) and  ( − 1  − 1). Assume that the topmost row of matrix  contains elements  (1 ·) and the leftmost column contains  (· 1). Partition the matrix into  stripes consisting of  consecutive columns (for simplicity, we assume that  is divisible by ), where  is the number of machines available. Each machine is responsible for filling in the entries of  in one stripe. Let the -th machine (for  1      ) be responsible for the -th 2 f g stripe from the left. See Fig. 1 for illustration. Assume that the topmost row of matrix D contains elements D(1, ) and the · leftmost column contains D( , 1). Partition the matrix into M stripes consisting · of l/M consecutive columns (for simplicity, we assume that l is divisible by M), where M is the number of machines available. Each machine is responsible for ﬁlling in the entries of D in oneA. stripe.Karczmarz Let the i -thet machineal. (for i 1,...,M ) 188 ∈{ } be responsible for the i-th stripe from the left. See Fig. ?? for illustration.

Fig. 1. The processFigure of 1: Thecomputing process of the computing matrix the . matrix StripesD. assigned Stripes assigned to different to diﬀerent machines have machines have been marked with diﬀerent shades of gray. The numbers inside been marked withthe matrix different specify shades the phase of gray. number, The when numbers the respective inside part the of matrix the matrix specify is the phase number, whencomputed the respective (see below). part of the matrix is computed (see below).

The first machine fills in the first (leftmost) stripe, starting from the topmost row. Consider the rightmost column in the first stripe. Observe that the con- The first machinetents of thisfills column in isthe everything first the (leftmost) second machine stripe, needs to know,starting in order from to the topmost row. fill in its stripe. The entries in the rightmost column of the first stripe are filled Consider the rightmostin by the column first machine in from the top first to bottom, stripe. and asObserve they are being that computed, the contents of this column is everything thethe first second machine sendsmachine them to needs the second to machine. know, The in secondorder machine to fill then in its stripe. The en- fills in its stripe and sends the contents of the cells in the rightmost column of tries in the rightmostits stripe column to the third of the machine, first and stripe so on. are filled in by the first machine from top The correctness of this approach should be clear. However, we need to to bottom, and as improvethey are it a littlebeing bit, computed, in order to make the it efficient.first Clearly,machine each sends machine them to the second requires O(kl/M ) time to fill in its cells. However, all machines (except for the machine. The secondlast one)machine send k messages, then fills each containing in its astripe single number.and sends This may the be verycontents of the cells in the rightmost columninefficient, of its but stripe can be fixed to easily,the third as a machine machine, may send and the contents so on. of cells in batches, each containing b numbers, thus reducing the number of messages to k/b . This obviously does not impact the running time of each machine. The correctness of this approach should be clear. However, we need to improve it a However, there is one more efficiency aspect that we should take care of. little bit, in order toNamely, make we needit efficient. to assure that theClearly, machines doeach not waitmachine long forthe requires numbers  () time to fill in its cells. However,they need to all have inmachines order to perform (except computation. for the If last each machineone) send sends  messages, each containing a single number. This may be very12 inefficient, but can be fixed easily, as a machine may send the contents of cells in batches, each containing  numbers, thus reducing the number of messages to  . This obviously does not impact the running d e time of each machine. However, there is one more efficiency aspect that we should take care of. Namely, we need to assure that the machines do not wait long for the numbers they need to have in order to perform computation. If each machine sends the contents of the rightmost column after filling in its entire stripe (i.e. sends batches of  =  messages), then our solution becomes essentially sequential. On the other hand, we know that  = 1 is also not a good choice, for performance reasons. Let us analyze how to pick a good value of . For the analysis, assume that all the machines are perfectly synchronized, that is, in each time unit a machine can fill in exactly one cell of its stripe (or wait for the data it needs to continue working). Consider now a phase of exactly  time units. Since each stripe has width , the first machine sends the first message to the second machine exactly after the first phase. In the second phase, the second machine starts working (fills in the first  rows of its stripe) while the first one keeps on working on the following rows. From the second phase, the second machine no longer needs to wait, as it receives the necessary data exactly the moment when it needs it. In general, the -th machine starts to work in the -th phase after ( − 1)  =  () time units. Thus, the -th machine starts after  () time units and then works for  () time units. Hence, all the machines finish within  ( + ) time units and send  () messages each. Distributed Tasks: Introducing Distributed Computing to ... 189

By setting  =  we assure that the waiting time is dominated by the computing b c time, which means that our solution parallelizes the single-machine solution in a perfect way. At the same time, the total number of messages sent is moderate ( ( 2)).

5. Task “Sabotage” (2015)

The city of Megabyteopolis was built upon a large lake and consists of a number of isles connected with bridges. The bridges may run above other bridges. A group of saboteurs wants the current president Byteasar not to be reelected. They plan to impact the public opinion by exposing Byteasar’s administration’s helplessness in the case of a major emergency. Specifically, they decided to blow up one of the bridges (they cannot afford blowing up more). The sabotage could be considered successful only if there was no other way between the isles previously connected by the destroyed bridge. Your task is to find the number of bridges that the saboteurs should consider when working out the details. Input ●●int NumberOfIsles(): returns  (1 ≤  ≤ 200 000) – the number of isles consti-

tuting the city of Megabyteopolis. The isles are numbered 0 through  − 1. ●●int NumberOfBridges(): returns  (1 ≤  ≤ 108 ) – the number of bridges in

the city. The bridges are numbered 0 through  − 1. ●●int BridgeIntA(int i): returns the first isle connected by the bridge . ●●int BridgeIntB(int i): returns the second isle connected by the bridge . Output The output should contain a single integer – the number of bridges whose blowup could result in the sabotage being considered successful.

5.1. Solution

In this task we are asked to solve a basic graph problem: for a given undirected graph  = (  ) we need to compute the number of bridges. A bridge is defined here as an edge of  whose removal results in an increase of the number of connected components of . Denote by  () the set of bridges of . A textbook algorithm (e.g., Sedgewick, 2002) for computing  () is based on an extension of the depth-first search (DFS) algorithm and runs in  ( + ) time. The number of vertices in our graph is quite small, i.e., the bound on the order of 105 is typical for graph tasks even in the traditional, non- distributed setting. The number of edges  in our case can be, however, much larger. Unfortunately, DFS is not an algorithm that can be parallelized easily. Nevertheless, we do not need to entirely abandon the idea of using DFS: our strategy is to use the multiple machines to reduce our problem instance to an instance with only  () edges. In such a reduced instance, we use DFS to find the bridges. 190 A. Karczmarz et al.

Definition 1. Consider a graph  = (  ). We define a bridge certificate of  to be a set  ⊆  such that for any  ⊆  ×  ,  ((   )) =  ((   )). [ [ As a result, replacing a subset of edges of  with its bridge certificate does not affect the set of bridges of . We are going to use the bridge certificates to detect and remove edges of . The following lemma describes a construction of a bridge certificate. For completeness we give its proof in the Appendix. Lemma 1. Let  = (  ) and  =  ,  =  . Then there exists a bridge certificate j j j j  of  such that  ≤ 2 which can be computed in  (+) time. j j By Lemma 1, we can take any subset  0 of edges of , find a bridge certificate of

(  0) and replace  0 in  with  ⊆  0 in  (+  0 ) time. We call this step a reduc- j j tion with respect to  0. It turns out that we can easily perform the reductions in a parallel fashion. For simplicity, first assume that we have only two machines, i.e.,  = 2. We partition the edge set  into two sets 0 1 of roughly equal size. The machine , for  = 0 1, performs the reduction step on the set , obtaining a certificate  of size at most 2, in  (+  ) time. Next, machine 1 sends the set  to machine 0. In the last step, j j 1 machine 0 runs DFS to find the set  ((   )). This, however, takes only  () 0 [ 1 time, as   ≤ 4. By the definition of a certificate, j 0 [ 1j  () =  ((   )) =  ((   )) =  ((   )) 0 [ 1 0 [ 1 0 [ 1 This concludes that indeed this approach finds all the bridges of . In order to develop a distributed algorithm using   2 machines, we perform multiple reduction phases. In the first phase, the set of edges is partitioned among the  machines, and each machine computes a certificate of the edges assigned to it. In the following phases the certificates are merged in pairs: two certificates produced in the previous phase are sent to a machine that takes their union and computes the certificate of the resulting graph. Let us describe this process formally. Assuming that the machines are numbered 0 through  − 1, we split the input edge set  arbitrarily into  parts 0      − 1, each of size  (). Our distributed algorithm runs in  = log  + 1 phases numbered d 2 e 0 through  − 1. In the -th phase ( = 0      − 1) only the machines with identifiers  divisible by 2 are active and actually do perform some work. With each active machine  we associate two sets  ,  (   ⊆ ) whose contents depend on the phase number     . Before the -th phase:

●●  is a bridge certificate of the graph

 = (        )   [ +1 [ [ +2 − 1 In the above we set  = if  +  ≥ . + ; ●● If  = 0 then  =  . Otherwise,  ≤ 4.   j  j After the -th phase the set  is a bridge certificate of and  ≤ 2. Note that   j  j it follows that after the phase  − 1,  is a bridge certificate of  and  =  (). 0 j 0j Distributed Tasks: Introducing Distributed Computing to ... 191

At that point the machine 0 runs DFS on (  0) to compute the set of bridges of  in  () time. It remains to show how to implement the phases so that the invariants imposed on the sets   are satisfied. Before the first phase we set  =  . Assume that the phase  − 1 has been completed. We perform a reduction of Lemma 1 on the set  in order to obtain the set  . This takes  ( +  ) time, which is  () for   0 and  ( +  j  j ) if  = 0. The last step is to initialize the sets  before the next, ( + 1)-th phase. +1  To do that, for each  divisible by 2 , we set  =    if  + 2   and   [ +2  =  otherwise. As for all ,  ≤ 2, clearly we now have  ≤ 4. To implement  j j j j this step, the machine  + 2 sends the entire set +2 to the machine . This requires a single message of () bytes. Fig. 2 depicts the phases of our distributed algorithm. In each phase every machine remains idle or sends  () bytes, or receives  () bytes. Consequently, the first phase takes  ( + ) time on each machine and each of the  − 1 remaining phases runs in  () time on each machine. Thus, the time complexity of this solution is  ( +  log).

5.2. Tests

The library providing the test data had to be robust enough to serve graphs with large edge sets and nontrivial 2-edge-connected components (i.e., connected components of  formed after removing all the bridges), given limited time and space. It seems that a hard test case is a graph with a maximum number of edges and possibly large number of bridges. In such a case at least some of the 2-edge-connected components should be very dense.

X0 Ph. 3 Y0

X0 X4 Ph. 2 Y0 Y4

X0 X2 X4 X6 Ph. 1 Y0 Y2 Y4 Y6

X0 X1 X2 X3 X4 X5 X6 Ph. 0 E0 E1 E2 E3 E4 E5 E6

Fig. 2. The phases of the distributed algorithm when  = 7. In this case  = 4 phases are Figure 2: The phases of the distributed algorithm when M =7. In this case performed. The blue arrows illustrate the communication between the machines in the cor- K =4 respondingphases phases. are performed. The blue arrows illustrate the communication between the machines in the corresponding phases.

components of G formed after removing all the bridges), given limited time and space. It seems that a hard test case is a graph with a maximum number of edges and possibly large number of bridges. In such a case at least some of the 2-edge-connected components should be very dense. The infrastructure for generating test graphs provided a general graph interface along with a few specialized implementations (a vertex, a path, a cycle, a clique, a pseudorandom graph, a set of loops) that could serve the edges in O(1) time with constant space consumption, regardless of the graph size. As an example, a cycle on n vertices numbered 0 through n 1 can be represented − with a single integer n: when asked for the cycle’s i-th edge, we just return (i, (i + 1) mod n). Such graphs could be then combined into larger and more sophisticated graphs by unions and direct sums and also extended by adding speciﬁed edges, which were typically used to ensure the desired structural properties of the served graph. For example, this allowed to easily generate a tree of size 100 with each vertex replaced with a random 2-edge-connected graph with 1000 vertices and between 100 000 and 500 000 edges. Such a graph had on the order of 107 edges in total, 99 bridges and could be represented with the number of bytes on the order of 102. At the highest level, the vertices were assigned random identiﬁers, whereas the list of edges was randomly permuted. The size of the in-memory representation of each test cases was O(n) per machine and the sophistication level of the served graphs was limited only by

16 192 A. Karczmarz et al.

The infrastructure for generating test graphs provided a general graph interface along with a few specialized implementations (a vertex, a path, a cycle, a clique, a pseudorandom graph, a set of loops) that could serve the edges in  (1) time with constant space consumption, regardless of the graph size. As an example, a cycle on  vertices numbered 0 through  − 1 can be represented with a single integer : when asked for the cycle’s -th edge, we just return ( ( + 1) mod ). Such graphs could be then combined into larger and more sophisticated graphs by unions and direct sums and also extended by adding specified edges, which were typically used to ensure the desired structural properties of the served graph. For example, this allowed to easily generate a tree of size 100 with each vertex replaced with a random 2-edge-connected graph with 1000 vertices and between 100 000 and 500 000 edges. Such a graph had on the order of 107 edges in total, 99 bridges and could be represented with the number of bytes on the order of 102. At the highest level, the vertices were assigned random identifiers, whereas the list of edges was randomly permuted. The size of the in-memory representation of each test cases was  () per machine and the sophistication level of the served graphs was limited only by the need to return the requested edge in time on the order of 100ns.

6. Conclusions

We described a novel format of programming competitions, aimed at familiarizing students with an increasingly important area of computer science – design of distributed algorithms. While there is a considerable engineering effort involved in preparing the backend for such a competition, we hope that an increasing number of competitions (maybe including IOI in the future) will feature tracks or problems of a distributed nature, to reflect the industry’s shift toward cloud-based and distributed computing.

References

Chávez, L.H. (2015). libinteractive: a better way to write interactive tasks. Olympiads in Informatics, 9, 3–14. Cormen, T.H., Leiserson, C.E., Rivest, R.L. Stein, C. (2009), Introduction to Algorithms. MIT Press. David, H.A. Nagaraja, H.N. (2003). Order Statistics (3rd Edition). Wiley. Dean, J. Ghemawat, S. (2008). MapReduce: simplified data processing on large clusters.Communications of the ACM – 50th anniversary issue: 1958–2008, 51(1), 107–113. Forišek M. (2013). Pushing the boundary of programming contests. Olympiads in Informatics, 7, 23–35. Fox, A. and Brewer E. (1999). Harvest, yield and scalable tolerant systems. In: Proc. 7th Workshop Hot Topics in Operating Systems (HotOS 99). IEEE CS, 174–178. Komm D. (2011). Teaching the concept of online algorithms. Olympiads in Informatics, 5, 58–70. Kulczyński, T., Łącki, J., Radoszewski, J. (2011). Stimulating students’ creativity with tasks solved using pre- computation and visualization. Olympiads in Informatics, 5, 71–81. Sedgewick, R. (2002). Algorithms in C++, Part 5: Graph Algorithms. Addison-Wesley Longman Publishing Co., Inc., Boston. Distributed Tasks: Introducing Distributed Computing to ... 193

A. Karczmarz (1990), PhD student at Faculty of Mathematics, Infor- matics and Mechanics, University of Warsaw, Poland, jury member of multiple programming contests organized by University of Warsaw, coorganizer of Algorithmic Engagements 2015. In his research he focuses on algorithms and data structures.

J. Łącki (1986), postdoctoral researcher at Sapienza University of Rome, Italy, IOI Scientific Committee elected member, appointed chair for 2016, responsible for task selection at Algorithmic Engage- ments for many years, former head organizer of Polish Training Camp. In his research he focuses on graph algorithms.

A. Polak (1991), PhD student at Department of Theoretical Computer Science, Faculty of Mathematics and Computer Science, Jagiellonian University, Kraków, Poland, judge at the ACM Central Europe Re- gional Contest in 2012, 2013, and 2014. His research interests lie in algorithms, complexity theory, and computer vision.

J. Radoszewski (1984), assistant professor at Faculty of Mathemat- ics, Informatics and Mechanics, University of Warsaw, Poland, and Newton International Fellow at King’s College London, UK, chair of the jury of Polish Olympiad in Informatics, co–chair of the Scientific Committee of CEOI’2011 in Gdynia, former member of Host Scien- tific Committees of IOI’2005, CEOI’2004, BOI’2008, and BOI’2015. His research interests focus on text algorithms and combinatorics. J.O. Wojtaszczyk (1980), Staff Software Engineer at Google, Warsaw, judge at the ACM ICPC World Finals in 2011, 2012, 2013, 2015 and 2016, coorganizer of the Google Code Jam since 2012, main organizer of the Distributed Code Jam. The primary focus of his engineering work is around cluster management. Appendix: ProofAppendix: of Lemma Proof ?? of Lemma ??

Lemma ??. Let H =(LemmaV,E) and ??.n =LetV H, m=(=V,EE ).and Thenn there= V exists, m = aE bridge. Then there exists a bridge | | | | | | | | certificate X of H suchcertificate that X X2nofwhichH such can that be computedX 2nA. inwhich KarczmarzO(n can+ m be)et time.al. computed in O(n + m) time. Appendix: Proof of Lemma ?? 194| |≤ | |≤ Proof. We compute theProof. set X Wein the compute following the way. set X First,in the compute following some way. span- First, compute some span- Lemma ??. Let H =(V,E) and n = V , m = E . Then there exists a bridgeAppendix: Proof of Lemma 1 | | ning| | forest FAppendix:(we identifyning it withforest Proof aF set(we of of itsidentify Lemmaedges) it of withH using a ?? set of any its graph edges) search of H using any graph search certificate X of H such that X 2n which can be computed in O(n + m) time. | |≤ algorithm. ThisAppendix: takes Oalgorithm.(n + m) Prooftime. This Then, takes of computeO Lemma(n + m) sometime. ?? spanning Then, compute forest F some spanning forest F  Lemma ??. Let H =(V,E) and n = V , m = E . Then there exists a bridge Proof. We compute the set X in the following way.of H First,=(V,E computeF ). Finally, someof H span-Lemma set=(XV,E= 1. FLetF )F.  Finally,=. Clearly,( |) set and| XX  ==|2F n| and,F . X= Clearly, E. Then. X there2 nexistsand aX bridgeE. certificate certificate\LemmaX ??.of HLetsuchH =( that\V,E∪X) and2nnwhich= V| , can|≤m = bej ∪ computedEj . Then⊆j inj thereO|(n exists|≤+ m) atime. bridge ⊆ ning forest F (we identify it with a set of its edges)We of H nowusing prove any that graphX searchisWe indeedof now  such prove a bridge that| that|≤  certificateX≤ is2 indeed |which| of H can a. bridge Let|be| computedY certificateV inV . of (H+. Let) time.Y V V . certificate X of H such that Xj j 2 n which can be computed⊆ × in O(n + m) time.⊆ × algorithm. This takes O(n + m) time. Then, computeFirst, let some usProof. show spanning thatWecompute forestFirst,the graphsF let the usH show set1 =(X thatV,E|in|≤ the theY following) graphsand HH2 way.1=(=(V,X First,V,E computeYY))haveand someH2 =( span-V,X Y ) have Proof. We compute the∪ set  in the followingu ∪ ∪way.v First,H compute someu ∪ spanningv H forest of H Appendix:=(V,E F ). Finally, Proof set X of= F LemmaF . Clearly,the ??sameX connectedning2Proof.n and forest components.XWeFthe compute(weE same. identify connected theClearly, it set withX if components.in a there set the of followingexists its edges) Clearly,a path way. of H First,ifusing there computein any exists2 graph a some path search span- in 2 \ ∪ | |≤u v ⊆  (weH identifyuH vit with a set Hof itsH edges)H of  using anyu graphH search algorithm.u This We now proveAppendix: that X is indeed aProof bridge certificate ofthen Lemma a path of Halgorithm..ning Let ?? forestYexists ThisthenVF in(we takes aV path.1 identify, asO(n+2 itism with) aexiststime. subgraph a set in Then, of1 itsof, ascompute edges)1.2 Conversely,is of asomeH subgraphusing spanning if any of graph forest1. Conversely, searchF  if ⊆ × 0 0 Lemma ??. Let H =(V,E) and n = V and, m v=areE connected. Then there in andH exists1 takesbyapathv are a bridge connected ( P+, then) time. in anyH Then,1 edgebyapath compute(a, b) P ,someP thenY spanning anysuch edge that forest(a, b ) ofP Y=such (  that ). Fi- First, let us show that the graphs H1 =(Appendix:V,E| | Y ) and| |H Proof2of=(algorithm.HV,X=(V,E ofY LemmaThis) haveF ). takes Finally,O ??(n set+ mX)=time.F F Then,. Clearly, computeX some2n spanningand X forestE. F  n ∪ ∪ 0 ∈ \ ∈ \ certificate XLemmaof H such ??. thatLetXH =(2nV,Ewhich)(anda, bcan) n be=E computedVX, mcan=u be inE replaced.O(va, Then(n\ bnally,+) Hm there)E by settime. a X exists path can= a a be bridge replacedb. containedClearly,∪ by  a entirely path≤ 2a |and in|≤ Fbcontained (recall⊆ . entirely⊆ in F (recall the same connected components.| Clearly,|≤ if there exists∈ |\ a| pathofWeH now|=(| V,E provein ∈F that2).\ Finally,X is indeed set[ X a= bridgeF Fj certificate.j Clearly, ofXH. Let2 n andY XV EV. . certificate X of H suchLemma that thatX ??.F2Letniswhich aH spanning=( canV,E be) forest computedandthatn ofF\=WeHis inV). anow,O spanningm(n prove=+ mE) that. foresttime. Then  ofis there indeed∪H). exists a bridge a bridge |certificate|≤ of ⊆. Let⊆×  ⊆  ×  . First, let then a path u  v exists in H1,X as H2 is a subgraph| |≤ of HFirst,1. Conversely,We let usnow show prove if thatu that theX graphsis indeedH1 =( a bridgeV,E certificateY ) and H of2 =(H.V,X Let Y Y ) VhaveV . Proof. We compute the set in the followingX way.H First, computeX some2n | span-| | |  = (O ∪(n + m ))  = ( ∪ ⊆  ×) and v are connected in H1 byapathP ,certificate then any edgeForof a(a, graph bsuchthe) samePG that, defineY connectedsuchGForus that show awhiche components. graphas thatG can withtheG, be graphs define computedthe Clearly, edgeH G1 =(e ine ifremoved.V,Eas thereG withY exists) Nowtime. theand aH assumeedge path2=( euV,Xremoved.v inY )H Nowhave assumethe same con- ning forest FProof.(we identifyWe compute it with the a set set ofX itsin edges) the following of H∈First,using way.\ let any| First, us|≤ graph show− compute search that the some graphs span- 1 − [ and 2  [ have2 (a, b) E X a bthat (u, v) is a bridge inFthatH1nected,( i.e.,u, v) (components.isu, av) bridgeB(H inClearly,1).H Then,1, i.e., if there there(u, v exists∪) is noB (aH pathpath1). u Then,v therein ∪2 then is no a pathpath u v exists can be replacedO(n by+ am path)  containedthen entirelythe a same path in connectedu(recall v exists components.F in H1, as H Clearly,2 is a subgraph if there exists of H1 a. path Conversely,u  v ifinuH2  ∈algorithm.\ Thisning takes forest F (we identifytime.Proof. it Then, withWeHcompute a set( ofu, vits) thesome edges) set spanningHX ofinH(H theu,using forest v) following(u, any v) graph∈ way.H search First,(Hu, compute v) (u, v) some∈ span- H (u, v) that F is a spanning forest of H). in 1 and.v Asare connected2 in inu1 1,isv inas aH  subgraph1.2 byapathis As a subgraphH2 ofP ,H 1of then is1. any aConversely, subgraph, edge there(a, is bif of) also H andP1 no Y aresuch connected, there that u is in also 1 by no a path of H =(V,Ealgorithm.F ). Finally, This set takesX ning=O(Fn + forestFm).time. Clearly,F −(weAppendix: Then, identifyXthen compute2 it an with pathand− someProof aX set− spanningofE. itsexists of edges) Lemma in forest of1H,−F asusing 2 ??−is any a graphsubgraph search of 1.− Conversely, if G \ G e G ∪path ue v |in|≤H2 (u,path v).,u ⊆then Moreover, anyv in edgeHH2 1(and( u,) vH).2 Moreover,have such the that sameH1 (and connected ) H∈ 2have\  thecan samebe replaced connected by a path  For aWe graph now, prove defineH =( thatV,EX isasF indeed) with a bridge the edgeX certificate= Fremoved.(Fa,and of b) Hv Now.E−are LetX connected assumeYXcanV be2n replaced inV .HX1 byapath− byE a pathP ,a then b anycontained edge (a, entirely b) P inYF such(recall that of  − . Finally,algorithm. setcomponents This takes and.O Clearly, thus(∈n +\(mu,components) vtime.) X Then,andY and. compute Thus, thus (.u, some v2) spanningXnB (HY).and Thus, forest consequently(Fu, v2) ∈ Bn( \H ) and consequently that (u, v) is a bridge in H1, i.e.,\(u, v) HB(H=(1)V,E. Then,YLemma there)∪that( isa,HF nob??.)=(is path aLetEV,X spanning|HuX|≤⊆=(canYv)V,E contained× forestbe) replacedand of ⊆Hnentirely).= byV a ,in pathm  =(recallaE2. bthat Thencontained  thereis a spanning exists entirely a forest bridge in2 F of(recall ). First, let us showWe that now the prove graphs that∈ofX1His =(B indeed(V,EH ) aF bridgeB)and(.H Finally,) certificate2 ∈ setB\X(H of=∈)HF.Bhave∪ LetF(H.Y) Clearly,V | XV|. ∈2n |∪and| X E. ∈ in H1 (u, v). As H2 (u, v) is a subgraph of H11∪\certificate(u, v)2that,For there. XF aof graphis isH a also spanningsuch∪1G no,For definethat∪ a graph forestX2G. ⊆  of2,e ndefineHas×which).| G|≤ with can−  thebe as computed  edge with⊆e theremoved. in edgeO(n + removed. Nowm) time. assume Now assume that ( ) the− same connectedFirst,− let components. us show that Clearly, theWe graphs now if there proveH−⊆1 =( exists thatV,EX a pathisY ) indeedandu H a2v⊆ bridge=(in HV,X2 certificateY ) have of H. Let Y V V . Finally, suppose that (u,Finally, v) B( supposeH| 2)|≤. Then,− thatu(u,and v)v are B( connectedH2). Then, inuHand2, butv are connected in H2, but path u  v in H2 (u, v). Moreover, H1 and H2 havethat the same(u,For∪ v) aconnectedis graph ais bridge a Gbridge, indefineH in1 ,∪G i.e.,1, i.e.,(eu, (as v) G) Bwith(H 1( the).1 Then,)⊆. edge Then,× theree thereremoved. is is no no pathpath Now u assume v in 1 − ( ). As then a path theu− samev exists connected in H1, components. asFirst,H2 letis a us subgraph showClearly, that of if the thereH1. graphs Conversely, existsH a1∈ path=( ifV,Euu  Yv )inandH2H2 =(∈ V,X Y ) have (u, v) X Y (u,no v path) BProof.u(H)v existsWe compute innoH2 path the(u,u set v).X Asv existsinH the2 −is in following aH subgraph2 ∈(u,2 way. v) of. AsH First,1,H(2u,is compute v) a subgraphE someY . of span-H1, (u, v) E Y . componentsv and thus H . Thus,P 2in that(andHa,1 b) consequently(u,(u,P v) v)isY. a As2 bridge− H(2  in) (isu,H∪ 1a v, )subgraph i.e.,is a(u, subgraph v )of B1(−H of(1)H∪. 1 Then,), there(u, there v )is, also there is nono is pathpath also u no v in 2 − ( ). and are connectedthen a path in∈ u1 byapath∪ v existsthe same in, thenH1 connected, any∈ as ningH edge2 is forest components. a subgraph−F (we identify ofu− Clearly,suchH1.v that it Conversely,− with if there a setH exists ifofu its(u−u, a edges) pathv∈) v u ofH vusing−inHH∈ any2 (u,∪ graph v) search ∈ ∪ B(H1) B(H2). Let us showpath thatHu no∈ pathv(Letu,\in v us)H2 show(Hexistsu, that v).(u, in noMoreover, v) path1 H1.and AssumeexistsH2 H inhave the1( contrarytheu, v) same.connected Assume the contrary (a, b) E Xandcanv are be replacedconnected by in aH path1 byapatha bucontainedP ,v thenin any entirely1 edgeH in(Moreover,a,.F bH) As(recallP 2 Y1 andsuch  that2is −have a subgraphH the same ofconnected1 u −components, there and is also thus no ( )   . ⊆ then a path algorithm. exists This in− 1 takes, as O−2(nis+ am− subgraph) time. Then, of 1 compute. Conversely, some− ifspanning forest F  2 [ Finally, suppose∈ \ that (u, v) B(H2). Then, andu and letv areP be connectedcomponents such au path. inandH andv2, letIf but thusPHP∈be(u,X(\ suchvu,) vY) , aX then path.Y .P Thus,IfwouldPH (u, beX v) a HYu,B then(Hv path)Pandwould in consequently be a u  v path in that F is a spanning(a, b) E forestX can of H be). and replacedv are by connected a path a inpathHb containedbyapath Thus,in entirelyP 2,(⊆ then )∪ in anyF. edge((recall Moreover,2) (anda, b )consequently⊆P1 andY∪such 2 have( that12) the⊆  same (2). connected ∈ H (u,of v),H which =(V,E is1 notHF possible.). Finally,(u, v),− Hence, which set2X∈ there is= notF∪ exist possible.F . some Clearly, Hence, edgesX∈( therea, b2)n existandP such someX E edges. (a, b) P such no path u  v exists in ∈H2 \(u, v). As H2 is a subgraph2 ofBH(componentsH1,1()u, vB) (H2E2 and). Y . thus (u, v) X Y . Thus,∈ \(u, v) B(H ) and consequently For a graphthatGF,is define− a spanningG e as forest(a,G b)with of EH− the).X edgecan bee removed. replaced\∈ by Now−∪ a pathFinally, assumea supposeb contained that∪ ( entirely )  in (|F2|≤)(recall. Then,∈ 2 and⊆  are connected∈ in 2, but no that (a, b) WeE nowX⊆ prove(thatu, v that)(.a, As bX) is(a,E indeed b) X/ ∈F aand( bridgeu,∪ v()a,. b certificate) As/2(F a,, b there) / ofF∈H existand. Let( pathsa,Y b) /VF ,V there. exist paths Let us show that no path u  v exists− in H∈1 (\u, v). AssumeBFinally,(H1 the) suppose contraryB(H2). that (u, v) B(H2). Then, u and v are connected in H2, but that (u, v) is a bridgeFor a graphin H1,G i.e.,, define(u,that v)GF Bis−(e aHas spanning1).G Then,with∈ forest there\ the edge\{ ofis noHe). pathpathremoved.} u ∈ v Now \exists∈ assume\{ in 2 −}( ∈). As ∈2 is a subgraph⊆ of∈ ×1, ( )   . Let us Q and QFirst, from leta usto showb⊆inQ bothand thatQ spanning the from graphsa foreststo∈Hb1in=(F bothV,Eand spanningF Y,) correspondingly.and forestsH2 =(FV,Xand AtFY,) correspondingly.have At and let P be such a path. If P X Y , then∈ −P would beno patha uFinally,u v pathv supposeexists in in thatH2(u,( vu,) v).B As(H2H).2 Then,is a subgraphu and v are of H connected1, (u, v) inEH2Y, but. 2 [ in H1 (u, vthat). As(u,H v)2 is a(u, bridge v) is in aH subgraphFor1, i.e., a graph(u, of v)HG1,B define((Hu,1 v).)G, Then, theree asshow there isG also withthat is no no no the pathpath edge u e vremoved. exists∪ in  Now1 − ( assume ). Assume∪ the contrary and let  be such a ⊆ ∪ least onethe of same these connected paths,least say one components.Q, of does these not− paths, containClearly,∈ say( ifu,Q there v,). does We exists not may containa replace path u(u, the v)v.in We∈ H may2∪ replace the H2 (u, v), which− is not possible.− Hence, there exist some∈ edgesLet−no us( patha, show b)−u P thatsuchv exists no path in Hu (vu,exists v). As inHHis1 a subgraph(u, v). Assume of H, the(u, v contrary) E Y . path u vininHH12 (u,(u, v v).). As Moreover,H2 ((u,u,H v v1))isand a subgraphH2 have theH of H same1 (path.(u, connectedu, v v) )If, thereB (⊆H2) is also , nothen  would2 be a u v path1 in  − ( ), which is not pos- −  that is(a, athen bbridge) a in path1,u i.e.,∈ Qv exists(a,X b) in H11,.− as Then,H2 Qis there a subgraphX is no− path(a, of bH) 1. Conversely,2 if∈(ua,∪ b) that (a, b) E X (u,−− v) . As (a, b) /−F andedge(a, b) / Fwithand,Let there thelet usP path exist showbeedge− such paths that a∈ no. path.with By path replacing the Ifu[P path vX allexists suchY , inthen. edges ByH P replacingwould(u, vwith). be all Assume paths a suchu edges thev path contrary in with paths componentspath and thusu (u,v vin) H2X in(u,Y H. v) Thus,. Moreover,(u,( vu,). v) AsHBH1(Hand2)(u,andH2 v)have consequentlyis⊆ athe subgraph same connected of H (u,⊆ v), there1( is also) no  ( )   ( ) ∈ \ \{ } ∈ 1 containedand∈ in vXare,2 we connected obtaincontainedsible. a inpathH Hence,1 inubyapathX, vthere wein obtainH ⊆exists1P , then(∪ au,some path v any), aedgeu edgecontradiction. −v(a,in bH) Psuch( Thus,u,Y v )thatsuch, a contradiction. that Thus, . Q and Q from a to b in both spanning∈ −∪ forests−F and F ,∈H correspondingly.2and−(u, let v)P, whichbe such At is not a path. possible. If P2 Hence,− X thereY , then exist P somewould22 edges be( aa,u b) vP pathsuch2 inn n f g B(H1) B(componentsH2). and thus path(u, v)u X v inY .H Thus,2 (u,(u, v) v.) Moreover,BAs(H 2() and H) 1 consequentlyand andH 2(have− )  the 0, samethere exist connected paths∈− \ and  0 from  to  in both spanning ⊆ Q (u, v()u, vB()a,(H b)1) −and,E X consequently,can(u, v be) replacedB(HB1()Hand,2 by) a consequently,B path(H⊆1)a. ∪ b containedB(H2) entirelyB(H1). in F (recall ∈ least one of these paths, say (u,, does v) B not(H contain) ∈ ∪u . Wethatv−∈H may2(a,\( bu, replace) v∈),E which theXH is( notu,2 v possible.) . As (a, Hence, b)2 / F thereand exist(a, b) some/ F , edges there(a, exist b) pathsP such Finally, supposeB(H1) thatB(H2). components2 . Then, and∈ andthat thusFare(isu, connected− a v spanning) ∈X forests\ inY∈ forest.\{2 Thus, , but ofandH( u,}).⊆ 0 v, )correspondingly.B(H2∈) and consequentlyAt least⊆ ∈one of these paths,∈ say , does not contain edge (a, b) with the path ⊆Q X. By∈ replacing all such edgesQ thatand(a, bQ()a,with∈from b) paths∪Ea toXb in( bothu, v) spanning.∈ As (a, b forests) / F andF and(a,F b),/ correspondingly.F , there exist paths At no path u  v existsFinally, in supposeH2 (u, that v)B.( As(Hu,1H) v)2 isBB a(H( subgraphH2)2.). Then, ofuHand1, (u,v are v) connectedE Y . in H2, but X ⊆ −u v H (u, v) For a graph G∈,( define∈\ ).∪ \{WeG maye as }replaceG with the the edge∈ edge ( e)removed. with ∈the path Now  assume ⊆ . By replacing all such contained in , we obtain a pathu v in 2 ⊆∈H ,( au, contradiction.least v)Q and oneQ of thesefrom Thus,a paths,to b in− say bothQ, spanningdoes not containforests F(u,and v).F We, correspondingly. may replace the At Let us showno that path nou path v exists inexistsHFinally,2 in−(u, v suppose1). AsthatH2 that(u,.is v Assume a)(u, subgraphis v a) bridge theBedges(H of contrary in2H) .(H1 Then,, ( ,u, i.e.,) vwith)u(andu, Epaths v)vYare .containedB( connectedH ). Then, in  in, thereHwe2 ,obtain but is no a pathpath u v in  − ( ), a con- (u, v) B(H1) and, consequently, B(H2) B−(H1). − edgeleast(a, one b) with∈ of these the path paths,1 Q say∈XQ.∪, By∈ does replacing not1 contain all such(u, edges v). We(a, may b) with replace paths the2 ∈and let P beLet such us a show path. that If P no pathno⊆X pathuY,u thenv existsv PexistswouldH in inH(H1 beu,2 v a()u,(uu, v) v.).Hv Assume AspathH(2u, inis vthe) a⊆ subgraph contrary of H1, (Hu, v) (u,E v)Y . ⊆ ⊆ ∪ in contained1 −(a,− in. b) AsX,tradiction. we2 obtain Thus, aQis path ( aX subgraph u) vin ( ofH1) and,1 (u, ∈consequently, v),∪ a, contradiction. there ( isa, ( balso)2) no Thus, (1). H2 (u, v), whichand let isP notbe possible. such a path. Hence, If thereP existX someY ,edge− then edgesuP (woulda,withv b) be theP− asuch pathu H v path(u, v.) in By2 replacing2 − all such edges with paths Let us show thatpath nou pathv inH2 exists(u, v in). Moreover,1 ⊆ .H1 Assumeand H−2 thehave contrary the same connected − ⊆ ∪ (u,contained v) B(H1 in) and,X∈, we consequently, obtain− a pathB(uH2) v inB(HH1). (u, v), a contradiction. Thus, that (a, b) HE2 X(u, v)(,u, which v) . is As notand(a, possible. b) let/ PF beand Hence, such(a, ba there) path./ F∈ exist, there If P some exist−X edges pathsY (,a, then b) PP wouldsuch  be⊆ a u 2 v path in ∈ \− \{ } ∈ components(∈u, v) andB(H thus⊆) (u,∪ v) X ∈ Y . Thus,B(H ()u, vB) (H−B)(H2) and consequently Q and Q fromthata(toa, bb) in bothE X spanningH(u, v)(u, forests. v As), which(a,F band) is/ notF ,and correspondingly.possible.(a, b) 1/ Hence,Fand,, there thereconsequently, At∈ exist exist∪ paths some edges2 (a,∈ b) 1 .P such ∈ \ \{ 2 − } B(H∈1) B(∈H2). ∈ ⊆ ∈ least one ofQ theseand paths,Q from saya Qto,b doesthatin both not(a, b contain spanning) E (Xu, forests v).⊆(u, WeF v) mayand. AsF replace(,a, correspondingly. b) / theF and (a, b) At/ F , there exist paths ∈ \ Finally,\{ suppose} that (u,∈ v) B(H2). Then,∈ u and v are connected in H2, but edge (a, b) withleast the one path of theseQ X paths,.Q Byand replacingsayQQ ,from does alla not suchto b contain edgesin both(a,(u, spanning b) v)with. We paths forestsmay∈ replaceF and theF , correspondingly. At ⊆ no path u  v exists in H2 (u, v). As H2 is a subgraph of H1, (u, v) E Y . contained inedgeX, we(a, obtain b) with a the path pathleastu Qv oneinXH of.2 By these( replacingu, vpaths,), a contradiction. sayall suchQ, does edges not Thus,(a,− contain b) with( pathsu, v). We may replace the ∈ ∪ ⊆ −Let us show that no path u  v exists in H1 (u, v). Assume the contrary (u, v) B(Hcontained1) and, consequently, in X, we obtainedgeB(H2 a()a, path b)Bwith(uH1) the. v in pathH2 Q (u,X v)., Bya contradiction. replacing all such Thus, edges (−a, b) with paths ∈ ⊆ and let P −be⊆ such a path. If P X Y , then P would be a u  v path in (u, v) B(H1) and, consequently,contained inBX(,H we2) obtainB(H1 a). path u v in H ⊆(u, v∪), a contradiction. Thus, ∈ H2 ⊆(u, v), which is not possible.2 Hence, there exist some edges (a, b) P such (u, v) B(H ) − B(H ) B(H− ) ∈ 1 thatand,( consequently,a, b) E X (u,2 v) . As1(a,. b) / F and (a, b) / F , there exist paths ∈ ∈ \ \{ ⊆} ∈ ∈ Q and Q from a to b in both spanning forests F and F , correspondingly. At least one of these paths, say Q, does not contain (u, v). We may replace the edge (a, b) with the path Q 19X. By replacing all such19 edges (a, b) with paths ⊆ contained in X, we obtain a path u v in H (u, v), a contradiction. Thus,  2 − (u, v) B(H ) and, consequently, B(H ) B(H ). ∈ 1 2 ⊆ 1 19

19 19

19 19 19