Assignment 4
Due 5 Nov by 16:59 Points 100 Submitting a file upload File types pdf
Start Assignment
This assignment is for students enrolled both in the PG and UG versions of this course. Note that this is a group assignment, with groups of
maximum three students. Please write the student ids at the top of the submitted report, and please make a single submission per team.
Objective
To understand the performance characteristics of a distributed consensus protocol and to communicate these characteristics in writing, through a
performance report.
Description
In this assignment, you will be evaluating the performance of the Paxos implementation you implemented in assignment 3. Alternatively, you can
also use an existing Paxos implementation, either found by yourself or from the list below.
You will be designing your own experimental plan and reporting on your findings through a written report. The experimental plan will aim to answer
the following questions:
What is the performance in terms of runtime (and other metrics) of your system given specific configurations, in a functioning mode without
failures?
What is the performance in terms of runtime (and other metrics) of your system given specific configurations, in a functioning mode with failures?
In your experimental plan, you will be using at least two metrics. The first is the runtime (measured in seconds or milliseconds as appropriate): you
will have to define this runtime, and you will have to think very carefully about what you are measuring. Regardless of the functioning mode
(with/without failures), there will be parameters that will influence this runtime. One could be the number of nodes in the system, whereas another
might be the number of simultaneous proposers. You are expected to present at least one plot showing how the runtime varies with one of these
parameters. Since the conditions of your experiments might vary slightly, even if your scenario is not stochastic, you will have to present an average
of at least ten runs, as well as the standard deviation, with your results.
You are responsible for defining the second metric, based on what you know about what the protocol does and also what you’ve learned about other
protocols, such as the voting protocols discussed in the lectures. Similarly to the first metric, since the conditions of your experiments might vary,
you are expected to present average metrics of at least ten runs (and the standard deviation).
The structure of the report is as follows:
1. Introduction (worth 5% of your mark)
1. In this section, you will be describing the protocol and the implementation you choose to use
2. Make sure you cover the purpose of Paxos as well as key points of the implementation that might influence you results
2. Experimental Setup (worth 20% of your mark)
1. In this section, you will be describing your experimental setup, including how you calculate the metrics and the experimental framework you
use to run your experiments (your scripts, how you save and interpret your data, etc.)
3. No Failure Mode (worth 30% of your mark)
1. In this section, you will be presenting the results of your experiments when there are no failures in your system
2. Make sure that all your plots have axes and all axes have labels and units
3. Make sure you include a discussion subsection where you discuss your results
4. With Failure Mode (worth 30% of your mark)
1. In this section, you will be presenting the results of your experiments when there are failures in your system
2. Make sure that all your plots have axes and all axes have labels and units
3. Make sure you include a discussion subsection where you discuss your results
5. Conclusion (worth 10% of your mark)
1. This section briefly concludes your report, using the discussions from sections 3 and 4 as starting points
2. Discuss what you have learned about Paxos and its performance
Your report should have a minimum of 3 pages and a maximum of 4 pages, Arial 11 with 1.5 cm margins. The quality and clarity of your writing is