The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 They are getting some seed funding from cant role in these developments. We now could go into the dark fiber market and sell the Alberta government and that is to be used have many carriers building national infra- dark fiber and a fraction of what everybody to enable every community have low-cost structure that is independent of the old else was selling it for in order to gain mar- bandwidth such that any competitor, ISP, PSTN. ket share. So the question that we keep ask- business, or school can buy fiber strands. ing ourselves is whether the economic ben- COOK Report: Do you have any feeling at efits that we’re seeing now merely because COOK Report: So they are setting a kind of all that the cost effectiveness may move even we are leading the market place or are they green field example of running the fiber train beyond the range of 1000 to one? maybe there is a much more fundamental through for the first time and showing ev- driver? eryone how to jump on board and play the St Arnaud: It’s hard to speculate on that. game. What about dark fiber networks in the Nevertheless we are, right now, really down We are starting to understand some network provinces in general? to the true cost. We know how much it costs fundamentals that show that carriers cannot to deploy dark fiber as well as the incremen- gain huge economies of scale, and that, as a St Arnaud: A number of other provinces tal costs and the equipment. For the first time result, their costs would still remain higher. have are looking into building condominium in history we have a telecommunications dark fiber networks like Alberta. Our fed- system that is based on real input costs. The COOK Report: And the reason for this? eral government has just created a National biggest cost component is the installation and Broadband Task Force to look at how the deployment of the fiber. In my opinion, for “Scaling Issues on government can help accelerate broadband the moment, we’ve pretty well on a plateau. to every community, residence and business However, we may get some increasing the Internet Networks” and in Canada. On the research and education economy of scale as we put in more fiber side Newfoundland has a 1700 km 4 chan- strands. OBGP nel GbE DWDM network and we are just finalizing 250 km 2 GbE network to Prince COOK Report: On the other hand some is- St Arnaud: It is what is called the “n” Edward Island. The British Columbia uni- sues are beginning to emerge that might put squared phenomena. Namely their costs will versities will announce their optical network an end to the increases in cost efficiency. If go up by the square of the number of users plans very soon which will be much more some of the issues with increasing traffic and connected and the squaring of anything pro- along the CA*net 4 concept than CA*net 3. connections about which you write in your duces really dramatic results. Today’s net- New Brunswick and Ontario are getting new “Scaling Issues on Internet Networks” works can be visualized as snowflakes. If close. Nova Scotia will also announce soon. paper turn out to be verifiable and accurate, we’re talking about a telephone network for we may find that there is a point at which example, traditional carriers get significant new investment in costly equipment may economies of scale because “n” is stable and A Cost Effective National have to be made just to maintain the status growing very slowly. In the snowflake model Infrastructure quo. Correct? you get then economies of scale by aggre- gating traffic at network nodes. Today, the Independent of the PSTN St Arnaud: That’s right. The other issue is bigger the telephone network, the more sig- that because of what someone called the “n” nificant the economies of scale and the more COOK Report: What seems to be highly sig- squared phenomena is the inverse economies profitable such a network can be. nificant about the situation in Canada and of scale. In other words that operating a very indeed unique in the world, except perhaps big network in proportion to the number of With the Internet “n” is always growing be- for Sweden, is that you have the school and users may be actually more costly than op- cause it is related to the number of comput- municipal dark fiber build outs merging into erating a smaller network. ers and networked devices. provincial wide public sector networks and finally into the CANARIE National back- COOK Report: A big network in the sense COOK Report: But in talking in terms of bone. You a have just about completed a of a global Internet? Or a big network in the the Internet are you saying then that a way national public-sector telecommunications sense of a large autonomous system? to escape what may become the problems infrastructure having for the first time abso- of “n” squared is rather than running a single lutely nothing to do with the public switched St Arnaud: All of our work on dark fiber huge global network is instead to run an telephone network. While the dark fiber and and OBGP was initially intuitive in that we Internet composed of many topologically gigabit Ethernet that is run over that fiber is could do some things considerably less ex- limited smaller peripheral systems, like cur- the major component, in order to make all pensively than the carriers with their man- rent Internet autonomous systems? And to this work independently of the public aged network services. The question in our connect all of these by means of OBGP ac- switched network and of the carriers, it mind then became one of whether or not cording to the patterns that the users in each seems that you need the optical border gate these savings were just an artefact of still of them desire? way protocol (OBGP). living in a world of monopoly inefficiencies or some other fundamental factors. Did these St Arnaud: That’s correct. We are discov- We know that with OBGP your end users savings materialize only because we were ering that, in the Internet at large, size, does can manage their respective parts of the new leading the marketplace? We wanted to not bring those economies of scale. In fact public sector network. Consequently, for the know if we face a situation where in time, that there is a certain size beyond which the first time you may have been some means as carriers themselves start to move in this economies of scale may be actually inverted. of demonstrating the enormous cost effec- direction of dark fiber networks that with tiveness of this new technology which when their large economies of scale, all our cost COOK Report: Are you speculating that one you and I first talked about this about three savings would start to disappear? way to get out from under this conundrum years ago you pegged at 100 to one and may be to have a series of customer owned which now may be 1000 to one. Do you Could carriers with their very large finan- networks that form the edges of the Internet agree? cial muscle reach a point where they could and, honeycomb like, are all interconnected achieve greater efficiencies and negate the with each other using OBGP ? St Arnaud: Yes. But I must point out that usefulness of customer owned dark fiber and our carrier partners have played a signifi- wavelengths? Hypothetically a huge carrier St Arnaud: That’s right. What we think may 61 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA scale far more efficiently is customer owned routers and sets up wave lengths between COOK Report: And some of the spectrum fiber with customer owned wavelengths and them. To do this in a satisfactorily cost-ef- of solutions at least are outlined in your pa- massive peering. Now by customer I mean fective way, we need a very simple switch per? either an ISP or a large organization. I don’t without a whole lot of legacy protocol ca- mean you or I. The reason why we still have pability built into it. For this one task we St. Arnaud: Correct . thousands of ISPs left in the world and why don’t need all the complexity and concomi- they may not be disappearing as rapidly as tant cost that most switches come with. COOK Report: Your awareness of these is- predicted is that at some point in their de- sues came primarily from where? The pa- velopment the economics of scaling actu- Scaling the Internet per by Andrew Oldyzko and issues raised ally begins to work in favour of the smaller by Mike O’Dell? ones. COOK Report: Let’s look at your new pa- St Arnaud: Yes. And what’s more all of this COOK Report: I understand that you are per on Internet scaling issues. If the hypoth- has happened within the last 60 days or so. working presently on two flavors of OBGP. esis on the problems in growth of connec- From our point of view the paper has been Would you please explain what they are. tions as well as bandwidth are verified, what driven by two things. First by issues raised is learned will have serious implications for with Mike O’Dell in the fact that Uunet’s St Arnaud: Currently we are going in two the topology of and way in which the entire network capacity is growing faster than its strategic directions with OBGP. One direc- global Internet is run. Never mind customer offered load. And second curiosity about tion which Marc Blanchet is working on is owned dark fiber. why all this dark fiber and customer owned to modify the existing BGP code. His group wavelengths seems to be intuitively such a is developing an IETF draft for that that will St Arnaud: Absolutely. good idea. We were trying to understand if use CFR-LDP for the setup and signalling it’s just intuitively a good idea or if there is of customer owned wavelengths. This would COOK Report: What is going to have to something fundamental underneath it is that mean that Juniper and Cisco would have to happen to get answers to these scaling is- is driving it. And this is what we think the implement OBGP in their routers to make it sues? “n” squared issue is doing. happen. At the same time we have another team at Carleton University at Ottawa that St Arnaud: We will still have to do a lot COOK Report: There are some other issues is developing OBGP as an external protocol more research and investigation to see how in the scheduling of BGP itself that people to BGP. This protocol would work out of an real the phenomena is. Will it be a minor like Jeff Houston and Tony Bates have been external box with the result that it would aberration because its impact can be elimi- talking about. work with existing routers but also be inte- nated by other technological means? Or will grated at a future date with existing BGP. it signal a major change in the operation of St Arnaud: This is a totally separate issue the Internet? We have to gather some data from the “n” squared problem. In the con- COOK Report: That sounds extremely sig- and do more research to find out how sig- trolling the size of the BGP routing table is nificant. Say you are looking for a change nificant the impact is. We actually have some a significant challenge. We haven’t really to the BGP standard and you go into the stan- data that indicates that the problem may be looked at that and in fact what we are doing dards arena to get its blessing. Next you must even more significant than we originally it with OBGP and customer owned wave rely on router manufacturers to change their anticipated. The first task is to analyze this lengths could make that worse. But I am code. You never know the speed with which data. And the second task, if we find out that confident that people like Geoff Houston, it will take place. If you are lucky it could the hypothesis is correct, is to look at tech- Tony Bates, Tony li, and Yakov Rekhter will happen in three to six months. If you are not nology solutions that would minimize the arrive at a viable solution. lucky and politics raises its head, it could be impact. three to six years. Therefore, getting these COOK Report: It looks like the next six capabilities is important enough to the issue COOK Report: With whom do you think you months to a year is on the incredibly fasci- of scaling the Internet and being able to get may work to ascertain all this? Is this a task nating and like the entire complexion of the where you want to go that you are covering that would be appropriate for an organiza- landscape may change. all your bets. You are doing this by develop- tion like CAIDA? ing an independent effort at Carleton in or- St Arnaud: That is for sure. der to be absolutely certain that you are able St. Arnaud: Absolutely. I think it would be to push control of network administration a perfect research project that CAIDA could COOK Report: to sum up -- and peering into the hands of as many of the tackle. smaller players at the edge of the network “A proposed strategy to make as possible. COOK Report: Do you have any thought as to whether it will be three months, six Canada the most networked country St Arnaud: That’s right. months or year before you know what is in the world and the first to have low happening? cost Gigabit Internet infrastructure COOK Report: And please say a bit more available to virtually all schools, St Arnaud: I think will have a good indica- about what the Carleton effort looks like hospitals, libraries and businesses by tion soon. But, we want to do two things. from the point of view of the stand-alone 2005” device. What is this device - a switch? First we will want to do a thorough theoreti- cal and mathematical analysis that will give The source of the above is slide 2 of presenation St. Arnaud: Yes for the most part. JDS us a model. And then we will compare that model to some empirical data that we must on optical communities found at http:// Uniphase has been very generous and has www.canet3.net/library/presentations/ provided us with an optical switch and other gather. Hopefully within a few months we will have that. Now if both the theoretical OpticalCommunity-Sept2000.ppt We thank technology. We have some software that Canarie and Bill St Arnaud for permission to use controls the switch. To the router it looks model and empirical model confirm our postulation, we will need to say OK this se- all the illustrations in this issue all of which were like it is communicating to another router. obtained from the Canarie web site. As a result you get a virtual router that sig- rious; this is real, and what are the solutions? nals the existence of OBGP to other OBGP 62 A RevolutionThe COOK Reportin onthe Internet Cost Light, IP, and Gigof E - NewAnnual Report Fiber February 2001 Networks IMS Executive Explains His Role in the Creation of Fiber Brokering and Condominium Dark Fiber Networks Plummeting Cost Makes it Possible to Bring Fiber to Most End Users Editor's Note: Robert Proulx graduated in ginning of 1997, employed 20 people and Yes, Bell Canada bought all the extra ca- Electrical Engineering from Ecole had a no nonsense style of operation hat I pacity, thinking to kill the competitor. So Polytechnique de Montreal in 1977 and ob- found very appealing. here is IMS which was there to help man- tained a Master's Degree from McGill Uni- age and commerialize Hyro Quebec's fiber versity in 1981. He is VicePresident, Tele- I joined IMS with the mission of starting the finding itself shoved aside by Bell Canada. communications and associate of IMS Ex- telecommunications department. At that Looking back we might say that Bell Canada perts-Conseils. Proulx has been involved in time, Hydro Quebec, the electric utility for made a mistake when they did not buy me the studies and the realization of private fi- Quebec province, had a large telecommuni- along with the excess capacity of Hydro bre optic projects in more than 25 school cation network which they wanted to com- Quebec. At the time, I was expecting that divisions in the province of Quebec, as well mercialize. Since I had a lot of contacts and what I had done for Hydro Quebec could be as more than 10 municipalities, including the clients in the telecommunications industry I replicated over long-distance links, and I had city of Laval, the second largest city in the put together a good bid and Hydro-Quebec expected to use the fiber of Hydro-Quebec. province. We interviewed him in Montreal gave me the contract to commercialize the Suddenly I didn't have a source of fiber any- on November 28, 2000. network. more. COOK Report: Did you become involved COOK Report: And was part of the idea of Building Private Networks in telecommunications, when you joined what they were doing in building their tele- IMS? communications network the use of fiber optic ground wire that they would lay along Now when I was at Hydro Quebec doing the transmission towers? commercialization of the network, IMS had Proulx: No, I started much earlier than that. many contacts with people who wanted to I was, up until 1996, working for a large have a private network. So while they told engineering firm in Montreal taking care of Proulx: Exactly. They did not build it to commercialize it, but they soon realized that me that it was too bad that my biggest cus- the telecommunications department. We tomer left, they reminded me that I still had were doing the engineering mainly for telcos it had so much capacity that they could com- mercialize it. They tried to do so for a few a lot of contacts who wanted their own so- and I ran the telecommunications department lution. at that time. years, but their approach was one of a big company with a big telecommunications The timing of these events in 1997 turned Eventually I received a call from headhunt- department. They did not know how to an- swer RFPs at the same speed as a small com- out to be critical. The CRTC was ers who they asked me if I was interested in deregulating the territories of the telecom- a position in a cable TV company. They of- pany and consequently they were missing a lot of opportunities. But, with the help of munication carriers. Previously a carrier was fered me the position of Vice President of restricted to operation in a geographical ter- engineering for Cogeco Cable for the prov- IMS, Hydro-Quebec was able to commer- cialize its network. The commercialization ritory. Suddenly, in 1997, with a license, you ince of Quebec. Cogeco Cable is the third could attach your own fiber or other infra- largest multi-service operator in Canada. went well. It went so well, that Bell Canada eventually realized that Hydro Quebec could structure to poles anywhere in Canada. This had never been tried before. To assume my new position I moved from become a big competitor. They offered to Montreal to Trois Rivieres. I was specifi- buy all of the excess capacity belonging to Hydro-Quebec. So the first project that we did was for the cally in charge of deploying the Internet on members of RISQ (Quebec's universities and Cogeco infrastructure in Quebec. Cogeco COOK Report: What did you define as ex- research institutes). It was a small project, was the first to deploy cable modems in a of ten kilometers, using poles. We took it on large-scale manner in the province. I had tra capacity at the time? How many fibers did Hydro Quebec have? in partnership with a small cable distributor worked all of my life in consulting as an in the area of Sherbrooke. The small cable associate in an engineering firm. But now I network only had 600 customers, but most was working for a company owned by a fam- Proulx: They had many fibers in their cable but were using only a few strands. importantly, it had a broadcast distribution ily. This was not a good fit. So after eight undertaking license. In other words, it quali- months, we decided on good terms how I fied as a carrier for access to the transmis- could leave the company. However, I liked COOK Report: Do you mean that Bell came in and bought the remaining strands? sion poles at a set and reasonable price. Con- the place where I lived (Trois Rivieres) and sequently, we used its license beyond its I wanted to stay there. Proulx: No. Not quite. They bought the ca- operating territory to attach on an additional pacity. Hydro-Quebec could use whatever ten kilometers to his network. We sent in I wanted to start my own engineering firm the request and it worked. in telecommunications, but since I am a tech- they wanted of the total capacity. They can nical person who likes to do field work and use as much as they want for their own needs, but everything that is not used by Hydro- COOK Report: This was a request for au- not just administration, I decided instead to thorization to use the fiber infrastructure? join another engineering firm called IMS. I Quebec, becomes Bell's to commercialize and to use. Hydro-Quebec may not resell it. don't like to administer and shuffle papers Proulx: Not the fiber, just the poles. The goal and have a secretary. IMS, which at the be- So that's the way they killed the competi- tion. is to gain access to the public infrastructure 63 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA of conduits and poles used by the telecom- for its maintenance. The director looked at Proulx: Yes. But having that, we stopped munications carriers. The CRTC has decided me and asked where is the rest? I responded negotiating with Bell. We said to Bell: If you that other carriers must be able to get access by saying that there is no rest. I told him that don't want to do it, we'll be doing it with to the same infrastructure at specified costs. we were prepared to give his company 6 fi- Quebectel. And we had the negotiation and bers everywhere, in the north of Montreal, Quebectel won the contract. We built this small project, and it worked. to places with considerable population den- After that, a school board called us and said, sity, and that we were prepared to do this in COOK Report: So you used this occasion we have a project to link all the schools by order to be able to use his license. We would to build up an infrastructure base so that you dark fiber. Can you help us? So, again, we build the network, put in a cable as large as could be considered a real company? used the same strategy. We used the license we wanted, and keep 6 fibers for the sole of a small cable distributor and we built a use of his company. I could approach this Proulx: We are a consulting engineering network of 60 kilometers with four partners. negotiation with confidence because by now firm. We are project builders. We have done At that time, Bell Canada simply regarded I had developed first hand knowledge of the this for a school board for a distance of 200 my efforts as not being serious and thought cost of building networks, and I knew that I kilometers. We were under contract from that what I was doing would not remain vi- could have a return on investment of less two merging schools boards-- Manoirs and able over time, than two years. However, I also knew that le Gardeur. We gained them a return on in- without a non dominant carrier license to vestment of close to two years. We have done We completed this project. But we had two gain access to the poles and conduits, I could that. But after that, we did it for Laval, a problems with this approach. The first prob- do nothing. major city of the province. And not just city lem is that the small cable distributor was of the province, but we have done the school very, very small, 600 customers. As a result COOK Report: A company with such a li- board and the municipalities. We are start- we didn't have the ability to work on a big cense can transfer its use to another com- ing to work with a lot of partners, building project. We were faced with giving a few pany? networks. million dollar project to a cable distributor that you don't know will be in business two Proulx: The license is not transferable. The Now one of the biggest projects we've done years from now. You run the risk of a change ownership of the cable stays with the tele- was for the RISQ, the Reseau in ownership of the cable on which your communications carrier. What ends-up be- Interordinateurs Scientifiques Quebecois, an project depends giving you a fresh problem. ing sold is an Indefeasible Right of Use organization owned by all the universities (IRU) to certain strands on the cable. This is in the province of Quebec. It was an OC-48 The other problem we have is that since this what the school board ends-up purchasing. between Montreal and Quebec City and an cable distributor doesn't have a lot of money, If and when the telecommunications carrier OC-48 between Sherbrooke and Trois- it absolutely wants to be the builder and the is comfortable with the idea of selling IRUs Rivieres; Montreal and Ste-Anne de Belevue contractor and the operator of the network. to its fiber, we know that we have a partner and Drummondville to Victoiaville. But in the public sector giving a contract for the buildout of the network. without going out for public bid can cause a COOK Report: I have heard about that from problem. So we had two problems. The costs COOK Report: But you gave them six fi- Bill St-Arnaud of Canarie, that there were were too high and the small cable distribu- bers, when there was many more than six something like 2,500 kilometers of fiber tor was taking too much for itself. But, at fibers that were being laid? connecting twenty or thirty educational in- the time it was the only way to do it. stitutions in Quebec? Proulx: Yes. What you must understand is The Importance of the that in building out fiber optic cable, the Proulx: Correct and in the last year, we have marginal cost to add some strands in the been involved in close to 5,000 kilometers Non Dominant License cable is very, very low. For an increase in of fibers, building or swapping or exchang- only 10% of the cost of the project, you can ing. We are involved with 40 school boards We were looking for another solution. We double the number of fiber strands laid. So in the province of Quebec and we are in- called Quebectel, which had recently been giving away six fibers out of 96 cost us vir- volved in 12 municipalities. We have bought by Telus, the incumbent telecommu- tually nothing. In particular it was a lot projects in six provinces in Canada. nications company, like Bell in the West of cheaper than using a contractor that would Canada. Quebectel was the incumbent local be likely to charge us its cost plus 30%. Finding Partners for carrier in the eastern parts of the province of Quebec that lay outside of the Bell Canada COOK Report: Understood, but this is only Condominium Fiber operating territory. We told Quebectel that the beginning of the project. In other words, we wanted to use its license to build a net- when you're planning should we put down COOK Report: In http://www.canarie.ca/ work on the operating territory of Bell 24 fibers or 48 or 96, at that point, once you advnet/canet3/fibre.html Andrew Bjerring Canada using the new CRTC decision en- put the fibers through a conduit, if you're and Bill St Arnaud write abling the deregulation of territories. We then going to add another 24, then it's a bit were surprised by their interest. more expensive, isn't it? "In turn, lower prices for fibre is leading to a shift away from carrier-owned infrastruc- We met with one of the directors in the com- Proulx: Yes in that case it's slightly more ture and towards more customer- or munici- pany and outlined our plan to build a net- expensive. Using figures for the initial build pally-owned fibre, as well as to innovative work of 200 kilometers in Laval, just north of 96 fibers, of which we give the telco six, sharing arrangements such as fibre "condo- of Montreal. The network would go to all the cost of the six to us is about 5% of the miniums". the schools and the places in the city. We total project. also said that we would give Quebectel six "Condominium" fibre is "un-lit", or "dark" fibers throughout the network. On top of this, COOK Report: You wanted an IRU from fibre that is installed by a private contractor that we offered to build the network and pay Bell on fiber that it already owned? on behalf of a consortium of customers, with 64 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 customers owning the individual strands of capacity planning. We have a team to do this. COOK Report: So when something got fibre. Each customer/owner "lights" their done, it would be Bell Canada that would fibres using their own technology, thereby COOK Report: So is it a part of your busi- just go out and do everything itself? deploying a private network to wherever the ness model that when you might be answer- fibre reaches, perhaps including carrier COs ing an RFP to build the network linking the Proulx: Bell Canada and Videotron and for and ISPs. The business arrangement is com- research institutes and universities, one way that matter, all other incumbent telecommu- parable to a condominium apartment build- you know you can come in most competi- nication companies. And they are doing it ing, where common expenses such as man- tively is to make a bid that will include other for an enormous profit because building agement and maintenance fees are the joint partners? public dark fiber services is still a business responsibility of all the owners of the indi- model that is not adequately understood. vidual fibres." The IMS Business Model How to cost a fiber optic network is still something unknown. However, knowing And the business model outlined by St Proulx: I would explain it a little bit differ- what the cost to build it is, I can also under- Arnaud and Bjerring (and used by your own ently. When you build a building, what do stand how much it costs to be stuck with an company) is one of 'net headed' horizontal you do? You hire an architect and an engi- established business model and the impli- thinking. It is a concept totally alien to the neer. You do some plans, drawings. You pre- cations of being a telco. Furthermore, I have bell headed view of telecommunications as pare a request for proposal and you go out the additional advantage of knowing that all vertically integrated monopoly. IMS is pri- to find a contractor to build it. of this can be built using a non-dominant marily a group of engineers who have expe- carrier license. rience dealing with the people you have to I work exactly the same way. I'm the engi- deal with in order to get right of way per- neer hired by the organization to build a net- Use of the Non Dominant mission to put on poles. You know how to work. Next I find a partner who wants to be order the fiber, you know where to go to an investor in the project. The partner be- License employ the people who actually tie the fiber comes an investor in the project and I say to the poles. we have a project that will be owned by five Consequently we have gone out and gotten or six companies and we prepare the docu- our own telco license. Now I am IMS, a non- Proulx: But more than that. We find part- ment and we go out with an RFP to build. dominant carrier. But I'm not using the li- ners. This is the major point of what we're And the same contractors that are being used cense. The only reason why I have my li- doing. We're not an engineering firm doing by the Bell will build my network. It's an cense now is help to negotiate. So I go to a project for someone, who says to us I want RFP to build. Not to provide services, not to see a telco and say do you want to be part- you to build fiber. No. What we do is we provide dark fiber. It's to build network that ner with me? And now they know that if find partners. Knowing that by having part- will be owned by a consortium. But it is a they say no I can build it alone since I too ners, we can share the costs. We enable consortium that we also build. am a carrier. It usually becomes more cost people to pay as little as possible to go from effective for the telco to partner than to go it point "a" to point "b". We are a development engine. We are build- alone. ing a network. Our RFP is to build. Our con- COOK Report: This is your business model, tractor will be a supervised contractor. It will COOK Report: And the license is a kind of in other words. be paid by the consortium fixed fees or vari- a certification of your technical capabilities? able fees - depending on the situation. Some- Proulx: Yes. I call the city and say, I'm build- times, the customer prefers to have a turn- Proulx: No, but it's saying that I'm the car- ing fiber, do you want to be a partner in this key project. When it's a turnkey project, we rier, I can offer the service of a carrier in project? I will call all the carriers that I know are doing the same thing, but we are having Canada. I have satisfied the condition to have of. Hey, we're building something, do you a team of partners, we hire the contractor my license. I have my license, but I'm not want to be partner of this network? Do you and we propose a price for the turnkey using it. I'm using it to negotiate. I go to see want to link your network there? I ask Telus project. But IMS is mainly an engineering a carrier and I say I'm building a project here. and Group Telecom and all those incumbents company. If the customer wants us to do it who want to deploy their networks in new for them, then we're working for them. If You remember at the beginning I told you I areas and I propose them to deploy their they want to do a turnkey project, it is the have to give six fibers? I don't have to do network at a fraction of what would be the same model. We put the consortium together that anymore, because now I have my car- cost if they would do it alone. So I'm find- and we select a contractor who wants to build rier license, I go to them and say, do you ing partners to build. For example if you put it with us and we establish a price and we want to be a partner? If he says no, I'm build- four partners together, it costs 25% for each bid on the project. ing it anyway. So now he has to have an of the partners and you have as much fiber interest, knowing that if he partners with me, as you want. But our model has been up to now more a it'll be half of the cost for his investment. So model of consortium and engineer project he has a choice, either he works with me or COOK Report: But do you have a certain builder working on professional fee paid by he pays full price for his own investment. percentage above and beyond the 25% left the organization. We have a project on the over that you can deal with? I mean, how do south shore of Montreal, which requires a You can imagine that Bell, who already has you decide what 25% of the fibers are? How network of 60 kilometers of fiber, that will structure there, panicked when they realized do you decide whether their 25% is 12 fi- pay for itself in 6 months. this. Because not only are they losing the bers and 24 fibers or whatever? customer for who I'm building the fiber, but COOK Report: I understand. Is there any- I'm bringing competition into their territory. Proulx: We do an analysis of that. We have one else in Canada who has tried to adopt So it's very hard for them and I agree, but it a white paper on the planning, and we have the same business model? is also inevitable. engineering department doing all phases of the planning from design to engineering, to Proulx: I don't know of anyone else. COOK Report: You say to Bell Canada: 65 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA "Deal with it." Proulx: That's it. Behind the proliferation Proulx: This is what they're saying. For the of networks that makes this possible is IMS. school board, it's very interesting. But for a Proulx: Yes. But on the other hand, I also So this is the model we're building and we telco, it's a nightmare. It's a real nightmare say to Bell, you are my greatest helper in have great success. I'll tell you something. and they don't know how to handle that. This this project. The price you charge for tele- I've never understood why we are alone do- is the future. There's no choice now. Nobody communications services is so high that they ing that. Now we are doing that in six prov- will go back. The future will be for the cit- help me to have a return on investment that inces. Maybe because we have a good ies. is quite rapid By doing so you ensure that I knowledge of the telecommunications in- will have many customers. dustry. Maybe because my reputation is COOK Report: Well, it will be customer- there, now they know that I can do it. It's not owned dark fiber and the telecommunica- COOK Report: And they ensure that you an easy task. It is really a fight. There's so tion companies will shrink down to the few have a lot customers. In the meantime what much money involved. In a school board people they employ with knowledge of how about the schools? where they build their own private network, to put equipment on the end of the fiber and we estimate the loss of business from Bell start sending bits. Proulx: The morning newspaper is talking at about $1 million from a school board. Proulx: Let me tell you my vision of the about this. The schools now are saying hat There are seventy school boards in the prov- future. Let's start with the past. Imagine that since they have their own fiber, Bell Canada ince. They can assume that they will lose ten years ago, the city of Montreal --did not can no longer offer them cost effective voice $70 million in revenues. And this is just for take care of installing sewer and water pipe services. the province of Quebec. And this is just the and that this was left for the private sector. school boards. Because we are working for Now imagine that as a building owner you The Economics and cities, we can expect that a similar loss of have to choose between a selection of ten revenues will come from the twelve Munici- and twenty people providing the pipe. Public Policy of pal governments that we are working with. Customer Owned Dark You decide who you take and this company Fiber COOK Report: But the school board can say builds the pipe for you. Or you can change to the taxpayers we're saving you money and the company. If you have the pipe, you say, COOK Report: They're getting rid of Bell that makes them happy. I don't have good service with you, the pres- Canada, in other words. sure is not good enough, you select another

Figure Two: The RISQ Quebec Research Network brokered by IMS at a tiny fraction of the single customer price. White boxes identify fiber exchanged with existing networks or construction with new partners.

66 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 provider of pipe and they build a new pipe Building City Owned on the one hand, but you also have the cable beside your home. It would be a nightmare companies, who are entrenched. Are the for the city, it's impossible to handle that. Infrastructure? cable companies as threatened by this as Bell But telecommunications is exactly what we Canada? have doing precisely this. IMS is sitting down with a lot of cities. We had an interesting meeting this morning with There's a lot of companies who put struc- an organization for cities, which covers all Organizing Horizontally tures in the city, both underground in con- Canada. We are starting a committee to pro- to Push Ownership to duits and on the poles, and the city has noth- mote this vision of the future and the reason ing to say about that. It was a monopoly be- why we're doing it for the school board pres- the Edge of the Network fore. Now, it's no longer a monopoly. You ently. It's only a question of timing. But what keep having streets being dug up everywhere we will see is that it's really community. Why Proulx: I think so. But for me broadband is by new telecommunication companies who the school and not the city, the library, the not it DSL nor is it a cable modem. It is gi- want to put some pipes. And if the city says, hospitals and so on? gabit Ethernet. But gigabit Ethernet is only no, you don't have the right, you say, Oh, it's today. We are working now with bits which unfair. I don't have the same treatment as Services should be provided by community. is like Morse code. Think of what can be Bell who is already there. It is impossible to It's like roads. When you build roads, the done with modulation of light. It is the be- survive like that. Here is my vision of the city is not building roads or highways. They ginning of an era. future: Telecommunication infrastructure are giving contracts for roads, but you only will be city-owned. When you'll build tele- have one administration in charge of build- COOK Report: Everyone is certainly push- communication, everybody is agreed it will ing roads. If you want to build a road in the ing the limits. Howard Frazier the distin- be fiber. The city will have fiber to all the city, you don't decide yourself, it's the city guished Cisco engineer whom I interviewed homes, to all the enterprises, going to a cen- that will decide. And if you are not happy, on in gigabit Ethernet last summer has only tral location and you will decide, as a build- don't re-elect them. Same thing for telecom- a few months later left Cisco and founded ing owner, do you want your telephony with munications. his own start up reportedly with the goal of Telus, do you want an Internet provider, who doing gigabit Ethernet over copper. will it be? You decide on this provider, we've COOK Report: What about the technology got one location and will provide service for that I heard about recently where robots take Proulx: They will try to push that on to cop- everybody from there. And the city will fiber strands through the sewer? Are you per because they have not yet amortized their handle that. looking at that? copper based local loop. In the meantime fiber is easy to install. We just commissioned If you put two pipes in it's expensive. But if Proulx: Yes. But the real solution is not that. a 93 km 100 megabit network with a cost of you put a new one and you share the cost It's just like trying to put the water pipe in $2,000 per connection. It is so easy to build with different telcos and the ISP and the the sewer pipe. You can do that, also. But networks with fiber that there is no reason Cable companies there, it will reduce the cost the future is to have dedicated structures for to do anything else. for the telcos and reduce the cost for the user telecommunications. You can have many, COOK Report: When you bring fiber into many ways of doing things. and for the city. It will remove a lot of night- home or apartment, how you do the demarc? mare of always opening the streets. Each city or each organization will decide Proulx: There is no clear answer to that. We And this company doesn't have to be man- how they want to do that. We have recently are in a new era of with a pilot project going aged by the city. A lot of people fought about witnessed a pilot project in Brossard on the on. We face a technical problem. I'm an en- it. We don't want the government or the city south shore of Montreal, where fiber to the gineer. An engineer always finds solutions to be involved in those kinds of things. It home, is being installed by a subsidiary of for technical problems. Sometimes the ad- could be an independent company manag- Hydro Quebec. Hydro-Quebec will bring ministrators don't like our solutions. But we ing the network for the city and why not the fiber to each of the home and bring fiber to are talking about technical solutions which telcos being the owner of this company, by a central location and the owner of the home do exist. law, saying that each telco has to buy shares will decide who will provide television, te- of this company. And if you want to pro- lephony, Internet and video-on-demand. The When I started looking at Cogeco, we were pose services to be delivered by the com- cost of building a home including that will paying a $1,000 per cable modem. Currently pany, you should buy a share of the distri- increase a little bit, because they have to pay the price is a lot cheaper than that. It is this bution. It's a vision. There's many ways of for the conduits and the fiber to go to the expensive only because we're not selling in looking at that. But it is this way that I think central location. sufficient bulk. Within two years there will it will be. I don't know what the structure of be many choices and within five years it will it will be but the current ways we have pres- COOK Report: And they have how much be a commodity. What we're doing now is ently in the cities, with all telcos being able bandwidth? building the infrastructure to support this to put what they want and when they want commodities based future. In the meantime just doesn't work. Proulx: They can have ten megabit band- the telcos are losing their monopoly. Con- width. The future is really there. With fiber sider me to be a plumber. I'm just building Again, this will cause a big problem for the to the home, the amount of bandwidth is vir- pipes. I've told people build the pipe and then existing telcos, which presently, because tually unlimited. When the subsidiary of decide what you will put in it. If you sepa- they are already there, have an advantage Hydro-Quebec makes an announcement of rate the three components of the network it over the others of time and cost. But this the technology being used, it will be pos- becomes easy. The first is the pipe. The sec- cannot last. And we should look at the infra- sible to give out more details. ond is the equipment and by means of which structure of telecommunications just like you will light the network. And in the third municipal infrastructure. This is my vision. COOK Report: What is the situation with is the management of the network. And these the cable companies? You have Bell Canada three things should be separate.

67 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA Four Year IMS Track Record in Dark Fiber Brokering Starting in 1997, IMS Experts-Conseils implemented several pri- City of Mirabel 60 km of fibre optics 10 buildings vate fiber networks in the province of Quebec and is now involved City of Québec 100 km of fibre optics 60 buildings in major projects in Ontario, Alberta, New Brunswick, Prince-Ed- City of Repentigny 10 km of fibre optics 4 buildings ward Islands and Nova Scotia. IMS worked jointly with Hydro- City of St-Eustache 6 km of fibre optics 8 buildings Quebec from 1997 to 1999 in the commercialization of its telecom- City of Terrebone 10 km of fibre optics 5 buildings munication network. More than $10M was spent on telecommuni- Ville St-Laurent 12 km of fibre optics 9 buildings cation projects carried out during this period. IMS’s primary project Ville de Fredericton (N. B.) 35 km of fibre optics 50 buildings was the deployment of a SONET OC-48 liaison between Montreal Ville de Sherbrooke 35 km of fibre optics 5 buildings and Quebec (250 km) using existing fibers owned by Hydro-Que- bec, as well as fibers owned by various carriers (exchange of fi- School Boards bers) and the construction of missing parts. Several other projects were also carried out during these two years with Hydro-Quebec Capitale 140 km of fibre optics 80 schools that used SONET, ATM, Frame Relay or ISDN. IMS has been also Region.de-Sherbrooke 180 km of fibre optics 66 schools very active with School boards, with the “Réseau Interordinateur Riviare-du-Nord 175 km of fibre optics 52 schools Scientifique Québecois”, with municipalities and with other orga- Sefimourie-des-Mille-lks 200 km of fibre optics 80 schools 4 nizations described below: partners Amiante 12 km of fibre optics 9 schools City of Ottawa Laval 170 km of fibre optics 111 schools 3 partners IMS carried out a preliminary study for the construction of a fibre Saint-Hyacinthe 250 km of fibre optics 51 schools optic network linking most of the research centres and universities Affluents 170 km of fibre optics 70 schools 4 in the City of Ottawa, for Canarie. The project was estimated at partners $700 000 and the construction or the acquisition of the network is Bois-Francs 60 km of fibre optics 12 schools 4 part planned for the year 2000. It is important to note that the main ad- ners vantage of carrying out a preliminary study according to the method Draveurs 90 km of fibre optics 40 schools proposed by IMS is to be able to know the real cost of a customer- Gmndes-Seigneuries 210 km of fibre optics 58schools based network: it is then possible to solicit competitive offers and Vailee des T’issarands 30 km of fibre optics 56 schools obtain a private fiber optic network at a competitive cost. Hautes-Rivieres 250 km of fibre optics 54 schools Laurentides 200 km of fibre optics 35 schools Alberta Minister of Science and Technologies Patriotes 2 km of fibre optics 3 schools Premieres-Seigneurie 190 km of fibre optics 73 schools Samarcs 460 km of fibre optics 72 schools IMS was also mandated by the Alberta Minister of Science and Trois-Lacs 45 km of fibre optics 15 schools Technologies to carry out a preliminary study for a pilot project that Chemin-du-Rov 29 km of fibre optics 11 sites aimed at connecting various public organizations in the province. Coeur des Vallees 53 km of fibre optics 19 sites Following this project, the Government was able to prepare tender Marie-Victorin 6 km of fibre optics 5 schools documents to equip all the public organizations with a private fiber Sir-Wilfrid-Laurier 92 km of fibre optics 20 schools optic network. Total of 3014 km of fiber for these school districts which represent Ontario Ministry Energy, Science and 75% of all school districts in Quebec Technology Other IMS Work Together with GSI international, IMS was involved in the ORION project for the Ministry of Energy, Science and Technology in the IMS did the study for the province wide fiber network in Alberta Province of Ontario. This project has the objective to link all uni- and another Alberta project with Netera University Research net- versities and college in the province by a dark fiber network. work IMS did preliminary study for New Brunswick for 1700 kilo- meter university network Prince Edward Island Community net- Cities and Municipalities of Québec work engineering study phase 1 University of Dalhousie fiber con- nection to University of Acadia in Nova Scotia (ongoing) Following the success in the fiber optic projects for the school boards, many municipalities mandated IMS to accompany them in the ex- Other companies involved in customer owned ecution of fiber optic projects to connect their buildings. The list of municipalities that mandated IMS and the scope of the mandate are dark fiber construction and brokering given in the following table. Universe to You (covers Southern Ontario) Municipality Scope Holland Construction (joint work in Ontario with IMS and GSI International) City of Brossard Pilot project: Fiber to the home City of Lachenaie 14 km of fibre optics 3 buildings Dixon Fiber INC (joint work with IMS in 2 Quebec projects, inves- City of Laval 90 km of fibre optics 45 buildings tigation of 3 additional in Quebec and 2 in Maritimes provinces)

68 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 Clash of Broadband Private vs Public Sector Business Models Will Impact Canadian Economy Francois Menard Describes Complexity of Canadian Regulatory, Carrier, and Content Struggles -- Suggests That Viable Outcome Likely Only With Community Networks Infrastructure Editor’s Note: Francois Menard last ap- ing services, technical support customer ser- have open access. peared in the pages of the COOK Report in vices, multicasting capabilities, address our October issue. He has worked for some management and other technical functions COOK Report: And you’re saying the FCC of the contending parties and through nu- of the cable system that affect customers’ will likely have to, as a matter of uniformity merous filings with the CRTC has made sig- experience with their ISP, AOL Time Warner in the U.S., impose this, because if they nificant contribution to the development of must provide them in a manner that does don’t, everybody everywhere will wind up regulatory policy. As he puts it, our conver- not discriminate in favor of AOL Time in court fights over it, as happened in Port- sation is designed to demonstrate the cur- Warner’s affiliated ISPs on the basis of af- land? rent and emerging business models that new filiation” open-access regimes and customer-owned Menard: Exactly and the recommendation networks are trying to compete against. In This specific condition is even more draco- from the judge in the Ninth Circuit Court of late December 2000 he joined IMS as Project nian than any measure taken to date by Ca- Appeal was that FCC leadership was re- Manager, Telecommunications. IMS Inc, is nadian regulatory authorities, which have quired. the Canadian dark fiber broker mentioned specifically excluded IP voice telephony on elsewhere in this issue. We interviewed cable networks from the decision Order COOK Report: So the Notice of Inquiry then Francois in Montreal on November 29, 2000 2000-789: came from in part from the Ninth Circuit and updated that interview in mid January Court calling for FCC leadership? 2001. “Decision 99-8 require a cable carrier to make higher speed access service available Menard: Correct. As a result of the NOI, COOK Report: What are we looking at in only to permit companies other than the the FCC will presumably come to the same the next six months to a year in terms of a cable carrier to offer higher speed retail conclusion as the CRTC in January 1996, possible transition in the use of network in- Internet service. For this purpose, higher that cable carriers, when offering non-pro- frastructure and cost allocation and payment speed retail Internet service does not include gramming services, in other words some- for access and content and so on? Internet protocol (IP)-based voice telephony thing other than TV, are subject to telecom service, multi-casting, virtual private net- law, that they have to file tariffs and that they Menard: I think that the United States may works or local area networks.” have to provide non-discriminatory intercon- catch up to Canada on Cable regulatory is- nection with third parties. sues. Cable TV providers in Canada have To this end, I believe that the FCC may have been viewed as telecommunications com- prevented the cable carriers in the USA from Technical Issues in Cable mon carriers with respect to third party ac- having a “carte blanche” to provide IP tele- cess to their infrastructure since January phony, multicasting and other services with- TV ISP Interconnection 1996. This has yet to happen in the United out giving any regard to non-discriminatory States, but I am optimistic that it will hap- interconnections. This decision from the If you want to have access to the cable net- pen as a result of the outcome of the Notice FCC will certainly make my life easier in work, you can either rent 6 megahertz chan- of Inquiry for which comments are due on Canada. However, those draconian license nels or have interconnection at a higher layer December 1, 2000. The FCC logically transfer conditions by the FCC would need than the physical TV channel. What’s been should come to the same conclusion that the to show up in the decision resulting from advocated in Canada is that cable carriers CRTC has come to, that cable carriers in the Docket Number 00-185, Inquiry Concern- can’t provide physical interconnections. States have dominant market power posi- ing High-Speed Access to the Internet Over They apparently cannot give spectrum fre- tion in terms of access to high speed internet Cable and Other Facilities, to really become quencies to other people. Thus they have to services. The cable network infrastructure meaningful to the whole of the industry. provide interconnection at a higher layer. But must be subject to telecom regulation and [End of January 17 addition.] this practice has not been uniform. There is that national leadership is necessary. the precedent of Cogeco having allowed Without national leadership from the FCC Queens University to share the coaxial spec- [Editor: Added by Menard on January we will become enmeshed with all the fran- trum to allow its own students to have ac- 17]For a long time I felt that the regulators chise transfer conditions that the local ad- cess to the university campus via a Cable in the USA were only playing a catch-up ministrations are trying to impose. Modem Termination System (CMTS) oper- game to what’s been happening in Canada, ated on behalf of the University by Cogeco. however, on January 11th 2001, the FCC has COOK Report: Like AT&T, with its TCI imposed the following condition as part of franchises? COOK Report: And the definition of a higher the approval of the AOL - Time Warner layer is what? merger: Menard: Like TCI, in Portland, Oregon, when they tried to get their license trans- Menard: A router on top of the cable infra- “To the extent AOL Time Warner provides ferred to ATT only to be told by the local structure. It may also be an Multi Protocol any Quality of Service mechanisms, cach- authorities that, as a condition, you need to Label Switching (MPLS) router. 69 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA COOK Report: At a higher layer in the pro- potential discrimination. Since CMTSs are Menard: Yes indeed. And it’s only going to tocol stack? incapable of ensuring that there will not be get worse with MPLS. An ISP has no way congestion on a given segment of the net- to trust that it gets as much bandwidth to the Menard: Correct. This is at Layer 3 in the work, they have to be treated as the last CMTS as does the incumbent cable carrier. OSI reference model. In the telephone world, bottleneck. Unfortunately, the various cable the interconnection is said to be performed carriers will not tolerate the idea that inter- COOK Report: In other words, Videotron’s at layer 1, that is the physical layer. In other connection be realized at the CMTS. (Note going to tell its customers, you get what? X words you can gain access to the copper that I say carriers rather than companies be- number kilobits upstream and Y down- going straight to the home. cause under Canadian law they are common stream? What do they advertise? carriers.) COOK Report: So interconnections for third Menard: They advertise nothing. They ad- parties on cable systems are done quite dif- Cable Modem Termination System (CMTS) vertise fast best effort service. They give ferently than on the PSTN? continue to practice traffic aggregation poli- absolutely no minimum through put either cies that are not guaranteed to be non-block- for download speeds or upload speeds. In Menard: In the cable world, there’s no ing, that is, that there is not always enough practice, the upload speeds are often system- physical, electrical connectivity between the bandwidth capacity available over the met- atically capped at 14 Kbytes per second and cable and some third party accessing the ropolitan area network backbone to carry all the download speeds are not guaranteed. As cable. The cable modems on the customer the traffic that CMTS are capable of putting part of the high-speed working group meet- premise are directly connected via a hybrid- out. When you consider that in many cases, ings of the CRTC, the cable carriers were fiber coaxial infrastructure to a Cable Mo- the metropolitan network aggregation is recently asked why rate limiting was not part dem Termination System (CMTS) typically done over an IP packet over OC-12 SONET of the tariffs that they had filed. They re- located at the head-end. The CMTS is the ring architecture, which doesn’t have that sponded that rate limitation was part of their Cable world’s equivalent of the access con- much capacity, you can understand why we ability to manage the network properly. centrators used for providing DSL services cannot be sure that the transport across the by the telcos. The CMTS then converts the metropolitan area network backbone may This was uncovered when Videotron re- Data Over Cable Service Interface Specifi- not be performed exactly the same way for vealed that they needed to second-level test cation, (DOCSIS), signal typically to everybody modems having passed the DOCSIS certi- Ethernet. And, from that, it gets converted fication of Cablelabs so as to ensure that they to TCP/IP in a router. COOK Report: So throughout Montreal, the respected the rate limitation policies set in cable companies have built a layer two high- the modems. Now ISPs providing third party The interconnection with ISP’s is then at the way on which TCP/IP and other things can access to cable modems of cable carriers can TCP/IP layer. As opposed to DOCSIS. Be- run. see their customers having their rate limita- cause DOCSIS is only specified as a local tions changed in their modems at any time, area network technology between the cable Menard: Correct. But shall we, as ISP’s trust with any value, for as long as the same is modems and the CMTS technology, you the cable carrier to aggregate traffic as done for everybody. This will certainly be cannot have interconnections at the DOCSIS equally for itself as it does aggregate traffic part of a formal application before the CRTC layer between an ISP and a cable carrier. for us competing ISP’s? to require cable carriers to add this value to Unfortunately, even if the most recent CMTS their rates. could allow for interconnection directly at COOK Report: Give me a definition of what the CMTS, the cable carriers will not allow it would mean to do it equally, versus what COOK Report: But the fear is that what they it to happen. it would mean to do it non-equally. deliver will be better for their customers and perhaps significantly so than what they de- COOK Report: Why? Menard: The case which scares me the most liver to an ISP. is that the cable carrier would be able to di- Menard: Cable carriers do not want this to vert certain traffic out of his metropolitan Menard: There are strong reasons to believe happen because they would loose control of area network before arriving to the point of that ISP’s customers would get very differ- their ability to eventually discriminate right interconnection and that after this diversion ent service than Videotron’s own custom- at the very first hop. point, the carrier would simply not care di- ers. mensioning its backbone adequately any- COOK Report: Would you like to see ISP’s more. This means that there could be a lot COOK Report: The “strong reason to be- being able to connect to that level? of packet loss beyond a point where the cable lieve’ is primarily because there is no neu- carrier doesn’t care about the quality of ser- tral party with access to the technology to Menard: If we follow the CLEC competi- vice. Therefore we must come back to the verify that everybody’s playing honestly? tion model, interconnection is always done fundamental rule that interconnection should at the last bottleneck before the Customer always be done at the last bottleneck. Menard: Exactly. For a phone company Premise Equipment (CPE). In this case, the third party interconnection, it’s simple. You last bottleneck is the CMTS on the other side COOK Report: Can they tune this layer 2 know you have access to a pair of copper of the cable modem. However, were the CPE network in such a way that if there’s one that’s dedicated to you. It’s a dedicated pair engaged in discriminatory rate limiting, then megabit of bandwidth, I get 6/10’s of it and of copper wire. But in the cable world, an interconnection would logically have to be you get only 4/10’s? ISP, there’s a whole shared infrastructure to the cable modem, which would require behind the cable company’s point of inter- competitors to build of a parallel coaxial net- Menard: Absolutely. connection. work infrastructure to reach the cable mo- dems. Since this cannot happen, then there COOK Report: Allegedly, Cisco was mak- COOK Report: Shared in the sense of sta- is a reasonably good cause for requiring that ing routers with that capability, weren’t they? tistically multiplexed. the cable modem not be a tool engaged in 70 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 Menard: Yes. More importantly than that, Menard: Correct. wouldn’t have a clue. not only is it shared, but there are QOS mechanisms available on that infrastructure COOK Report: So the MAC addresses of Menard: Interconnection with the PSTN that permit the common carrier to do voice- the customers in this room are going to be will look like heaven when people will start over-IP. But yet in the third party access rates randomized, but can they also have a tag or to figure out the ramifications of third party that Videotron is filing, they’re explicitly something that will show to me as the owner access to cable modem infrastructure. preventing ISP’s from doing voice-over-IP of the infrastructure, which are my custom- on their infrastructure. ers? In theory, I can set things to favor them? COOK Report: But one of the reasons it’s gone on this long is that if you took an aver- COOK Report: And the problem is that only Menard: All the way to the CMTS, you age cross-section here of people in Quebec the owner of the cable infrastructure has ac- can’t do anything. That’s why interconnect- and tried to explain what it was that was cess to the routers and can control them. ing right at the CMTS is important. For fa- happening, it’s too difficult for the average voritism to be exercised, it can only begin person to understand. Menard: The layer 2 knobs are at the sole to happen in the CMTS. But even having control of the incumbent cable carriers. access straight to the CMTS does not en- sure an ISP that the cable carrier is not con- Issues Involved in Pricing COOK Report: And I used the word ‘router’. figuring the CMTS to its own advantage. It Internet Access Should I have used ‘switch’? is still quite possible to favor that one port out of the box over the box’s other ports. Menard: Certainly, but not only that, even Menard: Same thing. Just that there are the ISP’s have a problem grasping what is knobs available to physically ensure that at COOK Report: And, in theory, it could put happening. Due to the fact that the current a given point in time, only one guy has ac- its customers on that one port. retail pricing scheme of the cable carriers cess to the network. With Ethernet, there’s for ISP third party access in Canada is so only one machine that can talk on the net- Menard: Absolutely. low, they are not paying enough attention work at a time. Because the cable network is a shared network, the cable modem ter- COOK Report: And somebody else’s on a COOK Report: The public just thinks they’re mination system is there to dispatch who has less favored port. getting a good deal. They don’t realize they the right to speak on the wire at any given can get the low price and still be screwed. point in time. Menard: In fact, that’s what they are trying to do now. Menard: ISP’s are not stupid. They know COOK Report: It’s like the time-sharing that they can’t provide high speed Internet principle, you’re convinced that you really COOK Report: How do you know that? Do access at the speed that cable carriers are have the whole machine, but you don’t. you know it in any way you can publicly providing, given the economics of transit say? bandwidth across the Internet. Menard: Correct. But on the uplink side, there’s even a time slot that’s called a Menard: I was publicly told recently that COOK Report: Because, where are they DOCSIS SID. During that time grant allo- Videotron is attempting to put MPLS tags going to get the upstream from? cation, you have a minimum bandwidth to directly into each and every cable modem. I the CMTS so you can ship your voice pack- can only conclude that this is done, among Menard: Correct. At home, on my cable ets back within a guaranteed delay window. other things, to avoid having to provide non- modem, I get 350 Kbytes per second, peak discriminatory layer 3 interconnection at the downloads. So for an ISP to be able to pro- COOK Report: And you’re saying it’s only point of interconnection with their network. vide continuous 350 Kbytes peak downloads the owner of the level 2 infrastructure that Because they figure, if they put MPLS tags to more than one person, it will need several has access to the knobs that establishes what in the cable modems, the VoIP traffic times that bandwidth upstream. kind of service each party attached at level Videotron’s own customer can escape to- 3 and higher will have? wards the PSTN or elsewhere before per- COOK Report: And there’s a question of formance on the shared media deteriorates. how do you scale the cost of bandwidth con- Menard: The owner of the CMTS gets to nection? dictate the time slots in which cable modems COOK Report: They set up PVC’s for their get to talk on the wire. customers to come to them and then go out? Menard: Videotron figures that bandwidth costs, at Videotron’s price, are of at least $4 COOK Report: Does it have anything to do Menard: Yes. So basically the point of in- per month per subscriber, with TCP/IP addresses? If I have certain terconnection becomes one of one thousand Canadian.[Editor's Note: All prices are TCP/IP addresses and my competitor ISP has network elements in Videotron’s network quoted as Canadian dollars. Each dollar is certain others, can I set the equipment to fa- acting as a firewall for third-party ISP’s. But worth 66 US cents.] That means that vor transmission to my addresses as opposed Videotron’s network continues to work as Videotron is probably paying roughly 500 to theirs? planned. There are multiple escape hatches bucks a megabit per second from Teleglobe within Videotron’s network that Videotron’s for its bandwidth. And they figure that this Menard: No, it doesn’t need to do that. It own customers can escape through while price will enable them to serve their custom- works by means of the Ethernet MAC (Me- every ISP’s traffic will be directed to a single ers at $4 per month. Meanwhile an ISP is dia Access Control) address, inside the mo- “hatch” at one point of interconnection. never going to be able to buy bandwidth dem. from Teleglobe at $500 a month per mega- COOK Report: But this is all so obscure that bit per second. The ISP will have to spend COOK Report: And the media access con- if you’re not telecom literate and interested, more like $1,200 Canadian a month. Or trol address is a different address for each it takes some intense effort to get an idea of $1,100 if it negotiates really hard. house or each customer, right? what’s happening. The average public 71 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA COOK Report: We’re talking about ISP’s fiber access and equipment and salaries that vices at these prices? with 3,000 customers, 8,000? One thousand? the ISP has to pay. Menard: Videotron is presently in customer Menard: Yes, as opposed to 100,000. Let’s Menard: That’s right. And ISP’s used to acquisition mode, at all costs. I don’t know say you go from the retail rate. Videotron’s make $10 per month per subscriber of net how long they can do that, but they seem to saying that its lowest rate that includes cable profit. Now cable carriers are telling them be able to afford it while the last remaining modem service is $34.50 a month, includ- that he should only be making $3, $4 at the independent ISPs are facing significant fi- ing $5 for the rental of the modem. So if most. nancial difficulties. The main issue for them you take $34.50, subtract $5 for the rental at this time is dominance of technology. The of the modem, it’s $29.50. Out of this $29.50, COOK Report: I’d wager that they don’t only way they can be profitable while sell- third party access, the rate approved by the think that’s progress. ing services at these prices is by way of long- CRTC is $19. That’s the best ISP’s can pay term contracts. for the cable modem loop. Videotron says Menard: Indeed. That’s not progress. So it’s at least $4 per month per subscriber, at we’re pointing to a future where ISP’s have COOK Report: With your upstream pro- their $500 per megabit per second price that no means of making money on the access, vider? they get such a sweet deal from Teleglobe unless they just have a million subscribers for. And you can expect an ISP to pay at and then it becomes a financial game, it be- Menard: No, with customer monthly fees least twice that. So if it’s twice that, then it’s comes an AOL kind of play. But the small that are paid for in advance. Videotron has like $8 per month. So add $19 plus $8 per ISP out there today is basically being de- this package where you can buy a year’s month for bandwidth, you’re at $27 and nied the chance to play in the high speed worth of cable modem service at Radio you’re supposed to turn around and sell this access arena that is being sponsored by the Shack, and then get in addition, substantial for cheaper than Videotron, which is selling regulators, which don’t seem to be able to discounts on your installation and your mo- it at $29.95. do anything about it. dem rental fee.

COOK Report: Which is a little more than COOK Report: What we’ve covered in the COOK Report: In other words, they’re say- the 25% discount, that they are getting while discussion so far is the comparative costs of ing to the customer: I’ll give you a $10 dis- resale lasts. providing Internet service from an ISP over count if you buy twelve month’s of service the cable infrastructure as opposed to the from me. Menard: Exactly, under third party access, owner of the cable infrastructure. And also the 19$ price represents close to a 33% dis- some of the issues, if the ISP wants to use Menard: Correct, however, in the case of count if you consider that the retail rates of the cable infrastructure for providing voice- Videotron, for a twelve-month engagement, the incumbent carrier remains 29.95$. Even over-IP, presumably in competition also with the rebate is worth $192 per year if you rent then, from that 33% discount, as an ISP, you the cable owner, right? the modem and $120 per year if you buy the must find your profit within that window, modem, which will be sold to you with a which requires you to find a way to find those Menard: The main, number one issue for $170 rebate as well. Those rebates are very megabits for Internet bandwidth at much the cable companies is that they’re in a battle high and difficult for an ISP to match. They better prices than 8$ per month, which would for dominance of access technology. So it’s also have been giving out free installations instantly kill any opportunity for profits. really a function of who wins — DSL or for the last couple of months. It doesn’t seem cable? Who is the first in the home, because without explanation as to why they claim Menard: For purposes of resale, the Internet once the customer, (the clientele) has been that they are loosing money for each of the bandwidth is included. But under third-party acquired, there is a higher probability that customers that they are adding to the net- access, you have to provide it. But it’s not the guy won’t switch. Although this is likely work. However the stock market seems to only Internet bandwidth, you also have to not as true as they believe it to be, they are, compensate them accordingly. They are us- add $80,000 that Videotron is charging one nevertheless, banking on it. ing the excuse that they are loosing every shot interconnection to Videotron’s infra- month to advocate to the regulators that they structure, plus the monthly fee for renting Their game plan is to acquire as many cus- shouldn’t be forced to sell service to ISP’s the fiber all the way to the point of intercon- tomers at all costs as quickly as they can. at a discount. Basically, they’re saying if nection. There may be other monthly fees They’re in a battle for dominance of tech- they’re forced to provide service to ISP’s, associated with the point of interconnection, nology. That means that they’re dumping they’re in the business of subsidizing ser- but we will only find out in the follow-up their service. They’re providing high speed vice to ISP’s. Whether we really believe that process to CRTC Order 2000-789 which will Internet access pretty cheap. They’re putting is another story as they would have the op- ultimately lead to the availability of the third lots of bandwidth on the network pretty portunity to increase their prices so that they party access service. cheap. And for $34.50 Canadian you can get become profitable. 350 Kbytes per second of speed with a cable COOK Report: These are ISP costs. modem. My 128 kilobits per second ISDN Remember that they are in customer acqui- link that I paid commercially several hun- sition mode, so they’re dumping their pric- Menard: Yes. Plus the Internet servers and dred dollars per month just a year ago is was ing. Next, they only break even because of what not. Basically, Videotron is indirectly slower than my cable modem for which I the long-term contracts. Finally, the only way saying that, in the cable modem business, only pay $30 per month for these days. that they can increase their mark-up is by you cannot have more, in any way, shape or adding more services - in other words by form, than a 14% mark-up. That is if you increasing the bill sent to the customer. And sell at the same price that Videotron sells The Economics of Market the number one means of increasing the bill the service for. Share sent to the customer, with little to no incre- mental cost in the network, is to allow voice- COOK Report: And a 14% mark-up can’t over-IP applications, which will command COOK Report: You question whether it is an extra $20 to $30 per month, on top of the cover all those other things in the way of sustainable to provide Internet access ser- 72 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 $34.50 monthly fee that they’re currently Color and is the largest printing company in of Videotron, was able to prevent it from charging. the world. being sold to the Ontarians, and with this deal,became an even bigger one, to the tune COOK Report: What do you get for that Videotron was owned by telecom people that of 2.2 billion dollars. dollar amount per month? were not too heavy into the content busi- ness. Sure, they owned a TV station called Now Quebecor is a very big printing press Menard: You get another phone line. You TVA, But compared to Quebecor not as yet company. They print phone books. They have the ability to operate a fully-compliant too heavy in the media. print newspapers. They print advertising CLEC business model. Meaning you can wherever they can and they sell all of it on port your current phone number to this new On February 14th 2000, Videotron an- TV. phone line if you want to. The line is life nounced that it had signed a definitive agree- line. But it has no more features than a tra- ment to merge with Rogers, the largest cable COOK Report: So it’s a bit like AOL/Time ditional Class 5 telephone switch phone line carrier in Canada., Warner - mixing content and distribution. service from your local incumbent telephone company. A call detail record needs to be But then On March 24th 2000 Videotron Menard: Exactly. If Quebecor can’t get ad- produced and reconciled for every call even announced that the Caisse de dépôt et place- vertising to your eyes on Television, they if the call is free. ment du Québec was granted a temporary will do it in the newspapers that they pub- court injunction preventing the merger. What lish. If then they cannot get to you in the It’s got all the heaviness of the billing sys- happened was because Videotron is a nation- newspapers, they will get to your eyeballs tem. But you get free long distance calls over alized company in Quebec, La Caisse de on the Internet. They are in the business of very extended areas. So the deal with dépôt Quebec, which is everybody’s pen- selling and printing ads anywhere, anytime. Videotron is, the whole province of Quebec sion fund in Quebec, supplied $2.2 billion For them, the opportunity to bundle adver- is one local calling area. But Bell Canada in cold hard cash of the monies necessary to tising across several media can make their could also do that. If they were really stuck buy Videotron. Critics complained that the offer more distinguishable from the compe- and they had to do it they could do it. How- Quebec provincial government in effect tition. The most valued customer for these ever it is uncertain that the CRTC would bought Videotron with people’s retirement folk are not residential customers, but the approve such a tariff. savings. people with advertising dollars to spend.

Cisco has been convincing Videotron by La Caisse de dépôt et and Quebecor together, COOK Report: But when did this purchase suggesting that with IP telephony, incremen- made a counter bid later in the summer, take place? tal revenues can be added to the infrastruc- which was not accepted because it wasn’t ture. Such revenues are touted as likely to all cash, but on September 13th 2000, they Menard: It was concluded on October 23rd generate cash flows that are bigger than the made a revised bid, all cash, which nailed 2000. However, the meger is still not ap- incremental cost of providing such services the deal. proved by the CRTC. It’s in the CRTC’s on the infrastructure. The infrastructure for hands as Quebecor already owned another Internet access is already there. Adding COOK Report: You mean the pension fund television station: TQS. They are still under voice-over-IP to mix means that you can bill of the Province of Quebec and this private fiduciary management. more for the same infrastructure. Because printing company made a joint bid? it’s still all on the same network, the actual Nevertheless, Quebecor recently cut $112 cost of adding it is small. Menard: Yes. It was widely perceived that million from the operations budget of the government didn’t want to lose TVA Videotron. Quebecor fired the VP of IP Te- Quebecor Buys which is known for being the television lephony of Videotron and a couple of other channel more in tune with nationalist inter- VP’s. Now Videotron before Quebecor Videotron and Changes ests in Quebec. stepped in and changed its gameplan has the Dynamic of the been burning money like crazy for the last COOK Report: French only? two years on IP telephony. All this has just Marketplace happened in the context where Videotron is Menard: Yes. French only and it caters to Cisco’s number one hope to demonstrate that Now, we’re getting into the contentious stuff. the politicians who are nationalist. voice-over IP over cable works. The Telecom Acts say that you have to pro- vide interconnection that is non-discrimina- COOK Report: Is that still an issue? COOK Report: Yeah, two years ago you tory. But interconnection begins to become were telling me this was coming. discriminatory here because Videotron Menard: Yes. You watch the TV news. If wants to keep voice over IP for itself. But you watch Radio Canada, you get the feder- Menard: Yes. That’s why it’s kept quiet, this is getting to be even worse now since alist news and if you watch TVA you get Cisco doesn’t talk about it, because they they have been acquired by Quebecor. They more or less the other nationalist point of know that their premier test bed has been did own a TV station and a few magazines view, although Radio-Canada does a fair job badly shaken. But in addition to all this, one by way of their participation in TVA, the at covering the nationalist interests as well, may ask how ISP’s in Montreal can believe biggest TV station in Quebec that aired the but not the converse. that Videotron is negotiating third party ac- first interview of pregnant Celine Dion. cess to their cable modem infrastructure in However Videotron was not an integrated After a long legal battle, on September 13th good faith? I have been closely involved with media company seeking to bundle content 2000, Quebecor and La Caisse de Depot du a lot of the players and the weight of the with its cable network. Quebecor compares Quebec, made a revised all cash bid which evidence that I have heard leads me to have its acquisition of Videotron as being similar terminated for good the merger agreement serious doubts about their actions. to the merger of AOL and Time Warner. with Rogers. The government of Quebec, Quebecor is the parent company of World which was already a significant shareholder But opinion inside Videotron as to who the 73 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA enemy really is, is divided between those tising dollar on the Internet? You need to per subscriber in order to make AOL the who believe it’s ISPs and those who believe have 80%, 90% penetration for Internet ac- default homepage of Videotron’s users. that the real battle is against Bell Canada. cess. They only way that they are ever go- Videotron could have gotten hard cold cash This latter group is trying to convince ing to make the increase internet market for the eyeballs business of their users but Videotron management that selling cable share necessary to get the advertising rates instead decided to invest in a non-profitable modem services to ISP’s is good business, that they want for their Internet ads is to give portal company and ended up making more because their number one priority must be Internet access away for free. So there’s ev- money than accepting AOL’s deal. This is dominance of cable technology. The issue ery reason to believe that they will eventu- for me a reference example on how the stock is DSL or cable, which one will prevail? So ally give away high-speed Internet access market can lead you to make very weird as- anybody that sells cable, whether it’s them- for free and that ISPs will have to be pre- sumptions about what are the best invest- selves or somebody else, is good for them. pared to compete with that. ments decisions you should make. You would think that they would jump on this opportunity. However, there have been COOK Report: And this dynamic, obviously, Videotron made more money on the stock public reports that higher executives of to readers, has to have tremendous parallels valuation of Netgraphe, the company that Videotron have said that they would not al- to AOL/Time Warner. runs the portal, that they would have ac- low for any deal which would permit ISPs cepted $3 per month per subscriber from to sell Cable Modem services at a lower price Menard: Absolutely. And seeing how AOL AOL. From the overvaluation of the com- than what Videotron charges. What is negotiating in Canada for third party ac- pany running the portal. This is kind of the Videotron doesn’t realize is that in doing so, cess, I have the impression that they do have astronomical, illogical math and finance it is the ISPs which are loosing the money somekind of a hidden agenda. AOL Canada game you can play if don’t have sophisti- while helping their cause for technology is a subsidiary of AOL in the States, but in cated investors. People are buying Netgraphe dominance. They have been accused of be- Canada, AOL doesn’t own any cable net- stock. For what? For online advertising and ing short sighted, and their acquisition by work. So they’ve been pursuing third party zero content? People are buying Excite stock Quebecor hasn’t changed anything. access negotiations. Whereas in the States, and Yahoo stock. There’s little in the way of they pulled out of all negotiations since the real, known, quantifiable cash flows in these At the same time this whole thing is hap- acquisition of Time Warner. companies. It is all, grossly exaggerated fu- pening, they are hit with new forces that once ture value. So this thing is volatile. It can go again are redefining their business model. In the United States, the openNET Coali- poof at some point in time. Now that they’ve been acquired by people tion has been after AOL and screaming that are in the eyeball business, mainly, sell- bloody murder, because AOL, before the Shared Revenue ing advertising at any cost, the only way they acquisition, was adamant and a supporter of can augment the value of their advertising open access, but now that they’ve acquired Business Models, dollar is if you distribute it to more eyeballs. Time Warner, they’ve pulled out of all open Portals and E-Commerce So how can you do that? access negotiations. Taxes COOK Report: You can charge higher rates. In Canada, I clearly get the feeling that AOL is getting instructions from people in the COOK Report: The assumed use of a Menard: But how do you do that when you States, because they’re moving ahead with Internet portal is to collect eyeballs, right? have 30% penetration of high speed cable third-party access, full blast, without any access on cable modems? How do you go consideration to the architecture and at any Menard: Yes, to sell online advertising. But above 30% Internet access? Two things. You price. I was told by people from Videotron, you can’t sell online advertising at the same make telephony an Internet feature, so by that AOL offered $3 per subscriber per price as TV advertising when you have pen- way of people buying telephony, they get month to make AOL the default home page etration of service that is in the neighbor- Internet anyway. Because if you need a cable of Videotron cable modem subscribers. hood of 20% to 30% of the population, while modem that provides Internet access to pro- However, because Videotron is a national- for TV, you have 80% to 90% of the popu- vide IP telephony, the modem will be there. ized company, they decided to create their lation. They have really clear incentives to Now it’s only a function of the customer own portal, called InfiniT. give away high speed Internet access. putting in the computer to it. COOK Report: They being AOL? COOK Report: Because giving it away, if COOK Report: So now they’re seriously in they get the penetration up there, could ac- competition with Bell Canada? Menard: No, Videotron invested money tually increase their total revenues. into a company called Netgraphe and they Videotron - Free Internet created this portal called InfiniT, which is Menard: Through advertising. And the last their default portal, in effect the @home thing on top of that, is after they’ve done the Access to Increase portal for Videotron subscribers. The fact advertising eyeball game, then they can force Revenue from Content that InfinitT is the default homepage of everybody on their network to give them a Videotron’s users certainly has to do with cut on all e-commerce transactions. As telcos Negraphe’s high-flying stock price on the charge rights of ways for the telephone poles, Menard: Oh, absolutely. And the second Canadian stock market. It’s its resemblance then they’re going to charge rights of ways thing, and the most important thing, is how with Excite@Home which, ultimately, gives for an Amazon.com transaction traveling on do you raise penetration to 80%, or 90% lev- Netgraphe such a high value. the portion of the network operated by this els, which ultimately justifies the existence captive portal based subscriber. of TV channels, because it’s a service that Because of the stock market hype for por- appeals to a vast majority of the population tals, Videotron ended-up making more COOK Report: How would that work? and thus justifies premium advertising rates. money from Netgraphe than from striking a Could they get away with that? How do you charge more money per adver- deal with AOL, which offered 3$ per month 74 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 Menard: Yes. And it’s been quoted that COOK Report: While some call centers can ISP’s, if any, going out of business. What Excite@Home has publicly announced that be slow, this is probably not true of the en- do you say that to an argument like that? they would tax e-commerce transactions tire mail-order industry, but I hear what made on the @Home network. you’re saying. Menard: In Canada as in the USA, dial-up has still been growing. We are just now fac- COOK Report: How could Excite@Home Menard: So it may be a sufficiently impor- ing the turn point where ISPs will begin to tell who comes onto Amazon through them? tant deterrent to say so that the site of Ama- loose more dial-up users to high-speed ac- Excite@Home would simply say to Ama- zon losing business potentially for lots of cess than turn-up new dial-up users. The zon, either pay or we’ll cut off access to you? @Home subscribers would see Amazon smaller ISP’s that operate dial-up business working a deal with @Home where, will- are still viable, so they don’t have the knife Menard: Exactly. It could go that far. Let ingly, without any network element to actu- at their neck yet. But six, nine months, a year me run by you a possible implementation of ally enforce this, they would kick back Ex- from now, if they’re not in the high speed my paranoia: They could say that here after cite for all transactions. business, they will be in huge trouble. any TLS, 128-bit encrypted traffic going to Amazon.com is cut off from a subscriber COOK Report: But, again, this is purely COOK Report: Is the mix of the business going through the cable modem of @Home. speculative, because Amazon has to think, different in Canada, percentage-wise? There would be some network box that re- if we agree to this, then we have to do it for directs payment sessions through a server everyone and we’re having a hard enough Menard: In Canada, it’s like 80% cable mo- that’s aware of the fact that Amazon is to time staying in business anyway. Why dem and 20% DSL. In the States it’s the re- kick back a specified amount to the owner should we capitulate and give in? If we don’t verse. of the infrastructure. give in, will there be enough of a fuss and we look like the good guys and the other COOK Report: So you Americans, look up COOK Report: It would be kind of a game guys look like the bad guys. here to see your future. In the sense of the of chicken, wouldn’t it? If Amazon and some market segmentation. of the big people have the guts enough to Menard: Yes. It’s a chicken and egg situa- say we won’t pay, would it get cut off and tion. Nevertheless there are people at Menard: Yes. Today several people within what would the subscribers say? @Home that are saying publicly that they’re Videotron are not taking too seriously the going to tax e-commerce. management’s wishes of attempting to find Menard: Well, then, Barnes & Nobles a business case for giving away cable mo- would be very happy to be the premier sup- COOK Report: You’re reminding me that dem services for free. But you have to re- plier of online books for @Home subscrib- people are saying this publicly and, if there’d member that it was the President of ers. been a tremendous outrage about it, presum- Quebecor, who said they would be giving ably they wouldn’t be saying it anymore. away cable modem services, not some crazy COOK Report: Because you think Barnes junior executive. & Noble, if they could do without Amazon Menard: True. But they seem to be able to they would pay something? get away with it. This whole thing points to The business modeling guys at Videotron a high speed Internet access over cable busi- know the cable modem business case by Menard: Probably. It’s all about shared rev- ness model which is highly volatile and be- heart, cent-by-cent, they can take the $34.50 enues now. The only way you can get into ing redefined as we speak. After all time we monthly fee and pry it apart with a plus or high-risers these days is to become a Build- have spent in this interview, we’ve only cov- minus one cent variation. $5.00 is for rental ing Local Exchange Carrier (BLEC) that is ered the cable television side of the high- of the modem, amortized over six years, sharing a significant portion of its revenues speed Internet access equation so far. The three years or whatever. And bandwidth is with the building owner. The same could same kinds of things are happening with allocated this way: da da da da. They know apply to electronic commerce if mega-me- those trying to stake their future on DSL, it all inside out. And now I’m supposed to dia empires are allowed to provide bundled although it is taking more time to happen. give this away for free and tax e-commerce telecommunications services without giving and do advertising-based captive portals. any regard to non-discriminatory intercon- Where Does this Leave There is tremendous resistance to this within nections with pure-play telecommunications Videotron. service providers. the ISP? COOK Report: Well, in order to avoid be- In thinking of an architecture that would Now, is an ISP supposed to react to this vola- ing laughed at, doesn’t the Quebecor presi- make it possible for an incumbent carrier tile and evolving model, with a business dent have to prove that he has a billion dol- operating a captive portal to intercept pay- model, which is entirely subscription-based? lars in the bank and that he can capitalize ment transactions, that’s technically possible. The monthly subscription will have to be this until they can charge more to justify the The worst that could happen is that the ac- high because it will not be reduced by ad- new model? tual payment transaction would move back vertising, e-commerce taxation, bundling of from the web to a voice telephone call. But IP telephony, etc. How would a pure-play The Content Based Business Model and Free I think there’s a sufficiently high number of traditional ISP offering only traditional high- internet Access people that have grown accustomed to not speed access, email and web browsing be want to do phone-based mail orders because ever able to remain profitable? This is why Menard: I would describe it this way: it’s they don’t want to spend half an hour wait- it’s so important to have proactive regula- really the content-based business model that ing for a call center agent to do their phone- tory policy from the government. is fueling the media mergers. However con- based mail order. The service provider may sider what Andrew Odlyzsko said recently, get away with asking for a small portion of COOK Report: Cable modem sales and DSL at the Canarie 6th Advanced Networking the monies involved in electronic commerce supposedly are going like gangbusters in the Workshop. “It is all about connectivity. transactions. U.S., but I haven’t heard so far of many People don’t care to pay for content. He said 75 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA that from a historical point of view in all from Quebecor is based on a super bundled irked so many people and imposed compres- major changes in telecommunications tech- price that, if you’re Kraft Canada and you sions of 112M$ has slowed Videotron down nology people were not paying for content, want to advertise Kraft dinner, you make a for a while as they were required not to make but always for connectivity. When the polled check to Quebecor and you get so much any further capital expenditures until the new the room, asking: “Who would lose email?” eyeball hits on the Web, so much air time on Quebecor strategy is fully in place. Nobody raised their hand. But when he TV, so much pages and columns in the news- asked in comparison: “Who would lose paper, all for a bundled price. It will be im- COOK Report: Do you mean that it made Web?” Everyone raised their hands. So it’s possible for you to evaluate how much Videotron a lot weaker and they don’t have connectivity. It’s not content. What’s the least you’re really paying for the TV portion of more market muscle? likely thing that they would lose? Email. If the advertising. So it’s a way to differentiate they were given the choice. yourself, because then you can hide from Menard: It’s slowed down the whole pro- customers the true price that they’re paying cess. If I were privy to the discussions be- COOK Report: Yeah, but the content people per element. You do this by using bundled tween these guys, I would find that it’s to who are buying in have different color eye- advertising. This kind of thing could be used the point where they’re really engineering glasses. to create situations of unfair competition. I their future moves. It’s war in there and they claim that the Bureau of Competition in have to win the battle over DSL to become Menard: Exactly and many investors keep Canada, or the Federal Trade Commission the dominant technology player above all. putting plenty of money into those people in the United States, could play a role in Meanwhile Bell Canada has not stood still that are producing content. making sure that these kinds of things do and acquired CTV, the largest Canadian TV not go on without being supervised. station, for the same purpose. COOK Report: But doesn’t the owner of Quebecor also have to say “I’ve got $500 Maybe I’ve turned a bit paranoid and I’m There is serious cross subsidization going million that I’m going to subsidize the saying things worse than they are. Remem- on here. While I want to know what’s the amount of time that it’ll take for me to use ber that the Quebecor acquisition of true cost of an ad on TV all I have is an offer this business model, to do what I claim it Videotron has shaken Videotron up. Clearly, from Quebecor. The offer states that the only will?” there are people out there in Videotron who kind of advertising I sell you is this bundled are saying that I am exaggerating the situa- scheme of newspapers, Internet, and TV. Menard: The only problem is that right now, tion. Some people within Videotron are con- the CRTC has not given its approval for the vinced that the president of Quebecor is not COOK Report: Has anyone ever done this acquisition. There’s some notary that’s ap- mad enough to give away cable modem ser- before? pointed to manage the company until the vices for free. But I really don’t care. He, transaction was approved by the CRTC. himself, said that he would do it, and if I Menard: It’s being done at Quebecor, but Until this happens, he cannot change the were an ISP out there, I would ask myself on a smaller scale. An example is Riptide, business model of Videotron. However I where will my business be three, four, five which is a small ISP owned by Quebecor, expect that it will change not too long after years from now, knowing full well that this well-known for having access to literally free the merger is approved. This is why I am could happen. advertising in the local newspaper. And that’s telling ISPs to do like me, and stay tuned how they leverage their ISP business is by while being prepared to face the worst. COOK Report: For how many months have having free advertising in the newspaper. the Canadian Association of ISP’s had to deal Whereas any other ISP would not get free COOK Report: Is it thought almost certain with this now? A month? Two months? How advertising in the newspaper. that it will be approved? long ago did this happen? COOK Report: Has this melting of the bar- Menard: Yes. It will certainly be approved Menard: I’ve known that this was going on riers between technologies and business and when it is watch for the announcement at Excite@Home since last April, and that models meant that any kind of government by Videotron of free Internet access. this has been going on at Videotron since its regulatory oversight can no longer keep up acquisition by Quebecor. with what’s going on? COOK Report: Sounds bad. Do you have laws designed to make the concentration of COOK Report: But when did it become pub- Menard: It’s very late for them to keep up. media in a few hands more difficult? lic? But not as late as it is for government to keep up with condo fiber builds. That’s really, in Menard: There’s foreign ownership restric- Menard: It leaked out after Quebecor finally French where we say ‘dépassé’. It’s got re- tions, and there is something preventing one acquired Videotron last September. ally run over by the situation. guy from owning all television stations. However, there is nothing saying that if you COOK Report: Is there much of an uproar COOK Report: Well, I want to hear some- own a newspaper, you can’t own a cable amongst ISP’s yet or are they too busy with thing about condo fiber builds, but can you company. There’s things that prevent you other things? sum up where we are in this. This is like the from owning every TV station, but that does whale coming up out of the ocean, the new not cross over to newspapers. So nothing is Menard: Too busy with other things. They business model that is fraught with all sorts preventing this from moving forward, aside can’t quite qualify this risk at this time. They of difficulties and is likely to create an from the Bureau of Competition. have little appreciation for it. I’m at a point Internet run by people who certainly don’t where I really believe that some ISP’s de- give a damn about the ethics and so on that However someone could eventually testify serve to die their own death, under their own we care about. before the Bureau of Competition that it’s debt. impossible to know how much Quebecor is Also, say more about captive portals, as you charging for advertising on TV now. Why? Fortunately, the whole acquisition process seem to see a world with more of these things Because the only way to buy advertising of Videotron and the fact that Quebecor has out there. 76 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 Menard: Exactly. I’ve introduced this new some perhaps unexpected turns of events that Menard: Yes, Hydro-Quebec is the electric term, which I’m trying out on people — cap- may have brought an ongoing process to a utility for us in Quebec. Most of the power tivity enforcement points. There’s a new state of maturation where the likely conclu- over here is generated with hydro-electric network element out there. sion looks ominous. But you’re also saying turbines attached to huge reservoirs up in that simultaneously and parallel with all of north of Quebec, so it is very common for COOK Report: Like a network access point? this commercial mayhem is all this dark fi- us to refer to electricity as hydro-electricity, ber going in the ground. You also said that thus Hydro. Menard: Somewhat. It’s really a captivity Bell Canada has to compete with these de- enforcement point. But it’s more than just velopments that are eating away its installed COOK Report: Your point is, if you abso- IP Internet related. There’s really captivity base. What are we to make of all this? Where lutely had to make a choice between elec- enforcement points in the telecom industry, are we headed? tricity and a telephone, you’d choose elec- telecom/media industry, where you have the tricity. captivity of the end user listening to your Menard: Let’s say we go twenty years in publicity, but you also have the captivity of the future. What will be said of the cable Menard: Yes, of course. your advertiser customer. For example, I companies and of the phone companies was would suggest that Kraft Canada is captive, that they ventured into monopoly market COOK Report: And how they bring elec- because now that the rules have changed, powers, but failed because they were not na- tricity to homes is by means of wires attached the barrier to exit is too high. Kraft may not tionalized companies that had access to the to poles. want to buy into online advertising, but finds rights of ways and the business models that it has no choice. It has to buy into online which didn’t give them any choice buy to Menard: Correct, or underground by way advertising, in order to get what it prefers, have a wireline infrastructure. Electric utili- of conduits. A significant percent of the poles because it’s all in this bundled price. ties and gas companies cannot provide their in Quebec province are owned by Hydro- services wirelessly. For energy utilities, op- Quebec. While the utility companies can’t Furthermore, Nortel is introducing a new erating a wireline infrastructure fits much be in the telecom business, they are indirectly box, which is a mix of the Nortel Passport closer with their business models as they providing an infrastructure that allows for ATM Switch and the Nortel Shasta broad- have no real competition from wireless play- competition by multiple parties. band service node. Nortel combined the two ers. Laying telecom infrastructure and pro- parts together at the request of certain cus- viding and operating Layer 1 and Layer 2 is COOK Report: They automatically are pro- tomers especially interested in providing ultimately the responsibility of the guy that viding an infrastructure over which telecom network-based services. This box can be has a natural business model for wireline in- can run? especially good at implementing captivity frastructure. enforcement points. Menard: Correct. Correct. Let's say you're But twenty years from now the cable com- a city or a utility and you have to build struc- COOK Report: As all this maneuvering of panies and the phone companies will have tures that can be used for telecommunica- customer and advertiser occurs, won’t Bell become integrated suppliers of publicity and tions to allow for fiber to be put underground Canada be forced to engage in a fight to the information services and content. They or on a pole. Then either you’re in the busi- death with Quebecor? won’t know whether they are in the Internet ness of conduits, whether they’re aerial ducts business or in the telephony business. Nei- or underground ducts, or you put in the fi- Menard: Absolutely. They’re getting ready. ther Bell nor the cable companies has a busi- ber and then you let other people use the They’ve bought CTV to do that, they’ve got ness model for putting in a telephone wire. fiber. If you let other people use the fiber, newspapers to do that. And they have an Whenever Hydro decides to pull a fiber op- then you interconnect them to your infra- ATM-based network architecture which can tic cable, the cable company or the telephone structure. You interconnect either at the easily take in this new box of Nortel. company has no business model for pulling whole dark fiber layer, where you basically a fiber optic cable. Whenever the city de- tell the phone company I’m going to give COOK Report: While all the time the dark cides to put a fiber optic cable, the cable you a 20-year IRU to this subscriber’s pair fiber is robbing them of revenue. company or the telephone company or the of fiber and you do whatever you want on energy companies have no business models it. Or you break that apart at another layer Menard: Yes. I would not put it as such. I for doing so. and you try to share that fiber there. would rather say that new technologies are making it possible for people to build net- COOK Report: If we are talking new net- And now it’s at a point where, almost by works with a much lower cost structure. But works are you saying that Hydro or munici- obligation, the management of Hydro-Que- rather than changing their networks, some palities have the basic advantage? bec realized that to be simply installing ducts of their executives are thinking that buying is not enough to remain viable, as there is content engines like sports team is a better Menard: Yes, of course. Hydro or the mu- not enough money that can be charged just investment. Therefore Rogers has bought the nicipalities will be the first company to dig for the rental of conduits. The case of the Blue Jays. And Rogers is looking into buy- in. Hydro is the first company to put in a City of Montreal which operates a network ing the Montreal Canadians. These folk can pole. Hydro is the only company that is ob- of conduits through the CSEVM, the Elec- see the world only in terms of traditional ligated to provide service to every house with trical Services Commission, is a specific case content. a wire. Because the only thing that’s not which holds true mainly for a downtown wireless in this world is electricity. If you’re environment where there will be tens of play- Enter Cheap Dark Fiber in going to have one wire in a house, it’s going ers interested in renting space in the con- the Hands of New Players to be an electrical wire. duits. COOK Report: So Hydro is the electric com- But this may not hold true for residential COOK Report: So where are we now? I pany. areas. With only two or three infrastructure think we’ve covered in the cable thing are owners, there may not be enough money 77 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA pulled-in from just the rental of conduits. Ultimately, that’s what subscribers want. the conventional telephony or conventional Renting the fiber itself, by giving an IRU to They want horizontally specialized ISP’s and television business. But this is also precisely all of the fibers is not sufficient either, be- a selection of providers. The result could be why I think that society cannot trust either cause it doesn’t provide means of ensuring that at home, I select Blockbuster as my ISP the established telephony or cable television competition. So then the idea becomes to for my set top box, I select AT&T as my ISP players to build the unified information high- open up one layer above the physical fiber. for my IP phone, I select Worldcom as my way. And the standard best use for that one layer ISP for Web browsing, I select Equinix as above the fiber today in equipment language my ISP for application server provider ser- Allowing electrical utilities to provide the is that which is made by Worldwide Pack- vices. There’s literally nothing that would telecommunications infrastructure would ets, for bringing gigabit Ethernet over fiber technically prevent that from happening. create a level playing field for telecommu- to the home. However the lack of foresight from regula- nication service providers. I believe that the tors allowing these media mergers to con- existence of a level playing field is neces- COOK Report: In other words Layer 2. tinue may result in this not happening. This sary in order to ensure the viability of com- is what we can build into the architecture of petition in telecommunications. But, in or- Hydro to the Rescue the Hydro sponsored Brossard trial for third der to create that level playing field on the party access. Therefore in looking at how telephone network, you basically have to Menard: Yes, Layer 2. But then, the history the architectural technology plays out in the deny the phone company’s ability to pro- of third-party access in Canada points to an regulatory environment, I can go from the vide Internet service. architecture that’s opened up at Layer 3. If complete worst nightmare of an ISP to the you are the electric company (Hydro) and most ideal situation of an ISP. There are some of us who believe that regu- are selling third party access, you will do so lators should have done that years ago, but as a function of your point of interconnec- COOK Report: In other words, if what they were too shy to do this. I am still con- tion. The question is should that point pro- Quebecor is doing with Videotron is now vinced that the monopoly that was granted vide a Layer 2 or a Layer 3 interconnection? the world of darkness out there, then going to phone companies was to recover costs of to the electric company or municipality and building an infrastructure to provide conven- Let’s say you have a pair of fibers going from showing it how to build their infrastructure tional telephone services. For as long as we the home to this distribution center. From so that it enables competition in telecom- believe that the ongoing costs are more this distribution center, you want to go to munications promises to be the knight on slanted towards entertaining an obsolete cir- ISP A, telephone company B, cable TV com- the white horse riding to the rescue. cuit switching technology and that the physi- pany C. How do you share access to that cal copper plant has been paid for, there fiber? Well, you can’t give everybody a Menard: Exactly. I am very concerned with should be no reasons why telephone com- lambda, because doing such takes a box on finding out what will be needed for this to panies would be allowed to provide high- the premises that listens to multiple lamb- happen. This is where I think that commu- speed Internet access over their copper plants das. Such a box is too expensive today. Since nity dark fiber networks and condo-style in order to recover the costs of this infra- it may work a couple of years from now, in constructions may have the most significant structure. It is based on the aforementioned my efforts to lobby the CRTC I strive to impact. I say this because it will be either reasoning that I believe that incumbent tele- make sure that the regulations that get passed these new builds which will cause this to phone operators should strictly be forbidden out there do not prevent such a thing from happen, or it will be the fear that these new from providing data services, especially at existing. You cannot build the network so builds will happen anyway which will cause rates forborne from regulation. that each competitor has access to different the carriers to wake-up and convince them- strands, as this would also be too cost pro- selves that they should do it or else face ob- Division of Content from solescence. hibitive. Transport as a Result of What the ISPs and the incumbent carriers Literally, you should be able to find situa- Community Fiber Builds have agreed on to this date in Canada, is that tions where everywhere in the world, elec- tric utilities and municipalities will be will- an Internet Protocol router could provide the However, I never make a big deal about di- necessary mechanisms for common-carriers ing to do that, in new developments. Doing it everywhere else will take a lot of leader- viding content from transport because I think to meet their requirements for providing non- that this is a can of worms. There are people discriminatory interconnections. The router ship from the politicians as any attempt to force this upon the electrical utilities will that have business models which bundles would be shared by multiple parties at a point content and transport and there are other of interconnection, enabling interconnec- certainly not be well received as they are heavily in debt and facing the issue of the people which have business models based tions to be formalized at a layer higher than on transport only. Sure I think that dividing the the pure electrical or optical layer. This introduction of competition in electricity production and distribution. Nevertheless, content from transport is a good thing, but makes it possible to use an inexpensive that would make cable television a money Layer 3 Ethernet Switch to do the job. Electrical utilities and municipalities are better suited for profiting from competition loosing proposal when the fact is that this in telecommunications as they do not have network has been able to grow because of What we have in fact been explaining is the such bundling. I think that the division of difference between the new business mod- business models based on vertical integra- tion of services. content from transport will naturally happen els of incumbent telecommunications carri- by way of community fiber builds. ers going wild with media mergers and the traditional ISP’s which cannot afford to buy The reason why this experimentation is es- pecially good for the Hydro company today COOK Report: Are you looking at working into this. What ISP’s are trying to provide is with the electric company, then, to take the a monthly subscription based service, not is because they have no telecom service in operation right now to cannibalize. So why fiber infrastructure they have and create an cross-funded by advertising or e-commerce alternative Layer 2 network that the ISP’s taxation or captive portal kinds of services. would they prevent IP telephony or IP tele- vision from existing, as they are neither in can jump off of where they are now, just 78 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 before the train hits them and jump onto this Menard: Just a portion of it. Hydro-Que- many fundamental issues that need to be other platform for doing business? bec has some fiber, but it is either for their addressed with regards to what a city can do own use or they have sold the excess capac- and cannot do and how telecom competi- Menard: Absolutely. My issues are to find ity to Bell Canada. Some of it could certainly tion is supposed to happen. If a municipally out if such an infrastructure will be able to be used to interconnect fiber to the home wants to rent fiber instead of conduits and attract a sufficiently high number of sub- trials to a point of interconnection down considers this fiber to be the public rights of scribers resulting in sufficiently high pen- town. The conduits that are owned by City way, then no regulation is out there to pre- etration rates to ensure the viability of the of Montreal electrical services commission vent them from doing so. This is why I be- ISP’s business model. The thing is those are (CSEVM) are used by as many as 20 carri- lieve sometimes that this industry is a zoo. I all experimental trials and that there’s not ers on the main routes. This makes some often think that the objectives are to get as going to be a million subscribers available portions of the conduit system very lucra- far as you can, when you can, by whatever on this kind of infrastructure overnight for tive to operate for the city. You can imagine means necessary and that Darwin’s law pre- these ISP’s to go after these new business that revenues on some portions of the net- vails, that is, only the fittest survive. models on a commercial basis. work bring in a lot of revenues at prices of 3 to 9$ per meter per year for as many cables I’ve originally thought that a business model COOK Report: Well, how much of this new as can fit in a pipe. In some portions of the of condominium fiber builds partially paid infrastructure has been built by the electric network, the CSEVM reports that there are for by the school boards to be related to this utilities? up to 18 to 19 people paying utility rates zoo. However I have recently convinced (4$) and up to 30 users paying user rates (7 myself that this isn’t the case because those Menard: Little to none. It’s all green field to 9$). This means that it is the conduits that new networks are being financed out of the stuff. The Brossard trial is a new neighbor- are rented, and not the fiber itself. But just same budgets that would otherwise be used hood of 150 homes initially that will grow to keep things interesting note that a project to purchase services from the various incum- to a maximum, of 1,500 homes over fifteen is before the city of Toronto where the city bent carriers. years. So it’s a missionary thing to go and would rent the fiber instead of the conduits. do that. How could you blame the public sector for COOK Report: Nobody’s challenged the more cost effectively spending the monies COOK Report: Well, of the fiber that’s un- city’s role? that they have been allocated for their tele- derground in Montreal is Already there, how communications needs? If the carriers chose much is owned by the electric company? Menard: Not to my knowledge. There’s not to offer services that offer the same bang

Figure Three: What Gigabit Ethernet to the Home Might Look like Under Ca*net4

79 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA for the buck that new business models for owned by Bell Canada is preferred is be- they go about making it a profitable busi- purchasing rights of use on infrastructure cause the CRTC has mandated attachment ness case of putting in the fiber. built by way of condominium style fiber prices that are much lower than what would offerings, can you blame the public sector cost to build your own support structure over COOK Report: And they’re looking to from going shopping somewhere else? a route of Hydro poles. The only other ma- people like you to help show them where jor reason why you may be denied to attach the alternatives are. Either in-house or an aside from a notice that the pole will soon external consultant? Hydro Quebec Finds that be replaced, is that the supporting metal Competition Is Really strand is already too crowded and that add- Menard: They ask a wide range of people Only Possible on Shared ing any more weight could damage the struc- who fill these mandates of refining the ar- ture. However, given the weight of optical chitecture. Even Worldwide Packets is Infrastructure fiber as opposed to copper, it is rarely the known to do this for Hydro-Quebec. case that adding optical fiber cables com- COOK Report: You’re saying that while promises the stability of the existing infra- COOK Report: What is the difference be- Hydro owns the poles, they haven’t put any structure. tween defining the interconnection and de- fiber on their poles to speak of yet. fining a general business model for them to COOK Report: So if Hydro Quebec looked begin to use? Menard: Correct. No fiber for use by third at its inventory of poles, it would find out parties to be precise. that some scattering of them have some fi- Menard: Very little. It’s really how much ber at this point, right? Owned by other par- money per month can Hydro expect to make COOK Report: At least in the Montreal re- ties? out of its fiber. Since you put in fiber and gion of Quebec, the fiber to the home trials you can share the fiber to do telephone, tele- that are being conducted are all out in the Menard: Yes. On several of them. vision and Internet access, it means that since suburbs, yes? the retail rate of telephony is $30 a month, COOK Report: I’m trying to get a better fix and the retail rate of TV is $30 a month and Menard: Yes, Brossard to be precise. Given on what you think about the electric utility the retail rate of Internet access is $30 a this latest trend where Bell Canada is no beginning to venture into telecom, and how month, that the customer is expected to pay longer the first entity with a business case that will affect the battle between Bell at least $90 a month. If it were only for tele- for constructing support structures, we also Canada and the cable TV Internet people. phony, Hydro couldn’t charge more than find that they are not taking the initiative to Maybe it won’t affect it that much? Maybe 40% of that $30 as a fee to Bell Canada to build the conduits where fiber-to-the-home it will be just one more force of privatization provide service. So the fiber would only be initiatives require that underground struc- of community networks feeding in and con- viable if it is used for several services. tures be constructed. necting to a commercial provider? What is motivating Hydro Quebec now to say: “Oh, COOK Report: So if they can use it for three COOK Report: These dark fiber builds that we better join the race and start buying our different services, a sustainable business IMS is doing can be on anybody’s poles? own fiber?” model becomes feasible. One service is not But who has poles? Only the electric com- enough. pany? Menard: The justification of Hydro Que- bec for doing the Brossard trial is the fol- Menard: Yes. Because at $90 a month, they Menard: No, both the electrical utility and lowing: In their 2000 to 2004 strategic plan, can charge $10 for each ISP service to one the local exchange carriers own a portion of they have determined that they should try to entity and attempt to make a profit on one- all the wooden poles. The inventory of poles put the electrical wire underground, wher- third of the basic fees charged to the is separated somewhat evenly between Bell ever possible. And they cannot find a busi- subcriber. Overall, the electrical utility can and Hydro-Quebec. This is because of his- ness model to do so unless they put more begin to make money because of competi- torical arrangements between the two com- stuff in the ground than just the electrical tion on the same infrastructure. I have yet to panies to which I cannot get to, dating to the wire. find an occurrence of viable competition on early eighties when both were monopolies. multiple, physically different wireline infra- Today, I think it would be difficult for Bell COOK Report: So they’re interest in put- structures in the residential sector. It hasn’t to find a business case to be the one plant- ting in their own fiber is only, as you said, worked to date. What has worked however ing new poles given the fees that were set green fields, absolutely brand new construc- is competition from multiple different infra- by the CRTC for the rental of space on elec- tion sites. But why are they having to put it structures, say wireline and wireless, cop- trical poles. I suspect that historically, many underground? per and coax. Nevertheless, this is what the poles have been planted by Bell Canada as decision for local competition is supposed they thought of this as a long-term invest- Menard: This neighborhood is a specific to make happen. I am still upset by the fact ment that they could make while not paying case of a rich neighborhood without any that the CRTC refuses to consider other monies to the electrical utilities. At the same above ground, unsightly poles. forms of competition than facilities-based time, the electrical utilities would continue competition. to plant their poles where the structures COOK Report: But it seems, then, that Hy- planted by Bell Canada would not accom- dro Quebec, as you said, just has barely a COOK Report: Because you’ve got to use modate the new higher-power lines. big toe in the water? That is compared to all different facilities. Except that, in reality, this other infrastructure? you’re talking about parallel, vertical facil- What’s important is that more recently, it ity empires. seems that all new poles are being laid by Menard: We’re just seeing the tip of the ice- the electrical utilities in the new develop- berg and it’s all green field. They’ve got the Menard: Correct, but with only three physi- ments. The reasons why a route of poles green light to do whatever the heck they cally different facilities, there can’t be com- want. It’s really up to them to define how 80 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 petition. I’m convinced that there’s not force of privatization of community net- Menard: So the future points to a compli- enough money generated by the system to works feeding in and connecting to a com- cated balance between commercial commu- have competition on more two physically mercial provider? nity networks, co-op community networks, diverse wireline infrastructures. By physi- and the end-result of the involvement of cally diverse, I mean something that is dif- Menard: I suspect that some residential energy utilities in the establishment of new ferent than various forms of copper such as telecom services will develop as a co-op infrastructures, all of which is combined to coax or twisted pair. In this case, I am refer- while others will evolve as community net- the efforts of the established incumbent car- ring to copper and optical fiber. works run by private companies but con- riers attempting to prevent the evolution to- trolled by municipalities. Can you see it de- wards this direction. Sure, there can be competition between wire- veloping in a different way? Some cities will less and wireline, but I don’t consider com- be too small to interest private companies COOK Report: What you’re saying is the petition between wireless and wireline to be and will have no choice but to develop their Hydro stuff that’s beginning now may what the regulators had in mind in their de- community networks as co-ops. Some evolve into a community network co-op kind cision to promote facilities-based competi- denser cities will get better services than co- of model. tion anyway. People say that for oligopolies ops. Generally, co-ops are not the better ser- to disappear, there need to be at least ten vices. People who buy their food at a co-op Menard: It will preserve the quality of the competitors. technically pay a higher fee for food at a co- telecommunications service that we have op than they would pay at a supermarket or today in a world where we clearly suspect at Wal-Mart. But yet people that buy food at that that if we leave it to the private sector, it Developing Community a co-op at least have a decent level of ser- will degrade rapidly as the telecommunica- Owned Business Models vice in the middle of nowhere. Because they tions services become focused on content own the store, they get a refund for every exclusivity and advertising-funded. The COOK Report: OK. What do you think time they buy food at the co-op. utilities and municipalities can change this about the electric utility beginning to ven- as they are entities clearly known to have ture into telecom. How will that affect the COOK Report: And there’s the satisfaction no economy of scale from the selling of ad- battle between Bell Canada and the cable of being in charge of their own infrastruc- vertising. IMS also can do its share of the TV Internet people? Maybe it won’t affect ture. effort to alter this trajectory, but it cannot be it that Much? Maybe it will be just one more

Figure Four: IMS Condo Fiber Builds in Montreal

81 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA out there on all markets at the same time. entitled: A Paradigm Shift for the Stupid the nature of the services that they offer. They Then again, it may not be all energy utilities Network presents this idea in great detail. want to do this in order to become more com- and all municipal administrations which may See http://www.tmdenton.com/pdffiles/ petitive in their service offerings. Condo fi- be receptive to these new opportunities. A_Paradigm_Shift_for_the_Stupid_Network.pdf ber networks serve a dual purpose in that they provide end-users with connectivity the Fallacy of Facilities COOK Report: Well, the question that seems way they want it and that they do so imme- to follow from that is what form of compe- diately while providing carriers with new Based Competition: No tition will regulators continue to promote? facilities upon which they can engineer new One Will Ever Over Build Are they going to make any changes? But, business models while not endangering too what about this condominium fiber? Where rapidly the business models of their exist- Bell Canada does this fit into the picture? Is it signifi- ing facilities. How rapidly the public carri- cant? ers are prepared to reform their business Let me return to my earlier point: the ’96 models is the question that will determine Telecom Act promoting local competition, Condo Fiber their survival. Until then, it may be that for and the CRTC Decision 97-8, promoting several customers, building their own path local competition are not to be trusted. Fa- on the telecommunications grid by leverag- cilities based competition won’t work with Menard: Condominium fiber is very sig- ing the condo fiber business model will be the established wireline players as it has not nificant, as it dramatically shakes up the the only way to ensure that they can have worked to date. Nobody is ever going to find landscape of facilities-based telecommuni- the services that they want at a reasonable a business model for overbuilding Bell cations. Condominium fiber will be ex- price. Canada for a long, long, time, for as long as tremely appealing for public institutions with Bell Canada can squeeze everything that it several locations within a finite geographi- What is Left for the can out of its copper while being allowed to cal area. We can think of school boards, mu- be involved in anti-competitive interconnec- nicipal buildings, government offices, hos- Incumbent Carrier? tion practices and content bundling. pitals and hospital satellite offices and clin- ics, etc. I see another wave of commercial We could imagine certain areas where the COOK Report: Did you think a year ago that condominium fiber and co-op condo fiber public sector would no longer be purchas- it might work? in the area of connecting communities, ing any services from the incumbent carri- where private companies should be sensible ers aside from local PSTN interconnection. Menard: No, I never thought that it would, to the needs of the private and corporate resi- The business that would be left for the carri- but there’s a bunch of CLEC’s out there be- dents of communities prepared to build their ers in these areas would be formed of a mix- lieving that it does. Sure it does for the busi- own facilities rather than waiting for carri- ture of shopping malls, business condos, and ness sector spending a lot of money on voice ers to do so. residential. It’s a lot of business taken away telephony services. In this case, some people from the incumbent carriers. From that per- find it possible to overbuild a downtown COOK Report: Used by police, fire depart- spective, the landscape of telecom will re- core. But you cannot overbuild residential ments as well? quire a tremendous amount of consolidation areas for basic telephony services. Who’s once all these guys are no longer buying ser- going to put in a piece of copper to compete Menard: Yes. By municipal buildings too, vices from the incumbent carriers. I would with Bell Canada? I would challenge any- like public libraries, 911 call centers, water suggest that we’ve proven that most phone body at the CRTC to convince me that they treatment facilities, etc. What strikes me as companies have no interest in providing ser- are acting in the best interests of the popula- problem however once all public sector en- vices based on the best networks that they tion by only promoting facilities-based com- tities are connected to each other with pri- can design because it would cannibalize too petition. The worst in all of this is that found vate fiber optic networks, there will not be a much their existing businesses. As the pub- in the CRTC decision 97-8, which affirms whole lot of business left for private tele- lic sector in Canada is showing to this date, that of facilities-based is to be the only form communications operators to fight over in you have to build your IP network on con- of competition aside from resale. This means certain areas. What is left for public wireline dominium fiber if you want it to reap all of that the CRTC has not considered other telecommunications operators may only be its potential benefits. forms of competition than resale or facili- residential, commercial and industrial build- ties-based. ings. It is also safe to bet that low-density COOK Report: And the phone companies areas will also be covered by higher-speed just think they’re going to survive on politi- I for one believe that there is the existence wireless communications. The most prob- cal muscle and cash in the bank? of another form of competition that I call lematic issue is to ensure that this revolu- facilities-less competition. It is more or less tion reaches rural communities. Here again, Menard: Let’s just say they are feeling their based on the notion that the sunset and un- condo fiber has an immense potential for dominant position weakening. This is why bundling rules that allow competitors the such communities, as some of them can be they are paying more and more attention to chance to buy unbundled network elements the ones providing transit to other adjacent condominium fiber. Fortunately for them, from the incumbents at reduced rates for communities, establishing connectivity be- the governments continue to buy into their some period of time should either eleminated tween two urban centers. I am convinced that requirements for the public subsidy of their immediately or extended on a permanent condominium fiber will dramatically alter business models, which provides them with basis. It also defines the notion of network- the landscape of telecommunications for a some breathing room. However, they can- layer interconnections as to offset the cost long time. not base their returns to their shareholders of certain facilities. However, this form of solely on the profitability ensured by gov- competition is not formally presented in the As a result of widespread interest in condo ernment subsidies, as this would destroy decisions giving birth to competition in tele- fiber, I suspect that a lot of the carriers which their speculative value on the stock market. communications. The paper that I published will have participated in the building of the They are beginning to realize that they can- condo fiber network will seek to improve 82 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 not continue to ensure their survival by ask- lowing the phone companies to believe that business model should remain the primary ing for monies through cost studies. They they can continue to buy into the cost struc- means of connectivity of our society for a are coming to understand that they better ture of the existing PSTN. long long time. I for one believe that we start selling telecommunications services to would all be better off, if fiber-optics were customers the way that the customers are For example, there was discussed recently omnipresent. I think that it is so important interesting in buying such services. For a lot in La Presse that the construction of a dark that I would be prepare to trade existing de- of customers, it is both connectivity and fiber network for the Montreal school board regulation of telecommunications to go back horizontally specialized services acquired will result in 2700 phone lines from being to a natural monopoly nationalized utility over the Internet that matters. The problem disconnected from the PSTN. The school with a mandate to build fiber to everybody. is that this is a business model which doesn’t district’s private fiber build will diminish the work well with public carriers which have requirement for PSTN phone lines by a fac- To this end, it may be that electrical utilities spent all their life finding ways to package tor of four. So they will go from 3,600 phone will assert control of the telecommunications services with their infrastructure in a verti- lines to 900 phone lines. That means that, industry given the compatibility of their busi- cally-integrated fashion. suddenly, there are quite a few phone lines ness models. We have seen that market out there that are available on those switches forces are making the business models of I refer to the monopolies of the phone com- for providing services to other people. But the incumbent carriers evolve from a gov- panies as amortization periods guaranteed who’s going to buy them? Yet, the PSTN ernment-entertained natural monopoly on by the government. It means that they can gear which was used by Bell Canada to pro- voice circuit switching to one of bundling buy whatever piece of equipment that they vide services to all these schools will con- high-speed Internet access with content. And want, even if they continue to be monsters tinue to appear in the cost studies of Bell we have also seen that energy utilities, like Nortel DMS switches or Lucent ESS Canada for the next 10-15 years rather than whether gas or electricity, are finding it to switches, and the government will consider Bell being required to take a loss for it. be a compliment of their natural monopo- that to be an integral part of their cost stud- lies to install telecommunications wires as ies, and guarantee a certain rate of return for COOK Report: The school’s new bottom well. We have also seen that some electric their investments. line is that they’ll be running 2,700 voice- utilities are considering the installation of over-IP phones of their own over their new optical fiber to be a key element of their abil- I contend that it robs the phone company of infrastructure? ity to justify the underground installation of much of the motivation it ought to be using their infrastructure for electricity delivery. to install more cost effective IP technology. Menard: Yes. The reason why there were The task at hand is to structure the resale of so many phone lines was that the school the telecommunications facilities of the elec- COOK Report: Is this sort of thing still go- board used ISDN modems to link all the tric utilities so that they provide telecom- ing through the CRTC? schools together. In Canada, there is a munications infrastructure with a level of monthly price cap on ISDN which makes it openness suitable for implementing compe- Menard: Yes, it is still going on today. You available for unlimited use for around $50 tition between telecommunications service should see all the paper that I am getting per month. But this is a real waste of digital providers which base their revenues on sell- these days as part of CRTC Public Notice central-office switching technology. ing advertising and those who base their rev- 2000-27, where the CRTC seeks comments enues on selling services. on certain incumbent carriers proposals to COOK Report: Which produced additional restructure their costing and rate bands. I am revenue that Bell has now lost. COOK Report: In other words, if you’re talking about the rest of the entire Canadian going to bring a gigabit to the home, the forest being cut down just to print that. Menard: Yes, but 64K bits per second is natural infrastructure for doing that is the They’re asking to be able to devaluate their pretty slow. Or even 128k bits per second. poles and conduits of the electrical grid. switches for another ten to fifteen years and What I’m saying is that Bell Canada used to to be able maintain rates so that they can make 3600 times $50 per month of revenues Regulated Access to continue to use their expensive toys. of business from just the telephones of the Montreal City schools. And now, as a result Telecom Public Rights of COOK Report: And maintain their present of the dark fiber build, there’s only 900 Way rates, which should be getting cheaper... phone lines left. So that’s a lot of money taken out of business of Bell Canada’s in- Menard: Absolutely. Society needs to de- Menard: Absolutely. They want to increase come statement every month. So they’re their subsidy requirements in the rural areas cide that it wants to do away once again with going to turn around and say, hey, we have something that is similar to the private road in order to be able to decrease their prices in competition. We can’t pay for this, we’re the metropolitan areas. This is essentially system or the private tramway system that going to go bankrupt, you want us to go we have had in the past. We know from the what the Canadian government is set to al- bankrupt? Then it becomes a very political low them to do with PN 2000-27. problems that we’ve seen in the last few game and these guys are equipped to lobby months about the sustainability of the busi- which they do with a vengeance and, as a ness models that were created for the intro- COOK Report: And the Catch-22 is that result the market stays paralyzed. these alternative infrastructures may put on duction of competition in electricity produc- tion and distribution, that it will not be an the streets at cheaper working rates that take COOK Report: Paralyzed in the sense of their customers from them anyway. easy challenge for our society. I was told that resulting in increased subsidies which just the debt of Hydro-Ontario is bigger than the keeps an unstable and doomed situation alive one of the whole province of Ontario. Menard: Yes, this is why the regulators have a bit longer? to be especially careful about rate rebalanc- Achieving the right balance between the in- terests of the public and the interests of share- ing, or else, it will be the rural areas that will Menard: Yes. Not that it is unstable, but be- pay the price. More importantly, the regula- holders seems to always require mediation cause it continues to suggest that the PSTN and intervention by governments. tors should be especially careful about al- based on copper circuits and its associated 83 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA COOK Report: What we are seeing is that your door pretty cheaply. This newspaper nate what is delivered through its delivery there is a face-off between the old business happens to be printed by the new owners of networks. But if you do not like the postal models and the new business models for tele- Videotron, Quebecor. If you like advertis- system, you can get it through a messaging communications. ing and material that is a notch better, you service, which also relies on the public road can subscribe to La Presse, which will also system, and regulated prices for gasoline to Menard: I would like to make the follow- be delivered to your door by a paper boy, deliver its goods. ing analogy. Today, we could conclude that but will cost you a little more. Either way, there is reasonable competition between both the delivery of La Presse or the Journal The goal of making the aforementioned anal- newspapers. If you live in a big city, you de Montreal will require somebody to drive ogy to telecommunications is that I would can probably get three or four different news- through the streets with a car. This street has like to become as comfortable with compe- papers delivered to your door. If you live in been paid for by everyone’s taxes. In the case tition in telecommunications than I am with a smaller city, you could get the local news- of the newspaper, there is a clear separation competition with newspapers. For the last paper probably delivered cheaper than a between the content and the delivery net- 100 years, we have been used to telecom- newspaper from one of the big cities. And if work. But there is also a lot of bundling in munications services in the shape of voice you live in a remote area, you can get your the fact that Quebecor buys more paper and switching, which did not rely on business newspaper by the mail. If you do not like has more printing operations than La Presse, models bundling content with delivery net- advertising, you can pay to subscribe to a hires cheaper journalists, and ends-up put- works. To the extent that the Internet was purist newspaper, or a specialized publica- ting out a product that is cheaper to receive allowed to become an overlay of the PSTN, tion which is more difficult to get to, be- than La Presse. the Internet remained free of any mandatory cause it is only available by the mail or to a advertising. ISP’s created the Internet as an local shop. Either way, you may have to use In any event, if you do not like advertising, overlay on the PSTN by converting voice the postal system or use the public roads sys- you can receive a specialized publication by circuits to data circuits with modems. tem to get to your content. the mail. In such a case, the means of deliv- ery, the postal system, is a good example of However today, those ISPs are facing incred- To make an example out of Montreal, if you another heavily regulated industry which ible competition from cable television net- accept a lot of advertising and you happen relies on the publicly-owned rights of ways works as a means of transport of Internet to like crime, sex, violence and sports, you to deliver content. The postal system is es- content. Telecommunications regulators in can get the Journal de Montreal delivered to pecially denied any opportunity to discrimi- Canada have found that when used to pro-

Figure Five: Schoolboard Condominum Fiber Builds in Southern Quebec Province 84 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 vide non-programming services, cable tele- there is always the possibility that existing access to the rights of ways and the non- vision networks were in fact, common car- unlicensed spectrum will be increased once discriminatory nature of interconnections. rier telecommunications. Now today, we are it runs out. The same way, there is always The FCC is responsible for controlling the witnessing the attempts of the cable televi- the possibility that more condo fiber deploy- monopolies that have been created with the sion operators to bundle content with adver- ments will occur once we run out of capac- FCC licenses. The Competition Bureau in tising in order to provide a cheaper Internet ity on the existing optical fibers. Canada or the Federal Trade Commission access for the masses. But, by doing so, they in the United States would be charged of must provide interconnection to ISPs before Let’s consider the current state of the tele- determining where anti-competitive prac- there is any content injected on the wire so communications industry and the threats that tices are used to prevent interconnections as to make it possible for end-users to buy new telecommunications business models where market forces consider them to be content and its distribution in an unbundled pose to the very existence of the Internet as appropriate. fashion if so desired. we know it today. The responsible things that we must now do as society are: When I say that the Ministry of Industry in The telecommunications industry has Canada or the Department of Commerce in evolved as a natural monopoly by way of (1) to set the rates for access to public rights the USA should manage the telecommuni- subsidies. In the same way, the television of ways and telecommunications support cations industry, I say this because, at least industry has progressed significantly be- structures, so that they are within reach of for Canada, it would be an extension of what cause of incredible amounts of subsidies. enough competitors. (2) clearly define the they are already doing. In Canada, it is the Finally, electricity production and distribu- nature of non-discriminatory interconnec- Spectrum Administration of the Ministry of tion is still heavily regulated and subsidized. tions between all telecommunications net- Industry (Industry Canada) which manages What is so encouraging about condo fiber works, at all layers of the OSI reference spectrum in the air. To a certain extent, business models is that with them, we are model, and (3) clearly identify how public whether we are talking about lambdas on finally breaking away from heavy regula- funds are to be spent to get enough com- fiber, NTSC channels on coax networks or tions to rely purely on softly-regulated mar- petitors to build out the telecommunications ADSL frequencies on PSTN copper loops, ket forces to provide services to end-users. grid to places where market forces will be all these allocations of frequencies could be In order to build such networks, access to insufficient to ensure a comparable level of handled by the government organizations public rights of ways are necessary. It is the competition and service quality. specialized in spectrum management. The responsibility of the governments to provide reasons why I think that this would be bet- access to such rights of ways on a non-dis- COOK Report: Does the CRTC have a role ter than a specific entity such as the CRTC criminatory basis. However, given the fact anymore? or the FCC is that it would force these orga- that it may be impossible for municipal ad- nizations to focus on dissolving the monopo- ministrations to deal with several carriers, it Menard: I doubt that they do because they lies that were created by licenses from such is in the interest of society to find how best can’t enforce their role as managers of pub- organizations, and in Canada, this means to regulate access to public rights of ways lic rights of ways and privately built tele- incumbent local exchange carriers and for telecommunications. communications infrastructure, and they broadcasting distribution undertakings don’t have a clear mandate under law to do which have had exclusive territories in the How society manages rights of ways and so. Sure they can say that every Canadian past. public funds to enrich certain telecommuni- ought to have a telephone and create mo- cations players rather than others, at the ex- nopolies to make this happen. However this In the face of the new optical Internet tech- pense of certain end-users, which cannot has been done. Case closed. It is not to say nology incumbent telephone company or afford to buy into condo fiber, is and always however that another body such as a Minis- incumbent cable television company busi- will remain a challenge full of little dirty try of Industry or a Competition Bureau can- ness models should go away naturally. Part secrets. The only dignifying task which can not ensure the smooth behavior of the tele- of this natural death should be that society be performed by governments with the help communications industry by requiring that stops sustaining their business models based of regulators is to ensure that the process is interconnections occur on a non-discrimi- on inefficient circuit switching architectures. conducted openly, with the public interest natory basis. The problem is that it is very likely that those always clearly in mind. who will be able to build infrastructures will COOK Report: Are we about to give up on be the ones that will pay for what they build To this end, I believe that the involvement them? through advertising. It is a bit like competi- of electrical utilities, municipalities, and tion in the newspapers, if you are fed-up with government agencies interested in partici- Menard: Well, I’ve said and it’s not the first advertising, you can subscribe to another pating in condominium fiber deployments time I’ve said so, the only responsibility of publication which doesn’t depend on adver- to be most positive thing that could happen the CRTC is to tame the monopoly monster tising. However, you may not be able to get to the telecommunications industry. Tele- that it has created. Once this monster is it delivered to you by a kid and you may communications public facilities deploy- tamed it should be the Competition Bureau need to get it by the mail, a day late. ment is, and will always remain, a first come and the Minister of Industry that controls first served opportunistic business. There what happens next. Society has been artificially sustaining in- will never be enough momentum behind any cumbent telephone and cable company net- government intervention to change the dark COOK Report: So the rough analogy in the works. Those networks should go away natu- side of this industry in our capitalistic United States would be the Department of rally. They should die a natural death. This economy. As society, we can only live at Commerce and the Federal Communications is not to say that the private sector cannot be peace together if we make the process open Commission (with perhaps the FTC thrown retained by society to build and manage net- to all interested parties, both large and small, in)? works. However I have serious doubts that which is something that wireless spectrum it is in the public interest to allow the legacy auctions have failed at doing. Fortunately, Menard: Yes, the Department of Commerce network operators born through public sub- should be the one that’s ensuring equitable sidy, to continue defining what telecommu- 85 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA nications services, or its business model are. ity might be to borrow Robert Metcalfe’s so in order to share the costs, it should not As we’ve concluded in the paper “Bellheads idea that they should be divested of services be denied the opportunity because it alters vs Netheads,” the last monopoly of the in- and allowed only to sell access to their physi- the business model of one particular tele- cumbent telecommunications operators is to cal plant to all comers. It’s a very frustrating communications carrier. define what services are. See http:// problem to one who clearly understands how www.tmdenton.com/netheads3.htm the current technology can do a vastly supe- This is more or less what recent telecom- rior job at a fraction of the price. munications regulation allows communities COOK Report: In other words, we should to do. Any entity in Canada can file for a stop artificially sustaining the life of large What ever happens, private fiber builds rep- non-dominant carrier license and build its vertically integrated networks that own and resent an irreversible process. In Canada the own infrastructure. It can do this on support control access to their own infrastructure? public sector will have an incredible incen- structures with mandated prices. This may tive to own and manage its own networks be in effect, the greatest success of recent until such time as dark fiber services are of- telecommunications regulation. Regulation Some Thoughts on fered on a general basis. Residential users in Canada barring franchise fees may the Public policy and the will typically go for whatever service is the single biggest advantage that facilities-based cheapest, and it is probable that they will wireline carriers may have in Canada over Private Sector shoot for advertising-funded telecommuni- their counterparts in the United States. cations services. However, I do not believe Menard: Look at it this way. The roadways that it is in the best interests of society to It may be that the involvement of munici- and the old legacy giant telecom networks allow carriers operating networks with a palities in the building of community net- are both on public property. They get a li- right-to-expropriate license, to charge rates works will be even more beneficial in the cense from the government to expropriate a for interconnection that are anti-competitive. United States than in Canada due to the fact piece of public or private property so they Such a right to expropriate, as granted by that this may be the only way for cities to can put in their own private telecommuni- the Telecom Act of Canada, comes with re- stop asking for franchise fees. If a city be- cations infrastructure and charge a rent for sponsibilities, the main one is to provide in- comes a participant in a project to build and accessing it. terconnection on a non-discriminatory ba- control a system of optical fiber cables across sis. the territory of the municipality, and grants Sometimes it seems that we’re going to have access on non-discriminatory basis to any the public switched telephone network for- If it becomes only possible to finance a tele- interested party, it has no reasons to charge ever. So perhaps we might as well just keep communications infrastructure by way of franchise fees to offset the costs of keeping the machine flowing like it is, but deny the cross-subsidy from advertising, let the rates the public rights of way in good shape. The phone company’s ability to be in the Internet for interconnection show, but do not allow Federation of Canadian Municipalities is access business or other enhanced services. for the nature of the interconnections to pre- considering asking the CRTC the permis- Tell them it’s a cost plus 25% business. vent service providers which do not want to sion to do the same for them as for their coun- That’s it. No marketing. Cost plus 25. It’s a develop a service package which isn’t based terparts in the United States. We believe that nice business model, but it’s not going to be on publicity, to be denied the opportunity to they should instead consider becoming an an exciting business. do so. In any event, our capitalistic society active participant in a community network should not artificially prevent any entity, that infrastructure rather than implement a com- But when you allow for monopolies to ex- can afford to do so, from accessing public plex system of franchise-fee management. ist, you have to be really, really, really care- rights of ways in order to build its own tele- ful as to how you trust that they’re really communications infrastructure, if it chooses COOK Report: So there are a lot of social operating at cost plus 25. Another possibil- to do so. And if a community decides to do

Figure Six: An Example of One of the Largest Customer Owned Municiple Government Customer Owned Dark Fiber Builds 86 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 consequences in letting the present direction such a way that what’s viable as a business company is going to be in business twenty go on unchallenged. model is very drastically changing. Conse- years from now. And if I want an Internet quently, your governmental oversight of it, network that behaves like an Internet, that Menard: Yes. In the light of what I just said, to the extent that it exists, is in a continual is not an MPLS network, that has IP/v6, that there are incredible incentives for citities to catch-up game. does multicast, I better own the network, become involved in community networks because I’m not going to get it from the in- projects. The condominium model is the one Menard: Absolutely. It’s absolutely clear. cumbent phone company. I’m not going to which is most advantageous to all parties as So I don’t know to what extent is the Cana- get it from Bernie Weber’s WorldCom. The it also allows for the improvement of the old dian National Broadband Taskforce is supposed to help that, ness of doing what’s in the best interest of but it’s this kind of initiative which may help their customers. They’re in the business of COOK Report: Is this good, bad or indiffer- to break out of the status quo. But Canada is doing what’s in the best interest of their ent? so big and population is becoming so disin- shareholders. terested with politics, that it is it is difficult Menard: I used to be one advocating the to evaluate what is the public interest in do- At the same time you have very different death of the incumbent operators. I have ing away with the current status-quo. public sector networks being built. Optical since evolved my thinking, and now only BGP, optical peering, optical cross-connects believe that their old business models are to Instead you have people electing people who on private fiber networks, linking public blame instead. I advocate that all interested are making promises. The next government utilities, schools, municipal buildings — the parties be given an equal opportunity to play could undo laws that the previous govern- whole public sector. In reality, the issue is a role in the construction of community net- ment makes. The telecommunications indus- that this public sector infrastructure is going works. The telecommunications business is try gives sense to Darwin’s law in a society to be so good that adding private sector traf- and will always remain an opportunistic disinterested with politics. Our society car- fic on top of it would be almost not notice- business. I think that telecom networks man- ries more and more inertia with time, and able. aged like cities is the right model. Sure you’ll making important decisions such as devel- have recurring fees to maintain the network, oping a a national policy framework for Not noticeable. So there you go. The public but this is the best way to allow for organi- high-speed Internet access carries more sector builds its own network and gives ac- zations to do away with exorbitantly high speculative promises than quantifiable val- cess to this unlimited bandwidth at cost to recurring fees. Today, we really have a good ues. Existing carriers can leverage their po- residents of the country and to businesses, opportunity to build networks so that they litical influence to extend the scope of their so that they can do business. do not stop functioning if everybody uses established business models which will cer- them at the same time. tainly delay for a long time the benefits that COOK Report: But the private sector in the society could reap from new business mod- US and Canada right now is trying the same COOK Report: But what is the predominant els for the telecommunications industry. scheme. However, it’s trying it from a busi- philosophy? In the United States at this point, ness model base that is bound to crash. the predominant philosophy would be to let COOK Report: Yeah, I agree with that. In all the private community networks do their Canada Bill St. Arnaud and Canarie are do- Menard: Correct. Because unlike printed own thing. That’s the so-called ‘free mar- ing some brilliant things with developing media, mandatory advertising cannot fi- ket’. It seems that’s what’s hollered out on cheap high bandwidth technology, but what nance the Internet. The lack of penetration every street corner, but I think you’re also you’re saying, that’s almost like one play for the Internet and the technical incompat- saying that it’s really deceptive in that it’s being enacted on a broader stage, where all ibilities between advertising and interactive fraught with the seeds of its own destruc- these other zoo-like things are going on to communications mitigates against it. I can- tion. determine how that stuff is going to be used not see how any ISP will ever be able to by the end user and if it’s even going to reach charge for a service which would interrupt a Menard: No, I agree with you. However, in the end user. multipary online game of Quake III every the United States, I get the feeling that sev- minute to display an ad for a new product eral political variables are preventing com- Menard: Private condo fiber builds and op- for adolescents. munity networks from being viable. On one tical BGP are the only avenues to carry for- hand, you have the cities with a history of ward and scale the Internet way of doing COOK Report: On the other hand we’ve certain municipalities granting franchises to things. Given my evaluation of what the tele- been talking about what will work. certain parties, and in turn, charging exorbi- communications industry will look like in tantly high franchise fees to them. There is twenty years from now, I believe that pri- Menard: It will if you’re willing to admit no apparent national leadership from the vate condo fiber builds and optical BGP are that the most stable thing to do, today, in FCC to define what are the prices to access also the most promising and stable thing to telecom is to build your own private network the support structures (poles + conduits). do. and your own Internet. Which, literally, is Finally the infamous e-rate (education-rate) what CaNET*4 is and what RISQ is doing. forces the various schools to purchase their COOK Report: Be didactic. In the sense of, They are applying this recipe all the way telecommunications services from the car- explain to me why indeed this is the most down to communities of interest on a met- riers at a significant discount. This prevents stable thing to do. ropolitan area network level. school boards from better investing the mon- ies that they should be allocated to purchase Menard: Because, if I were a school, I COOK Report: RISQ is what? telecommunications services, to buy their wouldn’t believe that the phone company stake in dark fiber community networks. would be able to provide me service of the Menard: RISQ is the Reseau d’Informations quality that I need for the next twenty years. Scientifiques du Quebec, an organization COOK Report: Are all these technologies Why? Because I don’t know if the phone owned by all the universities in the Prov- converging into overlapping platforms in ince of Quebec which has the mandate to 87 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA interconnect all universities together. RISQ COOK Report: So after seeing this come COOK Report: You almost won’t have to has built a dark fiber network of several thou- crashing down around them, the Quebec say that there’s a lesson here that we can sands of kilometres spanning the whole population who may no longer be adequately teach you. You can imply that there’s a les- province of Quebec, which is very large. If served by Quebecor and the rest of this 20th son to be taught by inviting people to hop we’re admitting that the most stable thing to century business model, should probably be on and see for themselves. do for the public sector to ensure the stabil- glad to have this other alternative? ity of its telecommunications strategy over Menard: Once you got fiber into the local time is to become a participant in the build- Menard: Yes. And it’s proving that if the school, I can bet you that I can convince all ing of community networks, it seriously public sector finds it financially justifiable of my neighbors to get fiber laid in the street dampens the relevance of the private sector to go out and building its own telecommu- and bring the connectivity right to the local being subsidized to provide services. You nication system for its own use, it’s point- school. It’s going to be a community-based are essentially proving that subsidized pri- ing towards the right business model that will decision, co-op based, the association of the vate networks have no reason to be. That’s a redefine how society purchases telecommu- residents of this neighborhood, one-time fee, big secret now, isn’t it? nications services. no monthly subscription fees, we go and we hook up to this public network. It will hap- COOK Report: It’s not a secret to you. And COOK Report: But it seems utilitarian, too. pen, because the cost of doing so will be low it’s no longer a secret to me, but it’s a big I mean, what you want to be saying is that enough that this can happen. secret to a hell of a lot of people. you can demonstrate, not necessarily whether you should do it or not, but it is what COOK Report: This also reminds me of Menard: If the private sector free market works. It’s what works in the sense it pro- what happened last year in Umea, Sweden. zoo in both the US and Canada amalgam- vides the most bandwidth at the best price There a family of computer geeks under- ates content with means of delivery and by for the most people. stood the possibilities of the technology. speculation consolidates markets, puts ISPs They costed out the construction of a100 out of business and leaves market share in Menard: Yep. It used to be because of old Mbps multimode-fibre network to all homes very few hands, we will see high prices, poor technology that you needed public subsidy and a 100 Mbps singlemode-uplink to the access and the best promises of the new to provide services at affordable prices for city’s largest ISP for their neighborhood of internet technology denied. the most people. Only fifteen years ago, 60 small homes on the basis of 95% partici- mechanical switches for placing a telephone pation. The cost came to $2000 per home. COOK Report: but in Canada you will have call across the country were still widely in Fifty-eight of the 60 homes signed up. Be- more and more community networks with operation. The responsibilities for manag- yond the initial investment, each home tremendous cost advantages of at least a ing this network were colossal in compari- owner pays $10 a month for hundred mega- thousand to one over the private sector net- son to today’s technologies. Looking at the bit per second internet access. The extraor- work? current state of the art technologies we must dinary website that tells the story in text and ask if residents are now well served by far several dozen excellent pictures is found at Menard: Yes. It is my hope and wishes. I distant entities which define what services http://www.acc.umu.se/~tfytbk/mattgrand. think it’s a fair thing to say that optical BGP are with the aid of a public subsidy license is the best possible architecture by which from national regulators. Or should we elimi- Menard: I think the network will be big government and public sector things would nate the public subsidy license in favor of enough that the owners of the rest of the in- likely to exchange traffic, by routing lamb- what can now be built by the residents of frastructure will have no choice but to come das on shared optical switches. those very communities in a free market? and interconnect.

And, in truth, it’s not adding residential ISP COOK Report: And what Canarie and all of COOK Report: That’s an interesting way to traffic on that network that’s going to cause you guys, are demonstrating is the power look at it. But, in the meantime, if OBGP that network to crash. And it’s because the capability and cost-effectiveness of what works satisfactorily, it’ll help the network most intense sources of bandwidth is going new technology networks can do. So what get big enough. to be entertainment. But it’s not the public if they’re built with some public funds, but sector’s responsibility to provide entertain- they’re built by people who in effect are Menard: Yes, community networks can ment. So maybe you need a parallel content operating on green fields, without any of the scale, at extreme levels of bandwidth, with- delivery network to provide the entertain- earlier industrial or technological hang-ups. out noticing the impact of residential sub- ment function. But you don’t need the scribers in joining on the network. Internet to provide the entertainment func- Menard: The key is no motivation to pro- tion. tect established revenue streams. You can’t trust a guy who has an embedded base of American Greenfield COOK Report: So you see the possibility of applications to give up on it. Fiber Companies and this being kind of a lifeboat for telecom needs for Canada to jump into when the sur- COOK Report: And pretty soon, when some Canada’s Public Net vival of the fittest perhaps wreaks havoc. of this stuff starts to unravel, if you guys have done a good enough job, it seems that will COOK Report: Won’t there be private green Menard: A lifeboat for whom? have a situation where the Internet model field companies like Level 3 and some of may be able to continue for a long time. the new fiber-built companies that will be COOK Report: For the telecommunication wanting to adopt the technologies of the needs for the population. Menard: The network’s will be so good that publicly built networks? the private sector will be able to hop on it Menard: No it’s a lifeboat to maintain the for free and it won’t make any difference Menard: They'll interconnect with these promise of Internet. for them to be bundling content and access publicly built networks. New carriers with at a reduced price than access alone. lots of capacity such as Aerie will be in a 88 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 privileged position to provide the missing building Bell Canada in the residential sec- private sector will not find a justifiable busi- links between the community networks that tor ever with a wire line infrastructure. So I ness case for pursuing its existing new busi- would otherwise remain isolated due to the think that we should stop to kid ourselves ness model of bundling exclusive content fact that they are not be ajacent to other such with facilities-based competition and instead with high-speed Internet access, so it may community networks. mandate the construction of new facilities quit on its own. It may say “I throw the blan- instead of allowing sunset rules to go on for- ket, I can’t compete with that.” COOK Report: And enough understanding ever. We should prevent explicit cross-fund- of the relative value of the technologies. ing of community networks infrastructures And if nobody in the private sector wants to with PSTN revenues. So this may be enough provide the interconnection, then what will Menard: Yes. Communities are communi- to put an end to this zoo that we have, people do? Why can’t the houses on my ties. You can’t force certain traffic patterns plagued with unreliable business models and street talk to the network that’s at the school? between communities. It will always be that no ability to validate that any CLEC will be Sometimes I hope that we may be seeing certain communities interact very little to- able to become viable in residential sectors. the beginning of a perfectly natural transi- gether. We can’t change the way public rights We can’t rely on the LECs to build the tion from privately owned, public telecom of way have been allocated today. We clearly Internet of the future, so communities will networks to community networks, some of agree that facilities-based competition have to build their own. And if it wasn’t for which will be owned and operated by the doesn’t work. Nobody’s overbuilding the the e-rates in the States, your schools would public sector entities directly. facilities of Bell Canada. There’s also not a be doing the same. single CLEC serving primarily the residen- COOK Report: Government action must tial sector in Canada. That being said, let’s see what happens. If enable edge decision and control rather than the schools do it, then the city does it, then impose a master plan on all communities. COOK Report: And the CLECs in the States the hospitals do it, then the fire stations do Nevertheless, the incumbent guardians of the are not working, either. it, then the governments do it, then the re- local loop will take every last penny and have gional offices of the government do it. every politician buy every remaining favor Menard: Indeed. So the government has the that they possibly can. But the only way to option of extending the sunset rule, which COOK Report: And a very few school dis- fight against it is educating as many people is to maintain the discount pricing of the lo- tricts in the United States, even with the E- about what it’s possible to do right now in- cal loop for a while, or get rid of it as soon rate go out and do the financial calculations cluding building locally owned public net- as possible to make it possible for people to and decide to build some of their own fiber works. Eventually what you are doing will justify the urgency of building new infra- anyway. become better known down here. The stu- structures. By renewing the sunset rule for- pidity of maintaining the E-rate and schools ever, or until such time as it determines that Menard: And what’s going to happen is the and library consortium subsidy of our local there are enough facilities, it is delaying the public sector will have its own public loop must become broadly understood and building of new infrastructure. Its a danger- Internet with acceptable use policies driv- that program over thrown. Then we may ous regulatory double-edged sword. ing demand for new Ethernet appliances for have an opportunity to follow the lessons using site offices, buildings, schools, you are teaching. In the meantime I don’t see anybody over- videoconferencing system and whatnot. And there will be an especially big market. The

Figure Seven: The Customer Owned Dark Fiber National Infrastructure Discussed in This Issue

89 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA "Scaling the Internet" - Excerpts from a Draft Bill St Arnaud in New Paper Examines Traffic Issues His Hypothesis Suggests That Scaling Problems May Force Traffic Off Backbones

Editor’s Note: The complete paper is 7,000 traffic load will increase as the computers in- year is a faster rate of growth than “N squared”. words in length. Below we have stitched crease in power and speed because the same Could it be that this growth rate reflects not together an abbreviated version of 2200 number computers can support more simulta- greater usage by humans but in fact an increase words. The paper is to be found at http:// neous sessions. in the number of average connections per com- www.canet3.net/library/papers/scaling.html puter resulting in total number of “N squared” We urge readers to grab the full version as SNIP connections across the Atlantic, each of which well. is carrying traffic equal to one connection in It is clear from this simple example the effect 1996? Introduction of increasing N is not so much greater traffic load but a dramatic increase in topology load. Unfortunately as yet we do not have any sta- Current Internet networks are built on network For a service provider an increase in topology tistical data to refute or support this interpre- architecture paradigm that remains unchanged can be much more expensive than increase in tation of the data. However if there is a direct from the early days of telephony where traffic traffic load. With increased traffic load a net- correlation between the number of connections is aggregated in a classic “snowflake” pattern work operator only has to increase the size of and increased traffic volume then the implica- of hierarchical nodes. This network architec- the pipes proportional to the actual usage. With tions are ominous for existing Internet service ture has worked well for traditional telephony increased N, however, all the pipes intercon- providers and consumers of bandwidth alike: when there is single hierarchical application necting potential users have to be increased in (telephony), a considerable degree of local traf- size at some function related to the square of 1.For organizations, even if there are no new fic versus long distance and where large mul- N. applications, Internet traffic will continue to tiplexing ratios can be achieved at each node. grow regardless if the number of employees However, increasingly network engineers are It is important to note that even if the offered and/or computers at the organization remains beginning to wonder if this network architec- traffic load remains the same an increase in N constant; and 2.For service providers the to- ture model will scale with the huge growth in will result in an increased number of connec- pology of networks will continue to increase traffic volumes on the Internet. There has been tions in the network. Because network in size even with a fixed number of customers some anecdotal evidence that the core infra- engineers cannot predict with certainty when and with no increase in offered load structure of the Internet may actually be grow- a circuit will be used they must reserve spare ing faster than the offered load. If this proves capacity in each circuit. This results in network If Internet networks start to exhibit N squared to be a systemic condition then network engi- capacity growing in some relationship to the phenomena of any degree then these networks neers and researchers may have to re-evaluate square of N regardless if the traffic volume will simply not scale for any large values of the fundamental network topology of the remains the same. N. If the value of N is growing to accelerate, Internet. the value of N squared is going to grow even SNIP more dramatically. SNIP It is a general rule of thumb that most Internet What is fascinating to note is that with large But what are the nodes “N” in an Internet net- networks double their traffic volumes every values of N, the economies of scale of large work? Is “N” the number of humans using the year. For the longest time it was assumed that networks are inverted. If a network infrastruc- network, the number of devices connected to this growth was directly user driven i.e. we ture must grow at some function related to the the network, or some other parameter? Under- are all using the Internet more and more in our square of N, presumably the network costs will standing what is “N” is fundamental to the ar- every day lives. But it is hard to imagine a also grow at a corresponding rate. Accordingly guments presented in this paper. human being increasing their usage of the a network with N-1 users will be less expen- Internet on compounded annual growth of sive than a network with N users, particularly SNIP 100% per year after year after year. Let us look for large values of N. at an excellent example provided by Andrew So on the Internet “N” is not set by the num- Oldyzko in a recent paper “Internet Growth: The effect of “N squared” on DSL and Cable ber of connected computers but by the num- Myth and Reality, Use and Abuse” http:// modem networks in particular could be perni- ber of simultaneous sessions that can be sus- www.cisp.org/imp/november_2000/odlyzko/ cious. These networks are deployed on a flat tained on those computers. More importantly 11_00odlyzko.htm .In that paper he shows monthly billing model and with average ex- with computers the number of simultaneous year-to-year traffic volumes across the Atlan- pected customer usage. Even if the traffic load connections is only limited by the CPU power tic for the Swiss research network called remains constant peer-to-peer applications on of the computer. As computers grow ever more “SWITCH”. The SWITCH network connects these networks will drive up costs of the core powerful they will be able to support greater up most of the universities and research cen- infrastructure of the carrier. and greater numbers of simultaneous connec- tres in Switzerland. For the past 5 years the tions. Accordingly on the Internet the size of trans-Atlantic link has shown consistent year- SNIP N is only limited by the number of connected to-year growth in traffic of 87% per year. computers and their respective CPU power. Internet service providers today are probably But we know that CPU power increases year SNIP at the early stages of dealing with the N squared to year by the famous Moore’s Law where of problem comparable to the early days of the doubling CPU power every 18 months. Con- The growth in the number of simultaneous telephony industry . Already many network sequently even if the number of computers connections may give us a better clue to the managers are complaining of the huge capital connected to a network remain the same, the source of this steady annual increase in traffic. costs that they are incurring because of the A compounded annual growth rate of 87% per dramatic growth in Internet traffic and the

90 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 number of users. This is most likely the first model will scale? It seems to work well for the other hand carriers that do not sell Internet symptom of the N squared phenomena. As an “customers” who are ISPs. Will it scale for service, but instead only sell pipes will be clear example the following e-mail exchange by large enterprise customers? Will it scale for beneficiaries. Operators of carrier neutral Mike O’Dell, Chief Scientist for UUnet on the individual homeowners? Internet exchange facilities should also ben- NANOG list further illustrates the challenge efit from N squared growth in the Internet. of the N squared phenomena: SNIP But the reality is that large ISPs will always “…for UUNET’s network to handle the 100% CANARIE is also leading an active research be needed to provide global Internet connec- increase in gigabit/sec offered load over 12 program on these issues with a concept called tivity. It would be difficult to imagine every months, the gigabits/sec-route-miles capacity customer empower networks. See http:// small ISP and large enterprise establishing a of the network must increase 100% about ev- www.canet3.net/library/presentations/ massive web of radiating bandwidth pipes to ery 4 months…. The deep intuition about net- CAnet4DesignDocument-Sept00.ppt for more all other similar organizations located around work growth dynamics developed over the details. As a result of this research CANARIE the world. years with voice networks simply does not is proposing the deployment of the optical net- yield workable results when applied to very work where customer controlled wavelengths One possible hybrid model is that as commu- large data networks which exhibit huge dy- are dark fiber and routed by the customers at nity based fiber networks become more preva- namic ranges of traffic slosh and the astound- the edge of the network to support massive lent small community based ISPs could offer ing doubling of offered load every year. (and direct peering. Leading edge optical technol- Internet services locally to the community as this is still the case even given how few people ogy in this case is not required to support band- they would be less susceptible to N squared currently enjoy “broadband” access)” width, but rather connectivity. costs. ISPs and large commercial enterprises usually host most of the common peer to peer SNIP In this architecture approach of radiating wave- application servers that generate N squared lengths, the traditional network requirements traffic such as e-mail, web hosting, caching, There is one possible network architecture that for restoral and protection are not necessary. etc. As such local fiber or wireless commu- might allow the costs of N squared to be passed The loss of a single radiating arm is not cata- nity networks connected to these servers can on directly to the consumer without special bill- strophic as in a traditional hierarchical network. be deployed at relatively low cost. The com- ing mechanisms. In this network architecture As a consequence simpler optical network ar- munity based ISP could then purchase dedi- the customer is responsible for building the chitectures can be deployed. cated wavelengths to a number of Internet ex- connections to support the N squared growth. change points within the community and else- Rather than buying an “Internet” service per It is important to note that “customer con- where in the region or internationally. Ulti- se from a “network cloud”, the customer in- trolled” wavelengths does not mean that the mately they would still have to purchase some stead purchases a number of dedicated links customer deploys their own optical network. volume of transit service from a large global radiating out from the customer to a number Instead it is expected that customers will pur- ISP. In this case, the large global ISP could of Internet exchange points where they inter- chase their wavelengths and fiber from com- bill not only for traffic volume, but topology connect with other like minded customers. In mercial carriers and other suppliers. However costs as well . this case if there is an increased traffic load rather the carrier offering a managed service, due to N squared phenomena the “customer” the customer controls the routing of the wave- SNIP rather than the Internet service provider is re- length to an Internet exchange point of their sponsible for building out new connections to choice instead of the carrier. Further Areas of other Internet exchange points. Rather than looking like a traditional snowflake the net- Initially, in this proposed network architecture Research work starts to look like a mesh of intercon- the customers will initially be the GigaPOPs necting radiating stars. located in each province. But over it time it is The N squared phenomena is not new. It has hoped that customers will be individual uni- been a basic feature of all networks since the In essence this network architecture is already versities and ultimately individual worksta- first two telephones. However with traditional in place with smaller Internet service provid- tions on the campus. The proposed CA*net 4 telephone networks it was never a serious is- ers who interconnect to each other at common network will give Canadian researchers and sue because of the large locality of traffic, hi- peering points. In this case the “customer” is industry partners the ability to study alterna- erarchical applications and the high multiplex- the ISP. Generally these ISPs do not charge tive approaches to the N squared phenomena ing ratios that were possible on such networks. for the exchange of traffic between themselves, and develop network product solutions that But the Internet is different. And network ar- but in most cases they still have to purchase might enable the large deployment of scalable chitectures that were designed for the good old Internet “transit” service from a major ISP to networks. Already in pursuit of that objective telephone may simply may not scale for a fun- interconnect to the global Internet. It has been CANARIE is leading a research effort to de- damentally new type of traffic type and fun- demonstrated in an excellent paper by Will- velop a new version of the Border Gateway damental change in the way we communicate. iam Norton http://www.nanog.org/mtg-0010/ Protocol (BGP) called Optical BGP (OBGP) tree.doc that this network architecture can dra- http://www.canet3.net/library/papers/ Further mathematical research and evidential matically reduce Internet transit fees from a OpticalBGPNetworks.doc to enable custom- data of the growth of the N squared phenom- larger upstream provider. For some time the ers at the edge to setup and tear down their ena needs to be carried out. How quickly are demise of many of the smaller ISPs has been own wavelengths in order to enable massive Internet backbones experiencing N squared predicted. But perhaps this is a classic vindi- direct peering with as many like minded net- traffic growth? Will new applications enhance cation of the inverse economies of scale with works as possible. N squared Internet traffic or decrease it? Will N squared growth in the Internet. broadband services to the home with DSL, Impact on the cable modems and higher exacerbate N Although smaller ISPs should be able to un- squared and the self similar nature of Internet dercut the costs of the larger ISPs, they in turn Commercial Internet traffic? Will we have to develop network to- will suffer cost pressures from their larger cus- pologies and protocols so that eventually ev- tomers who can implement similar solutions Clearly large commercial Internet service pro- ery network device will be able to set up di- for their network connectivity. The question viders will face some serious challenges if the rect BGP peering with other devices and net- then is how small in size this architectural arguments presented here are proven true. On works? 91 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA Optical Community Networks - A Canarie Presentation - September, 2000

Available complete as a 2.4 megabyte power point file at http://www.canet3.net/library/presentations/OpticalCommunity-Sept2000.ppt - excepts follow Optical Communities

Customer Empowered Networks (CEN) • School boards and municipalities throughout North America are deploying their own dark fiber networks in partnership with next generation carrier • Individual institutions – the customers – own and control their own strands of fiber • Fiber are configured in point to point private networks; or • Connect to local ISP or carrier hotel • Low cost LAN architectures and optics are used to light the fiber • Control and management of the optics and wavelengths is under the domain of the LAN customer at the edge, as opposed to the traditional carrier in the center • These new concepts in customer empowered networking are starting in the same place as the Internet started – the university and research community. • Customers will start with dark fiber but will eventually extend further outwards with customer control and ownership of wavelengths • Extending the Internet model of autonomous peering networks to the telecom world Market Drivers

• First - low cost • Up to 1000% reduction over current telecom prices. 6-12 month payback • Second - LAN invades the WAN – no complex SONET or ATM required in network • Network Restoral & Protection can be done by customer using a variety of techniques such as wireless backup, or relocating servers to a multi-homed site, etc • Third - Enables new applications and services not possible with traditional telecom service providers • Relocation of servers and extending LAN to central site • Out sourcing LAN and web servers to a 3rd party because no performance impact • IP telephony in the wide area (Spokane) • HDTV video • Fourth – Allows access to new competitive low cost telecom and IT companies at carrier neutral meet me points • Much easier to out source servers, e-commerce etc to a 3rd party at a carrier neutral collocation facility Examples of Dark Fiber Costs

• University network Urban Fiber Builds • Varennes: 50 km - $406K (maintenance $26K/year) • Montreal East: 14 km - $120K (maintenance $9K/year) • Laval: 33km - $213K (maintenance $15K/year) • University network Rural Fiber Builds • Sorel: 54km - $266K (maintenance $19K/year) • Megantic: 40km -$273K (maintenance $14K/year) • Schoolboards • Victoriaville school board -Average price for fiber(s) $2 - $7 per meter • Spokane School District - $US 800/mo for first 5 years then $US 400/mo 92 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 • Over 50 schools • Stockholm - $1200/mo – over 100 schools • Las Vegas School district – 240 schools – Telcordia (Bellcore) prime contractor • Many, many others in the works • Companies like Telcordia (Bellcore), IBM, etc are now leading development of dark fiber networks for schools Condo Fiber Build Examples

• Des affluents: Total cost $1,500,00 ($750,00 for schools) • 70 schools • 12 municipal buildings • 204 km fiber • $1,500,000 total cost • average cost per building - $18,000 per building • Mille-Isles: Total cost $2,100,000 ($1,500,000 for schools) • 80 schools • 18 municipal buildings • 223km • $21,428 per building • Laval: Total cost $1,800,000 ($1,000,000 for schools) • 111 schools • 45 municipal buildings • 165 km • $11,500 per building • Peel county: Total cost $5m – 100 buildings Cost per building $50,000

Figure Eight: Canarie Speaks of the Customer Owned Networks it advocates as Customer Empowered Networks something which considering their impact is appropriate.

93 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA Ottawa Fiber Condominium • Consortium consists of 16 members from various sectors including businesses, hospitals, schools, universities, research institutes • 26 sites • Point-to-point topology • 144 fibre pairs • Route diversity requirement for one member • 85 km run • $11k - $50K per site • Total project cost $CDN 1.25 million • Cost per strand less than $.50 per strand per meter • 80% aerial Due to overwhelming response to first build – planning for second build under way

Figure Nine:the street map for Ottawa's Condo Fiber Network Ottawa Original Estimates

• Original Engineering Estimates • Original estimates turned out to be 10% higher than RFP responses • Estimated cost to connect 22 institutions with 6 fibers to each institution in a star configuration • Total cost $615,000 or approximately $30,000 per institution “on average” • Actual costs range from $5K to $60K depending on how far institution is from center of star in downtown Ottawa • If condo fiber contractor were to double capacity of network (i.e.12 strands to each customer) cost of project would only increase by 10%

94 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001

Figure Ten: Above - the key sections of the center of the Ottawa Network. Below - each section is identified by cost of installation.

95 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA • Or doubling number of participants would increase cost by only 10% (plus cost of laterals for additional institutions) connected with fiber Typical Payback for school

(Real example – des affluents – north of Montreal) • Over 3 years total expenditure of $1,440,000 for DSL service • Total cost of dark fiber network for 100 schools $1,350,000 • Additional condominium participants were brought in to lower cost to school board to $750,000 • School board can now centralize routers and network servers at each school • Estimated savings in travel and software upgrades $800,000 • Payback typically 8 –16 months Independent Study by Group Secor available upon request Advantage of CEN for business

• Significant reduction in price for local loop costs • No increase in local loop costs as bandwidth demands increase • Ability to outsource LAN and web servers to distant location as LAN speeds and performance can be maintained over dark fiber • Access to lower cost competitive service providers at carrier neutral hotels • New entrants cannot afford high cost of building out their own fiber networks Even small businesses with less than 20 employees can realize significant savings and benefits Advantage of CEN for cities

• In downtown core minimizes digging up streets • If N carriers are trying to deploy service then number of times roads has to be torn up is N squared • However with condominium fiber road only has to be torn up once • Produces a competitive market place and level playing field • New competitive carrier can meet customers at carrier neutral collocation facilities • Eliminates market advantage of incumbents • In suburban areas eliminates duopoly of cable and TV companies • The first company to install fiber into suburban neighborhoods will likely have a natural monopoly • In Stockholm home owners have a choice of 4 cable companies • Makes cities a much more attractive place for new high tech businesses and service • Opportunity for new revenue stream for city • New entrant fiber companies are willing to pay city percentage of revenue for access to right of way Critical role for Government

• Customer Empowered Networks increases facilities based competition, levels the playing field and provides greater choice to the consumer • One of government’s key mandates • Governments might be able to encourage by CEN by giving preferential access to ROW to those fiber installers who will sell condominium fiber at an agreed upon price • E.g Dublin OH • Stockholm • Governments can also encourage building carrier neutral collocation facilities • In downtown cores will likely be done by private sector

96 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 In suburbs will probably have to be municipal facility like school board office, etc Possible framework for the last mile

• The R&E community leadership in customer empowered networks is pointing to a possible solution for the last mile to the home • An architecture concept also based on open access customer owned dark fiber using well known LAN architectures • Many competitive service providers share in the cost of condominium fiber • Ensures facilities based competition • Telcos are unlikely to build FTTH in existing neighborhoods because of the huge capital investment and ROI needed on that investment • Solutions for high speed Internet to the home may not come from the carriers but from the R&E community Preliminary analysis

• Early Fiber to the Home (FTTH) was too expensive because it assumed all services would be converged – date, voice and video • expensive terminal equipment required to segregate voice, data and video services at the home • but voice traffic is going wireless • and broadcast is going by satellite • Lifeline voice can significantly increase costs • DC battery power, 911 services • The big driver for residential broadband is not voice or video • It is the Internet • Very soon Internet will carry video and second line voice • So instead of building a converged network such as FSAN, HFC, etc build an Internet network only • Divergence rather than Convergence may be the key to low cost FTTH Gigabit internet to the home • With condominium fiber builds multiple carriers share in the cost of fiber build out to network nodes serving approximately 250 homes • Governments can help accelerate the process by funding public institutions like schools and libraries to acquire customer owned fiber • Build an architecture that guarantees competitive equal access to every neighborhood node • It is impractical to have multiple carriers own individual strands to each and every home: • Therefore let the customer have title to individual fiber from the residence to the neighborhood node * The customer connects to the service provider of their choice at the neighborhood node • The result is third commercial network running in parallel to telephone and cable for high speed Internet only • Avoids regulatory and technical issues of 911, number portability, etc • Encourages SMEs and entrepreneurs to build the infrastructure • Customer premise device is very simple and cheap Gigabit Internet - Possible Architecture 1

• A community consortia would put together a plan to fiber up all public sector buildings in their community • A community can be a province, a municipality, village, etc

97 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA

• A fiber splice box that terminates the fiber at the street side nearby each public sector building such as school, hospital, library is called a “Node” • The fiber build consortia must insure that potential facilities exist near the for private sector equipment to connect up future home owners – colo facility • Small colo facility in basement of school or separate jiffy boxes for each vendor, etc • Private sector responsible for costs of such a facility • Public sector buildings will have dedicated fiber strands that connect to a “Supernode” which is a fiber splice box on the street beside outside of major public sector central facility such as school board office, city hall, university, etc • The fiber build consortia must insure that facilities exist nearby the Supernode for the private sector to install equipment to service home owners and businesses – colo facility • At least one carrier neutral collocation facility plus central offices and head end • Private sector responsible for costs of such facilities Gigabit Internet - Possible Architecture 2

• Additional fibers are made available from the Supernode to all Nodes such that competitive service providers can purchase such fiber to the node • Service provider can extend fiber to individual home owners or businesses at a later date if title and ownership of the fiber belongs to the customer; or • Service provider can offer wireless connectivity from node • Service provider can offer traditional “open access” HFC, PON or VDSL connectivity • A satellite network or provincial research and education network would connect the Supernodes to the closest CA*net 4 GigaPOP. • The CA*net 4 network would be provide national and international connectivity for all public sector traffic • Application grids of “e-commerce” , forestry, high performance computing • The additional strands that would be used by private sector to provide services to home would be technology “neutral” • service provider could deploy any technology they want – FSAN, HFC, GbE, wireless, etc Prince Edward Island community network model

• Assume all supernodes (school board offices, city halls, etc) are along arterial highway (500 km) and interconnected with 48 strands @ $13/m • Cost of provincial research and education backbone - $6.5m • This cost would be provincial responsibility • Assume all nodes (schools, libraries, hospitals) are along collector highways (850 km) and interconnected with 24 strand @ $10/m • Cost of connecting nodes - $8.5m (half the fibers reserved for future connection to home owners) so only $4.25m is charged to public sector • Based on Alberta density of 3800 public sector buildings on population of 2m, then PEI there would 250 public sector buildings • Average cost to bring fiber to every node - $34,000 • But half of fibers are to be used for connectivity to home, so average cost per public sector building then $17,000 which would be paid out of cost savings • Assume all other roads (3600 km) go by all homes and farms with 12 strand fiber @ $8/m • Cost of going by all homes and farms $28.8m • There are 47,000 dwellings in PEI, so cost to go by each home $612. • Cost to connect up home would be less than $20/mo with 4 year amortization

98 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 CA*net4 Design Document - A Canarie Presentation - October 30, 2000 - excerpts A 650 kilobyte PDF file at http://www.canet3.net/library/presentations/CAnet4DesignDocument-Sept00.ppt Design Philosophy

* Research and Education networks must be at forefront of new network architecture and technologies * But should not be duplicating leading edge developments occurring in private sector * Should be undertaking network technology development that is well ahead of any commercial interest * But any network architecture can only be validated by connecting real users with real applications and must solve real world problems * Test networks per se are not sufficient * There is a growing trend for many schools, universities and businesses to control and manage their own dark fiber * Can we extend this concept so that they can also own and manage their own wavelengths? * Will “empowering” customers to control and manage their own networks result in new applications and services similar to how the PC empowered users to develop new computing applications? * Biggest concern to R&E networks (and most ISPs) is cost of Internet transit Overall Objective

* To deploy a novel new optical network of distributed optical IXs that gives GigaPOPs at the edge of the network (and ultimately their participating institutions) the ability to setup and manage their own wavelengths across the network and thus allow direct peering between GigaPOPs on dedicated wavelengths and optical cross connects that they control and manage * To allow the establishment of wavelengths by the GigaPOPs and their participating institutions in support of QoS and grid applications * To allow connected regional and community networks to setup wavelength direct peering relationships with similar like minded networks to reduce the cost of Internet transit * To offer an “optional” layer 3 aggregation service for those networks that require or need such a facility * To partner with private sector in building “carrier neutral” distributed optical Internet exchange facilities across Canada Business Case for Direct Peering

* Typical Internet transit costs - $1000/Mbps per month * For 100 Mbps Internet transit then $100,000/mo * But coast to coast 100 Mbps channel is $1000/mo (e.g. www.Cogent.com) * New optical technology will reduce that cost further * Compelling business case to do as much no-cost direct peering as possible * See http://www.nanog.org/mtg-0010/tree.doc * OBGP is a proposed protocol that will allow massive direct peerings * Each optical switch is in effect a mini-IX to allow direct no cost peering * OBGP will also automate peering relationships * Significant business opportunity for carriers who want to partner with CANARIE in building next generation optical Internet * For example Telia claims that they save 75% in Internet transit fees with massive direct peering * Speeds up convergence time on BGP routing

99 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA

Opportunity for Carrier and Industry Partners

* To participate in a novel new Internet architecture that will allow customers to manage and control their own wavelengths anywhere across the network * Very attractive technology for Tier 2 ISP, research networks and ASPs * Yahoo and Cable and Wireless have already started down this path * It will allow them to create their own network topologies * To provide a valuable new service for customers that will allow them to reduce Internet transit costs by as much as 75% * To develop new value added services in IX brokering and management * To develop new value added services in distributed web hosting, distributed no-cost IX peering, etc * To experiment with new long haul optical technologies that will dramatically reduce cost of long haul transmission CA*net 4 Requirements

* To partner with private sector to build national Internet optical distributed exchange points * See. LayerOne Distributed Optical Exchanges – www.layerone.com * National dark fiber to support new ultra long haul optics and switches where the wavelengths and switches are controlled by the end user * Existing optical networks and switch technologies are still designed from a “carrier centric” view of the world * Carrier neutral colo facilities for the interconnection and location of optical switch IXs * Additional wavelengths or SONET channels to interconnect layer 3 routers and aggregation services * Layer 3 service will be an optional service to GigaPOPs * Wavelengths across the Atlantic and Pacific * Propose to deploy OBGP or similar technology to enable Customer Empowered Networking and support massive BGP peerings

CA*net 4 Possible Architecture

* CANARIE builds heterogeneous network made from many sources e.g (illustrative purposes only): * dark fiber from St. John to Halifax using ULH 16 channel POS * dark fiber condo from Halifax to Fredericton using 16 channel 10GbE * wavelengths from RISQ from Edmonston to Ottawa sharing 32 channel 10GbE system * dark fiber from Ottawa to Winnipeg with Onet using 16 channel POS at OC-192 * wavelengths from Bell Canada from Montreal to Chicago as part of a 140 wavelength system * wavelengths from Telus from Chicago to Winnipeg as part of a 140 wavelength system * dark fiber from GT Telecom from Winnipeg to Calgary using 16 channel 10GbE * wavelengths from Shaw from Calgary to Vancouver as part of a 32 channel 10GbE systems * wavelengths from 360 Networks from Halifax to London as part of a 400 wavelength system * Wavelengths from Teleglobe from Seattle to Honolulu – Sydney – Tokyo - Seoul * On all cross sections a minimum of 16 wavelengths dedicated to CA*net 4 * 2 wavelengths dedicated to layer 3 aggregation service (looks like old CA*net 3) * 6 channels (and OCX ports) reserved for temporary applications like Grids or OBGP QoS * 8 channels (and OXC ports) to be assigned to ORANs or participating carriers based on contribution to overall project * E.g. if ORAN donated fiber through its territory then it gets extra channels and OXC ports to use elsewhere * A wavelength and OXC port bartering and exchange mechanism will be required

OBGP

* Proposed new protocol to support control and management of wavelengths and optical switch ports across CA*net 4 * Control of optical routing and switches across an optical cloud is by the customer – not the carrier * Use establishment of BGP neighbors or peers at network configuration stage for process to 100 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001

establish light path cross connects * Customers control of portions of OXC which becomes part of their AS * Optical cross connects look like BGP speaking peers – serves as a proxy for link connection, loopback address, etc * Traditional BGP gives no indication of route congestion or QoS, but with DWDM wave lengths edge router will have a simple QoS path of guaranteed bandwidth * Wavelengths will become new instrument for settlement and exchange eventually leading to futures market in wavelengths * May allow smaller ISPs and R&E networks to route around large ISPs that dominate the Internet by massive direct peerings with like minded networks OBGP Features

* OBGP Traffic Engineering * To build a mesh of optical switches such that the network operator can carry out traffic engineering by moving high traffic BGP peers to an optical cross connect * OBGP QoS * To build an optical network that will support the establishment of direct optical channels by the end used between BGP speakers to guarantee QoS for peer to peer networking and or grid applications between attached regional research or community networks * OBGP Optical Peering * To provide a peering transit service such that any BGP speaking regional research or community network can establish a direct peer with any other BGP speaking peer through the establishment of an direct optical channel in response to the request to establish the peer. * OBGP Large Scale * To prototype the technology and management issues of scaling large Internet networks where the network cloud is broken into BGP regions and treated as independent customers Target Market for OBGP

* University research and community networks who are deploying condominium fiber networks who want to exchange traffic between members of the community but who want to maintain customer control of the network at the edge and avoid recreating the need for aggregating traffic via traditional mechanisms * E.g. Ottawa fiber build, Peel County, I-wire, SURAnet, G-Wire, CENIC DCP, SURFnet, etc etc * Next generation fiber companies who are building condominium fiber networks for communities and school boards and who want to offer value added fiber services but not traditional telcommunications service * E.g. C2C, Universe2u, PF.net, Williams, QuebecTel, Videotron, etc * Next generation collocation facilities to offer no-cost peering and wavelength routing * Metromedia, Equinix, LINX, PF.net, LayerOne, Westin, PAIX, Above.com, Colo.com, etc etc * Over 500 Ixs and carrier hotels worldwide

OBGP vs ODSI or MPlS

* Existing overlay or peer optical networks are designed for central carrier control and management * Use intra-domain protocols for topology and network discovery e.g. variations of OSPF, PNNI, RSVP, CR-LDP * OBGP is intended for the end user to control wavelengths and OXC * OBGP is complementary to ODSI and MPLS * OBGP is an inter-domain protocol * ODSI or MPLS clouds can be abstracted down to a single OBGP switch and part of a much larger OBGP network * OBGP ( like BGP) is not intended for complex network optimization and path discovery

101 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA

Inverting the Layers

Elimination of restoral or protection * With massive peerings to the edge, the loss of one peer is not catastrophic * Networks look more like “star bursts rather than “ring of rings” * See C Labovitz ACM Sigcomm Aug 2000 – massive peering helps faster convergence

OBGP Traffic Engineering

* Network operator assesses the largest traffic flows between BGP peers within the network * Traffic flows can between iBGP or eBGP peers * If the BGP peers with large traffic flows have WDM channels then network operator can determine through the OBGP message if there is a match in wavelengths and protocols between the 2 peers * If there is a match in wavelengths network operator establishes an optical cross connect to look like a BGP router within the network operator’s AS but with a better routing metric than through original electrical router * Reduces forwarding load on a traditional packet forwarding router * Existing peering relationships and routing policies maintained but allows intermediary ISP to offload high traffic users through an optical switch that looks like a BGP router

102 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001

OBGP Traffic Engineering Process

* A single predetermined default wavelength is used to establish initial peering * Standard BGP configs are used to setup the initial peering * In the initial OBGP message specifies which additional wavelengths that are available on all neighbours, the port address, loop back address of possible virtual router, link protocol, etc * If the router in the middle sees there is a match in wavelengths, protocol, etc OBGP then spawns a virtual router at the optical cross connect * The virtual router has its own loop back and IO port addresses * The routers on either side have to modify their configs to establish BGP peering with the new virtual router * Network operator can change selection of routers by terminating BGP session with existing routers and establish new BGP peering session with new router e.g. AS 3 * Router in the middle can keep track of which AS generates the largest data flows and create a routing “flap” if it finds an AS pair with higher data flows

OBGP QoS

* External BGP router wishes to have a direct optical path through a series of ASs to guarantee a QoS link * Routers along the path use OBGP messages to notify each other of presence of additional optical paths in the link * When a router receives a routing update from a source that has more than one optical path then a special attribute is added to the AS path * E.g. AS 4 advertises to AS 1 x.x.x.1 with a special attribute indicating additional optical path * AS 1 advertises to AS 2 x.x.x.1 with the same attribute as long as there exists additional optical paths between AS 1 and AS 2. If not, the attribute is dropped * Through routing updates the edge router discovers that a direct optical path is possible COOK Report March 2001 103 Optical BorderCOOK Network Gateway Consultants, 431 Greenway Protocol Ave, Ewing, NJ 08618 USA is CA*Net3 Experiment To Enable Peering Without Purchase of Transit Through Carrier Cloud Direct Connection of Lambdas Could Reduce Backbone Transit Traffic Enabling Internet to Cope with Gig E Bandwidth Explosion Editor’s Note: Bill St Arnaud is the direc- could only be found by transiting a back- scale. tor of network projects at Canarie. Since bone to what we might call the “legacy” SONET-based Internet. Let’s call the other approach to building net- 1998 he has been responsible for the coor- works the “capitalist economy” approach. dination and deployment of the world’s first St Arnaud: To answer your question we With this approach different companies per- national optical research and development must examine the current context of data net- form the different tasks necessary to mak- network, CA*Net3 . working. We are witnessing a profound up- ing the over all network function. This is a COOK Report: It has been nearly three years heaval in the traditional customer carrier re- much more horizontal form of networking since we last talked. Most of the predictions lationship. The carriers look at building their and one that allows much more control to you made ten have come true and the networks from the same old standard of a be exerted from the edge. While some users progress made has been stunning. Gigabit centrally controlled network. Consequently, may say that they want to have “cradle to Ethernet and 10 gigabit Ethernet are going they tend to be able to see the only solution grave” network services other users may say to bring an explosion of bandwidth at the to the problem of their customers’ growing I want to be able to route and control my edges of the Internet. But if traffic at the traffic as being one of just adding more of own wavelengths independent of a central periphery of the network is going to grow the same kinds of SONET channels that are controlling authority. The company that pro- by three and four orders of magnitude, already there. They then turn around and vides the fiber, may be separate from the people are wondering how backbone traffic offer to their customers these new channels company that provides the wavelengths can possibly grow by four and five orders as managed SONET services. The major which may be separate from the companies of magnitude. challenges to this approach of networking that provide Internet and other advanced is scalability. services. Moreover I want to be able to con- But first, what is thought to be the limit for nect to my preferred Internet service pro- speed increases at the periphery of the net- COOK Report: When you say carriers, vider who may be halfway around the world. work? Are we ever likely to see one hun- would you make any distinction between the dred gigabit Ethernet? legacy carriers like Bell Canada, AT&T, and It is the later form of “capitalist economy” Sprint and the new green field players like network architecture that will be enabled by St Arnaud: One hundred gigabit Ethernet Level 3, Global Crossing and Williams? all these optical channels. When you can get in serial mode is rather unlikely. When you a thousand wavelengths on a single fiber you get to ten-gigabit speed in serial mode, you St Arnaud: Rather than divide the field into can sell individual wavelength to individual begin to experience all kinds of non linear old and new carriers, I would prefer to talk customers. Your customers may even be able effects in the fiber. For example you get po- about two different models of building net- to bypass the traditional transit supplier by larization dispersion and Raman scattering. works. The first I would call the centrally constructing their own wavelength network. Consequently, while I do not see one chan- controlled command economy and the sec- nel at 100 gigabits on one strand of fiber, I ond what I would call the capitalist or more COOK Report: Are you saying then that if do see the possibility of 1000 channels of free-market approach to networking. Most you are one of the large installers of dark ten-gigabits on that one strand of fiber. One networks today are designed and operated fiber like level 3 or Williams or Qwest, you company recently announced that they have through a “command economy” architec- will have a variety of solutions at different developed a Bragg grating that will support ture. levels of complexity and cost for your cus- up to 880 channels on a single strand of fi- tomers. Whereas if you are working with an ber. Although this may have been a “hero” Two Different Ways to older, more mature service provider like experiment in the lab, it does point to the Build Networks AT&T, MCI and Sprint, you will find that future direction of fiber capacity. you will have an obstacle in migrating your You have a single carrier with a large and own business in this direction? There are a number of ways to multiplex sometimes even global vertically-integrated optical channels on a single strand of fiber. infrastructure where the carrier is accus- St Arnaud: I think that it will always re- One is called a Bragg channel. Another is tomed to providing services at all levels of main a mixed market. I think that you will called an optical wave guide. There are a the protocol stack. Such a carrier does ev- have customers out there who will pay a pre- number of others. They are competing with erything. It deploys the fiber. It lights the mium to have “cradle to grave” managed each other in a search for the best ways to fiber. It adds SONET services to the fiber. It Network Services. These customers do not multiplex all these optical channels together. provides ATM services and on top of that want to manage their own wavelengths and adds E-Commerce and web hosting services. they don’t want to learn how to run Gig E COOK Report: How then would you pose Such an integrated a carrier finds itself run- over dark fiber. They are willing to pay a the current problem? We are having an ex- ning a large and very complicated bureau- premium to all this taken care of for them. plosion of bandwidth at the edges of the net- cracy and also finds that it must have a large These folk still form a large part of the mar- work. We are having enterprise networks and expensive management system to pro- ket and it will take a long time for these building their own private networks out of vide and integrate all of these complex and things to evolve. The market then will evolve gigabit Ethernet. How then would this one varied services and activities. Some people along with the carriers, some of which will group at the edge of the network communi- are beginning to ask whether these huge in- die and some of which will become success- cate with another group somewhere else on tegrated a vertical carriers can continue to ful. the edge of the network with a group that

104 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 Right now there are thousands of carriers kilometers of fiber comes to less than 2 mil- and a relatively small number of backbones worldwide and yet we only have perhaps 50 lion US dollars. This is done as a partner- facing a collision with the huge amount of different automobile companies worldwide. ship with five or six new carriers. By con- data being generated at the edges of the net- I think we will see a very significant aggre- tributing to the cost of laying the fiber, the work. How do we cope? gation of traditional backbone carriers in this outcome is that, at the end of the project, new marketplace. Those who adopt quickly each of these new carriers has access to a Internet Backbones: in order to take advantage of new technolo- network as powerful as that of the incum- gies and economies of scale will survive. bent telco. Changing Demands

COOK Report: In evaluating the carrier COOK Report: Who are these new carriers? St Arnaud: You’re asking precisely the right market what do you look for to distinguish questions. I think that the first generation of those who will survive? St Arnaud: They are Cogeco., Gateway Internet backbones were represented by Telecom, GTE Telcom, Dixon Fiber, Sprint, MCI, UUNET and GTE St Arnaud: Those carriers who partner with QuebecTel, Videotron and VDN. You wrote Internetworking. One might also refer to the research and education community I about VDN and its activities in Montreal in these as the first business model for Internet think will be the survivors. I believe that you your last issue. backbones. The second generation or Busi- will begin to see considerable innovation in ness Model is represented by Cogent Com- developing new network technologies and COOK Report: I have heard the thought munications whom you interviewed in your business models coming from the research expressed that in some of the research and September newsletter and I believe the third and education community. Five years ago education community in North America generation or business model is a back plane most people believed that the research and high-speed networking is seen more as a built from customer controlled lambda’s or education community’s role in the develop- means of soliciting subsidies to pay down wavelengths of light. ment of the Internet had diminished to the the campus cost of connection than as a way point of insignificance. The assumption was to develop these more innovative industrial For example, right now if I were a small ISP that the private sector would be successful partnerships. Are you willing to comment in upper New York state, I would usually in pushing the rollout of broadband Internet on this opinion? peer with an upstream provider or two that services to businesses, schools and homes. that had POPs close to your network. For a St Arnaud: Yes I think there is a mixture of fee, these large upstream providers would COOK Report: Why didn’t this happen? Are interest out there with some of the Univer- transit my traffic to the rest of the Internet. you in the research and education commu- sity CIO s playing a vital role to help their But if I am an ISP and I can control my own nity in possession of a flexibility that the respective communities to get advanced wave lengths, I have the possibility of by- private sector business market does not Internet services. Others are interested just passing or routing around a transit provider. have? in their own internal needs. Perhaps a uni- versity in a big city where there are many If ten percent of my traffic were going to St Arnaud: Perhaps because the university other universities doesn’t feel the need of and ISP in Hollywood California, it might research and education community with high being a strong community player. But oth- be cheaper for me to buy a wavelength of bandwidth demands can afford to be an early ers like the University of Indiana or the Uni- light and route around a traditional transit adopter of new technologies that production versity of Illinois have been playing a very supplier by taking my wavelength directly networks cannot afford to experiment with. strong role model in their respective com- to that ISPs border routers on a no cost peer- These institutions are now partnering with munities. ing basis. I think that the availability of these the private sector so that both communities wavelengths where you can peer directly win in the process of building next genera- COOK Report: Are you aware of the emer- with other smaller ISPs will allow you to tion dark fiber networks. gence of many new carriers in the U.S.? Of build what I call octopus networks. In such course we all know about Qwest, Level 3, a network you take the central part of your [Editor’s Note: On the NANOG list on Sep- Williams, Global Crossing and similar large network and then you extend all these ten- tember 1 Sean Doran answered a question players. But who else? tacles to other networks with whom you about Internet 2 (the U.S. next generation share significant amounts of traffic. If nec- network) by saying: “I’m sorry, but parts of St Arnaud: You certainly have many new essary such a tentacle could extend across my network [Global Telesystems] are AL- CLECs and DLECs. You have many more the entire continent and enable you to peer READY faster than Internet2’s network by of these in the U.S. than we have in Canada. directly with a like-minded network. As a factor of four. Other fully commercial net- I would keep my eye on a new carrier called wavelengths become cheaper and more works are also well beyond the technology PF.net. They have a new management team prevalent, at some point in the future it will that most academics brag about. The ONLY that I believe who truly “get it”. be cheaper to buy a wavelength rather than interesting large-scale academic networking the equivalent transit bandwidth. project is currently being done by COOK Report: And by DLECs you mean? CANARIE. “ We agree with Sean’s assess- COOK Report: That is rather fascinating. A ment. Bill St. Arnaud is the Director of the St Arnaud: A CLEC is a competitive local year or two ago if I were a regional network CANARIE Ca Net3 effort.] exchange carrier focused on the voice mar- in New England or the Northwest of the ket where a DLEC is the same thing only United States or somewhere else, the eco- Public Private focused on the data market. nomics of the business model would dictate that I should look for an exchange point into Partnerships COOK Report: What then is the implica- which I could bring my network. I would tion of all this growth in traffic and data at then peer there with as many similarly geo- St Arnaud: What we have seen in Canada the periphery of the network for what one graphically situated networks of my size as is that this partnership led by the university might call the first generation of the Internet would agree. My objective would be to do research community benefits everyone. In backbones that grew up and matured be- as much no cost peering as possible and drive Quebec, for example, the universities are tween 1995 and 1997? Given what is hap- down to the absolute minimum the amount building a fiber network some 3,500 kilo- pening now, you have the old hierarchical of traffic going into and out of my network meters long that will connect all of their in- question of who gets to peer with whom, that I would have to pay a transit provider to stitutions. The cost for installing the 3500 105 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA carry. multiplies and the prices those wavelengths ated very much like telecommunication net- comes down, ISPs will find that they are able works today - the same organization that What I think I hear you saying now is that to buy wavelengths from their networks to owns the infrastructure also controls the rout- there is technology developing that would dozens and perhaps even hundreds of ex- ing and scheduling of the trains. However, enable smaller regional ISP is to achieve change points some of which will be in other with highways the organization that builds these goals without taking leased lines from countries and on other continents halfway and maintains the roadbed has little control the borders of their network to several net- around the world. This is the first step. It’s over the routing and scheduling of the auto- work exchange points. Is this correct? And doable today and is beginning to happen as mobiles and trucks that use the highway. The if this could we explore how this technol- Williams and others are now selling wave- user makes the choice on what interchange ogy will work? lengths to ISPs. they want to get off and on the highway. St Arnaud: Yes. The current business model The next step is where OBGP comes in. COOK Report: This is very powerful stuff. is for ISP to go to 3 or 4 local exchange points Rather than buying a wavelength to a fixed Who is behind OBGP? Where is it being and then buy transit for the bulk of its traffic place like the PAIX in California one could developed? What needs to be done in order from one or more upstream ISPs. But if it buy a wavelength to an appropriate optical to finish it? can purchase affordable wavelengths, it can switch. Each optical switch in the path to connect to many more exchange points or the PAX, in fact, then becomes an exchange The OBGP Testbed by privately interconnecting and directly point. With OBGP any optical switch in a peering with many more ISPs. optical network can be treated as an internet St Arnaud: First of all I must emphasize exchange point such that autonomous ISPs that this is a very experimental concept. At COOK Report: Then this does not cut ex- can interconnect and peer with each other this stage it is high risk and there maybe change points out of the picture. It gives ISPs anywhere along the network - in essence a some kind of fundamental flaws that we many more options and very likely less ex- truly distributed IX. haven’t thought of yet. The background pensive options for interconnection with the event for the OBGP effort was the dark fi- rest of the Internet? Now imagine an ISP in California that wants ber build in Ottawa to the various universi- to peer with an ISP in Florida. Further imag- ties and research institutes. All these orga- St Arnaud: That is correct. Currently most ine that both of those ISPs own wavelengths nizations speak BGP in their networking en- commercial DWDM fibers have anywhere that travel to or through the same switch in vironment. Moreover they are usually multi from 32 to 40 and up to 80 wavelengths on Salt Lake City. With OBGP that ISP in Cali- homed. The only problem is that it is not them. DWDM systems supporting over 800 fornia, which owns a wavelength to the practical to build a total mesh of fiber to wavelengths are being tested and it seems switch in Salt Lake City, can see your wave- every institutional user. We have 26 differ- very likely that within a year or two, a single length coming in from Florida to that same ent sites and laying down a full mesh of fi- fiber will be able to support wavelengths switch. Consequently, you will be able to ber between the sides would require thou- numbering to well over a thousand. With route traffic to and from the California ISP’s sands of different fibers — something that such a proliferation, a wavelength should be wavelength without any intervening AS clearly is impractical. cheap enough so that an ISP in New York number or network control by a central au- could easily afford to take a high bandwidth thority. You will, in effect, have established For example we have dark fiber going from connection to the PAIX exchange in Cali- no cost peering with that ISP at the switch our offices here in downtown Ottawa out to fornia rather than purchasing a transit ser- in Salt Lake City . Furthermore, if another a research facility on the west end of the city vice from a major upstream provider. ISP in Colorado connects to that same where a number of fibers terminate. Also a switch, your can reroute your wavelengths number of fibers terminate in the east end COOK Report: What we are talking about and re-establish your peering session to the access point. Now I would like to be able to then presumably is optical BGP which you Colorado ISP instead of the Florida ISP. The route my traffic to any one of the institu- have proposed as an extension to the border beauty of this is that you don’t need a router tions to which I do not have a direct fiber gate way protocol. If so, please explain in as in Salt Lake City - only control of the opti- connection. One model of doing this is to much detail as possible how these extensions cal ports. But for all intents and purposes, to put a carrier in the middle. The carrier would will work. your peers, it looks like you have full fledged say “OK I’ll take care of it. I can build this layer 3 IX in Salt Lake City. cloud whether it be ATM or one that is made Optical Border Gateway out of wavelengths. I will route your traffic It is important to note that many of these and don’t you worry about it." Protocol concepts originated with ATM. What we are doing is stealing some of the best ideas out When it comes to optical networking the St Arnaud: the Optical Border Gateway of the ATM world and converting them to carriers can’t conceive of anyone besides Protocol (0BGP) is an experimental concept the optical layer. But instead of a traditional themselves owning the cloud and doing the which at this point is unproven. It is very “central managed” approach to networking routing for their customers. But we found radical and there may be many potholes that as is currently being planned for optical net- that owning the dark fiber ourselves seri- we encounter on the way to implementation. works, we are allowing the users at the edge ously changes our perspective on the possi- The first enabling step is dark fiber and the control the routing and management of the bilities of networking. We no longer want to availability of many dozens of wavelengths wavelengths. go back to the model where somebody has such that an ISP can then purchase its own to be in control in the middle in order to ag- wavelength and use it to connect to an ex- Now this has some profound consequences. gregate and redistribute traffic. Customer change point several thousand miles away If you let users at the edge control the rout- owned dark fiber has given us freedom. We if need be. At such an exchange point you ing and topology of the network that means like the freedom and independence of con- will be able to do standard peering. The ad- the carrier in the middle will be very limited trolling our own routing and never want to vantage given you by OBGP is that you can in how it can optimize and manage the wave- go back to having a carrier in the middle. get to that exchange point without having to length routing in the network. pay a transit costs to an upstream provider. We then realized that one way to avoid this The analogy I like to use is building rail- might be to have an optical switch in the As the number of available wavelengths ways versus highways. Railways are oper- middle and for each of us have access and 106 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 control of our own separate ports on that COOK Report: What then is an extension of pages. It is not a big radical new protocol switch. If you give each of institution con- to a protocol? Exactly how you create such but rather a very simple modification to an trol of ports on that switch, we can then de- a thing? existing one. The bigger challenge is in cide for ourselves how to route among us modifications to the routing software itself. and do so without the interference of any St Arnaud: In most protocols you have an In our proof of concept project we are using central organization. Now to look at how this options field. BGP has such a field and a public domain software. Once we have suc- might work in other situations, the dark fi- number of proprietary products have been cessfully completed the proof of concept ber supplier may be the one who installs the using that field for their own special pur- project, we certainly hope that Cisco and switch to be used by his customers. In such poses. We are proposing to use the options Juniper and others will modify their own a situation the fiber supplier provides the field in BGP to turn networking upside router operating system software to incor- electrical power for the switch, leaving the down. Today when you connect an ISP to porate our optical extension to BGP. control of the switches ports to his custom- an upstream ISP, the first thing they do is to ers. He says to his customers: “you cross install the physical fiber. Then they put in COOK Report: So much of what you need connect to whomever you want in that switch the link layer which can be ATM or SONET. to do is to get everyone to agree to an appro- and we don’t even want to have anything to Next they establish IP connectivity and fi- priate way to modify the options field in do with managing the switch expect provid- nally BGP connectivity. We are saying let BGP. Once you have done that Cisco and ing electrical power” us reverse that whole process. others will have to change their running soft- ware so that it will recognize optical exten- COOK Report: So the provision of these ser- So first we would establish BGP peering. I sion data from this particular options field? vices would be an attractive business model would say I want to peer with Gordon Cook. for a company like Williams? I have your AS number and to start a BGP St Arnaud: Yes. And then the commercial session where I instruct my router to con- vendors will also add their own set of bells St Arnaud: Yes as well as for Metro media nect us. It is the router then that establishes and whistles — management software and Fiber and for companies like Equinix. This the physical connection between us. Right things like that. They will have to see value is a service that would be natural for anyone here in Ottawa most of our institutions are in what we’ve done and have to be confi- running an exchange point to provide, par- going to put in either for four or eight wave- dant that there is a market for adding our ticularly for those who don’t want to func- lengths. They can have their wavelengths set extension into their software. tion as a full services provides. They just up to go to whichever universities they want the minimum services of the electrical choose. Then if someone says let’s set up a COOK Report: Why wouldn’t they think power and a secure place to house the switch BGP session with Gordon Cook, a router will there is a market for it? on which you depend. take one of those wavelengths and steer it towards a connection that will peer with your COOK Report: You are looking at 0BGP as network. OBGP Runs Against the a means by which to enable these kinds of Accepted Grain interconnections ? When you say you are COOK Report: Does bringing OBGP to uncertain of the feasibility at this point, what market depend on developing a common St Arnaud: Right now in the Optical Internet do you mean? What kind of obstacles might means of modifying the options field in Forum and in the ODSI working group the you run into? Might they be technology or BGP? And therefore you just need to de- traditional industry view is completely op- the expense of integration with existing soft- velop a common way of going about this so posed to this concept. The traditional equip- ware and hardware? that everyone operates with the same expec- ment suppliers are working on a completely tation. different strategy. In the traditional central- St Arnaud: As of today we don’t see any ized, carrier-centric view of the world, the such obstacles. But for every great idea that St Arnaud: Yes. The first thing we will do paradigm is to take the ATM model of net- succeeds, there are a hundred that fail. It is a proof of concept that will demonstrate working and replace the ATM virtual circuits could be market reasons, or business rea- these capabilities in the next couple of with wavelengths or lambda’s. In this model sons, or there could be a technology com- months. We will put together and test a very the carrier totally controls everything. ponent that we’ve so far overlooked. When simple version of the protocol. We already we build a prototype as proof of concept, have a number of industry partners involved COOK Report: this is what Francois Menard we will have a better feel of whether or not in the project who have indicated to us that meant when he said that Sycamore was go- it is really workable. they’re ready to take the concept commer- ing to do circuit switching in colors? cially and go with us into the IETF. COOK Report: Are you working on a pro- St Arnaud: That indeed is what they’re talk- totype in the Ottawa area or somewhere else? COOK Report: You will have “bof” and then ing about. There are two approaches to tra- form a working group to make this an offi- ditional optical networking. One is an over- St Arnaud: Exactly. I have the switch sit- cial IETF standard? lay model where you have a circuit switched ting right here in front of me. We hope to optical network and run IP on top of it. You have the first prototype in operation later this St Arnaud: That’s correct. But we don’t would do this like you run IP over ATM to- fall. want to actually start on standardization ef- day. The other approach is called a peer net- forts until we are certain that it is workable working approach where the wavelengths COOK Report: What do you have to do be- and that there are no unexpected “ gottchas“ are treated as MPLS Logical Switched Paths. tween now and then to get it working? You in the project. said that that is merely an extension to the COOK Report: What is the advantage of border gate way protocol. Is the only thing COOK Report: How complex is the final doing it this way? Is it being pushed by the you have to do to write some new code and product that we’re looking at? Is it a docu- carriers who have big bucks? add that code to working BGP software? ment of 10 pages or hundred and 50? St Arnaud: Yes it is something that the car- St Arnaud: Yes. That is why we are and St Arnaud: To anyone who is knowledge- riers who have big bucks are very comfort- 99.99 percent certain that it will work. able in BGP it is a relatively minor change able with. The big carrier market is a huge and could probably be done in only a couple market. If you want to sell to them and make a lot of money from doing so, you better 107 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA deliver the products they want. Therefore the fiber will really be the big factor in this and on a no cost basis. It will then expand from reason that the optical companies are focused not so much the protocol itself. In jurisdic- the university research community to large on this approach is that they expect to be tions where fiber is readily available like in commercial enterprises. selling to carriers for the next hundred years Chicago and Canada, you will see it happen just as they’ve sold to them for the last hun- relatively quickly. In other places it may take COOK Report: I have heard comments dred. a longer time. about GigaPops. Is this where someone at the University goes out and builds a re- COOK Report: So what then are you doing Cost Implications gional exchange point for that University and with all of this in Chicago? for local ISPs? Or perhaps will it be Equinix or some other commercial exchange point St Arnaud: We are working closely with COOK Report: When I first started talking to you two and a half years ago you were and the universities will come to that? And Joel Mambretti, Tom Defanti and Charlie if I’m an ISP in the local area, it would be to Catlett and other people at Internet 2 on these saying way back then that you saw two or- ders of magnitude in cost changes coming my advantage to connect and there as well. concepts and on the development of the Right? OBGP protocol. They are working on build- down the pike. in other words in the net- ing a dark fiber network in Chicago. The working power that you could buy for $100 in a new SONET based network you could St Arnaud: Yes. You would interconnect network will hopefully include the first op- your wavelengths there and try to have as tical network exchange where we will put buy today for a dollar in with the new IP over fiber networks. many direct peering sessions there as is pos- these types of protocols and technologies to sible in order to reduce your transit costs. use. They are working with our Canadian St Arnaud: Oh yes. No doubt about it. Just GigaPOPs and commercial exchange points industry and research partners on building seem to be the most logical places. the optical border gate way protocol. look at the Cogent business model as an example. COOK Report: If you look at carrier’s atti- Strategies to Cope With tude where they want to do circuit switch- COOK Report: Indeed on the level that you the Traffic ing in colors, do you think it safe to assume first enunciated it has been amply borne out. that this will happen? Now if you look at the new OBGP technol- ogy, do you think there’s any likelihood of COOK Report: OK. If you look at how back- St Arnaud: Some form of circuit switching anywhere as near a significant cost savings bones are being run now, you have a situa- will continue on its own for a considerable as was gained with the new network tech- tion where all the traffic generated by the period of time with perhaps much smaller nology after 1995. Could another order of explosion of gigabit and 10 gigabit Ethernet proportion of the telecommunications mar- magnitude of cost savings be added to the at the edges of the Internet, can perhaps not ket doing it this way. first two? even be sent through all the backbones that we have now. How are you going to cope COOK Report: At the same time, is it your St Arnaud: Yes and no. First of all if you with it? How are you going to get it from belief or hope that you can begin to grow use dark fiber you get a dramatic savings in one side of the network to the other? From a and develop the other approach dependent cost. For one time investment as little software and hardware technology point of on OBGP? Do you see some possibility of $10,000 you can get your institution hooked view is it feasible to think in terms of three the two approaches coexisting with each up with dark fiber that will last for 20 years. orders of magnitude or even an increase of other for some period of time? It has gotten very very cheap. The biggest four orders of magnitude in backbone speeds cost now for many of our institutions who needed to balance the delivery of the even St. Arnaud: Yes and I think the circuit- have dark fiber is the actual Internet access higher bandwidth generated at the edge of switched approach will likely be overwhelm- itself. Our biggest single cost is the purchase the net? ingly predominant for the next five to 10 of Internet transit service. Now if OBGP years. This is due to the huge legacy infra- works out as we hope it means we can sig- St Arnaud: Definitely one of the challenges structure and the huge critical mass of equip- nificantly reduce our transit costs. We will facing us is whether the core routers on the ment belonging to the existing carriers. We be able to peer directly with the networks of backbone will be able to keep up with all are at about the same development stage with our choice and be able to reduce our transit the increase in bandwidth at the edge. OBGP that the Internet was in 1975. Because traffic to say no more than 20% of the total Scalability is one of our biggest challenges. of the huge existing investment in the tradi- traffic rather than the 80 to 90% that it is One way to get around the bottleneck of the tional models of networking, I think that today. router is to do what is known as cut through OBGP will start with our university friends wavelength routing. If I have enough traffic across Canada and with our colleagues in COOK Report: I understand and this leads to one destination my upstream provider Chicago. It may evolve a little bit faster. I me to recall that while, on the one hand, we may take that traffic and put it on a wave- may be too pessimistic. But I don’t think it have a handful of extremely large ISPs like length of its own and send it directly to its will be mainstream for a number of years. AOL and Earthlink , on the other hand we appointed destination rather than try to still have probably four to five thousand packet process it in its local core router . COOK Report: What will be the critical de- small ISPs in North America. It seems to termining factors of progress? Economics? me that these small ISPs better hope for the COOK Report: This sounds like you are set- Inertia? Politics? What will be the critical success of OBGP which should enable them ting up a permanent virtual circuit but is what determinant of what you see as a rather slow to spend far less on transit and it remained you’re doing something different from what rollout and evolution? more cost competitive with those who are you see if you were using MPLS . far larger than they. St Arnaud: I think the speed of the evolu- St Arnaud: yes. It’s a matter of how is car- tion of deployment will be directly related St Arnaud: Indeed. I think that is a correct ried out. This cut through concept in both to the speed of the rollout of dark fiber. When assumption. What I think will be the first ATM and MPLS is well-established and universities and schools and other institu- thing to happen first is that a large univer- well-known. The cut through routing is done tions can get their hands on dark fiber, this sity with good networking skills sets, instead by the carrier inside its “cloud.” The cus- becomes a very appealing course to take. of spending all its money on transit services tomer is unaware of what’s going on. As far Therefore the speed of the rollout of dark will be able to peer directly with many ISPs as you know you are sending all your pack- 108 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 ets to a router and it’s the router that makes as in a LAN with standard Ethernet frames. of political economic issues could knock it the decision whether to cut through or not. You can transmit data via a wavelength. For off the track? But with OBGP you the customer gets to that option there is a slightly different form make the cut through decisions - not the car- of the ten gigabit protocol. Or you can map St Arnaud: Of course we believe that this rier. your data to a SONET frame. Nortel, Lu- is going to be the next biggest thing since cent, and Alcatel, the traditional telephony TCP and that it will revolutionize the indus- COOK Report: Does the existence of this oriented companies, are very strongly pro- try. But, in the past, many people have come OBGP technology create an opportunity for posing that ten gigabit Ethernet be mapped up with similar visions that have not deliv- companies like Equinix who are building to ten-gigabit SONET frames. Such a map- ered on their promise. Therefore I don’t want Internet exchange points to step forward and ping will obviously leverage their massive to be overly optimistic. We think it will have say to ISPs we have the kinds of optical market and investment in SONET. a major impact on the future of the network, switches at our exchange points that will but the reality is that many other people have enable you to implement OBGP and we can COOK Report: How do you do the SONET had similarly good ideas in the past that have assist you in learning how to accomplish framing? ended up in the trash heap. these implementations? St Arnaud: You take your 10 gigabit COOK Report: How much of the outcome St Arnaud: Yes we believe this is a very Ethernet frame and strip out the header and either in the U.S. and Canada depends on significant opportunity for organizations like all extraneous information and repackage the regulatory point of view that you take? Equinix or any of the other exchange point what is left into the SONET frame. Doesn’t the predominant regulatory point of or GigaPOP operators. They would provide view focus on a vertical approach where the that neutral site and electric power for the COOK Report: You do need a separate piece regulator looks at the relationship of huge switch and let their customers do their own of equipment to do this don’t you? And this vertically integrated companies to each other cross connects using OBGP. piece of equipment adds cost and complex- and permits the charging of one price for ity. voice and another price for data packets. The COOK Report: If you are looking at a back- question is does it make any sense to do this bone like UUNET are you really going to St Arnaud: Absolutely. There is a standard in view of the Internet? Or is continuing to have to assume now that such a backbone is called Ethernet over SONET (EOS). The big do this merely a carrier over from the regu- already a mesh of OC 48 and before long companies behind this are saying: let’s map latory world view of the legacy voice net- OC-192 permanent virtual circuits? many Ethernet channels onto one SONET works? channel. If we do this, we will benefit from St Arnaud: I think that is a fair assumption. aggregation because people will not be us- St Arnaud: This likely derives from the situ- For example they may own a fiber from New ing their Ethernet channels at full capacity. ation where a large percentage of voice is York to Chicago on which they will run ten By multiplexing them together we can gain still carried over switch voice networks OC-192s terminating on a router at each end. efficiencies into the use of a single SONET which are very much more expensive to run Now they could have another 10 wave- channel. But then you have to do buffering than packet networks are. When a large per- lengths on that same fiber and of those ten and queuing and essentially make yourself centage of all voice is moved to the packet five might go through Chicago terminating a router. Some people say why go with network, then you are absolutely right. At in Salt Lake City and the other five go Ethernet over SONET? Why not use packet that point there will be no difference in cost through Chicago and terminate in San Fran- over SONET? At the present time there is between delivering the two services. cisco. quite a debate going on over these choices. Say your are an ISP and that 20 percent of COOK Report: With the carriers then, you COOK Report: What would be a reason that your voice traffic stays on your IP network. will find almost exclusively SONET-based you do packet over SONET rather than But you also have to connect to the PSTN wavelengths because of their legacy of net- Ethernet over SONET? for the remaining 80 percent of your traffic. work equipment? The cost of this PSTN traffic will always be St Arnaud : Let’s assume that you’re carry- much higher to you, than the cost of voice St Arnaud: Yes. Almost exclusively. ing IP packets and that you are carrying traffic generated purely as Internet data. And Ethernet frames over SONET. These voice traffic for the next 50 years probably COOK Report: If I am a Cogent Communi- Ethernet frames are adding only complex- will always be more expensive than data traf- cations and all the pair of fibers I can do my ity and inefficiency. IP, in other words packet fic. backbone somewhat differently from over SONET, is much simpler. UUNET, can I not? COOK Report: Is it fair to assume that this COOK Report: Can you deliver then IP over will be influenced by a kind of regulatory St Arnaud: The Cogent backbone is prob- SONET to the edge of the network and right inertia? You assume that there is going to be ably very similar to UUNET’s. While they into the office building? a difference in cost between voice and data. both use SONET framing they do not use Because the cost is greater if you permit the SONET rings or SONET protection features. St Arnaud: Yes. But generally it is not done companies to charge more for the old way that way. Usually it is converted into an of doing things, you’re really subsidizing the COOK Report: But ten-gigabit Ethernet will Ethernet frame at the local POP and trans- continuation of that way of providing ser- include a wide area network framing as part mitted over dark fiber to the customer. As a vices. Are you not? of the standard. In about year or so when result some people say you might as well products using these standards are available, carry that Ethernet frame all the way across St Arnaud: Yes and no. The dramatic drop it should be possible to use 10 gigabit the network. in long distance voice costs have not really Ethernet framing in wide area network back- been caused by the Internet but have been bones? caused by competition. The carriers had to Politics of OBGP wring more savings out of their legacy net- St Arnaud: With the ten-gigabit standard works or incur losses. Traditional voice net- you have three ways that you can do data COOK Report: Let’s go back to the ques- working is actually very very hard to do. transmission. You can transmit to it native tion of OBGP. Assuming that this will work Real time delivery of the human voice re- from the technology side of things, what kind 109 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA quires the addition of all sorts of quality con- of the extremely stringent reliability require- Internet and the only uncertainty is how rapid trol mechanisms to the network to avoid such ments imposed upon their voice network the migration will be. things as echo and jitter. Building a good functionality by regulators. Because of the network for voice is very difficult. insistence on the traditional “ five nines “ of St Arnaud: Yes, and I can see the two net- reliability, the traditional telephone compa- works existing in parallel for 50 years. Many companies announced that they will nies can take up to five years to evaluate a do voice-over IP and totally eliminate the new technology. Their time scales are on COOK Report: Yes. But 10 years from now telephone company. . Voice turns out to do entirely different order of magnitude than I can imagine that we might be paying $20 very difficult to do especially when you have those of the Internet where we are swapping a month for our always on, five nines reli- to interface with the traditional public out boxes every six months. able telephone service and $80 a month for switched telephone network with its SS 7 a suite of Internet data services including signaling. We simply cannot ignore the COOK Report: Is there a kind of the institu- voice-over IP, e-mail and Web access, and legacy PSTN, in building a voice-over IP tional dilemma and the setting a regulatory access to high-speed video and audio. networks. Until the last telephone and the policy? Can you imagine a situation where last country in the world is switched over to if we had regulatory leadership with a “net St Arnaud: Perhaps. Except I see these two IP, we will still have to interface with the headed“ disposition and we could begin to figures reversed. The suite of Internet ser- old legacy system. This need to interface promote the acceptability of a world where vices may be $20 a month and the legacy, with the legacy PSTN means that we have if you’re IP over fiber service goes out you but highly reliable, phone services $80 a to have complex of protocols and signaling would have wireless as an alternative re- month because Internet data services will and things like that which drives up the cost placement for the time of the outage. Or will benefit greatly from economies of scale. of delivering voice whether it is over IP or everything be governed by the inertia of the the PSTN. legacy networks where those people will COOK Report: Well that is an intriguing make it impossible for such a radical change thought because, if voice services are that COOK Report: If you look at voice over IP of priorities to take place? expensive, it will add to the pressure to mi- as run on large corporate virtual private net- grate away from them to the Internet data works, can you get quality in those environ- St Arnaud: I’m afraid it will be the latter services. ments equal to the quality you get with a with incumbent IXCs and ILECs trying to very different and more expensive equip- milk all possible profit out of their legacy St Arnaud: We think that there will be three ment on the public switched telephone net- infrastructure. parallel networks in existence. In fact we work? would suggest that policy not to try to con- COOK Report: Do you think there’s any verge all telecommunications into IP because St Arnaud: Absolutely. This is happening possibility that some say three to five years doing so will increase the cost of IP. The today within many corporate private net- from now someone could begin to make the residential telephone market will continue works. Voice in the networks belonging to argument that because of the ubiquity of to provide delivery of telephone by twisted large corporations is being packetized and various telecommunications systems that pair. There is nothing wrong with a voice- moved rapidly into IP Transport. you could rely on the ability to switch over over copper which in fact works very well. momentarily from one system to another in A second network may well be broadcast COOK Report: Say then that you have in- case of an outage? Could this be justifica- video over coax which again works very dependent parallel networks — voice con- tion for the promotion of regulatory policy well. There is no really compelling reason tinuing on the legacy public switched tele- in favor of doing just this rather than taking to deliver broadcast video over IP. I believe phone network and a voice-over IP on cor- another 20 years to write off the investments there will be a third network dedicated to porate networks. You cannot easily go from in legacy SONET equipment? only IP and this network can under some corporate network to the public network? circumstances carry voice or video or both. St Arnaud: Increasingly they do. I’m sure St Arnaud: That is the problem. Going be- the different approaches to these problems Your primary mechanism for receiving tween a voice over IP network and a circuit will be developed. However the other prob- broadcast video will be either an antenna or switched networks is very difficult. This is lem that traditional telephone companies coax . The primary mechanism for voice will especially the case with the big carrier have is traditional customer expectations. be either a cell phone or your current cop- switched networks which have extremely They expect their phones to work all the per phone. IP will include a voice-over IP stringent demands as a result of their SS 7 time. With 99.999% availability your phone and other value-added services but for the signaling. Carrier grade voice-over IP is a is almost always going to be up and ready to next 50 years I am don’t see it as entirely long way off and it is going to be quite some reach anyone anywhere at anytime. In the displacing the other two networks. time before it becomes a cost-effective so- Internet we have a different set of expecta- lution. tions and ways of evaluating reliability. Go- Fiber to the Home ing from the culture in the telephone com- panies to the world of the Internet is going Regulatory Issues to take a long time. But focusing on a voice COOK Report: What do you see as the eco- is focusing on the wrong problem. I say for- nomics of bringing fiber into the home ? You COOK Report: When American and Euro- get voice because voice will be a niche mar- have talked about bringing gigabit data ser- pean carriers buy new equipment for their ket within less than three years. vices to every home in Canada by 2005. networks, to what extent to do they buy the What will be the dynamics that drive what newer technology of the IP networks or are COOK Report: So one might think of the only a few years ago most people would they almost forced into a continuing to buy future as a group of parallel telecommuni- have considered an unlikely outcome? Yet the older technology that is compatible only cations services. Hear you would have you now see it as possible and Worldwide with their legacy circuit switched networks legacy voice services going forward into the Packets has built its entire business model ? future, but you also have the Internet data on this as a premise. What will drive this? services and voice-over Internet data ser- St Arnaud: They buy some equipment de- vices . you will have a migration of people St Arnaud: This is our greatest challenge signed to tie the to networks together but from the legacy voice services toward the and we believe this is where the research they’ve been rather slow to do so because and education community in partnership 110 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 with the government and carriers can really the other path to scaling be the one of taking proach is ours. In this we say let BGP be the help to accelerate the deploying fiber to the light paths to exchange points and use of the controlling mechanism and let decisions be home. For example. In Quebec a number of optical border gate way protocol? made more upon the lines of traditional municipal governments and school boards Internet direct peering relationships. have gotten together and put fiber into ev- St Arnaud: It’s hard to say. OBGP is too ery public sector building for a one time cost immature at this point make a difference. COOK Report: Would you clarify the dis- as little as $10,000 per building. This is a We like to think that in a few years it might tinction that you are making between MPLS one time charge for the next 20 years. If these be the dominant protocol. But that remains and circuit switched? municipalities put in a dozen extra strands to be seen. Over the next couple of years I to the school, it becomes very easy for the think that MPLS will be the dominant means St Arnaud: Traditional circuit switching is private sector at minimal cost to run fiber to of traffic engineering backbone networks. tantamount to running IP over ATM. that’s individual homes from a node at the school. the ODSI approach. The wavelengths are We are looking at the feasibility of partner- COOK Report: is a third possibility the con- like ATM virtual circuits and they are ships to get public-sector buildings fiber tinued scaling of large point to point back- switched all the time all over the place. In connected in a way that will save them bones? the ODSI model switching has merely been money. Once we do this, if we do it right, it moved down to the optical level and the IP is relatively easy for the private sector to St Arnaud: There are probably three possi- layer is unaware of topology changes in the extend that fiber to individual homes. bilities. I am not a fan of the MPLS approach. network. The MPLS way of doing things is MPLS was a technique designed to cope the cut-through model. In the ODSI world COOK Report: Do your politicians under- with shortage of bandwidth. Now with the if something breaks, the optical plane takes stand the importance of getting this technol- ability to buy wavelengths of light on fiber care of it. In the MPLS world the MPLS ogy installed as infrastructure? bandwidth is not an issue. Consequently router has more responsibility to do the cut micro engineering the network is probably throughs. St Arnaud: Yes, Increasingly they do so. not necessary today. Also it does not look like we can build routers big enough to ag- COOK Report: In other words does your COOK Report: if you survey the scene in gregate all the traffic. We will require opti- routing take place at layer two or layer three? both the U.S. and Canada from the point of cal “cut thru” or “bypass” circuits. The chal- view of viable business models for getting lenge that faces is who controls the cut St Arnaud: That’s right. In OBGP is entirely fiber to the home, what do you see as viable through circuits - the carrier in the middle a layer 3 or even a layer four model in which approaches and who are the major players? or the customer at the edge. the BGP itself actually controls the set up of wavelengths. Arnaud: At the present this a small market, Now, doing it this way is not building a tra- but it is growing rapidly. There are a num- ditional circuit switched model. Circuit COOK Report: And from a structural point ber of companies in the area. Homefiber is a switched models imply that for every flow if you, if what you’re running is going to company doing it in the Palo Alto area and you must start a new circuit. The service will take place at layer two, does that equate to SBC is doing it in the Atlanta area. It is pretty set up a switched circuit and send a web page the only place where scaling is possible be- well a slam-dunk to put fiber in new hous- to you, and then switch to another circuit ing the very large telco kind of model? ing developments, especially if you have to and send a web page to someone else. This build new roads and trench new sewer lines. will not scale. Having direct peering with St Arnaud: OBGP is a model that we be- The challenge is to get it into existing homes. each other with wavelengths and doing by- lieve can scale the most because all the con- In the Western US, WIN is a company that pass around other ones may achieve the same trol is at the edge of the network. And why? is doing it as well as several public utilities ends. Because each small group of people at the who are now building a fiber networks to edge the network is able to come to their existing homes and who have developed COOK Report: so the 50 to one hundred own decisions and implement or change compelling business cases showing that this largest carriers of the world will be going to these decisions very quickly. The other two can be done. a large number of exchange points and these models assume the existence of a huge cen- carriers might develop a model of relying tral carrier in the middle. If at some point in COOK Report: Who are the experts in this upon cut through circuits to reach each of the future you have millions and then per- area? the 50 to one hundred other carriers from haps even billions of wavelengths that have the exchange points. Yes? to be centrally controlled and monitored how St Arnaud: World Wide Packets is one of this could possibly scale? the leading companies in this area. Another St Arnaud: That is correct. effort worth looking at is Grant County in COOK Report: The see the MPLS model as Washington state where the state govern- COOK Report: And then the third way, if it the one that can transition to the control at ment leadership was critical. Washington works, would be the optical border gate way the edges model of the optical border gate State passed a law encouraging utilities to protocol. way protocol? Or do you see it as just an- get into the fiber to the home marketplace. other way of keeping central control? But they also wisely said we don’t want you to be telephone companies. We just want you Three Models of St Arnaud: I think the intention is to rely to install dark fiber and sell access to that. Networking on it indefinitely. After all their are very few people at the present time who know any- COOK Report: if we look at the issue of St Arnaud: That’s right. So right now, in thing about OBGP. If you are in Nethead scaling of Internet backbones needed to deal the standards bodies, you have three ap- from the Internet World, the MPLS model with gigabit traffic generated at the edge, proaches. One is called the overlay and this of cut through is probably the one that is perhaps we can see the existence of two is basically the circuit switched model. This most prevalent and popular in the IETF. If paths? One path relies upon a mesh of per- is being promoted by under the label of you are a Bellhead from the telephony world manent virtual circuits and MPLS and is ODSI. Another one is called peer network- the ODSI model of an overlay is more popu- sometimes referred to as the path preferred ing. This is where all the wavelengths are lar. by UUNET and Mike O’Dell . Now would treated like MPLS tunnels. The third ap- 111 COOK Network Consultants, 431 Greenway Ave, Ewing, NJ 08618 USA COOK Report: So in this sense your tech- nology out look is determined by the tech- nology origin of its investors? Since it looks Some Optical like these will be competing with each other for a while can we ask if there is any evi- Networking News dence for a distinction in the cost effective- ness of the two different approaches? Does the first one depend on SONET where is the Alcatel Transmits Five cut through MPLS model doesn’t? Terabits on Single Fiber St Arnaud: Yes and no to both. It is just that both assume a large organization exerting [From an Alcatel press release Sept 9] central control. OBGP assumes that you Alcatel sets a new world record for DWDM have dark fiber and that there is no central backbone networks at 5.12Tbit/s and paves control organization. the way towards 10Tbit/s transmission on a single fiber Paris, 7 September 2000- Alcatel, COOK Report: Is there any reason is that the world leader in optical networking, to- you can see over the period of the next few day announced that it has set a new world years where the cut through central control record for DWDM (Dense Wavelength Di- model might become dominant over the vision Multiplexing) transmission capacity OSDI circuit switched model? Or do you over backbone networks. The company has think that for the next several years those achieved a unidirectional transmission of two models will coexist and keep similar 128 channels each modulated at 40Gbit/s, market shares? reaching a total throughput of 5.12Tbit/s (5,120Gbit/s) over 300 km. This result was St Arnaud: It’s difficult to say because it is achieved with three technologies developed so new and still untested. But in my personal by Alcatel : 40Gbit/s DWDM systems, an opinion MPLS will hold a far larger market innovating technique to make the most of share because it is more in line with the existing bandwidth, and TeraLightTM opti- Internet view of the world, while ODSI is cal fibers. This demonstration once again more in line with the traditional carrier view highlights the technological leadership of of the world. Alcatel in 40Gbit/s DWDM systems as well as the unequalled potential of the commer- COOK Report: To conclude — if you were cially available TeraLightTM fiber for 2.5, to offer any advice someone who is making 10 and 40Gbit/s transmission. decisions about acquiring telecommunica- tions infrastructure in their community or to This breakthrough transmission would al- someone who is making investment ses- low for the transport of 640,000 simulta- sions, what would your advice be? neous high-bit rate ADSL Internet lines or over 100 million simultaneous voice calls St Arnaud: Number one advice. Wherever on a single optical fiber. It will allow opera- you are. Get dark fiber. Doing so changes tors and service providers to dramatically your whole outlook on life. it puts you in increase in the future the capacity transported control and will start to open your mind to over a single fiber, thus further optimizing all sorts of new ideas and concepts. Imagine the cost of their networks. a world before electricity. If you never had electricity and I am trying to explain to you To achieve such a performance, Alcatel has the concept of what a blender does, you sim- developed a new technique to optimally use ply can’t understand. But when you have the bandwidth of optical amplifiers. When electricity your whole world starts expand- carried over an optical fiber, wavelengths ing. And until you get dark fiber you really generate interferences. It is therefore neces- cannot adequately grasp the possibilities of sary to have a minimum spacing between OBGP. each wavelength in the networks, which lim- its the number of wavelengths that can be November 2000 COOK Report transported over the same fiber. The higher the bit rate, the stronger the interferences and the more limited the number of wavelengths. The innovating technique developed by Alcatel allows to minimize the effect of these interferences and to pack an unprecedented number of DWDM 40Gbit/s channels on a single fiber, thus reaching the world record capacity of 5.12Tbit/s. This laboratory dem- onstration also exploited Alcatel's leading- edge NZ-DSF optical fiber, TeraLightTM, which offers an optimized design for DWDM systems. This result, achieved in

112 The COOK Report on Internet Light, IP, and Gig E - Annual Report February 2001 when 14 months ago it paid over 8 million offer voice, data, and television services. realistic conditions with in-line amplifiers, dollars for a 25 year IRU on a trans Atlantic The small, tightly knit consortium will ini- underlines the Alcatel strategy to optimize STM-1 only to see the price of such an IRU tially offer digital voice, TV, and Web host- all the key technologies required to offer plummet to less than 3 million a year later.” ing over fiber, under the name SpectraDyne cost-effective, advanced optical networks to We asked Above Net’s CTO Dave Rand Services. The team includes Sierra Pacific cope with the Internet explosion. whether we had errored in our description Power, Hewlett-Packard, Oracle and in any way. TelecommUnity Systems. Sycamore's ODSI in Major Rand replied: “You are correct, but we made This alliance will be the first to connect and Setback our money back by being one of the first interact with all consumers, retailers, medi- with large capacity. Time is everything, that cal facilities, libraries, schools, industrial [From: Light Reading] DENVER -- Ef- capacity got us customers worth far more companies, local merchants, and local gov- forts by Sycamore Networks Inc. (Nasdaq: than the apparent ‘loss’ of $5 million.” ernment over computers, telephone, or tele- SCMR) to gather industry support for its way vision in their coverage area, said a senior of automating tomorrow's telecom networks “One of the major advantages of fiber is that executive at TelecommUnity Systems. The has suffered a significant setback. the technology is future-proof. Each addi- initial rollout of services in southern Nevada tional DS3 costs an average of $3000 today. is set for this summer with projects and dis- Its way of doing things was rejected by other Using our definition of a broadband user cussions under way between the consortium vendors at an August 17th meeting of the (using more than 3 DS3’s), fiber is always and at least 30 other utilities.” Optical Internetworking Forum (OIF). And more cost efficient. As an example, a com- that's dented Sycamore's prospects of being pany using 4 DS3’s would be paying around On February 4 according to the Washington first to market with standards-based software $12,000 per month to telcos for the capac- Post Ford Motor Co had announced that it that enables edge equipment like routers to ity. Maybe they would get a deal on the 5th was going to provide every single one of its set up and tear down connections on demand one and get it for $2500/mo. But it’s $2500 350,000 employees with a home PC and over optical backbones. per month, forever. color printer — plus unlimited Internet ac- cess for $5 a month. See http:// It might also mark the beginning of the end To provision the same capacity over dark search.washingtonpost.com/wp-srv/WPlate/ for the Optical Domain Service Interconnect fiber might cost $10,000/mo for the fiber. ut 2000-02/04/095l-020400-idx.html coalition (ODSI), a Sycamore-led group of you still need to add equipment. Today, vendors that was set up earlier this year to equipment costs around $30,000 per end. On the 5th Of February Delta Airlines an- try and speed up the development of stan- Amortized over 5 years, that might be a nounced PC s and access for its 72,000 em- dards in this area. It was ODSI's proposals $1000/mo. Toss in $250 per month for main- ployees for $12 a month. See http:// that got rejected at the recent OIF meeting, tenance - now you are at $11,250/mo for washingtonpost.com/wp-dyn/business/ and that setback has led to some members those same four circuits. Already you are A13291-2000Feb5.html Internet service is questioning whether it's worth carrying on. saving money. But that’s not the point. Now, on the way to becoming like telephone ser- when you need another DS3, add $2000 vice in accessibility and usage. News of these developments have only ma- worth of cards to the existing equipment, and terialized in the past day or so, when the OIF you have another DS3. Perhaps $33 per March 2000 COOK Report got around to publishing a press release month, amortized over 5 years. And you can about its meeting, which was held in *keep* adding them. So it’s $2500 per Barcelona. month, or a $2000 one-time-charge. For the full story, please go to: ‘But wait - bandwidth pricing is falling’. Yes, http://www.lightreading.com/ we know. In fact, we are *enabling* the price document.asp?doc_id54 to go down. So, let’s say that there is a price decrease next yet, and the price of bandwidth November 2000 COOK Report drops by 67% - from $3000 per DS3 to $1000 per DS3. That same 4-DS3 user is now a 12-DS3 user. Still paying $12,000 per month. But for fiber, it’s just another 8 cards, $16,000, and you are *still* only at $11,516/ Explosion in mo (amortized over 5 years - its less if you do 10 years, and our average fiber contract Capacity Chased is 12 to 14 years in length). And, if you need more capacity, you can use DWDM avail- by Explosion in able today - which allows 6144 DS3’s to ride on that same $10,000 worth of fiber. Soon, Use so the various manufactures tell us, they will be able to increase even that threefold.” Fiber to the Home from HP Oracle and Power Certainly new uses for the fiber are pooping up daily. On January 24, Ephraim Schwartz, Companies For Less than of InfoWorld http://www.pcworld.com/ $15 a Month -- Abovenet on pcwtoday/article/0,1510,14909,00.html wrote the Need to Own Fiber “This summer, a new nationwide network Recently we wrote: “It is a buyer’s market created in secret by an alliance of utility com- but a dangerous one as AboveNet found out panies and computer industry giants will

113