House of Commons Public Accounts Committee
BBC Digital Media Initiative
Written Evidence
Only those submissions written specifically for the Committee and accepted by the Committee as evidence for the BBC Digital Media Initiative inquiry are included.
List of written evidence
• Bill Garrett, former Head of Technology, BBC Vision Productions.
• John Linwood, former Chief Technology Officer, BBC
BBC’s management of its Digital Media Initiative Written Evidence – William Peter Garrett - “Bill Garrett”
In summary of my submission to the committee;-
• I find it difficult not to conclude that the National Audit Office was mislead during their previous 2010/11 investigation specifically with reference to the progress of DMI after the project was brought in-house by the BBC. [paragraph 16, 17]
• I consider evidence given by the corporation to the committee hearing of the 15th February 2011 misleading and did not portray the true performance of the DMI project. [paragraph 21, appendix 090]
• I believe the revised investment case (sometimes referred to as the “benefits case” or “business case”) of April/May 2010 which sought funds to continue DMI’s development in-house was a fundamentally flawed proposition consequently placing public funds at significant risk. I believe a number of staff knowingly falsified estimates of financial benefits to ensure case approval. [paragraph 8,9,10, appendix 040,042]
• I raised concerns about various aspects of DMI in 2010 and again in 2012. [paragraph 5,6,7,14,16 appendix 024,031,084]
• By virtue of its narrow remit I believe the PwC investigation has not been permitted to fully answer the fundamental question “What went wrong?”
• I have provided additional documents as evidence to support assertions within this statement (attached as an appendix).
As the culture surrounding DMI and the behaviours of individuals involved are key to understanding the events which I believe lead to the failure of the project this written evidence is provided in the form of a narrative witness statement.
Introduction
1. From 2006 to 2010 I was Head of Technology BBC Vision Productions, a department of the BBC sometimes referred to as the corporation’s television programme making arm. Prior to 2006 I was an investigative journalist for the BBC working in business programming, I specialised in commercial misconduct, corporate finance and technology transformation. I am a professional engineer with significant experience in the broadcast industry. I am the author of the letter to Lord Patten of Barnes – Chairman of the BBC in May 2012 in which I raised a number of concerns about the corporation’s management of its Digital Media Initiative.
2. In January 2007 I had interaction with a team of BBC staff and hired consultants from Siemens and Deloitte as they prepared a proposal for DMI. This initiated a work stream of consultation with the production divisions of the corporation, effectively a requirements gathering exercise. It indicated the need for a "work in progress" asset management system for TV production and a digital "archive" for completed master recordings. The subsequent requirements document formed part of a business case for DMI submitted to the Director General’s Finance Committee in Q2 calendar year 2007.
3. Throughout the following two year period I had regular contact with the DMI team during their attempts to deliver a working system within the Siemens delivery contract. I represented the interests of the production communities and regularly liaised between senior production figures and DMI management. Within a year it became obvious that DMI was in significant difficulty as it continued to miss delivery commitments. By the end of 2008 the credibility of the project was very low and it became increasingly challenging to explain its failings to senior production figures who relied on my technical understanding. By early 2009 I was aware that problems with the project were causing tensions between the corporation and its technology partner Siemens.
Siemens contract terminated
4. In the autumn of 2009 in a conversation with a senior technology manager who reported to John Linwood, I describe the decision to tear up the contract with Siemens as a "reckless act" adding "They (DMI team) will probably never be able to deliver the project now". I qualify my comments indicating that a key technical challenge and therefore risk for DMI was the capability of the BBC's computer network to support the system. I indicated that as Siemens operated and managed this network for the BBC, Siemens had effectively underwritten this risk by also being the contractor who had committed to deliver DMI. By tearing up the contract and bringing the project "in-House" the BBC took on this risk. The individual indicated that he understood this "dynamic".
5. In January 2010 I am asked to provide my subject matter expertise for a newly prepared benefits case for DMI. During a meeting I raise a question whether taking specific component parts of the DMI technology in isolation could more effectively deliver benefits. I highlight that this approach could also reduce project delivery risks by focusing in on one or two components. It was agreed between myself and a senior finance manager in BBC Vision that I should seek to interview the delivery leads of DMI to ascertain whether this would be an opportunity. It was also suggested that I raise some additional questions.
6. In the following days I meet with some members of the DMI leadership team for a Q & A session. I conduct the interview and take notes indicating that I will provide a copy of the notes back to the interviewees so they have an opportunity to moderate the recorded responses. Although the meeting is cordial and friendly I detect a note of frustration towards some of my latter questions particularly where finances of the project are questioned. Specifically one of the team becomes animated when I raise questions about DMI funding for the Salford Quays project (sometimes referred to as “Project North”/“North”), he indicates that the questions "were not for this forum". [page 024-026]
7. I send my notes and the moderated responses to senior staff of the Vision and Technology divisions setting out a number of my concerns. As these responses had been moderated by the DMI team I considered these to be an accurate reflection of their view. I felt the clarity of the responses were not of a standard that informed me satisfactorily about the future DMI proposition. They raised many questions about the funding for DMI specifically wrt to Project North where there appeared to be a significant gap in operational expenses, additionally a critical risk associated with the BBC’s computer network had not been mitigated or addressed. During various discussions with fellow senior colleagues within Vision Productions and Children's television I repeated my concerns that the finances for the project looked unclear and that I believed there to be a significant number of questions around the technology yet to be resolved. [page 024, 031-034]
Preparation of the 2010 revised Benefits Case
8. I am invited to take part in a meeting to further address the benefits case for DMI. The proposed meeting is organised to consider a new benefit line item – “cost avoidance”. At the meeting we discuss what costs production divisions would be exposed to if DMI did not go ahead. These consisted of estimated costs of standalone data storage systems and their associated support costs, these have subsequently been labelled “digital islands”. Myself and executives from the relevant production divisions did some rough estimates of costs, our calculations totalled approximately £7 Million for the project lifespan. All agreed that these numbers were based on rough workings however some rationale had been applied. The meeting concluded with these cost avoidance figures being the only numbers discussed during the two hour session.
9. The following day I am forwarded an email, it is a copy of a summary briefing of the cost avoidance meeting [page 040,042]. Despite the importance of the email and that it requested feedback I had not received this directly. It was provided to me by one of the production staff who also attended the meeting and who wished to draw my attention to it. Attached was a spread sheet which purported to be a recording of the figures discussed, however there were three further columns detailing additional cost avoidance figures I had not previously seen (marked as columns A, B and C in appendix document 042). These additional columns totalled £8.4Million. Although the sheet indicated that these figures had been produced based on information gathered during the meeting, these columns had not featured in our discussions. I became concerned about the veracity of these numbers as I had not had opportunity to clarify them nor was I aware that any of my production colleagues had had a similar opportunity. One of the cost items was "Datacentre & Storage Management costs" and equalled £3 Million. This line item alone defied reason as this cost would only occur if DMI went ahead, not a cost avoided if it didn’t. It is my belief that the reason I was not recorded as having attended this meeting or been on the original distribution list of the email and summary document was because the authors would have known I would consider the additional £8.4Million unjustified. In looking at these costs it was clear to me that the numbers had simply been made up.
10. Sometime later in February 2010 I have a brief meeting with a Vision Productions finance colleague. They wished to clarify their understanding of some of the benefits and cost avoidance of DMI. I detect an element of concern in their own view and they show me a document which I believe to have been a section of the revised business case. It highlights the cost avoidance numbers, these assert the costs avoided if DMI goes ahead to be £15Million, it uses words to the effect "these figures are ratified by the CTO of the BBC". I indicate my immediate concern that these were unjustified figures in my view. I describe the numbers as "like Enron" referring to the approach of adjusting estimated figures to suit needs rather than being based on pragmatic rationale. I felt qualified to draw this comparison as a few years earlier I had made a film about Enron for the BBC in which many of the interviews with former Enron executives described similar behaviours in the preparation of business cases for the failed energy company.
11. Around the time of the preparation of the revised business case a workshop was held between Project North and the DMI, it aimed to enlighten Project North about the technology solution from DMI. During the workshop 36 questions were raised by Project North members and subsequently submitted in written form to the DMI team. Answers to these were critical for North to progress with its own project. During the meeting I am offered a demonstration of a key piece of DMI functionality which I email to confirm my interest. I and many colleagues considered the performance of the DMI representatives during the workshop as chaotic and without a clear understanding of their technology solution. A number of colleagues describe the meeting as “farcical”. Later I read an article in industry press magazine TVB Europe in which Mr Linwood explains the DMI would be based on “Cloud” technology. [page 035-036] This brought into question the project’s entire approach and was, in my view a high risk solution at the time.
12. During February 2010 I receive an email from a member of the DMI project team withdrawing the previously made offer of a demonstration of a piece of system functionality. I forward on the email to relevant stakeholders adding the words "It just gets better and better! Files based import not ready!" [page 049]
13. In the following days, I receive an email from a senior figure in BBC management who had been, or possibly still was, the creative lead for DMI. [page 050] In it he raises matters of communications with the DMI team and in particular requests for demonstrations of technology. At the time I take his communication to be referring to my email (17/02/2010). He sets out a protocol for all future communication with the DMI team. He appears to imply that any other approaches were at risk of being deemed insensitive to priorities, having a disrespectful tone and creating noise. He writes "These are our colleagues, not a third party we are testing for credibility." I am shocked by his statement, it was crucial to determine the credibility of DMI ahead of its business case application for further funding. I cannot speak for the mindedness of this individual when he wrote this particular email however the net effect of his communication was to inhibit the freedom of concerns being raised by those it was directed at. A production colleague commented to me “I think we’re being told to shut up”.
14. Towards the end of February 2010 In a routine meeting with a senior Vision Productions colleague and a senior technology manager who reported to John Linwood. I raise my concerns about the benefits case for DMI directly with the technology manager, specifically the subject of the cost avoidance figures. He immediately becomes extremely agitated, he doesn't attempt to address the concerns but starts raising concerns he has about another unrelated matter discussed earlier in our meeting. I try to refer back to the issue but he ignores the point again drawing our focus to an irrelevant matter. The meeting concludes and myself and my Vision Productions colleague walk back to the Vision Productions management office where she refers to his behaviour using words to the effect "He's just not objective"
15. During March and April 2010 there were various levels of contact for me and other stakeholders with the DMI team. During this time I did not receive any material answers to the 36 questions that had been minuted and submitted as part of the DMI workshop held on the 17/02/2010. In the main I and many BBC colleagues remained unconvinced by the credibility of the DMI team to deliver on their assertions, particularly as they appeared not to be meeting their deadlines, nor could they provide any particular insight into their proposed technology solution. Specifically during this time I was unable to ascertain exactly what functionality was going to be delivered in "Release 2 – production tools" that the project proposed would occur in May 2010.
16. Towards the end of April 2010 I engage in dialogue with a colleague who was a senior technology manager who I understood to have recently assumed some oversight of the DMI project. I forwarded him copies of various matters including the mostly unanswered 36 questions raised with the DMI team in the previous months. I particularly raise questions over the claim that the “Archive” functionality for DMI had been delivered in February 2010 [page 082,084]. During my dialogue with the individual it became clear to me that he shared my concerns about the project. At around the same time I write to the DMI team about the Archive solution and other matters but did not get a response [page 085]. I casually meet a member of the DMI team who I had a good working relationship with who admits that he considered the claim that Archive solution had been delivered as questionable.
17. In early May 2010 it became clear to myself and stakeholders in BBC Children's that DMI would not deliver the proposed release in May/June 2010. Children’s television was the only area in the production community slated to have a delivery at this proposed release. For the removal of doubt no release was made to anywhere in Vision Productions or Children's during this period. The Natural History Unit was expecting DMI to be ready for the series “Frozen Planet” which this was due in post- production. The delay by DMI was likely to cause production issues, as a result a stand-alone (digital island) solution was put in place. This was not DMI connected to any archive functionality however it was funded by the DMI project.
Following my resignation from the BBC
18. (04/06/2010) I leave the BBC, in the months following I remain in close contact with many key stakeholders who confirm that no DMI functionality had been delivered.
19. (July 2010) I read an article in trade paper Broadcast, it is an interview with John Linwood in which he indicates that DMI is "live and in production and being used in the BBC". I know John Linwood's statement to be misleading. [page 087-089]
20. (Jan 2011) I read the NAO report and associated press coverage. The press coverage appears to focus on lost benefits from the project during its time under the Siemens contract. The NAO report states the deliveries of DMI that John Linwood had claimed in the press article. Again I know these to be misleading and I am amazed that these are recorded in the NAO findings. I consider the report does not reflect what I know about the project from the point when it was brought in house by the BBC
21. (June 2011) I read a press article about the BBC contracting a consortium to design the infrastructure for DMI. The article supports my view that the claims by Mr Linwood and subsequent NAO findings and the BBC’s evidence to the PAC hearing were not only misleading they were not technically possible. Essentially it could not be claimed that the DMI system was working if the design and delivery of the infrastructure had yet to be commissioned at the time of the report and subsequent hearing. [pages 090]
22. (During 2011 – 2012) I have regular catch ups with colleagues still working at the BBC and in areas close to the DMI project. I hear that nothing has been delivered during this period. I am told that a small trial of functionality had been attempted in the BBC London Factual department but the component was just a fraction of the workflow. Further I meet at a dinner party former colleagues of my wife (another former BBC production staff member), they indicate that they had "tried some sort of system for logging but it was pointless and didn't work very well” I realise they are referring to DMI.
23. (May 2012) I visit BBC Salford Quays, I had been part of the work stream that had defined much of the non-DMI technology for the facility. During my visit I became aware that DMI had not delivered any functionality. I was told there were developers attempting to deliver the “Archive” function but no useable system was in use in production. This was extremely disappointing to learn in light of the time that had passed. Following this visit I write to BBC Trust Chairman Lord Patten.
Conclusions
24. It is clear that this project failed because despite claims the DMI team did not deliver a working solution. It is my belief that this was as a result of very poor technical direction by the BBC evident in the DMI team’s inability to show me technical designs for the proposal. From Q1 2010 when the project was claiming a delivery schedule which included a “Basic Archive” and “Production Tools” they did not have a design that would deliver a viable implementation.
25. From the outset the technical leadership of DMI appeared mismatched with the skills required for such a project. This was compounded by a lack of understanding by the team of contractual boundaries the BBC had in place with many of its technology providers who would be necessary stakeholders. Adding these factors to the challenging environment created by the scale of the BBC and some of the investment needed around infrastructure (such as the network) meant the chances of success were extremely low. In essence the DMI team did not grasp the scale of the challenge.
26. A year later when Mark Thompson and Erik Huggers gave oral evidence to the committee hearing on the 15th February 2011 the BBC was still to commission the design and delivery of the infrastructure for DMI which brings into question their evidence. The infrastructure would have glued DMI together, it is therefore difficult to reconcile the lack of a fundamental element of DMI with the picture the corporation painted at the time.
Why was DMI allowed to continue?
27. Too many staff members and contractors jobs depended on DMI continuing, many of them recognised the project had little chance of success however speaking up would impact their careers and livelihood. Many senior figures had reputations invested in DMI. The resulting behaviours ensured that concerns external to DMI were either dismissed or shut down and business cases and other approvals were simply waived through without proper scrutiny. This culture ensured the project continued receiving funds, this was not one person but a very large group of people. There were a number of individuals senior in the technology division who held the responsibility to safe guard public funds in this area but did not act despite being aware of this situation. In some cases I believe certain individuals acted wilfully to subvert governance processes and falsify value propositions so as to deceive the various governance panels. This most serious matter is one I believe the BBC has a responsibility to investigate.
28. The 2013 PwC report into DMI followed a mandate set by the BBC Trust consisting of seven questions which narrowed the investigation to governance issues. I believe there remains a risk of a repeat of these circumstances because there are no conclusions about the real causes of DMI’s failure and therefore recommendations to mitigate them. I indicated to the Trust and the PwC team the significance of the 2010 business case and my concerns that this crucial step required examination. This business case permitted spending on the DMI project despite a questionable proposition, accordingly placing considerable public funds at significant risk. I am unaware if there has been any review of the preparation of this case.
29. The BBC does deliver large scale technology projects very well. Many of my former colleagues are committed to delivering value for money for the licence fee payer, however there remains a number of individuals within the leadership of the Technology and North divisions of the BBC who need to account for their actions in relation to the management of DMI.
A It is recognised that the current networking infrastructure is not fit for purpose and as such John Linwood has committed to upgrading the infrastructure from FM&T’s budgets. This must be done by Release 3 of DMI during 2010
Q3 Does DMI still anticipate to replace the production paperwork system P4A, if so is this fully funded from DMI and when would it deliver?
A Yes DMI is expecting to fund and deliver the replacement for P4A and this is expected in Release 4 or later. There is a short term solution for production paperwork called Silver Mouse currently under consideration as an interim solution but it’s costs are in the tens of thousands pounds and therefore not a considerable sum.
Q4 Is it possible to optimise the benefits case in a particular production genre or business area by only taking part of the DMI functionality, for example just the Archive? In context as other systems are expected to feed the DMI Archive in isolation for example Northern Ireland and A&M it must be technically possible to do this?
A Taking either the Archive or the Production Tools components is possible but this would not be DMI, The benefits case does not stack up unless the system is complete with all the metadata management making the Archive searchable. DMI was tasked to provide and end to end architecture.
Q5 If the system has to be complete are there any benefits from DMI as it’s functionality is rolled out, in particular in Children’s when they receive just the production desktop tools in May 2010 ahead of the rest of the system?
A Yes there are some financial benefits.
Q6 Are there additional costs that production areas will have to meet during the proposed benefits case? For example will they have to pay for any of the following: - Client machines for the DMI desktop tools; Training; Storage; Support.
A - DMI is designed to operate on standard BBC desktop machines and therefore there is effectively no additional cost over a normal machine. In some cases specific performance may need to be met with higher performance machines. Training is cost covered and will not impact on production financially. Storage for predicted volumes of media is covered, additional media capacity can be purchased from a rate card.
Q7 What are the breakdowns of costings for DMI? The original case from my understanding was £83M but I now see a figure of £105M what is this additional cost?
A The new costs cover lost benefits in the old case resulting from the late delivery. I am not providing a breakdown of costs as this could effect commercial negotiations with suppliers.
025
Q8 Had DMI delivered on time Children’s DMI solution will have effectively been fully funded, Will DMI cover the specific Opex costs of DMI in Salford for the benefits case period of 8 years.
A Project North has a budget line to cover capex for certain DMI elements not covered in the original £83M DMI business case.
Q9 I understand that the capex budget line in the Project North case, approximately £5M was to help design and build the “Share” enabler of DMI which was specifically needed for Salford, but what about the operational costs of running the Production Tools storage ie the back end of Cynergy?
A- This is a discussion for the Project North team who are all fully aware of this.
Q10 Aside of who may actually meet the costs of this ongoing year on year support in Salford, as DMI is the design and build authority what is the expectation cost of this element? You must have an understanding of what your solution will cost?
A This is not a question for this forum and I suggest it is directed towards the Project North Leadership Team of .
026 soon as is possible, I hope to get a finalised version to by 10am tomorrow.
– Question 7 on reflection I’m not sure I understand you response in that “lost benefits from the old case are included in the new figure”, is it possible you could provide me with an explanation of how this has become an additional £22M element to the current business case?
Bill
Q1 – Is the proposed DMI solution still an end-to-end system with tapeless delivery, and if the system is to be phased in when would the tapeless delivery functionality be available to production?
A – Yes the system is end-to-end and it will include tapeless (file based) delivery. This functionality is fully funded from DMI and file based delivery is expected to be available to production in Release 4 by the end of 2010. Benefits case is predicated on a four month lag between release and start of benefits realisation in order for productions to familiarise themselves with system, training etc
Q2 – Is the proposed system going to operate over the existing BBC networking infrastructure or a separate network? Are there costs of any additional networking work?
A – It is recognised that the current networking infrastructure is not fit for purpose and as such John Linwood has committed to upgrading the infrastructure from FM&T’s budgets. Delay to this would consrain the speed of roll-out from release 3 onwards.
Q3 – Does DMI still anticipate to replace the production paperwork system P4A, if so is this fully funded from DMI and when would it deliver?
A – Yes DMI is expecting to fund and deliver the replacement for P4A and this is expected in Release 4 or later. There is a short term solution for production paperwork called Silver Mouse currently under consideration as an interim solution. DMI is not funded to deliver an interim solution so there would need to be a decision about funding and delivery ownership before a decision is made at the end of january. These issues are outlined in the paper that is pulling together for
Q4 – Is it possible to optimise the benefits case in a particular production genre or business area by only taking part of the DMI functionality, for example just the Archive? In context as other systems are expected to feed the DMI Archive in isolation for example Northern Ireland and A&M it must be technically possible to do this?
A – Taking either the Archive or the Production Tools components is possible but this would
032 not be DMI, The benefits case does not stack up unless the system is complete with all the metadata management making the Archive searchable. DMI was tasked to provide and end to end architecture and to future-proof the production processes in line with creative futures aspirations.
Q5 – If the system has to be complete are there any benefits from DMI as it’s functionality is rolled out, in particular in Children’s when they receive just the production desktop tools in May 2010 ahead of the rest of the system?
A – Yes there are some financial benefits.
Q6 – Are there additional costs that production areas will have to meet during the proposed benefits case? For example will they have to pay for any of the following: - Client machines for the DMI desktop tools; Training; Storage; Support.
A - DMI is designed to operate on standard BBC desktop machines and therefore there is effectively no additional cost over a normal machine. In some cases specific (e.g. If a production chose to do high-end finishing within the DMI toolset) performance may need to be met with higher performance machines. Training is cost covered and will not impact on production financially. Storage for predicted volumes of media, as indicated by production departments, is covered, additional media capacity can be purchased from a rate card.
Q7 – What are the breakdowns of costings for DMI? The original case from my understanding was £83M but I now see a figure of £105M what is this additional cost?
A – There is no increase in the cost to the BBC. DMI will deliver more functionality to more areas for the same amount of money. The suggested increase is an accounting necessity to illustrate loss of benefits due to delays against old case. The DMI programme is currently negotiating substantial deals with 3rd parties around kit, professional services and managed service and it is not appropriate to highlight the monies available as this is commercially sensitive. All deals go through correct scrutiny with BBC Procurement and industry benchmarking in place to ensure that value for money is realised.
Q8 – Had DMI delivered on time Children’s DMI solution will have effectively been fully funded, Will DMI cover the specific Opex costs of DMI in Salford for the benefits case period of 8 years.
A – DMI will cover the operating costs and deliver a managed service for the duration of the case. There are some specifics around the Salford infrastructure and the commercial deals 033 that are being worked on that may shift some of this responsibility to other areas. This is being worked on collaboratively between project north, DMI and MCSL. It is not anticipated that this would have any impact on production departments
Q9 – I understand that the capex budget line in the Project North case, approximately £5M was to help design and build the “Share” enabler of DMI which was specifically needed for Salford, but what about the operational costs of running the Production Tools storage ie the back end of Cynergy?
A- There is a capex element in the project north funding for building DMI infrastructure. The operational costs are outlined in the previous answer.
Q10 – Aside of who may actually meet the costs of this ongoing year on year support in Salford, as DMI is the design and build authority what is the expectation cost of this element? You must have an understanding of what your solution will cost?
A – DMI will deliver within the capex budget. The other elements of spend, as outlined in answer 7 are, for very good reason, being negotiated by project north and a team is in place to drive this forward. This team includes senior management from project North, Dmi and FM&T.
034