All sounds a bit dodgy to me, I can't imagine that a single power supply or even a whole substation going out would, in this day and age, completely mangle a major company's operations in the way that this has. Most companies have done full risk analysis, and have fail-safe systems and disaster recovery scenarios that cost many hundreds of thousands to maintain, and often carry out regular dress rehearsals.
What do we think has really happened?
I wonder whether the denial that it was a cyber attack gave any clues...
|
|
Some t*** applied an inadequately tested patch or upgrade, would be my bet.
|
Outsourced to India according to the news this PM and staff advised that the subcontractors have no experience of dealing with this type of problem!!!
Perhaps the accounts will show the false economy of outsourcing....
Savings from outsourcing: £100m.
Cost of todays debacle: £150m.
;-)
|
Much as I don't like outsourcing UK jobs abroad, I'm also not sure that that in itself would be a cause. There are some quite experienced people abroad, and some incredibly stupid people running systems here. Outsourced people are often only running stuff as instructed by people here so I don't really see that as a potential cause.
The faulty patch is a possibility.
I wonder if whatever system it is that has caused the issue was simply not seen as a show stopper and therefore didn't have the belt and braces which a core system would have, though this too would be a bit daft.
I doubt we will ever know the truth as it is almost definitely an embarrassment for BA.
|
Outsourced people are often only running stuff as instructed
>> by people here so I don't really see that as a potential cause.
The people who know how to run the system, i.e. instruct, are not here any more. Plus those abroad were educated by people over here who knew their jobs were going there.
The major RBOS bank failure, and their inability to recover it quickly was a direct result of outsourcing.
Last edited by: Zero on Sun 28 May 17 at 21:59
|
would never have happened in the old days on the Mainframe.
Whatever the cause, there is no excuse for a crap fallback/recovery strategy. Mind even the banks have now fallen at that hurdle.
Its because at board level its claimed "IT is not our core business"
Sorry CEO, but in 2017 it is very much your core business.
|
>>Sorry CEO, but in 2017 it is very much your core business.
Totally agree.
Look at Amazon, from books to almost anything and they use their vast computing capacity to stream films and music and host other organisations' websites as well as providing cloud services.
|
It is not their core business.
That is not say it is not essential.
|
>> It is not their core business.
>>
>> That is not say it is not essential.
Hmm.
Let me see. A banks core essential is its ledger. On what ever format that is. Lose it and you can't do business. In fact technically you are insolvent
An airlines core business is flying passengers. Its essential. They couldn't and didn't.
Your statement is typical of the thinking I was referring to. If it prevents you from doing business, its essential
Last edited by: Zero on Mon 29 May 17 at 07:14
|
The UK government is a classic example of "getting it wrong" in IT - failed IT systems where it is outsourced - failings either in function and / or failures to keep to the development budgets.
The "amount of computing knowledge" of many government departments at a senior level can be worryingly poor. Faced with the task of ordering new systems, hardware & software, bad decisions on function & cost are made & the taxpayer picks up the bill.
Outsourced computing is often decided upon as it "saves money" within a few years the Computing knowledge within the company or government department is diluted as staff leave/retire. This is then exploited by the contractor with amendments & upgrades costing much more than would have been possible with "in-house staff" knowing what needed to be bought & what time tasks would take.
|
Couldn't have put it better myself. Every word is so true, based on my experience of IT within at least one Govt. department.
Governments never understand IT. You only need to see the shambles of certain projects to see that.
|
Govt. IT!
First proper job was accounting in a Govt dept. 700 people to keep proper book ledgers and to hand write month end statements to debtors. Mainframe in huge warehouse was to keep control accounts only!
IBM PCs were left unused in the corner because management were frightened that we would break them!
A modern PC, Sage Line 50 and half a dozen staff could have replaced the lot!
Last edited by: zippy on Mon 29 May 17 at 10:38
|
>> IBM PCs were left unused in the corner because management were frightened that we would
>> break them!
Similar in an agency of the then Lord Chancellor's Dept. At time Pentium was new there were dozens of well spec'd PCs in a warehouse quietly obsolescing!!
Those who did have PC's were not allowed to use them for correspondence because one or two people had written carp letters. The officers concerned continued to write carp letters either by hand or via the typing pool.
|
>> If it prevents you from doing business, its essential
Of course it is. But that doesn't make it core.
|
|
By the way, as a general rule I am not a fan of outsourcing. However, not outsourcing needs to be at least as smart a decision as outsourcing, and neither are helped by preconceived ideas or prejudices.
|
....an ex-colleague of mine is (or possibly now was!) their head of IT operations.
I might, at some time in the future get to know the full story.
As for outsourcing, the main issue is the arms-length relationship that reduces both visibility and control, making it easier for things to go tits-up, and more difficult to respond.
Outsourcing to India brings additional cultural problems as, great people as they are, they just can't say 'no'. (Even when they should)
And IT has been core business for airlines since the mid to late 70's when I first worked with IBM's ACP; increasingly, the airline industry just cannot work without it.
|
>> And IT has been core business for airlines since the mid to late 70's when
>> I first worked with IBM's ACP; increasingly, the airline industry just cannot work without it.
Ah the ACP, OS/360 with all the fancy bits stripped out, use to fall over and recover so quickly no-one knew it had failed.
|
It certainly ran on system/360, but was an OS in its own right from the ground up.
AFAIK, when I was working on it BA were the only other UK user (kept offering me a job).
It was an odd thing to work on, all programming was in assembler in 1K segments, and the whole system had to be rebooted to initiate any changes. That led to a practice of leaving some of the 1K segment unused (as a patch area) and inserting machine code into a live (though usually test) system to try things out.
Happy days!
|
>> IT is not our core business
New version should be:
IT is not our core business but without IT we have no business. So, lets make sure we have a properly running IT system!
Management often see as "cost" instead of "enabler". This mind shift is paramount to prevent such issues in future.
>> Outsourced people are often only running stuff as instructed by people here
Agree with above. Outsourced services to Indian IT vendors are all about making sure operational activies (mostly mundane) are performed as instructed. These vendor rarely influence any strategic decision.
Just like an aircraft accident (as shown on TV) happens because of series of events going wrong (almost no aircraft accident happened because of single thing failed/done wrongly) this kind of catastrophic IT failure is result of many things go wrong. This is a strategic failure from leadership team.
While I understand CEO doesn't get involved in this directly but it is his task to figure out what the strategy should be and who would execute this. The IT director should definitely be held accountable for this failure.
|
Seems it was one bloke's fault. Bet he's crowing about it down the pub (Henning Wehn does a bit of his stand-up show about how the Brits go to work, screw something up and brag about it)
www.bbc.co.uk/news/business-40159202
|
British Airways IT issues - No FM2R
