Thursday, June 15th, 2017

Time	Event
12:00p	HPE Trips Over Australian Tax Office’s SAN There’s been a bit of a kerfuffle in Australia over a series of storage outages of a 3PAR 20850 SAN solution Hewlett Packard Enterprise built for the Australian Tax Office, an organization that services over 12 million taxpayers. Although the exact cause of the problem is still waiting for an official report from HPE — expected in “late 2017” — a report from the ATO indicates that a poorly designed system, compounded by a lack of preparedness by HPE, was largely responsible for the event. The report reads like a cautionary tale for service providers on how not to instill confidence. Although designed specifically to meet the ATO’s needs, the SAN was owned and operated by HPE, with ATO staff having no direct access to the system. A little before 1:00 a.m. on December 12, a little over a year after it went online, the system went down, with a cascading effect of data volumes entering preserved states to protect data, therefore becoming unavailable to applications and services. According to the report, this eventually “resulted in a systems outage, causing the majority of the ATO’s online services to become unavailable.” By 3:35 a.m. that morning, 455 out of 3,063 data volumes were out of service, as evidently faulty “firmware supporting impacted disk drives in the SAN prevented those drives from re‑booting.” By this time, enough conditions prespecified by ATO in its contract with HPE had been met to push categorizing the event as a Priority 1 incident. HPE didn’t do so until around 7:00 a.m. The SAN wouldn’t return to full functionality for about eight days. Luckily for the ATO and Australian taxpayers, no data was lost, but “the impact of pre‑incident design and build decisions were material in extending the time to recover data and bring production and supporting systems online.” It turned out that the SAN had been indicating issues that needed addressing for a while. “Analysis of SAN log data for the six months preceding the incident indicated potential issues with the Sydney SAN similar to those experienced during the December outage. While HPE had taken some actions in response to these indicators – including the replacement of specific cables – alerts continued to be reported, indicating these actions did not resolve the potential SAN stability risk.” In all, from May through November of 2016, 77 events related to components that later failed were logged by ATO’s incident resolution tool. And although HPE went to work swapping out some cables and such in an attempt to remedy the situation, the tax agency was “not made fully aware of the significance of the continuing trend of alerts, nor the broader systems impacts that would result from the failure of the 3PAR SAN.” It turned out that some stressed or ill fitting fiber optic cables were the most likely culprit to have initiated the failure (a similar outage happened again in February when a data card was dislodged while replacing a cable), but design issues with the SAN had made it much worse. According to the report, the system had been designed for performance over stability, with some monitoring and resilience features disabled. “This particular SAN configuration leverages a feature known as wide‑striping which is designed to significantly improve performance by reading and writing blocks of data to and from multiple drives at the same time, preventing single‑drive performance bottlenecks. When several physical disk drives were impacted by a drive firmware issue which prevented those drives from re‑booting, the result was that a small number of drives temporarily and in some cases permanently prevented access to a significant amount of application data. This also had the effect of extending the duration and complexity of the recovery effort.” Another issue that delayed recovery was that recovery tools for restoring the system were kept on the failed SAN. Although the ATO report lays much of the problem at the feet of HPE, it admits to sharing some of the blame. For instance, there was another backup SAN running in a separate data center that due to a design issue couldn’t be used for recovery, partly because ATO had “relied heavily on HPE recommendations,” which in hindsight was a mistake. “Full automated fail‑over for the entire suite of applications and services in the event of a complete SAN failure in Sydney was not part of the storage solution for the SAN. The cost of automatic fail‑over systems, as they exist in some areas of critical infrastructure or in large financial institutions, is very high.” In response to the failure, the ATO has commissioned a new 3PAR solution. When the data from the old system is transferred, it will be taken out of service and undergo forensic analysis by HPE. The tax agency also indicates that when its current contract with HPE expires, it might consider other options. (Comment on this)
3:00p	New Data Center Provider eStruxure Enters Growing Canadian Market With Canadian companies clamoring to store data at home, and key data center locations in Toronto and Montreal expected to grow 20 percent annually over the next few years, newcomer eStruxture announced this week it is entering the data center services market. The Montreal-based company said it will use $80 million in raised capital from Canderel and Caisse de dépôt et placement du Québec to expand its footprint. “Cloud adoption and the overall proliferation of data are driving increased demand for hyper-scaled, enterprise class data centers to support processing and storage capacity,” Todd Coleman, president and CEO of eStruxture, said in a statement. “Customers are looking for experienced data center operators with the expertise and financial ability to grow with customer needs.” eStruxture also announced it has acquired Montreal-based Netelligent Hosting Services, one of the largest data center operators in the city with more than 850 customers using its downtown data center facility. “This first acquisition gives us access to high-quality infrastructure and a strong enterprise customer base,” Coleman said. “Thanks to the experience and expertise of the team already in place, we will benefit from growth opportunities in Montréal, Toronto, Vancouver, and elsewhere in the country.” The Canadian market has advantages that global customers looking to process and store data in North America may find appealing, including relatively low energy costs, cooler climate, and a business-friendly regulatory environment. (Comment on this)
3:30p	Is it Time to Downsize Your Integration Efforts? John Joseph is VP of Marketing at Scribe Software. The shift to cloud applications created a proliferation of data silos that seemed to set us back to square one in our efforts to integrate enterprise data.  But now more cloud application providers are improving their apps’ ability to connect to both on-premises and cloud applications.  It is a reflection of the growing importance of data integration. It represents a milestone in the evolution of cloud applications, as they become more connected out of the box as a way to differentiate the application, and it’s an opportunity for business users to rethink their data integration strategy as cloud-based data integration alternatives become the norm. But, does it mean that an enterprise can start to think about shrinking their data integration efforts? For SaaS providers, the pressure is increasing to add more out of the box connectivity. Whether the application itself is mature or emerging, how well the application connects to other cloud applications and on-premises applications has become a primary evaluation criterion when businesses purchase SaaS applications. Until recently, integration was usually addressed as a “phase 2 requirement” – after the software or service was purchased and in use. It meant that when businesses purchased SaaS applications, such as social media platforms or sales productivity tools, the burden was on them to figure out how to get them connected to the rest of their cloud and on-premises systems. That is changing. Increasingly SaaS providers are deciding to solve their integration challenges by embedding a commercially available cloud-based integration platform, or integration platform as a service (iPaaS), with their own applications. For businesses that may purchase a variety of SaaS applications this is a great opportunity to re-evaluate their own integration strategies. They need to add more integration capabilities, therefore, to keep pace with the rate of SaaS adoption – especially they need to add agile integration capabilities to address the needs for customer-facing teams to change applications, fields, and aggregations. Consider what’s happening to the application landscape, where there is an explosion in the number and variety of applications available and in use at a single enterprise.  Some research organizations have reported that a single enterprise can be have hundreds of SaaS applications in use within their walls. Plus, with innovations such as the Internet of Things promising to spur new application needs, you should expect the number of applications you use to continue to rise. There is a clear and present need, therefore, to integrate more apps and to deal with the complexities of applications and APIs from different manufacturers. So as you think about your integration approach moving forward, you should evaluate your options on these points: Lifecycle Management Costs – If you are going to be using a wide array of SaaS applications from various manufacturers you can benefit from having an integration platform that provides a single control layer for your applications and integration developers. The platform’s maker likely takes on the burden of ensuring compatibility with the applications it supports, meaning you will spend no time revising integrations because of API changes to the various SaaS applications. A single integration platform can also make monitoring and lifecycle management easier as you have a standard integration technique and a single pane of glass for managing integrations for various manufacturers. Consider the case where you decide to switch from one CRM system to another down the line. If you had used integration capabilities built into that CRM system, you would have to reintegrate all of your other SaaS applications to the new CRM system. But if you used a third-party integration platform, your workload could be much lighter because you may just have to connect the new CRM system to the platform. Depending on your future plans and how frequently you change applications, using a third-party integration platform to integrate core applications from multiple manufacturers may lower your overall lifecycle management costs. Custom Application Deployments – If your application’s manufacturer offers some integration capability that allows you to develop custom extensions to their application, it could be your fastest way to reach your goal, but consider downstream management time and whether the application will change before you decide to use that interface. A dedicated integration layer can add a layer of abstraction that will allow your application to work with any CRM system without having to rebuild the application from scratch. The Field-Level Capabilities of Integrations – If two organizations separately created connectors between two applications, they could differ wildly in their capabilities. One may ensure that additional information is captured and made available to the other application. This could create a differentiator for your business or could be a better fit for your existing sales process. So consider the capabilities of an integration offering, not just the names of the sources they supposedly connect. Support for more fields and entities means you will have more flexibility to integrate what your business users want. Business Agility You Require – Connectors between different applications can run the spectrum between being flexible and fixed. Usually that comes with a tradeoff of how complex they are to manage and adapt. Consider not just often you expect to add/subtract applications, change fields or add new orchestration steps between applications, but also who will be making the changes. Increasingly the task of creating and changing powerful integrations is being done by business users and not by an exclusive set of developers in the IT department. It is likely that you will answer these questions differently when considering your CRM, ERP, or other applications. So your aggregate integration approach may end up being a well-thought out mix of approaches. You may rely on an out-of-the-box integration from your marketing automation system to connect to your social media platform because the fields you need integrated are few and won’t change, but use a third-party platform for integrations to connect your marketing automation and CRM systems because of the need to continuously update fields and workflows based on those systems. Opinions expressed in the article above do not necessarily reflect the opinions of Data Center Knowledge and Penton. Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.   (Comment on this)
4:30p	British Airways Owner Says Data Center Outage Cost £80M Maria Tadeo and Christopher Jasper (Bloomberg) — British Airways owner IAG SA said a power outage that led to the cancellation of hundreds of flights last month probably cost it about 80 million pounds ($102 million) in lost revenue and the expense of accommodating, re-booking and compensating thousands of passengers. The figure is based on an initial assessment and London-based IAG will update the market on the exact figure in due course,  Willie Walsh, the group’s chief executive officer, said at its annual shareholder meeting Thursday. “It was a dreadful experience for many of our customers and we are truly sorry,” Walsh said in Madrid, adding that “BA passengers will be compensated as soon as possible.” See also: Delta: Data Center Outage Cost Us $150M At least 75,000 travelers found themselves grounded over three days from May 27 as the U.K. carrier’s information-technology systems crashed. Walsh has said previously that the  meltdown seems to have happened after an engineer disconnected a power supply at a data center near London Heathrow airport, causing a surge that resulted in major damage when it was reconnected. Read more: British Airways Says Engineer Wrongly Disconnected Data Center Power The CEO said that an independent investigation into the failure is progressing, while adding that there’s nothing to suggest the incident was connected to cost-cutting efforts ordered by BA chief Alex Cruz, or an attendant trend toward outsourcing some activities to less costly third-party providers. Citigroup had estimated the expense of the outage at 100 million euros ($112 million), comprising 40 million euros in lost revenue and 60 million euros of customer compensation. The breakdown at British Airways came as another blow to the aviation industry, which has been plagued by passenger outrage over a number of incidents since a passenger was dragged from a plane at United Airlines. BA suffered a further setback Thursday, when a failed Heathrow baggage system meant thousands of passengers had to leave without checked bags during the morning departure peak at two terminals used by the carrier. See also: Most Data Center Outages aren’t Caused by Tech Failure (Comment on this)
7:12p	Switch’s Las Vegas Data Center Stronghold Reaches North of 2 Million Square Feet Switch has officially launched the latest massive facility on its homebase Las Vegas data center campus, bringing its total capacity in Sin City to more than 2 million square feet and about 315MW. Switch, the largest data center provider in Vegas, recently started expanding to other markets. Known for its proprietary data center design, futuristic interiors, and ex-military security guards armed with machine guns, the company builds hyper-scale colocation facilities and lists among its top customers Amazon Web Services, eBay, Hulu, and NASA. Its latest Las Vegas 10 data center adds about 350,000 square feet of data center space and can provide up to 40MW of power. It is designed to the same specifications as the previously existing Las Vegas 8 and Las Vegas 9 facilities. The design, according to Switch, is a brain child of its founder and CEO, Rob Roy, who designed everything from mechanical and electrical systems to the roof and conference-room interiors. The company leans heavily on its data center design for setting itself apart from competitors, and earlier this month announced its own design standard, called Tier 5 Platinum, which includes a long list of characteristics that aren’t covered by the industry’s most widely used and recognized data center design rating system created by the Uptime Institute. Switch had portions of Las Vegas 8 and 9 data centers certified by Uptime. Both received Tier IV Gold certification, the highest rating in the system designed to evaluate data center infrastructure reliability. Switch said it would not pursue Uptime certification for any of its future facilities because the system doesn’t take into account elements such as network carrier redundancy and availability of renewable energy, among many others. It also complained that Uptime doesn’t do enough to police misuse of its terminology by data center providers. Read more: Switch Backs Away from Uptime’s Tiers, Pushes Own Data Center Standard Switch’s current design is called Switch MOD 250 (Modularly Optimized Design). Modularity enables data center providers to expand capacity in a building quickly by installing standardized, pre-fabricated infrastructure components. Switch.com – The Core Campus The company launched its first non-Las Vegas data center in February of this year. The first building on its Citadel Campus outside of Reno, Nevada, has eBay as the anchor tenant and measures 1.3 million square feet; it can support up to 130MW of power. The following month Switch announced the launch of a data center in Michigan, inside a re-purposed pyramid-shaped former office building, and in May said it had secured land to build data centers in Atlanta. The company is also building data centers in Italy and Thailand. Both international projects are partnerships with local investors. (Comment on this)
9:54p	Xpanxion test content – do not publish Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi sed enim ex. Maecenas pulvinar cursus velit, ac mattis dolor sodales efficitur. Fusce libero mi, tempor vel neque id, bibendum ultrices elit. Nam venenatis tincidunt posuere. Nunc tincidunt libero purus, at volutpat nisl facilisis non. Nulla porttitor bibendum ex, sed tincidunt metus porta vel. Sed quis tortor nisi. Aliquam sed tortor leo. Ut luctus, ipsum finibus feugiat lobortis, purus massa tincidunt lacus, sit amet euismod nulla ante eu tellus. Pellentesque condimentum iaculis mattis. Sed vel tellus consectetur, maximus nunc sed, feugiat purus. Integer quis mi ullamcorper, placerat nibh at, pharetra velit. Nunc volutpat arcu quis est ornare, ac aliquam nunc vulputate. Praesent sit amet nisi nisi. Bulleted list items: Bullet 1 Bullet 2 Indented Bullet 1 Indented another level bullet 1 Indented Bullet 2 Bullet 3 Nulla id arcu eget urna tempus volutpat nec id enim. Etiam enim arcu, porttitor eget auctor eu, efficitur sit amet ipsum. Sed eleifend, diam non interdum venenatis, diam ipsum imperdiet nibh, non cursus felis ante vitae lectus. Vivamus ultrices lectus sed vulputate facilisis. Phasellus euismod facilisis tortor, eget scelerisque nisi aliquam in. Praesent porta sem lacinia, ornare lectus in, ullamcorper libero. Sed maximus pulvinar felis, quis blandit eros efficitur non. Nunc ex magna, fringilla fermentum fringilla ac, semper vel enim. Nunc sit amet placerat neque. Cras imperdiet maximus urna sit amet auctor. Nam nisi turpis, tincidunt sed nisi sed, finibus rhoncus ante. Vestibulum mauris nunc, semper ac ullamcorper id, efficitur nec dui. Pellentesque facilisis nibh sed est consectetur rhoncus non eu sem. Numbered list: Number one Number two Indented number one Another indented number on Indented number two Number three Sed a dapibus quam. Morbi rutrum egestas felis et sollicitudin. Donec tincidunt justo eu metus porta mollis. Curabitur cursus purus at congue eleifend. Suspendisse euismod faucibus justo, ut pulvinar nunc efficitur ut. In sed faucibus leo, at dapibus metus. Donec sit amet lectus felis. Maecenas at augue vitae ex rutrum porta eget lacinia ipsum. Vivamus euismod leo sed dapibus feugiat. Sed ornare non turpis consequat viverra. Nullam placerat placerat leo. Vestibulum pulvinar ligula tempor neque auctor, id posuere ligula pulvinar. Praesent consectetur egestas viverra. Donec pulvinar dolor mauris, eu cursus metus suscipit maximus. Nam non scelerisque eros. Aliquam ac elit leo. Integer molestie, turpis ut pharetra auctor, sapien risus condimentum purus, vitae pulvinar est justo placerat erat. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Nullam ullamcorper dui sed odio fringilla faucibus. Vivamus posuere sapien ut felis ullamcorper porttitor feugiat ac lacus. Donec et nibh cursus, maximus justo sed, consequat risus. Etiam rutrum luctus finibus. Proin vulputate neque eget justo tincidunt mattis eget nec nulla. Mauris rhoncus vulputate tellus eget malesuada. Nulla viverra ante dolor, id ullamcorper tellus laoreet sit amet. Donec rutrum, dui eu molestie placerat, metus massa sodales tortor, eu blandit justo augue ut sem. Pellentesque urna ligula, auctor sed elementum ut, fringilla vel ipsum. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Cras sed pellentesque lacus, eget finibus felis. Curabitur in ex mi. Vivamus eros diam, porttitor ut mollis ac, semper a risus. Morbi dapibus quis ipsum id molestie. Etiam et risus rutrum, condimentum dui et, ultrices arcu. Proin semper massa diam, quis ornare lacus pretium et. (Comment on this)

<< Previous Day

2017/06/15 [Calendar]

Next Day >>