Tuesday, February 17th, 2015

Time	Event
4:00p	Pivotal Open Sources Entire Big Data Suite As enterprise IT grows increasingly comfortable with using open source technologies, presence of enterprise vendors in the open source ecosystem grows too. Pivotal, the San Francisco-based company majority owned by EMC and VMware, whose mandate is to enable enterprise developers to build and deploy analytics-enabled software using modern agile development methods, has led three open source projects: the well-known Platform-as-a-Service software Cloud Foundry; enterprise messaging software RabbitMQ; and Redis, an in-memory key-value store. Now, the company has decided to open source the remaining components of its suite of big data analytics software. Pivotal announced the decision Tuesday, the day it also announced a partnership with enterprise Hadoop company Hortonworks and formation of a new industry association around open source big data technologies. The components of Pivotal’s Big Data Suite being open sourced are its Hadoop distribution, called Pivotal HD; its massively parallel processing SQL engine called HAWQ; Greenplum Database; and GemFire, its in-memory NoSQL database. The main reason to open source the entire suite is simply to give enterprise customers what they want, Sundeep Mandra, vice president of Pivotal’s Data Products Group, said. Traditionally averse to open source, enterprises are increasingly realizing that there are some big benefits to the approach. Those benefits are preventing single-vendor lock-in and being involved in shaping the technology they use. “They want to impact the roadmaps,” he said. Open Source With Paid Premium Features Pivotal is taking the common approach to making money on open source software: open the code but sell a commercial distribution with advanced features. Those include things like WAN replication in GemFire, which is a way to have a fully replicated database at a remote data center, ready to be queried without delays when the primary data center is having problems. Another advanced GemFire feature is continuous query, which continuously retrieves results of a SQL-type query as they become available. In a trading application, for example, a query may be used to display stocks over a certain price point. As more stocks cross the price point during the day, they will be retrieved for the application’s use. The open source version of HAWQ will be fully functional but will not include next-generation query optimization and management tools. Pivotal has not yet worked out exactly which components will be open source and which will be kept behind the paywall. “We’re not necessarily finalized on the list,” Mandra said. A Common Core for Hadoop The new industry association, called Open Data Platform, will promote big data technology based on open source software in the Apache Hadoop ecosystem. The consortium has a hefty list of founding members, consisting of major vendors and service providers. In addition to Pivotal, it includes EMC, VMware, GE (which owns a minority stake in Pivotal), IBM, Teradata, Verizon, CenturyLink, Capgemini, Splunk, Hortonworks, and AltiScale. First order of business for ODP will be creation of a tested core reference platform of Hadoop and Apache Ambari, the open source software for provisioning, managing, and monitoring Hadoop clusters. Member companies will build offerings based on the common core. The point is to create a standard platform to simplify and accelerate integration of applications and tools with Hadoop. Instead of testing different tools by different vendors for compatibility on their own, users will simply know that the tools will work on any system that’s compliant with the standard. The goal of ODP is to avoid segmentation in the Hadoop ecosystem, Mandra explained. ODP wants to do for Hadoop what creation of the Linux kernel did for the Unix ecosystem, which was getting fragmented. “That’s what we want to do with the Open Data Platform,” he said. “We think this will be a really big advancement.” Linking Up with Hortonworks Pivotal is joining forces with Hortonworks, the Yahoo spinoff that last November became the first enterprise Hadoop company to file for an initial public offering, to integrate products and to collaborate on engineering and support. Pivotal plans to enable its Big Data Suite to run on Hortonworks’ platform, but it will continue supporting its own Hadoop distribution, Mandra said. Notably, Cloudera and MapR, the other two major enterprise Hadoop players, were not part of the ODP announcement. Mandra said he would love for the two Hortonworks competitors to be part of the group. Customers Set Tone in Hadoop Ecosystem Hadoop, the open source framework for storing across clusters of commodity servers for parallel processing, is enjoying widespread use in the enterprise nowadays. Mike Hoskins, CTO at Actian, which provides a SQL analytics solution based on Hadoop, said nearly every company has a Hadoop cluster, and those are no longer experimental deployments. “It’s hard to find a major account that doesn’t have a deep, serious project and investment in Hadoop clusters,” he said. Customers are driving technology roadmaps more than ever before, he added. “People don’t appreciate to what degree the power pendulum has swung from the vendor to the customer. Customers are now more empowered to set the ground rules.” Perhaps this explains Pivotal’s decision to let users play a bigger role in further development of its “crown jewels” through participation in the open source development process. Another possible explanation is wanting to focus on higher-level tooling, while letting go of the control over lower-level infrastructure components, Hoskins said. “The value is all higher in the stack.” (Comment on this)
4:29p	Keeping the Video Streaming Lance Smith is CEO of data virtualization company Primary Data. YouTube, Netflix, Amazon and a host of other on-demand broadcast content providers have changed the way we consume Internet media, and video is now the vast majority of content we check out online. Many technology advances improved and redefined media play, transfer and storage to support this culture shift. Let’s review how some of the breakthroughs we’ve seen in recent years will bring even more on-demand media innovation in the near future. The Video Stream Becomes a River Many of us stream video today, but we also access video content via standard broadcast (cable, satellite and over-the-air); capture (DVR); and disc (Blu-ray/DVD). Much more content is moving to the cloud, and we are increasingly watching video on social media platforms. In fact, a recent Business Insider article notes that 63 million people in the U.S. watched a video on Facebook during the month of April 2013. These videos now come from more sources than ever before, as our smartphones can produce high-quality videos, and specialized cameras like the GoPro enable all of us to shoot professional-caliber footage. Keeping video streaming to all of our screens requires more innovation in IT. The shift to social networks, networked screens, and mobile devices as our primary viewing platforms will push most video content to be streamed exclusively. Enabling offline and mobile viewing on all of these platforms will require a number of innovations in video delivery technologies. Location, Location, Location Supporting the increasing volume of streamed video data requires that we solve the hosting performance and delivery bandwidth challenges that currently frustrate online video viewers. These two factors play a significant role in providing high Quality-of-Service to the customer. Whether at the source or the last mile, insufficient bandwidth creates interruptions that result in dreaded buffering pauses. In addition, to ensure smooth performance, the content provider must provide enough processing horsepower to support sudden spikes in simultaneous users and user activity. This can be common when a beloved celebrity dies and suddenly makes archived films “must see” content. Today, processing performance issues are resolved by using memory caches and low latency storage media like flash. Content providers address basic bandwidth challenges by compressing data and bring hot content closer to users by placing the data on edge servers at key geographical locations around the world. Telco companies are continually expanding en-route bandwidth with scalable solutions such as 4G and LTE for mobile devices. Households insulate against peak-hour congestion with Fiber Optic connections of up to 1Gbps of dedicated bandwidth. While these solutions work today, they are not sustainable under our current growth rates. Flash storage is currently too expensive to host the massive content libraries of a leading entertainment provider. The manual and semi-automatic movement of data to edge stores is too slow to scale with an unexpected pop culture event. The next five years will bring technologies that enable video content hosting and delivery to adapt dynamically to rapid changes through data virtualization. Intelligence applications will provide real-time analysis to predict what data is becoming hot, what is cooling off, and then automatically move the data to match resource supply with data demand. This will both improve the viewing experience and increase content providers’ ability to scale cost-effectively. Making the Most of Metadata While the bandwidth and storage infrastructure requirements for Full HD and Ultra HD (4k) video files are enormous, an even bigger challenge arises from the significant value that businesses can derive by analyzing metadata about the video and the viewer. Pulled from a variety of different sources, including Web pages, browsing history, search criteria, file attributes and social media, metadata allows companies to gain insight about viewers, market segments, or general trends. This information can then be used for ad-matching, recommendations and future program planning to ensure that content matches our evolving interests. Making the most of metadata presents two significant challenges for content providers. First, significantly more server and storage resources are required because metadata volume is growing faster than the data itself. Second, conventional databases are ill-suited to handle metadata because it comes from many different sources. This has led to the development of unstructured databases such as Cassandra and MongoDB that scale performance and capacity quickly. Still, much work remains ahead, and we’ll soon see databases that enable content providers’ applications to access unstructured data more easily. As discussed in the eWeek article, “Hybrid Databases Gaining Favor for Enterprise Big Data Analytics,” new database platforms will accept inputs from many sources, cleanse data to minimize redundancy and worthless data, format it for different applications and high-level databases, and present views for end users and applications. Data virtualization technologies will dynamically manage where data should be stored based on access patterns and other metadata attributes. Cold data will be immediately moved to low-cost storage, while active data will be moved to Tier 0 and Tier 1 media. The Road Ahead Over the next few years, the technologies that support online video will continue to be a key source of innovation. What’s even more interesting is how many of these advancements will transfer seamlessly to all data centers as data virtualization is widely adopted. With policy-based dynamic data mobility placing data on the optimal resource for an application’s performance, price, and protection requirements, data virtualization will disrupt conventional centralized architectures, improve customer service levels and accelerate time to market, all while reducing data center costs. Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library. (Comment on this)
5:19p	Datapipe Adds Managed Cloud Services for Microsoft Azure Datapipe has extended its managed cloud services to Microsoft Azure. Datapipe has developed a solid business providing managed services atop of Amazon Web Services in the last few years and now extends that same managed cloud support atop of another public Infrastructure-as-a-Service giant. Datapipe offers end-to-end support in planning, building, and running a managed public cloud environment. For Azure, it can utilize both Hyper-V for private cloud and the public cloud infrastructure for a hybrid topology. The end game is to support any kinds of infrastructure and multiple deployment models, providing managed hosting regardless of setup. Furthering its mission to enable hybrid, Datapipe recently partnered with Equinix for a colocation and managed AWS combo. The New Jersey-based company has made several acquisitions over the years to extend its hybrid infrastructure capabilities. It recently acquired GoGrid, extending its multi-cloud capabilities around Big Data deployments. GoGrid came with proprietary orchestration and automation technologies, specializing in one-click Big Data infrastructure deployment. It also acquired Layered Tech in August, which formed the foundation for its government cloud business, deepening compliance capabilities. Managed services make public cloud resources more accessible to a wider swath of customers who can’t use vanilla public cloud, or those that desire to leverage multiple cloud platforms. It also opens up Datapipe’s potential audience to customers already on Azure, or those on Microsoft tech that want to extend infrastructure out to Azure. “These new services will enable Datapipe customers to more effectively plan, build, and run enterprise applications on Azure,” said Rob Allen, CEO of Datapipe, in a release. “This collaboration with Microsoft further strengthens our leadership in offering the choice and control of managed cloud and hybrid IT solutions.” “Datapipe’s capabilities to manage hybrid environments and ability to incorporate Microsoft infrastructure into hosted private clouds, as well as integrate with Azure’s public cloud environments, make the company a natural fit to work with Azure,” said Aziz Benmalek, general manager of Microsoft’s Hosting Service Providers business. (Comment on this)
6:00p	Cloud DR Provider CloudVelox Nets $15 Million Hybrid cloud disaster recovery provider CloudVelox has closed on $15 million from Cisco Investments and others to help build out sales and marketing and further expand product development efforts. Previous investors Mayfield Fund, Pelion Venture Partners, and Third Point Ventures also participated in the round. CloudVelox offers automated cloud migration and disaster recovery software to move application workloads across data centers and clouds. The company’s One Hybrid Cloud platform involves a five-step automated process to discover, blueprint, provision, synchronize, and initiate service for workloads, according to the company. CloudVelox says that this process allows a company to develop disaster recovery processes that use the public cloud as a secondary data center, paying only for what is needed. The City of Asheville, North Carolina, recently revamped disaster recovery plans, citing the company’s cloud DR as the perfect solution for avoiding the costly venture of building their own redundant data center locally. Jonathan Feldman, Ashville CIO, was quoted in a statement saying that a secondary data center is a “terrible use of taxpayer funds.” Feldman said the CloudVelox platform was a game changer. “Compared to any other option, we were able to deploy our important apps on AWS for DR with minimal budget impact and maximum compatibility.” CloudVelox is listed as an advanced technology partner for Amazon Web Services, where cloud DR solutions feature architectures from ‘pilot light’ to ‘hot standby’ environments. Last year the four-year-old Santa Clara, California-based company was named one of the “Cool Vendors in Business Continuity Management and IT Disaster Recovery Management” by Gartner. CloudVelox CEO Rajeev Chawla noted that the Series C funding was a “key milestone for CloudVelox, which supports our view of the market for hybrid cloud automation software. The funding will help us to continue our product leadership while scaling our teams and resources for growing market demands for automated cloud DR and migration.” (Comment on this)
6:30p	Sophisticated Self-Destructing Equation Malware Infects Thousands of Servers Worldwide This article originally appeared at The WHIR Researchers have discovered a sophisticated, self-destructing malware that has the ability to affect hard drive firmware. According to a report released on Monday by Kapersky Labs, a group called Equation has been using multiple types of malware since 1996 to stage cyber attacks. “The Equation group is probably one of the most sophisticated cyber attack groups in the world,” said the report, “and they are the most advanced threat actor we have seen.” Kapersky identified more than 500 victims worldwide but because the malware contains a self-destruct mechanism, it estimates the actual number of victims to be in the tens of thousands. Servers, domain controllers, data warehouses, website hosting and other types of servers have been found with infections. Command and control infrastructure used by Equation includes 300 domains and more than 100 servers in several countries including the US, UK, Italy, Germany, Netherlands, Panama, Costa Rica, Malaysia, Colombia and Czech Republic. Data collection and spying continues to make news with a never before seen complex surveillance software. In November, Symantec discovered Regin. “The level of sophistication and complexity of Regin suggests that the development of this threat could have taken well-resourced teams of developers many months or years to develop and maintain.” The newly identified software used by the Equation group is even more complex. Called Equation for its love of algorithms and sophisticated threat methods, the group uses at least six kinds of malware to wreak havoc on unsuspecting indows systems. Equationdrug, doublefantasy, grayfish, equestre, triplefantasy and fanny are the varieties of malware identified in the report. Grayfish is the most complex of the tools residing in the registry and using bootkit to execute at startup. Doublefantasy is initially used on victims to identify if they are interesting enough to be targeted and to keep a backdoor open in the system. If the target is deemed interesting, the Equation group moves forward with installing equationdrug on older operating systems and grayfish or triplefantasy on systems using Windows 7 or newer. These programs allow for full control of the operating system. The malware is primarily designed for Windows systems, however Kapersky did find evidence indicating there may be a Mac version of doublefantasy as well. It also noticed different code being shown to iPhone users which indicates they are also being targeted. “GRAYFISH is the most modern and sophisticated malware implant from the Equation group,” according to the report. “It is designed to provide an effective (almost “invisible”) persistence mechanism, hidden storage and malicious command execution inside the Windows operating system.” This scariest thing about this piece of software is that it’s the most complex that Kapersky has ever seen suggesting “developers of the highest caliber are behind its creation.” Based on the level of development and the most targeted countries of attack being Iran, Russia, Pakistan, Afghanistan, India, China, Syria and Mail, it is speculated that the NSA or other government entity may be behind the Equation group. Target groups included governments and diplomatic institutions, telecommunications, aerospace, energy, nuclear research, oil and gas, military, nanotechnology, Islamic activists and companies developing cryptographic technologies, giving further support to the idea that the Equation group may be backed by government interests. “Although the implementation of their malware systems is incredibly complex, surpassing even Regin in sophistication, there is one aspect of the EQUATION group’s attack technologies that exceeds anything we have ever seen before,” the report stated. “This is the ability to infect the hard drive firmware.” The malware can survive a hard drive reformat and operating system reinstall. This article originally appeared at: http://www.thewhir.com/web-hosting-news/sophisticated-self-destructing-equation-malware-infects-thousands-servers-worldwide (Comment on this)

<< Previous Day

2015/02/17 [Calendar]

Next Day >>