What Works, and What Doesn’t, for Media Storage in the Cloud


BenFoakes_DavinciTheatre 08-12-16The cloud and its advancing capabilities are startling compared to what they were just two years ago. There’s renewed awareness that as broadcast and production infrastructure virtualize and continue to be driven more by global connectivity, a variety of Software as a Service (SaaS), workflow, Infrastructure as a Service (IaaS) & Platform as a Service (PaaS) offerings are a far better fit for the project-based demands of this business than traditional CapEX bound storage purchases.

Yet storage that’s appropriate for media in the cloud is an area that’s both advanced – and still has a way to go. All cloud storage is not created equal. Consumer-grade file sync and share products may be OK for your family reunion video, but lack the professional features required for multi-petabyte HD and UHD (4K and 8K) client work. Object storage may be required for certain projects, while editing, rendering, transcoding or color correction typically require file storage. And not least, Digital Asset Management often employs databases, which require block storage.

“An elastic storage system that can be sited at various cloud providers around the world is critical to the growing global content creation Industry,” says Steven B. Cohen, a 35-year veteran of Hollywood, who has created digital workflow solutions for top producers and directors.

“When we built our first 100% cloud-based media solutions, we assessed the standard public cloud offerings and realized very quickly that they did not seamlessly integrate with our media workflow applications, the bandwidth and IOPS weren’t there at a price point that made sense and the cost models for data retrieval (Egress) would kill our business model,” says Ben Foakes, founder of BASE Media Cloud and 15-year hands-on post-production expert.

Fortunately, the industry has made tremendous advances with storage that delivers cloud global access and economics with media and entertainment suitable high performance. Here’s a guide to help you select what works, and what doesn’t, for your next project.

What Works In the Cloud

1). File distribution hubs that provide a single place for storing and sharing large media files during production, post-production and final delivery, suitable for collaborating across global locations.

For example, BASE Media Cloud provides a broad portfolio of media and entertainment storage and workflow solutions powered by an advanced cloud-based infrastructure. The company provides cutting-edge data wrangling services for clients including ITV Studios, MAKE TV and Tracsis, handling the online transfer and backup of tens of Terabytes of HD and 4K media across multiple countries every month. Within the Soho Post Production community, the team provides automated DR, file sharing and archiving services to facilities including Green Rock, Fifty Fifty, and Directors Cut Films, and for this year’s global Formula E motorsports series, BASE provided the entire storage and distribution platform enabling hundreds of hours of HD content to be delivered to multiple broadcasters, sponsors and rights holders via a simple web UI. All of these solutions have leveraged specialist Hybrid Cloud architecture, built around the company’s underlying Zadara Storage cloud platform that provides data center-grade file, block, and object storage as a service.

2). Dailies Review & Approval. Original Camera RAW Dailies can readily add up to many PB of storage – and the low res proxies must be shared with collaborators around the globe. Cloud-enabled modalities are well suited to this need, with web-based Media Asset Management and View & Review tools often being utilized by companies to enable Creatives and Execs to check out material from production within hours of shooting. As modern cloud-based embedded file transfer and hyper-scale transcoding continues to evolve, these types of solutions are getting better and better.

3). Editorial workflows. Traditional video editing has always required expensive, proprietary hardware and software systems to be deployed on customers’ sites, with high bandwidth local networks and crucially, expensive real estate.  With recent advances in PCoIP streaming technology, spearheaded by Teradici, editing is now made possible in the cloud, via low bandwidth domestic broadband speeds by securely streaming ‘just the desktop’ to an end user. This revolutionary step in remote working has created a tipping point for post-production, where high-res media can now be stored in the cloud and the editorial workstations can be moved to the content.

In this approach, you eliminate the need for highest speed Internet access for work in progress, except to push content into a far-away cloud.  An editorial system in the cloud can use compute and GPU acceleration from large-scale IaaS providers such as Amazon Web Services (AWS), Microsoft Azure or IBM SoftLayer, with high-performance video storage deployed at or near the public cloud resources.

For instance, BeBop Technology provides cloud-based Adobe Premiere Pro or Avid Media Composer editorial workstations and storage that deliver the same quality and fidelity as a local workstation. The core BeBop solution stack includes full integration with AWS or IBM SoftLayer Clouds, Teradici’s PCoIP, Zadara Storage’s enterprise storage as a service, Vidispine’s broadband content management systems, NVIDIA workstations, IBM SoftLayer workflow orchestration and automation, and Aspera for expedited file transfers.   For media and entertainment organizations preferring a turnkey approach, an end-to-end managed service is also available.

BenFoakes_4Kshoot 08-12-16

4). VFX. Even with its massive storage requirements, it’s now possible, instead of purchasing many petabytes for a project, to leverage new On-Premise as a Service (OPaaS) options, where the firm can acquire storage, yet pay by consumption with a minimum commitment length shorter than a television season. In that manner, the production house avoids issues related to the size of the internet pipes and is able to gain the benefits of elasticity and OpEx-only payments. Of course with large enough Internet bandwidth and certain size files, the use of public clouds for compute and specialized cloud-based storage is also an option.

5). Rendering. Production firms are indeed rendering in the public cloud in the US, China, and Germany, with both unleashed content and with post-theatrical work. The MPAA is working on a suggested cloud security recommendation, but that has not stopped the producers of The Walk or Deadpool recently. The hyperscale processing capacities of public clouds like Google and AWS are already being utilized by large-scale VFX houses to hugely condense the time required to render complex frames.

6). Mastering. Where traditional on-premise storage and processing servers have physical capacity ceilings, the cloud now enables almost unlimited resources to be thrown at high-quality media file processing, on demand. For example, groundbreaking SaaS developers SunDog Tools are already using AWS compute at scales of literally thousands of cores at a time to process high-framerate 4K feature film content and create Digital Cinema Packages for theatrical release. The innovative software can also interconnect with private clouds for added data security and sovereignty for clients who require guarantees of the physical location and disc-level location of their data.

6). Media companies with their own data centers (Private Clouds) that often need additional capacity. In those instance data center-grade cloud storage that is collocated at their facility, such as the aforementioned on-premise as a service (OPaaS) offering, but with a cross-connect to the public cloud compute resource such as at AWS would beneficial. OpenStack instances like this have been deployed for ABC/Disney for several TV Series.

7). CDN-like applications, where hundreds of outlying servers help deliver content faster. Having cloud-based storage tied to those nodes would be extremely beneficial.  The CDN wouldn’t have to capitalize the storage – but simply focus on distribution and delivery mechanisms.

When The Cloud Doesn’t Work

  • Direct editing of High-Resolution files: if the editing application is local to the customer but the storage that the editor is accessing is in the cloud via the Internet, it may be latency bound and ineffective.
  • Where customers have not yet invested in good quality broadband or leased line connectivity, uploading large media files can prove impractical. However, as even the lowest cost domestic broadband services tend to run ‘Asymmetric’ bandwidth, meaning a bias towards higher-speed downloads, this does still open up the capabilities for remote editing, administrative control of web tools and high quality streaming video. As connectivity becomes ever more ubiquitous and costs are driven down, cloud will become even more popular across the sector.
  • Not much else, really.

With advancing software-defined storage capabilities, almost any media and entertainment cloud storage application is now possible – both doable and cost-effective. Storing video in the cloud can equalize the playing field between big shops and savvy boutiques – allowing M&E groups with limited means to get high performing technologies without capital purchases – and often at a performance advantage over shops that are stuck with older CapEx-based storage solutions that are not yet fully depreciated and not ready for a refresh. If you are not yet taking advantage of the cloud, it’s worth reexamining assumptions you may have had about what types of applications benefit from cloud-based video storage.

Author’s Bio:  Noam Shendar is chief operating officer of Zadara Storage, an enterprise Storage as a Service (STaaS) provider to media and entertainment companies.

Images courtesy of Ben Foakes, BASE Media Cloud


1 Comment

Leave A Reply