Video dominates the media landscape…not just on your TV but on your smartphone, tablet, desktop and laptop computer. It’s broadcast, streamed and sent over satellite and cable. That’s great news for the creative people who make content and the audiences that consume it and equally great news for production and postproduction companies. But it also creates a new set of challenges.
If you work at a production or post house, you know that the pace of technological change since the transition to HDTV has been relentless. The move to UHD/4K, High Dynamic Range, High Frame Rate, file-based post and delivery…each tick upwards in technology demands more from the production and post facilities and practitioners.
Workflow is the key ingredient in allowing individual productions and production and post facilities to run smoothly and meet client needs. Perhaps no challenge has been as great as the need to store, manage and archive massive amounts of material. The huge increase in content has meant dramatically increased demands not just for storage, but also for content to be searchable with metadata, kept at a storage level that makes it readily accessible and archived in a way that it can retain its value. Without a properly constructed workflow, even a single project can become hopelessly muddled, with multiple versions scattered or lost throughout a facility.
Increased volume is only part of the issue. With the advent of UltraHD TV and 4K, demands for space and management have increased. HDR and HFR are only making the issue more acute.
As motion picture studios and broadcast networks have realized the value of repurposing their sizable video assets, they have made increasing demands on their vendors. For every production and post facility, implementing robust systems for storage, maintenance and archiving is no longer an option. It’s an imperative, even for the smallest post or production facility.
Whether you’re an editor, a film executive or a post house owner, you’ve got questions about how to create the ideal workflow and the best environment to manage and store your video and other assets. What are the pitfalls? What are the must-haves for the best solution? Can I afford to move to a more robust and reliable way of handling my content?
In this feature, SHOOT brings you the experts with the answers. We talk to two companies in the field that have years of experience and reams of success stories. Regardless of size of your company or project and the state of your budget, Quantum and SGL Broadcast detail the solutions that can fit your needs.
Quantum: Optimizing Workflow with Storage Innovations
Quantum has specialized in high throughput, high performance computing for archive and data protection for 35 years. Many in the media and entertainment industry are familiar with Quantum through its StorNext shared file system that controls shared access to storage across an entire workflow. This Quantum product has been ubiquitous in the industry for the last 20 years, a feature at companies ranging from Framestore VFX house in London and Park Road Post in New Zealand to Disney and CBS and many other major studios and broadcasters.
“About five years ago, we started to look at where the future of media production was going,” says Alex Grossman, Quantum VP of Media and Entertainment. “We saw HD, more collaboration, more workgroups, not quite in the cloud, but people working remotely.” The company rewrote a lot of the over 1 million lines of code to StorNext, turning it into a platform for the workflow for today…and tomorrow.
But that’s not where they stopped. “We realized that we had moved into a true tapeless workflow,” Grossman adds. “We no longer had tapes on the shelf for archive. Virtually every facility had some kind of LTO backup, but they didn’t have the concept of the repurpose-able archive yet.” Facilities were doing more and more high-resolution work but lacked the end-to-end workflow to completely optimize efficiency and costs. “Often people let their workspace grow in an ad hoc fashion,” he says. “You’ll have 22 shows on an online disc because with offline, it was hard to get it back.”
The need, Quantum saw, was to make storage more cost effective and manageable, as well as faster for ingesting and outputting content in a fully automated manner. “We said, let’s build tools that let you archive in a couple of different ways,” Grossman says. “We already had an automated tool to back-up the archive on LTO, but there are a lot of technologies as resilient as LTO with the speed of disk.”
Object Storage became the focus of Quantum’s next generation solution, dubbed Lattus. “Traditional RAID is inefficient for Petabyte scale storage, and traditional block storage addressing can’t match the scale, access or durability needs of today, “ says Grossman. “Lattus, Quantum’s next generation cloud object storage, addresses these issues, and ultimately allows the business to unlock more value from the data. Now we can offer people the fastest performance, from ingest to delivery.”
At the same time, continues Grossman, Quantum focused on refining the way people handle content at ingest. “With higher resolution content, you need to ingest the content but you also need a copy of that raw content to be maintained for a long time, so it’s an archive ingest. Using StorNext Storage Manager, the platform now lets you even archive at ingest.”
Workflow Optimized Storage was the next step. “You can lower the overall cost of production by how you use storage,” says Grossman. “In broadcast, for example, you are getting content from many different sources in a lot of different codecs. You need to transcode it to get it into a working format and build a proxy format too. We automate that process completely in a way that doesn’t slow down the people already working on the content.”
Quantum is able to achieve this by building intelligence into the use of different kinds of storage. “In today’s higher resolution workflow environment, when you work with bigger files, we have lots of different storage components and we choose the right one for each step,” says Grossman. “For example, you might need SSD for ingest. Then we use fast storage for editing and, for delivery, we can use a slower storage because it’s not real time. The user doesn’t notice the difference but our system is in the background, moving things along.”
With compatibility with tools from 54 different manufacturers, Lattus also allows control of the system from other parts of the workflow. “These tools can take control of the system, while Quantum Lattus executes those processes in the background,” says Grossman. Quantum’s StorNext Pro Solutions includes all the pieces necessary to build a highly customized yet universal workflow. And if a client has a favored transcoder or nonlinear editing system, Quantum can optimize the workflow around those tools. “Lattus gives a tremendous amount of flexibility,” says Grossman.
For the UHD/4K, High Dynamic Range and High Frame Rate future, Quantum offers unlimited scalability in performance and capability. “We have a StorNext solution, StorNext Pro 4K, designed for facilities who want an entry-level 4K system,” says Grossman. “So many facilities in Hollywood and New York are getting requests from the studios to deliver in 4K. With our system, you can work in 4K as easily as you did in HD, as well as migrate content back and forth.”
In addition to 4K, many content distributors are interested in producing High Dynamic Range (HDR) material. “You need increased capacity, and we’ve done all our testing,” says Grossman. “With Quantum solutions, you’ll have the headroom to do 4K plus HDR without a problem.”
“One of our advantages is that, with our system, you can deliver HDR in 4K in a collaborative manner,” he continues. “Unlike other systems, we can do it all simultaneously, with a large number of users. In principle, our system can handle hundreds of users working in 4K on a large project.”
Quantum has done a tremendous amount of testing on how to work with high frame rates and applied its expertise with existing technology specifications. “In today’s high resolution workflow with HDR and HFR, about half the operations happen in real time—editing, color correction, audio sweetening to some extent, and finishing,” Grossman says. “The rest of the operations—ingest, transcode, render and deliver—all happen in non-real-time. We separate these two operations out, which lets us do 4K, HDR and HFR with less cost and hardware, managing it carefully in StorNext 5, which provides the fastest streaming performance in the market, so the non-real-time doesn’t clog up real-time operations.”
In the wings is Quantum’s Q-Cloud solution, for those clients interested in cloud-based solutions, a choice that few are making today—but more may make in the future. “We wanted an offering we could give all our customers, but we understand it’s not for everyone, at least today,” says Grossman. “Meanwhile, what we offer today is an end-to-end workflow with an unique and very important archive piece at the end.”
SGL Broadcast: Scalable Solutions for Storage Management
For the past 15 years, SGL Broadcast has devoted its energies into the media and entertainment space, from production and broadcasting to editing and education. “Anywhere there’s video,” says Director of Worldwide Sales Bernie Walsh. “We focus on managing video assets throughout their life cycle, from ingest through the process of editing, play-out, and archive, including storing, managing and duplicating copies.”
Maintaining and retrieving video assets has never been an easy job, even in the tape world. But the complexities have dramatically increased as the amount of video has exploded and, with it, the numbers of versions, operations and retrieval needs. But SGL Broadcast has a solution for that: FlashNet.
“FlashNet is a content management application to manage video from its ingest through editing and archive,” says Walsh. “Whereas large organizations, such as broadcasters, have the money to put a fairly complex solution in place, that leaves out most post houses and other smaller facilities where budgets aren’t there to get a solution that does the job right.”
SGL now offers several lower-cost, entry-level bundles that make it possible for a facility of nearly any size and scope to be able to enjoy the benefits of storage management. “We provide you with the server to run our software on, the connectivity into other parts of your organization, including the storage, production and post, and a tape library where all the material can be stored,” says Walsh. “We also provide a very nice user-friendly interface. So many facilities don’t have any kind of asset management or automation, which makes it very difficult to find assets in the tape library. We make it easy with search criteria.”
For facilities that want a turnkey archiving system, the entry-level FlashPack, with FlashNet at its heart, offers all-in-one hardware and software services and comes with 60-terabytes of storage and 12 months of software support and service.
FlashNet is highly scalable, says Walsh. “The majority of our customers who invest at the low end can still use the hardware they’ve invested in to increase the size of the archive,” he says. “That’s the philosophy behind our software. We have customers we one server and 30 or 40 tape drives. But we have other much larger customers, such as NBC, CBS, BBC and WWE, with 12 or 14 servers and a Petabyte of disc archive and 10 tape drives in a clustered, fully automated scenario. The idea behind the cost-effective bundle lets you get into this world relatively simply and then grow without wasting the investment you already made.”
The FlashNet solution, which is ideal for news and sports, also works well with major manufacturers of production and post gear, including Avid, Grass Valley, EVS and VSN. “If you’ve got a facility with ISIS online storage, we would provide an interface from our software to connect with that Avid environment,” says Walsh. “When someone is finished editing a piece, he’ll select that project and send it to the archive, and we provide an interface that does that, totally transparently to the user.” Facilities can set up the system in different ways, to accommodate multiple projects and editors.
The most important thing is that three weeks—or three months—later, the editor can easily find the clips he archived with search criteria. “If the post house has created a low-res version, the editor can play it and make sure it’s the right clip,” says Walsh. “Then he can mark in and out points, so if he only wants to restore two minutes of a one-hour sequence to make a promo, he can do that in low-res, then restore and deliver it.”
Walsh notes that for facilities that either already have or want to develop an in-house management system, SGL provides a set of APIs that will allow them to create a fully integrated system, with their own software and FlashNet. “We let them hook into our system so when they get to a point where they outgrow it, they can buy a production asset management system,” he says. “We don’t just provide them with a sophisticated archive system but the ability to integrate their own in-house system.”
FlashNet is also used as a near-line storage system. “When your material is ingested, it generally goes to a high-performance but expensive disk,” says Walsh. “We can put another layer in there, from the online storage to our archive, that can be a combination of more near-line disks and near-line tape library. Storage Manager creates life cycle rules that can offload content from online storage when needed to near-line storage. You can create a rule that it makes a copy for the tape library or that, if it hasn’t been accessed for a defined period of time, that it can be deleted and create more space.”
“It’s easy to move from online disk to near-line disk,” he adds. “What’s difficult is managing it, making sure you have the right components and getting it back in a way that works. We can provide that. You need software that’s going to manage those videos throughout their life cycle, and that’s what we do.”
Conclusion
Up until now, only the facilities that could afford a customized system could manage, store and archive their assets in an integrated, efficient way. Now, as video assets increase and become more valuable, every facility, no matter how small, must find a way to store, manage and archive video in the most robust way possible. That imperative becomes clearer as UHD/4K (and beyond), High Dynamic Range and High Frame Rate become more common. An ad hoc system or a patchwork of solutions just don’t work anymore.
Assets created today can be repurposed months from now, and every production and post expert knows that accessing assets for repurposing is an essential—and increasingly important—task.
The chance to automate those tasks with proven hardware and software is a compelling solution to storage and archive issues facing today’s post facility. The good news is that there are now solutions for every budget and capacity. With scalability at the core of the solutions we’ve discussed here, solving this ongoing problem answers today’s problems and prepares the facility for the future.
