Shared Storage Part 3: Management, Permissions, and Support

5 THINGS - Simplifying Film, TV, and Media Technology

Have you checked out the 2 prior episodes on shared storage? You should!

This 3 part series consists of:

Part 1: SANs, NASes, Bandwidth, and Connections.
Part 2: Drives: Size, Spindles & Protection (RAID)
Part 3: Management, Permissions & Support

On this episode, we’ll be wrapping up our exciting storage adventure with part 3 in this 3 part series, and examining Storage, with Management, Permissions, and Support. We’ll be holding off on ODA, LTO and Cloud storage for a future episode.If you’ve missed part 1 or Part 2 of 5 THINGS on storage, I highly suggest you go back and get your tech on…lots of good stuff.

Let’s get started. Wadsworth Constant be damned.

1. Why do permissions matter?

Permissions are the fundamental building blocks of using shared storage. It’s the reason you can read from a SAN or NAS, and it’s the reason you can write to a SAN or NAS. With improper permissions, your access may be restricted, you may corrupt your own media, or your NLE will simply refuse to use the storage. We can classify permissions into one of two categories: File-level locking, or Volume (also known as Block level) locking. Each has their place, and each has their own considerations. Let’s look at these.

First, we have Volume, or Block level, locking. This is the fastest way of sharing storage, and the least expensive. Volume level allows multiple users to mount the drive or volume and READ from it, but only ONE user can write to it. This is great for NLEs that don’t support concurrent editors opening and using the same project at the same time. (we’ll get more into that in question 2). It also forces some manual file management and organization. Honestly, I like this method – It gives you the most throughput and it causes editors to think about file management and make conscious decisions of media usage as opposed to one big bit bucket to dump footage into.

Once we move into file locking, the price goes up, and so does the file count, usually. File level is what we commonly refer to as one big bit bucket. Every user, if given permission can read and write to the same volume. While this is the easiest in terms of access, it becomes very unwieldy when it comes to media management. No one wants to delete media because someone else may be using it. Performance is also decreased because of the extra layer of checking that goes on during usage. The plus side is that this is the basis for all shared projects – such as Avid or Adobe Anywhere.

Now, while File Level locking is the underlying technology behind shared projects when projects are shared via a NAS protocol, no management is responsible for taking on different users using the same project and media at the same time. This leads to corrupt files and projects. Don’t assume file level locking allows for shared projects. Don’t try it.

2. How do I share my projects and media?

Currently, and I stress currently, one1 out of the big 3, support this out of the box. Avid has supported this for almost 15 years. This is one the reasons it’s the de facto standard here in Hollywood for feature films and television. When using Avid shared storage – ISIS – multiple concurrent editors can open the same project at the same time. The only restriction is what is called bin locking. Everyone can read the bins, but only one can write to it. Thus, only one user can be editing a timeline at any one time. This aside, shared projects allows almost simultaneous sharing of cuts and media. If ISIS is out of your price range, several other manufacturers have spoofed the AVFS – the Avid Video File System.

Premiere supports shared projects (although, when this feature is introduced, it becomes a “Production”), however, this is dependent on shared storage as well as using the Adobe Anywhere solution – which I outlined in Episode 1 of 5 THINGS. Anywhere allows the sharing and receiving of changes to a production by several concurrent users. As of December 2014, this is ONLY available with an Anywhere solution.

FCP X, like FCP 7 before it, does not support multiple concurrent editors opening the same project at the same time and making changes to it. Final Cut Server followed a check in, check out methodology for FCP 7, however, Apple canned that product, so it didn’t get much screen time.

3. What management features should I be looking for?

Glad you asked.

Obviously adding and removing users from utilizing the volumes is necessary to prevent misuse or the accidental deleting or moving of files. This includes granting or restricting use of volumes to users.

Checking the status of the drives: including free space and health is paramount.

Another great feature is the ability to grow and shrink volumes. Let’s say you allocate 5 TB each for your editors. One Editor never uses up more than 2TB. But another editor is always running close to 5TB. Perhaps you want to re-allocate some space to the second editor. Some shared storage solutions would require you to move all of the data off of Editor 1’s volume, delete it, create a new, smaller volume, then copy it back. This is a major pain. More robust shared storage solutions allow for the ability to dynamically increase the size of the volume….or, if the solution is really good, shrink it as well.

As we discussed in Episode 2 on shared storage, RAID protection is of paramount importance. Perhaps you want to have an added layer of redundancy for your projects – to be more protected in a catastrophe – let’s say, RAID 1 – for your projects. However, your raw media is backed up elsewhere, so you would not be dead in the water if the media was lost. In that case, the media may be a RAID 5 – or even RAID 0. This can help free up more space.

Throttling is also a great feature to have. Perhaps you only want to allow offline editors a fraction of speed, but your online editors need more bandwidth to accommodate higher video bitrates and track counts. Being able to set these on a per-user basis, as well as monitor performance in real time – is a fantastic feature.

Other beneficial features may include interoperability with backup, archive, or TIER II storage solutions, migration tools to move to larger storage, diagnostic or testing tools to verify speed and connection integrity, or deployment tools to aid in the connection of new clients, including drivers, client software, etc. Also, the ability to run other apps on the computer running the storage, sure as a render or transcode volume, or asset management software.

4. What kind of support do I need?

Many shared storage companies have a spares kit – which may include a spare Hard drive and caddy and may even include a spare power supply. I highly recommend getting this with any solution. It seems to be Murphys Law during crunch time, and how long can you afford to be down? As we’ve discussed in previous episodes, professional shared storage solutions don’t use off the shelf hard drives – so running to the local electronics store to get one isn’t an option.

Many shared storage integrators may have their own support plans – including a technician on call to remote into the system and troubleshoot…or perhaps a spare unit or replacement parts on hand if needed.

Lastly, our shared storage has a ton of moving parts. A higher probability of something going wrong. Many users have Tier II storage in conjunction with their fast, edit storage. This ensures that if your media is backed up to the TIER II storage, you won’t be dead in the water if your edit storage goes down.

5. What shared storage solutions do you recommend?

Yes, the question I get asked the most frequently. Here are my top 3 solutions.

As I’ve said many times before, I prefer the one throat to choke philosophy. If you’re in an Avid environment, go with an ISIS. The management tools are deeper than any other editorial shared storage solution out there. The deployment tools are built in and accessible via a web browser. It’s one of the most stable storage solutions I’ve ever worked with.

Avid ISIS 5500 Primary Engine — 32TB System with System Director Function. Includes 1 Year ExpertPlus w/ hardware Support: $38,000

If you’re in a mixed environment or looking to save a few bucks, I’m a big fan of Facilis. The Avid project sharing was ironed out a few years ago, and the throughput, when used as a SAN, is better than most solutions out there. You can also choose to work with Ethernet or Fibre connected clients. In addition, the management is dead simple.

Facilis Terrablock 24EX/16 – 32TB (16x2TB): $22990

A newcomer is Platform by ProMax. In a relatively short time, they’ve developed a very solid product at a very attractive price point. They’ve also done something not many other solutions do: they utilize the horsepower on the shared storage server that is not being used for file management, and offload processes to them, such as Asset Management, or cross platform After Effects rendering. The price point, as mentioned, is also very attractive.

ProMAX Platform Studio [X] T1-8 $6995
ProMAX Platform Studio Module SHARE 32TB $6995
ProMAX Platform 4 Port 1GbE Expansion Card $695
(Platform systems are a la carte, so this price can scale)

Did I miss anything about Management, Permissions, and Support? Or anything in this 3 part series? Let me know in the comments, via Twitter, or on Facebook, and please, share this series with your friends. I plan to be back after the holidays with new Episodes of 5 THINGS.

As always, thanks for watching.

Have you checked out the 2 prior episodes on shared storage? You should!

This 3 part series consists of:

Part 1: SANs, NASes, Bandwidth, and Connections.
Part 2: Drives: Size, Spindles & Protection (RAID)
Part 3: Management, Permissions & Support