Do you see the laptop graphic to the left here? This is is a dramatization of what you'll want to do should you ever mis-manage your file naming or folder structure on a project. Luckily, there are several tips from around the web to help with file management. Let's pick ourselves up by the bootstraps, and dig in with some helpful (and ultra exciting!) tips for file management.
First, let's hit on general file management. The folks at the American Society of Digital Photographers have an outstanding site dedicated to this called dpBestflow. Granted, this site is geared more toward still photography, but a lot of the best practices also pertain to motion picture as well.
The main page of their Best Practices contains several sections including File Management, Metadata, and Backup. Of interest in the File Management section is the Overview, which contains some great tips concerning organization by folders. They hit on an important point about organizing by Job, vs Client. If you decide to organize by Job Number, you're going to have to have your own spreadsheet or cataloging software to retrieve projects whenever archived. I tend to use a simple Microsoft Excel spreadsheet for this. The clear advantage to this is that you're going to have a cascading list of numbers that will never change -- even if the job suddenly cancels for whatever reason. This really helps with billing as well. Alternatively, you could organize by Client Name. The advantage to this is when you work with several clients, and feel more inclined to organize by a folder structure. The caveat, however, is that you will not have a consolidated list to refer to in the future. These images from dpBestflow illustrate this difference:
 |  |
| Date-Based | Client-Based |
Once you decide how to organize your master structure, you'll definitely want to make sure that every project is also organized the same way. I've included how I typically organized projects below:
At the root level, I sort by anything that is considered Source, anything that is considered an Export, and Project Files.
Let's start from Project Files. When I do daily backups, I tend to zip the entire Project Files folder and upload to Google Drive or Dropbox, in addition to another local server or hard drive. It's certainly unwieldy to backup the entire project daily, but this structure works for me to make sure that all of my work (the Project Files) are safe from the terrible reality of hard drive failure. At the end of the project I will usually Collect Files in After Effects, Meda Manage in Final Cut, or do whatever file collation my creative program will do, and then back that up to three drives (as described in my previous post on Amazon Glacier).
My Exports folder contains all final deliverables, arranged by date. Inside the folder I will organize by "Rounds" if I think there will be multiple versions of multiple videos that day. However, if I'm working on a single video deliverable, I will typically still organize by version. This can be done easily by having a simple "_v#" number per export. You can always just organize by date and see when the latest file was exported, but this is just another assurance you're on the right export. You certainly don't want to just have a file be "final", as The Beat by PremiumBeats explains below:
With video editing, photography and graphic design, it's typical that a final version may not stay a final version for long. Clients may request changes, weeks, months or even years down the road. Worse, they have multiple rounds of changes after you’ve sent them a “final” deliverable. So, using the word “final” in your naming can quickly get you into hot water. Instead, use a version number so there will be no confusion as to what file is the latest version.
They have several other great tips as well, like hyphenating for example. But I think that kind of goes without saying, if you're at a level where you're responsible for building projects... no spaces!! No. None. Not now, not never. (Double negatives are clearly okay though?)
Finally, my Source folder contains the source files for every program, organized by the type of program and type of file it is. This may be a lot of minutia, granted, but I'd rather be over-organized than have to rely on searching for every file on my system. I also want to acknowledge that I've heard it is typically best to have separate capture sources/drives for audio and video, however this keeps things simple for me, and I haven't noticed extreme performance drops.
What do you guys think? Is this at all similar to how you organize your projects?
Links: dpBestflow, The Beat - File Naming for Video Editors, Designers & Photographers
Your Comment
35 Comments
i'm using post haste to organize my projects.
it is free and you can select different types of projects, like film, animation, photography, 3d etc.
you have the same structure every time.
September 24, 2012 at 7:18AM, Edited September 4, 7:54AM
Sebastian - solid suggestion, and worth a look: http://www.digitalrebellion.com/posthaste/ I like what I see so far perusing the manual/screen caps.
September 24, 2012 at 7:32AM, Edited September 4, 7:54AM
Additionally, Pro Versioner could come in handy, seems to be like an automator for project files or something: http://www.digitalrebellion.com/proversioner/
September 24, 2012 at 7:38AM, Edited September 4, 7:54AM
I'm interested in employing Time Machine to do the daily backups of my project files. I wonder, is it possible to have time machine back up only specific folders on a drive, like your auto-save vault or for example all the folders named "3_Project files" as in this example.
September 24, 2012 at 7:50AM, Edited September 4, 7:54AM
I know that you can choose which folders and the things that you want to EXCLUDE from the backup, so as long as the things you want are included you'd be good to go.
September 24, 2012 at 10:33AM, Edited September 4, 7:54AM
Be careful when using Time Machine to automatically back up files...it may look like it's backing up based on hours or days, but over the course of months it deletes individual days so that you have only one backup per week to save space...so if you had multiple versions of file exports during that week and Time Machine only saves one of those days, you may lose some of them...I had this happen while working on a short film a couple years back. I spent hours looking for my files, I didn't know where they went, since it had been a few months and Time Machine automatically deleted 'excess' backups. I prefer doing it by hand whenever possible...no automatic backups unless it's truly reliable and you know exactly how the backup works. Doing it by hand also means you get to organize it EXACTLY how you want it. But beware of Time Machine...
September 27, 2012 at 12:39PM, Edited September 4, 7:54AM
We have been using Post Haste for few years now and it's brilliant. Even some of our less-organised members find it a useful tool to keep themselves organised within projects, not to mention whoever needs to pick up the project from them!
We've enforced a rule where no member are allowed to import/copy/transfer any file to our system unless it's been 'Post-Hasted' by our admin. Works like a treat.
September 24, 2012 at 9:11AM, Edited September 4, 7:54AM
Also, something we found very helpful is adding initials of the person doing changes to new versions of documents, i.e. TheProjectFile_v4_PJ especially if we're amending someone else's work. This helps to hunt down the person responsible for the latest changes.
September 25, 2012 at 2:08AM, Edited September 4, 7:54AM
Yeah, we do the initials thing as well, particularly with AI files as those are used by mograph folks, as well as designers.
September 25, 2012 at 7:09AM, Edited September 4, 7:54AM
I have a similar setup. But I let time-machine cover my daily backs, as opposed to manually zipping and moving them to dropbox.
I highly suggest when you've created your folder structure to create a ZIP file of it and name it "TEMPLATE" or something to your liking. This way, with each new project, it's quick to Unzip that folder structure and not have to build it from scratch!
September 24, 2012 at 7:20AM, Edited September 4, 7:54AM
Yes, thats what i do, but my system consists in the master folder named after the project then 3 subfolders named material, project and render. super clear never had a problem, in the render folder i have 2 subfolders, one is called client, thats where all the diferent client versions go, and then emision, where only the finished files go (mov,m2v etc) also in the material folder i have 3 subfolders named 4afx, 4fcp and Audio.
here is the template folder if somebody wants it: https://dl.dropbox.com/u/4878243/Template%20carpeta.zip
September 24, 2012 at 9:00AM, Edited September 4, 7:54AM
Great post, but what's wrong with spaces in file and folder names? That's an honest question. I often use spaces in names when organizing (I have a MOTION GRAPHICS folder in my projects, for example).
September 24, 2012 at 8:07AM, Edited September 4, 7:54AM
The primary reason is that if you ever want to do anything on the command line, spaces in names make things far more complicated. On a command line, spaces delineate between commands, arguments and files, so to have a space inside a name itself you must type "\ " instead of just " ". This can get tricky and confusing fast.
September 24, 2012 at 8:18AM, Edited September 4, 7:54AM
As you enter the first few unique chars of a name, press Tab to have the rest filled in for you...including the escaped spaces. It's not that bad on the Mac shell.
September 24, 2012 at 8:33AM, Edited September 4, 7:54AM
um yeah... that only works if the file has a unique name that isn't like anything else in the folder. Try going into your applications folder to get into the Adobe After Effects app in command line... still gotta escape the spaces manually which is a pain because of all the similar spelled folders.
September 24, 2012 at 8:39AM, Edited September 4, 7:54AM
cd /Applications/*Aft*
September 24, 2012 at 8:51AM, Edited September 4, 7:54AM
There shouldn't be a problem with using spaces in 2012. There are enough nerdy programmers however who still believe in the dreaded underscore etc. and don't debug their own code to handle spaces at all times. If you're using mainstream products on a Mac certainly you won't have problems with spaces, but mix in legacy software and goober software and you may.
For backup I use Time Machine for the OS but Carbon Copy Cloner for the data (it's a Mac shell for rsync). I have a CCC OS drive as well ready to boot immediately to avoid downtime. I use Aperture for stills and FCPX for video primarily, though I have MC5.5 as well. And I use those programs' basic approach to data management rather than swimming upstream (though I don't use the Aperture vaults, which are worse than just having duplicate libraries afaics). Each project has its own name and a monotonically increasing version number (not the date) and each is archived by year of start.
September 24, 2012 at 8:30AM, Edited September 4, 7:54AM
I think Peter is correct that more often than not, you'll be alright if you have spaces in filenames. Legacy can be an issue as he says. Also, I work with a lot of Mac to PC server workflows, and I've had some problems there before, especially with files going unlinked. Then there's the occasional hiccup when placing a video file online in the JW Player or equivalent, when for some reason the spaces aren't auto-renamed to percent 20 (%20) signs, I assume. (I'm not a hardcore html coder, so forgive me if I'm off here a bit with how browsers handle spaces in url filenames.)
September 24, 2012 at 8:40AM, Edited September 4, 7:54AM
If you want specific, flexible, granular, automated, etc, control of your backups (verified, too), you should check out ChronoSync for Mac. Each photo/video client gets a license from me as a perk!
The books listed on the dpBestflow site are nice, too, and gets you in the mindframe for archiving and using DAM tools.
Speaking of DAM, without Final Cut Server, what do you rely on these days?
September 24, 2012 at 10:14AM, Edited September 4, 7:54AM
This file structure works great for me for all or at least a variant anyway my question regards Pro Tools that is very rigid about its own file hierchy. Is it better to create your Pro tools file do all of your imports finish your session and then once complete bring into this system? Deleting music files if they are saved in your source music? Examples such as this or keep the entire file structure and maintain all links? Mostly looking for best way to manage those files without having to do a large amount of re-linking later
Thanks for all help
September 24, 2012 at 10:47AM, Edited September 4, 7:54AM
I'm curious, does anyone still maintain complete, untouched disk images of all their AVCHD footage?
I've done so religiously for years, but I'm really beginning to think that it's a paranoid waste of time. There doesn't seem to be much disadvantage with breaking the MTS files out, remuxing them into something easier to use like Quicktime or MP4, and just storing those without worrying about preserving the original file structure. In fact the main advantage of such an approach is that you can delete all the wasted takes and useless screwups instead of wasting storage preserving them forever inside an unwieldy .dmg or .iso image.
So what say ye, NFS fans? Am I crazy for thinking of doing this? Or was it crazy to ever care about "perfect" AVCHD archives to begin with...since there's always been one way or another to get their contents into even the most annoying NLE if you know how to finesse them?
September 24, 2012 at 9:31PM, Edited September 4, 7:54AM
I was told it's just a header change putting them into .mov's. I didn't diff them to check. Try that.
September 24, 2012 at 9:35PM, Edited September 4, 7:54AM
It's not a question of how perfectly the video is repackaged, I'm plenty confidant of that. It's all that extra nonsense inside the AVCHD file structure...I suppose it has the potential for value but I've never seen it, personally. Maybe it's just a bunch of unrealized planning someone put into the specification? Dunno. Feels like a waste of time and energy to keep it in tact though. Would be much nicer to just keep the video files, delete the ones that aren't valuable, and nuke all the rest of the folders and little data files and stuff. Any reason not to?
September 24, 2012 at 11:18PM, Edited September 4, 7:54AM
I'm pretty sure I know what you're talking about... like the structure the GH2 builds when capturing movie files? I'm not aware it's the same for every AVCHD shooter, but ultimately I think it depends what editor your company/you typically use. For FCP, I'd archive in the way you're suggesting and preserve the (annoying) folder structure. For Premiere/AVID I'd be tempted to nix it and just grab all the .MTS's. One of the reasons I've decided to move from FCP to Premiere is the ability to better natively handle DSLR movie files.
September 25, 2012 at 7:07AM, Edited September 4, 7:54AM
As the heading suggests.That was exciting!
September 25, 2012 at 1:26AM, Edited September 4, 7:54AM
Thanks!
September 25, 2012 at 7:03AM, Edited September 4, 7:54AM
I store project files on a completely different hard drive from source files when I edit and when I back it up for easy separation.
September 25, 2012 at 6:12AM, Edited September 4, 7:54AM
For Windows Backups I use Allway Sync, its $20 USD but it verifies file version bi directionally, It can be set to a timer, manually triggered or when changes are made :D
September 25, 2012 at 11:48PM, Edited September 4, 7:54AM
I love this little program for the PC called Cathy. It creates a log of all hard drives, discs, anything, which is searchable. In a project folder, I can create a simple searchable text file with the date, client name, project name, whatever, which can be used in future searches. I just key in word or two, and Cathy tells me which hard drives it is located on.
It is a very simple, tiny program written by an individual. This website does not look very professional, but don't let that fool you. It's a great program. You will find it about 1/2 way down this page: http://www.mtg.sk/rva/
September 27, 2012 at 12:53PM, Edited September 4, 7:54AM
typo check > adobe glacier
September 27, 2012 at 1:06PM, Edited September 4, 7:54AM
Don't know how we missed that, thanks.
September 27, 2012 at 1:11PM, Edited September 4, 7:54AM
Double-thanks Paul!
September 27, 2012 at 9:35PM, Edited September 4, 7:54AM
Just a minor corrrection and expansion. dpBestflow is an initiative of the Library of Congress, who in essence "hired" ASMP - The American Society of Media Photographers (not "Digital" Photographers) to build the site for them. The site is free to use, and you don't even have to register, and it has vast resources for both still and motion of all kinds - way past DAM issues. This post is a great take on DAM, so thanks.
The entire site is currently managed by Peter Krogh, who literally wrote the book - The DAM Book.
ASMP is a trade association with over 7,000 members, mostly still photographers, but video is something many of thte members do routinely now. Truth-in-advertising, I'm on the national board. For those that are entreprenuers - or work for entrepreneurs - ASMP today hosted a day-long symposium at the New York Times Theatre about the current state of the media business. It was all live-streamed and recorded and will be available to watch for free as soon as it's posted. www.asmp.org
September 27, 2012 at 1:08PM, Edited September 4, 7:54AM
Hi Barry, thank you for your minor correction and expansion. I will have to check out The DAM Book, if for no other reason than for respect of the title haha. Also, I did dig through some of the other resources at dpBestflow, and certainly appreciate you mentioning them again as well. In particular, I found this link of interest - basically a complete rundown of the video production process: http://dpbestflow.org/node/635
September 27, 2012 at 9:43PM, Edited September 4, 7:54AM
Anyone have advice on verification software to use for downloading cards and other backups? Chronosync (Mac) has a basic verification but I recall there is a more professional software but I can't remember the name. Are most people verifying during primary downloads off CF cards?
Thanks
September 29, 2012 at 2:09AM, Edited September 4, 7:54AM