June 5, 2013

Remix, Isolate, and 'Photoshop' Audio with Celemony's Melodyne & Sony's SpectraLayers

melodyne celemony music audio sound production studio suite software daw digital workstationAnyone who's ever learned the basics of sound manipulation in a waveform editor, such as Audacity, knows how difficult it is to separate sounds from a mixed-down recording. I grew up with Sound Forge back when it was still owned by Sonic Foundry, and quickly came to understand how impractical it can be to isolate a clean vocal track from, say, a CD rip. A company called Celemony and their audio editing platform Melodyne is now making it incredibly fast, easy, and intuitive to do just that -- and much more, like autotuning those isolated vocals for remixing. Meanwhile, Sony is likewise doing the "impossible" with its impressive frequency editor/clean-up tool SpectraLayers.

In the specific case of stripping the vocals off a mixed down track to create an instrumental (or the opposite, discarding everything but the vocals themselves), there are several workarounds and methods for doing so. That said, Celemony's Melodyne makes this process visually intuitive -- and rather powerfully automated -- in preparation for further creative manipulation down the line, because it's specifically engineered to do so. This news comes to us from RedShark, including the following demonstration/tutorial:

The phrase "blobs" is certainly not in the traditional lexicon of audio -- nor sound at large, for that matter -- but in a strongly unique interface such as this, fresh terminology must follow. As both a commenter and the article's writer (Managing Editor David Shapton) over at RedShark mentioned in the comments of the original post, Melodyne isn't the first (or only, at least) piece of software to break mixes down in previously difficult-to-achieve ways. Other offerings such as Sony's SpectraLayers basically puts the power of Photoshop's toolset to a multi-layerable spectrogram view of a mixed-down track, except instead of visually manipulating something visual, SpectraLayers allows for visually managing the manipulation of something auditory:

http://www.youtube.com/watch?v=Fy687hvNcD4

On top of going over some pretty solid basics of sound and audio -- such as phase cancellation, which the above workarounds use more simplistically to do something similarly to, though with less control than, SpectraLayers -- this video is pretty easy to get excited about. Celemony's software is more dedicated to facilitating 'autotune'-esque control via isolation, but both of these products spell a bright future for audio production (not just for DJs). The possibilities of cleaning up a dual-system production track, which more often than not is being recorded by one microphone as a single mono channel, is very good news for filmmakers, too.

Again, tools for remixing and noise clean-up have existed forever, but never with this level of fine-tunable control. The Photoshop analogy rings strangely true, I feel, given the importance of the visual gamma and 'level multiplier' controls to SpectraLayers' efficiency. Why not shift the "contrast" or force "choke points" of a spectrogram plot to for tracing, and to work more easily with encoded material the way you can in Photoshop? Well, thanks to apps like SpectraLayers -- and from a bit of a different angle of attack, Celemony's Melodyne -- you can.

What audio clean-up (or mash-up) experiences have you had in which you really could have used a tool like this? What are the implications of technologies like this on your workflow as a filmmaker?

Links:

[via RedShark News]

Your Comment

7 Comments

Thanks for posting this, Dave. I've been working in audio for years and have never seen editing tools as useful. Especially, Melodyne. Wow!

June 6, 2013 at 2:59AM, Edited September 4, 11:21AM

5
Reply
Lorenzo S.

...this is veeeery old "news" - celemony is 12 years old...you probably also read that Redshark article ;)

June 6, 2013 at 3:32AM, Edited September 4, 11:21AM

0
Reply
mariano

The Sony demo is so unrealistic. The file he is working on has been prepared specially for the demo. The bird song, the woman's voice and the siren are conveniently very different in terms of their spectral signature. There's very little ambient sound or reflections. If it were real life, all of these would be blended together and the task would be much less straightforward.

June 6, 2013 at 4:17AM, Edited September 4, 11:21AM

1
Reply
Graham Kay

I was just about to post the same. They all sit in totally different frequencies so isolating them are very easy, you could probably do it with a parametric EQ just as easy, with bit of automation.

They should've used a song or a actual natural situation where most sounds don't sit as nicely in the spectrum, they go all over the place due to reverberating off of various different surfaces etc...

Cheeky demo.

June 6, 2013 at 6:06AM, Edited September 4, 11:21AM

0
Reply

Both of these softwares have existed for years! Melodyne is used heavily in studio settings for pitch correction while SpectraLayers for sound restoration.

The key is to capture the sound right the first time and not have to rely on post-sweetening for correction.

June 6, 2013 at 6:11AM, Edited September 4, 11:21AM

1
Reply
Dennis

I've been using melodyne but for musical matters. It's nice to give it more uses jeje

June 6, 2013 at 7:15AM, Edited September 4, 11:21AM

1
Reply
Jesuan

Both of those tools give fairly ugly results. There's no shortcut for great audio. That's why we ADR. If you were to try to remove a siren from a dialogue track and then play it in a theater, the frequency artifacts would be totally obvious.

June 6, 2013 at 12:53PM, Edited September 4, 11:21AM

0
Reply
steve