Last month, Yusuf / Cat Stevens' 1967 debut album Matthew & Son and the swift follow up which appeared the same year, New Masters, were reissued on high quality 180 gram vinyl, which marks the first time the records have been available on vinyl since 1982. Abbey Road’s James Clarke has breathed new life into the original masters with the reissues making use of his unique De-mix technology. The software has enhanced the original vocals and helped to amplify the bass, which mixes in the late 1960s were often unable to do.We spoke with James about how this groundbreaking technology works, its use on the Cat Stevens reissues, and how he plans to further develop the software.
De-mixing with James Clarke
So, the first question is more of an overview of the mixing. Can you give us a brief understanding of the process and how De-mix technology works? As much as you can give away as possible.
It's not about keeping things secret, the process is that you create spectrogram models of the target instrumentation you're looking for, so vocals, guitars, bass, drums, stuff like that. And then the software starts to look for patterns within the mixed version that matches those models. It then creates what are called masks, which effectively, think of them like a specific sieve, you just drop the audio through it and the mask catches the bits it wants to keep and lets everything go through. It then does the same for all the other instruments and eventually it works out that this bit of audio belongs to the drum, or the vocal or bass guitar.
When did you first start building it, and how long did it take to develop the whole way through?
Started looking at the idea or the concept in 2009 after conversations with engineers in the garden at lunch...
Do you think that was an issue they were facing?
Well no. It was more that I had just started here and I was just having a general conversation and I thought 'oh is there any software to un-mix music’ and they all say 'No! No! That's the holy grail. It can't be done, it's impossible' and I thought 'No, it'll be possible, there must be a way to do that' and that's where it all started from.
What projects have you used De-mix on so far?
First project that was completed was the Hollywood Bowl. That was done in 2011, but not used or released until 2016 or 17. After that, I then worked on A Hard Day's Night, the movie, the Beatles 1+ project, and then Sgt. Pepper and the White Album. Not all the White Album and not all of Pepper, just the bits that needed to be worked on. And then there's also been Life on Mars by David Bowie, Rush 2112 - The Live Concert - we were just able to enhance a bit of the bass and the vocals and stuff like that. One of the very first projects was Cliff Richard and The Shadows, which was back in 2011. So that was after the Hollywood Bowl work had been done (it could've easily been 2012), on the 50th Anniversary box set, there was a couple of tracks that had never been released before and they supposedly had stereo versions, but when we played the tapes back obviously there had been an issue with the tape machine, so it was only mono. We basically attempted to create as much separation as possible and then remixed it and got it remixed to stereo. That work was done by Peter Mew a long long time ago.
Talking more about the Cat Stevens project...What De-mix technology did you use for it? I know it was announced that there were elements applied done to the vocals of the track. Can you confirm what you actually did to the Cat Stevens reissues?
Right, with the Cat Stevens project. the issue has always been around how buried and how low his vocal was in the mix. So, I was just primarily asked to target his vocals and bring the vocal out as they were already stereo mixes (in terms of the actual backing tracks). We weren't creating anything new in terms of stereo. Once we were able to lift the vocal off so that it could be re-balanced to be more in line and make it more prominent, we were able to do a little bit of separation on some of the instrumentation. Therefore, it could be enhanced, EQ where the backing was quite flat (we always work off the flat tape transfers, not the remastered versions). Flat transfers are always the best to work from because it's as recorded on the tape before it goes to mastering and so forth. So, you create your mixes from that point, then you've got a better chance to create a new re-master. So this re-master becomes a new master.
What were the main challenges with using the de-mix technology on these reissues?It's the vocals. Vocals are one of the hardest things to work with when you're doing the de-mixing and source separation, because the peoples vocal are quite broad when their singing. Parts of vocals will be attached to bits of instrumentation; if you pull out vocals and drums, you tend to get sibilance (s and t sounds) that get themselves attached to the drum track. If you listen to a drum track, in its isolated form, you'll hear the drum and then you'll hear the occasional word coming through. When you listen to the vocal track, you'll hear the vocal but it'll be with a lisp, as parts of those vowels and consonants disappear. So that's why you have to be careful. It's really about listening to the types of songs that are being played, and then think 'that one separates well as there are very few instances with those types of words and drums at the same time’.Did you use it on everyone single one of the tracks?Every single track was De-mixed.And it followed the same process every time?Same process each time. So, we'd isolate the vocals and once we've got an isolated vocal that was usable, we would then see if we could expand the extractions further on the instrumentation, if we deemed it necessary. Then, we handed it to the engineers, and they did their magic.When you gave it to the engineers, what was their role? Were they mixing it all together? How much mixing did you do during this?I don't get involved in the mixing process. I would tend to sit in with engineers in case they've got any issues with a certain bit of the song. They would say this is what’s happening when we're trying to remix it and I would go away and tweak the code, tweak the models and see if I could resolve those issues and then hand them back with another iteration of the audio.Your thoughts on the final outcome; were you happy with the results?Oh excellent! I think when you hear it played back through the monitor speakers in the control room and you listen to the raw recording in the better format before it's got to mastering (like the mastering guys have done a superb job on top of what the engineers have done) you just think 'wow that's how it should have probably sounded at the time!' We're not sure if it's a case of when they recorded it, we believe, (and this is what some of the engineers think) the orchestra tracked themselves to recordings, so they weren't playing live together. The orchestra was playing along to a recording themselves, therefore the response isn't the same, and because of that, we think the vocals were mixed low into the mix. This is why we also discovered that there were no multi-tracks for these songs. We searched and searched through the Universal archives. It probably explains it as we couldn't find any multi-tracks and even though the multi-tracks were from the late '60s, there are all his other multi-tracks for the other four sessions. It's just these two albums that don't exist, so we're thinking maybe they just recorded it live against a recording, but that's just an assumption or a guess on what we've been listening to.Final question - What is the future of De-mixing? How much can you reveal that you've got in the pipeline?It's always a difficult one because I'm working on quite a few projects at the moment, and you're sort of under a non-disclosure, so I can't explicitly state which artists I'm working for. But I've moved into a deep learning approach which uses the same concept of generating these masks to un-mix the audio, however the masks are learnt rather than derived. With the current process (or what I call version one), you derive the mask that is specific for that song. The new approach is to derive a learned mask that can be applied to any song. So, you feed it multiple examples of audio - from its raw state to its mixed state. The system learns how instruments get mixed together to produce an output. If you do hundreds of hours of that type of learning, then when you give it a piece of music that needs to be De-mixed or un-mixed, it knows that 'well, I know how to get to this form, that's been done this way' so the inputs fall out. It's producing some stunning results at the moment.Next generation of De-mixing?Next generation of De-mixing coming up very very very soon. I would say within the next few months some releases will be coming out that uses version two.Exciting stuff!!Find out more about De-mixing here.
