An Interview with Frank Serafine by Randall D. Larson
Originally published in CinemaScore #11/12, 1983
Text reproduced by kind permission of the editor and publisher, Randall D. Larson
Frank Sernfine was brought in to provide the synthesized sound effects for TRON. Operating a small sound effects studio in Los Angeles, SFX, Serafine’s background was in the music and audio synthesis recording field before he began to work with motion picture sound effects in STAR TREK: THE MOTION PICTURE. Subsequently, Serafine has been involved with creating sounds for THE FOG, THE SWORD AND THE SORCERER, TRON, BRAINSTORM and TV’s THE DAY AFTER. This interview, recorded while in the midst of working on TRON on May 7, 1982, details Serafine’s efforts on the film and his views on synthesized sound effects.
Would you describe what SFX does in regard to motion picture sound effects?
Traditionally, motion picture sound has been recorded with microphones on location, then transferred to 35mm mag stock and cut by editors. What we’re doing is taking the technology from the music industry and using multi-track machines in sync with video machines. We’re laying all of our sound effects right down onto multi-track machines in sync with the picture, which is kind of a breakthrough. We’re using a lot of synthesizers, and we’ll take organic sound effects such as a monkey screaming, cats purring, and we transfer them onto discs. In TRON, in order to create a deadly sound on a disc, we took monkey sounds and recorded them into a digital synthesizer, and then we played them back in sync to the picture, and created a very deadly sound.
How do you go about creating a sound on a computer?
We sample the sound into the computer, digitally, and then it calculates the frequencies and all the characteristics of that sound and gives it back to you. Then you can manipulate or re-create the sound the way you want it. We also use computers, like Atari and Apple, to keep a log of all our sound effects. We have something like an address filing system where we file all of our sound effects – we have ten and a half hours of them. They are logged into the computer and we can access anyone of them, whether by name, sound number, how long it is.
Would you describe how the process of running sounds through a computer works?
It’s called sampling. You just take whatever given sound you have and you store it into the computer memory. If it’s a monkey scream, if it’s a cat purr, you just run it into memory and you call it whatever name, and it files that sound as if you were just putting it on tape. And then you have the ability to turn it upside down or backwards, and loop it, or whatever.
Would you describe the advantages of synthesized sound effects over those recorded by earlier methods?
You have all the control, especially for science fiction films. With organic sound effects, you hear that sound and that’s it. It can be very powerful – like if you have an explosion or a train passing, you can’t really recreate that with a synthesizer; a train going by has a certain “air” quality, it’s pushing air instead of being a static sound. With synthesizers you can come close to that but it’s not the same; you’re dealing with electronics that are creating frequencies instead of actual air. But with synthesizers you have the advantage of creating surreal sounds, sounds that have never been heard.
Once you have decided that a particular visual effect requires a certain sound, how do you go about creating that sound?
It has a lot to do with how the director feels it should sound. I usually try and work with the director to get the feeling of what he’s trying to convey with that kind of effect. After a certain point of working on sounds and analyzing and studying and doing R&D, you just get to a point where you can look at a certain visual effect and get an overall view of what’s going on: “Well, that’s a hum sound” or “that’s a high frequency bell” or “that’s pixie dust”. After doing it long enough you just know where to go. I just step into the library and start experimenting with sound effects. I’ll run them backwards, then I’ll experiment with the picture, and if that doesn’t work I’ll sample them into the digital computer. It takes a great deal of experimenting before you can really find out what’s what.
I’d imagine that the research on some of these sounds takes longer than actually creating it on the synthesizer.
Right. I’ve done R&D on TRON for a year now, because there’s absolutely no real sound effects that can convey what’s happening in the film. It takes place in a video game world, and you have to do a lot of experimenting for that.
One of your earlier assignments was on THE FOG. What kind of work did you do there?
I had to create scary sound effects, working in conjunction with John Carpenter. He was trying to convey a lot of horror, and I’d never really done that before. STAR TREK was ominous and celestial, but for this I had to create a lot of jarring sound effects, something that would hit the audience real fast; also ghost sounds. We went into a lot of percussive sounds and gongs; we ran fiddle bows across the sides to create a really shearing, barren, strange sound texture. But it was pretty standard. I just did THE SWORD AND THE SORCERER, and we played a lot of very weird things. We’d take balloons and we’d crackle them with our hands, and we’d run that through the synthesizer; and burn offs – we’d take a soldering iron and put it into a hot dog for melting flesh!
How do you work with the music composer in achieving a balance between music and sound effects?
I look at the film score and I try and create the sound effects within the score. I’d look at the key that it’s in. If there’s heavy orchestral tendencies in the score I’ll try and stay high above them with synthesizer sound. If there’s nothing there, I get real heavy and I’ll create whatever is needed for a certain texture. Often times I’ll use harmonizers to fit the sound effect perfectly in place; if it’s not in key, I’ll raise the pitch until it works perfectly with the key of the music. We had to do a lot of that on STAR TREK because there was so much music to go with the sound effects.
What sort of work did you do on SATURN 3?
I did the trailer score for that. They came to me and they knew they had a dog on their hands, in terms of the film. They said “We’ve got to sell this film!” So I created a whole synthesizer sound effect and electronic music score for that, and consequently the film did very well for about a month and then once everybody saw the film it pretty much went downhill. But the trailer on it was very nice, and worked out very well.
We did the same thing on QUEST FOR FIRE. The score for the film was actually very nice but it was very subtle and mellow, and what they were trying to do with the trailer was to convey the action and the suspense, so I did a synthesizer sequencer type score, which we orchestrated with violins and sound effects on top of it.
I’d imagine that, coming from the music industry, you’re looking at sound effects not only as just sounds, but from a musical perspective as well?
Right. That’s quite a bit different from the way that the sound effects editors work, because they never really do anything in “real time.” They just take sound effects, cut them in, and that’s basically it. What I’m doing is orchestrating sound effects and placing them to the picture in real time, very much like musical sound effects. Orchestrating the sound.
Real time sound effects means mainly to compose the sound directly to the picture, laying it down with the video and multi-track recorder in sync together. There’s no real editing process that goes into it.
What kind of effects are you creating for TRON?
I’m using a lot of video game sounds. I went up to Atari and researched their video game sounds, and now we are developing our own video game sound effects for the film. I’ll analyze a lot of those sounds and I’ll recreate them on the synthesizer, and I’ll also sample them into the Fairchild digital synthesizer and then lay them in real time to the picture.
In other words, video game players may recognize some sounds in the film?
Oh, yeah, we’re using Pac-Man and a lot of those. We’re playing jokes with video games.
How are you working with composer Wendy Carlos on creating the audio atmosphere for TRON?
Well, she’s in New York and I’m out here in L. A., and we just communicate over the telephone, so that she knows what’s happening with the sound effects and I basically know what’s happening with the music. That way we don’t interfere with each other.
TRON may be considered a landmark film in terms of the use of computerized visual animation. How will your synthesized audio effects contribute to its uniqueness in terms of sound?
By matching everything that’s happening with those visuals. Also the style in which we’re doing it; I think it’s the first film that’s ever been done with SMPTE time code synchronization. This is a universal code (developed for the Society of Motion Picture and Television Editors) that is able to lock in all video and audio machines. Every source, the 35mm film, all the synthesizers, will be triggered with this code, so that whenever you want a certain action to happen you can trigger a series of events.
Say that the clock is at 01000. You say “I want the sound effects to trigger at that point.” All the reels, everything, stop, and they hold at that certain point. All the equipment is able to trigger at whatever point you give it.
You’ve said that synthesizers are revolutionizing the sound effects capabilities, much the same way that computers are revolutionizing visual effects, particularly in the case of TRON. Would you elaborate on the future of such technology in films?
Eventually, everything’s going to become digital, and I feel that synthesizers are going to play a real important part in just the overall logging. If you have, say, a hundred sounds, you can program them to file onto the picture at any specific point, then you can go to lunch and come back and it’ll be edited for you! The process would just take a lot less time. There’ll also be a lot less animosity in putting films together; you’ll be able to do it with a lot less effort than it takes at the present moment, which is really a painstaking process. With all this computerized technology and synthesizers in filing organic sounds into digital memory, you’ll be able to really do it fast.