E*D Films Scene Track Interview – Daniel Gies, Art + Tech Director

E*D Films Scene Track Interview – Daniel Gies, Art + Tech Director

March 31, 2018 – Excellent Interview by ED Films. Interview by , Art + Tech Director, Filmmaker, Partner. Today, he talked about ED Films Releases Scene Track, its In-House Unity Media Exporter for Free.

Where did the idea for Scene Track: The Game Media Exporter come from?

Scene Track is the answer to a big question for us. How can a small studio with limited budgets, time and resources create artist-driven, world-class animated content?

Anyone who is familiar with animation knows how expensive and labour intensive the process is. Unlike live action, where the director can try many different things on set and build the film in the editing room, there is almost no room for experimentation once animation production is set in motion.

In many of our past 2D projects, we used to combine a lot of procedural animation techniques with real-time motion to increase the quality of our work. This gave us a lot more freedom to experiment and overshoot without needing more animators. But this process also required a lot of custom scripts and coding that had to execute during playback; we also weren’t getting the real-time help we needed out of the 3D animation packages we were using. So we started looking at game engines.

In game engines, we found the potential to achieve some of our loftier projects without having to scale up as a studio. Since they are designed for real-time, and a slew of other helpful things like character AI, interactive IK, and event-driven actions, we could now realistically imagine directing our characters and scenes like actors or puppeteers. We could play games to make movies. Animation was about to get more fun, but then we faced a real problem.

There was no record and export feature in any of the game engines. So if we wanted to bring what we were playing into our existing pipeline, we were stuck. Scene Track was our solution.

Since Unity was the most approachable for us, and currently has the most number of assets available for purchase and download, we made Scene Track for Unity first. But if the Scene Track gets the kind of traction that we expect it to, our plan is to make it platform agnostic.

What are the current features in Scene Track?

Scene Track can record any described data (we utilize schemas) in real-time while Unity is in Play Mode, be it Transform or event driven.

This allows us to generate data that can be converted into any format you want. We chose to focus on FBX, MIDI, video and image sequences for now as they are the most useful for our production process.

What are the basic hardware requirements for Scene Track? Which software applications will it work with?

Any computer purchased in the last 4 years should have no trouble running it. If you can open and run Unity, you can use Scene Track. It functions outside of Unity so it doesn’t actually get in the way while playing.

Developing and programming an open source tool is challenging! How did you manage that?

We are an animation studio first and foremost and were not equipped to develop software at this level so we hired the talented engineers at dotBunny, a Canadian developer, whom we had worked with on gaming projects the past.

How many days did it take to develop the Scene Track and what software did you use to develop it?

We used Visual Studio 2015 (FBX SDK limitation) and XCode4.

The actual development of the prototype started in 2015. The production happened between November 2016 and April 2017. Tweaking and updates happened over the course of one month.

The actual coding of Scene Track was quite fast, it was the conceptualisation, planning, financing and testing that took the most amount of time. Once we knew what we were trying to make, and how to go about it, the entire thing was relatively painless to develop on the coding side. It was the workflow and production implementation that proved to be the challenging aspect.

Why was Scene Track released as a free open source tool?

Our focus as a studio is ultimately to make high-quality, animated film productions. We are not setup to manage software development and distribution in any deep capacity. We had initially conceived this tool hoping to open game engines up for our style of workflow. When we realised the potential of it, we were able to work with the Canadian Media Fund and the National Research Council of Canada to expand the development into something other people could use as well.

Once we had it working, our plans were to partner with a developer and be the creative content producer that could expand and teach the pipeline while they managed distribution and sales. That never materialised and instead of letting the tool get lost to the digital ether, we decided to let it out into the wild, where it can grow and become more than we could ever imagine it could be.

We know that Unity and Unreal have, and are continuing to develop, their own versions of FBX exporters so we can’t compete on that level. The edge that our exporter has is that its API, “The File Temple”, runs completely independent of Unity so, with the right people, it could be made to capture data in real time from any application and turn it into any format you can think of. The fact that it captures MIDI, video and images is also a differentiator. This is really a tool made by artists, for other artists. It is not limited to just the game or animation industry. Often, when we present this project, people from theatre and performance realms get very excited as well.

How can Scene Track be useful at very big and small production studios who are making animated films? Also, how it is useful for Animators and Game Designers working in an advanced manner?

Using game engines as a pre-viz tool and for rapid prototyping of animated content has already been proven on small and big budget productions. In fact, since Unity added the Timeline feature its capacity for staging a scene from scratch is amazing. The only problem we ran into, before we had Scene Track, is that all of the animation had to be sourced from outside of Unity.

If you wanted to move a character or scene object around you had to manually trigger events and keyframe paths, which is not really an entry-level user experience. With Scene Track, you can record the gameplay of an animated character and drag it right back onto the animation timeline to use in your scene. This allows you to populate a set with animation very quickly. You can sit and work with a director, playing through the scenarios and building things up in a couple of hours using stock assets and affordable plugins with very little experience in Unity.

Once you are finished playing, recording, re-timing. etc. whatever you made in Unity and the director is happy, it can all be imported into your CG animation package and polished for a finished shot. No one has to rebuild or start over. In fact we have been using Maya’s HIK rigging and mocap workflow to rapidly create sophisticated rigs for complex, humanoid characters and apply the Unity animation back onto them in a few quick steps. This is a massive time saver! You can go from a character without a rig to an animated asset in Unity, back to Maya and polishing the animation in under an hour.

Additionally you can use any off the shelf or free character generation package like Mixamo, DAZ, Fuse, Maya Character Generator, etc. to populate a game scene with performable avatars, combine it with a pre-existing or character controller from the asset store, mash in some Final IK for advanced, drag and drop, procedural animation and behaviors, throw on some crowd AI for background characters and you are off. There are so many resources out there that are already artist friendly and ready to use.

Imagine submitting an RFP with an added animated sequence that took one afternoon of work, from one person to create. In two days you could go from a game-y looking sequence using stock assets to a fully realized, beautifully animated, simulated and rendered sequence that uses your existing cinematic assets. No optimization, no coding, no game specialists just straight up kit bashing and old fashioned animation polish.

Big studio or small, you benefit from greater freedom of experimentation, faster turnaround and smaller teams that ultimately use less resources. The added benefit for a big studio, or a studio already familiar with game engine driven productions is that you can develop more custom tools for interactivity and performance and may already have a wide variety of game ready animations to run in engine, so less time can be spent tweaking or polishing the recorded gameplay.

Every time you do something in the engine you are adding to your library of assets to be used in the next project. Game assets are very expandable and flexible and ultimately designed for reuse and variation, far more so than traditional CG film assets.

If you are doing far more sophisticated things, it only means less polish. One of our goals is to develop a multi-pass, puppeteering approach that would allow us to control a single character in many passes. Very similar to the complex, animatronic puppets from The Dark Crystal. Except for us, one person could do the whole thing and augment the performance in Unity’s Timeline before sending it to Maya for a final polish.

Has E*D Films developed any other open source tools?

No, we have not. However, we have, and continue to, provide free tutorials on our process and methods on both YouTube and Vimeo. While it isn’t open source, it is open and accessible to anyone who is interested.

Currently we have a number of projects in production or in development, including two VR animations and one series, so we are focusing on refining and further developing our animation production pipeline so that we can take full advantage of Scene Track and Unity’s existing features.

The exciting thing is that we know whatever time we invest in Unity now can feed directly back into our existing animation production process.

Are you testing Scene Track with the help of any animation studios or 3D application companies?

We are actively pursuing the student market with the tool. We have, and will continue to, give presentations and workshops at animation schools to show the tool to students and help inspire them to think about new ways of production. We are also reaching out to colleagues, both independent makers and small studios like our’s, to get them experimenting with the tool in their pipelines. We’re also conceptualizing animation-jam style projects, to show how fast the tool can be implemented into the creative process. There is a perceived steep learning curve that can make game engines intimidating. We’re encouraging people to dive in and play, it’s the only way to see how easy and fun it can be.

Any challenges you faced in creating Scene Track? Can you share some fun stories that happened during the creation of it?

While conceptualizing and developing Scene Track we were constantly faced with the question of ”Why would you do this?” and “Why take it out of the engine?” What seemed obvious to us and independent animators was extremely difficult to communicate to investors, studios and developers. Additionally, because we don’t come from a game development background there was a lot of learning to be done, and a lot of second guessing ourselves in the beginning. This made it challenging to gain the support we needed to actually make the tool happen.

Thankfully we received funding and support from the Canadian Media Fund’s Experimental Stream and the National Research Council of Canada, who saw the potential of what we were trying to do and threw their hats in early to help develop a prototype and, subsequently, the current version.

It was a deceptively complex and difficult process though. Creating a film is quite straightforward compared to developing software for public use.

What animation projects have you used Scene Track on?

We have already used Scene Track on two of our own productions Giant Bear and Retour a Hairy Hill. Additionally we have used it on a number of experiments and tests that may see the light of day when we come out of our current work tunnel.

Can you share something about Giant Bear and Retour à Hairy Hill?

Giant Bear is the story of an Inuit hunter who is forced to confront the last remaining giant polar bear on earth. It’s an action-adventure short, based on an oral story recounted by an Inuit elder and translated for the screen by Iqaluit-based Taqqut Productions. Giant Bear will be finished production in Summer 2018.

Return to Hairy Hill / Retour a Hairy Hill is a short film based on the true story of my grandmother who, after decades away, returns to her childhood home in rural Northern Canada. It’s a piece of family folklore about memory, transformation, and blood memory. I’m a third generation doll maker, the importance of which makes sense when you see the film. The project has been in development for 10 years.

Retour a Hairy Hill is our most robust technical development, as it will not only incorporate everything we learned on Giant Bear, but will harness the game engine’s power of performance-driven animation to bring our characters and world to life. Unity is helping us capture complex and subtle multi-take, multi-layer performances, which we can then transpose onto our Maya characters.

The rendering and compositing style is also very specific and takes advantage of an existing workflow from Maya to After Effects that will help blend 2.5D, hand-painted backgrounds, 2D hand-animated assets and 3D characters into a seamless, unified composite. It will be the perfect example of why the conventional game production river could be best if it flowed uphill.

Tell something about E*D Films Online Store. What are the collections you have in it?

When we work on a short film, we create custom elements to make our processes more efficient. For example, we might create custom Tree Brushes for painting forest backgrounds in Photoshop. Over time, we’ve been able to amass a substantial collection of Photoshop Brushes, animated environmental sequences, custom character rigs, sounds, and more. And as we’ve shown them off in our tutorials, our audience/dedicated fans have asked how they can get their hands on them. So we started putting together asset bundles for other artists so they could bring their projects to life.

The first phase of our store currently houses various collections of Custom Photoshop Brushes, mainly focused on environment painting.

Animated Elements are collections of high-resolution image sequences with an environmental or atmospheric focus such as collections of falling snows or bushes blowing in the wind; these sequences are PNG sequences that can be downloaded and imported into After Effects projects as a sequence to help populate and bring animated life to a scene.

While Scene Track is currently our only plan for an open source tool, we will be releasing more production tools in the next year on the store. We’ll also be launching new sounds, rigs, After Effects organizers, and more.

Our next step is to accompany each product with a how-to tutorial in order to share our processes with our audience and truly teach them how they can use these collections to make projects of their own and make their stories move.

What about your future plan and upcoming projects? Anything you want to share with us?

We have recently secured a co-development agreement from Corus (Teletoon) a passion project of ours called “Elemented”.

Elemented is a sci-fi series that that chronicles the life of 15 year old Wren and her friend Ash who live on an isolated mining outpost on a distant planet. The main story arc follows Wren, as she and her friends work together to save their home planet from destruction as attempts to advance to the next stage of evolution. After uncovering an ancient and powerful artifact Wren’s own humanity will be put to the test as she unlocks its mysteries and learns of its terrible purpose.

It was this project that was real impetus behind the creation of Scene Track. The project won a number of pitch competitions and garnered international recognition before it was even written. So we knew we had an audience, but one thing kept coming up over and over again from every broadcaster and potential distributor, “This is too risky and it is going to be too expensive to create.” And that is true. Using existing studio approaches to animation production, this series could never be made to the quality it deserved within a reasonable budget. However, it was designed to be made using game engines and now that we can record what we play we can finally make this rich, complex and dynamic world come to life.

A big thanks for your time.

For more info:

Official site of E*D Films