AI in Filmmaking at the Yorkton Film Festival

AI is Revolutionizing Filmmaking – Are You Ready?

As someone who has been deep in the AI trenches for the last year, experimenting with custom language models, generative AI, and video and voice cloning/deepfake tech, I can tell you we are living through a period of rapid and profound change. Artificial intelligence is coming for the filmmaking industry in a big way, opening up incredible new creative vistas while also raising important ethical questions we all need to grapple with.

I recently got a glimpse of the AI-powered future of film at the “Unlocking the Cinematic Future: AI in Filmmaking” panel I participated in at the Yorkton Film Festival in Saskatchewan. The panel, which I was a part of along with fellow panelists Leif Kaldor and Janine Steele, moderated by Erin Mussolum, promised to showcase how Artificial intelligence is emerging as a powerful force offering filmmakers unparalleled tools to enhance creativity, streamline production processes, and captivate audiences in entirely new ways.

We discussed the first steps in creating an AI mindset, how policymakers are responding, and dove into the pros, cons, fears and frontiers of AI’s intersection with creativity across the entire filmmaking process.

“I’ve been up to my eyeballs in the enigmatic world of generative AI lately, and the possibilities have really got my wheels turning,” I told the crowd. We covered everything from using AI for specific production tasks like lip-syncing and multi-lingual voice dubbing, to the bigger policy implications as lawmakers race to establish guardrails around this powerful technology.

One thing is clear – AI is not just a tool for tedious automation, it’s a new co-creative partner that can augment and enhance human creativity in unprecedented ways if we’re willing to embrace an “AI mindset”. As I put it in Yorkton, the latest AI systems demonstrate remarkable capacity for ideation, world-building, and generating entirely new visuals and audiovisual experiences from scratch, going far beyond mere automation of existing tasks.

The demonstrations at the festival were a window into this new reality. My video/voice cloning experiment used deepfake tech to create multilingual versions of myself delivering the same message in seven different languages – something impossible for a single human actor to accomplish naturally.

“I literally just immediately worked on bringing people into my world,” I explained, through these AI-generated multilingual video clones speaking languages from Japanese to Swahili. This opened the door to “inviting everyone, whether you’re from California like me, or from as far away as Okayama or Kampala, into our inclusive global digital community.”

Animator Leif Kaldor also showed how AI could drastically accelerate the tedious, labor-intensive process of lip-syncing animation across multiple languages for a new preschool series. Instead of taking a week for a junior animator to manually lip-sync each language, Kaldor’s team created “an AI tool set that basically does it in about an hour and a half or two hours,” with just some touchup work remaining.

But these production examples are just the tip of the iceberg. What if an AI co-pilot could not only automate tasks like dialog dubbing, but go further by rewriting scripts, proposing alternative storylines, character arcs, or even generating entirely new scenes, settings, and animated characters from scratch? Some of these capabilities already exist and are rapidly improving thanks to large language models and text-to-video generators.

For filmmakers willing to change their mindset, AI can become a truly creative collaborator, not just a fancy tool for automating tehind-the-scenes workflows. As I put it in Yorkton, “I’ve always wondered what it would sound like to hear myself speaking flawless Japanese, Swahili, or even Russian…I’ve built a multilingual deepfake experience to break down not just technical boundaries, but also cultural and linguistic ones.”

Of course, with great power comes great responsibility. As this generative AI tech goes mainstream, we’ll have to grapple with a host of ethical issues around disclosure, consent, copyright and ownership of AI-generated content. There are also valid concerns about the impact on jobs, with some roles like transcription and rotoscoping potentially being disrupted by automation.

These were hot topics at the Yorkton panel. Sarah Spring Executive Director at Documentary Organization of Canada (DOC) issued a stern warning that “this is like a really important territory we’re stepping on” where “human lives and stories” were “shared in a context of trust” – making that content “fair game” for unconsented AI remixing and reproduction is a breach of ethics.

Policymakers are moving rapidly to try to get ahead of these challenges. As Canada New Media Fund’s Janine Steele outlined, the EU’s AI Act aims to manage risks around privacy and safety, while Canada’s proposed legislation focuses more on citizen protection from things like deepfakes through measures like pre-approval requirements. The US is largely leaving it up to the market and courts to sort out.

My view is that the film industry needs to be proactive in defining ethical guidelines and best practices, rather than remaining hands-off. Full transparency is important – if you use AI-generated visuals, audio, characters or storyline elements in your productions, you should disclose that upfront to maintain trust with audiences.

But I don’t think it’s realistic or desirable to expect credits listing every single AI tool involved. We don’t ask directors to credit every einzelne software used. What matters is clarity on the final creative output itself – did you incorporate AI-generated elements or not? Simple “AI-generated” or “No AI” badges could provide that transparency without bogging down in excessive detail.

Ultimately though, I believe humans will remain at the creative center as these AI tools evolve and mature. We are the ones providing the prompts, curation, intentionality and core artistic direction. AI is a collaborator amplifying and extending our creativity, not automatically replacing human storytellers.

“My goal is to develop an AI that intimately understands my personal creative voice, thought patterns and perspectives – sort of like an AI version of my inner muse,” I told the Yorkton audience. I want an AI co-pilot that speaks my language, with my idiosyncrasies and nuances baked in through training on my entire body of work.

In the meantime, my process involves taking a multimedia approach, using AI to transcribe recordings and interviews, generate ideation keywords and visual concepts, and then having it weave all those inputs together into an edited review, narrative or video artworks.

I’m bullish on the amazing potential for generative AI to open up vast new frontiers of storytelling and worldbuilding. But I’m also sober about the ethical risks, and believe it’s crucial to have proactive, society-wide conversations about developing appropriate policies, moral limits and safeguards as we move forward into this future.

Because make no mistake – this technological revolution is rapidly approaching. As I warned the Yorkton audience, “The future of film is AI-powered, and those who embrace the mindset and the tools today will be the ones shaping that future.” The cinematic universe is about to be reborn. The real question is whether we’ll be midwives ushering in that rebirth responsibly, or merely bystanders.