Over the past two years, developments in artificial intelligence (AI) have driven an amazing wave of innovation in a number of different fields. For app developers, this comes with a parallel challenge to make sure that AI technology is reliable, effective, and safe. AI’s capabilities are constantly evolving, so we need to always be on the lookout for things we can learn, integrate, or develop better ourselves.
At Lightricks, riding the cutting edge of AI has been both exciting and challenging. After integrating dozens of new generative AI (GenAI) functionalities into Videoleap, Photoleap and Facetune, our top visual content creation apps, we took on a new challenge, developing LTX Studio over the past year. LTX Studio is a filmmaking app that uses GenAI to push the envelope by integrating AI with film production, enabling users to transform a single idea into a cohesive, AI generated video – amplifying creativity through new forms of storytelling.
As a team, we have several years of experience working with visual AI apps, but in many ways this project crossed uncharted territory – not just for us, but for the entire tech world. Researching, building and refining the app involved staying abreast of the rapidly-evolving AI landscape, identifying potential technologies that could be integrated or developed in-house, and orchestrating it all with some creative problem-solving. We’re proud of what we’ve accomplished with this product, but our work is hardly over.
Here are some of the things that we’ve learned along the way.
Building models carefully, with intention
Every generative AI tool is constrained by the quality and scope of the data behind it. Because of these complexities, our researchers and engineers put a lot of effort into assuring the consistency and reliability of our models.
It’s also important to remain vigilant about the biases that can develop within AI models. Our engineers are constantly refining the way the engine interprets user prompts, ensuring that the output is as bias-free and as close to the user’s vision as possible.
Taking usability testing to new heights
AI tools tend to be highly technical, and there’s a risk that they can be too complicated for the typical user to apply effectively. We needed to wrap LTX Studio in a user-friendly and accessible interface, with the kinds of features, actions, and terminology that are already familiar to video creatives. In addition, we are working directly with leaders in the filmmaking and advertising industries to create the features that they need to meet their always-changing challenges.
We’ve always had a user research team and taken a data-driven approach to product iteration based on user interaction data. However, this time we added to the user feedback mix a beta community with a Discord server as its hub, and we work with active members who post and share feedback on Discord and directly with our CX teams. Their feedback has been extremely valuable, helping us to refine the interface considerably.
Building on past project experiences
Ensuring the stability and scalability of any AI tool is a challenge, especially with the landscape changing so rapidly. It’s even harder when you’re blazing a new frontier in AI.
We are fortunate that we already had previous experience creating similar software.
Additionally, our previous dev projects meant that we already knew a lot about the challenges we’d be facing.
Balancing the need for speed… and quality
The challenge when creating generative AI content – particularly as we move into the world of generative video – is that rendering and creating high-quality, high-resolution video content is complex and slow. The tradeoff to faster video rendering is usually lower quality, but it’s most important to keep users engaged while maintaining a frictionless editing experience.
How do we do this? First, by breaking the creation experience into smaller parts, steering the experience so users move scene-by-scene and shot-by-shot. Because people don’t need to wait for the entire movie to render, they can incrementally build their project. Next, we built a rendering process that operates in steps, so users can make faster decisions about what will eventually be the end result. This way, they can choose between different versions during the creative process (avoiding sub-optimal, lower-quality videos) and wait only for the relevant results.
Multi-tenant scalability
AI video makes heavy demands on computing resources. Even with faster rendering, it’s still challenging to be able to scale an AI video editing platform for many users at the same time, without significant buffering and long wait times.
Our engineers have created best practices through the management of global spikes in usage of our mobile apps, and react quickly to maintain the best user experience.
Bringing GenAI video into reality
Producing a generative AI video platform is an ongoing but exciting challenge. At Lightricks, we are continually stretching ourselves to find new ways to integrate AI into innovative tools. By learning from experiences across our portfolio, from our editing apps Facetune for portraits, to Photoleap for images and Videoleap for video content, we are able to integrate new elements and experiences for our users.
Building LTX Studio has highlighted how important it is to have an ongoing dialogue with the user community, so as to ensure that technology evolves in the right direction, one that continues to meet their needs and standards. We’re excited to see where it all goes next.
Alon Yaar is a VP Product at Lightricks, an award-winning developer of AI-first photo and video editing tools that have been downloaded over 730 million times worldwide. A seasoned freelance designer, Yaar first joined the Lightricks team in 2018 as a machine learning researcher.
