At VideoTranslator we do a lot of work in the Internationalization And Localization industry.
But we are not transcribers, translators, or voice over artists. We use AI to do transcription, translation and synthetic (AI) dubbing.
This happens when a client is looking to try out tech and/or better understand the value proposition, or has their own reasons for us to provide this service.
Sometimes however, the most useful thing we can do is just have a conversation.
The email and discussion that follows is with Curt Jaimungal, an indie film maker from Toronto, who sent in an email.
Curt is a very interesting entrepreneur, and you should totally check out his channel - Theories of Everything with Curt Jaimungal!
The below images are used with his permission.
The Use-Case
Right - so what is Curt asking about here?
The use case is quite straight forward to understand, “… it would be great if I could simply drag and drop it into a program and it would output the lecturer speaking in English”
If we were to restate it, the use case here is that the client wants to:
- Upload a German video and have it transcribed.
- Translate the previously transcribed captions into English.
- Use an English speaking AI to synthetically dub the content into English and speak it out.
After which, he quite practically asks, “Is there an easy drag and drop interface to accomplish this?”
The ask is totally doable.
In fact, this is the exact service we provide managed service clients.
But the ask is can the app be used to do this? Absolutely.
Its just … not as nice as one would hope…
Example Of AI Voice Over, Or Synthetic Dubbing
Have a look at this blog post - How QAChef Is Reaching Clients In A Tough Market - it covers the business side of such a process…
This is a more fleshed out version of what Curt is asking for - but the blog post looks at this from the point of view of ROI (Return On Investment) and how to use video translation to achieve specific marketing goals.
You should totally click on the link if you want to get a better handle on synthetic, or AI, dubbing.
How To Convert Your Video From English To German
Have a look at this blog post - English To German Video Translation: What is a Babybündel?
It covers the mechanical aspects of how the process works, as in what to click within the app.
The process is a little ugly at the moment - and is an ongoing priority in our development schedule.
Send us an email if there is a specific use-case you would like us to cater to for you’re preferred workflow.
What do AI voices sound like?
So lets say you are reading this blog post, but looking to translate into French or some other language?
Have a look at this blog post - Samples From A Voice Over Robot: French, German and Dutch - it covers how we use different AI’s for different gender effect, age effects etc.
But what are the implications of an AI workflow?
So we just pointed out how we can do what Curt wants, so why are we saying its messy?
To fully translate a video from German to English, we would do the below.
Upload the German video and use an AI to transcribe, resulting in a caption file (*.srt).
After this, a human subject matter expert needs to clean up the captions.
Now, the Text-To-Text translation AI can be used. Again, a human subject matter expert should be used to sanity check the translation.
Finally, the third AI can be used to provide the synthetic voice over (*.mp4).
What happens without human oversight?
As we move up information complexity, the quality of the AI’s output gets worse… but why?
High complexity information is valuable. You pay lots to go to a doctor or lawyer.
The information the doctor or lawyer has is scarce. This is why they are expensive - not many people have the skills that the doctor or lawyer has…
This scarcity means, there is less information on the internet, to train an AI with - this is why the quality falls a little bit.
How Do I Start With Video Translation?
Just start with English (or in this case, German) transcripts - don’t translate to begin with… Why?
The value add of captions is all about search.
Search engines don’t understand audio or video! They only understand text.
Without captions, search engines do not know what is in you’re video!
This is the whole point of VideoTranslator.
When you (later) translate your captions - it means people searching for the video (in their words - i.e. in the other language) will find your video - even though it is still in English!
English captions only!
Practically, use the VideoTranslator app and get the captions.
Make any changes you’d like.
Download these as an *.srt file.
Upload to multiple channels
Upload the *.srt to YouTube.
Read this blog post about how internationalisation can boost ROI, specifically the part about how duration impacts engagement by channel!
What Should A Video Transcription And Translation Workflow Look Like?
We can offer advice, or just a sympathetic ear - shout out to my translation friends 😃 - but this is the question!
What level of ROI do you require, on a per video basis, to invest in the transcription and translation workflow?
So - general advice only - reach out and ask for specifics.
Make small slices of the best bits of the video and use Captions + Auto-Overlay to measure engagement.
Honestly, all you can do is try, measure, and refine.
No silver bullets (or snake oil) here sorry.
Thank you very much for the kind words Curt!
Please let the team know if we can help out in any other way.
