Open Captions: Enhancements November 2019

Working with a prospect, we added a number of enhancements to our open captions workflow. If you are interested in what open captions are and what its use cases are please have a look at this blog post.

Very simply open captions are generally used as an accessibility tool focused on people who have hearing impairments. More recently due to the rise of streaming services, open captions have been making a come back.

• As an example, a simple use case would be a new mum at home with her child. If the child is sleeping, the mum can watch video content (on TV or online) with open captions on mute.

• Another example is people watching video on a social media platform, where the video content is often auto-played but in a muted environment.

We have added one simple enhancement to the Auto-Overlay functionality. The new functionality allows a user to add the open captions anywhere in the video.

We will be using the same RSTV sample which we worked with in an earlier blog post. To see how we used an AI to caption the Indian English video, please read this blog post.

Context + The Enhancement

A client we have been working with for a while had the following requirement. He wanted to add the open captions in a specific place in the video. Historically this was possible but was very hard to use effectively.

The enhancement is that we now allow X and Y coordinates to be placed, allowing the user to have fine grained control over the placement of the overlay.

What Used To Be Possible?

Assuming you followed the steps in this visual guide, you would be aware we can add open captions using the Auto-Overlay feature.

Basically we used the Auto-Overlay feature as shown below. This is a two step process - we already have the captions from the previous steps.

1. Use Action -> Auto Overlay and choose a text colour, size and font. We have chosen black colour text, with font size 36 and font Ubuntu.

   

2. In the second step, it used to only be possible to choose from three options Top, Centre and Bottom. This is shown below.

   

3. It is important to understand what is happening here. Its basically a mathematical function called a text writer that is writing-out (or baking-in) the open captions into the video. The yellow highlight was probably not the best choice but its useful in demo.

4. Ok - now we are going to make some changes. First we use Action -> Remove Auto-Overlay to take away the open captions as we want to move them around. This looks like the below image.

   

How Has Auto-Overlay Changed?

The change is slightly different for (a) the stripe, and (b) the highlight in the Auto-Overlay function. First, lets talk about the stripe and then we will look at the highlight.

Use the Action -> Auto-Overlay and we use the pre-filled values for Step 1. The below images are from the point-of-view of Step 2.

No Stripe Or Highlight

In this use case we don’t want a stripe or a highlight. This is shown in the image below.

Depending on the colour scheme of the underlying asset the lack of a stripe or highlight may not offer the best user experience.

Using A Stripe

In this case we want a stripe. The idea of the stripe is a bar across the screen. Here there are two options, show the stripe always, or show it only where there is text on the screen.

First of course, go ahead and remove the no-stripe/no-highlight overlay from your asset. Then, trigger the Auto-Overlay process again.

Text Only Vs No Text Only

Why would you use this? The use case is to do with the speed of the speech in your video.

• In the case where the speech is very quick, with minimal breaks in the speaking, we would recommend No Text Only
• Alternately, if there are gaps in the speech, use Text Only.

The idea is when the underlying speech is relativity fast the flickering of the stripe looks terrible, so you want the stripe to be in place irrespective of whether the text is showing or not showing.

The below example is of a Stripe with Text Only and we are using the blue colour.

This does not look amazing but you can see the effect of the Stripe with Text Only.

Using A Highlight

In this case we want to use a highlight. The idea here is a highlight around the text in question. Using the highlight option looks like below. Text only is not a thing with the highlight for obvious reasons.

In the video below, we used the Centre option to make it very obvious what is happening. Transparency is also turned down for the highlight itself.

The Big Change: Advanced Options - Stripe

The stripe example is the slightly more complicated of the two options. The first thing to note is the actual dimensions of your video. In our case, the video is 640 x 360 pixels. This is shown in the image below.

Add a stripe with the below advanced settings. Note that these numbers are actually wrong, but very useful to understand what is happening.

What we are doing here is setting X and Y coordinates for the text at 20, 20. We are also setting the X and Y for the stripe at 40, 40. Normally you would set the stripe to X and Y coordinates similar to the text.

Ok - so that was nice, but not really useful. Lets try again with the below settings.

This looks much better. The idea here is to put the stripe at the bottom of the video. But we have made a mistake in the alignment of the text.

With the examples above, hopefully it makes sense how to use this functionality.

Important: If you are using the stripe, please note that the height of the stripe is 10% of the height of the video itself. You may need to manually fix up the captions so that all the captions are single lines only.

The Big Change: Advanced Options - Highlight

Highlights are easier to use than the Stripe - mostly because we only require one set of X and Y coordinates. The setup used is shown below.

This results in the following video using the highlight. Please note this setup is not optimised and make the changes required on your side.

Important: If you are using the highlight, it is well worth making sure that each sentence is roughly the same length and centring the open captions into the video nicely.

Conclusion

We hope that this visual guide is helpful to you! Open Captions are very useful, and a critical part of the value proposition our video translator offers to clients.

Please connect with us on LinkedIn, YouTube or Facebook for any comments, questions, or just to keep up to date with the work we do!

We are very grateful for your support!

If you are interested in trying out our technology, please try our platform or drop us an email at hello@videotranslator.ai.