Just-in-time prepared captioning for live transmissions

Latency remains one of the most significant factors (1) in the audience’s perception of quality in live-originated TV captions for the Deaf and Hard of Hearing.

Once all prepared script material has been shared between the programme production team and the captioners, pre-recorded video content remains a significant challenge – particularly ‘packages’ for transmission as part of a news broadcast. These video clips are usually published just prior to or even during their intended programme – providing little opportunity for thorough preparation.

This paper presents an automated solution based on cutting-edge developments in Automatic Speech Recognition research, the benefits of context-tuned models, and the practical application of Machine Learning across large corpora of data – namely many hours of accurately captioned broadcast news programmes.

The challenges in facilitating the collaboration between academic partners, broadcasters and technology suppliers are explored, as are the technical approaches used to create the recognition and punctuation models, the necessary testing and refinement required to transform raw automated transcription into broadcast captions and methodologies for introducing the technology into a live production environment.

INTRODUCTION

Over the last 30 years the volumes of ‘SDH’ captioning (Subtitles for the Deaf and Hard of Hearing), both live and pre-recorded, have increased considerably across the globe. As coverage approaches 100% in certain markets, the attention of the regulatory bodies has shifted from volume to quality (2). A closed caption is fundamentally a short section of timed text, and three simple measures of quality can be viewed as:-

  • its fidelity to the original spoken word
  • the textual accuracy of the transcript
  • and the timeliness with which it’s presented

For a pre-recorded programme it is possible to ensure that fully accurate, perfectly timed verbatim captions can be presented; with the majority of productions there is sufficient time between the completion of editing and publication for the captioner to create and review the caption file to the required standard.

Live programming presents a considerable challenge; the caption data will need to be streamed in real-time from the captioner to the point of insertion, and synchronisationmethods such as timecode cannot be relied upon.

Whilst a sizeable proportion of a live programme may be spontaneous or ad-libbed, possibly 100% for sporting events, much live programming relies on a running order, sections of pre-scripted material, and prepared and pre-edited video ‘packages’.

Much effort has been made by live programme makers to ensure that teams producing captions have access to running orders and scripts in advance of broadcast – most of the major broadcasters in the UK, France and Australia provide access to such data for their captioners.

This data can be used directly (with suitable correction to phonetic spelling and grammar) or to prepare the captioning platform for any names and places likely to be mentioned on air, but it is less common for access to be made available to video content pre-broadcast. Where this access is granted, the dynamic nature of live production, especially in news, can mean that there is little time between completion of the clip and its inclusion within the broadcast.

Therefore there is a considerable challenge to transcribe and prepare captions for this prepared content within the time available. Techniques based on live origination can be used to accelerate the production process (such as respeaking – a speaker-dependent ASR technique used to transcribe the spoken word into captions via dictation), but these take production effort (which may be focussed elsewhere) and time.

Ideally it should be possible to utilise an Automatic Speech Recognition (ASR) engine to generate the transcripts, but these typically produce transcripts that require more editing time to reach broadcast quality than manual origination would have taken.

Previous experimentation as part of the EU- Bridge Speech Technology consortium indicated that ASR engines trained on very specific domains, such as Broadcast Weather, could achieve very high levels of accuracy – typically over 90% (3). It was recognised that the broader News domain would impact the overall accuracy of such an engine, but it would prove usable if a target accuracy of at least 95% was achieved, or 90% accuracy with fewer than 10% of errors being omissions, for the clips processed, with system confidence scores being used to pre-filter the poor quality transcripts.

Scores lower than these would require sufficient manual review and editing that it would render any time-saving by using ASR worthless; for this project scores were calculated using the approaches described in the section below.

This paper will illustrate how such an automated ASR engine was devised and created with research partners, showing the benefits of context-tuned models and the practical application of Machine Learning across large corpora of data – namely many hours of accurately captioned broadcast news programmes. The challenges of introducing such a system into the live production environment will be discussed, as well as the testing required to ensure that each iteration of the engine reaches the target accuracy levels. The paper concludes with a summarisation of the success so far and some considerations for future applications.

DOWNLOAD THE FULL TECHNICAL PAPER

 

46 thoughts on “Just-in-time prepared captioning for live transmissions

  1. I definitely wanted to send a message so as to say thanks to you for those remarkable items you are placing on this website. My rather long internet investigation has at the end been compensated with professional facts and techniques to talk about with my great friends. I would suppose that most of us visitors actually are undoubtedly blessed to dwell in a superb community with so many awesome people with good tips. I feel pretty lucky to have used the web page and look forward to many more pleasurable times reading here. Thanks a lot once more for a lot of things.

  2. I wanted to write you a very little remark to thank you so much once again for all the amazing principles you have featured in this case. This has been so surprisingly generous with people like you to grant easily all that a number of us could have sold for an electronic book to end up making some money for their own end, even more so given that you might well have tried it in the event you decided. These good ideas in addition served like a great way to be certain that other people have the same dream just as my own to realize way more regarding this problem. Certainly there are several more pleasurable instances ahead for individuals that look over your blog.

  3. My husband and i ended up being so excited when Peter managed to finish up his studies from your ideas he was given in your web site. It is now and again perplexing to simply be releasing helpful hints which most people may have been selling. Therefore we do know we’ve got you to give thanks to for this. All of the explanations you have made, the straightforward website navigation, the relationships you will assist to engender – it’s many exceptional, and it is facilitating our son in addition to our family recognize that that issue is satisfying, and that’s wonderfully serious. Thanks for everything!

  4. I and my friends have already been analyzing the excellent tips and hints from your website and then suddenly developed an awful feeling I never expressed respect to the site owner for them. All of the young boys came absolutely very interested to learn all of them and have in reality been taking pleasure in those things. Many thanks for being very considerate and then for figuring out this sort of superb guides millions of individuals are really desperate to learn about. My sincere regret for not saying thanks to earlier.

  5. I wanted to post a simple word to be able to appreciate you for the marvelous items you are posting at this website. My time intensive internet investigation has finally been recognized with useful facts and strategies to go over with my guests. I would assume that we visitors are very fortunate to live in a really good network with many awesome professionals with valuable ideas. I feel extremely lucky to have discovered your entire web pages and look forward to many more fun minutes reading here. Thanks a lot again for everything.

  6. I together with my guys were found to be looking through the nice secrets and techniques on your web site and then quickly I had a horrible feeling I never thanked the blog owner for them. Those young men appeared to be consequently joyful to read them and already have extremely been taking advantage of these things. Thank you for indeed being very helpful and then for picking this sort of awesome subject areas most people are really needing to discover. My personal honest apologies for not expressing appreciation to you sooner.

  7. Thank you so much for providing individuals with an extremely superb chance to read in detail from this blog. It’s usually very pleasing and as well , packed with a good time for me personally and my office colleagues to visit your web site at the very least 3 times every week to read the newest stuff you have. And lastly, we’re at all times motivated considering the perfect techniques you give. Certain 3 points in this article are ultimately the finest we have all had.

  8. I have to express my affection for your generosity supporting men and women who should have help on the field. Your personal commitment to passing the solution throughout ended up being exceptionally functional and have frequently permitted associates just like me to realize their targets. Your own important guideline signifies much to me and extremely more to my mates. Thanks a lot; from all of us.

  9. A lot of thanks for every one of your hard work on this web page. My aunt enjoys managing internet research and it’s obvious why. I notice all of the powerful manner you offer very important strategies via the website and cause contribution from others about this content so our favorite daughter has always been learning a lot of things. Take advantage of the remaining portion of the year. You’re the one carrying out a pretty cool job.

  10. I precisely had to say thanks once again. I am not sure the things I would’ve followed without the techniques shared by you concerning this problem. It truly was an absolute frightening issue in my view, but coming across this specialised form you processed it forced me to weep with joy. Now i’m happier for the service and thus sincerely hope you know what a powerful job you were carrying out instructing some other people thru your websites. Most likely you have never come across all of us.

  11. I simply had to appreciate you once again. I do not know the things I would have gone through without the actual information provided by you on such a theme. It seemed to be an absolute distressing scenario in my circumstances, nevertheless witnessing the very specialized approach you resolved that forced me to jump with happiness. Extremely happier for your information and thus wish you find out what an amazing job you have been putting in teaching others thru your web site. I’m certain you have never come across any of us.

  12. I wanted to write you one very little observation so as to thank you yet again considering the pleasant pointers you’ve provided on this site. It is extremely open-handed with people like you to allow publicly all that most people could possibly have distributed as an ebook to make some profit on their own, and in particular given that you could have tried it if you ever wanted. Those tricks as well served as the fantastic way to know that many people have similar eagerness the same as my own to realize very much more concerning this issue. I am certain there are numerous more fun opportunities ahead for individuals who read carefully your blog.

  13. I as well as my buddies ended up checking out the good guidelines from your web page and so then developed a horrible feeling I had not expressed respect to you for those tips. All of the young men are already so joyful to learn them and already have quite simply been tapping into these things. Thanks for genuinely quite helpful and for having certain essential information millions of individuals are really needing to understand about. My personal honest regret for not saying thanks to you earlier.

  14. I am only commenting to make you be aware of what a fantastic experience my cousin’s child found browsing your web site. She figured out too many pieces, which include what it’s like to possess a wonderful helping style to have men and women with no trouble thoroughly grasp a variety of specialized matters. You truly surpassed our desires. Many thanks for displaying those necessary, healthy, explanatory and even unique tips on your topic to Janet.

  15. I simply wanted to compose a brief comment in order to thank you for all the great techniques you are posting on this website. My incredibly long internet look up has finally been rewarded with good quality concept to write about with my partners. I would suppose that many of us site visitors actually are truly lucky to dwell in a perfect website with many brilliant individuals with useful principles. I feel rather blessed to have seen your entire webpages and look forward to really more pleasurable moments reading here. Thanks once more for all the details.

  16. I together with my guys have been looking at the excellent helpful hints on your web site then at once came up with an awful suspicion I never thanked you for those tips. Those people happened to be for this reason excited to learn all of them and now have in fact been taking advantage of these things. Thanks for simply being really thoughtful as well as for pick out this form of essential subject matter most people are really wanting to learn about. My personal sincere apologies for not saying thanks to you sooner.

  17. I wanted to create you one little note just to thank you very much the moment again regarding the great things you’ve provided here. This has been incredibly open-handed with you to offer unreservedly what a few people could have distributed for an ebook to make some profit on their own, primarily given that you might have done it if you decided. Those concepts also worked to be a good way to realize that the rest have the identical zeal similar to my own to learn whole lot more related to this matter. Certainly there are millions of more pleasant situations in the future for those who look over your blog.

  18. I am also writing to let you understand what a useful encounter our child undergone studying the blog. She even learned numerous issues, with the inclusion of what it’s like to have an excellent helping mindset to get most people with no trouble fully understand specific very confusing matters. You truly exceeded people’s expectations. I appreciate you for distributing those beneficial, safe, revealing and as well as fun tips about your topic to Mary.

  19. I must show appreciation to this writer just for bailing me out of this trouble. After exploring through the world wide web and coming across tips that were not beneficial, I was thinking my entire life was done. Living without the presence of strategies to the issues you’ve sorted out as a result of your review is a serious case, as well as the ones that would have badly affected my career if I had not discovered your web site. Your own personal talents and kindness in controlling all the pieces was invaluable. I don’t know what I would have done if I had not encountered such a subject like this. It’s possible to now look forward to my future. Thanks a lot so much for the reliable and result oriented help. I will not be reluctant to refer your web blog to any individual who should receive care on this subject matter.

  20. I intended to compose you a bit of word to say thanks yet again for your incredible views you’ve discussed here. It is certainly extremely generous with you to provide unhampered all most of us could possibly have sold as an ebook to end up making some money for their own end, precisely considering that you might have done it in the event you desired. The guidelines additionally acted to be the good way to fully grasp some people have the same passion the same as mine to find out a good deal more on the topic of this condition. I know there are several more pleasant moments ahead for people who scan through your site.

  21. I would like to point out my passion for your kind-heartedness giving support to those who really need help on this one question. Your special commitment to passing the solution all through became wonderfully valuable and has really made women like me to get to their objectives. This invaluable publication means a lot to me and especially to my colleagues. Many thanks; from everyone of us.

  22. Thanks for your whole effort on this web page. My daughter takes pleasure in working on research and it’s really easy to see why. A number of us know all relating to the powerful way you offer good tricks on your web blog and therefore invigorate contribution from website visitors on that matter and our own daughter has been learning a lot. Enjoy the rest of the new year. You are always performing a dazzling job.

  23. I simply wanted to type a remark in order to say thanks to you for the amazing guidelines you are giving at this website. My considerable internet investigation has now been compensated with sensible know-how to go over with my co-workers. I would say that we readers are definitely endowed to exist in a magnificent place with so many brilliant professionals with interesting tips and hints. I feel truly happy to have discovered your webpage and look forward to many more amazing moments reading here. Thank you once again for all the details.

  24. I am glad for commenting to make you know of the fabulous discovery my wife’s daughter went through reading through your blog. She came to find lots of details, which included what it is like to have a great coaching mindset to get folks very easily learn about specified complicated matters. You really surpassed our expectations. Thanks for supplying these informative, trustworthy, explanatory and as well as easy tips on that topic to Julie.

  25. My husband and i felt absolutely more than happy that Raymond managed to finish off his survey from the ideas he received from your own site. It’s not at all simplistic just to continually be handing out strategies people might have been trying to sell. We already know we have got you to be grateful to because of that. The specific illustrations you have made, the easy site navigation, the friendships your site give support to create – it is all spectacular, and it’s really leading our son in addition to us recognize that the issue is enjoyable, and that is wonderfully important. Many thanks for all the pieces!

  26. Thank you for each of your labor on this website. My mom enjoys conducting investigation and it’s easy to see why. Most people learn all concerning the compelling means you convey precious solutions by means of this web blog and in addition cause response from website visitors about this topic while our girl is understanding a great deal. Have fun with the rest of the new year. You’re the one performing a remarkable job.

  27. I and also my friends appeared to be studying the excellent recommendations on your site and then before long developed a horrible feeling I had not thanked the web blog owner for those strategies. All the ladies appeared to be consequently excited to see all of them and have in effect pretty much been enjoying these things. I appreciate you for simply being very helpful and for using such beneficial things millions of individuals are really eager to know about. Our sincere regret for not expressing gratitude to you sooner.

  28. Thanks for each of your labor on this website. My aunt loves getting into internet research and it’s really easy to see why. My partner and i learn all concerning the dynamic way you give powerful suggestions via this web blog and even foster response from the others about this theme while my girl is always being taught a whole lot. Take advantage of the remaining portion of the year. You’re conducting a tremendous job.

  29. I simply desired to say thanks yet again. I’m not certain the things I would’ve done in the absence of the actual ways contributed by you directly on such area of interest. It became an absolute troublesome matter for me, but viewing your specialized mode you treated the issue forced me to jump for delight. Now i’m thankful for this work and even expect you comprehend what a powerful job you have been carrying out teaching other individuals via a blog. I’m certain you haven’t met any of us.

  30. I want to show my respect for your generosity giving support to those people that really need help with that study. Your real commitment to passing the solution all through had been extremely informative and has usually empowered those much like me to reach their goals. Your entire insightful tutorial signifies a whole lot a person like me and still more to my office colleagues. Warm regards; from each one of us.

  31. I just wanted to type a quick comment so as to say thanks to you for these fantastic ideas you are giving at this site. My time consuming internet look up has at the end been rewarded with professional knowledge to share with my contacts. I would believe that most of us readers are very much fortunate to be in a decent community with many special professionals with insightful strategies. I feel somewhat blessed to have seen your website page and look forward to tons of more enjoyable minutes reading here. Thank you once again for all the details.

  32. I precisely needed to thank you very much all over again. I do not know the things I could possibly have worked on without the information revealed by you over this industry. It was before an absolute depressing difficulty in my opinion, however , finding out the specialized approach you dealt with that forced me to leap over delight. I’m just thankful for this service and in addition wish you really know what a great job you happen to be doing training others by way of your site. I’m certain you have never got to know all of us.

  33. There are definitely lots of particulars like that to take into consideration. That is a nice point to carry up. I provide the thoughts above as general inspiration however clearly there are questions just like the one you carry up where crucial thing can be working in trustworthy good faith. I don?t know if greatest practices have emerged around issues like that, however I’m certain that your job is clearly recognized as a good game. Both girls and boys really feel the affect of only a moment抯 pleasure, for the remainder of their lives.

  34. My wife and i got absolutely satisfied when Emmanuel could carry out his inquiry through your ideas he gained from your very own weblog. It’s not at all simplistic to just be releasing key points which often many others might have been selling. And we all know we have you to give thanks to for this. The type of explanations you made, the straightforward website menu, the friendships you aid to create – it’s got everything unbelievable, and it is making our son in addition to the family believe that the theme is cool, and that is seriously fundamental. Thanks for everything!

  35. I抦 impressed, I have to say. Actually hardly ever do I encounter a weblog that抯 both educative and entertaining, and let me tell you, you have hit the nail on the head. Your idea is excellent; the difficulty is something that not sufficient people are talking intelligently about. I’m very completely happy that I stumbled throughout this in my search for one thing referring to this.

  36. I used to be very pleased to find this net-site.I wanted to thanks on your time for this glorious learn!! I undoubtedly enjoying each little bit of it and I have you bookmarked to check out new stuff you blog post.

  37. Once I initially commented I clicked the -Notify me when new comments are added- checkbox and now every time a comment is added I get four emails with the identical comment. Is there any means you’ll be able to take away me from that service? Thanks!

  38. An impressive share, I just given this onto a colleague who was doing a bit analysis on this. And he in actual fact purchased me breakfast because I discovered it for him.. smile. So let me reword that: Thnx for the deal with! However yeah Thnkx for spending the time to discuss this, I really feel strongly about it and love studying more on this topic. If doable, as you develop into experience, would you thoughts updating your weblog with extra particulars? It’s highly helpful for me. Big thumb up for this weblog publish!

  39. I simply wished to thank you so much once more. I am not sure the things that I would have undertaken in the absence of those tips and hints documented by you directly on that topic. It previously was a real fearsome scenario for me, but encountering this expert technique you resolved that forced me to cry with happiness. Extremely thankful for your information and thus expect you are aware of a powerful job you’re putting in training some other people via a web site. Probably you haven’t met all of us.

Leave a Reply

Your email address will not be published.