:
AWS (Amazon) Unveils New Technology Using Artificial Intelligence To Automatically Convert Horizontal Videos To Vertical For Mobile Phones At The NAB Convention This Week
Amazon announced today at the NAB convention in Las Vegas that it has developed two new artificial intelligence tools using Elemental Inference that will allow television networks to convert horizontal videos into vertical for mobile phone viewing. It was stated by Amazon’s developers that this tool will be useful for television networks looking to attract the growing number of young adults watching short-form sports videos such as those found on Tik Tok.
A demonstration of how the new technology works was given on Tuesday by the developers of the Elemental Inference. A demonstration of how the new technology works was shown using a real basketball game, demonstrating the ability of the technology to track the action on the court and display a vertical view of the play with a lag of only 200 milliseconds. According to reports from the NAB convention, the Elemental Inference is a completely managed service that performs both live encoding and AI inference simultaneously. The live encoding provides a standard output of the video, while the AI refines the output based upon where the primary subject matter is located within each frame. Therefore, no additional camera operator is needed to perform the necessary editing tasks.
This announcement comes at a time when there are many television networks experiencing fragmentation in terms of audience demographics. Younger viewers are watching sports clips on mobile devices in vertical format (such as Tik Tok and Instagram), while older viewers are still primarily watching sports on their televisions in horizontal format. One result of this demographic shift is that television networks are finding it increasingly difficult to create separate vertical video products for mobile phones without duplicating much of their production staff or creating lower quality automated versions of these products. However, with the introduction of Elemental Inference and AI-based tools like this, television networks may now have an alternative way to produce vertical video products quickly and efficiently.
Get the latest model rankings, product launches, and evaluation insights delivered to your inbox.
TVTechnology.com published a report from the NAB convention, which reported that the Elemental Inference is part of Amazon Web Services' larger strategy to support AI-based video production workflows. Reportedly, other demonstrations at the NAB convention included AI-based clipping tools for sports producers and AI-based tools for identifying athletes during live broadcasts. All three technologies address the common goal of allowing television networks to provide content to more diverse types of consumers with limited changes to current production processes.
As described above, Amazon's technology uses AI algorithms to analyze video in real time, identify key players and actions, and determine optimal framing boundaries for a vertical screen. While Amazon has not revealed details about the underlying neural network models used for the Elemental Inference or sources of data used to train these models, it is clear that large amounts of high-quality video data would be required for this type of application.
It remains uncertain how well this product addresses less predictable edge cases. As previously mentioned, sports programs typically involve repetitive player movement in specific locations on the field. On the other hand, live events often feature unpredictable elements such as audience reaction, sideline drama, multiple-angle replay sequences, etc. The fact that Amazon reports that the system produces results in approximately 200 milliseconds indicates that little buffer room exists for correcting errors.
Finally, Amazon has yet to announce pricing information for this product or indicate when this product will become available to customers. However, according to TVTechnology.com's report from NAB 2019, AWS has been marketing its Elemental Media Services product line (which includes Elemental Inference) as a comprehensive set of video processing services for use by media companies around the world. These services reportedly handle video processing for several prominent media companies (including Fox Sports and Discovery).
One reason why this technology may represent an opportunity for broadcasters is that the process of providing platform-specific content does not require additional production crews. Further, the relatively low latency inherent in the AI-based reframing process should enable near-live streaming capabilities. Therefore, sports leagues can begin generating vertical clips during games; local news outlets can generate mobile-friendly content without significantly changing their overall work flow; etc. However, due to the significant infrastructure requirements associated with performing real-time AI inference at broadcast scales (especially with 4k resolutions), this capability is likely to be available first to well-funded and well-equipped broadcasters.
In addition to AWS's demonstration of its AI-based reframing capabilities at NAB 2019, there were other vendors at the conference showing similar AI-based tools for video production. Thus, it seems that this trend toward AI-driven video production is becoming an industry wide phenomenon.