Google is using EXO file format to save videos


there is no way to open the .exo file since it is encrypted and the video file is split into chunks of .exo files. Even if you have one offline video stored in your phone you can find more than one .exo file under Internal Storage/Android/data/com.youtube.com directory. They are nothing but the encrypted exo files that can only be opened through the Youtube App. From the xda-developers forum post,
1- Google is using EXO file format to save videos. This is apparently a file format used for system files and while we managed to copy these EXO files onto a computer, it can be apparently played only through the YouTube Android app.
2- When you request a download for an offline video in the YouTube app, the video is downloaded in chunks. For example, a 3-minute file we downloaded was saved in five parts. It is possible that only the YouTube app for Android can make sense of these chunks and join them together.
3- We also feel there is real-time encryption going on here. When the video is downloaded, not only it is broken into parts but is also converted to EXO format, compressed and encrypted in real time. This has several benefits, the one obvious benefit is that people can't copy-paste the video and share it indiscriminately. The compression, meanwhile, saves space on the disk. For example, the 3-minute long video that we downloaded actually consumed around 40MB data. But when stored on the phone, the video took only around 25mb space.
4- When a user plays the video, it is decrypted in real time. It is possible that only YouTube's app can do this decryption.
5- If you share the offline video using Bluetooth, instead of the actual file, a web link to the video is sent to the recipient.
However you may want to look into this Exoplayer Library on Github and see if that helps.
ExoPlayer is an application level media player for Android. It provides an alternative to Android’s MediaPlayer API for playing audio and video both locally and over the Internet.