Updated: August 31, 2022
In our First and Second parts we examined the topic of real-time video streaming, how it’s done on iOS and took to pieces the process itself. And now:
Let’s Compare The Results Of Video Encoding On Various iOS Devices!
Disclaimer
In order to convey an informative experiment in comparing various video encoding methods, we created a test environment, an application that allows us to measure the results properly.
The app uses three encoders:
- Hardware – accessible via VideoToolbox library.
- Hardware – accessible via AVAssetWriter. For this purpose the realization from kickFlip Library was used – OS broadcasting solution for your iOS applications.
- Software – compiled ffmpeg 3.0 library with compiled as dependency x264 library.
We wanted to find out the limits of each method and conducted research for multiple resolutions used by AVCaptureSession: 352×288, 640×480, 1280×720, 1920×1080, 3480×2160. The handheld devices chosen for the tests: iPhone 4S – the weakest iOS8 device, iPhone 6 plus, and iPad Air 2 – one of the most powerful devices in the market. During the tests we determined and measured the CPU usage and delays when encoding the video in H.264 format.
There is a certain margin of error and the results might differ from the results acquired during such tests in a different environment here. However, our results show the difference in encoding efficiency on various devices and show it quite well.
| ffmpeg with x264 (sw) | AVAssetWriter (kickflip realization) | VideoToolbox | |
|---|---|---|---|
| 352×288 | delay: ~ 0.60 s. CPU Used: ~ 55%-60% | delay: ~ 0.39 – 0.46 s. CPU Used: ~ 8% – 12% | delay: ~ 0.07 – 0.087s. CPU Used: ~ 7% – 9% |
| 640×480 | delay: ~ 0.75 – 0.85 s. CPU Used: ~ 130% – 160% | delay: ~ 0.46 – 0.5 s. CPU Used: ~ 8% – 12% | delay: ~ 0.067 – 0.087s. CPU Used: ~ 7% – 9% |
| 1280×720 | delay: ~ 1.55 – 1.63 s. CPU Used: > 160% | delay: ~ 0.688 – 0.77 s. CPU Used: ~ 8% – 12% | delay: ~ 0.114 – 0.118s. CPU Used: ~ 7% – 9% |
| 1920×1080 | delay: ~ 3.8s. CPU Used: > 160% | delay: ~ 0.84 – 0.88s. CPU Used: ~ 8% – 12% | delay: ~ 0.177 – 0.181s. CPU Used: ~ 7% – 9% |
It is clear that the software encoder overloads the CPU and it fails to deliver reasonable results even working with 640×480 resolution. You can also notice that there is more than 100% load of CPU meaning that some frames will be left out as the CPU won’t be able to process them in time. Obviously, the device battery will die faster.
Hardware encoders work wonderfully and showed great results regardless of resolutions. The average CPU usage was kept within 7-12% limits. We found that AVAssetWriter has a longer delay and the difference is quite noticeable.
iPhone 6 Plus and iPad Air 2
Here are the results after testing the devices.
iPhone 6 Plus:
| ffmpeg with x264 (sw) | AVAssetWriter (kickflip realization) | VideoToolbox | |
|---|---|---|---|
| 352×288 | delay: ~ 0.49 – 0.57 s. CPU Used: ~ 26%-37% | delay: ~ 0.21 – 0.276 s. CPU Used: ~ 9% – 10% | delay: ~ 0.03 – 0.04s. CPU Used: ~ 7% – 8% |
| 640×480 | delay: ~ 0.49 – 0.57 s. CPU Used: ~ 40% – 70% | delay: ~ 0.22 – 0.24 s. CPU Used: ~ 9% – 10% | delay: ~ 0.035s. CPU Used: ~ 7% – 8% |
| 1280×720 | delay: ~ 0.64 – 0.70 s. CPU Used: ~120% – 170% | delay: ~ 0.23 – 0.3 s. CPU Used: ~ 9% – 10% | delay: ~ 0.044 – 0.045s. CPU Used: ~ 8% – 9% |
| 1920×1080 | delay: ~ 1.08 – 1.26s. CPU Used: > 160% | delay: ~ 0.26s. CPU Used: ~ 9% – 10% | delay: ~ 0.0615 – 0.069s. CPU Used: ~ 9% – 10% |
iPad Air 2:
| ffmpeg with x264 (sw) | AVAssetWriter (kickflip realization) | VideoToolbox | |
|---|---|---|---|
| 352×288 | delay: ~ 0.53 – 0.62 s. CPU Used: ~ 40%-50% | delay: ~ 0.29s. CPU Used: ~ 9% – 10% | delay: ~ 0.026s. CPU Used: ~ 6% – 8% |
| 640×480 | delay: ~ 0.53 – 0.58 s. CPU Used: ~ 45% – 60% | delay: ~ 0.29 s. CPU Used: ~ 9% – 10% | delay: ~ 0.029s. CPU Used: ~ 7% – 10% |
| 1280×720 | delay: ~ 0.57 – 0.61 s. CPU Used: ~90% – 170% | delay: ~ 0.3 s. CPU Used: ~ 9% – 10% | delay: ~ 0.03s. CPU Used: ~ 9% – 11% |
| 1920×1080 | delay: ~ 1.76s. CPU Used: 180% – 270% | delay: ~ 0.32s. CPU Used: ~ 9% – 12% | delay: ~ 0.038s. CPU Used: ~ 10% – 12% |
In order to make the data more comprehensible and easy-to-read, we decided to put them on a bar chart. On the charts below you can see how three encoding methods fare against each other.
Apparently, powerful CPUs handle software encoding much better than previous iterations, but such a high CPU load is unacceptable even for modern handheld devices with improved batteries. To top it all off, the software encoding efficiency is much lower than that of hardware encoders.
This small test shows the true advantage that hardware encoders have over the software solutions. VideoToolbox functionality is much more diverse and efficient when it comes to compressing and broadcasting videos.
It is important to note that the delay of AVAssetWriter solution may increase depending on the encoding method used by a developer. If the minimal delay is a goal then VideoToolbox is much more preferable.
Long Live The Battery!
In order to show just how much impact encoding has on an end-user, we conduct a test on the batteries of the devices. We measured how long they can supply power to the device broadcasting a stream. The test was conducted using iPhone 5s with 100% battery and for 1080p resolution. Here’s the data.
The results show clearly that with software encoding the battery lasted less than 2 hours while hardware solutions extended this period to more than 3 hours.
Conclusion
The tests allowed us to determine whether the hardware encoders do their job better than software ones. Moreover, we successfully measure the effectiveness of popular encoding methods for iOS devices and now know exactly which methods to use when making a video broadcasting feature on a iOS device!
Read more about Video Streaming here:
