Abstract

In recent years, the use of video conferencing has been extended to games and remote experiments, both of which rely on computer vision algorithms. However, it has been shown that these algorithms can be negatively impacted by the compression necessary for real-time video. To increase the robustness against this issue, a semantic encoding pipeline was built that is based on H.264 and works in combination with WebRTC. Two ROI detection methods based on saliency and face detection, were tested using six videos at two different bitrates (750 and 1500 kbit/s). To evaluate these methods, the resulting encoded videos were transmitted via WebRTC and analyzed using an rPPG and an AU extraction algorithm. The results showed that our semantic encoding methods improve accuracy while only decreasing encoding speed by a small amount. These findings show that such a pipeline is technically viable and, in the future, can help to make games more robust and experiment results more trustable.

Results/Implementation/Project Description

Conclusion

[ PDF (optional) ] 

[ Slides Kickoff/Final (optional)]