In few words: Face Hallucination describes Super-Resolution for facial images. It is a technique to improve the quality of noisy or low resultion facial images into high resolution images using knowledge about typical facial features. This technique can subsequently help to identify a person from an image faster and more effectively.

 

Sample Images

On the left image of figure 1 one can see the input from for example a surveillance camera in low resolution and upsampled via bicubic interpolation in order to make the size of the image larger. It is not easy to identify a unique person on the image. Image (d) shows a hallucinated face with a State-of-the-Art face cascaded bi-network (CBN). Image (b) and (c) of figure 1 are intermediate steps and not of interest in this example. The image on the right (e) show the ground truth image for comparison.

Figure 1: Visualization of the inputs outputs and intermediate results with an example image. (1)

It is obvious that a hallucinated face helps to improve detection performance and thus became a strong research area in the recent years.

Network Structure

One State-of-the-Art network shall be described in detail in this section to understand the functionalities. The Cascaded Bi-Network (CBN) was published in 2016 and has the following architecture.

Figure 2: Network architecture of CBN. Two branches are merged together with a gate branch. (1)

The network can be devided into three parts. First part is the common branch. This branch acts as a simple super resolution neural network. It predicts the residual image between bicubic interpolation and ground truth. It´s input are bicubic interpolated images. The second part is the high frequency branch. This part gets bicubic interpolated images and 10 contour maps, which are generated from all training examples prior the training. Each map contains the contour or a facial feature like, mouth, nose, etc. These two branches are merged with a gate which is the third part of the network. This network has as input all input from the branches before plus their output. All parts consist of convolutional neural networks and are trained independently.

Figure 3 illustrates some results from this network:

Figure 3: Results of CBN with three examples compared to 2 other approaches. (1)

 

The CBN approach outperforms in comparison to other state of the art techniques.

Common metrics to measure the performance of Face Hallucination are SSIM and PSNR. See Super-Resolution for more information about the metrics.

 

Applications

Current application possibilities are:

  • Improving automatical face recognition or face identification
  • Improving facial landmark detection
  • Image enhancement
  • Image compression

Literature

  1. http://personal.ie.cuhk.edu.hk/~ccloy/files/eccv_2016_hallucination.pdf (Deep Cascaded Bi-Network for Face Hallucination; Shizhan Zhu, Sifei Liu, Chen Change Loy, Xiaoou Tang)
  1. https://people.csail.mit.edu/celiu/FaceHallucination/FaceHallucination_IJCV.pdf
  2. http://ijarcet.org/wp-content/uploads/IJARCET-VOL-4-ISSUE-4-1633-1639.pdf

2 Kommentare

  1. Unbekannter Benutzer (ga63muv) sagt:

    Hi, Martin,

    All the comments are SUGGESTIONS and are obviously highly subjective!

    From comments:

    1. Normally, you need to introduce abbreviations when they are first used(e.g. CBN). Then you can use them in the later parts. Apart from this, you may forgot to explain what are "CSCN, SSIM and PSNR" in the picture.
    2. It would be better if you label the figures so that you can refer to them and avoid such expressions like "the following figure".
    3. In my opinion, the content starts from "The following figure" in chapter Network Structure could be transformed to Conclusion to make your content more plentiful.
    4. Maybe it is not enough when you only showed the structure of the network and did a quite brief introduction of those 3 parts from the network. 
    5. You may need explain every variable in the structure.

     

    Confusions:

    1. What does the word "larger“ mean in the first sentence from the section "Sample Images"? 
    2. After looking through the paper your article based on, I found that maybe you ignored some important(maybe unimportant) messages, for example, the Iteration？
    3. I'm quite confused about how does this Neural Network work.

    Some Suggested Corrections:

    • On the left Image you can see the input from i.e. a surveillance camera in low resolution and upsampled via bicubic interpolation in order to make the image larger. 
    • On the left Image you can see the input from (e.g. ?) a surveillance camera in low resolution and upsampled via bicubic interpolation in order to make the image larger. 
    • It is not easy to identify a unique person on the image.
    • It is not easy to identify an unique person on the image.
    • The Image on the right show the ground truth for comparison.
    • The Image on the right shows the ground truth for comparison.

    • It is obvious that a hallucinated face helps to improve detection performance and thus became a strong research area in the recent years.

    • It is obvious that a hallucinated face helps to improve detection performance and thus has become a strong research area in recent years.

    •  It´s input are bicubic interpolated images.

    •  Its inputs are bicubic interpolated images.

    •  which are generated from all training examples prior the training. 

    •  which are generated from the same data sets used in the training process.

    • Each map contains the contour or a facial feature like, mouth, nose, etc.

    • Each map contains the contour or a facial feature, like mouth, nose, etc.

    • This network has as input all input from the branches before plus their output.

    • This network has all inputs from the branches before plus their output as inputs.

    • The CBN approach is compared to other state of the art techniques and outperforms them quite well.

    • Compared to other state of the art techniques, the CBN approach outperforms them quite well.


    Conclusion:
    • This wiki maybe a little bit too brief, because after finishing reading I just know this network is qualified for Face Hallucination but I still confused how to do it.
    • Most of the keywords are not mentioned in the wiki.
    • The resources in the Weblinks are quite well. They are very helpful to make the reader understand the principle of Face Hallucination. It maybe better if you can add more content from those 2 papers in the Weblinks.

  2. Unbekannter Benutzer (ga69taq) sagt:

    General problems and suggestions:

    • Please refer to your images with figure labels if you wish to explain them in your text.
    • Add image sources to your bibliography/weblinks instead of under the image without a hyperlink.
    • Please define your abbreviations at least once before using them (CSCN, SSIM, PSNR etc. )

    Consistency problems and suggestions:

    • Your capitalization is inconsistent throughout the article.
    • Your bibliography style is inconsistent with the rest of the wiki (and with template).

    Corrections (note that some of the corrections have more than one changes):

    • In few words, Face Hallucination describes
    • In a few words: Face Hallucination describes (or Briefly, Face Hallucination describes)
    • This can later help to identify a person on a picture faster and more effective.
    • This technique can subsequently help to identify a person on a picture faster and more effectively.
    • On the left Image you can see the input from i.e. a surveillance camera in low resolution and upsampled via bicubic interpolation in order to make the image larger.
    • Figure x (left) shows the input (for example from a surveillance camera) in low resolution and upsampled via bicubic interpolation in order to make the image larger. (i.e. means "id est" (namely), not "in example")
    • to identify a unique person on the image.
    • to identify a unique person from the image.
    • Image (d) shows a State-of-the-Art hallucinated face.
    • Figure x (d) shows a state of the art hallucinated face.
    • (b) and (c) are intermediate steps
    • Figure x (b) and (c) are intermediate steps
    • The Image on the right show
    • The image on the right in figure x shows
    • that a hallucinated face helps to improve detection performance
    • that a hallucinated face improves detection performance
    • One State-of-the-Art network shall be described in detail in this section to understand the functionalities.
    • An exemplary state of the art network is described in detail in this section in order to explain the functionalities.
    • and has the following architecture
    • and has the architecture shown in Figure x.
    • The network can be devided into 3 parts.
    • The network can be divided into three parts.
    • the common Branch.
    • the common branch.
    • It´s input are
    • Its inputs are
    • is a high frequency branch.
    • is the high frequency branch.
    • prior the training.
    • prior to the training.
    • Each map contains the contour or a facial feature like, mouth, nose, etc.
    • (This sentence was difficult to understand for me. The contour of what?)
    • are merged with a Gate
    • are merged with a gate
    • This network has as input all input from the branches before plus their output.
    • The inputs of this network are all inputs from earlier branches plus their output.
    • The following figure illustrates some results from this network:
    • Figure x illustrates some results from this network:
    • The CBN approach is compared to other state of the art techniques and outperforms them quite well.
    • The CBN approach outperforms in comparison to other state of the art techniques.
    • Nowadays applications can be:
    • Current application possibilities are:
    • Improve automatical face recognition or face identification
    • Improving automatical face recognition or face identification
    • Improve facial landmark detection
    • Improving facial landmark detection

    Final comments:

    • It is important to note that some of the corrections / suggestions are subjective and for you to decide
    • Most of the issues I pointed out under the general topics are omitted in the correction. Also, some of the stuff I point out are most likely duplicates of the previous review.
    • A really good and detailed explanation one of the cutting edge methods featuring. The "Network Structure" could be clarified some more.