More and more photos are captured in our daily life due to the ubiquitous presence of capture devices such as digital camera, mobile phone, and camcorder. It is possible that we capture thousands of photos in a travel, which we could not imagine before. Along with the exposure of the large number of photos, most of the photos are just stored in the computer because it is difficult to manage the photos manually.
Nowadays, social network becomes a new popular platform for people to interact with each other. Thus, the users of social networks upload their photos onto the social network not only for sharing their life experience but also jogging their own memory.
Moreover, users can tag the people appearing in a photo with their names so that the system can notify the tagged users and get a good way to manage and search the photos easily afterward. However, tagging all uploaded photos is a time-consuming task for general users. Hence, if some tags can be recommended automatically by the system, users can accomplish the tagging much quickly. For that reason, with the benefits arising with social network development, the photo tagging recommendation is the goal to achieve in this thesis.
To provide an appropriate tagging recommendation, using the face detection and recognition to identify the people in a photo is a direct method to achieve this goal.
However, with the increasing number of people, the features of faces are insufficient to distinguish the difference between them. As a result, the accuracy of face recognition decreases when the number of people increases. In addition, uncontrolled situation such as ambient illumination and capture angle of faces are also the bottlenecks for practical face detection and recognition. Furthermore, most of people might use the graphics editing program to make the photos look better before
2
uploading.
In general, to recognize the people in a photo, we can quickly make judgments regarding many aspects including their demographic description and identity if they are familiar to us. Some questions related to the activities of, emotional states of, and relationships between people in a photo can be answered by us. In other words, we draw conclusions based on not just what we see, but also a life time of experience of living and interacting with other people. Based on the observations, we adopt the relationship among the people to filter out a portion of people which are more possible to appear in the photo, and then apply face recognition on the filtered people.
In real world, it is difficult to quantize the relationship among people. Thanks to the rapid development of social network, the relationships between people can be retrieved by their interactions on the social network. For instance, if person A and person B usually appear in photos simultaneously, we can say that person B is closer with person A than others people. Afterward, when person A uploads a new photo, the probability of appearance with user B is higher than other persons. This kind of information becomes a complementation when we recognize a person in a photo.
In social network, the data which we use to retrieve the relationship is called social context. Two kinds of social context are used in our thesis: the number of co-occurrence of two persons in a photo and the number of common friends that two persons share. The relationship can be evaluated from the social contexts, and we are able to apply the relationship on the face recognition. For example, we define a photo that is going to be tagged as the query photo and the faces in the photo as the query faces. Assume that the person uploads a photo with one face already tagged. We can
find a group of people that are related to that person of the tagged face, and only the people in the group need to be considered in face recognition. As a result, we can recommend a list of people who are more possible to appear in the photo for the faces
3
that are not recognized yet. In existing studies, only the relationship between the user who uploads the photo and other users are considered. Actually, the relationship among the people in a photo is also an important cue to recommend a list of people.
In this thesis, for a query photo, we assume that the uploading user has tagged one face, termed as the known face and our goal is to find a group of people who are related to the identity of the known face. The people are retrieved from their faces by using face recognition at first. Then the relationships among the people are evaluated, and a list of people is recommended to the photo tagging.
In experiments, we use the photos on Facebook from 94 volunteers to demonstrate the performance of the proposed framework, and satisfactory experimental results are obtained.
In Chapter 2, we review previous works on tag recommendation using face detection and face recognition. In Chapter 3, we present our proposed system, including Face recognition and social context used. Chapter 4 shows the experimental results. At last, we will make a conclusion and discuss the future work in Chapter 5.
4