We have shown a structure-based image segmentation framework that use structure con-sensus for foregrounded and background separation. Our method combines principles from image completion and foreground extraction approaches into a powerful unified en-gine. And we demonstrate the high precision of our method in indistinguishable images by only one box as input. Our method can greatly decrease user interaction and benefits to further automatic segmentation systems.
Our method still has a few limitations: the input image should content sufficient back-ground information. And if there is another object identical to the target object in the image, our system may regard the object as parts of background and will fail in this situation.
26
Bibliography
Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. ”grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph., 23(3):309–
314, August 2004. ISSN 0730-0301. doi: 10.1145/1015706.1015720. URL http:
//doi.acm.org/10.1145/1015706.1015720.
Victor S. Lempitsky, Pushmeet Kohli, Carsten Rother, and Toby Sharp. Image seg-mentation with a bounding box prior. In ICCV, pages 277–284. IEEE, 2009. URL http://dblp.uni-trier.de/db/conf/iccv/iccv2009.html#LempitskyKRS09.
Y.Y. Boykov and M.-P. Jolly. Interactive graph cuts for optimal boundary amp; region segmentation of objects in n-d images. In Computer Vision, 2001. ICCV 2001. Proceed-ings. Eighth IEEE International Conference on, volume 1, pages 105–112 vol.1, 2001.
doi: 10.1109/ICCV.2001.937505.
Carsten Rother, Tom Minka, Andrew Blake, and Vladimir Kolmogorov. Cosegmentation of image pairs by histogram matching - incorporating a global constraint into mrfs. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1, CVPR ’06, pages 993–1000, Washington, DC, USA, 2006. IEEE Computer Society. ISBN 0-7695-2597-0. doi: 10.1109/CVPR.2006.91. URL http://dx.doi.org/10.1109/CVPR.2006.91.
Nir Friedman and Stuart Russell. Image segmentation in video sequences: a probabilistic approach. In Proceedings of the Thirteenth conference on Uncertainty in artificial in-telligence, UAI’97, pages 175–181, San Francisco, CA, USA, 1997. Morgan Kaufmann
27
Reference 28
Publishers Inc. ISBN 1-55860-485-5. URL http://dl.acm.org/citation.cfm?id=
2074226.2074247.
Stas Goferman, Lihi Zelnik-Manor, and Ayellet Tal. Context-aware saliency detection.
Pattern Analysis and Machine Intelligence, IEEE Transactions on, 34(10):1915–1926, 2012. ISSN 0162-8828. doi: 10.1109/TPAMI.2011.272.
Yuri Boykov, Olga Veksler, and Ramin Zabih. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell., 23(11):1222–1239, 2001. URL http://dblp.uni-trier.de/db/journals/pami/pami23.html#BoykovVZ01.
C. Nieuwenhuis and D. Cremers. Spatially varying color distributions for interactive multi-label segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(5):1234–1247, 2013. ISSN 0162-8828. doi: http://doi.ieeecomputersociety.org/10.
1109/TPAMI.2012.183.
Yin Li, Jian Sun, Chi-Keung Tang, and Heung-Yeung Shum. Lazy snapping. ACM Trans.
Graph., 23(3):303–308, August 2004. ISSN 0730-0301. doi: 10.1145/1015706.1015719.
URL http://doi.acm.org/10.1145/1015706.1015719.
V. Gulshan, C. Rother, Antonio Criminisi, A. Blake, and A. Zisserman. Geodesic star convexity for interactive image segmentation. In Computer Vision and Pat-tern Recognition (CVPR), 2010 IEEE Conference on, pages 3129–3136, 2010. doi:
10.1109/CVPR.2010.5540073.
A. Criminisi, P. Perez, and K. Toyama. Object removal by exemplar-based inpainting.
In Computer Vision and Pattern Recognition, 2003. Proceedings. 2003 IEEE Computer Society Conference on, volume 2, pages II–721–II–728 vol.2, 2003. doi: 10.1109/CVPR.
2003.1211538.
Yael Pritch, Eitam Kav-Venaki, and Shmuel Peleg. Shift-map image editing. In IEEE 12th International Conference on Computer Vision, ICCV 2009, Kyoto, Japan, September 27 - October 4, 2009, pages 151–158. IEEE, 2009. doi: http://dx.doi.org/10.1109/
ICCV.2009.5459159.
Reference 29
Soheil Darabi, Eli Shechtman, Connelly Barnes, Dan B. Goldman, and Pradeep Sen. Im-age melding: combining inconsistent imIm-ages using patch-based synthesis. ACM Trans.
Graph., 31(4):82, 2012. URL http://dblp.uni-trier.de/db/journals/tog/tog31.
html#DarabiSBGS12.
L. Grady. Random walks for image segmentation. Pattern Analysis and Machine In-telligence, IEEE Transactions on, 28(11):1768–1783, 2006. ISSN 0162-8828. doi:
10.1109/TPAMI.2006.233.
Jiangyu Liu, Jian Sun, and Heung-Yeung Shum. Paint selection. In ACM SIGGRAPH 2009 papers, SIGGRAPH ’09, pages 69:1–69:7, New York, NY, USA, 2009. ACM. ISBN 978-1-60558-726-4. doi: 10.1145/1576246.1531375. URL http://doi.acm.org/10.
1145/1576246.1531375.
Bryan C. Russell, Antonio Torralba, Kevin P. Murphy, and William T. Freeman. Labelme:
A database and web-based tool for image annotation. Int. J. Comput. Vision, 77 (1-3):157–173, May 2008. ISSN 0920-5691. doi: 10.1007/s11263-007-0090-8. URL http://dx.doi.org/10.1007/s11263-007-0090-8.
Qiong Yan, Li Xu, Jianping Shi, and Jiaya Jia. Complex scene saliency dataset.
http://www.cse.cuhk.edu.hk/leojia/projects/hsaliency/dataset.html.
M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PAS-CAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html.
Connelly Barnes, Eli Shechtman, Dan B. Goldman, and Adam Finkelstein. The gen-eralized patchmatch correspondence algorithm. In Kostas Daniilidis, Petros Mara-gos, and Nikos Paragios, editors, ECCV (3), volume 6313 of Lecture Notes in Com-puter Science, pages 29–43. Springer, 2010. ISBN 978-3-642-15557-4. URL http:
//dblp.uni-trier.de/db/conf/eccv/eccv2010-3.html#BarnesSGF10.
Reference 30
Yonatan Wexler, Eli Shechtman, and Michal Irani. Space-time completion of video.
IEEE Trans. Pattern Anal. Mach. Intell., 29(3):463–476, 2007. URL http://dblp.
uni-trier.de/db/journals/pami/pami29.html#WexlerSI07.
Johannes Kopf, Wolf Kienzle, Steven M. Drucker, and Sing Bing Kang. Quality pre-diction for image completion. ACM Trans. Graph., 31(6):131, 2012. URL http:
//dblp.uni-trier.de/db/journals/tog/tog31.html#KopfKDK12.
Patrick P´erez, Michel Gangnet, and Andrew Blake. Poisson image editing. In ACM SIGGRAPH 2003 Papers, SIGGRAPH ’03, pages 313–318, New York, NY, USA, 2003.
ACM. ISBN 1-58113-709-5. doi: 10.1145/1201775.882269. URLhttp://doi.acm.org/
10.1145/1201775.882269.
Hui Fang and John C. Hart. Detail preserving shape deformation in image editing. ACM Trans. Graph., 26(3):12, 2007. URL http://dblp.uni-trier.de/db/journals/tog/
tog26.html#FangH07.
Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B. Goldman. Patchmatch:
a randomized correspondence algorithm for structural image editing. ACM Trans.
Graph., 28(3), 2009. URL http://dblp.uni-trier.de/db/journals/tog/tog28.
html#BarnesSFG09.
Yu-Shuen Wang, Chiew-Lan Tai, Olga Sorkine, and Tong-Yee Lee. Optimized scale-and-stretch for image resizing. In ACM SIGGRAPH Asia 2008 papers, SIGGRAPH Asia
’08, pages 118:1–118:8, New York, NY, USA, 2008. ACM. ISBN 978-1-4503-1831-0. doi:
10.1145/1457515.1409071. URL http://doi.acm.org/10.1145/1457515.1409071.
Vivek Kwatra, Arno Sch¨odl, Irfan Essa, Greg Turk, and Aaron Bobick. Graphcut textures:
image and video synthesis using graph cuts. ACM Trans. Graph., 22(3):277–286, July 2003. ISSN 0730-0301. doi: 10.1145/882262.882264. URL http://doi.acm.org/10.
1145/882262.882264.
Laurent Itti, Christof Koch, and Ernst Niebur. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell., 20(11):1254–1259,
Reference 31
November 1998. ISSN 0162-8828. doi: 10.1109/34.730558. URL http://dx.doi.org/
10.1109/34.730558.
Vivek Kwatra, Irfan Essa, Aaron Bobick, and Nipun Kwatra. Texture optimization for example-based synthesis. ACM Trans. Graph., 24(3):795–802, July 2005. ISSN 0730-0301. doi: 10.1145/1073204.1073263. URL http://doi.acm.org/10.1145/1073204.
1073263.