Extracting low-level features in content based image retrieval systems using open CV environment Essay

EXTRACTING Low LEVEL FEATURES IN CONTENT BASED IMAGE RETRIEVAL SYSTEMS USING OPEN CV ENVIRONMENT

Abstraction—Feature extraction is a important and besides the initial stage of Content Based Image Retrieval ( CBIR ) where the characteristics of an image are extracted utilizing a peculiar technique and thenceforth stored in the signifier of a characteristic vector. This work fundamentally shows the characteristic extraction of images utilizing OPEN CV platform. OPEN CV is a library of programming maps chiefly aimed at real-time computing machine vision. We have used its built-in maps and libraries and extracted the characteristics that include cardinal points, contours and texture of images. The end product to the characteristic extraction can be taken as the input at the following degree of CBIR.

Keywords—Feature extraction, SIFT, SURF, OPEN CV.

I. Introduction

The Content based image retrieval is a procedure of bringing the image from a database of image based on the question produced by the user. It is composed of assorted stages that include feature extraction, similarity matching and indexing [ 4 ] .Feature extraction is one of the of import stages of CBIR without which the whole procedure can be considered to be impossible. At this stage features that comprises of the informations or content of the image are extracted and stored for taking as an input to the following stage. Figure 1 shows the assorted stages of CBIR where being the accent of our work, the characteristic extraction stage is highlighted.

There are several techniques followed under characteristic extraction where SIFT and SURF is one of them as discussed in this work. For the execution portion OPEN CV platform is put into usage. It is an environment provided for image processing execution utilizing C++ interface. Open CV ( Open Source Computer Vision Library ) is a library of programming maps chiefly aimed at real-time computing machine vision, developed by Intel, and now supported by Willow Garage and Itseez [ 1 ] . The chief end behind the launch of OPEN CV was

1. PHASES OF CBIR

to acquire a portable and an optimised codification with programming library maps available for free. OPEN CV was developed by Intel Microprocessor Research Lab which was distributed under a BSD manner licence and allows free commercial or research usage. It is supported under Windows and Linux, but it is besides portable at many other runing systems. It is a unfastened beginning computing machine vision library in C or C++ and optimized for existent clip applications. The paper is organized into assorted subdivisions: Section II explains the characteristic extraction and farther the Sub Section A types of characteristics as loosely classified into degree 1, degree 2 and level 3 characteristics. In the subdivision III the two major techniques under characteristic extraction procedure viz. SIFT and SURF are covered. The following subdivision shows the execution stage of our work.

II. FEATURE EXTRACTION

This is an initial stage of CBIR where the characteristics stored in an image are extracted and farther stored in a characteristic vector. Features are fundamentally the informations or content stored in an image that describes the image. This procedure transforms the content of the images into the signifier of content characteristics. The characteristics therefore extracted are utilised further in choice and categorization stage [ 2 ] .We can see the image to be a two dimensional array taken as { F ( x, Y ) : x=1,2, ….. , X, y=1,2, …. , Y } .

For colored images ( x, y ) is taken as colour value at pel and that for grey graduated table it is taken as grayscale strength value of that pel [ 2 ] .

Feature extraction stage performs the grid creative activity used for the designation of the characteristics on an array and so computes the strength value for each characteristic [ 3 ] .In CBIR systems the images are stored at their corresponding indices [ 4 ] .After the characteristic extraction procedure the characteristics are stored utilizing the characteristic vector as a information construction and hence retrieval of images is done by executing the similarity matching of these characteristic vectors [ 4 ] .The distance ‘D’ calculated between the characteristic vectors of query image ‘Q’ and image database ‘T’ is defined as,

D ( Feature ( Q ) , Feature ( T ) ) ? T ( 1 )

A.Types of Features

Features are fundamentally the information or the content stored by the image that gives a proper description of the image. They are maps defined for one or more measurings taken for the certain features of the object [ 4 ] .The characteristics are represented utilizing the characteristic forms. Features are of following types:

1 )Degree 1 characteristics: These are fundamentally based on the whole image or on the portion of an image every bit good.

• Color: The colour characteristics are extracted and studied by utilizing the colour histograms. Here the matching is done on the footing of the colour histograms which resemble to the colour histogram of the question image most closely [ 6 ] .Other representations for colour characteristic are colour minutes, colour correlograms, coherency vectors [ 8 ] .
• Texture: It is a powerful regional form that supports the image retrieval procedure. It helps in categorization, acknowledgment and happening the similarities between the images from multimedia databases [ 2 ] .Similarity done on the footing of texture helps in separating between countries of images with similar colour [ 6 ] .Texture can be represented utilizing structural that represent texture as texels ( texture component, or texture pel ) that are arranged on a regular basis on surface based on peculiar agreement [ 8 ] and statistical methods that includes Fourier power spectra, accompaniment matrices, shift-invariant chief constituent analysis ( SPCA ) , Tamura features, Wold decomposition, Markov random field, fractal theoretical account, and multi-resolution filtrating techniques such as Gabor and ripple transform, etc [ 2 ] .
• Shape: Shape features comprises of facet ratio, disk shape, Fourier forms, minute invariants, back-to-back boundary sections, etc. Shape forms are used for the representation of form characteristics [ 7 ] .
• Spatial location: It is used in part categorization. This peculiar characteristic may be used to distinguish when colour and texture characteristics are same. It is defined as upper, bottom, top harmonizing to the location of the part in an image [ 7 ] .

2 )Degree 2 characteristics: These are besides known every bit high degree characteristics. They give a semantic or a rational significance to the characteristics of an image. There is a semantic spread between low degree and high degree characteristics [ 9 ] which is defined as a infinite of letdown between the high degree characteristics of CBIR and low degree characteristics that are used for question intent. The low degree characteristics can be translated into their semantic reading by utilizing assorted techniques. The semantic information fundamentally extracts the logic of the low degree characteristics like colour, texture etc as given by the user and processes it further and eventually sets a relevancy feedback for the user and concludes whether the fetched end product is every bit desired or non. The relevancy feedback is obtained in several loops. It fundamentally reduces the semantic spread in CBIR systems [ 10 ] .Following are some major techniques under representation of degree 2 characteristics:

• SFL-Semantic Feature Layer: This technique is fundamentally a design semantically related categories incorporating low degree characteristics and besides includes extra cognition like patterning information, Domain cognition etc. This attack is implemented for high degree characteristics depicting human universe belongingss and for rating of 300 questions [ 9 ] .
• Fuzzy production regulations: This technique takes the infusions low degree characteristics like contrast of chromaticity, light-dark contract, form characteristics and texture characteristics and change over them into high degree semantic characteristics utilizing fuzzed production regulations those are derived from an image excavation technique. The list of constructions incorporating the content for the high degree semantic characteristics is created on the footing of Dempster-Shafer theory [ 11 ] .
• Semantic grouping algorithm: It is a statistical algorithm for keyword notes based on user feedback. This is an intelligent system that can bit by bit larn the user’s seeking wonts in footings of semantic dealingss among constructs and farther utilizations this information for betterment of the retrieval performances [ 12 ] .

3 )Degree 3 characteristics: These characteristics are used for automatic image retrieval [ 6 ] . These are used to analyze the semantic dealingss of user’s hunt. Here semantic grouping of keywords is done utilizing a peculiar algorithm. Keywords are generated from textual and manual notes of images [ 12 ] .

III. TECHNIQUES FOR FEATURE EXTRACTION

There are two celebrated techniques used under characteristic extraction named SIFT and SURF.

A.Surf

SURF stands for Speeded up Robust Features. It is an algorithm following multiple phases and therefore rush up the characteristic extraction procedure in-case of images. The SURF form is based upon the Hessian based scale-space pyramid for seeking of image characteristic points [ 13 ] .It is a method that is perform ant graduated table and rotation-invariant involvement point sensor and form. It is much better than other feature extraction techniques in footings of repeatability, peculiarity, velocity every bit good as hardiness. It is better since it depends upon built-in images for image whirls [ 14 ] .It is the speeded up version of SIFT characteristic extraction algorithm. Surf is three times faster than SIFT and same can be viewed in footings of public presentation besides [ 15 ] .It works good for the images with blurring and rotary motion but does non win good in managing [ 16 ] .

SURF besides helps in automatic image note where it selects the right figure of characteristics and the characteristics itself for note [ 16 ] .Medical image note can besides be done utilizing SURF form and the SVM classifier. There is a Fast-Hessian sensor used to execute characteristic extraction and characteristic matching is performed utilizing SVM with a quadratic meat and on comparing with SVM categorization it resulted into the improved categorization of lung images with 96 % truth. It is counted as a strong tool for medical image note [ 17 ] .

B.SIFT

SIFT stands for Scale Invariant Feature Transform. It was proposed by David Lowe in 1999.Its name conveys its significance as the content of the image is converted to scale invariant co-ordinates relative to local characteristics [ 18 ] .It is one of the techniques in image processing and computing machine vision that is used to capture the scene semantics [ 19 ] .The major stairss covered under SIFT algorithm are as follows [ 20 ] :

1. Find scale infinite extreme point.
2. Key point localisation and filtering.

• Improve key points and throw out bad 1s.

3. Orientation assignment.

• Remove effects of rotary motion and graduated table.

4. Create form.

• Using histograms of orientations.

First two stairss are the parts of Detector and last two are portion of Descriptor [ 21 ] .The algorithm has points or characteristics that are local lower limit and upper limit in a scale infinite and further composed of differences of Gaussians besides known as DOG. Normalized histograms of the gradient waies around those points are used for depicting them [ 21 ] .The ocular contents of a query image and database images can be extracted and further described utilizing 128-dimensional SIFT characteristic vectors. Here indexing and fiting of these SIFT characteristics can be done utilizing KD-tree and Approximate Nearest Neighbor ( ANN ) Search algorithms [ 22 ] .

2. SYSTEM ARCHITECTURE

IV. Execution

This subdivision shows the execution portion of our work where we have extracted some common characteristics from two trial images utilizing OPEN CV platform. It is an environment that provides c++ interface for image processing algorithms. The OPEN CV library is used by a big figure of companies like Intel, IBM, Microsoft, SONY, Siemens, Google, and research Centres Stanford, MIT, CMU, Cambridge, INRIA etc [ 23 ] . It is a unfastened beginning computing machine vision library in C or C++ and optimized for existent clip applications. Figure 2 shows the

system architecture of the work that has been done. The libraries and files can be downloaded and installed from [ 25 ] .

1. We load two different images taken in jpeg format.
2. Infusion characteristics utilizing SURF in OPENCV.
3. Expose the different images where three different characteristics are highlighted individually.

There are three basic characteristics extracted:

1 )Cardinal points: Cardinal points are fundamentally those points in an object of a peculiar image that give the description of characteristics in that object of that image. Such a description can be used to turn up that object in an image incorporating many other objects [ 24 ] . The cardinal points are detected utilizing SURF sensor. The cardinal points are detected and its forms are calculated and eventually those extracted cardinal points are stored in a vector array. After this the cardinal points are drawn and so displayed in the signifier of a new image.There are assorted other algorithms available in OPEN CV for feature sensing like SIFT, ORB, FAST.

2 )Contours: The contours are the lineations of an image. It includes borders and boundaries. They are extracted by declaring and specifying a map in OPEN CV that detects the borders of the image utilizing cagey border sensor. Then the contours are found and drawn from an initial point up to a concluding point therefore making a hierarchy of points. Hierarchy is a vector that contains the information about the image topology. [ 25 ] .Finally, the contours are shown in the signifier of a new image.

3 )Texture: The texture is extracted by utilizing al algorithm provided in OPEN CV.As a consequence the textured informations will be printed on the new image. Therefore, as an end product we get texture of the original image in the signifier of a new image.

V. EXPERIMENT RESULTS

We foremost laod the images and expose them utilizing the imshow map of OPEN CV.The images taken are grey graduated table. Then we have extracted the characteristics of both the images simultaneosly one after the other and these feeatures therefore extracted are stored in a characteristic vectors.Each characteristic is stored in its corresponding characteristic vectyor.Figure 3 shows the two trial images taken whose characteristics are to be taken and below them there matching extracted characteristics are shown in series where at first keypoints are shown, following are the contours and at the terminal we have extracted the texture.The images store a batch of content and are complex to hive away since there are a figure of matrices that are formedSo, the hardware constellations do besides matter while implementing image processing algorithms.

The end product of the characteristic extraction stage can be taken as the input to the following stages of CBIR that may include indexing and similarity matching.

VI. Decision

The paper shows the most important stage of CBIR that is feature extraction and how we have extracted out the characteristics utilizing OPEN CV platform. The characteristics include cardinal points, contours and texture. The extracted characteristics are stored in a characteristic vector and further can be taken as an input to the following stages of CBIR. This is a consecutive execution and to obtain better velocity ups and effectual executing clip we would execute it in parallel utilizing Graphics Processing Unit and CUDA.

3. Consequence

Recognition

The writers would wish to thank Dr.Satvir Singh and Mr. Sarabjit Singh from Shaheed Bhagat Singh, State Technical Campus, ,Ferozepur, India for their valuable aid in explicating this work.

Mentions

1. hypertext transfer protocol: //en.wikipedia.org/wiki/OpenCV
2. Image Feature Extraction Techniques and Their Applications for CBIR and Biometrics Systems, Ryszard S. Chora?s, INTERNATIONAL JOURNAL OF BIOLOGY AND BIOMEDICAL ENGINEERING,2007.
3. Feature Extraction and Normalization Algorithms for high denseness Oligonucleotide Gene Expression Array Data, Erich E Schadt, Cheng Li, Journal of Cellular Biochemistry Supplement,2001.
4. Content based Image Retrieval with Graphical Processing Unit, Bhavneet Kaur, Sonika Jindal, ITC 2014,2014.
5. Feature Extraction with Graphics Processing Unit, Bhavneet Kaur, Sonika Jindal, ICAET 2014, Under the procedure of publacation,2014.
6. Content based image retrieval, John Eakins, Margret Graham,1999.
7. A suvey of content-based image retrieval with high degree semantics, Ying Liu, Dengsheng Zhang, Elsevier,2006.
8. Content-based Image Retrieval: Feature Extraction Techniques and Applications, Amandeep Khokher, Rajneesh Talwar, iRAFIT2012.
9. Semantic Feature Layers in Content Based Image Retrieval: Execution of Human World Features, Horst Eidenberger.
10. Relevance Feedback: A Power Tool for Interactive Content-Based Image Retrieval, Rui, Y. , Huang, T. , Ortega, M. and Mehrotra, IEEE Transactions on Circuits and Systems for Video Technology,1998.
11. Content Based Image Retrieval Using High Level Semantic, Prof. Sharvari Tmane,2008
12. CBIR: From Low-Level Features to High-Level Semantics, Xiang Sean Zhou* , Thomas S. Huang.
13. Fast Scale Invariant Feature Detection and Matching on Programmable Graphics Hardware, Nico Cornelis, Luc Van Gool.
14. hypertext transfer protocol: //www.vision.ee.ethz.ch/~surf/
15. www.opencv.org/Surf/
16. Automatic image note utilizing SURF forms, Muhammed, A.V. , Kumar, G.S. , Sreeraj, M. , IEEE,2012.
17. ANNOTATION OF MEDICAL IMAGES USING THE SURF DESCRIPTOR, Anna Wojnar, Ant?onio M. G. Pinheiro
18. SIFT applied to CBIR, Jurandy Almeida, Ricardo da S. Torres and Siome Goldenstein,2009
19. Qi Zhang, Yurong Chen, Yimin Zhang, Yinlong Xu, SIFT Implementation and Optimization for Multi-Core Systems, IEEE,2008.
20. Distinctive image characteristics from scale-invariant keypoints. David G. Lowe, International Journal of Computer Vision, 60, 2 ( 2004 ) , pp. 91-110
21. CBIR in Cultural Databases for Identification of Images: A Local-Descriptors Approach, Eduardo Valle, Matthieu Cord, Sylvie Philipp-Foliguet
22. Application of Image SIFT Features to the Context of CBIR, Xu Wangming,2008.
23. Pisarevsky, V. : 2007, Intel Corporation, Software and Solution
24. homepages.inf.ed.ac.uk
25. www.opencv.org.