Data mining and data warehouse Essay

An Overview

Data excavation and information warehouse go new phenomenon in this new digital epoch. Both of this nomenclature gives the greatest impact in an organisation. As we know, the information crucially generated for some intents. Nowadays, there is much information system such Decision Support System ( DSS ) , Executive Information System ( EIS ) and Management Information System ( MIS ) which assist an organisation to back up day-to-day operational degree of the concern and aid in doing better determination devising. Most of companies today merely specialized and focused on specific field or countries of the concern such human resource, finance, procurance and etc. As a consequence, users face troubles in accessing complete information. Normally, information between these systems is non well-integrated and shared consistently. When there is the demand of certain information from different positions, some of the systems non able to supply the coveted information, whereas this will take to black determination devising. From the above state of affairs, an organisation should hold better integrated solution so that it will assist the managerial degree in better determination devising and job resolution.

Data Warehouse Concept

Top direction degree or the executive degree ever faces the troubles in doing better determination. In organisation, determination normally used for some intents such to develop concern policies or some processs for assorted concern activities. For the quality and long-run managerial determinations, an organisation should hold to use tremendous volumes of informations which is kept in the information systems or databases. Data warehouse, which is act like a databases, shop a immense figure of informations that can be used in back uping determination devising activities. Friedland, Liam ( 1998 ) stated that “data warehouse fundamentally focused on the database system whereby it provides users to entree desired informations. By utilizing informations warehouse, informations can be presented in legion ways to heighten the apprehension on how good the concern working hence giving speedy response for certain query.”

In database direction system, both informations warehouse and on-line dealing processing ( OLTP ) is interrelated with each other. OLTP is more refering on entering minutess, where this system able to optimise the velocity of recorded minutess. The usage of OLTP can be seen on the recognition card use, booking a plane ticket, or either purchase goods or merchandises via online. Each of the minutess will be recorded and will be kept in the database. The relation between informations warehouse and OLTP is the information which located in informations warehouse system is originally from OLTP beginnings.

What Is Data Mining?

Data excavation can be defined as the procedure of pull outing valid informations from different position of point and sum uping it into meaningful information. Basically, information excavation consisting with particular package, whereby it helps in finding unknown relationship among informations, particularly the informations comes from legion of databases. In concern environment, informations excavation can be used to deduce forms and tendencies that exist in the information. For illustration, an organisation can calculate their company gross revenues, and aiming mailings towards specific clients. Technically, informations excavation is the procedure of detecting relationship among a immense sum of informations in big relational databases.

Mena ( 1999 ) place informations excavation as the procedure of finding actionable and important theoretical accounts or form ; profiles and manner by utilizing acknowledgment engineerings manner or tools such familial algorithm and determination tree methods. Data excavation can besides be defined as the ability of sophisticated informations hunt in utilizing statistical algorithms to find the relationships and forms in a information ( Newton ‘s Telecom Dictionary ) . Therefore, informations excavation has become a research country whereby it go on pull more attending refering on concern and other scientific communities.

Another significance that can be used is data excavation is a procedure of detecting new, which involves interesting cognition including the tendencies, ordinances, anomalousnesss, relationships and other important characteristics from a immense volume of informations that is stored in information depositories. Data excavation can play as a cardinal component in the Knowledge Discovery in Databases ( KDD ) procedure including informations choice ( relevant informations retrieved from database ) , informations cleansing ( noise, losing informations is handled ) , informations integrating ( data enrichment ) , informations transmutation ( coding ) , informations excavation and cognition representations ( techniques used to show cognition ) .

Data Warehouse And Data Mining Relationship

Both informations warehouse and information excavation drama an architectural foundation of determination support systems. Data warehouse fundamentally act as the phases for successful informations excavation activities. Although informations excavation can be done without informations warehouse, but informations warehouse facilitate in heightening the opportunities of great accomplishment in informations excavation. In informations warehouse, the characteristics such incorporate informations, elaborate and summarized informations, historical informations and metadata can be considered as the phases of successful informations excavation procedure. Integrated informations will help the mineworker in seeking across a huge sum of informations rapidly and without troubles. Integrated information warehouse has all these undertakings including reconstituted, encoded values reconciled, constructions of informations standardized and many more so that the mineworker does n’t necessitate to pass excessively much of clip cleaning and conditioning the information before informations excavation procedure could be started successfully.

Both item and summarized informations comprised in informations warehouse. Detail information will guarantee the child to detect informations in farinaceous signifier. At the same clip, summarized informations will avoid the mineworker to reiterate the work person else already done before get downing the geographic expedition procedure. Besides that, summarized informations besides guarantee the mineworker to construct on the work of others instead than to construct everything from the base so that the mineworker does n’t necessitate to make unneeded occupation. In informations warehouse, historical information is really critical to the mineworker because valuable information stored at that place. It facilitate in giving the apprehension on the chronological of concern. On the other manus, metadata is used to depict the context of the information, non the content. When there is no justification on the significance of informations within the natural content of informations, this will take troubles for informations mineworkers to work with. Apart from the account above, informations excavation can work successfully and expeditiously by collaborating with informations warehouse. Data warehouse provides the phase for efficient geographic expedition of informations.

Data Mining Tools / Techniques

In order to pull out the coveted information from a big measure of informations successfully, there is the demand of techniques or tools so that the procedure can work expeditiously. Data excavation tools or techniques may assist to calculate the hereafter of concern tendencies therefore facilitate in doing good knowledge-driven determination. By engagement package and hardware platform, informations excavation techniques used in order to better the necessity of bing information resources. Other than that, the techniques can besides incorporate with merchandises and systems via on-line environment. In concern field, the major purpose of informations mining techniques is to back up redesign concern procedure, where the important cognition hidden in a immense volume of informations will be extracted and maintained by an organisation via informations excavation theoretical accounts.

In informations excavation techniques and systems, it can be classified based on the database, the cognition to be discovered and the techniques to be utilised. As we know, there are legion types of databases used in companies like relational databases and object oriented databases. Data excavation system can be classified depends on the types and the design of databases used. Besides, informations excavation system can sort assorted types of cognition based on the degree of the cognition discovered. Data excavation techniques such goaded method and other attacks can besides categorise the information excavation systems. Therefore, there are several major informations mining techniques including determination tree attack, familial algorithm, constellating, visual image, and associatory memories.

Decision Tree

Decision tree is the most popular technique which is used to foretell and sort. Decision tree is arboreal constructions that represent sets of determination. These determinations will build regulations for the categorization of a dataset. This tool is fundamentally based on simple tree theoretical accounts where the information set is classified strategically into different categories and subclasses at each subdivision during tree growing. This method will ease in doing good picks, particularly determinations that engage in high cost and hazards. Decision tree technique is compatible for undertaking director, concern analyst, or project decision-maker who involves crucially in determination devising procedure. The chief advantage of determination tree is the user can stand for determination options, possible results, and opportunity events schematically. Besides that, the user can rapidly show complicated options clearly. Even without complete information, determination tree may help the user to compare viing options or options.

The determination tree notation. The root node is the early determination from which a determination tree is established. Decision subdivision qualify a peculiar determination option. Two or more determination subdivisions are drawn to the right from a determination node. A opportunity node clarifies an event in determination tree where the point of ambiguity exists. At this point, at least two possible consequences will be generated. The end point terminates a subdivision, where a determination tree is ended when all subdivision waies consequence in an end point with a final payment value. Chance subdivision or opportunity result is the possible consequence which is generated from the opportunity subdivision. Two or more opportunity subdivisions are lines drawn to the right from a opportunity node. Double-hatch Markss are a brace of little lines which is placed over a subdivision to demo that peculiar subdivision is non to be considered in an estimated value computation. For the last one, determination node, acts as the way on a determination tree where a determination between at least two likely options can be made.

Clustering

Clustering techniques fundamentally related to informations records in the database, where the information records are classified based on the similarities. This technique engages informations preprocessing that covers each reduced information record which contain merely those properties to be analyzed has an impartial influence in the categorization procedure. The purpose of this technique is to sort and categorise anticipation jobs. This technique rather similar with determination tree technique, but the distinguishable difference is at the procedure degree. Extra preparation is needed in order to picture this technique. In bunch, a set of random which is normalized representatives is created chiefly. The similarity between closest representatives will be identified until fulfilling categorization is made. The advantages of this technique are easy to be understood ; where the theoretical account able to manage noisy, multidimensional informations sets. Even though the preparation takes long period clip, the solution is rapidly obtained during the execution stage.

Visual image

This method is good for mining big database. Visual image is the best attack to show complex mutualities among legion properties where the user can obtain coveted informations or cognition based on the analysis done. Some of the major tools including the colourss, practical world, sound and forms are available in this technique. Visual image can be used in both standalone and assorted combinations. For the illustration, the user can expose the consequences in two and three dimensional images, visualized multidimensional records, and aid in making determination trees by utilizing this technique.

Familial Algorithm

Familial algorithm is familial plan which is based on the biological mechanism of natural choice and endurance of the fittest. This technique is appropriate for job resolution that involve optimisation together with some estimable environment. High velocity computing machine is required in order to work out big and complex jobs in a rational sum of clip. This technique is compatible to happen solutions regarded on complex optimisation jobs, where it can bring forth optimum solutions to such jobs. Other than that, this method is good in turn uping good parts, so that the local optimizer has the ability to go on with more accurate local hunt. The advantage obtained by utilizing this technique is many possibilities can be recommended as the best solutions ; therefore familial algorithm is easy to be explained, plus have the capableness to manage multidimensional jobs.

Associative Memories ( AMs )

This technique is based on braces or larger groups, in which interrelated informations points are memorized or forgotten utilizing long term memory ( LTM ) web theoretical account. Data brace retrieval happened when the long term memory web produces web activities which ensuing in retrieved informations brace. Associative memories has the capableness to instantly memorise or bury, bring forth originative solutions and provides short and precise preparation times for the users.

Statisticss

Statisticss technique is used to measure consequences or the effects of informations excavation in order to divide the good from the bad. This technique plays as the constituent in informations choice, extracted cognition rating and besides in trying. The major functions of statistics are it has the capableness to observe interlopers, smooth the information when is needed and guess noise. By utilizing appraisal method, statistics can cover with the losing information. However, there are some issues originate sing to this technique. This technique merely concentrates on theoretical facets of technique and theoretical accounts.

Based on the account above, the techniques used in informations excavation have the difference and similarities among each other. By holding these tools, the organisations have the competences to pull out information from big sum of informations successfully. There is other myriad of informations excavation tools or techniques used such as fuzzed logic, regulation initiation methods, unreal nervous webs and other myriad of techniques.

Data Mining And Data Warehouse: Benefits

Today, many organisations have realized the important usage of both informations warehouse and information excavation as the tools for peculiar job solutions and besides in determination devising activities. Data warehouse and information excavation are largely used in the concern Fieldss. For illustration, informations warehouse assist the user in concern procedures such planning, analysis, and determination support activities by dividing the determination bed from operational bed. Because of informations warehouse acts as an incorporate database, it has the characteristics to combines informations from internal and external beginnings and bundles which are ready for usage in legion mold and determination support activities. Besides, informations warehouse is applied in bring forthing studies and besides help in rebuild multidimensional positions of the information which is intentionally for on-line querying and analysis. Other than that, informations warehouses can even help concerns to happen forms, information and study informations which was hidden or was non possibleby utilizing OLTP database systems.

By holding informations excavation tools, the users particularly from managerial degree and for those knowledge workers are able to beef up their cognition and set up the company. Data excavation is largely used by concentrating on a strong relationship among the mark consumers. Some factors such monetary value, merchandise placement, client satisfaction and the organisation ‘ profitableness can be determined by utilizing informations excavation tools.

Data warehouse has the ability to take incompatibilities before the information being lading where the user can merely understand informations and do usage the informations efficaciously. Furthermore, it can execute better with cumulative questions that stored in a critical volume of information. The feature of informations warehouse which is integrated and interrelated to each other makes easier for the users to manage, understand and besides in question that can traverse mention between different sections of company ‘s operations. For illustration, the information of finance can be compared with human resource informations even there are located in different databases and holding different constructions.

In order to heighten the privateness and security in an organisation, informations excavation is the good tools which can be used. In banking environment, informations excavation can be used to observe the forms of deceitful recognition card use. Furthermore, it can besides be used to place the loyal clients and the stocks trading regulations from historical market informations.

Decision

Data warehouse and information excavation have the ability to work out many concern or any scientific jobs successfully in order to accomplish concern ends. By the outgrowth of engineerings, the informations and information quickly developed. This will take to the jobs in inability to entree and recover the coveted information efficaciously. The important job faced by an organisation nowadays is relates on holding the right information at the right clip for determination devising intents. Apart from the jobs, this is proved that many organisations have usage and implemented both informations warehouse and information excavation in order to make the success solutions and to accomplish competitory advantages.

The Challenges And Issues In Data Mining

There are some challenges faced by an organisation towards the usage of informations excavation tools. The major challenge is the ability to manage different types of informations. As we know, databases comprise of complex informations type such hypertext and multimedia informations. Data excavation tools or techniques should hold the ability to execute efficaciously on legion classs of informations constructions. Besides that, informations excavation tools should mining information from different types of beginnings. The characteristic such flexibleness in mining information from heterogenous databases can be a strength for the information excavation in order to manage informations from different beginnings. Other than that, the challenges in protection of privateness and informations security is besides relates to data excavation tools. By holding the latest engineering, the informations can be merely viewed fro different positions.

The current issues besides arise by holding the information excavation into the organisation. The privateness issues become the major jobs. There are some companies violate privateness jurisprudence by selling the personal information of their clients. Because of this issue, some people are non interested in online shopping which because of person can entree their personal information and usage it in unethical manner. The abuse of information and inaccurate information besides become the issues in informations excavation. In concern environment, some of the information obtained via informations excavation can know apart against a certain group of people. Furthermore, informations excavation technique is non 100 % accurate, whereby errors can be occurred which can take serious effects within organisations.