Data Mining for Not Yet Techies

Data mining is a comprehensive term for a whole set of techniques and technologies to find and summarize the patterns, trends and anomalies to be found in the vast quantities of data in databases. The technical basis of data mining is know as machine learning.

Data mining analyzes data to discover relationships not immediately obvious. Its most common use is probably now in sales and marketing. Many companies have collected data for years but did not know how to properly use it. Now they are applying these techniques to learn where and who their customers are. This allows them to stop wasting money on ineffective marketing and concentrating their marketing budget on the media most likely to reach potential customers.

1. Decision trees
2. Rules
3. Linear

Data mining organizes infosets of patterns or expectations. It "is an extraction of implicit, previously unknown and potentially useful information from data."

It can solve problems by analyzing data already stored, by discovering patterns in the data.

We don't want to know that the country of Jordan gets 15 inches of rain a year, we need to know if that means it's a desert or a jungle.

Data mining uses learning algorithms. It is also helpful for the human beings designing data mining techniques and applications to know data about the data of the domain being studied - this is known as metadata. For instance, if you're analyzing the sales of grass seed, it's useful to know that most people do not plant grass seed during the winter.

That seems like common sense, which is necessary for extracting useful information. Also useful is the understanding of complex statistics to know how patterns can be extracted from random events. For instance, there are people who believe that there are patterns to roulette wheel spin results and if only they could figure out the exact system, they'd make a fortune. Unfortunately for them, unless the wheel is somehow rigged or needs repair, each spin produces an independent result that, combined with the spins before and after, does NOT produce a meaningful result. Do not bet real money on such gambling systems.

Text mining is the form of data mining that looks for patterns in text files.

Next: Data modeling