Sunum yükleniyor. Lütfen bekleyiniz

Sunum yükleniyor. Lütfen bekleyiniz

Classification Part 1 CSE 439 – Data Mining Assist. Prof. Dr. Derya BİRANT.

Benzer bir sunumlar


... konulu sunumlar: "Classification Part 1 CSE 439 – Data Mining Assist. Prof. Dr. Derya BİRANT."— Sunum transkripti:

1 Classification Part 1 CSE 439 – Data Mining Assist. Prof. Dr. Derya BİRANT

2 Outline ◘What Is Classification? ◘Classification Examples ◘Classification Methods –Decision Trees –Bayesian Classification –K-Nearest Neighbor –Neural Network –Genetic Algorithms –Support Vector Machines (SVM) –Fuzzy Set Approaches

3 What Is Classification? ◘Classification –Construction of a model to classify data –When constructing the model, use the training set and the class labels –After the construction of the model, use it in classifying new data

4 Classification (A Two-Step Process) 1.Model construction –Each tuple/sample is assumed to belong to a predefined class, as determined by the class label attribute –The set of tuples used for model construction is training set –The model is represented as classification rules, trees, or mathematical formulae 2.Model usage (Classifying future or unknown objects) –Estimate accuracy rate of the model Accuracy rate is the percentage of test set samples that are correctly classified by the model –If the accuracy is acceptable, use the model to classify data tuples whose class labels are not known

5 Classification (A Two-Step Process) Mining Model DM Engine Training Data Data To Predict DM Engine Predicted Data Mining Model

6 Classification Example Training Data Classification Algorithms IF rank = ‘professor’ OR years > 6 THEN tenured = ‘yes’ Classifier (Model ) Classifier Testing Data Unseen Data (Jeff, Professor, 4 ) Tenured? Process (2): Using the Model in Prediction Process (1): Model Construction

7 Classification Example ◘Given old data about customers and payments, predict new applicant’s loan eligibility. –Good Customers –Bad Customers Age Salary Profession Location Customer type Previous customers Classifier New applicant’s data Rules Salary > 5 L Prof. = Exec Good/ bad

8 Classification Techniques 1.Decision Trees 2.Bayesian Classification 3.K-Nearest Neighbor 4.Neural Network 5.Genetic Algorithms 6.Support Vector Machines (SVM) 7.Fuzzy Set Approaches

9 Classification Techniques Decision Trees Bayesian Classification K-Nearest Neighbor Neural Network Genetic Algorithms Support Vector Machines (SVM) Fuzzy Set Approaches … Classification

10 Decision Trees ◘Decision Tree is a tree where –internal nodes are simple decision rules on one or more attributes –leaf nodes are predicted class labels ◘Decision trees are used for deciding between several courses of action age? student?credit rating? <=30 >40 no yes no FairExcellent Yes NoNo Attribute Value Classification

11 Desicion Tree Applications ◘Decision trees are used extensively in data mining. ◘Has been applied to: –classify medical patients based on the disease, –equipment malfunction by cause, –loan applicant by likelihood of payment, –... Salary < 1 M Job = teacher Good Age < 30 Bad Good House Hiring

12 Decision Trees (Different Representation) Minivan Age Car Type YES NO YES <30>=30 Sports, Truck 03060Age YES NO Minivan Sports, Truck DT Splits Area ( Different representation of decision tree) short medium tall

13 Decision Tree Adv. DisAdv. Positives (+) +Reasonable training time +Fast application +Easy to interpret (can be re-represented as if-then- else rules) +Easy to implement +Can handle large number of features +Does not require any prior knowledge of data distribution Negatives (-) -Cannot handle complicated relationship between features -Simple decision boundaries -Problems with lots of missing data -Output attribute must be categorical -Limited to one output attribute

14 Rules Indicated by Decision Trees ◘Write a rule for each path in the decision tree from the root to a leaf.

15 Decision Tree Algorithms ◘ID3 –Quinlan (1981) –Tries to reduce expected number of comparison ◘C 4.5 –Quinlan (1993) –It is an extension of ID3 –Just starting to be used in data mining applications –Also used for rule induction ◘CART –Breiman, Friedman, Olshen, and Stone (1984) –Classification and Regression Trees ◘CHAID –Kass (1980) –Oldest decision tree algorithm –Well established in database marketing industry ◘QUEST –Loh and Shih (1997)

16 Decision Tree Construction ◘Which attribute is the best classifier? –Calculate the information gain G(S,A) for each attribute A. –The basic idea is that we select the attribute with the highest information gain.

17 Decision Tree Construction HavaSıcaklıkNemRüzgarTenis GüneşliSıcakYüksekHafifHayır GüneşliSıcakYüksekKuvvetliHayır BulutluSıcakYüksekHafifEvet YağmurluIlıkYüksekHafifEvet YağmurluSerinNormalHafifEvet YağmurluSerinNormalKuvvetliHayır BulutluSerinNormalKuvvetliEvet GüneşliIlıkYüksekHafifHayır GüneşliSerinNormalHafifEvet YağmurluIlıkNormalHafifEvet GüneşliIlıkNormalKuvvetliEvet BulutluIlıkYüksekKuvvetliEvet BulutluSıcakNormalHafifEvet YağmurluIlıkYüksekKuvvetliHayır Which attribute first?

18 Decision Tree Construction HavaSıcaklıkNemRüzgarTenis GüneşliSıcakYüksekHafifHayır GüneşliSıcakYüksekKuvvetliHayır BulutluSıcakYüksekHafifEvet YağmurluIlıkYüksekHafifEvet YağmurluSerinNormalHafifEvet YağmurluSerinNormalKuvvetliHayır BulutluSerinNormalKuvvetliEvet GüneşliIlıkYüksekHafifHayır GüneşliSerinNormalHafifEvet YağmurluIlıkNormalHafifEvet GüneşliIlıkNormalKuvvetliEvet BulutluIlıkYüksekKuvvetliEvet BulutluSıcakNormalHafifEvet YağmurluIlıkYüksekKuvvetliHayır Gain(S, Hava) = 0,246 Gain(S, Sıcaklık) = 0,029 Gain(S, Nem) = 0,151 Gain(S, Rüzgar) = 0,048

19 Decision Tree Construction ◘Which attribute is next? Hava Güneşli Bulutlu Yağmurlu ? Evet ?

20 Decision Tree Construction HavaSıcaklıkNemRüzgarTenis R1GüneşliSıcakYüksekHafifHayır R2GüneşliSıcakYüksekKuvvetliHayır R3BulutluSıcakYüksekHafifEvet R4YağmurluIlıkYüksekHafifEvet R5YağmurluSerinNormalHafifEvet R6YağmurluSerinNormalKuvvetliHayır R7BulutluSerinNormalKuvvetliEvet R8GüneşliIlıkYüksekHafifHayır R9GüneşliSerinNormalHafifEvet R10YağmurluIlıkNormalHafifEvet R11GüneşliIlıkNormalKuvvetliEvet R12BulutluIlıkYüksekKuvvetliEvet R13BulutluSıcakNormalHafifEvet R14YağmurluIlıkYüksekKuvvetliHayır Hava Güneşli Bulutlu Yağmurlu Nem YüksekNormal Hayır Evet Rüzgar Hafif Kuvvetli Evet Hayır [R3,R7,R12,R13] [R9,R11] [R4,R5,R10] [R1,R2, R8] [R6,R14]

21 Another Example At the weekend: - go shopping, - watch a movie, - play tennis or - just stay in. What you do depends on three things: - the weather (windy, rainy or sunny); - how much money you have (rich or poor) - whether your parents are visiting.

22 Another Example

23 Classification Techniques Decision Trees Bayesian Classification K-Nearest Neighbor Neural Network Genetic Algorithms Support Vector Machines (SVM) Fuzzy Set Approaches … Classification

24 Classification Techniques 2- Bayesian Classification ◘A statistical classifier: performs probabilistic prediction, i.e., predicts class membership probabilities. ◘Foundation: Based on Bayes’ Theorem. Given training data X, posteriori probability of a hypothesis H, P(H|X), follows the Bayes theorem

25 Classification Techniques 2- Bayesian Classification Class: C1:buys_computer = ‘yes’ C2:buys_computer = ‘no’ Data sample X = (Age <=30, Income = medium, Student = yes Credit_rating = Fair)

26 Classification Techniques 2- Bayesian Classification ◘X = (age <= 30, income = medium, student = yes, credit_rating = fair) ◘P(C 1 ): P(buys_computer = “yes”) = 9/14 = P(C 2 ): P(buys_computer = “no”) = 5/14= ◘Compute P(X|C i ) for each class P(age = “<=30” | buys_computer = “yes”) = 2/9 = P(age = “<= 30” | buys_computer = “no”) = 3/5 = 0.6 P(income = “medium” | buys_computer = “yes”) = 4/9 = P(income = “medium” | buys_computer = “no”) = 2/5 = 0.4 P(student = “yes” | buys_computer = “yes) = 6/9 = P(student = “yes” | buys_computer = “no”) = 1/5 = 0.2 P(credit_rating = “fair” | buys_computer = “yes”) = 6/9 = P(credit_rating = “fair” | buys_computer = “no”) = 2/5 = 0.4 ◘ P(X|C 1 ) : P(X|buys_computer = “yes”) = x x x = P(X|C 2 ) : P(X|buys_computer = “no”) = 0.6 x 0.4 x 0.2 x 0.4 = P(X|C i )*P(C i ) : P(X|buys_computer = “yes”) * P(buys_computer = “yes”) = P(X|buys_computer = “no”) * P(buys_computer = “no”) = Therefore, X belongs to class (“buys_computer = yes”)

27 Classification Techniques Decision Trees Bayesian Classification K-Nearest Neighbor Neural Network Genetic Algorithms Support Vector Machines (SVM) Fuzzy Set Approaches … Classification

28 K-Nearest Neighbor (k-NN) ◘An object is classified by a majority vote of its neighbors (k closest members). ◘If k = 1, then the object is simply assigned to the class of its nearest neighbor. ◘Euclidean Distance measure is used to calculate how close

29 K-Nearest Neighbor (k-NN)

30 Classification Evaluation (Testing) categorical continuous class Training Set Model Learn Classifier Test Set

31 Classification Accuracy True Positive True Negative False Positive False Negative Which classification model is better?

32 Validation Techniques ◘Simple Validation ◘Cross Validation ◘n-Fold Cross Validation Training set Test set Training set Test set Training set Test set ◘ Bootstrap Method


"Classification Part 1 CSE 439 – Data Mining Assist. Prof. Dr. Derya BİRANT." indir ppt

Benzer bir sunumlar


Google Reklamları