整合類神經網路與多元適應性云形回歸

1、類神 路料類例李行1.tw .tw理類神 路兩數類神路數類神 路良類神路識力來更精刺料行率參料類類神路AbstractThe objective of the proposed study is to explore the performance of data classification by integrating the artificial neural networks with the multiva

2、riate adaptive regression splines (MARS) approach. The rationale under the analyses is firstly to use MARS in modeling the classification problem, then the obtained significant variables are then used as the input variables of the designed neural network model. To demonstrate the inclusion of the ob

3、tained important variables from MARS would improve the classification accuracy of the networks, classification tasks are performed on one fine needle aspiration cytology (FNAC) breast cancer data sets. As the results reveal, the proposed integrated approach outperforms the results using discriminant

4、 analysis, artificial neural networks and multivariate adaptive regression splines and hence provides an efficient alternative in handling breast cancer diagnostic problems.Keywords: data mining, breast cancer, classification, neural networks, multivariate adaptive regression splines論料(datamining, D

5、M)來了量料療利料流率療料料量量若料料諸行識來療2行()8589 年惡（年臨女罹良 良惡(ultrasound)(mammography)行(needle aspiration)（惡（良(logistic regression, Neter 羅 (regression analysis)了羅analysis, Johnson and Wichern, 1998)et al., 1996)(autocorrelation)(discriminant數數(multi-collinearity)料料類來ANNs)類神數了 年來路論 類神路 (artificial neural networks;類神

6、不類(Lee et al., 1997)(Malhotra (supervised) 路類神力(Zha ng路et al.,神行Shavlik, 1997; Chung and Gray, 1999; Lee 了 羅 類神 (multivariate adaptive regression splines, MARS) MARS Friedman 念數(kn ots)數不易MARSet al. , 1998)數類神 路 類神 路 et al., 1992)1999; Salchenberger(unsupervised)理類路度類神數et al., 2002)路(Craven and料(los

7、s of fit, LOF) 來 度 料 連料(Friedman, 1991)理數 (Friedman, 1991)度 料 料(Friedman, 1991) 料4了羅類神路不類神路利MARS數行力MARS類神路MARS 類數類神路數類神 路良(better initialsolution)降類神路練行料569行異類力論論說論論類神路行論料類神路類神路類類論類神路年來類神路都類神路(patternclassification)(pattern recognition) 力 (Zhang et al., 1998)(supervised)(unsupervised)類神 路數量行理力力聯力濾(R

8、umelhart et al. ,1986)領例數率/利率(Fish etal., 1995; Berry and Linoff, 1997; Leeet al.1997; Zhang et al.,1998; Vellido et al., 1999)類神路聯(associate learning)類路路類神路Vellido et al. (1999)19921998 年領類神 路78%例類神路類神路度度力類神 路路率參數參 Anderson andRosenfeld (1988) Rumelhart et al. (1986) Lippmann (1987) Nelson and Illi

9、ngworth (1990) Stern (1996)(MARS)數 Friedman(1991)來料MARS念數料理料數(basis function, BF)數料參數(loss of fit, LOF)數(kn ots)數度料理立連數BF 數 利Craven and Wahba(1979)GCV(generalized cross validation)數數數數MARS數 不 數數數理量來度MARS 數累了(forward and backwardstepwise procedure)MARS數 (dummy variable)(missing value)數不MARSFriedman(1

10、991)來不量說MARS數數ANNs 力MARSANNs行MARS數 ANNs數立類神 路數不參MARS類神路() (Department of Surgery, Human Oncology andComputer Sciences, University of Wisconsin, Madison, USA)料357 良212 惡569行料數滑度度10數30 數良惡例398(70%) 練171(30%)留(不列)類來力參類劣MARS類神路行類神路Vesta Services Qnet (1998) MARSSalfordSystemsMARS 2.0(2001)行CPUPentium II

11、 733MHz IBM PC行(stepwise discriminate analysis)行 U數Wilks ' Lambda (Johnson and Wichern, 2002)數行30 滑度數10數11率95.91%69軸軸 0£ww0£(6860 |B；9 >|uuoh(686 0o>|U9qA9%99Z6 虛(% £96L9 (%69)£(蜃)乙(%£60兒(%Z0'66)90l，(囹兒(訕SdVIAl £%££93%巾眈乙% WZ乙%Z9Z'6t%右|/西%80t

12、9%t900Z:%000001，39NVldOdl/ll3AllV13d aiaVIVA(2-2%Z0'6699Z6SdVIAl馬 SdVIAl 導SdVIAl£w跚W0£wtt%Z0'66A99 (V AONV) WWW 肚 SdVIAl #WSdVIAl 乙#SdVIAl%L6G6 虛(%ooooO79 (%00 0)0(%9'9)Z(%9P£6)0(H(訕l(囹：%9P£6 I小%0000L 虛%000(H盍 (2-2數58596061 62行路1 神良惡路參數率0.010路率0.006 0.0080.010練練料RMSE0

13、.0001練 3,000練料 RMSE路路路30-60-130神60 神1神率0.010料 RMSE430-60-1類神路4率 98.25%率2-2率100.00%2類2類率100.00%1-1率97.20%類神路MARS率不數力不4類神 路1（良）2（惡）1（良）104(97.20%)3(2.80%)2（惡）0(0.00%) 64(100.00%)率 98.25%類神路神MARS8 神(見2)數8 神神數121314151617181920路1神良惡0 1數路參數率0.005路率0.0010.0030.005練練料RMSE0.0001練 4,000練料 RMSE路路路8-13-18 神13神

14、1 神率0.005料 RMSE58-13-15率98.25%率2-2率100.00%2 類2 類 率 100.00%1-1率97.20%51(良)2（惡）1(良)104(97.20%)3(2.80%)2(惡)0(0.00%) 64(100.00%)率 98.25%61-12-2率93.46%100.00%95.91%MARS99.07%95.31%97.66%類神 路97.20%100.00%98.25%97.20%100.00%98.25%7率率率6.54% 0.00%MARS0.93% 4.69%類神 路2.80% 0.00%2.80% 0.00%類神路8830數類神路數行參數360兩12

15、0不料易不論率率不類神路便8數數()類神路303608120了MARS類神理 6698.25 %率率）不料率(type II error 惡率力力理路了率類神路類神路數率了(misclassificati on cost)量率力率(type Ierror 良惡良率）率（7)類神率量路力類神 路MARS數 力9類神 路(classification and regression tree, CART) 力料來了量料療料料量量若料料諸行識來療料MARS類神路MARS類神路類更精MARS類神路兩MARS數類神路數類神 路良類神路識力更精數料了類神路率( 類神路數度)路率度惡良量精降降類神路不論率率(

16、fuzzy discriminant analysis) 類1. Anderson, J. A. and Rosenfeld, E.,Neurocomputing: Foundations of Research” , MITPress, Cambridge, MA, 1988.2. Berry, M. J. A. and Linoff, G., “ Data Mining Technique for Marketing, Sale, andCustomer Support ” , Wiley Computer, 1997.3. Box, G. E. P., Jenkins, G. M. an

17、d Reinsel, G . C., “Time Series Analysis-Forecasting and Control (3rd edition)” , Holden-Day, San Francisco, CA, 1994.4. Chung, H. M. and Gray, P., Guest Editors,“ Special Section: Data Mining” , Journal ofManagement Information Systems, Vol.16, 1999, pp. 11-16.5. Craven, M. W. and Shavlik, J. W.,“

18、Using Neural Networks for Data Mining” , FutureGeneration Computer Systems, Vol. 13, 1997, pp. 221-229.6. Craven, P. and Wahba, G., “ Smoothing Noisy Data with Spline Functions. Estimating the Correct Degree of Smoothing by the Method of Generalized Cross-Validation” ,Numberische Mathematik, Vol. 31

19、, 1979, pp. 317-403.7. Cybenko, G., “ Approximation by Superpositions of a Sigmoidal Function” , MathematicalControl Signal Systems, Vol. 2, 1989, pp. 303-314.8. Desai, V. S., Crook, J. N., and Overstreet, Jr. G . A., “ A Comparison of Neural Networks and Linear Scoring Models in the Credit Union En

20、vironment” , European Journal ofOperational Research, 1996, Vol. 95, pp. 24-37.9. Dillon, W. R. and Goldstein, M., Multivariate Analysis Methods and Applications , Wiley, New York, 1984.10. Fish, K. E., Barnes, J. H. and Aiken, M. W.,“ Artificial Neural Networks: A NewMethodology for Industrial Mark

21、et Segmentation”, Industrial Marketing Management,Vol. 24, 1995, pp. 431-438.11. Friedman, J. H., Multivariate adaptive regression splines, The Annals of Statistics, V ol. 19(1), pp.1-141, 1991.12. Hornik, K., Stinchcombe, M. and White, H.,“ Multilayer Feedforward Networks areUniversal Approximation

22、s”, Neural Networokls.,2V, 1989, pp. 336-359.13. Johnson, R. A. and Wichern, D. W., Applied Multivariate Statistical Analysis (Fourth Edition) , Prentice-Hall, Upper Saddle River, NJ, 1998.14. Lee, H., Jo, H. and Han, I., “ Bankruptcy Prediction Using Case-Based Reasoning, Neural Networks, and Discr

23、iminant Analysis”, Expert Systems with Applicatoiol.n1s3, ,V1997,pp. 97-108.15. Lee, T. S., Chiu, C. C., Lu, C. J., and Chen, I. F., “ Credit Scoring Using the Hybrid Neural Discriminant Technique ”, Expert Systems with Applications, Vol. 23 (3), 2002, pp. 245-254.16. Lippmann, R. P.,“A Introduction

24、 to Computing with Neural Nets”, IEEE ASSP Magazine,April 1987, pp. 4-22.17. Malhotra, M. K., Sharma, S. and Nair, S. S., “ Decision Making Using Multiple Models ” European Journal of Operational Research, Vol. 114, 1999, pp. 1-14.18. MARS V2.0 for windows 95/98/NT, Salford Systems, San Diego, CA, 2

25、001.19. Nelson, M. M. and W. T. Illingworth, A Practical Guide to Neural Nets , Addison-Wesley, Reading, MA, 1990.20. Neter, J., Kutner, M. H., Nachtsheim, C. J. and Wasserman, W., Applied Linear Statistical Models , IRWIN, Chicago, IL, 1996.21. Qnet 97 Neural Network Modeling for Windows 95/98/NT , Vesta Services, Winnetka, IL, 1998.22. Rumelhart, D. E., Hinton, D. E. and Williams, R. J.,“Learning Internal Representationsby Error Propagation in Parallel Distributed Processing”, MIT Press, Cambridge, MA,1986, pp.318-362.23. Salchenberger, L