On Sensitivity of Bayes Factors for Categorical Data with Emphasize on Sparse Multinomial Models

H. C. Liu, W. S. Chen, B. C. Shia, C. C. Lee, S. L. Ou, Y. C. Ou, C. H. Su 5

High Order Lambda Measure Based Choquet Integral Composition Forecasting Model

Hsiang-Chuan Liu1, Wei-Sung Chen2, Ben-Chang Shia3, Chia-Chen Lee4,

Shang-Ling Ou5, Yih-Chang Ou6, Chih-Hsiung Su7

1Department of Biomedical Informatics, Asia University

2Department of Computer Science and Information Engineering, Asia University

3Department of Statistics and Information Science, Fu Jen Catholic University

4Department of Statistics, Xiamen University

5Department of Agronomy, National Chung Hsing University

6Department of Finance, Ling Tung University, Taichung

7 Department of Accounting Informatiot, ChihLee Institute of Technology

Abstract: In this paper, a novel fuzzy measure, high order lambda measure, was proposed, based on the Choquet integral with respect to this new measure, a novel composition forecasting model which composed the GM(1,1) forecasting model, the time series model and the exponential smoothing model was also proposed. For evaluating the efficiency of this improved composition forecasting model, an experiment with a real data by using the 5 fold cross validation mean square error was conducted. The performances of Choquet integral composition forecasting model with the P-measure, Lambda-measure, L-measure and high order lambda measure, respectively, a ridge regression composition forecasting model and a multiple linear regression composition forecasting model and the traditional linear weighted composition forecasting model were compared. The experimental results showed that the Choquet integral composition forecasting model with respect to the high order lambda measure has the best performance.

Key words: Composition forecasting model, fuzzy measure, Choquet integral, extensional lambda measure, high order extensional lambda measure

1.  Introduction and motivation

The well-known composition functions of forecasting model are linear, in our previous works, the non-linear composition forecasting model were considered by using Choquet integral with respect to some fuzzy measures 4. For any given fuzzy density, both Zadeh’s P-measure and Sugeno’s λ-measure are univalent fuzzy measures with only one formulaic fuzzy measure solution, a multivalent fuzzy measure with infinite formulaic fuzzy ……

Fuzzy measures

Definition 1. Fuzzy measure, A measure on a finite set is called a fuzzy measure, denoted - measure, if its measure function satisfies following conditions:

(i) (ii) (1)

2.  Experiment and Results

A real data was obtained from the paper of Zhang, Wang and Gao. For evaluating the proposed new density based composition forecasting model, an experiment by using the 5 fold cross validation mean square error was conducted. The performances of Choquet integral composition forecasting model with high order λ-measure, L-measure, λ-measure and P-measure, respectively, a ridge regression composition forecasting model and a multiple linear regression composition forecasting model and the traditional linear weighted composition forecasting model were compared. The result is listed in Table 1, which shows that the composition forecasting model based on Choquet integral with respect to -measure and O-density has the best performance

Table 1: MSEs of 7 Composition forecasting models

Composition forecasting models / 5-fold CV MSE
Choquet Integral
Composition forecasting model
with O-density / –measure / 43012.03
L –measure / 43319.99
λ-measure / 44504.19
P –measure / 43727.90
Ridge forecasting model / 48109.21
Multiple forecasting model / 57095.67

Figure 1: CRISP-DM

3.  Summary

In this paper, based on high order λ-measure and O-measure, a novel Choquet integral composition forecasting model is proposed. The experiment with a real data by using the 5 fold cross validation mean square error shows that the new model is better than others.

References

[1] Akbani, R., Kwek, S. and Japkowicz, N. (2004). Applying support vector machines to imbalanced datasets, in Proceedings of the 15th European Conference on Machine Learning, 39-50.

[2] Batuwita, R. and Palade, V. (2010). Efficient resampling methods for training support vector machines with imbalanced data sets, in Proceedings of the International Joint Conference on Neural Networks. 1 - 8.

[3] Batuwita, R. and Palade, V. (2013). Class Imbalance Learning Methods for Support Vector Machines, in Imbalanced Learning: Foundations, Algorithms, and Applications (eds H. He and Y. Ma), John Wiley & Sons, Inc., Hoboken, NJ, Ch5.

[4] Burges, C. J. C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2, no. 2, 121–167.

[5] Chang, C.-C. and Lin, C.-J. (2011). \LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:1 - 27:27.

[6] Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002). Smote: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321-357.

[7] Chawla, N.V., Japkowicz, N., Kolcz, A. (2004). Editorial: special issue on learning from imbalanced data sets. SIGKDD Explorations Special Issue on Learning from Imbalanced Datasets 6 (1):1-6.

[8] Choi, J. M. (2010). A Selective Sampling Method for Imbalanced Data Learning on Support Vector Machines. Graduate Theses and Dissertations. Paper 11529.

[9] Cortes, C. (1995). Prediction of Generalisation Ability in Learning Machines. PhD thesis, Department of Computer Science, University of Rochester.

[10] Cortes, C. and Vapnik, V. (1995). Support-vector networks. Machine Learning 20, no. 3, 273-297.

[11] Cristianinio N. and Shawe-Taylor J. (2000). An introduction to support Vector Machines: and other kernel-based learning methods. Cambridge University Press.

[12] Hart, P. (1968). The condensed nearest neighbor rule. IEEE Transactions on Information Theory, IT-14, 515-516.

[13] Kubat M., and Matwin, S. (1997). Addressing the curse of imbalanced training sets: one-sided selection. In Proceedings of the 14th International Conference on Machine Learning (ICML1997), 179–186.

[14] Menardi, G. and Torelli, N. (2013). Training and assessing classification rules with imbalanced data. Data Mining and Knowledge Discovery, DOI:10.1007/s10618-012-0295-5, to appear.

[15] Osuna, E., Freund, R., Girosi, F. (1997). Training support vector machines: An application to face detection. In Proceedings Computer Vision and Pattern Recognition ’97, 130-136.

[16] Parpoula, C., Drosou K., Koukouvinos, C. (2013). Large-Scale Statistical Modelling via Machine Learning Classifiers, Journal of Statistics Applications and Probability 2, No. 3, 203-222.

[17] Pepe, M. S., (2000). Receiver operating characteristic methodology, Journal of the American Statistical Association 95, 308-11.

[18] Scholkopf, B. and Smola, A. (2001). Learning with Kernels: Support Vector Machines, Regularization, Optimization, Beyond. Cambridge, MA, USA: MIT Press.

[19] Schmidt, M. (1996). Identifying speakers with support vector machines. In Proceedings of Interface ’96, Sydney.

[20] Swets J.A. and Pickett R.M. (1982). Evaluation of Diagnostic Systems: Methods form Signal Detection Theory. Academic Press, New York.

[21] Tong, S. and Chang, E. (2001). Support Vector Machine Active Learning for Image Retrieval. Proceedings of ACM International Conference on Multimedia, 107-118.

[22] Vapnik, V., (1995). The Nature of Statistical Learning Theory. Springer-Verlag New York, Inc.

[23] Veropoulos, K., Campbell, C., Cristianini, N. (1999). Controlling the sensitivity οf support vector machines. In Proceedings of the International Joint Conference on Artificial Intelligence, 55-60.

[24] Yuan, J. Li, J. and Zhang, B. (2006). Learning concepts from large scale imbalanced data sets using support cluster machines in Proceedings of the 14th annual ACM international conference on Multimedia, 441-450, ACM.

Received December 12, 2013; accepted July 26, 2014.

Hsiang-Chuan Liu

Department of Biomedical Informatics

Asia University

Taichung 41354, Taiwan, ROC.

Wei-Sung Chen

Department of Computer Science and Information Engineering

Asia University

Taichung, 41354, Taiwan.

Ben-Chang Shia

Department of Statistics and Information Science

Fu Jen Catholic University

New Taipei City, 24205, Taiwan.

Chia-Chen Lee

Department of Statistics

Xiamen University

China.

Shang-Ling Ou

Department of Agronomy

National Chung Hsing University

Taichung, 40227, Taiwan.

Yih-Chang Ou

Department of Finance

Ling Tung University

Taichung, 40852, Taiwan.

Chih-Hsiung Su

Department of Accounting Informatiot

ChihLee Institute of Technology

New Taipei City, 22050, Taiwan.