Modeling

VOC Classification

Overview

The VOC device is designed to tell whether the tobacco is heathy or infected. Since this is an inquiry experiment, algorithms in data analysis are widely used in our modeling. We did data preprocessing, data analysis, and algorithm optimization on the data collected by VOC device. Finally, we used Logistic regression and detected the infected tobacco with 91% confidence.

Data preprocessing

First we defragmented the raw input data, and reorganized them into a matrix. 10 VOC factors were served as features, and the status(heathy or infected) was served as a tag to be predicted.

Then we analyzed the data using box plot and discovered that most data were normal, but some records were singular, whose box plot is shown as follows:

We removed those records with singular value, it turned out that the data left obey the normal distribution:

Data analysis

Our target was to create a model to predicted tobacco's status according to 10 input features. This is a classic two classification problem, which we had several algrithm to solve. The sampling algorithm is cross validation and the scoring policy we applied is ridit test

Decision Tree

First we used decision tree, which is based on information theory. ID3 decision tree was used to reduce the most information gain, while CART tree was used to reduce the GINI index. The performance of these two algorithm is almost the same. R = 0.83

MLP

The second algorithm we applied is Multi-Layer Perception, also called neural network. In this model, we used more than 100 neurons in each layer and the activation function is relu.

The result of MLP is much better than decision tree.R = 0.89

Leaner Model

Although the performance of MLP had been good enough, it's difficult to extract konwledge learnt by algorithm, which means the interpretability is weak. Why not try a simple model with high interpretability? First we tried LDA algorithm to compress the 10 dimensions data into 2 dimensions.

$$J=\frac{||w^T\mu_0-w^T\mu_1||^2}{w^T\Sigma_0w+w^T\Sigma_1w}=\frac{w^T(\mu_0-\mu_1)(\mu_0-\mu_1)^Tw}{w^T\Sigma_0w+w^T\Sigma_1w}$$

We define $S_w$ as within-class scatter matrix

$$S_w=\Sigma_0+\Sigma_1=\sum_{x\in X_0}(x-\mu_0)(x-\mu_0)^T+\sum_{x\in X_1}(x-\mu_1)(x-\mu_1)^T$$

We define $S_b$ as between-class scatter matrix

$$S_b=(\mu_0-\mu_1)(\mu_0-\mu_1)^T$$

$$J=\frac{w^TS_bw}{w^TS_ww}$$ $J$

The result of LDA algorithm is as follows: $R=0.89$ :

This result proved the data are linear separable, which enabled us to chose logistics regression algorithm.

We difine $logitP=ln\frac{y}{1-y}\in (-\infty,+\infty)$

$$p(y=1|x)=\frac{e^{w^Tx+b}}{1+e^{w^Tx+b}}$$

$$p(y=1|x)=\frac{1}{1+e^{w^Tx+b}}$$

$$l(w,b)=\sum_{i=1}^{m}lnp(y_i|x_i;w,b)$$

Then we can apply maximum likelihood method algorithm to estimate the paramaters.

The result is as follows:

                            Weight:
                            [[ 0.1819504 0.38788225 0.01350023 0.39594948 0.17799418
                            0.42087034
                            -0.57733395 -0.23876003 -0.00532918 -0.46174515]]
                            Intercept:
                            [ 0.00937812]
                            Effect:
                            D    35.300735
                            B    22.596339
                            F    18.289277
                            E    10.265025
                            C     0.393225
                            I    -1.575564
                            A   -10.679026
                            H   -14.398440
                            G   -26.211964
                            J   -39.130542
                            dtype: float64
                            Score:
                            0.894333333333
                        

Algorithm optimization

From the result of logistics regression, factor C and I and etc. are with less important weight, these factors may disturb the classifaction. We tried to reduce insigfinicant factors to simplify the model.

Finally, we reserved 4 factors with which we can predict the tobacco's status with 91% confidence and also reduced the VOC device.

                    Weight:
                    [[ 0.53196697  0.3404023  -0.53555988 -0.45588715]]
                    Intercept:
                    [-0.01204088]
                    Effect:
                    D    33.217011
                    F    15.492680
                    G   -17.319760
                    J   -33.967849
                    dtype: float64
                    Score:
                    0.912444444444
                

Summary

In this model, we tried different algorithm to abttain a robust, interpretable, and accurate solution to predict whether the tobacco is infected only according to 4 features in 91% confidence. Since there are 6 VOC sensors left unused in this model, the device can also be simplified in the future by reducing them. We can also try to add more functions to this device by making use of the left sensors.