1 Introduction

Over the last several decades, it is a great challenge for scientists to develop, manufacture, and deploy advanced materials as fast as possible. In June of 2011, the materials genome initiative (MGI) for global competitiveness was proposed by the National Science and Technology Council of America for the development of an infrastructure to shorten the materials development cycle. The most important and fundamental goal of MGI is to accelerate materials design through the use of computational capabilities, data management, and an integrated approach to materials science and engineering ^[1].

In principle, there are two strategies for materials design. One strategy is to start from the first principle, i.e., from quantum mechanics and statistical mechanics, to predict the properties of unknown materials. Although the first principle method has been widely used in materials design ^{[2, 3]}, up to now, it is still impossible to solve most of complicated problems in materials exploration work by using this strategy solely. The other strategy is to start from the semi-empirical way, i.e., from the known data of some materials to find semi-empirical rules, which can be used to predict the properties of unknown materials. In general, the second strategy is more practicable than the first one in materials design or new materials exploration area, since a variety of data mining methods can be utilized to construct statistical models for a lot of data sets available from scientific experiments ^{[4, 5, 6, 7, 8, 9]}.

In the research of materials design by using data mining methods, principal component analysis (PCA), partial least squares (PLS) and artificial neural networks (ANNs) are very helpful because of their relative good performance, speed, simplicity to construct statistical models ^{[10, 11]}. However, ANN may give rise to over-fitting problems ^[12] (i.e., may lead to good performance in fitting but poor performance in prediction) in treating finite, multivariate data set. At the meanwhile, nonlinear relations can only be modeled in limited way by using PCA or PLS algorithm ^[13].

In the semi-empirical method of materials design, the data of known materials is usually used as the training set. In most cases, the numbers of available known data in the training sets are rather limited, which means that the data processing tasks usually deals with the problem of small sample size, and hence cause serious over-fitting problem. As an effective way to overcome the problem of over-fitting, support vector machine (SVM) based on statistical learning theory (SLT) has been proposed by Vapnik ^[14]. SVM has been shown to perform well in various applications including drug design ^{[15, 16]}, materials design ^{[17, 18, 19]} and chemistry researches ^[20].

In new materials exploration work, there are two questions with general significance in need of answers: the first question is "what is the chemical composition of the substance having desirable properties?", and the second one is "what are the optimal conditions of preparation or production for this material at low cost?". Since both of these two questions involve with very complicated systems or processes, we have to solve these problems by using some semi-empirical methods. To answer the first question, the relationships between the microscopic structure of materials and their properties need to be addressed. This relationship is usually known as quantitative structure- property relationship (QSPR). To deal with the second question, mathematical models are usually set up for the optimization of the processes.

Based on the problems concerning materials design, the tasks of materials design can be classified into four different categories. The first type of task is to solve the "formability problems", i.e., to find some mathematical model or criterion for the stability of some unknown substances. The second type of task is the "property prediction", i.e., to make mathematical models for the structure-property relationships and use these models to predict the properties of new materials (or the inverse problem: to search the unknown new materials with some pre-assigned properties). The third type of task is to solve the "optimization problems", i.e., to find the conditions for optimizing some properties of certain materials. The last but not the least type of task is to solve the "problem of control", i.e., to find the mathematical model to control some index of materials within a desired range. Different data mining techniques should be adopted for these different purposes. In this paper, we demonstrate some examples of applying SVM methods including support vector classification (SVC) and support vector regression (SVR) as a relatively new tool to meet the different tasks of materials design in our lab. The advantage of using SVM for materials design is discussed based on the applications presented. 2 Methods of SVM

The foundations of SVM have been developed by Vapnik ^[14] and are gaining popularity due to many attractive features, and promising empirical performance. In this paper the term SVM will refer to both SVC and SVR methods, which can be used for solving qualitative and quantitative problems respectively ^{[21, 22, 23, 24, 25]}. 2.1 SVC

SVC has been recently proposed as a very effective method for solving classification problems, which can be restricted to consideration of the two-class problem without loss of generality ^{[14, 20]}. In this problem the goal is to separate the two classes by a classifier induced from available examples. It is expected that the classifier constructed has good performance on unseen examples, i.e., it generalizes well.

The geometrical interpretation of SVC is that it determines the optimal separating surface, i.e., a hyperplane, which is equidistant from two sets of data points. This hyperplane has many interesting statistical properties as discussed by Vapnik ^[14]. Consider the problem of separating the set of training vectors belonging to two separate classes, (y₁, x₁), (y₂, x₂) ..., (y_n, x_n); x ∈ R_m, y ∈ -1, +1; with a hyperplane

where w and b are the weight vector and bias, respectively.

If the training data are linearly separable, then there exists a pair of parameter set (w, b), for which we can write

where P is the set of positive sample, and N is the set of negative sample.

The decision rule is

The pair (w, b) can be rescaled without loss of generality min

The learning problem is hence reformulated as follows. Let us minimize ||w||² subject to the constraints of linear separability. This is equivalent to maximizing the distance, normal to the hyperplane, between the convex hulls of two classes and the optimisation becomes a quadratic programming (QP) problem

subject to y_i(w^Tx_i+b) > 1 i = 1, 2, ..., l. This problem has global optimum, and the Lagrangian is written as

where Λ ={λ₁, λ₂, ..., λ_l} are the Lagrange multipliers, one for each data point. Hence we can write

The Lagrange multipliers are only non-zero when y_i(w^Tx_i+b) = 1. Vectors fulfilling this requirement are called support vectors since they lie closest to the separating hyperplane. Then, the optimal separating hyperplane is given as follows

and the bias is given by

where x_r and x_s are any support vectors from each class satisfying the following equation

The hard classifier is then

In the case where a linear boundary is inappropriate, the SVC can map the input vector, x, into a high dimensional feature space, F. By choosing a non-linear mapping Φ, the SVC constructs an optimal separating hyperplane in this higher dimensional space. Among acceptable mappings are polynomials, radial basis functions and certain sigmoid functions. Then the optimisation problem becomes

In this case, the decision function in SVC is as follows

where the x_i is the set of support vectors and K(x, x_i) is called the kernel function. 2.2 SVR ^{[14, 20]}

In SVR, the basic idea is to map the data x into a higherdimensional feature space F via a nonlinear mapping Φ and then to do linear regression in this space. Therefore, regression approximation addresses the problem of estimating a function based on a given data set G = (x_i is input vector, and d_i is the desired value). SVR approximates the function in the following form

where {Φ(x_i)}_i^l = 1 is the set of mappings of input features, and {Φ(x_i)}_i^l = l and b are coefficients. They are estimated by minimizing the regularized risk function R(C):

where

and ε is a prescribed parameter.

In Eq. (17), is the so-called empirical error (risk), which is measured by ε-insensitive loss function Leed; yT, which indicates that it does not penalize errors below ε. The second term, , is used as a measurement of function flatness. C is a regularized constant determining the tradeoff between the training error and the model flatness. Introduction of slack variables ξ leads Eq. (17) to the following constrained function:

Thus, decision function Eq. (16) becomes the following form:

In Eq. (21), α_i α_i^* are the introduced Lagrange multipliers. They satisfy the equality: α_i · α_i^* = 0 α_i > 0, i = 1, 2, ..., l, and are obtained by maximizing the dual form of Eq. (19), which has the following form:

with the following constrains:

Based on the Karush-Kuhn-Tucker (KKT) conditions of quadratic programming, only a number of coefficients α_i · α_i^* will assume nonzero values, and the data points associated with them could be referred to as support vectors. In Eq. (21), K(x, x_i) is the kernel function. The value is equal to the inner product of two vectors x and x_i in the feature space Φ(x) . That is, K(x, x_i) = Φ(x) Φ(x_i). The elegance of using kernel function lied in the fact that one can deal with feature spaces of arbitrary dimensionality without having to compute the map Φ(x) explicitly. Any function that satisfies Mercer’s condition can be used as the kernel function. 2.3 Implementation of SVM

According to the Ref. ^[14], the SVM software package ChemSVM including SVC and SVR has been programmed in our lab. The free version of ChemSVM can be downloaded on the website of Laboratory of Computational Chemistry in Shanghai University (http://chemdata.shu. edu.cn:8080/MyLab/Lab/download.jsp). The validation of the software has also been performed in the applications of chemistry ^[20]. 3 Applications 3.1 SVC applied to the formability of perovskite or BaNiO₃ structure

The most exciting achievement of materials research is to find some new compound (or new phases) with specified structure and outstanding properties. In this work, the materials design problems of compounds with perovskitetype structures or BaNiO₃ structure will be discussed based on SVC model.

There are numerous complex oxides or halides with general formula ABX₃ (X = oxygen or halogen) having perovskite-type crystal structure and outstanding functional properties ^[26]. Since 1945, when the ferroelectric properties of barium titanate were discovered, a series of complex oxides and complex halides with perovskite-type structure have been found to be valuable functional materials. In recent years, searching new complex oxides and complex halides with perovskie-type structure has become an active research field of new materials exploration.

The crystal structure of compounds with ideal perovskite structure is illustrated in Fig. 1. It is the structure of a unit cell of SrTiO₃ crystal. In this structure, tetravalent Ti⁴⁺ cation is surrounded by 6 oxygen anions to form octahedral structure, and bivalent Sr²⁺ cation is surrounded by 12 oxygen anions to form cubo-octahedral structure.

Based on the understanding of such type of crystal structure, Goldschmidt proposed a famous crystal-chemical criterion of the formability or the stability of perovskite structure for ABX₃-type compounds:

where t is called tolerance factor. R_a, R_b are cationic radii of A, B ions respectively, and R_x is the ionic radius of anion of X. According to Goldschmidt, the cubic perovskite structure is stable only if the value of tolerance factor has an approximate range of 0.8 < t < 0.9, and the distorted perovskite structure can be stable in a somewhat larger range of tolerance factor. This criterion is widely used in the exploration work of new compounds with perovskitetype or perovskite-like type structure. Owing to the accumulation of the crystallographic data of compounds with perovskite structure, it is now widely recognized that the range of tolerance factor for the stability of perovskite structure should be 0.75 < t < 1.00.

Although Goldschmidt’s tolerance factor t is indeed very useful for the exploration of new compounds with perovskite structure, it is only a necessary condition but not the sufficient condition for the formation or the stability of perovskite structure ^[27]. Many systems having t in the range of 0.75-1.0 do not form perovskite-type compound.

For example, MnSiO₃ has t = 0.856, but it has CdGeO₃ structure; R_bMnCl₃ has t = 0.88, but it has hexagonal BaTiO₃ structure. NaI-MgI₂ system has t = 0.826, but it has no intermediate compound at all. Therefore it is desirable to investigate the complementary conditions for the formability of perovskite structure, in order to help the computerized materials design for new materials with perovskite structure. Atomic parameters and SVM technique can be used for this purpose.

Since BX₆ octahedra and AX₁₂ cubo-octahedra are the basic sub-structures of perovskite lattice, as shown in Fig. 1, it is easy to see that the stability of BX₆ octahedra and AX₁₂ cubo-octahedra are also necessary conditions for the stability of perovskite lattice. It is obvious that the condition 0.75 < t < 1.00 is not enough to assure the stability of the BX₆ octahedral and AX₁₂ cubo-octahedral structure. It is necessary to find the suitable criteria for the above-mentioned stability requirements.

Some ABX₃ type compounds, such as R_b NiCl₃, although having 0.75 < t < 1.00, do not form perovskitetype lattice but the crystal lattices with BaNiO₃ structure. The chief difference of BaNiO₃ structure (or hexagonal BaTiO₃ structure) from perovskite structure is that in Ba- NiO₃ structure (or hexagonal BaTiO₃ structure) the BX₆ octahdra are shared by their face with each other (see Fig. 2), while in perovskite structure they are shared with each other by their corners.

In order to find the criterion of relative stability between BaNiO₃ structure and perovskite structure, the data set containing 23 samples is used for data mining ^[28]. By using SVC combined with atomic parameters of compounds, the mathematical model of SVC was found to differentiate between perovskite structure and the hexagonal ABX₃ structures involving face-shared octahedral. The SVM model with linear kernel function manifests that 100 % of separation of the chlorides with perovskite structure and the chlorides with face-shared structures can be achieved. In this work, the leaving-one-out cross validation (LOOCV) method was undertaken to evaluate the performances of the models obtained. As such, the data set of n samples was divided into two disjoint subsets including a training data set (n-1 samples) and a test data set (only 1 sample). After developing each model based on the training set, the omitted data was predicted by the model developed. In LOOCV test, the rate of correctness of prediction is 91 %. The criterion for the formation of faceshared structure found by SVC can be expressed as follows

where R_a and R_b denote the Shanon-Preweit ionic radii of A and B in ABCl₃ respectively. X_a and X_b denote the Basanov electronegativity of A and B respectively. N_d denotes the number of d electrons in the unfilled shell of d electrons of B ions. It implies that large A⁺ cation, small B²⁺ cation and large electronegativity of B favor the formation of face-sharing of BX₆ octahedra. This fact can be explained as follows: In perovskite-type lattice the network of corner shared BX₆ octahedra form cages of A⁺ ions. The repulsive force due to the large A⁺ and small cage formed by small B²⁺ and X^- will make perovskite-type lattice unstable, while the BX₆ octahedra in face-shared structure form parallel chains, and A⁺ ions are located between these chains without strict confinement. Thus large A⁺ and small B²⁺ favor the face-shared structure. 3.2 SVR applied to the prediction of energy gaps of binary compounds

III-V and II-VI binary compounds are important semiconductors for microwave, optoelectron and infrared devices. The band gaps (E_g) are essential properties of these compounds. It would be helpful for materials scientists to estimate the E_g of a compound before synthesizing it. On the basis of known data set available, it is reasonable to predict the properties of unseen samples by using data mining methods. Since there are a lot of data mining methods available, one has to deal with the troublesome problem about model selection for a particular data set with finite number of samples and multiple features. It is very important to select a proper model with good generalization ability, i.e., low mean relative error for the properties of new compounds (unseen samples).

In this work, the data set consists of 25 compounds, including AlP, AlAs, AlSb, GaP, GaAs, GaSb, InP, InAs, InSb, ZnS, ZnSe, ZnTe, CdS, CdSe, CdTe, HgS, HgSe, HgTe, AlN, GaN, InN, PbO, PbS, PbSe, and PbTe ^{[29, 30]}. Based on the data set available, the SVR model for predicting E_g of A^IIIB^V and A^IIB^VI binary compounds was constructed by using atomic parameters as features including electronegativity, valence, radius, atomic mass and their functions. The data mining results indicated that the sum of proportion of atomic electrovalent and covalent radius ∑(z=r_cov) ^[31], mean atomic number, N atomic electrovalent Z_A and Z_B should be selected as parameters in the model of band gap,

In the present work, the LOOCV test was undertaken to find the suitable capacity parameter C, ε-insensitive loss function and kernel function for SVR model. In order to measure the generalization ability of SVR model, we defined the mean error function (MEF) U_m as Eq. (28)

where e_i is the experimental value of sample i, p_i the predicted value of sample i, n the number of the whole samples. e_max, e_min are the maximum and minimum experimental value of whole samples respectively. In general, the smaller the value of U_m obtained, the better generalization ability expected. It is found that the optimal SVR model with the least U_m is available when the kernel function is polynomial form of while ε = 0.07 and the regularized constant C = 70. By using above kernel function and parameters optimized, the trained SVR model for E_g of A^IIIB^V and A^IIB^VI binary compounds with original data is available as follows

where (α_i - α_i^*) Tis the Lagrange coefficient corresponding to support vector. Figure 3 illustrates the relationship of predicted E_g and experimental E_g of AIIIBV and AIIBVI binary compounds, with related coefficient (R) of 0.97.

In this work, the LOOCV method was also undertaken to evaluate the performances of the models obtained. Figure 4 is the plot of the predicted values employing LOOCV of SVR versus experimental values for E_g of binary compounds.

From Fig. 4, it can be concluded that the predicted results are in good agreement to experimental ones ^[25].

3.3 SVR applied to the prediction of sintered cold modulus of sialon-corundum castable

Although the discovery of new materials is very exciting in materials research, the most part of tasks of materials research everyday is to try to improve the preparation technology of known materials. The economic effect of such kind of improvement is very significant because these efforts eventually determine the cost and the quality of products or the competitive ability in international market. Here the example of using SVR in materials optimization of preparing Sialon Ceramic will be described.

Sialons are silicon aluminium oxynitride ceramic materials with a range of technically important application, from cutting tools to specialized refractories. Furthermore, they can have a wide range of compositions and occur in several different families of crystal structures, the properties of sialons can be tailored for specific purposes ^[32]. bsialon corundum find applications as high temperature, corrosion resistant, thermal shock resistant, high strength and toughness structural material ^[33]. The sintered cold modulus of sialon-corundum castable is an important property of sialon material, but the relationship between the property and process parameters is very complicated ^[34]. Hence it is necessary to find some computational methods to correlate the properties of sialon-corundum with their process parameters.

In this work, the data set consists of 20 samples from our experiments. The root mean square error (RMSE) V_m of LOOCV was adopted to estimate the quality of the model for predicting the sintered cold modulus of sialon-corundum castable. The V_m is defined as follows

where e_i is the experimental value of i sample, p_i the predicted value of i sample, n the number of the whole samples in LOOCV. Based on the data mining work, it was found that the linear kernel function with C = 7.0 and ε = 0.16 can be used to construct SVR model for the quantitative relationship of sintered cold modulus with their process parameters. Finally, the SVR model obtained can be presented as follows ^[19]

where S_pred means the experimental values of sintered cold modulus of sialon-corundum castable (strength, unit: MPa), while C_Water, C_SiO2, C_Al2O3 and C_Disperser are the content (mass %) of water, SiO₂ powder and ρ-Al₂O₃, dispersant and water respectively.

Figure 5 illustrates the experimental values versus predicted values of sialon-corundum cold modulus using LOOCV of SVR model with linear kernel (C = 7.0 and ε = 0.16).

According to the Eq. (31), in order to increase the Spred, the content of SiO₂ and dispersant should be increased, at the mean while the content of ρ-Al₂O₃ should be decreased, which is consistent with the mechanism as follows. Addition of SiO₂ improves the flowability, but it reduces the added water content of castable. Simultaneously, SiO₂ reacts with water and then forms net structure of siloxene, which accelerates the sintering of castable and improves its cold strength after sintering. However, ρ- Al₂O₃ which usually hydrates into Al(OH)₃ and AlOOH as bonder, cannot accelerate the sintering with water increasing, because mass agglomeration γ-Al₂O₃ exists in the commercial ρ-Al₂O₃, which holds lots of water resulting bad flowability. Therefore, the increase of ρ-Al₂O₃ restrains the sintering and reduces the sintered cold strength of castable. As for dispersant, it improves the flowability of castable by avoiding the flocculation structure of micelle and making water difficultly into this structure. 3.4 SVM applied to the optimization of electric resistances of VPTC semiconductors

VPTC materials are a kind of ceramic semiconductors for electronic uses. The task of the research work of VPTC materials is to search the optimal composition and the optimal preparation conditions for high value of ρ₀/ρ_min (the ratio of the electric resistance at zero degree centigrade to the minimum electric resistance) of these materials. There are five influencing factors including Yb₂O₃ content (W₁), excess TiO₂ (W₂), sintering temperature (T_c), sintering time (T_k), and relative cooling rate (V). By using linear kernel function (C = 10 and ε = 0.15), the trained SVR model for predicting ρ₀/ρ_min is available as follows

It is found that the relationship between the property of ρ₀/ρ_min and the five influencing factors is nearly linear one. Figure 6 shows the comparison between the experimental values and the predicted values of ρ₀/ρ_min by SVR in LOOCV test. By using the SVR model combined with pattern recognition techniques, some unseen samples can be designed with new compositions and technological conditions for the optimization of VPTC semiconductors. The experimental results prove that the property ρ₀/ρ_min of new sample designed by using data mining increases to 27, which is much higher than that of the best sample (ρ₀/ρ_min = 21) obtained before the optimization.

3.5 SVM applied to the thickness control of In₂O₃ semiconductor film preparation

In₂O₃ semiconductor nanometer film is a new material for combustible gas detector uses. It can be prepared by sol-gel method. How to control the thickness of the semiconductor film is one of the crucial problems in the preparation work. There are several factors influencing the thickness of film: the mass percentage of In₂O₃ and PVA in the bath, the viscosity of coating liquids, the drawing rate and the drawing number in preparation. So it is desirable to have a mathematical model for the automatic control in the film production. In our lab, SVM methods have been used for data mining of this purpose.

It has been found that the SVR with polynomial kernel of second degree can make the mathematical model for the thickness control of the semiconductor films. Figure 7 shows the comparison between the experimental thickness data and the predicted thickness in LOOCV test ^[35].

4 Discussion and conclusions

Generally speaking, how to choose the right balance between model flexibility and over-fitting to a limited training set is one of the most difficult obstacles for obtaining a model with good generalization ability to predict properties of materials. In the computation ofSVMmodel, it should be noted that the selection of appropriate value for the regularization parameter C is very important because of its possible effects on both trained and predicted results, since it controls the tradeoff between maximizing the margin and minimizing the training error. Usually, C should be optimized for fear of neither under-fitting nor over-fitting. It is also noticed that the predicted results are largely affected by the kernel functions and its parameters adopted.

It should be emphasized that the advantage of SVM is workable with a small size of sample set. In many cases, obtaining a sufficient number of experimental samples is still time-consuming and costly in the development of novel materials. Therefore, efficient learning from a limited number of samples becomes increasingly important for shortening the materials development cycle.

Although our research results indicate that the performance of SVM outperforms those of traditional data mining methods, it should be realized that different data mining methods would have their own advantages and disadvantages in different applications. Sometimes the best approach is a combination of different methods since the complementary approaches can provide helpful information from different point of views.

From the examples introduced in this paper, it can be concluded that the SVM is an effective modeling tool with great potential in materials design. Therefore, it can be expected that the SVM method will be further applied in various fields of materials science.

Acknowledgments Financial supports to this work from the National Natural Science Foundation of China (Grant No. 21273145) and the 085 Project of Materials Genome Engineering of Shanghai University are gratefully acknowledged.