Guang Pu Xue Yu Guang Pu Fen Xi
September 2012
Partial least squares (PLS) has been widely used in spectral analysis and modeling, and it is computation-intensive and time-demanding when dealing with massive data To solve this problem effectively, a novel parallel PLS using MapReduce is proposed, which consists of two procedures, the parallelization of data standardizing and the parallelization of principal component computing. Using NIR spectral modeling as an example, experiments were conducted on a Hadoop cluster, which is a collection of ordinary computers. The experimental results demonstrate that the parallel PLS algorithm proposed can handle massive spectra, can significantly cut down the modeling time, and gains a basically linear speedup, and can be easily scaled up.
View Article and Find Full Text PDFNear infrared spectroscopy (NIRS) is a process analysis and monitoring tool with many advantages, while it needs to set up quantitative or discriminative calibration models in advance, and needs to adjust these models when the process conditions are varied, which makes it difficult for ordinary user to take its full advantage of it. To tackle this problem, this paper presented a novel, simple and model-free methodology for online process monitoring based on two reciprocal viewpoints of measuring the variability of spectroscopy-both the similarity and dissimilarity of process spectrum, i.e.
View Article and Find Full Text PDF