The overview of our approach in the prediction phase is visualized in figure 4.1 and will be explained in the following sections.
4.1 Data Preprocessing
The samples used in the experiment are provided by Everest Textile Co., Ltd, Taiwan. Each sample consists of a recipe with size ranging from one to four, re-flectance spectra, CIEL∗a∗b∗, and CIELCh values. The reflectance spectra and
Figure 4.1: The overview of the method proposed in the thesis, where L, a, b are CIEL∗a∗b∗ and ˆC is the vector of the recipe. CPM is a color prediction model and a soft-proofing model is used to evaluate how well the predicted recipe vectors are.
CIEL∗a∗b∗ values are measured and recorded by machines and software developed by Datacolor, and then entered manually into the CSV file.
There are 8518 real-world samples, 487 standard samples, and 3 white samples.
White samples are the data of the three fabrics that are not dyed by any colorant.
Standard samples are the ones that are obtained by dying three fabrics with each colorant at concentrations of 0.06, 0.5, 1.5, and 3. The real-world data are the samples that are dyed to match the color that customers ask for. The data is then cleaned by the following procedures. First, we drop any sample with values are outside reasonable ranges or not numbers. In terms of CIEL∗a∗b∗, values of L should be between 0 and 100, values of a or b should be between -128 to 128.
The values of reflectance spectra should be between 0 and 200% (fluorescent dyes).
The values of concentration should be between 0 and 4 mg/L. Second, we drop samples whose second derivative of reflectance spectra exceeding a certain threshold because it is usually typo making the spectra too sharp. Third, we calculate the CIELCh values from CIEL∗a∗b∗ values. For each sample, it will be dropped if the calculated CIELCh value is not the same as the given ones. Last, the fabric of each sample is categorized into one of the three basic fabric which are cotton-based fabric, polyester-based fabric, and nylon-based fabric.
After dirty samples are removed, 7604 samples are left. Three of them are the white samples, 450 of them are standard samples, and 7151 are real-world samples.
In real-world samples, 24 of the recipes have sizes of 1, 733 have sizes of 2, 6333 have sizes of 3, and 61 have sizes of 4.
To make recipe be able to be inputted to our model, vectorization of the recipes are needed. The way we vectorize recipes is straightforward. First, we make a 38-dimension vector corresponding to 38 candidate dyes and the value of each 38-dimension is the concentration of the dye in a recipe, which is between 0 to 4. Then we concatenate the vector with the one-hot vector of three fabrics, which becomes the 41-dimension recipe vector. As to the outputs, we use standarization to normalize
CIEL∗a∗b∗ as well as reflectance.
4.2 Training Phase
In the training phase, we train two color prediction models, one is the model that will later be used to find inverse values (denoted as CPM) and the other is used as a soft-proofing model (denoted as SPM). Different structures and seeds are used to train the two models. We use the recipe vectors as input and the CIEL∗a∗b∗ or reflectance spectra as output, to train the models. The models are implemented with TensorFlow and MSE is the loss function. We use ten-fold validation on the real-world samples, and the training sets of each fold will be concatenated with standard and white samples. The standard and white samples are used as anchor points in the training set. We will discuss it later in section 5.1.
4.3 Prediction Phase
In the prediction phase, we are given a target fabric with color that we want to reproduce. In our experiment, we assume that the dye combination of the dye recipe is known and we just try to find concentrations of them. In practice, dye combinations can be easily picked by the colorists, and decoupling the picking of dye combinations and concentration can provide more flexibility so that the colorist can try different dye combinations to get different recipes for a color. On the other hand, we can also pick the dye combination that produces the closest color in the real-world samples.
After a dye combination is picked, we can then find the inverse value, that is, the recipe, of the CPM with CIEL∗a∗b∗ of the target fabric. We will discuss the ways we adopted to find inverse values in the next section. The predicted recipe will be inputted to the SPM to get a set predicted CIEL∗a∗b∗ values or reflectance spectra. There two metrics that we can evaluate the method. The first one is the
similarity of a predicted recipe to the ground-truth recipe. We calculate the mean absolute error as well as the relative error of recipe vectors in the experiment. For a recipe, MAE and relative error are shown in equation 4.1 and 4.2 respectively. For the other metric, we calculate the CIE ∆Eab∗ between the predicted values generated by SPM and the ones of the target fabric.
M AE = sum(|Cgt− ˆC|)
size of the recipe (4.1)
Relatvie Error = sum(|Cgt− ˆC|)
sum(Cgt) (4.2)
4.4 Finding Inverse Value
Two ways can be used to find the inverse value of CPM, grid search and gradient descent. Grid search is to enter as many different concentrations to the CPM as possible to get the corresponding CIEL∗a∗b∗ output, and the concentration that yields the smallest error to the CIEL∗a∗b∗ of the target fabric is the recipe we want. We use stride equals 0.01 for the recipes whose size is 1 or 2 and for those with size greater than 2, the stride must be increased otherwise it will take minutes even hours in run time.
Gradient descent is the other way to find the inverse value of a neural network.
This technique is leveraged in many works. For example, in the so-called ”deep dream” project [21] developed by Mordvintsev, Olah, and Tyka, the input image is updated by gradient of the loss between the output value and the one hot vector chosen. And in asm2vec [29], the predicted vector of an assembly function is also trained in run time.
The way we use gradient descent to find the inverse values is visualized in figure 4.2. First, we initialize recipe vectors with random values at the dimensions that correspond to the chosen dye combination, 1 to the dimension corresponds to the
Figure 4.2: Visualization of finding an inverse value using gradient descent. Cji is the ith recipe vectors at epoch j, (L∗, a∗, b∗)ij is the ith CIEL∗a∗b∗ value at epoch j, (L∗, a∗, b∗) is the CIEL∗a∗b∗ value of the target fabric, and µ is the learning rate.
When initializing and updating Cji, only the values of the dimensions involved will be changed and the rest remain 0.
fabric and 0 at the rest of the dimensions. Then we input the initialized vectors to the CPM and calculate the gradient of losses, which are the MSE between predicted result and the fabric, to update the input recipe vectors instead of the weights. Each time the recipe vectors are updated, we only update the values at the dimensions with the chosen dye combination as well as the fabric and keep the rest to zero. The process is repeated for certain epochs or until the losses are small enough. After that, we take the mean of the recipe vectors or the one that gives the minimum loss to the CIEL∗a∗b∗/reflectance spectra of the fabric. The former one is denoted as
”G.D. mean” and the latter one is denoted as ”G.D. min loss” in the result.