Supplementary Material
1 Sensitivity Analyses of Hyperparameters
0 200 400 600 800 1,000
0 10 20 30 40 50
λ = 0.000 λ = 0.125 λ = 0.250 λ = 0.375 λ = 0.500
Epoch
Accuracy(%)
(a) Hyperparameter λ
0 200 400 600 800 1,000
0 10 20 30 40 50
β = 0.0 β = 2.5 β = 5.0 β = 7.5 β = 10.0
Epoch
Accuracy(%)
(b) Hyperparameter β
Figure 1: The sensitivity analyses of hyperparametersλ and β in the case of 400 diseases.
We perform the sensitivity analyses of hyperparametersλ and β in the case of 400 diseases. First, we analyze the behavior of REFUEL with varyingλ ∈ {0, 0.125, 0.25, 0.375, 0.5}. Recall that we define our potential function as
ϕ(s) :=λ × |{j : sj= 1}| if s ∈ S \ {S⊥}
0 otherwise ,
whereλ controls the magnitude of reward shaping. As shown in Figure 1a, the agent with higher λ learns faster at the beginning, but the agent withλ = 0.25 (red line) reaches the highest accuracy at the end. Therefore, we chooseλ = 0.25 as our hyperparameter.
Next, we conduct the experiments with differentβ ∈ {0, 2.5, 5, 7.5, 10}. In our objective function J = Jpg(θ) − βJreb(x, z; θ), the hyperparameter β controls the importance of the feature rebuilding taskJreb. Figure 1b shows that higherβ yields better performance. Therefore, we select β = 10 as our hyperparameter.
32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montréal, Canada.
2 Comparison with Decision Tree
0 10 20 30 40 50
0 20 40 60
Decision Tree REFUEL (Avg Steps 8.01)
Average Steps
Accuracy(%)
(a) 200 diseases
0 10 20 30 40 50
0 10 20 30 40 50
Decision Tree REFUEL (Avg Steps 8.09)
Average Steps
Accuracy(%)
(b) 300 diseases
0 10 20 30 40 50
0 10 20 30 40
Decision Tree REFUEL (Avg Steps 8.35)
Average Steps
Accuracy(%)
(c) 400 diseases Figure 2: Experiments on3 datasets of different disease numbers.
We apply the CART decision tree algorithm to the disease diagnosis problem. In Figure 2, the red line represents the performance of REFUEL; the blue line is the result of the CART decision tree algorithm. Whereas REFUEL requires about8 symptoms from a patient, the CART decision tree algorithm requires about50 symptoms to reach the same performance as REFUEL, which is impractical to the disease diagnosis problem.
2
