This section we have design four different experiments procedure and running benchmark on two real SSDs. The SSDs is mentioned as before. We choose two benchmark for our experiment: Iometer and Postmark. Iometer is an I/O subsystem measurement and characterization tool for systems. It was developed by Intel Corporation. The postmark is a benchmark to measure performance about the class of application like electronic mail. it means that the postmark will frequently access large amounts of small files. The postmark execute procedure is sperate in three phases. First phase it will create file pool, we can setting the file numbers and file size of file pool. Second phase is transitions, the transactions will perform create, write, read, and delete operation for file pool, we can set the transaction times and ratio of create/delete, write/delete. The last phase is delete, it will delete all the files that the tool created on disk.
Experiment1
Figure 4.1: This figure shows the Experiment 1 result running iometer benchmark, figure 4.1a shows iometer on Windows, figure 4.1b shows iometer on Linux
The first experiment is to know how garbage collection benefited from trimming. To achieved the purpose, the experimental procedure has four step, first step is to fill up 90%
disk size, and second step is delete all the files that step 1 created on disk. The step 1 and step 2 is order to produce large amounts of TRIM commands in our experiment, and we want to compare trim-enable and trim-disable performance and analyze it. The third step is to wait an idle time. This step is order to give the system time to handle TRIM
0
Plextor-TRIM enable Plextor-TRIM disable Intel - TRIM enable Intel - TRIM disable
(a) Windows
Plextor-TRIM enable Plextor-TRIM disable Intel - TRIM enable Intel - TRIM disable
(b) Linux
Figure 4.2: This figure shows the Experiment 1 result running postmark benchmark, figure 4.2a shows postmark on Windows, figure 4.2b shows postmark on Linux
commands, the idle time is variable, which can be 0 minutes, 3 minutes, or 5 minutes. The fourth step is running benchmark on the real SSD, we use two benchmark: iometer and postmark. When using postmark, we skipped file deletion in transactions phase to avoid further trimming. We expect that the performance of trim-enable will better because it have less garbage collection cost.
The result of Running iometer is shown in Figure 4.1. We can observed that the intel SSD seems no different with trim-enable or trim-disable, no matter the environment is Windows or Linux. And in using plextor SSD the trim-enable performance is less than trim-disable when idle time is 0 because the TRIM overhead effect the performance, system will spent lots of time to handle large amounts of TRIM commands, after handling TRIM commands the performance shows better because the benefit of TRIM that trim-enable system has less garbage collection cost. The effect shows that when idle time is 3 minutes and 5 minutes, trim-enable performance is better than trim-disable performance. This phenomenon shows in both operation system Windows and Linux. The result of Running postmark is shown in Figure 4.2. the intel results are neither stable nor conclusive. In plextor SSD when idle time is 0 the trim-enable performance is less than trim-disable, after handling TRIM the performance is better except in Linux idle 5 minutes, the reason is same as before.
But the performance shows not obviously and the performance is drop in Linux idle 5 minutes. The reason why with trim-enable on Linux with idle period is 5 will drop can explained when we observe postmark every file operation handles files, see Figure 4.3. It shows how many files process per file operation per second. The delete file operation shows
that the trim-enable handle less files than trim-disable. So the postmark write performance is not obviously because it is effect by delete operation. If the performance is expect delete operation than the performance will obviously like running iometer’s result that the trim-enable system when idle period is 0 minutes the performance will less than trim-disable, and the trim-enable performance is better when idle period is 3 minutes and 5 minutes.
0
Figure 4.3: This figure shows every operations performs files per second on Linux with postmark in every idle period, figure4.3a shows result in idle period 0, figure4.3b shows result in idle period 3. figure4.3c shows result in idle period 5.
4.3.1 Experiment2
The second experiment is to know how garbage collection effect under high flash space utilization. To achieved the purpose, the experiment procedure is that, first step is to fill up 90% disk size, this step is to make high utilization disk. Second step is running benchmark to measure the write performance. We compare the performance with trim-enable and trim-disable and analyze it. In our procedure, it does not produce TRIM because there are no delete operation, so we set create/delete file in postmark phase 2
transactions, and the ratio of create to delete is 9:1. We expect that the trim overhead will drop the performnce. The iometer dose not performs delete itself and there are no delete operation in out procedure, so we expect when running iometer it will make no different between trim-enable and trim-disable.
Figure 4.4: This figure shows the Experiment 2 result running iometer benchmark, figure(a) shows result with plextor SSD, figure(b) shows result with pintel SSD
0
Figure 4.5: This figure shows the Experiment 2 result running postmark benchmark, fig-ure(a) shows result with plextor SSD, figure(b) shows result with pintel SSD
Running Iometer result is shown in Figure 4.4, the little difference is because we use full random write data, it will cause the different result in every run. The iometer result shows no different no mater the environment is Windows or Linux. The postmark result is shown in Figure 4.5. as expect that the performance of trim-enable is less than trim-disable, it because we trim overhead is affect the write performance. When use intel SSD in Linux the trim-enable performance is much less than the trim-disable, it shows that the TRIM overhead is bigger when using intel on Linux.
4.3.2 Experiment3
The result is shown in Figure 4.6. In figure 4.6c and 4.6d, intel SSD seems no different between trim-enable and trim-disable. The result is same as experiment 1 with intel SSD.
In figure 4.6a and 4.6b, the result of plextor SSD is as expect. When the request size is 4 KB, the performance with trim-enable is much better than the trim-disable. And when request size becomes bigger the performance is becomes closer with trim-enable and trim-disable. The reason is as before said, the bigger the request size the less the garbage collection cost, and TRIM does not have much benefit with the large request size.
0
Figure 4.6: This shows the result of Exepriment3, figure(a) use plextor SSD on Windows, figure(b) use plextor SSD on Linux, figure(c) use intel SSD on Windows,figure(d) use intel SSD on Linux
The result is shown in Figure 4.6. In figure 4.6c and 4.6d, intel SSD seems no different between trim-enable and trim-disable. The result is same as experiment 1 with intel SSD.
In figure 4.6a and 4.6b, the result of plextor SSD is as expect. When the request size is 4 KB, the performance with trim-enable is much better than the trim-disable. And when request size becomes bigger the performance is becomes closer with trim-enable and
trim-disable. The reason is as before said, the bigger the request size the less the garbage collection cost, and TRIM does not have much benefit with the large request size.
4.3.3 Experiment4
The fourth experiment is to know Different garbage collection pressures with a high flash space utilization. To achieved the purpose, the experiment procedure is that, first step is to fill up 90% disk size, as experiment 2 this step is to make high utilization disk. Second step is running postmark to measure the write performance. The reason why choose postmark is because the experiment procedure step 1 does not perform delete operation. So we have to use benchmark who has delete operation itself to produce TRIM commands to compare.
We set create and delete file operation in postmark phase 2 transactions, and the ratio of create to delete is 9:1. The variable of file size is selected among 4KB, 64KB, 1024KB. We expect that the trim-enable performance is much less than trim-disable when request size is 4 KB because the garbage collection cost is higher and now plus the overhead of TRIM commands.
The result is shown in Figure 4.7. In plextor SSD, when the request size select 4 KB the performance shows no different between trim-enable and trim-disable, it because the garbage collection is heavy in high disk utilization. The performance is worse that TRIM benefit can not help much, so the performance can not see obviously different. The reason is same in intel SSD on Windows when the request size is 4KB. The intel on Linux the TRIM overhead is too much that trim -enable performance is much less than trim-disable.
Expect the intel on Linux, we compare the performance when request size is 64 KB and 1024 KB, the write performance of trim-enable with request size is 1024 KB is much close trim-disable than with request size is 64 KB, the reason is because that the bigger request size has less numbers of TRIM commands. It also means it has less TRIM overhead, so the performance has bigger difference in request size 64 than the 1024 KB.
(a) Plextor in Windows (b) Plextor in Linux
(c) Intel in Linux (d) Intel in Windows
Figure 4.7: This shows the result of Exepriment4, figure(a) use plextor SSD on Windows, figure(b) use plextor SSD on Linux, figure(c) use intel SSD on Windows,figure(d) use intel SSD on Linux