Description
I benchmarked various optimization flags for building the Ruby interpreter. The benchmark is
MatrixBenchmark.rb. Ruby is built from the Debian source package, building only the ruby interpreter
itself (libruby and ruby1.8), using the libruby-static library as building SO files using -fprofile-generate
is not allowed
[~]% ruby --version
ruby 1.8.4 (2005-12-24) [i486-linux]
[~]% apt-cache policy ruby1.8
ruby1.8:
Installed: 1.8.4-5
[~]% gcc --version
gcc (GCC) 4.1.2 20060814 (prerelease) (Debian 4.1.1-11)
The benchmark is MatrixBenchmark.rb, which inverses matrices using a full Ruby implementation. The matrix
size was 256×256 elements.
All benchmarking files can be found
here, or for darcs users
darcs get http://www.jhaampe.org/Darcs/ruby-gcc-opt
Build flags
- optimization flags: -Os, -O2, -O3 (comparison here)
- architecture flags: -march=i386, -march=athlon-xp, -mtune=i586 (comparison here)
- profiling-based compilation: using -fprofile-generate, running either the benchmark (profbench) or Ruby’s test
suite (proftest) and then rebuilding with -fprofile-use (comparison here)
Computer
The machine is a Debian sid, Athlon XP 2000+ with 512 MB, with linux kernel 2.6.17
Results
Overall results
| name |
user |
system |
cpu |
real |
diff (%) |
| O3-arch-athlon-xp-profbench |
114.99 |
11.62 |
126.61 |
131.68 |
-22.80 |
| O2-arch-athlon-xp-profbench |
115.69 |
11.26 |
126.95 |
129.43 |
-22.59 |
| O3-arch-athlon-xp-proftest |
117.12 |
11.72 |
128.84 |
128.90 |
-21.44 |
| O2-arch-athlon-xp-proftest |
118.65 |
11.27 |
129.92 |
129.93 |
-20.78 |
| O3-arch-athlon-xp-std |
124.06 |
11.11 |
135.17 |
135.18 |
-17.58 |
| Os-arch-athlon-xp-profbench |
125.81 |
11.23 |
137.04 |
137.06 |
-16.44 |
| Os-arch-athlon-xp-proftest |
127.72 |
11.67 |
139.39 |
139.41 |
-15.01 |
| O2-arch-athlon-xp-std |
129.27 |
11.64 |
140.91 |
140.93 |
-14.08 |
| O3-tune-i586-proftest |
136.63 |
10.63 |
147.26 |
147.25 |
-10.21 |
| O3-tune-i586-profbench |
136.90 |
10.53 |
147.43 |
147.45 |
-10.10 |
| O2-tune-i586-proftest |
137.81 |
10.74 |
148.55 |
154.50 |
-9.42 |
| O2-tune-i586-profbench |
138.25 |
11.02 |
149.27 |
155.24 |
-8.98 |
| O3-arch-i386-profbench |
142.23 |
10.53 |
152.76 |
152.77 |
-6.85 |
| O3-arch-i386-proftest |
142.45 |
10.90 |
153.35 |
153.37 |
-6.49 |
| Os-arch-athlon-xp-std |
143.58 |
10.99 |
154.57 |
154.57 |
-5.75 |
| O3-tune-i586-std |
145.05 |
10.90 |
155.95 |
155.96 |
-4.91 |
| O2-arch-i386-profbench |
145.03 |
11.35 |
156.38 |
156.39 |
-4.65 |
| O2-arch-i386-proftest |
145.59 |
10.99 |
156.58 |
156.61 |
-4.52 |
| O2-tune-i586-std |
146.18 |
11.97 |
158.15 |
164.49 |
-3.57 |
| Os-tune-i586-proftest |
148.20 |
10.90 |
159.10 |
159.11 |
-2.99 |
| O3-arch-i386-std |
148.08 |
11.26 |
159.34 |
159.36 |
-2.84 |
| Os-tune-i586-profbench |
148.69 |
11.28 |
159.97 |
159.98 |
-2.46 |
| Os-arch-i386-profbench |
151.59 |
10.80 |
162.39 |
162.40 |
-0.98 |
| O2-arch-i386-std |
152.76 |
11.24 |
164.00 |
166.88 |
0.00 |
| Os-arch-i386-proftest |
152.89 |
12.11 |
165.00 |
165.46 |
0.61 |
| Os-tune-i586-std |
154.91 |
11.01 |
165.92 |
165.95 |
1.17 |
| Os-arch-i386-std |
160.78 |
11.53 |
172.31 |
172.42 |
5.07 |
Effect of -Ox
| name |
user |
system |
cpu |
real |
diff (%) |
| O3-arch-athlon-xp-proftest |
117.12 |
11.72 |
128.84 |
128.90 |
-0.83 |
| O2-arch-athlon-xp-proftest |
118.65 |
11.27 |
129.92 |
129.93 |
0.00 |
| Os-arch-athlon-xp-proftest |
127.72 |
11.67 |
139.39 |
139.41 |
7.29 |
| O3-tune-i586-std |
145.05 |
10.90 |
155.95 |
155.96 |
-1.39 |
| O2-tune-i586-std |
146.18 |
11.97 |
158.15 |
164.49 |
0.00 |
| Os-tune-i586-std |
154.91 |
11.01 |
165.92 |
165.95 |
4.91 |
| O3-arch-i386-proftest |
142.45 |
10.90 |
153.35 |
153.37 |
-2.06 |
| O2-arch-i386-proftest |
145.59 |
10.99 |
156.58 |
156.61 |
0.00 |
| Os-arch-i386-proftest |
152.89 |
12.11 |
165.00 |
165.46 |
5.38 |
| O3-arch-athlon-xp-profbench |
114.99 |
11.62 |
126.61 |
131.68 |
-0.27 |
| O2-arch-athlon-xp-profbench |
115.69 |
11.26 |
126.95 |
129.43 |
0.00 |
| Os-arch-athlon-xp-profbench |
125.81 |
11.23 |
137.04 |
137.06 |
7.95 |
| O3-tune-i586-proftest |
136.63 |
10.63 |
147.26 |
147.25 |
-0.87 |
| O2-tune-i586-proftest |
137.81 |
10.74 |
148.55 |
154.50 |
0.00 |
| Os-tune-i586-proftest |
148.20 |
10.90 |
159.10 |
159.11 |
7.10 |
| O3-arch-athlon-xp-std |
124.06 |
11.11 |
135.17 |
135.18 |
-4.07 |
| O2-arch-athlon-xp-std |
129.27 |
11.64 |
140.91 |
140.93 |
0.00 |
| Os-arch-athlon-xp-std |
143.58 |
10.99 |
154.57 |
154.57 |
9.69 |
| O3-tune-i586-profbench |
136.90 |
10.53 |
147.43 |
147.45 |
-1.23 |
| O2-tune-i586-profbench |
138.25 |
11.02 |
149.27 |
155.24 |
0.00 |
| Os-tune-i586-profbench |
148.69 |
11.28 |
159.97 |
159.98 |
7.17 |
| O3-arch-i386-profbench |
142.23 |
10.53 |
152.76 |
152.77 |
-2.31 |
| O2-arch-i386-profbench |
145.03 |
11.35 |
156.38 |
156.39 |
0.00 |
| Os-arch-i386-profbench |
151.59 |
10.80 |
162.39 |
162.40 |
3.84 |
| O3-arch-i386-std |
148.08 |
11.26 |
159.34 |
159.36 |
-2.84 |
| O2-arch-i386-std |
152.76 |
11.24 |
164.00 |
166.88 |
0.00 |
| Os-arch-i386-std |
160.78 |
11.53 |
172.31 |
172.42 |
5.07 |
Effect of arch-related options (march, mtune)
| name |
user |
system |
cpu |
real |
diff (%) |
| O3-arch-athlon-xp-profbench |
114.99 |
11.62 |
126.61 |
131.68 |
-17.12 |
| O3-arch-athlon-xp-proftest |
117.12 |
11.72 |
128.84 |
128.90 |
-15.66 |
| O3-arch-athlon-xp-std |
124.06 |
11.11 |
135.17 |
135.18 |
-11.51 |
| O3-tune-i586-proftest |
136.63 |
10.63 |
147.26 |
147.25 |
-3.60 |
| O3-tune-i586-profbench |
136.90 |
10.53 |
147.43 |
147.45 |
-3.49 |
| O3-arch-i386-profbench |
142.23 |
10.53 |
152.76 |
152.77 |
0.00 |
| O3-arch-i386-proftest |
142.45 |
10.90 |
153.35 |
153.37 |
0.39 |
| O3-tune-i586-std |
145.05 |
10.90 |
155.95 |
155.96 |
2.09 |
| O3-arch-i386-std |
148.08 |
11.26 |
159.34 |
159.36 |
4.31 |
| O2-arch-athlon-xp-profbench |
115.69 |
11.26 |
126.95 |
129.43 |
-18.82 |
| O2-arch-athlon-xp-proftest |
118.65 |
11.27 |
129.92 |
129.93 |
-16.92 |
| O2-arch-athlon-xp-std |
129.27 |
11.64 |
140.91 |
140.93 |
-9.89 |
| O2-tune-i586-proftest |
137.81 |
10.74 |
148.55 |
154.50 |
-5.01 |
| O2-tune-i586-profbench |
138.25 |
11.02 |
149.27 |
155.24 |
-4.55 |
| O2-arch-i386-profbench |
145.03 |
11.35 |
156.38 |
156.39 |
0.00 |
| O2-arch-i386-proftest |
145.59 |
10.99 |
156.58 |
156.61 |
0.13 |
| O2-tune-i586-std |
146.18 |
11.97 |
158.15 |
164.49 |
1.13 |
| O2-arch-i386-std |
152.76 |
11.24 |
164.00 |
166.88 |
4.87 |
| Os-arch-athlon-xp-profbench |
125.81 |
11.23 |
137.04 |
137.06 |
-15.61 |
| Os-arch-athlon-xp-proftest |
127.72 |
11.67 |
139.39 |
139.41 |
-14.16 |
| Os-arch-athlon-xp-std |
143.58 |
10.99 |
154.57 |
154.57 |
-4.82 |
| Os-tune-i586-proftest |
148.20 |
10.90 |
159.10 |
159.11 |
-2.03 |
| Os-tune-i586-profbench |
148.69 |
11.28 |
159.97 |
159.98 |
-1.49 |
| Os-arch-i386-profbench |
151.59 |
10.80 |
162.39 |
162.40 |
0.00 |
| Os-arch-i386-proftest |
152.89 |
12.11 |
165.00 |
165.46 |
1.61 |
| Os-tune-i586-std |
154.91 |
11.01 |
165.92 |
165.95 |
2.17 |
| Os-arch-i386-std |
160.78 |
11.53 |
172.31 |
172.42 |
6.11 |
Effect of profiling
| name |
user |
system |
cpu |
real |
diff (%) |
| Os-tune-i586-proftest |
148.20 |
10.90 |
159.10 |
159.11 |
-4.11 |
| Os-tune-i586-profbench |
148.69 |
11.28 |
159.97 |
159.98 |
-3.59 |
| Os-tune-i586-std |
154.91 |
11.01 |
165.92 |
165.95 |
0.00 |
| O3-arch-i386-profbench |
142.23 |
10.53 |
152.76 |
152.77 |
-4.13 |
| O3-arch-i386-proftest |
142.45 |
10.90 |
153.35 |
153.37 |
-3.76 |
| O3-arch-i386-std |
148.08 |
11.26 |
159.34 |
159.36 |
0.00 |
| Os-arch-athlon-xp-profbench |
125.81 |
11.23 |
137.04 |
137.06 |
-11.34 |
| Os-arch-athlon-xp-proftest |
127.72 |
11.67 |
139.39 |
139.41 |
-9.82 |
| Os-arch-athlon-xp-std |
143.58 |
10.99 |
154.57 |
154.57 |
0.00 |
| O3-arch-athlon-xp-profbench |
114.99 |
11.62 |
126.61 |
131.68 |
-6.33 |
| O3-arch-athlon-xp-proftest |
117.12 |
11.72 |
128.84 |
128.90 |
-4.68 |
| O3-arch-athlon-xp-std |
124.06 |
11.11 |
135.17 |
135.18 |
0.00 |
| O2-arch-athlon-xp-profbench |
115.69 |
11.26 |
126.95 |
129.43 |
-9.91 |
| O2-arch-athlon-xp-proftest |
118.65 |
11.27 |
129.92 |
129.93 |
-7.80 |
| O2-arch-athlon-xp-std |
129.27 |
11.64 |
140.91 |
140.93 |
0.00 |
| O2-arch-i386-profbench |
145.03 |
11.35 |
156.38 |
156.39 |
-4.65 |
| O2-arch-i386-proftest |
145.59 |
10.99 |
156.58 |
156.61 |
-4.52 |
| O2-arch-i386-std |
152.76 |
11.24 |
164.00 |
166.88 |
0.00 |
| Os-arch-i386-profbench |
151.59 |
10.80 |
162.39 |
162.40 |
-5.76 |
| Os-arch-i386-proftest |
152.89 |
12.11 |
165.00 |
165.46 |
-4.24 |
| Os-arch-i386-std |
160.78 |
11.53 |
172.31 |
172.42 |
0.00 |
| O3-tune-i586-proftest |
136.63 |
10.63 |
147.26 |
147.25 |
-5.57 |
| O3-tune-i586-profbench |
136.90 |
10.53 |
147.43 |
147.45 |
-5.46 |
| O3-tune-i586-std |
145.05 |
10.90 |
155.95 |
155.96 |
0.00 |
| O2-tune-i586-proftest |
137.81 |
10.74 |
148.55 |
154.50 |
-6.07 |
| O2-tune-i586-profbench |
138.25 |
11.02 |
149.27 |
155.24 |
-5.61 |
| O2-tune-i586-std |
146.18 |
11.97 |
158.15 |
164.49 |
0.00 |