Direkt: 3 42000000000
Getter: 350 42000000000

Right now – without Server – I get

Direkt: 1656 42000000000
Getter: 1656 42000000000

on my Thinkpad. With server I get

Direkt: 0 42000000000
Getter: 1031 42000000000

I’d think the JVM removes the object completely and just inlines the variable.

Direkt: 2 42000000000
Getter: 552 42000000000

which is approximately 3oox slower for the getter. I’m stumped and stunned.

Any ideas?

David

long sumGetter= 0L;
long tGetterStart = System.currentTimeMillis();
for (int i=0; i<1000000000; i++) {
sumGetter += test.getValue();
}
long tGetter = System.currentTimeMillis() – tGetterStart;

System.out.println(“Direkt: ” + tDirekt + ” ” + sumDirect);
System.out.println(“Getter: ” + tGetter + ” ” + sumGetter);

And here is the output:
Direkt: 401 42000000000
Getter: 407 42000000000

So back to the original example. Perhaps JVM detects that the sum is zeroed out after the first loop, and completely bypasses the loop.

Direkt: 3 42000000000
Getter: 401 42000000000

What’s interesting is when I marked the value field final
I got the following results:

Direkt: 3 42000000000
Getter: 73 42000000000

Marking the getter final does not have any affect.

Test3: On Windows XP

with -server option

Direkt: 250 42000000000
Getter: 938 42000000000

Direkt: 0 42000000000
Getter: 1063 42000000000

java version “1.6.0_10-rc”
Java(TM) SE Runtime Environment (build 1.6.0_10-rc-b28)
Java HotSpot(TM) Client VM (build 11.0-b15, mixed mode, sharing)

with -server option

Direkt: 1765 42000000000
Getter: 938 42000000000

Test4: Ubuntu Running in Sun VirtualBox on Windows XP (This test may not be of any relevance as the base instructions are again run on windows, but still thought of putting it here)

with -server option

Direkt: 78 42000000000
Getter: 1362 42000000000

This is getting Interesting !!!!!!!!!!!!!!!!!!!!!

java version “1.6.0_07″
Java(TM) SE Runtime Environment (build 1.6.0_07-b06)
Java HotSpot(TM) Client VM (build 10.0-b23, mixed mode, sharing)

Direkt: 1688 42000000000
Getter: 1687 42000000000

Test2: Ubuntu Running in Sun VirtualBox on Windows XP (This test may not be of any relevance as the base instructions are again run on windows, but still thought of putting it here)

java version “1.6.0_06″
Java(TM) SE Runtime Environment (build 1.6.0_06-b02)
Java HotSpot(TM) Client VM (build 10.0-b22, mixed mode, sharing)

Direkt: 1817 42000000000
Getter: 1860 42000000000

Java(TM) SE Runtime Environment (build 1.6.0_06-b02)
Java HotSpot(TM) 64-Bit Server VM (build 10.0-b22, mixed mode)

Direkt: 12 42000000000
Getter: 1002 42000000000

Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_11-b03)
Java HotSpot(TM) Server VM (build 1.5.0_11-b03, mixed mode)

Direkt: 106 42000000000
Getter: 642 42000000000

2.) There is no Test object in the code anymore, no direct access only a variable because “… the JVM might completly inline an object if it detects only one read-access object inside a loop.”

3.) See my comment #25 and above to make the JVM use objects and prevent eliminating them alltogether (what the JVM seems to do for 1 and 2 object but not 3 and above). The non-server JIT doesn’t eliminate objects.

a) I modified the benchmark with extra loops to “warm” up the hotpot vm before the actual benchmark:

public class Test
{
public int value = 42;

public int getValue()
{
return value;
}

public static void main( String[] args )
{
Test test = new Test();

int dummy;

long sum = 0L;
// extra “warm-up” direct loop
for (int i = 0; i < 1000000000; i )
{
sum = test.value;
sum = test.getValue();
}

sum = 0L;
long tDirektStart = System.currentTimeMillis();
for (int i = 0; i < 1000000000; i )
{
sum = test.value;
}
long tDirekt = System.currentTimeMillis() – tDirektStart;

sum = 0L;
long tGetterStart = System.currentTimeMillis();
for (int i = 0; i java -server -version
java version “1.6.0_03″
Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
Java HotSpot(TM) Server VM (build 1.6.0_03-b05, mixed mode)

C:\Projects\java\trunk\misc\target\classes>java -version
java version “1.6.0_03″
Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
Java HotSpot(TM) Client VM (build 1.6.0_03-b05, mixed mode, sharing)

C:\Projects\java\trunk\misc\target\classes>java -server -cp . Test

Direkt: 15 42000000000
Getter: 1250 42000000000

C:\Projects\java\trunk\misc\target\classes>java -server -cp . Test

Direkt: 0 42000000000
Getter: 1266 42000000000

C:\Projects\java\trunk\misc\target\classes>java -cp . Test
Direkt: 3282 42000000000
Getter: 3297 42000000000

@lumpynose:
With the -server jvm I see at a 2 orders of magnitude higher performance with direct access. If the server jvm was optimizing the getter to an inline call, why is this difference?

Direkt: 2496 42000000000
Getter: 2490 42000000000

java version “1.6.0_10-rc”
Java(TM) SE Runtime Environment (build 1.6.0_10-rc-b28)
Java HotSpot(TM) Client VM (build 11.0-b15, mixed mode, sharing)

int NO = 10;
test = objects[i % NO];

is slow

final int NO = 10;

is much faster.

As is 256 (obviously) compared to 257.

Another interesting tidbit:

The JVM is very fast for NO=1 or NO=2 (one or two test objects), where direct access is 0 or 1 ms for the number of calls on my machine. For NO=3 (three objects) direct access and getters have the same speed (with -server, JDK 1.6.10, Windows XP)


Test[] objects = new Test[100];
for (int i = 0; i < 100; i ) {
Test t = new Test();
t.value = i;
objects[i] = t;
}


and


test = objects[i % 100];


results in


Direkt: 6140 49500000000
Getter: 6032 49500000000
on my setup above.

Not sure how much the array access (and mod) masks performance though.


public class Test {
public int value = 42;

System.out.println("Direkt: " tDirekt " " sum);
System.out.println("Getter: " tGetter " " sum);
}
}


C:\Documents and Settings\Dhananjay>java Test
Direkt: 4734 42000000000
Getter: 3985 42000000000

C:\Documents and Settings\Dhananjay>java Test
Direkt: 4687 42000000000
Getter: 3969 42000000000

C:\Documents and Settings\Dhananjay>java Test
Direkt: 4672 42000000000
Getter: 3953 42000000000

C:\Documents and Settings\Dhananjay>java -version
java version “1.5.0_09″
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b01)
Java HotSpot(TM) Client VM (build 1.5.0_09-b01, mixed mode, sharing)


Windows XP SP2, Java 1.6.03, DualCore2, 6300, 1.86Ghz, 2gb

Direkt: 2312 42000000000
Getter: 2297 42000000000


I’ll take a look, perhaps Eclipse / IDEA only used a JRE not a JDK, and the code is faster with the JDK. (Server?)

5ms for 100.000.000 would be very fast though. Perhaps some server JIT kicks in.

At least that would be significant and people should use more direct access? ;-)

Not sure why that is the case.

OS : Ubuntu Gutsy Gibbon 7.10
Kernel : 2.6.22-15-generic
CPU : Intel® Core™ Duo CPU T2600 @ 2.16GHz
RAM : 2GB
JVM :
java version “1.6.0_03″
Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
Java HotSpot(TM) Server VM (build 1.6.0_03-b05, mixed mode)

Output of the program

Run 1 (250 times faster)
Direkt: 5 42000000000
Getter: 1223 42000000000
Run 2 (a little less than 100 times faster)
Direkt: 14 42000000000
Getter: 1333 42000000000
Run 3 (more than 150 times faster)
Direkt: 7 42000000000
Getter: 1216 42000000000

I’m not sure about memory consumtion – the JVM/JDK got better in recent releases. Most of the memory consumtion might be the base JDK classes – as you said – and an empty heap.

>> Before DHH started flying around the world doing conferences, having RoR on the resume was a good thing. These days, I would consider it a warning sign.

So true. RoR has been hijacked by these mediocre wannabe programmers who sound like they think Ruby is the coolest thing because “hello world” could be scaffolded in a few minutes.

Especially after the Twitter debacle and Zed Shaw’s “Rails is a Ghetto” rant I think the days of the Ruby hype are over.

In my opinion, these are random numbers. We can move to other things.

sum = iTest.getValue();

into

sum = iTest.value;

Yes, getters and setters are slow if the method/function call is really made. But in the case of the Java JIT it inlines the code and removes the call.

With the following code, my test result is as follows:
========================================
many objects – getter/setter 4826 ms
one object – direct access 5293 ms
one object – getter/setter 4747 ms
many objects – direct access 4911 ms
========================================

Here’s the code:
=Dummy.java===============================
public class Dummy {
String data;
public String getData() { return data; }
public void setData(String data) { this.data = data; }
}
========================================

=DummyRunner.java===========================
import java.util.HashMap;
import java.util.Map;
import java.util.Random;

public class DummyRunner {

public static final Map TIME_KEEPING =
new HashMap();
public static final int LOOP_SIZE = 1000000;
public static final Random rand = new
Random(System.currentTimeMillis());
public static final int RANDOM_STRING_SIZE = 12;

public static void main(String[] args) {
String temp = “”;
Dummy d = new Dummy();

String test1 = “one object – direct access”;
printHeading(test1);
startWatch(test1);
for (int i = 0; i < LOOP_SIZE; i ) {
d.data = randomString();
temp = d.data;
}
stopWatch(test1);

String test2 = “one object – getter/setter”;
printHeading(test2);
startWatch(test2);
for (int i = 0; i < LOOP_SIZE; i ) {
d.setData(randomString());
temp = d.getData();
}
stopWatch(test2);

String test3 = “many objects – direct access”;
printHeading(test3);
startWatch(test3);
for (int i = 0; i < LOOP_SIZE; i ) {
Dummy d1 = new Dummy();
d1.data = randomString();
temp = d1.data;
}
stopWatch(test3);

String test4 = “many objects – getter/setter”;
printHeading(test4);
startWatch(test4);
for (int i = 0; i < LOOP_SIZE; i ) {
Dummy d1 = new Dummy();
d1.setData(randomString());
temp = d1.getData();
}
stopWatch(test4);

printResult();
}

private static String randomString() {
String randomString = “”;
for (int i = 0; i > ” testName “…”);
}

private static void stopWatch(String testName) {
TIME_KEEPING.put(testName,
System.nanoTime() –
(TIME_KEEPING.get(testName))
);
}

private static void startWatch(String testName) {
TIME_KEEPING.put(testName, System.nanoTime());
}
}
========================================

My roundabout way of doing the same test is an attempt to show that VM does its job in a variety of situations, which should be comforting news of Java folks.

Mike

Read the blog post: “I know, micro benchmarking, I’m not sure what the JVM does in this case, inlining, loop unrolling, object removal, virtual methods and all – and it shouldn’t matter because the JVM optimizing getters is the whole point [...]“

If … Java … is … clever … enough … to … inline …. 90% of getters …. then … I …. don’t …. care … how … Java … is … able … to … do … that.

On a more serious note: Because of the sum, Java cannot optimize completly away the for loop.

More astonishing is this:

public class Test implements ITest {
  public int value = 42;

  public int getValue() {
    return getRealValue();
  }
  public int getRealValue() {
    return value;
  }

  public void change() {
    value = (int) System.currentTimeMillis() / 10000;
  }

  public static void main(String[] args) {
    Test test = new Test();
    ITest iTest = test;
    int dummy;
    for (int i=0; i<1000000; i  ) {
      dummy = test.value;
      dummy = iTest.getValue();
    }

    long tDirektStart = System.currentTimeMillis();
    long sum = 0L;
      test = new Test();
    for (int i=0; i<100000000; i  ) {
      sum  = test.value;
    }

    long tDirekt = System.currentTimeMillis() - tDirektStart;

    sum= 0L;
    long tGetterStart = System.currentTimeMillis();
      iTest = new Test();
    for (int i=0; i<100000000; i  ) {
      sum  = iTest.getValue();
    }
    long tGetter = System.currentTimeMillis() - tGetterStart;

    System.out.println("Direkt: "   tDirekt   " "   sum);
    System.out.println("Getter: "   tGetter   " "   sum);
  }

}

with


Direkt: 343 4200000000
Getter: 719 4200000000

And object creation inside the loop (clever object caching):


Direkt: 1844 4200000000
Getter: 1828 4200000000


Microbenchmarks. They just prove that it's dangerous to say things like "300 times faster" with an optimizing JIT.

get/set WILL be slower that direct access, on every language, everywhere, every time. Maybe not on quantum computing but we’ll die before that :)

“If you’ve ever done a serious project in an OO language then you’ll almost certainly have run into the fragile base class problem.”
I’ve heard and read this statement a couple of times. It is so dumb. It like saying “Structural programming is bad. It makes the code so full of deep indentation that you have to scroll horizontally when reading the source code. That’s not a problem if you use GOTOs.”

another thing you could have done is to use System.nanoTime(); instead to see how accurate your “microbenchmark” is.

BR,
~A

keep these posts coming. I know that you’re not the SANITY KEEPER OF THE INTERNET. All the same, if you have time, for those issues that really matter, you can blog just to make the rest of us java developers lives easier.

BR,
~A