Interesting Benchmark

I just found how dramatic this is, and I think I've posted enough in resources, so I am putting it here.

Most plugins use List<Value> = new ArrayList<Value>. But why just not use a ArrayList initially? List is an interface around ArrayList and LinkedList. From what I have seen, few people ever use the LinkedList methods inherited. Basically, there is no difference for most plugins between List<Value> = new ArrayList<Value> and ArrayList<Value> = new ArrayList<Value>, except for a significant performance dip.

I used the following code to benchmark this:

Code:Java

 
public class Test {
 
    public static void main(String[] args) {
 
    long start = System.currentTimeMillis();
    System.out.println("Beginning test: List. Time: " + System.currentTimeMillis());
    List<Integer> listA = new ArrayList<Integer>();
    for(int i = 0; i != 5000000; ++i) {        
        listA.add(1);      
    }
    long finish = System.currentTimeMillis();
    System.out.println("Test finished. Time it took: " + (finish-start));
 
    start = System.currentTimeMillis();
    System.out.println("Beginning test: ArrayList. Time: " + System.currentTimeMillis());
    ArrayList<Integer> listB = new ArrayList<Integer>();
    for(int i = 0; i != 5000000; ++i) {        
        listB.add(1);        
    }
    finish = System.currentTimeMillis();
    System.out.println("Test finished. Time it took: " + (finish-start));
 
    }
}
 

Not the most accurate of benchmarks, but the results returned are stunning.

Code:

Beginning test: List. Time: 1347062977015
Test finished. Time it took: 1125
Beginning test: ArrayList. Time: 1347062978140
Test finished. Time it took: 688

List was 2x slower than ArrayList in the long run!

Conclusion: don't use lists in high usage areas (PlayerMoveEvent handlers, for one). Following this will increase performance.

I hope this was informative! If you find any major flaw in it, feel free to point it out and I will rectify it!

 

This doesn't make sense at all. The type of the variable doesn't change the type of the object. Here are my results:

Code:

Beginning test: List. Time: 1347063523024
Test finished. Time it took: 47
Beginning test: ArrayList. Time: 1347063523071
Test finished. Time it took: 52

Almost identical, but I will say that the second test was consistently slightly longer.

Edit: I changed the code to do 10x more iterations, and got these results:

Code:

Beginning test: List. Time: 1347063697435
Test finished. Time it took: 456
Beginning test: ArrayList. Time: 1347063697891
Test finished. Time it took: 899

Getting worse for the second test. But, then I manually called the GC before each test, and here are the new results:

Code:

Beginning test: List. Time: 1347063725752
Test finished. Time it took: 482
Beginning test: ArrayList. Time: 1347063726678
Test finished. Time it took: 396

Then I reversed the order of the tests, and got these interesting results:

Code:

Beginning test: ArrayList. Time: 1347064031598
Test finished. Time it took: 456
Beginning test: List. Time: 1347064032369
Test finished. Time it took: 392

So, basically, there is pretty much no meaningful difference between the two, and the state of your GC and order you run these tests in matters more than the type of variable you define.

 

Likely there are some just-in-time compilation tricks the JVM is doing. This makes java very difficult to test

 

nisovin
Perhaps. Your results are very interesting, I must say that. For all my tests, however, the difference was much greater, averaging at 500ms difference. Still, it might still be more efficient (even if by a little) to just use ArrayList... I mean, there really isn't a great difference from day-to-day use.

Sagacious_Zed
Good point.

EDIT: I have moved the instantiation of the List/ArrayList before getting currentMilliseconds, and the results do change. The difference is now averaging at 300.

 

Go read up on dynamic dispatch and how it pertains to Java and the jvm.

 

Did you try completely reversing the order of the tests?

 

nisovin Yep, forgot to include that in my edit. The results stayed the same.
Sagacious_Zed Will do

 

I'm not really sure how you can claim this, given that my test results don't show this at all. Every test I ran, the order of the tests was what caused the difference, not the variable type. In fact, you can run two identical tests in a row (use ArrayList both times) and get a similar result.

Edit: Here's a better test. Comment out the second test, then run the first on its own. Then comment out the first test, and run the second on its own. Then see what kind of results you get.

 

As I said, it MIGHT be more efficient. For you the time changed when you reversed the tests, yet for me they stayed very similar. Either way, I changed the title to make it a bit more truthful.

I got your general range of results when I used ArrayList twice, yet it the previous test's (ArrayList <> List). Its most likely the state of the GC, as you said. For my curiosity, what JRE version do you have, and how are you running these tests? I am running it on Java 6, by the way (not that a different JRE version should change anything, but maybe they changed the way ArrayLists/Lists are handled, and the difference between our JREs may be causing it (slim chance)).

 

I have heard people talk about this before. I have seen statistics that accessing a method through an interface reference is 30% slower... I tested this to see if it is true. In all my tests it has been only 0.1%-2.0% slower. Apparently, it depends heavily on the JRE and what JIT compiling it is doing.

EDIT: Even if the performance is identical... I always use the narrowest identification I can.

 

Using JRE 7, I made it return an average of 200 fills, results:

List Average Time: 30
ArrayList Average Time: 36

It's irrelevant how you initialize a list, Java being Java is the cause of the strange deviation.

EDIT:

Here's the result of 1000 fills:

List Average Time: 31
ArrayList Average Time: 31

And here's the result of 1000 fills of BOTH ArrayList:

ArrayList 1 Average Time: 32
ArrayList 2 Average Time: 36

Finally, here's the average of 1000 fills of BOTH List:

List 1 Average Time: 32
List 2 Average Time: 32

 

Same

My friend's (a Python programmer) answer to all my questions of something like "why do you have to do new Pair<String, new Triplet<String, Integer>()>()?" His answer remains the same, 19 questions later: "BECAUSE ITS JAVA!". And then goes trying to convince me to use Python instead of Java. I must say, Python is much more straightforward. Too bad all the juicy things to mess around with are written in Java.

 

All the real juicy things are made in C++

Java is just... strange to say the least.

 

Yes but Java is easy to hijack with ASM et alia. For C++ you'd have to write your own injector for the fun to begin, and I haven't gotten there yet