[TUTORIAL|ADVANCED] Beyond Reflection - AspectJ - Tracing

Discussion in 'Resources' started by Icyene, Sep 1, 2012.

Thread Status:
Not open for further replies.
  1. Offline

    Icyene

    So as some of you might have noticed, I love reflection, so it was only time before I made a tutorial series on it.

    Have you ever had one of those moments where you need an event in net.minecraft.server, but can't submit a pull request etc because it is very specific to YOUR needs? You could make an "installer" for your plugin, that would swap the necessary classes from craftbukkit.jar with your modified classes, but that's plain nasty.

    In this tutorial I will show you how to create a simple AspectJ based method interceptor. This guide assumes you have a basic understanding of reflection, and (hopefully) some knowledge of dynamic proxies.

    Now you may be wondering, what the heck is AspectJ? AspectJ is an aspect-oriented extension to the Java language. Basically it allows cool things to be done, things previously impossible with plain Java.

    A basic 'Hello World' program in AspectJ is here. Please view that before proceeding.

    In that video, a pointcut is created to match the execution of HelloWorld.sayHello(). That means that that program will say 'World!' whenever sayHello() is called, regardless from what class its called! Cool, huh?

    Now without further ado, lets make our AspectJ tracer. Note that installing the AJDT Eclipse plugin will help you greatly if you do not want to do this int notepad++.

    First, define your aspect. Aspects are like classes, but have a 'aspect' identifier instead.

    Code:Java
    1.  
    2. public aspect LogAspect{}
    3.  


    Now we need to create a pointcut. A pointcut is "called" whenever what its cutting is called. Our tracer will be made to log all public method calls. Our pointcut would look like this:

    Code:Java
    1.  
    2. pointcut publicMethodExecuted():
    3. execution(public * *(..));
    4.  


    That pointcut will be called on the execution of a public method. Note: the second * acts like a wildcard, so if you changed it to, say, B* the pointcut will only be called for all public methods beginning with a 'B'. The first '*' is for the modifier of the methods you are pointcutting. These are final, volatile, transient, etc. The * represents that the tracer will trace all method calls, regardless of modifier.

    Now lets make something that handles the calling of the pointcut publicMethodExecuted(). AspectJ does have some strange constructs, I can say that. This is what the handling would look like:

    Code:Java
    1.  
    2. after(): publicMethodExecuted(){
    3.  
    4. }
    5.  


    Now if you put a System.out.println() in that, it would print whenever a public method is executed! The extra sauce resides in an object named thisJoinPoint. It stores the information about your cut. So lets use thisJoinPoint to print out the arguments that the traced method is passed. You have neat methods like .getArgs(), .getSignature() etc. With this knowledge, we can easily construct a tracer. Note that while AspectJ DOES have different constructs, its basis is Java, so all Java code is functional AspectJ code. In the end, our tracer would look like this:

    Code:Java
    1.  
    2. public aspect LogAspect{
    3.  
    4. pointcut publicMethodExecuted(): execution(public * *(..));
    5.  
    6. after(): publicMethodExecuted(){
    7. System.out.printf("Enters on method: %s. \n", thisJoinPoint.getSignature());
    8.  
    9. Object[] arguments = thisJoinPoint.getArgs();
    10. for(int i =0; i < arguments.length; i++){
    11. Object argument = arguments[I];[/I]
    12. if(argument !=null){
    13. System.out.printf("With argument of type %s and value %s. \n", argument.getClass().toString(), argument);
    14. }
    15. }
    16. System.out.printf("Exits method: %s. \n", thisJoinPoint.getSignature());
    17. }
    18. }
    19.  


    That would go in a separate aspect file (.aj extension). Note that the standard Eclipse compiler will NOT be able to compile your code. AspectJ relies on "weaving" the AspectJ code with bytecode, and utilises BCEL. Its kind of like LOLPython. Its a different language, but compiles into Python. You will need to use the AspectJ Compiler (ajc.exe).

    And that's it! Give yourself a pat on the back if you got this far and got the program to work. If not, keep trying!

    AspectJ opens up alot of doors for plugin development. If you had a program that made custom entities behave differently on moving, with AspectJ you wouldn't have to create a separate custom entity class. You would just have to pointcut the movement method, and AspectJ would do its magic!

    Uses of AspectJ in Plugin Development
    • Debugging - just pointcut your methods! No need to modify source files that can be thousands of lines long!
    • Hacking net.minecraft.server - its amazingly fun.
    • Modularity - the actual use of AspectJ.
    Additional Links
    • Tutorials, the AspectJ Eclipse plugin, docs, everything you can think of and more is located at the AspectJ Eclipse project page.
    • AspectJ: In Action. Its a wonderful book that teaches you how to use AspectJ to its full potential.

    Small Demo Class:
    To test your tracer, run this class.

    Code:Java
    1.  
    2. public class Main {
    3.  
    4. public static void main(String[] args) {
    5. System.out.println("Testing AspectJ Tracer.");
    6. aMethod(1, 2, 3);
    7. }
    8.  
    9. public static void aMethod(int a, int b, int c){}
    10.  
    11. }
    12.  


    If you made your tracer correctly, it will output:

    Code:
    Testing AspectJ Tracer.
    Enters on method: void com.github.Icyene.AspectJ_Demo.Main.aMethod(int, int, int). 
    With argument of type class java.lang.Integer and value 1. 
    With argument of type class java.lang.Integer and value 2. 
    With argument of type class java.lang.Integer and value 3. 
    Exits method: void com.github.Icyene.AspectJ_Demo.Main.aMethod(int, int, int). 
    Enters on method: void com.github.Icyene.AspectJ_Demo.Main.main(String[]). 
    With argument of type class [Ljava.lang.String; and value [Ljava.lang.String;@1128f5a. 
    Exits method: void com.github.Icyene.AspectJ_Demo.Main.main(String[]).
    
    With your package names, of course. Fun, fun, fun!

    [EXTREME] Beyond Reflection - ASM - Runtime Retransforming


    md_5 / Jacek would Bukkit plugins that use AspectJ be rejected on the basis that code can't be properly decompiled? AspectJ woven decompiled code looks like this:
    [​IMG]
    The majority of the plugin can be understood, but the AspectJ parts become impossible to understand.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 28, 2016
    one4me likes this.
  2. Offline

    md_5

    Oh. My. God.
    This is amazing, I hope it works with Netbeans!

    No, the use of AspectJ is not a reason for rejection, we deal with far worse code. Use it all you like :D
     
    Siggy999 and codename_B like this.
  3. Offline

    Icyene

    md_5 Yay! There is. Thanks! Means alot!:)
     
  4. Offline

    md_5

    I'm just reading the docs now, and I see that I can specify my stuff to be executed after(). But for hacking the server the best thing to do is execute before() and then stop the REAL server code from running. How can this be done, to say replace an entire method in the server.
     
  5. Offline

    Icyene

    No clue yet :D Possibly there may be a cancel function that can be run before(), but I wrote this tutorial right after I figured out how to make the tracer :p I will create a part 2 or update this thread once I figure it out. It SHOULD be possible. I'm using after() as a way to add a CanSnowFormEvent.
     
  6. Offline

    md_5

    An extended Google reveals that a before() aspect will only execute the real method if the proceed();method is called.
    That's what should happen anyway, will have to test.
     
  7. Offline

    Icyene

    That's half the battle! But then you'd have to add in your own return value... This can be easily achieved with proxies, but that would require the method you are hacking to 1. be in an object and 2. be able to replace all references to that object to your proxied object. An alternative would be to use ASM and inject your own method, but that would only work if Bukkit had a premain - type - thing that plugins could execute their ASM code before NMS loads.
     
  8. Woah this is awesome! Could we use this on Zombie Ablockalypse? Also would users using the plugin have to use an AspectJ program to run the plugin?
     
  9. Offline

    Icyene

    Yes, we could. But for our purposes overriding is much easier. No, AspectJ "weaves" itself into normal bytecode. If you don't want it to weave, compile like a normal jar and add the contents of ajruntime.jar into the plugin jar, and delete META-INF.
     
  10. Ok, if we used AspectJ though, wouldn't it be less maintenance to update the plugin?
     
  11. Offline

    Icyene

    iKeirNez Yes. I'd do it if and when I can actually use AspectJ for that. At the moment, I am a n00b at it. Just Skype me... Much faster than posting...

    md_5 I got it! I got it! I got it! (does happy dance :D)
    The trace aspect:

    Code:Java
    1.  
    2.  
    3. public aspect LogAspect {
    4.  
    5. pointcut publicMethodExecuted(): execution(private * *(..));
    6.  
    7. after(): publicMethodExecuted(){
    8. System.out.printf("Enters on method: %s. \n",
    9. thisJoinPoint.getSignature());
    10.  
    11. Object[] arguments = thisJoinPoint.getArgs();
    12. for (int i = 0; i < arguments.length; i++) {
    13. Object argument = arguments[I];[/I]
    14. if (argument != null) {
    15. System.out.printf("With argument of type %s and value %s. \n",
    16. argument.getClass().toString(), argument);
    17. }
    18. }
    19. System.out.printf("Exits method: %s. \n", thisJoinPoint.getSignature());
    20. }
    21.  
    22. Object around() : publicMethodExecuted() {
    23.  
    24. Object result = proceed();
    25. System.out.printf("Method %s returns %s \n",
    26. thisJoinPoint.getSignature(), result);
    27. return result + " was Intercepted and changed!";
    28.  
    29. }
    30.  
    31.  


    And the main:

    Code:Java
    1.  
    2. public class Main {
    3.  
    4. public static void main(String[] args) {
    5. System.out.println("Testing AspectJ Tracer.");
    6. System.out.println(aMethod(1, 2, 3));
    7. }
    8.  
    9. private static String aMethod(int a, int b, int c) {
    10.  
    11. return "aString";
    12. }
    13.  
    14. }
    15.  
    16.  


    It prints out this:

    Code:
    Testing AspectJ Tracer.
    Enters on method: String com.github.Icyene.AspectJ_Demo.Main.aMethod(int, int, int).
    With argument of type class java.lang.Integer and value 1.
    With argument of type class java.lang.Integer and value 2.
    With argument of type class java.lang.Integer and value 3.
    Exits method: String com.github.Icyene.AspectJ_Demo.Main.aMethod(int, int, int).
    Method String com.github.Icyene.AspectJ_Demo.Main.aMethod(int, int, int) returns aString
    aString was Intercepted and changed!
     
    
    :D

    Here we are using around() to get the return value before its returned, and swapping it. Let the hacking begin! NMS doesn't stand a chance...:D

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 28, 2016
  12. Offline

    md_5

    Amazing, I see there is a Maven addon for AspectJ so in theory it will compile fine but Netbeans will gawk at the source file.
     
  13. Offline

    Icyene

    md_5 There is also a AspectJ plugin for Netbeans. If Netbeans is anything like Eclipse that will give you a new view of the source file in a different editor. Also I haven't gotten how to actually weave the file >.< I got it to run fine in Eclipses console but on compilation the decompiler class is importing org.aspectj...
     
  14. Offline

    caldabeast

    codegasm
    that is all.
     
    bobacadodl likes this.
  15. Offline

    Icyene

    md_5 Sorry for tagging you again, but: "hacking" NMS is much more difficult then it seems, now that I have learned a bit more. I got the AspectJ: In Action book and read the first 8 chapters so far. The methods posted above will NOT be able to "hack" NMS, because NMS is not available at compile time for weaving. For this, you must use load-time weaving, which is considerably harder. Even that might not solve the problem... See my post here, maybe by the time you see this its already been answered.

    Also, my second comment on the post in the link is obviously a lie; I had to make it to avoid confusion :)
     
  16. Offline

    md_5

    So you need the source to compile time weave?
     
  17. Offline

    Icyene

    md_5 It seems so =/ But Java 6 introduced the Attach API, which allows agents to attach to a JVM to do their dirty magic. If I can figure out how to get my ASM-based profiler to use the API, then its a two way win: 1. It works 2. ASM is 69kb, vs AspectJ which is 159kb. OFC, not all the files are needed. For the profiler only 3 ASM classes are used. A shrinker like Proguard could probably remove all the unnecessary classes, making it more like ASM: 6kb, AspectJ: 20kb.
     
  18. Offline

    md_5

    Oh well, good luck with that :)
     
  19. Offline

    Icyene

    Small progress report: I got attaching to a JVM working, and have a minimalistic attach-based profiler working. The plugin creates a new JVM onEnable, and attaches itself to the JVM used by Bukkit (The process for getting the JVM id of Bukkit remains sketchy, haven't found a way to do it automatically yet. Does Bukkit have any value that can be accessed by reflection or otherwise that determines its PID?).

    The JVM started by the plugin is then terminated. Then I use the Java late-binding agent. It allows to reinstrument Bukkit/NMS classes at runtime, AFTER they are loaded. At the moment, I am using the JavaAssist library for reintrumenting for ease-of-use. Its 2.3MB when zipped, however. I am rewriting the instrumentation class to use ASM. Much smaller (69KB, when unzipped).

    This method is undeniable very, very hacky. It was pulled from the code of a program made to hack into a RuneScape client... Indubitably nasty. Then again, so are most monkey patches, and they usually provide wonderful things.
     
  20. Offline

    md_5

    You can use another part of the Java instrumentation framework to get the PID of a process.

    Code:
            String jvm = ManagementFactory.getRuntimeMXBean().getName();
            String pid = jvm.substring(0, jvm.indexOf('@'));
    Code only tested on Linux.
     
  21. Offline

    PandazNWafflez


    h31ix And the rest of the BukkitDev approval team:

    uMad?
     
  22. Offline

    Gravity

    Files that make it unable for us to do our job will be rejected, period. No questions asked.

    Are we MAD about this? No, but you will be when you try to use it and can't. I can't tell if you're making a joke or actually being rude, but you would think for a person who develops plugins yourself you would have a bit more respect for the people that have reviewed tens of thousands of plugins in order to keep you safe.
     
    Pew446 and bobacadodl like this.
  23. Offline

    one4me

    It can still be decompiled, it's just that it may take longer for someone to understand what's going on (similar to obfuscated code). Anyways md_5 already responded in the 5th post saying that using this was fine.
     
  24. Offline

    Icyene

    h31ix I'm confused... Will using AspectJ reject your plugin? Or not? md_5 stated that it wouldn't, but you imply that it would... So, would it? :O

    md_5 How could I have not realized that the plugin would be sharing the same VM with Bukkit... >.<

    one4me No, the parts using AspectJ are unreadable (completely), MUCH worse than obfuscation. Its filled with stuff like int i = '???' and aMethod$SomeVal$AClass etc.
     
  25. Offline

    Gravity

    Like I said before, if something you do to your file prevents us from doing our job (decompiling and reading it) it will be rejected. If AspectJ does that to a file, it will be rejected. If not, it will not be rejected. It's all about our ability to do our job.
     
  26. Offline

    Icyene

    h31ix Its a very fine line, then, is it not? It does not make a decompiler fail to decompile it, but depending on the extent that you use it you may end up having String s = ???, And other similar things. People who use AspectJ can't really determine how AJ weaves their files... If specific classes of a plugin look like the image I posted way above, would they be rejected? Just based on that, for future reference :)
     
  27. Offline

    Gravity

    It's... not really a fine line, it's more of like if you screw with your stuff so that we can't read it, it will be rejected.

    The photos you posted would indeed be rejected, if there's a variable declaration we can't read, or the decompiler fails to convert the Java byte code into readable Java, then that's not ok.
     
  28. Offline

    Icyene

    Ouch... That makes AspectJ use in release plugins useless, and limits its use to debugging =/ But its not like we want to screw with our stuff; the AJ compiler does that... I would argue but its sure I wouldn't win... For future note, is the use of raw bytecode in plugins utilizing ASM a point for rejection, too?
     
  29. Offline

    Gravity

    Welp, sorry. Security and safety has always and will always be our #1 priority on BukkitDev, it's a necessity to make people know they are safe and discourage potential malicious people from uploading their stuff.

    To address your question, it's like I've been saying; if it cannot be properly decompiled and reviewed for safety, it cant be used. We can't read straight bytecode, and therefor if you're using that instead of java in a way that we cannot decompile, it's against the rules.
     
  30. Offline

    Icyene

    It would be more like

    Code:
      this.visitLdcInsn(methodName);
                this.visitMethodInsn(INVOKESTATIC, 
                    "sample/profiler/Profile", 
                    "end", 
                    "(Ljava/lang/String;Ljava/lang/String;)V");
                break;
    
    Where the last line can be much longer. Its not a decompiler failing to decompile; it looks like that in an editor as well.

    md_5 I got it working! Works like a charm, attaches after JVM loads. ATM is huge, and very inconvenient: The Attach API needs tools.jar, which comes only with JDK and is 14mb in size. Additionally, it also needs to point to attach.dll (or .so on linux). Running it with JRE instead of JDK java.exe produces the following error:

    Code:
    java.util.ServiceConfigurationError: com.sun.tools.attach.spi.AttachProvider: Provider sun.tools.attach.WindowsAttachProvider could not be instantiated: java.lang.UnsatisfiedLinkError: no attach in java.library.path
    Exception in thread "main" java.lang.RuntimeException: com.sun.tools.attach.AttachNotSupportedException: no providers installed
    
    Working on fixing, have come to the conclusion that only 5 classes from tools.jar are needed, and they are pretty small. As is the attach library.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 28, 2016
Thread Status:
Not open for further replies.

Share This Page