How to unit test your plugin (with example project)

Discussion in 'Resources' started by Pandarr, Jun 28, 2011.

Thread Status:
Not open for further replies.
  1. Offline

    Pandarr

    Alright... for super small projects I don't think it makes a ton of sense to unit test a project, but for some bigger ones it can be very helpful.

    What is unit testing? In short, it's making sure your methods do what you think they are doing. Sure you can test in game. But that can be tedious. Plus it's not repeatable all the time nor good for regression.

    Example project functionality
    In my example project I have created a plugin that does the following:

    During the player join event, if the players name begins with "P", send them a message. Otherwise do not send them a message. Yes it's a dumb project. Feel free to run it on your server if you so desire that functionality... ;)

    Crappy way of testing
    Now how can I test this? Normally I would compile, start up my server, go into my client, and see if the message comes up. My Minecraft login doesn't start with a "P" so I would not get a message. Now I either have to change my code to the letter that is the beginning of my minecraft login to test the other case, or find somebody whose name does start with "P" to login to my server. Kind of a pain either way.

    So... unit testing with Mock Objects is a good way to do this. You still *should* go into the game and ensure various things are working. But unit testing will make you feel like it's working as desired for the most part.

    Better way of testing
    Since most of Bukkit is filled with Interfaces and Abstract classes, it's really hard to instantiate them. With a mock object though, you can in a sense. You might think, "well that's a lot of fake data I have to generate!" and you'd be wrong.

    In this instance all we care about is the players name. We need to test it in the onPlayerJoinEvent(). So first, we make a mock "PlayerJoinEvent" object. PlayerJoinEvent has a fair amount of methods. The only one we care about is .getPlayer() though. So we tell our mock event that when .getPlayer() is called on it, return our mock Player that we've also made.

    With the player, we also know we're only going to call .getName(). So we tell that mock player object that when .getName() is called on it, return this String.

    Then we send our event into our method. Now of course we have no client to actually see the message. So how to do we know if the message was sent? That is what verify is for. We still have a reference to our mock player object. So we ask the mocking library to verify that .sendMessage() was called on our player. We can give it specific text to check for or just anyString(). Likewise for our other player whose name does not start with P, we can check that that method was not called on the player.

    A header to break this section from the prior one
    Mock objects for your plugin are *not* meant to test the functionality of Bukkit. They are meant to ensure that when Bukkit calls these methods that it does the codepath you expect. If Bukkit's sendMessage() method is jacked up then there's not much you can do about that in your plugin. However you can ensure that the correct code path was called.

    Once you have a good amount of testing in place and users report errors, add new test cases for those errors to ensure they never happen again! Also, by rerunning your test cases after a bug fix, you can ensure that you have not broken other functionality during your fix. As another advantage, you have unlimited "Player" objects so you can even test what happens when multiple players are using the plugin.

    I know I know... write MORE code? Trust me... this extra code will save you a ton of time and headaches. No more, "Well I fixed the issue, I hope it didn't break anything else" compile/submitting. Some people even write their test cases before they get into the code (see Test Driven Development).

    This is not the solution for everything, but is a good tool to have access to.

    The link to the project header
    Anyway that's enough long windedness... here is the project:

    https://github.com/Pandarr/PluginTesting

    Libraries used
    PowerMock with Mockito
    JUnit 4
    Bukkit (duh)

    Test Passed!

    passed.png

    Test Failed! :(
    I changed the .startsWith() to look for "C" instead of "P". The first player was supposed to get .sendMessage() called but did not, so the test fails.

    failed.png
     
    HeroCC, xGhOsTkiLLeRx, Goblom and 4 others like this.
  2. Offline

    Kainzo

    Lovely.... I may have jumped the gun on this... I would love to see more ways to benchmark and test plugins before they hit mainstream.

    We've noticed that with Perms 3+ that the only time we find an error is under full load.

    If we can find a way to overload the plugin through unit tests, we'd have happier end users and devs.
     
  3. Offline

    Pandarr

    You might be able to start a bunch of Tasks that just spam call Permissions stuff. I would think that would be able to overload the plugin pretty well. 100 "plugins" calling Permissions 20 times a second....

    To be honest today was my first day using PowerMock. I'll see if I can find anything regarding load testing.

    You might try http://jakarta.apache.org/jmeter/

    I got it to load up the junit test cases but running them errored out. However I didn't have any of the Mock libraries or Bukkit in my Path so they wouldn't have worked anyway. But it looks like good unit tests in combination with JMeter may fit the bill. You'd write a test case to get the status of various users and their permissions and have JMeter thread that method to hell... :p

    http://jakarta.apache.org/jmeter/usermanual/junitsampler_tutorial.pdf

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 17, 2016
  4. Offline

    Pandarr

    For my next plugin, EduCraft, I'm going to attempt to fully unit test it. This should provide more examples of how to test various functionality. I'm sure the tests and code will evolve as I get further in and learn new ways to do things. But aside from being a plugin it can be used as a reference for getting unit test examples.

    https://github.com/Pandarr/EduCraft

    I am finding it interesting that thus far I haven't even compiled it (aside from auto-compile to run the unit tests of course) and ran it as a plugin yet, but I feel like the current functionality is sound. Granted there's not much functionality at the moment... but normally I'd be compiling and launching Minecraft like crazy. Now I spend that time writing test cases that I can rerun later when updates happen.
     
  5. This:
    Code:
        @[URL='http://forums.bukkit.org/members/test.36465/']test[/URL]
        public void testOnPlayerChatPlayerChatEvent() {
            HerobrineNPC NpcManager = mock(HerobrineNPC.class);
            Player p = mock(Player.class);
            PlayerChatEvent event = mock(PlayerChatEvent.class);
            when(event.getPlayer()).thenReturn(p);
            when(event.getMessage()).thenReturn("its him");
            when(NpcManager.checkHerobrine()).thenReturn(true);
            when(NpcManager.getPlayer()).thenReturn(p);
            HerobrinePlayerListener listener = new HerobrinePlayerListener(mock(Core.class), NpcManager);
            listener.onPlayerChat(event);
            verify(event).setCancelled(true);
        }
    Gives an error on the when(event.getPlayer()).thenReturn(p);
    something about private fields or something :(
    @Pandarr
     
  6. Offline

    Pandarr

  7. Alright thanks.
    The only other thing is that when i import:
    import org.powermock.modules.junit4.PowerMockRunner;

    it doesn't know what im talking about.
     
  8. Offline

    Pandarr

    Check out the pom.xml for the project.
     
  9. yea, fixed. thanks for the great guide!
     
  10. Offline

    SniperFodder

    This is awesome. I was wondering how I was going to do Unit Testing with my plugin and this fits the bill perfectly. I'm sure this will also be useful when writing other software as well.

    Going to try this when I get home from work today.
     
  11. Offline

    AmoebaMan

  12. Offline

    krisdestruction

    I know this thread is old, but does anyone know if this still works or not?
     
  13. Offline

    nverdo

    I tried it on 1.7.9.R02 but the event handling was changed drastically.

    See below for some changes you need to make:


    Class PluginTesting:
    Code:java
    1. import org.bukkit.event.Event.Priority;
    2. import org.bukkit.event.Event.Type;

    should be changed to:
    Code:java
    1. import org.bukkit.event.EventPriority;
    2. import org.bukkit.event.player.PlayerJoinEvent;

    and
    Code:java
    1. pluginManager.registerEvent(Type.PLAYER_JOIN, myPlayerListener, Priority.Monitor, this);

    should be changed to:
    Code:java
    1. pluginManager.registerEvent(PlayerJoinEvent.class, myPlayerListener, EventPriority.MONITOR, myPlayerListener, this);



    Class PTPlayerListener:
    Code:java
    1. import org.bukkit.event.player.PlayerListener;

    should be changed to:
    Code:java
    1. import org.bukkit.event.Listener;
    2. import org.bukkit.plugin.EventExecutor;

    and
    Code:java
    1. public class PTPlayerListener extends PlayerListener {
    2. public void onPlayerJoin(PlayerJoinEvent event) {
    3. Player player = event.getPlayer();

    should be changed to:
    Code:java
    1. public class PTPlayerListener implements Listener, EventExecutor {
    2. public void execute(final Listener listener, final Event event) {
    3. Player player = ((PlayerJoinEvent)event).getPlayer();



    Class PTPlayerListenerTest:
    Code:java
    1. myPlayerListener.onPlayerJoin(mockEvent);


    change the two occurrences to:

    Code:java
    1. myPlayerListener.execute(myPlayerListener, mockEvent);


    Finally if you need to update pom.xml to the latest dependencies:

    HTML:
    <dependencies>
    
    <dependency>

            <groupId>org.bukkit</groupId>
    
        <artifactId>bukkit</artifactId>
    
        <version>1.7.9-R0.2</version>
    
        <type>jar</type>
    
        <scope>provided</scope>
    
    </dependency>

        <dependency>

            <groupId>junit</groupId>
    
        <artifactId>junit</artifactId>
    
        <version>4.11</version>

            <type>jar</type>
    
        <scope>test</scope>

        </dependency>

        <dependency>

            <groupId>org.powermock</groupId>
    
        <artifactId>powermock-module-junit4</artifactId>
    
        <version>1.5.5</version>
    
        <type>jar</type>
    
        <scope>test</scope>
    
    </dependency>

        <dependency>

            <groupId>org.powermock</groupId>

            <artifactId>powermock-api-mockito</artifactId>
    
        <version>1.5.5</version>
    
        <type>jar</type>
    
        <scope>test</scope>
    
    </dependency>

    </dependencies>
    
    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 17, 2016
Thread Status:
Not open for further replies.

Share This Page