Unit Test SailPoint Rule

[#1]

• Background
• Challenges
• About Sail4j
• Mockito
• Use Mock
• Use Spy
• Use Mockito.verify
• Use Custom object
• Conclusion

Background

Unit testing plays a very important role in any successful software development. If you strictly follow the Test Driven Development methodology, you should code unit test first before writing any component. The build or release will be successful only if all the unit tests pass. This is a good practice not just for the initial phase of development but also for the future enhancement. Test code will help prevent any defects introduced by the changes for the new features. The unit test code essentially becomes a valuable asset to relieve the burden of regression testing.

Another important aspect of unit testing is it is usually very quick and cheap to run. Unlike integration test, unit test is supposed to be lightweight, not dependent on downstream systems (such as database), and repeatable as it will not have any side effect on the system.

Now you may ask this question. Is it easy to write unit test for IdentityIQ or IdentityNow? The answer is probably not. For components like Workflow, Application, Bundle or Task, it may not be possible to write pure unit test code (yes you can write integration tests which are executed in a real IdentityIQ orIdentityNow environment by using some testing tools/frameworks developed by SailPoint community). But for Rules, there are ways to write unit test code.

Challenges

One of big challenges that stops us from writing any meaningful unit test is that Rule is written in Beanshell programing language. Beanshell, as a scripting language, gives you a lot of flexibilities to update logic in runtime without requiring application rebuilding or redeployment. But its downside is it doesn't provide much support for unit testing. Another challenge is Rule needs an environment (IdentityIQ or IdentityNow) to run. So without a simulator, the only way to test a Rule is to run the Rule in a full-fledged IdentityIQ installation or IdentityNow tenant. Based on the previous discussion, this way of testing doesn't qualify as true unit testing. So now how to address these 2 main challenges?

About Sail4j

To address the first challenge, we need to look from the Rule manufacturing point of view. How do we create the Rule? The Beanshell code of Rule is essentially a snippet of Java code. In fact, many of us tend to write the body of Rule in a separate Java project with the help of IDE (Integrated Development Environment) software like Eclipse or IntelliJ,  and then copy that part of code to an XML file with proper wrapper to save it as a legitimate Rule file. Sail4j is a tool to do the similar thing but in a automated fashion, and seamlessly integrated with the building process commonly used in an IdentityIQ or IdentityNow project. For example, if we have an IdentityIQ project using SSB (Standard Service Build), then a Custom Apache Ant Task from Sail4j can be configured in the build.xml to convert a Java class to Rule XML. Similarly, if you happen to use Maven in an IdentityNow or IdentityNow project, the Maven Plugin from Sail4j can be used. Now, if you take a step back and look at this from a software development perspective, the Java class becomes your source code and Beanshell Rule XML is the output of the build. This way of development switch provides 2 obvious big benefits:

• You pretty much eliminate all the programing syntax error. Remember all the simple and annoying errors (such as these errors because we forget to import a java class in the head) we encountered as a Rule developer in the earlier days. This will never happen again in runtime because Eclipse simply won't allow your Java class to compile and therefore build will fail.                                
•  It opens the door to unit testing This is huge because it means you can write proper unit testing code using popular frameworks like JUnit or TestNG. And the Rule will be throughly tested before being committed to a code repository and deployed to an installation. Just think about this in an IdentityNow project, this practice can save time and effort of many people involved (Rule developer, reviewer and deployer). If a Rule passes the unit tests you've diligently written, it is very unlikely that it will not pass the Rule Validator.   

Refer to the following page for more details about how to use and setup Sail4j in an IdentityIQ or IdentityNow project.

https://github.com/renliangfeng/sail4j-iiq-idn

Mockito

Before touching the second challenge mentioned earlier,  I would like to talk a bit about unit testing. One key principle of unit testing is it should ONLY test the component that needs to be tested. Taking a 2 layers application as an example, you have a service layer component and a data access layer component where service layer calls the functions/methods from data access layer. When you write unit test for the service, you should assume everything in data access layer functions exactly as it is supposed to behave. If you try to code to connect data access component to the Database to read to write data in you test case, then you are effectively testing both service and data access components which theoretically categorize it an integration test rather than unit test. So how can you write a unit test without having to write code to get the dependent components and downstream systems running? The key technique is to mock the downstream layer. In this example, you can simulate the methods/functions of data access component in you unit test code and trick your service to execute the scenarios you would like to test.

We can apply the same techniques to an IdentityIQ or IdentityNow project. Here we want to unit test the Rule and the Rule's downstream is IdentityIQ or IdentityNow runtime environment. If you take a close look at most of Rules you have implemented, the commonly used SailPoint objects are SailPointContext, IdentityService etc. So if we can find a way to mock these SailPoint objects, then we should be able to write unit tests. When it comes to Java API mocking, one of my favorite frameworks is Mockito. It provides a rich set of features and user-friendly APIs for mocking to suit different testing scenarios and works very well with JUnit testing framework. In the following sections, I will break down into multiple use cases where you can use different features of Mockito to mock the SailPoint object to test your Rules.

Use Mock

One of the main features of Mockito is to mock a Java interface. Suppose you want to test a Java class which calls a method from a Java interface, and the Java class has an object reference with type of that Java interface. In your JUnit test case you can mark that object reference with Mockito Mock annotation, then Mockito will automatically instantiate a mocking stub object of that interface. Then you can use Mockito to setup the expected behaviors of the stub object to suit your different testing scenarios.  One of the most commonly used SailPoint interface in Rule is SailPointContext.  Below is a simple example of mocking the method getObjectByName of SailPointContext in a JUnit test case using org.mockito.Mockito.when method. The Rule we will test here is called MyRule which calculates the value of Display Name based on firstName, lastName and middleName.

.

  import static org.mockito.Mockito.doReturn;
  import static org.mockito.Mockito.when;
  import org.mockito.Mock;
  import org.mockito.junit.MockitoJUnit;
  import org.mockito.junit.MockitoRule;
...
  @Mock
  SailPointContext context;

  @Rule 
  public MockitoRule mockitoRule = MockitoJUnit.rule();

  @Test
  public void testDisplayName() throws Exception {
    Identity identity = new Identity();
    identity.setFirstname("Jane");
    identity.setLastname("Doe");
    when (context.getObjectByName(Identity.class, "JaneDoe")).thenReturn(identity);

    MyRule myRule = new MyRule();
    myRule.context = this.context;
    assertEquals("Doe, Jane",myRule.getDisplayName(context,identity));
  }

In another more complicated case, you may have a Rule (Identity Attribute) to generate a unique username. To make the username unique, the Rule calls the search method of SailPointContext to validate if the potential username is already used.  To simulate the searching result for the testing scenarios that username is already used, the following snippet of code will achieve the goal. With small modification, you can easily mock other scenarios that you would like to cover.

import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.when;
...
QueryOptions queryOptions = new QueryOptions();
queryOptions.addFilter(Filter.eq("application.name", applicationName));	
queryOptions.addFilter(Filter.ignoreCase(Filter.eq("searchableUsername", userName)));	
List<Object[]> lst = new ArrayList<Object[]>();
if (!expectEmpty) {
	String nativeIdentity = "test";
	Attributes attributes = new Attributes();
	attributes.put("id", "dummyid001");
			
	Object[] array = new Object[2];
	array[0] = nativeIdentity;
	array[1] = attributes;
		lst.add(array);
}
when(context.search(Link.class, queryOptions, "id")).thenReturn(lst.iterator());

Use Spy

Mock function of Mockito is relatively easy to understand and use. However Mock function only works for Java Interface (such as sailpoint.api.SailPointContext in the previous examples). If the Rule uses Java class like sailpoint.api.IdentityService, then you can't use Mockito's Mock function. In general, this type of Rule (or the method of Rule library) is not testable (at least from unit testing perspective).  That being said, with a bit of tweak, the Rule can become testable largely with the help of another Mockito feature called Spy. The technique I normally use is to extract that section of code (untestable due to use of unmockable Java class of SailPoint APIs) into a separate method. Then in JUnit test, you spy the Rule object itself with Mockito Spy annotation. Spy essentially allows you to mock the methods (which are untestable) of the Java class that you are testing so that you can still test the remaining parts. In this example, you mock the behavior of new method with code of calling IdentityService (this part of code is not testable and we just have to accept that), but you can still test the other parts of Rule.  Now let's use an example to demonstrate how to use Spy in JUnit test. Suppose you have a Rule that is used to get email address in a writeback process (to a target system after provision). The email fetching has following logic:

• if email Identity attribute exists, use this value;
• otherwise use the mail attribute of primary Active Directory account (user can have multiple AD accounts, but onlyprimary one).

The Rule is developed in a Java class as shown below:

@SailPointRule(name = "Rule-Get-Generated-Email", fileName = "Rule-Get-Generated-Email.xml")
public class RuleGetGeneratedEmail {
	private static Log ruleLog = LogFactory.getLog("rule.GetGeneratedEmail");
	
	@IgnoredBySailPointRule
	SailPointContext context;
	
	List<Link> getLinks(Identity identity, Application app) throws GeneralException {
		IdentityService idService = new IdentityService(context);
		List<Link> adLinks = idService.getLinks(identity, app);
		return adLinks;
	}

	public String fetchEmail(Identity identity) throws GeneralException {
		String emailFromIdentity = identity.getEmail();
		if (StringUtils.isNotBlank(emailFromIdentity)) {
			ruleLog.debug("Get email from Identity email attribute: " + emailFromIdentity);
			return emailFromIdentity;
		} else {
			String primaryAdAppName = (String)identity.getAttribute("primaryAdApplication");
			Application primaryAdApplication = null;
			if (StringUtils.isNotBlank(primaryAdAppName)) {
				primaryAdApplication = context.getObjectByName(Application.class, primaryAdAppName);
			}
			if (primaryAdApplication != null) {
				List<Link> adLinks = getLinks(identity, primaryAdApplication);
				if (adLinks != null && adLinks.size() == 1) {
					Link primaryAdAccount = adLinks.get(0);
					String emailFromAD = (String)primaryAdAccount.getAttribute("mail");
					if (StringUtils.isNotBlank(emailFromAD)) {
						ruleLog.debug("Get email from primary AD mail attribute: " + emailFromAD);
						return emailFromAD;
					}
				}
			}
		}
		return "";
	}
}

As you can see, the first method getLinks is untestable as it uses a SailPoint class IdentityService which is un-mockable (note: this is based on my knowledge of Mockito, maybe it's possible by using other mocking frameworks).  So now in my JUnit test case, I will use Spy annotation to spy the whole Rule which allows me to simulate the method getLinks using method org.mockito.Mockito.doReturn.

import static org.mockito.Mockito.doReturn;
import static org.mockito.Mockito.when;
import org.mockito.Mock;
import org.mockito.Spy;
import org.mockito.junit.MockitoJUnit;
import org.mockito.junit.MockitoRule;
...
public class RuleGetGeneratedEmailTest {
	
	@Mock
	SailPointContext context;
	
	@Rule 
	public MockitoRule mockitoRule = MockitoJUnit.rule();
	
	@Spy
	RuleGetGeneratedEmail rule = new RuleGetGeneratedEmail();
	Identity identity;
	Application adApp = new Application();
	List<Link> adLinks = new ArrayList<Link>();
	
	@Before
	public void setUp() throws Exception {
		rule.context = context;
		identity = new Identity();
		identity.setFirstname("Test");
		identity.setLastname("Smith");
		identity.setName("Test.Smith");
		adApp.setName("Active Directory");
	}

    @Test
	public void testEmailFromAD() throws Exception {
		when (context.getObjectByName(Application.class, "Active Directory")).thenReturn(adApp);

		identity.setAttribute("primaryAdApplication", "Active Directory");
		Link adLink = new Link();
		adLink.setAttribute("mail", "abc.efg@gmail.com");
		adLinks.add(adLink);

		doReturn(adLinks).when(rule).getLinks(identity, adApp);

		String fetchedEmail = rule.fetchEmail(identity);
		assertEquals("abc.efg@gmail.com", fetchedEmail);
	}

Use Mockito.verify

The method Mockito.verify is another feature commonly used to validate the logic in Unit test cases. It allows you to validate expected arguments are passed when the method of a mocked or spied object is called. You can also specify exactly how many times the method is expected to be called. This can be useful to test the provisioning related Rules such as Before Provision Rule or After Provision Rule. For example, you have a After Provision Rule which writes back some attributes to a HR application once Active Directory account is provisioned successfully.

@SailPointRule(name = "Rule-Writeback", fileName = "Rule-Writeback.xml", 
	type = RuleType.AFTER_PROVISIONING, referencedRules = {"Rule-Library-Writeback"})
public class RuleWriteback {
    @IgnoredBySailPointRule
	SailPointContext context;

	@IgnoredBySailPointRule
	public String writeback(String empId, Map attributes) {
		return null;
	}

    @SailPointRuleMainBody
	public void execute(ProvisioningPlan plan, Application application, ProvisioningResult result) 
        throws Exception {

	    //complex code is ommitted here; our main goal of unit testing is to test the code included here.	

        writeback(employeeID, attributes);
    }
}

In this example, the writeback method is inside a referenced Rule library and will not be tested in this Unit test (and it is not testable anyway). But at least we can validate if the arguments passed to this method are correct by calling Mockito.verify with the expected values as shown in the JUnit test code snippet below:

@Spy
RuleWriteback rule = new RuleWriteback();

@Test
public void testSuccessJoiner() throws Exception {
		
	sailpoint.object.ProvisioningPlan plan = init(sailpoint.object.ProvisioningPlan.AccountRequest.Operation.Create);
		 
	addAttribute("mail", "test.smith@gmail.com");
	addAttribute("employeeID", "4000871");
	addAttribute("sAMAccountName", "P4000871");
		
	rule.execute(plan, adApplication, null);
		
	Map map = new HashMap();
	map.put("PrimaryEmail", "test.smith@gmail.com");
	map.put("LogonUserName", "P4000871");
		
	Mockito.verify(rule).writeback("4000871", map);
}

To build some sophisticated test cases, you can even use Mockito ArgumentCaptor to capture arguments passed to a method that is mocked or spied. This allows you to examine the arguments to validate real values against expected values. Refer to the Mockito document for more details about how to use this function.

Use Custom object

The sailpoint.object.Custom object is quite often used in a Rule. In JUnit test case for the Rule, you can write code to populate the Custom object to reflect what's defined in the Custom XML, this will work just fine. However there is a better way to do this. You can use the method populateCustomFromFile from the following test helper class provided by Sail4j to populate the values from the real Custom XML file:

com.sailpoint.sail4j.test.Sail4jUnitTestHelper

Below example shows how to use this helper class to populate a SailPoint Custom object in JUnit before test method. The benefit of this not only reduces the lines code in your test case, but also allows you to test the Custom object itself. On top of that, when there are changes in the Custom object in the future, you don't need to update JUnit tests to reflect the changes.

import sailpoint.api.SailPointContext;
import sailpoint.object.Custom;
import com.sailpoint.sail4j.test.Sail4jUnitTestHelper;
..

@Mock
SailPointContext context;

@Rule 
public MockitoRule mockitoRule = MockitoJUnit.rule();

MyRule rule = new MyRule();

@Before
public void setUp() throws Exception {
	Custom buCustom = Sail4jUnitTestHelper.populateCustomFromFile("config/Custom/Custom-BusinessUnit.xml");
					
	rule.context = context;
	when (context.getObject(Custom.class, "Custom-BusinessUnit")).thenReturn(buCustom);
}

The test helper class Sail4jUnitTestHelper is included in the Sail4j test helper module. To use it, you need to include the following jar file in your project. Refer to the link provided in the previous section for details about how to setup and use Sail4j in your project.

sail4j-test-helper-1.1.jar

Conclusion

The goal of unit testing in an IdentityIQ or IdentityNow project is never going to have the 100 or 90 percentage testing coverage due to the fact that we can't mock all SailPoint APIs used in the Rules plus other constraints. But as you have discovered in the previous sections, with a bit of creativity, you can write code to test the majority of logic. Some of these tweaks may seem to be inconvenient, but in my view it is usually align with the best coding practice in general. It makes code more testable, manageable and readable. As a true believer of TDD (Test Driven Development) methodology, I always think having unit testing is alway better than no unit testing.

Another thing I want to point out is Mockito and JUnit are just the tools that I personally like and am more familiar with. There are many other excellent frameworks in the market you can consider which may better suit your needs or styles. Once you start writing Rule in plain Java code, the door is open and the world is your oyster. There are a lot of open-source solutions for Java unit testing you can explore. For example, in my old days as a developer before discovering Mockito, I have used EasyMock which was pretty good as well. As an alternative unit testing framework to JUnit, TestNG is probably worth checking out as well.

[developerthang]

Hi,

Thanks for the article.

However, whenever I try to mock SailPointContext and try to run the test, I received this error:

Caused by: java.lang.IllegalStateException: Unable to determine branding
at sailpoint.tools.BrandingServiceFactory.getBrand(BrandingServiceFactory.java:50)
at sailpoint.tools.BrandingServiceFactory.initializeBrandingService(BrandingServiceFactory.java:21)
at sailpoint.tools.BrandingServiceFactory.getService(BrandingServiceFactory.java:14)
at sailpoint.object.Identity.<clinit>(Identity.java:137)
... 4 more

Any idea why and how I can work around it? @bruce_ren 

Thanks

[bruce_ren]

Hi @developerthang ,

You need to add a dummy iiq.properties (even empty one is fine) to your classpath. For a maven project, it should be like in the screenshot below:

[developerthang]

Thanks for getting back to me @bruce_ren 

I used the SailPoint SSB and directly added my JUnit test classes to the src/ folder inside the default SSB: https://community.sailpoint.com/t5/Professional-Services/Services-Standard-Deployment-SSD-v7-0-2/ta-... 

I also run JUnit tests directly through Eclipse.

I already tried adding the iiq.properties file inside the SSB folder, but the "Unable to determine branding" error still exists.

Do you have any other suggestions? 

[developerthang]

Please ignore my comment above. I got that part to work now. Just needed to put the dummy iiq.properties at the right place in the project.

Now I just need to figure out how to mock the SailPointFactory.getCurrentContext() part in the method that I'm testing

[ajeet_bunny]

<?xml version='1.0' encoding='UTF-8'?>
<!DOCTYPE Rule PUBLIC "sailpoint.dtd" "sailpoint.dtd">
<Rule created="" id="" language="beanshell" modified="" name="AddRemoveEntitlmentRule" type="FieldValue">
<Description>This rule can be used to generate a field value (eg - an account name) using data from the given Identity. If this rule is run in the context of a workflow step then the arguments passed into the step will also be available. Also, any field values that have been processed so far from the policy related to the Application/Role will be available.</Description>
<Signature returnType="String">
<Inputs>
<Argument name="log" type="org.apache.commons.logging.Log">
<Description>
The log object associated with the SailPointContext.
</Description>
</Argument>
<Argument name="context" type="sailpoint.api.SailPointContext">
<Description>
A sailpoint.api.SailPointContext object that can be used to query the database if necessary.
</Description>
</Argument>
<Argument name="identity" type="Identity">
<Description>
The Identity object that represents the user needing the field value.
</Description>
</Argument>
<Argument name="link" type="Link">
<Description>
The sailpoint.object.Link that is being acted upon. If the link is not applicable,
this value will be null.
</Description>
</Argument>
<Argument name="group" type="AccountGroupDTO">
<Description>
The sailpoint.web.group.AccountGroupDTO that is being acted upon. If the AccountGroupDTO
is not applicable, the value will be null.
</Description>
</Argument>
<Argument name="project" type="ProvisioningProject">
<Description>
The provisioning project being acted upon. If a provisioning project is not applicable,
the value will be null.
</Description>
</Argument>
<Argument name="accountRequest" type="ProvisioningPlan.AccountRequest">
<Description>
The account request. If an account request is not applicable, the value will be null.
</Description>
</Argument>
<Argument name="objectRequest" type="ProvisioningPlan.ObjectRequest">
<Description>
The object request. If an object request is not applicable, the value will be null.
</Description>
</Argument>
<Argument name="role" type="Bundle">
<Description>
The role with the template we are compiling. If the role is
not applicable, the value will be null.
</Description>
</Argument>
<Argument name="application" type="Application">
<Description>
The sailpont.object.Application with the template we are compiling. If the application
is not applicable, the value will be null.
</Description>
</Argument>
<Argument name="template" type="Template">
<Description>
The Template that contains this field.
</Description>
</Argument>
<Argument name="field" type="Field">
<Description>
The current field being computed.
</Description>
</Argument>
<Argument name="current" type="Object">
<Description>
The current value corresponding to the identity or account attribute that the field represents.
If no current value is set, this value will be null.
</Description>
</Argument>
<Argument name="operation" type="ProvisioningPlan.Operation">
<Description>
The operation being performed.
</Description>
</Argument>
</Inputs>
<Returns>
<Argument name="value">
<Description>
The string value created.
</Description>
</Argument>
</Returns>
</Signature>
<Source>import sailpoint.tools.GeneralException;
import sailpoint.tools.Util;

boolean value=true;
if(operation!=null)
{
if( Util.nullSafeCaseInsensitiveEq(operation, "Add and Remove Entitlement"))
{
value=false;
}
else
{
value=true;
}
}
return value;</Source>
</Rule>

[LGD]

Hello @bruce_ren,

Thank you very much for this very exciting tool and valuable documentation!

However, as it seems to be closed-source and not supported by SailPoint or the PS, I’m a little reluctant to adopt it if there's no way of making it evolve in the future in the event of new rules in future IIQ versions or bugs in Sail4j.

Are there any plans to make it open-source?

Best regards,

Louis-Guillaume Dubois

[bruce_ren]

Hi @LGD , right now we don't have plan to make it open source. However we do release new version of Sail4j to support new rules introduced in new version of IIQ and fix bugs.

Bruce

[VasanthRam]

Hello @bruce_ren  this documentation was very much helpful for me to understand mockito.This is something very much similar to Rule Development Kit in Sailpoint Identity Now right..? It is providing me with all the configurations in a Github repo and with all the rule templates with xml wrapper.
Am i right or how does this differ from Rule Development Kit? 
And can you say me some pros and cons of it?

[VasanthRam]

Hello @bruce_ren  this documentation was very much helpful for me to understand mockito. Thanks for the detailed explanation. I have been working on IDN for the past few months..This is something very much similar to Rule Development Kit in Sailpoint Identity Now right..? It is providing me with all the configurations in a Github repo and with all the rule templates with xml wrapper.
Am i right or how does this differ from Rule Development Kit? 
And can you say me some pros and cons of it?

[sgattani]

Hello,
I have a rule which uses tokenization.

String inputFilePath =" %%APP_INPUT_FILE_LOCATION%%";

After compilation, tokens are replaced. So when junit runs after compilation it is not able to override the variable. How to override inputFilePath at runtime?

I have tried writing a getter getInputFile and mocking using Mockito. 
(fileInput is file path that I want to override)

Mockito.doReturn(fileInput).when(console.runRuleMethod(rule, "getInputFile", args1));

console.runRule(rule, args);

Rule:
String inputFilePath = getInputFile();