IdentityNow Account Filtering during Account Aggregation

IdentityNow Account Filtering during Account Aggregation

Overview


Often there is a need to filter accounts on a source during an account aggregation process.

While some source connectors offer an ability to filter accounts natively, other source connectors may not, usually due to technical limitations - such as a lack of filtering abilities in APIs the connector is calling.

Luckily, IdentityNow does have an ability to filter accounts on its side, as part of the aggregation process. This is configured by setting a filterString property on the source configuration. Once configured, the aggregation process matches accounts as they are aggregated against the filter string. Accounts which match the filter string will be filtered. Accounts which do not match, will be sent to IdentityNow as normal.

The trade-offs are summarized here:

Advantages

  • Ability to filter accounts during an account aggregation, despite connectivity.
  • Applies to all account aggregations and delta aggregations.

Disadvantages

  • This offers no aggregation performance improvements. This still forces an aggregation to run in its entriety, and filters the accounts as they are iterated back to IdentityNow.

    If performance improvements are desired, we recommend leveraging filtering on the connector, where applicable. Not all source connectors have filtering abilities, as described above.

  • Excessive filtering can produce aggregation timeouts. This is because IdentityNow (in the cloud) never receives any accounts (due to filtering) and keeps waiting for a response from the virtual appliance. The more accounts which are filtered, the greater chance for an aggregation timeout.

 

Note: For dedicated connectors, filterString will not work on the following attribute names unless they are marked as Account Name in the source.

  • displayName
  • identity
  • instance

We don’t recommend changing the default Account Name of any dedicated connectors as it may have some unforeseen issues.

Configuration

In order to configure this, you need to modify the source configurations to add in a filterString property. This can be done with a simple partial update to the source, using the REST APIs.

As an example, we may want to only bring in accounts which belong to employees. If employee was denoted by the "type" attribute on the account, we might configure our filterString to look like this:

Attribute Value
filterString ( type != "Employee" )

 

Different Filters for Different Objects - The filterString applies to all objects which are aggregated. If you want to get more specific to accounts or groups which are filtered, you can use account.filterString or group.filterString to denote specific filters for those particular objects. This document assumes filterString but you can also use the others as well.

 

Any account which matches this criteria will be filtered. The value you see is filter string syntax. The filter string reference guide follows this section. More complex values can be implemented, such as groupings, logical operators, etc.

The configuration of this is done via the IdentityNow Source Partial Update API. The details are as follows:

Request

PATCH /beta/sources/{id}

 

HTTP Headers

Key Value Description
authorization Bearer {token} This is the JWT OAuth token.
content-type application/json-patch+json This is needed for PATCH operations.

 

Path Parameters

  • id - The ID of the source. e.g. 2c9180835d191a86015d27ac132112ae

 

Query Parameters

  • (Not applicable)

 

Request Body

JSON Patch syntax representing the change:

content-type: application/json-patch+json

[
  {
    "op": "add",
    "path": "/connectorAttributes/filterString",
    "value": "( type != \"Employee\" )"
  }
]

Example

curl -X PATCH \
  https://example.api.identitynow.com/beta/sources/2c9180835d191a86015d27ac132112ae \
  -H 'Authorization: Bearer eyJ...BRM' \
  -H 'Content-Type: application/json-patch+json' \
  -H 'cache-control: no-cache' \
  -d '[
  {
	"op": "add",
	"path": "/connectorAttributes/filterString",
	"value": "( type != \"Employee\" )"
  }
]'

 

Response

 

Response Codes

HTTP Code HTTP Status Description
200 OK Returned if the request was successfully processed.
401 Unauthorized Returned if there is no authorization header, or if the JWT token is expired.
403 Forbidden Returned if the user you are running as, doesn't have access to this end-point.
429 Too Many Requests Returned in response to too many requests in a given period of time - rate limited. The Retry-After header in the response includes how long to wait before trying again.
500 Internal Server Error Returned if there is an unexpected error.

 

Response Body

content-type: application/json

A modified source object.

 

Filter Reference

This is a filter reference to show how various account filters might be constructed. The expressions genreally follow the form:

{property} {operation} {value}

 

Here are additional rules for parsing correctly:

  • String literals should have double-quotes.

    • e.g. firstname == "Neil"
  • True / false values are treated as boolean literals

    • e.g. inactive != false
  • Digits are treated as numbers

    • e.g. age < 18
  • The string value 'null' (no quotes) is treated as null

    • e.g. name != null
  • Everything else is assumed to be the property name

    • e.g. email == contactAddress

 

Composite Filters

Filters can also be grouped and used together as composite filters:

Composite Filter Pattern Example
Grouping ( {expression} ) !( type == "Employee" ) || (type == "Contractor" )
AND ( {expression} && {expression} ) ( type == "Employee" && location == "Austin" )
OR ( {expression} || {expression} ) ( type == "Employee" || type == "Contractor" )
NOT !( {expression} ) !( company == "SailPoint" )

 

Operations

Note: Any comparison operator can be prepended with an 'i' to signify a case-insensitive comparison (eg - i==, i!=, etc...).

Operation Pattern Example
Equals {property} == {value} firstname == "Neil"
Not Equals {property} != {value} lastname != "Smith"
Less Than {property} < {value} age < 18
Greater Than {property} > value age > 18
Less Than, Equals {property} <= {value} age <= 18
Greater Than, Equals {property} >= value age >= 18
Is Null {property}.isNull() email.isNull()
Not Null {property}.notNull() company.notNull()
Is Empty {property}.isEmpty() Groups.isEmpty()
Like, Exact {property} == {value} firstname == "Neil"
Like, Start {property}.startsWith( {value} ) lastname.startsWith( "Mc" )
Like, Start
(Ignoring Case)
{property}.startsWithIgnoreCase( {value} ) lastname.startsWithIgnoreCase( "Mc" )
Like, End {property}.endsWith( {value} ) email.endsWith( "@sailpoint.com" )
Like, End
(Ignoring Case)
{property}.endsWithIgnoreCase( {value} ) email.endsWithIgnoreCase( "@sailpoint.com" )
Like, Anywhere {property}.contains( {value} ) email.contains( "sail" )
Like, Anywhere (Ignoring Case) {property}.containsIgnoreCase( {value} ) email.containsIgnoreCase( "sail" )
Contains All {property}.containsAll({ {value}, {value}, {value}, ... }) Groups.containsAll( { "A", "B", "C" } )
Contains All
(Ignoring Case)
{property}.containsAllIgnoreCase({ {value}, {value}, {value}, ... }) Groups.containsAllIgnoreCase( { "A", "B", "C" } )

 

Labels (2)
Comments

Is there other ways to update and aggregate a single identity from a source or is account filtering the best approach? For example, if i wanted to update a delimited file connected source can I use the IdentityNow API to add or update a single identity instead of aggregating the entire source?

this filter also seems to work on entitlement aggregation, at least on the Azure AD source. Is that intended or should I create a ticket?

what if the attribute name has a space at middle? For instance, in AWS direct connect source, it has an attribute, named as "Access Keys". We would like to set up the filter so we aggregate only AWS users that have "Access Keys"

Tried to set filter like, 

Access Keys.isNull()

but it doesn't work. If we put "Access Keys".isNull(), then "Access Keys" is treated as string literal.

Does the attribute have to be part of the Account Schema for the Source?

Yes @chrisp , Based on my testing it looks like attribute must be part of account schema 

Trying to use this with Azure to only pull in hybrid accounts. This is the filter that was setup and Azure connector still pulling everything in. 
 
"value": "(dirSyncEnabled == true && userType == \"member\")"

Hello,

@neil_mcglennon   @piyush_khandelwal  @lisa_ivy 

To fully understood the flow with a custom openconnector:

1 - I create a filter in a Java "fashion" (looks like in Java)
2 - I update the Source Configuration of the openconnector
3 - When an aggregation starts, the system get the filterString attribute
 a - parse the content (if error what append ?)
 b - Convert Java like filter as pojo Filter object
 c - Launch the iterate method with this pojo (see iterate(Filter filter) in openconnector.Connector)
4 - When iterate method is launch, the connector parse and validate the pojo
5 - Transforms the Filter pojo to something that filter data

Is that correct ?

Or the system launch the Aggregation (the pojo Filter is null) and when iterate on the result apply the filter on it  ?

Best regards,

Stéphane POPOFF

Hello can anyone explain if it works correctly?

here some tests:

( lastname.startsWithIgnoreCase( "value1" ) || jobtitle.startsWithIgnoreCase( "value2" ) || lastname.startsWithIgnoreCase( "value3" ) || lastname.startsWithIgnoreCase( "value4" ) )  ---> no effect

( LastName != \"value1\" && !(lastname.startsWithIgnoreCase( \"value2\" )) ) --> no effect

( LastName != \"value1\" && lastname.startsWithIgnoreCase( \"value1\" ) ) --> no effect

( LastName != \"value1\" || !(lastname.startsWithIgnoreCase( \"value2\" )) ) --> no effect

(lastname.startsWithIgnoreCase( \"value1\" )) --> no effect, returns all

!(lastname.startsWithIgnoreCase( \"value1\" )) --> returns 0

( LastName != \"value1\" && LastName != \"value2\" ) --> return 2, the only that works

!(lastname.startsWith( \"value1\" )) --> return 0

(lastname.startsWith( \"value1\" )) --> no effect, returns all

In particular how to have the .startsWith functioning?

Thanks

Just a few observations from my attempts on using this to get accounts only for a specific manager and his team:

  • in case integer attributes type matters
    ManagerId=="123456" --> no effect
    ManagerId==123456 --> filters out the team, see next point
  • reminder that you define which accounts to EXCLUDE
    ManagerId==123456 --> does not work
    ManagerId!=123456 --> works
    (PersonId != 123456 || ManagerId!=123456) --> does not work, see previous point
  • with grouped composite filters you need all brackets
    !( PersonId==123456 || ManagerId==123456 ) --> does not work
    !((PersonId==123456 || ManagerId=123456 )) --> works
  • we did not run encounter any timeout issues when applying this filter to  get ~10 out of ~16000 accounts in our sandbox  

I hope this info might help others succeed :wink:

Hi, I'm back on this feature. How to query attributes with complex name such as "Password Deactivated"?

Thanks

Version history
Revision #:
10 of 10
Last update:
Sunday
Updated by: