Knowledge Article

SailPoint SaaS Change Management and Deployment Best Practices

Author

  • neil_mcglennon

    SailPoint

Overview

This document describes SailPoint’s recommended best practices for managing configuration changes across customer tenants. It includes a description of the SailPoint SaaS release process, the purpose of various SailPoint tenant environments, as well as recommended promotion processes for configurations.

Ultimately SailPoint recommends that all organizations follow a controlled configuration, test, and deployment process, though the specific details of that process are left to each organization to define in accordance with their own business practices.

SailPoint SaaS Releases

When discussing how to manage changes across tenants, it is important to understand of how SailPoint releases SaaS components so you can be prepared for what they contain, under what cadence they are released, how they are communicated, and how they may impact your business.

In general, SailPoint’s SaaS releases fall under four general categories:

  1. Feature Releases
  2. Maintenance Fixes
  3. Security Improvements
  4. Architectural Changes

Releases can contain features which are actual net-new capabilities, or enhancements to existing capabilities. These functionalities can be impactful - as is the case with user interface or behavioral changes; they can also be less impactful - such as additions to REST APIs, event triggers, new connectors, AI improvements, or even micro-service functionality changes.

Releases can also contain maintenance fixes, which are updates to existing feature capabilities to make them more resilient, and potentially fix or address particular issues that have been reported by the user community. These releases are typically less impactful, and welcomed to help ensure that the SailPoint SaaS platform remains a quality, stable service that customers enjoy using.

Releases can contain security improvements. SailPoint takes security of the SailPoint SaaS platform very seriously; and is constantly making improvements to ensure that the platform is resilient to attack and hardened with multiple layers of security. A lot of these security improvements come from internal SailPoint guidance, as well as results from regular independent penetration tests done on the platform. These improvements can be released at any time from standard security patches for technologies we use, or from SailPoint engineering itself for code that was developed in-house. These releases are transparent to customers, and are a welcome part of what makes the SailPoint service trusted by its customers.

Releases can also contain architectural changes. These are changes to how the SaaS product operates, scales, communicates, or even where and how it stores data. These releases ensure the platform runs optimally and can scale to meet its intended performance envelopes. These releases are usually not very transparent to customers, as they are intended to deploy with little to no visible impact on end users.

Release Cadence, Rollout, and Feature Flagging

SailPoint SaaS releases are deployed at a high rate. At current writing, our current average is sixty (60) releases a week, spanning a multitude of SailPoint micro-services that underpin the Identity Security Cloud platform. While releases occur frequently, most of these are transparent to anyone using the product; a smaller percentage of changes are communicated and announced ahead of time as they would impact the end user experience.

Further Reading: Another great resource on this topic was published to the SailPoint Tech Blog entitled “Our Path to 60 Releases Per Week.”


When feature releases are deployed, they are not always enabled for live usage by all tenants at the same time. SailPoint uses a widely-adopted SaaS concept called feature flagging to identify which tenants should be using which features. This allows SailPoint to control rollout of certain features in a dynamic way. This has many advantages over existing static, “all or nothing” deployment strategies.

SailPoint can roll changes out incrementally for different populations of tenants based on need.This could mean that changes are rolled out to sandbox first and then enabled in production later. Most feature releases will roll out to sandbox first, and then production second. The amount of time between tenant releases typically depends on how impactful the release is, with a targeted cadence of two weeks between sandbox and production deploys.

Via feature flagging, SailPoint has more control over how features are used and adopted. Feature flags let customers opt-in to Early Access (EA) programs if they are interested, and preview features ahead of when a feature might be Generally Available (GA) to all customer tenants. This allows SailPoint to collect feedback on new features, possibly improve the features further before a wider release, and hone communication and documentation information for widespread rollout.

Feature flagging also allows SailPoint to roll out new code for emergency fixes, if needed, or even rollback to previous versions of micro-services if a particular code deploy does not work as expected. Taken in with continuous monitoring and alerting, this allows for a more resilient platform with less negatively impactful changes. SailPoint Engineering and DevOps personnel, along with SailPoint Change Approval Board (CAB) guidance, are integral to the release planning and execution process.

Further Reading: Another great resource on this topic was published to the SailPoint Tech Blog entitled “CI/CD Superhero: Feature Flags.”

Impact and Communication

SailPoint’s mission is to provide best-in-class features within a robust and stable SaaS service. To do that, communication of new features is critical.

All features which are queued for release are announced and communicated in several ways.

The SailPoint SaaS feature roadmap is publicly visible at SailPoint SaaS Product Release Updates and can be tracked through there.

Releases are also announced and communicated on the SailPoint Compass Community which includes rollout, timing, and impacts. This community not only serves as a place for announcements, but also serves as a place for roadmap visibility, customer discussion, and relevant product documentation and best practices.

Another way releases are announced and communicated is through our in-service messaging, which is visible and located in the lower right corner of the screen. This announces new features, and even offers walk-throughs and assistance in the service spotlighting changes once features have been released.

Lastly, SailPoint SaaS status and health can easily be monitored at status.sailpoint.com which includes uptime metrics and outage information.

SailPoint Tenants and Environments

A separated customer tenant ecosystem, with controls in place for testing and migrating changes between environments, is essential to preventing avoidable outages and minimizing the introduction of misconfigurations in production environments. SailPoint provides two distinct tenant environments for every customer by default:

  1. A sandbox tenant
  2. A production tenant

Sandbox Tenants

The sandbox or test tenant is an environment used for initial implementation and configuration of the SailPoint service, and serves as a place to test artifact configurations before live usage in a production deployment. Since this environment may act as a precursor to a production tenant, but not actual live usage, it is important to understand this environment is not typically given the same level of resources that are required for production operation.

The purpose of this environment is to test configurations before production. It allows the implementer or administrator to set configurations that will be used in the implementation; essentially, all core configuration should be initially done or represented in the sandbox tenant. Implementers or administrators can test these items without affecting production behavior. Once testing has been satisfactorily passed, these same configurations can be promoted to the production environment in a controlled manner. It is also important to note that sandbox environments are always left in situ - as they are - and are never refreshed from any other tenant or periodically restored to a "bare metal" state like sandbox environments from other SaaS vendors.

Typically, a sandbox tenant only contains a subset of identity and account data, since the implementer only needs the data relevant to the functional tasks they are implementing. Using full, large-scale production data sets are discouraged, because some data processes, such as aggregations and identity refreshes, may take longer, or may impede configuration or testing cycles, which are better executed against smaller data sets for iterative configuration and testing. SailPoint recommends a set of 1,000 identities in this environment. Requests for larger sets of data might be accommodated, with SailPoint approval, assuming the multi-tenant space (pod) has room for that number of identities. A limit of 5,000 identities for a sandbox tenant is strongly encouraged but not enforced.

From a source and data quality perspective, the sandbox tenant data and sources should be as close to a copy of production as possible, within technical and budgetary constraints. Ultimately, final testing for the entire solution is validated in this environment prior to production rollout, so the more similar the environments are, the more valuable the test results will be and the less risk there is in migration to production. Poorly planned and implemented sandbox tenants are usually the root cause of failed production deployments. Processing on the sandbox tenant will be less performant than a production tenant as it does not have the same resources. That said, sandbox tenants should work smoothly, just not as quickly when processing bulk changes.

Many configuration objects created in this environment are often exported from the sandbox tenant and migrated to the subsequent production tenant. Artifacts may optionally be checked in via a version control system, or part of a larger CI/CD process if so desired. This is described in more detail later in this document.

NOTE: As required by any standard software release process, testing of even small, incremental changes should be done in this environment before promoting those changes to production.

Production Tenants

The production tenant is the environment which is designed for actual live usage by end users, ongoing operations, and actual integrations with other target source systems. It is the ultimate end-goal for various business, IT, and security processes.

Because of this pattern of usage, this is the place with the most sensitive information and must be managed with the strictest quality and change controls. Ideally, great care should be taken for any changes to this environment, and everything should be tested and validated appropriately before putting those changes into live production usage.

Additional Tenants

Some organizations may want or require additional tenants for specific purposes. Some examples include additional testing environments, performance testing environments, or even pre-production tenants which mirror production configuration. Additional tenants can certainly be accommodated with additional subscription cost, subject to the nature of the environment, and licensed according to the SailPoint Software-as-a-Service (SaaS) Subscription Agreement. If you need additional tenants to meet your needs, contact your SailPoint Customer Success Manager (CSM) for details.

Recommended Promotion Process

Due to the multi-tenant software-as-a-service nature of the Identity Security Cloud platform, many factors of releasing, updating, security, and running the platform are handled for you.

Beyond the service itself, are the unique combination of configurations and integrations which make the SailPoint SaaS solution work for your organization. These are features uniquely tailored to meet your needs as a customer, and SailPoint recommends following a controlled configuration, test, and deployment process to manage these.

There are two important considerations in managing configurations.

  1. Which configurations to manage
  2. How to manage configurations between tenants

This section discusses how and which configuration to manage and how to integrate these with a cross-tenant promotion process.

Managing Configurations

When managing configurations, it is important to understand which items should move from one tenant to another. These items are the various objects that work together to manage customer data and support business practices.

Not all objects in the SailPoint SaaS solution are created equal. Some objects relate to setup and configuration of a tenant, and others may be a byproduct of usage of certain features or processes in the SailPoint SaaS Solution. These generally break down into three categories:

  1. Always-Managed Objects
  2. Conditionally-Managed Objects
  3. Non-Managed Objects

Always-Managed Objects

This category of objects are things that pertain to configuration and setup of the SailPoint SaaS setup; and are typically something that SailPoint recommends managing via some sort of change control process between tenants. Below are common configurations which we recommend managing along with a brief explanation of what they are:

  • Sources - These define the technical details of how to connect to a target system using a SailPoint connector. There may be some minor differences in these configurations across tenants which commonly pertain to connectivity or security information - e.g., the sandbox Active Directory source communicates with a test domain, while the production Active Directory source communicates with a production one.
  • Source Schemas - These are configurations which control what account and access data to collect during data import. These generally match between sandbox and production, and the only reason they would differ is if data is somewhat different between tenants (e.g., account structures are different).
  • Source Correlations - These are configurations associated with sources which control how accounts are linked to their owning identities. These generally match between sandbox and production, and the only reason they would differ is if data is somewhat different between tenants (e.g., account data is different).
  • Source Account Profiles - These are configurations which determine provisioning policy around how an account is provisioned - most commonly during account creation.
  • Source Attribute Sync Configuration - These are configurations which determine which identity attributes should be synced to account attributes.
  • Workflows - These are configurations which define a sequence of steps for governance related processes.
  • Identity Profiles - These are configurations which determine the mapping of identity attributes for a set of identities associated with the identity profile’s authoritative source.
  • Transforms - Configurations which allow for transformation or determination of account or identity attribute data in a “no-code” manner.
  • Rules - Configurations which allow for calculations in the cloud, or connector-level operations using custom code.
  • Custom Connectors - Developed and packaged connector artifacts which are uploaded to a tenant for execution in the virtual appliance.
  • Service Desk Integration Modules (SDIM) - Configurations which map sources to a ticketing system, for manual service-desk fulfillment.
  • Password Policies - Configurations which define sources' password policies for password resets, synchronization, or expirations.
  • Tenant Configurations - Configurations which control system-level settings of the tenant.

Conditionally-Managed Objects

This category of objects falls into a bit of a “grey area,” where some customers choose to migrate them across tenants while others prefer to recreate them independently in each environment. The decision on whether or not to migrate these generally falls to the customer’s preference and their business and change control processes.

  • Roles - These are configurations which determine assignment of access to identities. While role assignment criteria generally ports across tenants, oftentimes access profiles and entitlements may be somewhat different between sandbox and production source systems. For this reason, these objects are generally not managed between tenants.
  • Access Profiles - These are configurations which determine a bundle of entitlements. Access profiles are based on one or more entitlements, which may be somewhat different between sandbox and production source systems. For this reason, these objects are generally not managed between tenants.
  • Separation of Duties Policies - These are configurations which define risky or disallowed combinations of access and are generally data dependent - especially around access models, which may be somewhat different between sandbox and production source systems. For this reason, these objects are generally not managed between tenants.
  • Event Trigger Subscriptions - These are configurations which define a subscribed target to send event triggers to. While there is no reason these can’t be migrated between tenants, they generally aren’t managed as the event trigger subscriptions tend to be environmentally specific.
  • Branding - These are configurations which define the colors and logos with which to brand the SailPoint SaaS user interface. Each customer handles branding slightly differently; some customers share similar branding across tenants, while others differentiate their sandbox and production source systems via very different branding schemes (e.g., production is red; sandbox is blue).
  • Email Templates - Configuration templates which define the email messages that the SailPoint SaaS services generate in email-based communications. Each customer handles email templates slightly differently; some customers share similar email templates across production and non-production environments, while others differentiate their sandbox and production tenants' email communications schemes (e.g., production has a different email footer than sandbox).

Non-Managed Objects

This category contains objects that are not designed for import and export across tenant. Most are data objects created during the normal course of product usage (e.g., audit events), data import tasks (e.g., identities, accounts, or entitlements), or feature usage (e.g., certification campaigns, tasks).

  • Virtual Appliances - Every tenant has different virtual appliance clusters. While these can be named similarly across tenants, they are usually part of one-time tenant setup and specific to that tenant, not something which needs to be migrated between environments.
  • Identities - Identities are built in accordance with the identity profile mappings, and based on aggregated account and authoritative source information. Alternately, these may come from Non-Employee Lifecycle Manager (NELM) processes, which are a by-product of actual tenant usage. Identities are not something that can be migrated from one tenant to another.
  • Accounts - Accounts are imported from source account aggregations and are not something that can be migrated from one tenant to another.
  • Entitlements - Entitlements are imported from source entitlement (group) aggregations and are not something that can be migrated from one tenant to another.
  • Audit Events - These objects are generated as a by-product of actual tenant usage, and because they relate to auditing, they are very tenant-specific; they cannot be migrated from one tenant to another.
  • Account Activities - These objects are system generated to describe in-flight or recently completed access request or provisioning activities. These are generated as a by-product of actual tenant usage, and they cannot be migrated from one tenant to another.
  • Certification Campaigns - These objects are created through the usage of Identity Security Cloud's compliance feature set. These are based on identity and account data, so they cannot be migrated from one tenant to another. Similar certification campaigns can be configured in each tenant, they just can’t be migrated from one environment to another.
  • Tags - Tags are created as a by-product of usage of the search features, and cannot be migrated across tenants.
  • Tasks - These objects are a result of manual provisioning or approval processes and are a by-product of actual tenant usage. They cannot by migrated across tenants.

Don’t see something? Feel free to ask on our Compass Community!

Promotion Process Between Tenants

There are several ways that people move configurations between Identity Security Cloud tenants. Each approach has tradeoffs depending on the level of technical ability or desired level of automation needed. These generally break these down into three categories:

  1. User Interface
  2. REST APIs
  3. SailPoint SaaS Configuration Tools

User Interface

The user interface is the most obvious choice in configuring the SailPoint SaaS service, as it merely involves an administrator configuring things the same way in one tenant as they are in another tenant. This is very approachable, and doesn’t require any special knowledge or technical tooling in order to accomplish. Any administrator can simply open two browser windows and mimic configurations in each environment.

While simple, this approach does have its drawbacks. Managing configurations this way is fairly tedious, and since it is manual, is prone to human error. For this reason, we do not recommend this for large, complex deployments.

One other drawback is that if there are advanced configurations made via REST APIs, these may not be immediately obvious in the user interface. These are typically settings that are only visible via REST APIs.

REST APIs

The REST APIs are a technical interface which can discover or configure almost anything in the system. The SailPoint SaaS user interface actually relies on the REST APIs to function, so anything you set up, administer, or configure in a tenant is actually executed via REST API calls.

Because REST APIs are a technical interface, they can be leveraged to configure the Always-Managed or Conditionally-Managed Objects in the system in an automated fashion. This can eliminate the tedious configuration promotion which exists with UI-based methods.

SailPoint finds this process is often also used by customers to plug SailPoint SaaS into a Continuous Integration / Continuous Deployment (CI/CD) pipeline or Change Management (CM) process. This is seen as a way of easily automating the tedious process from the user interface, while making sure that configuration changes are managed. The configuration data in this process is also often captured and committed in a Version Control (e.g. GitHub) system if so desired.

SailPoint does offer REST APIs specifically for managing bulk import and export of configurable objects. Detailed information can be found here: SaaS Configuration Import and Export

While REST APIs are a great way to ensure accuracy in any configuration promotion between tenants, there are a few tradeoffs. REST APIs are fairly technical in nature, and require a decent amount of technical skill and knowledge in order to use. Invoking REST APIs may require some development or scripting knowledge, and many REST API calls will be necessary to migrate all configurations.

Tip: For a reference of available REST APIs to support promotion of configurations between tenants, see developer.sailpoint.com. This is a great resource around all SailPoint SaaS APIs.

SailPoint SaaS Configuration Tools

Beyond the user interface and the REST APIs, SailPoint has fully-supported tooling designed specifically for promoting artifacts between customers' tenants: Configuration Hub. This is an incredibly valuable feature which allows backup/restore, cross-tenant migration, string and object tokenization, and draft comparisons directly within the Identity Security Cloud UI. The capability is underpinned by fully-exposed REST APIs as well, which can support CI/CD pipelines for customers who leverage highly-automated promotion practices. SailPoint strongly encourages customers and implementers leverage Configuration Hub for change management needs as a primary option, and only lean upon UI-based or REST API-based methods if strictly needed.

Other tools and frameworks also exist, and we encourage our partners to develop and share these kinds of tools on the SailPoint Developer Community.