Scripted REST connector
The Scripted REST connector is an implementation of the Scripted Groovy connector toolkit. It can interact with any REST API, using Groovy scripts for the ICF operations. This connector type lets you develop a fully functional REST-based connector for in-house applications or any cloud-based application not yet supported with the standard set of connectors.
To use this connector, you must write a Groovy script for each operation that you want the connector to perform (create, read, update, delete, authenticate, and so on). No sample scripts are bundled with the connector, but IDM customers have access to the Scripted REST connector source code in the connectors-customers-ga repo. This repository includes sample scripts for all the ICF operations.
You cannot configure the Scripted REST connector through the UI. Configure the connector over REST, as described in Configure Connectors Over REST.
Alternatively, a sample connector configuration and scripts are provided in the /path/to/openidm/samples/scripted-rest-with-dj/
directory and described in Connect to DS with ScriptedREST. The scripts provided with this sample demonstrate how the connector can be used, but most likely cannot be used as is in your deployment. They are a good starting point from which to base your customization. For information about writing your own scripts, refer to Scripted connectors with Groovy.
Script custom behavior
The Scripted REST connector uses the Apache HTTP client library. Unlike the Scripted SQL connector, which uses JDBC drivers and a Tomcat JDBC connection pool, the Scripted REST connector includes a special script to customize the Apache HTTP client.
This customizer script lets you customize the Apache HTTP client connection pool, proxy, default headers, timeouts, and so on.
The customizer script is referenced in the connector configuration, in the CustomizerScriptFileName
property:
{
...
"configurationProperties": {
...
"customizerScriptFileName": "CustomizerScript.groovy",
...
}
}
The script can implement two predefined Groovy closures — init {}
and decorate {}
.
init {}
-
The Apache HTTP client provides an HTTPClientBuilder class, to build an instance of the HTTPClient. The Scripted REST connector injects this builder into the
init
closure when the connector is first instantiated. Theinit
closure is the ideal place to customize the HTTP client with the builder.You can customize the following elements of the client:
-
Connection pool
-
Connection timeouts
-
Proxy
-
Default HTTP headers
-
Certificate handling
Example init closure
/** * A customizer script defines the custom closures to interact with the default implementation and customize it. * Here, the {@link HttpClientBuilder} is passed to the customize closure. This is where the pooling, the headers, * the timeouts etc... should be defined. */ customize { init { HttpClientBuilder builder -> //SETUP: org.apache.http def c = delegate as ScriptedRESTConfiguration def httpHost = new HttpHost(c.serviceAddress?.host, c.serviceAddress?.port, c.serviceAddress?.scheme) PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager() // Increase max total connection to 200 cm.setMaxTotal(200) // Increase default max connection per route to 20 cm.setDefaultMaxPerRoute(20) // Increase max connections for httpHost to 50 cm.setMaxPerRoute(new HttpRoute(httpHost), 50) builder.setConnectionManager(cm) // configure timeout on the entire client RequestConfig requestConfig = RequestConfig.custom()/* * . * setConnectionRequestTimeout * ( 50). * setConnectTimeout * (50) * .setSocketTimeout * (50) */.build(); builder.setDefaultRequestConfig(requestConfig) ... } }
Call the builder methods to fit your requirements. The
init{}
closure does not need to return anything. -
decorate {}
-
The
init
closure configures a Java instance of the HTTP client, which is injected into every CRUD script. In addition to the libraries provided by the Apache HTTP client, Groovy provides a number of libraries to deal with requests and responses.The
decorate
closure lets you inject a "decorated" instance of the HTTP client into your scripts. For example, the sample scripts use thegroovyx.net.http.RESTClient
library.This excerpt of a sample
delete
script shows the injection of thehttpClient
andconnection
variables into the script. Theconnection
variable is the output of thedecorate
closure.Example decorate closure
def operation = operation as OperationType def configuration = configuration as ScriptedRESTConfiguration def httpClient = connection as HttpClient def connection = customizedConnection as RESTClient def log = log as Log def objectClass = objectClass as ObjectClass def options = options as OperationOptions def uid = uid as Uid log.info("Entering " + operation + " Script"); switch (objectClass) { case ObjectClass.ACCOUNT: connection.delete(path: '/api/users/' + uid.uidValue); break case ObjectClass.GROUP: connection.delete(path: '/api/groups/' + uid.uidValue); }
When you use the
|
Example OAuth2 Authentication Implementation
This example shows how to use the customizer script to implement OAuth2 authentication in the Scripted REST connector.
Although grant types are largely standardized across OAuth2 authentication providers, the way in which different providers handle flows, headers, attribute names, and so on, often differs. This makes it difficult to include a single implementation of OAuth2 authentication in the Scripted REST connector. To make sure that OAuth2 authentication works in your specific use case, you use the customizer script, which can be adapted without requiring a new version of the connector itself.
The Scripted REST connector includes a simple implementation of the OAuth2 Client Credentials grant type. The connector needs to get an access token, using the Client ID and the Client Secret, cache it, and renew it when it expires or when the server revokes it. The Apache client provides interceptors for requests and responses. These interceptors can be used in the customizer script to manage the access token:
-
In the request: If the access token is absent or expired, renew the token and cache it in the Scripted REST connector property bag.
-
In the response: If the server returns a 401 error, delete the Access Token from the connector property bag. This will ensure that the next connector request gets a new access token. The HTTP POST query to get the access token is also handled by the customizer script.
This example shows a complete customizer script for the OAuth2 implementation:
init { HttpClientBuilder builder ->
switch (ScriptedRESTConfiguration.AuthMethod.valueOf(c.defaultAuthMethod)) {
// ......
case ScriptedRESTConfiguration.AuthMethod.OAUTH:
// define a request interceptor to set the Authorization header if absent or expired
HttpRequestInterceptor requestInterceptor = { HttpRequest request, HttpContext context ->
if (null == context.getAttribute("oauth-request")) {
def exp = c.propertyBag.tokenExpiration as Long
if (c.propertyBag.accessToken == null || exp < System.currentTimeMillis() / 1000) {
new NewAccessToken(c).clientCredentials()
}
request.addHeader(new BasicHeader(HttpHeaders.AUTHORIZATION, "Bearer " + c.propertyBag.accessToken))
}
}
// define a response interceptor to catch a 401 response code and delete access token from cache
HttpResponseInterceptor responseInterceptor = { HttpResponse response, HttpContext context ->
if (HttpStatus.SC_UNAUTHORIZED == response.statusLine.statusCode) {
if (c.propertyBag.accessToken != null) {
c.propertyBag.remove("accessToken")
Log.getLog(ScriptedRESTConnector.class).info("Code 401 - accessToken removed")
}
}
}
builder.addInterceptorLast(requestInterceptor)
builder.addInterceptorLast(responseInterceptor)
break
default:
throw new IllegalArgumentException()
}
}
class NewAccessToken {
static final String GRANT_TYPE = "grant_type"
static final String REFRESH_TOKEN = "refresh_token"
static final String CLIENT_CREDENTIALS = "client_credentials"
static final String CLIENT_ID = "client_id"
static final String CLIENT_SECRET = "client_secret"
static final String OAUTH_REQUEST = "oauth-request"
Log logger = Log.getLog(NewAccessToken.class)
ScriptedRESTConfiguration c = null
final CloseableHttpClient client = null
final HttpPost post = null
NewAccessToken(ScriptedRESTConfiguration conf) {
this.c = conf
this.client = c.getHttpClient()
this.post = new HttpPost(c.getOAuthTokenEndpoint())
post.setHeader(HttpHeaders.CONTENT_TYPE, "application/x-www-form-urlencoded")
post.setHeader(HttpHeaders.ACCEPT, "application/json")
}
@Synchronized
void clientCredentials() {
boolean expired = (c.propertyBag.tokenExpiration as Long) < System.currentTimeMillis() / 1000
if (c.propertyBag.accessToken == null || expired ) {
if (c.propertyBag.tokenExpiration != null && expired) {
logger.info("Token expired!")
}
logger.info("Getting new access token...")
final List<NameValuePair> pairs = new ArrayList<>()
pairs.add(new BasicNameValuePair(GRANT_TYPE, CLIENT_CREDENTIALS))
pairs.add(new BasicNameValuePair(CLIENT_ID, c.getOAuthClientId()))
pairs.add(new BasicNameValuePair(CLIENT_SECRET, SecurityUtil.decrypt(c.getOAuthClientSecret())))
post.setEntity(new UrlEncodedFormEntity(pairs))
CloseableHttpResponse response = null
try {
HttpClientContext ctx = HttpClientContext.create()
ctx.setAttribute(OAUTH_REQUEST, true)
response = client.execute(post, ctx)
int statusCode = response.getStatusLine().getStatusCode()
if (HttpStatus.SC_OK == statusCode) {
def jsonSlurper = new JsonSlurper()
def oauthResponse = jsonSlurper.parseText(EntityUtils.toString(response.getEntity()))
c.propertyBag.accessToken = oauthResponse.access_token
c.propertyBag.tokenExpiration = System.currentTimeMillis() / 1000 + oauthResponse.expires_in as Long
} else {
throw new InvalidCredentialException("Retrieve Access Token failed with code: " + statusCode)
}
} catch (ClientProtocolException ex) {
logger.info("Trace: {0}", ex.getMessage())
throw new ConnectorException(ex)
} catch (IOException ex) {
logger.info("Trace: {0}", ex.getMessage())
throw new ConnectionFailedException(ex)
} finally {
try {
if (response != null) {
response.close()
}
} catch (IOException e) {
logger.info("Can't close HttpResponse")
}
}
}
}
}
Using the Scripted REST connector with a proxy server
If the IDM server is hosted behind a firewall and requests to the resource are routed through a proxy, you must specify the proxy host and port in the connector configuration.
To specify the proxy server details, set the proxyAddress
property in the connector configuration. For example:
"configurationProperties": {
...
"proxyAddress": "http://myproxy:8080",
...
}
Run scripts through the connector
Groovy toolkit connectors have two operations that allow you to run arbitrary script actions: runScriptOnConnector
and runScriptOnResource
. runScriptOnConnector
is an operation that sends the script action to the connector to be compiled and executed. runScriptOnResource
is an operation that sends the script to another script to be handled.
runScriptOnConnector
The runScriptOnConnector
script lets you run an arbitrary script action through the connector. This script takes the following variables as input:
configuration
-
A handler to the connector’s configuration object.
options
-
A handler to the Operation Options.
operation
-
The operation type that corresponds to the action (
RUNSCRIPTONCONNECTOR
in this case). log
-
A handler to the connector’s log.
To run an arbitrary script on a Groovy toolkit connector, define the script in the systemActions
property of your provisioner file:
"systemActions" : [
{
"scriptId" : "MyScript",
"actions" : [
{
"systemType" : ".*ScriptedConnector",
"actionType" : "groovy",
"actionFile" : "path/to/<script-name>.groovy"
}
]
}
]
If you want to define your script in the provisioner file itself rather than in a separate file, you can use the actionSource
property instead of the actionFile
one. A simple example follows:
"systemActions" : [
{
"scriptId" : "MyScript",
"actions" : [
{
"systemType" : ".*ScriptedConnector",
"actionType" : "groovy",
"actionSource" : "2 * 2"
}
]
}
]
It is optional to prepend the last script statement in actionSource with return .
|
Running MyScript
will return:
{
"actions" : [
{
"result": 4
}
]
}
If your script accepts parameters, you may supply them in the request body or the query string. For example:
curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request POST \ --data-raw '{"param1":"value1"}' "http://localhost:8080/openidm/system/groovy?_action=script&scriptId=MyScript¶m2=value2"**
You can also call it through the IDM script engine. Note that the system can accept arbitrary parameters, as demonstrated here:
openidm.action("/system/groovy", "script", {"contentParameter": "value"}, {"scriptId": "MyScript", "additionalParameter1": "value1", "additionalParameter2": "value2"})
runScriptOnResource
To run an arbitrary script using runScriptOnResource
, you must add some configuration details to your provisioner file. These details include a scriptOnResourceScriptFileName
which references a script file located in a path contained in the scriptRoots
array.
Define these properties in your provisioner file as follows:
"configurationProperties": {
"scriptRoots": [
"path/to/scripts"
],
"scriptOnResourceScriptFileName": "ScriptOnResourceScript.groovy"
},
"systemActions" : [
{
"scriptId" : "script-1",
"actions" : [
{
"systemType" : ".*ScriptedConnector",
"actionType" : "groovy",
"actionFile" : "path/to/<script-name>.groovy"
}
]
}
]
When you have defined the script, you can call it over REST on the system endpoint, as follows:
curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request POST \ "http://localhost:8080/openidm/system/groovy?_action=script&scriptId=scriptOnResourceScript&scriptExecuteMode=resource"
Implemented interfaces
This table lists the ICF interfaces that are implemented for the scripted REST connector:
OpenICF Interfaces Implemented by the Scripted REST Connector
The Scripted REST Connector implements the following OpenICF interfaces. For additional details, see ICF interfaces:
- Authenticate
-
Provides simple authentication with two parameters, presumed to be a user name and password.
- Create
-
Creates an object and its
uid
. - Delete
-
Deletes an object, referenced by its
uid
. - Resolve Username
-
Resolves an object by its username and returns the
uid
of the object. - Schema
-
Describes the object types, operations, and options that the connector supports.
- Script on Connector
-
Enables an application to run a script in the context of the connector.
Any script that runs on the connector has the following characteristics:
-
The script runs in the same execution environment as the connector and has access to all the classes to which the connector has access.
-
The script has access to a
connector
variable that is equivalent to an initialized instance of the connector. At a minimum, the script can access the connector configuration. -
The script has access to any script arguments passed in by the application.
-
- Script on Resource
-
Runs a script on the target resource that is managed by this connector.
- Search
-
Searches the target resource for all objects that match the specified object class and filter.
- Sync
-
Polls the target resource for synchronization events, that is, native changes to objects on the target resource.
- Test
-
Tests the connector configuration.
Testing a configuration checks all elements of the environment that are referred to by the configuration are available. For example, the connector might make a physical connection to a host that is specified in the configuration to verify that it exists and that the credentials that are specified in the configuration are valid.
This operation might need to connect to a resource, and, as such, might take some time. Do not invoke this operation too often, such as before every provisioning operation. The test operation is not intended to check that the connector is alive (that is, that its physical connection to the resource has not timed out).
You can invoke the test operation before a connector configuration has been validated.
- Update
-
Updates (modifies or replaces) objects on a target resource.
Scripted REST Connector Configuration
The Scripted REST Connector has the following configurable properties:
Basic Configuration Properties
Property | Type | Default | Encrypted(1) | Required(2) |
---|---|---|---|---|
|
|
|
|
No |
The Remote user to authenticate with. |
||||
|
|
|
Yes |
No |
The Password to authenticate with. |
||||
|
|
|
|
Yes |
The service URI (example: http://myservice.com/api). |
||||
|
|
|
|
No |
The optional Proxy server URI (example: http://myproxy:8080). |
||||
|
|
|
|
No |
The username to authenticate with the proxy server. |
||||
|
|
|
Yes |
No |
The password to authenticate with the proxy server. |
||||
|
|
|
|
No |
Authentication method used. Can be: BASIC, BASIC_PREEMPTIVE, OAUTH or NONE. |
||||
|
|
|
|
No |
Default HTTP request content type. Can be: JSON, TEXT, XML, HTML, URLENC, BINARY. |
||||
|
|
|
|
No |
Placeholder for default HTTP request headers. |
||||
|
|
|
|
No |
When using OAUTH, this property defines the endpoint where a new access token should be queried for (https://myserver.com/oauth2/token). |
||||
|
|
|
|
No |
The client identifier. |
||||
|
|
|
Yes |
No |
Secure client secret for OAUTH. |
||||
|
|
|
Yes |
No |
The refresh token used to renew the access token for the refresh_token grant type. |
||||
|
|
|
|
No |
The optional scope. |
||||
|
|
|
|
No |
The grant type to use. Can be: |
||||
|
|
|
|
No |
Defines throttling for read operations either per seconds ("30/sec") or per minute ("100/min"). |
||||
|
|
|
|
No |
Defines throttling for write operations (create/update/delete) either per second ("30/sec") or per minute ("100/min"). |
(1) Whether the property value is considered confidential, and is therefore encrypted in IDM.
(2) A list of operations in this column indicates that the property is required for those operations.
Groovy Engine configuration
Property | Type | Default | Encrypted(1) | Required(2) |
---|---|---|---|---|
|
|
|
|
Yes |
The root folder to load the scripts from. If the value is null or empty the classpath value is used. |
||||
|
|
|
|
No |
Classpath for use during compilation. |
||||
|
|
|
|
No |
If true, debugging code should be activated. |
||||
|
|
|
|
No |
Sets a list of global AST transformations which should not be loaded even if they are defined in META-INF/org.codehaus.groovy.transform.ASTTransformation files. By default, none is disabled. |
||||
|
|
|
|
No |
Sets the minimum of time after a script can be recompiled. |
||||
|
|
|
|
No |
If set to true recompilation is enabled. |
||||
|
|
|
|
No |
Base class name for scripts (must derive from Script). |
||||
|
|
|
|
No |
Gets the extensions used to find groovy files. |
||||
|
|
|
|
No |
Encoding for source files. |
||||
|
|
|
|
No |
Directory into which to write classes. |
||||
|
|
|
|
No |
The error tolerance, which is the number of non-fatal errors (per unit) that should be tolerated before compilation is aborted. |
||||
|
|
|
|
No |
If true, the compiler should produce action information. |
||||
|
|
|
|
No |
Warning Level of the compiler. |
||||
|
|
|
|
No |
Custom Configuration script for Groovy ConfigSlurper. |
||||
|
|
|
Yes |
No |
Custom Sensitive Configuration script for Groovy ConfigSlurper. |
(1) Whether the property value is considered confidential, and is therefore encrypted in IDM.
(2) A list of operations in this column indicates that the property is required for those operations.
Operation Script Files
Property | Type | Default | Encrypted(1) | Required(2) |
---|---|---|---|---|
|
|
|
|
|
The name of the file used to perform the AUTHENTICATE operation. |
||||
|
|
|
|
|
The name of the file used to perform the CREATE operation. |
||||
|
|
|
|
No |
The script used to customize some function of the connector. Read the documentation for more details. |
||||
|
|
|
|
|
The name of the file used to perform the DELETE operation. |
||||
|
|
|
|
|
The name of the file used to perform the RESOLVE_USERNAME operation. |
||||
|
|
|
|
|
The name of the file used to perform the SCHEMA operation. |
||||
|
|
|
|
|
The name of the file used to perform the RUNSCRIPTONRESOURCE operation. |
||||
|
|
|
|
|
The name of the file used to perform the SEARCH operation. |
||||
|
|
|
|
|
The name of the file used to perform the SYNC operation. |
||||
|
|
|
|
|
The name of the file used to perform the TEST operation. |
||||
|
|
|
|
|
The name of the file used to perform the UPDATE operation. |
(1) Whether the property value is considered confidential, and is therefore encrypted in IDM.
(2) A list of operations in this column indicates that the property is required for those operations.