Scripted SQL connector
The Scripted SQL connector is an implementation of the Scripted Groovy connector toolkit. This connector lets you interact with any SQL database, using Groovy scripts for the ICF operations.
To use this connector, you must write a Groovy script for each operation that you want the connector to perform (create, read, update, delete, authenticate, and so on). No sample scripts are bundled with the connector, but IDM customers have access to the Scripted SQL connector source code in the connectors-customers-ga repo. This repository includes sample scripts for all the ICF operations.
Configure the Scripted SQL connector
You cannot configure the Scripted SQL connector through the UI. Configure the connector over REST, as described in Configure Connectors Over REST.
Alternatively, a sample connector configuration and scripts are provided in the /path/to/openidm/samples/scripted-sql-with-mysql/
directory and described in Connect to a MySQL database with ScriptedSQL. The scripts provided with this sample demonstrate how the connector can be used, but most likely cannot be used as is in your deployment. They are a good starting point from which to base your customization. For information about writing your own scripts, refer to Scripted connectors with Groovy.
Run scripts through the connector
Groovy toolkit connectors have two operations that allow you to run arbitrary script actions: runScriptOnConnector
and runScriptOnResource
. runScriptOnConnector
is an operation that sends the script action to the connector to be compiled and executed. runScriptOnResource
is an operation that sends the script to another script to be handled.
runScriptOnConnector
The runScriptOnConnector
script lets you run an arbitrary script action through the connector. This script takes the following variables as input:
configuration
-
A handler to the connector’s configuration object.
options
-
A handler to the Operation Options.
operation
-
The operation type that corresponds to the action (
RUNSCRIPTONCONNECTOR
in this case). log
-
A handler to the connector’s log.
To run an arbitrary script on a Groovy toolkit connector, define the script in the systemActions
property of your provisioner file:
"systemActions" : [
{
"scriptId" : "MyScript",
"actions" : [
{
"systemType" : ".*ScriptedConnector",
"actionType" : "groovy",
"actionFile" : "path/to/<script-name>.groovy"
}
]
}
]
If you want to define your script in the provisioner file itself rather than in a separate file, you can use the actionSource
property instead of the actionFile
one. A simple example follows:
"systemActions" : [
{
"scriptId" : "MyScript",
"actions" : [
{
"systemType" : ".*ScriptedConnector",
"actionType" : "groovy",
"actionSource" : "2 * 2"
}
]
}
]
It is optional to prepend the last script statement in actionSource with return .
|
Running MyScript
will return:
{
"actions" : [
{
"result": 4
}
]
}
If your script accepts parameters, you may supply them in the request body or the query string. For example:
curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request POST \ --data-raw '{"param1":"value1"}' "http://localhost:8080/openidm/system/groovy?_action=script&scriptId=MyScript¶m2=value2"**
You can also call it through the IDM script engine. Note that the system can accept arbitrary parameters, as demonstrated here:
openidm.action("/system/groovy", "script", {"contentParameter": "value"}, {"scriptId": "MyScript", "additionalParameter1": "value1", "additionalParameter2": "value2"})
runScriptOnResource
To run an arbitrary script using runScriptOnResource
, you must add some configuration details to your provisioner file. These details include a scriptOnResourceScriptFileName
which references a script file located in a path contained in the scriptRoots
array.
Define these properties in your provisioner file as follows:
"configurationProperties": {
"scriptRoots": [
"path/to/scripts"
],
"scriptOnResourceScriptFileName": "ScriptOnResourceScript.groovy"
},
"systemActions" : [
{
"scriptId" : "script-1",
"actions" : [
{
"systemType" : ".*ScriptedConnector",
"actionType" : "groovy",
"actionFile" : "path/to/<script-name>.groovy"
}
]
}
]
When you have defined the script, you can call it over REST on the system endpoint, as follows:
curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request POST \ "http://localhost:8080/openidm/system/groovy?_action=script&scriptId=scriptOnResourceScript&scriptExecuteMode=resource"
Implemented interfaces
This table lists the ICF interfaces that are implemented for the scripted SQL connector:
OpenICF Interfaces Implemented by the Scripted SQL Connector
The Scripted SQL Connector implements the following OpenICF interfaces. For additional details, see ICF interfaces:
- Authenticate
-
Provides simple authentication with two parameters, presumed to be a user name and password.
- Create
-
Creates an object and its
uid
. - Delete
-
Deletes an object, referenced by its
uid
. - Resolve Username
-
Resolves an object by its username and returns the
uid
of the object. - Schema
-
Describes the object types, operations, and options that the connector supports.
- Script on Connector
-
Enables an application to run a script in the context of the connector.
Any script that runs on the connector has the following characteristics:
-
The script runs in the same execution environment as the connector and has access to all the classes to which the connector has access.
-
The script has access to a
connector
variable that is equivalent to an initialized instance of the connector. At a minimum, the script can access the connector configuration. -
The script has access to any script arguments passed in by the application.
-
- Script on Resource
-
Runs a script on the target resource that is managed by this connector.
- Search
-
Searches the target resource for all objects that match the specified object class and filter.
- Sync
-
Polls the target resource for synchronization events, that is, native changes to objects on the target resource.
- Test
-
Tests the connector configuration.
Testing a configuration checks all elements of the environment that are referred to by the configuration are available. For example, the connector might make a physical connection to a host that is specified in the configuration to verify that it exists and that the credentials that are specified in the configuration are valid.
This operation might need to connect to a resource, and, as such, might take some time. Do not invoke this operation too often, such as before every provisioning operation. The test operation is not intended to check that the connector is alive (that is, that its physical connection to the resource has not timed out).
You can invoke the test operation before a connector configuration has been validated.
- Update
-
Updates (modifies or replaces) objects on a target resource.
Scripted SQL Connector Configuration
The Scripted SQL Connector has the following configurable properties:
Configuration properties
Property | Type | Default | Encrypted(1) | Required(2) |
---|---|---|---|---|
|
|
|
Yes |
No |
The connection password to be passed to our JDBC driver to establish a connection. Note that method DataSource.getConnection(username,password) by default will not use credentials passed into the method, but will use the ones configured here. See alternateUsernameAllowed property for more details. |
||||
|
|
|
|
No |
The connection properties that will be sent to our JDBC driver when establishing new connections. Format of the string must be [propertyName=property;]* NOTE - The "user" and "password" properties will be passed explicitly, so they do not need to be included here. |
||||
|
|
|
|
No |
Set this to true to propagate the interrupt state for a thread that has been interrupted (not clearing the interrupt state). Set the value as false for backwards compatibility. |
||||
|
|
|
|
No |
Set this to true if you wish to put a facade on your connection so that it cannot be reused after it has been closed. This prevents a thread holding on to a reference of a connection it has already called closed on, to execute queries on it. |
||||
|
|
|
|
No |
The default catalog of connections created by this pool. |
||||
|
|
|
|
No |
avoid excess validation, only run validation at most at this frequency - time in milliseconds. If a connection is due for validation, but has been validated previously within this interval, it will not be validated again. |
||||
|
|
|
|
No |
Flag whether ignore error of connection creation while initializing the pool. Set to true if you want to ignore error of connection creation while initializing the pool. Set to false if you want to fail the initialization of the pool by throwing exception. |
||||
|
|
|
|
No |
Register the pool with JMX or not. |
||||
|
|
|
|
No |
If autoCommit==false then the pool can complete the transaction by calling commit on the connection as it is returned to the pool If rollbackOnReturn==true then this attribute is ignored. |
||||
|
|
|
|
No |
Flag to log stack traces for application code which abandoned a Connection. Logging of abandoned Connections adds overhead for every Connection borrow because a stack trace has to be generated. |
||||
|
|
|
|
No |
The maximum number of connections that should be kept in the pool at all times. Idle connections are checked periodically (if enabled) and connections that been idle for longer than minEvictableIdleTimeMillis will be released. (also see testWhileIdle). |
||||
|
|
|
|
No |
The indication of whether objects will be validated by the idle object evictor (if any). If an object fails to validate, it will be dropped from the pool. NOTE - for a true value to have any effect, the validationQuery parameter must be set to a non-null string. This property has to be set in order for the pool cleaner/test thread is to run (also see timeBetweenEvictionRunsMillis). |
||||
|
|
|
|
No |
Flag to remove abandoned connections if they exceed the removeAbandonedTimeout. If set to true a connection is considered abandoned and eligible for removal if it has been in use longer than the removeAbandonedTimeout Setting this to true can recover db connections from applications that fail to close a connection. See also logAbandoned. |
||||
|
|
|
|
No |
Connections that have been abandoned (timed out) wont get closed and reported up unless the number of connections in use are above the percentage defined by abandonWhenPercentageFull. The value should be between 0-100. The value 0 implies that connections are eligible for closure as soon as removeAbandonedTimeout has been reached. |
||||
|
|
|
|
No |
The minimum number of established connections that should be kept in the pool at all times. The connection pool can shrink below this number if validation queries fail (also see testWhileIdle). |
||||
|
|
|
|
No |
The default read-only state of connections created by this pool. If not set then the setReadOnly method will not be called (Some drivers dont support read only mode, ex: Informix). |
||||
|
|
|
|
No |
The maximum number of milliseconds that the pool will wait (when there are no available connections) for a connection to be returned before throwing an exception. |
||||
|
|
|
|
No |
Set this to true to log errors during the validation phase to the log file. If set to true, errors will be logged as SEVERE. Set the value as false for backwards compatibility. |
||||
|
|
|
|
No |
The fully qualified Java class name of the JDBC driver to be used. The driver has to be accessible from the same classloader as tomcat-jdbc.jar. |
||||
|
|
|
|
No |
Returns the name of the connection pool. By default a JVM unique random name is assigned. |
||||
|
|
|
|
No |
If a statement proxy is set, wrap statements so that equals() and hashCode() methods can be called on closed statements. |
||||
|
|
|
|
No |
A custom query to be run when a connection is first created. |
||||
|
|
|
|
No |
The timeout in seconds before a connection validation queries fail. This works by calling java.test_sample.Statement.setQueryTimeout(seconds) on the statement that executes the validationQuery. The pool itself doesnt timeout the query, it is still up to the JDBC driver to enforce query timeouts. A value less than or equal to zero will disable this feature. |
||||
|
|
|
|
No |
The SQL query that will be used to validate connections from this pool before returning them to the caller. If specified, this query does not have to return any data, it just cant throw a SQLException. Example values are SELECT 1(mysql), select 1 from dual(oracle), SELECT 1(MS Sql Server). |
||||
|
|
|
|
No |
If autoCommit==false then the pool can terminate the transaction by calling rollback on the connection as it is returned to the pool. |
||||
|
|
|
|
No |
By default, the jdbc-pool will ignore the DataSource.getConnection(username,password) call, and simply return a previously pooled connection under the globally configured properties username and password, for performance reasons. The pool can however be configured to allow use of different credentials each time a connection is requested. To enable the functionality described in the DataSource.getConnection(username,password) call, simply set the property alternateUsernameAllowed to true. Should you request a connection with the credentials user1/password1 and the connection was previously connected using different user2/password2, the connection will be closed, and reopened with the requested credentials. This way, the pool size is still managed on a global level, and not on a per schema level. |
||||
|
|
|
|
No |
The JNDI name for a data source to be looked up in JNDI and then used to establish connections to the database. See the dataSource attribute. |
||||
|
|
|
|
No |
The name of a class which implements the org.apache.tomcat.jdbc.pool.Validator interface and provides a no-arg constructor (may be implicit). If specified, the class will be used to create a Validator instance which is then used instead of any validation query to validate connections. An example value is com.mycompany.project.SimpleValidator. |
||||
|
|
|
|
No |
Timeout value in seconds. Similar to to the removeAbandonedTimeout value but instead of treating the connection as abandoned, and potentially closing the connection, this simply logs the warning if logAbandoned is set to true. If this value is equal or less than 0, no suspect checking will be performed. Suspect checking only takes place if the timeout value is larger than 0 and the connection was not abandoned or if abandon check is disabled. If a connection is suspect a WARN message gets logged and a JMX notification gets sent once. |
||||
|
|
|
|
No |
Set to true if you wish the ProxyConnection class to use String.equals and set to false when you wish to use == when comparing method names. This property does not apply to added interceptors as those are configured individually. |
||||
|
|
|
|
No |
Timeout in seconds before an abandoned(in use) connection can be removed. The value should be set to the longest running query your applications might have. |
||||
|
|
|
|
No |
The default auto-commit state of connections created by this pool. If not set, default is JDBC driver default (If not set then the setAutoCommit method will not be called). |
||||
|
|
|
|
No |
Validate the connection when connecting to the database for the first time. Set to true if you want to use the validationQuery as an init query. |
||||
|
|
|
|
No |
A semicolon separated list of classnames extending org.apache.tomcat.jdbc.pool.JdbcInterceptor class. See Configuring JDBC interceptors below for more detailed description of syntaz and examples. These interceptors will be inserted as an interceptor into the chain of operations on a java.test_sample.Connection object. |
||||
|
|
|
|
No |
The initial number of connections that are created when the pool is started. |
||||
|
|
|
|
No |
The default TransactionIsolation state of connections created by this pool. One of the following: NONE, READ_COMMITTED, READ_UNCOMMITTED, REPEATABLE_READ, SERIALIZABLE If not set, the method will not be called and it defaults to the JDBC driver. |
||||
|
|
|
|
No |
Property not used in tomcat-jdbc-pool. |
||||
|
|
|
|
No |
The URL used to connect to the database. |
||||
|
|
|
|
No |
The indication of whether objects will be validated before being borrowed from the pool. If the object fails to validate, it will be dropped from the pool, and we will attempt to borrow another. NOTE - for a true value to have any effect, the validationQuery parameter must be set to a non-null string. In order to have a more efficient validation, see validationInterval. |
||||
|
|
|
|
No |
Set to true if you wish that calls to getConnection should be treated fairly in a true FIFO fashion. This uses the org.apache.tomcat.jdbc.pool.FairBlockingQueue implementation for the list of the idle connections. This flag is required when you want to use asynchronous connection retrieval. Setting this flag ensures that threads receive connections in the order they arrive. During performance tests, there is a very large difference in how locks and lock waiting is implemented. When fairQueue=true there is a decision making process based on what operating system the system is running. If the system is running on Linux (property os.name=Linux. To disable this Linux specific behavior and still use the fair queue, simply add the property org.apache.tomcat.jdbc.pool.FairBlockingQueue.ignoreOS=true to your system properties before the connection pool classes are loaded. |
||||
|
|
|
|
No |
Property not used. Access can be achieved by calling unwrap on the pooled connection. see javax.test_sample.DataSource interface, or call getConnection through reflection or cast the object as javax.test_sample.PooledConnection. |
||||
|
|
|
|
No |
Time in milliseconds to keep this connection. When a connection is returned to the pool, the pool will check to see if the now - time-when-connected > maxAge has been reached, and if so, it closes the connection rather than returning it to the pool. The value 0 implies that connections will be left open and no age check will be done upon returning the connection to the pool. |
||||
|
|
|
|
No |
The minimum amount of time an object may sit idle in the pool before it is eligible for eviction. |
||||
|
|
|
|
No |
The number of milliseconds to sleep between runs of the idle connection validation/cleaner thread. This value should not be set under 1 second. It dictates how often we check for idle, abandoned connections, and how often we validate idle connections. |
||||
|
|
|
|
No |
The indication of whether objects will be validated before being returned to the pool. NOTE - for a true value to have any effect, the validationQuery parameter must be set to a non-null string. |
||||
|
|
|
|
No |
Use a lock when performing operations on the connection object. Set to true if you will use a separate background thread for idle and abandon checking (e.g. JMX clients). If the pool sweeper is enabled, a lock is used, regardless of this setting. |
||||
|
|
|
|
No |
The maximum number of active connections that can be allocated from this pool at the same time. |
||||
|
|
|
|
No |
The connection username to be passed to our JDBC driver to establish a connection. Note that method DataSource.getConnection(username,password) by default will not use credentials passed into the method, but will use the ones configured here. See alternateUsernameAllowed property for more details. |
(1) Whether the property value is considered confidential, and is therefore encrypted in IDM.
(2) A list of operations in this column indicates that the property is required for those operations.
Groovy Engine configuration
Property | Type | Default | Encrypted(1) | Required(2) |
---|---|---|---|---|
|
|
|
|
Yes |
The root folder to load the scripts from. If the value is null or empty the classpath value is used. |
||||
|
|
|
|
No |
Classpath for use during compilation. |
||||
|
|
|
|
No |
If true, debugging code should be activated. |
||||
|
|
|
|
No |
Sets a list of global AST transformations which should not be loaded even if they are defined in META-INF/org.codehaus.groovy.transform.ASTTransformation files. By default, none is disabled. |
||||
|
|
|
|
No |
Sets the minimum of time after a script can be recompiled. |
||||
|
|
|
|
No |
If set to true recompilation is enabled. |
||||
|
|
|
|
No |
Base class name for scripts (must derive from Script). |
||||
|
|
|
|
No |
Gets the extensions used to find groovy files. |
||||
|
|
|
|
No |
Encoding for source files. |
||||
|
|
|
|
No |
Directory into which to write classes. |
||||
|
|
|
|
No |
The error tolerance, which is the number of non-fatal errors (per unit) that should be tolerated before compilation is aborted. |
||||
|
|
|
|
No |
If true, the compiler should produce action information. |
||||
|
|
|
|
No |
Warning Level of the compiler. |
||||
|
|
|
|
No |
Custom Configuration script for Groovy ConfigSlurper. |
||||
|
|
|
Yes |
No |
Custom Sensitive Configuration script for Groovy ConfigSlurper. |
(1) Whether the property value is considered confidential, and is therefore encrypted in IDM.
(2) A list of operations in this column indicates that the property is required for those operations.
Operation Script Files
Property | Type | Default | Encrypted(1) | Required(2) |
---|---|---|---|---|
|
|
|
|
|
The name of the file used to perform the AUTHENTICATE operation. |
||||
|
|
|
|
|
The name of the file used to perform the CREATE operation. |
||||
|
|
|
|
No |
The script used to customize some function of the connector. Read the documentation for more details. |
||||
|
|
|
|
|
The name of the file used to perform the DELETE operation. |
||||
|
|
|
|
|
The name of the file used to perform the RESOLVE_USERNAME operation. |
||||
|
|
|
|
|
The name of the file used to perform the SCHEMA operation. |
||||
|
|
|
|
|
The name of the file used to perform the RUNSCRIPTONRESOURCE operation. |
||||
|
|
|
|
|
The name of the file used to perform the SEARCH operation. |
||||
|
|
|
|
|
The name of the file used to perform the SYNC operation. |
||||
|
|
|
|
|
The name of the file used to perform the TEST operation. |
||||
|
|
|
|
|
The name of the file used to perform the UPDATE operation. |
(1) Whether the property value is considered confidential, and is therefore encrypted in IDM.
(2) A list of operations in this column indicates that the property is required for those operations.