Command line tools for OA4MP

These are a set of programs that are accessible from a CLI (command line interface) allow an administrator to do CRUD (Create, Read, Update and Delete) operations on client records and approvals. There is also the option to copy a store completely from one store to another.

Getting the tools

Generally you should use the installer and follow the instructions for that. This will deploy and configure a fully functional CLI for your server. If you really insist on downloading and running the jar manually, please the Appendix.

Configuration

The tool uses a standard configuration file exactly like the server. Indeed, you can (and generally should) just point the tool at the server configuration file. You will need to supply the name of the configuration in the file you want to use. Configuration file aliases (these are internal to the can be quite handy here. Given the abilities of the CLI to do quick copies of the store, it pays to have a backup (file) store configured and occasionally simple copy the active store to that. There are many possible uses. It is assumed that the tools are installed in a location called $OA4MP_SERVER. Normally the default for this is

OA4MP_SERVER=/opt/oa4mp/server

The assumption of the script is that files are in the following directory structure

  • $OA4MP_SERVER/etc/admin-cfg.xml - the configuration file
  • $OA4MP_SERVER/lib/cli.jar - the jar containing the CLI
  • $OA4MP_SERVER/bin/cli - the script for running this. Be sure it is set to be executable.

The default name of the configuration is assumed to be "default" (no quotes). If you invoke the script with no arguments it will use this and the above configuration file. Invocation syntax is

java -jar $OA4MP_SERVER/lib/cli.jar  [configName configFile env]
  1. configName = name of the configuration in the file
  2. configFile = full path to the configuration file
  3. env = full path to a properties fiel that has environment variables set in it. (This is specialized and can usually be omitted.)

Note that you may shorten the list, but cannot, e.g., just supply the configuration file.

Help!

For every command you may add the switch --help and you will get the most up to date information on the topic available. The interactive help (as it is called) is always up to date. E.g. if you are using transactions and want to view the help for the ls command:

    transactions> ls --help
    ls [-l  | -v | -E] | [-key key | -keys array] id
      Usage: Lists information about the contents of the store, an entry and
         individual values of the entry.
      When listing multiple entries, tools will use the most numbers from the most recent call to this.
      A line listing is tabular and will shorten entries that are too long, ending them with ...
      A verbose command will format every bit of every entry within the margins.
      Note: The argument idiom '-key key_name' may be replaced with '>key_name' as a shorthand
    E.g.
      ls -l  -E
      Prints out the line form of *every* object in this store. This may be simply huge
    E.g.
      ls
      Prints out the short form of *every* object in this store. This may also be huge.
      If you are using this to find things, you probably want to look at the search command
    E.g.
      ls -l  /foo:bar
      Prints a line format for the entry with id foo:bar
    E.g.
      ls -v /foo:bar
      prints out a verbose listing of the entry with id foo:bar.
    E.g.
      ls -key id /foo:bar
      >   foo:bar
      Prints out the id property for the object with identifier foo:bar

      You may also supply a list of keys in an array of the form [key0,key1,...].
    E.g.
      ls -keys [id,callback_uris,create_ts] /foo:bar
      would print the id, callback_uri and create_ts properties for the object with id
      foo:bar.

  See also list_keys, search, archive
    For transaction stores, you may also specify listing by using the access token or refresh token:
    ls [-at | -rt token]
    Note that other switches, such as -v work as well.

You see that this contains the command line switches, various examples, specific switchs (-at) only found in the transactions component and a See also block the point you at related or useful other commands.

Meta commands

The OA4MP CLI extends the NCSA CLI framework, so meta commands are more fully discussed there. There are commands to do tasks and there are meta-commands that control how the CLI itself operates. These are prefixed with a backslash /. E.g. to quit a component, issue

/q

A full listing of meta commands is done with the ?:

/?

Commonly used meta-commands are a command history, executing a command in another component without switching from the current one, saving you command history (useful if you want to use a set of commands as a basis for some scripting). To list the commands available for the admin component, you would issue

admin>/commands
    approve          copy             deserialize      get_id           ls               rm               set_env          status_search
    approver_search  count_clients    echo             link             print_env        rs               set_id           unlink
     .
     . (more commands)
     .

An example of using the // command

One specific meta command that is quite useful is the // which will execute a command for another component.For example, if you are working with transactions and want to check how many there are then see how many token exchanges there are, you would do this:

    transactions>size
        Current store has 10 entries (excluding versions).
    transactions>//tokens size
        Current store has 3 entries (excluding versions).

Here the // tells the CLI to take the token component then the rest of the line (in this case just the size command) is passed to the component. Note that no argument will just change to that component and exiting it will return to you the original component.

Running the tool

The CLI consists of utilities for each component that can be used. Once invoked you will see the startup banner and prompt:
[mybox bin]# ./cli
**********************************************************
* OA4MP CLI (Command Line Interpreter)                   *
* Version 6.0                                          *
* By Jeff Gaynor  NCSA                                   *
*  (National Center for Supercomputing Applications)     *
*                                                        *
* type 'help' for a list of commands                     *
*      'exit' or 'quit' to end this session.             *
**********************************************************
oa4mp>
If you type "--help" You will get
oa4mp>--help

Here are the commands available:
use
load
To get more information on a command type

command --help
Basic supported commands are
  • use - specify a component to use.
  • load configName [configFile] - load the given named configuration from the current config file. Specify the config file if you need a different one.
To see the help for, say, the use command you would issue
   oa4mp>use --help
    Choose the component you wish to use.
    you specify the component as use + name. Supported components are
    clients - edit client records
    approvals - edit client approval records
    copy - copy an entire store.
    keys - create a set of signing keys.
    permissions - basic permission management.
    admins - create or manage administrative clients.
    parser - write/debug scripts from the command line.
    tokens - manage tokens created in the token exchange endpoint
    vo - manage virtual organizations
 e.g.

 use clients

 will call up the client management component.
 Type 'exit' or /q when you wish to exit the component and return to the main menu
 --> and /h prints your command history, /r runs the last command
Each of these components; clients, approvals and copy is described in detail below. But first, a few preliminary notions.

Index vs. Unique ID

Since most objects have unique identifiers, you can specify these are arguments at any time it makes sense to do so by prefixing the id with a forward slash, "/". If there is a list of objects, you can also specify the index on the list as the target of the operation. The ls command will list everything in the active component.

After you issue an ls (no arguments) you will see a complete list of items in the store. These are numbered on the right hand side. This is the item's index. You may then specify the index directly. E.g. to print out a long version of the item with index 4 issue

ls 4

Typically you will know the unique identifier for an item and you can enter this if you escape it with a forward slash (/). To give the long listing of an object with unique identifier myproxy:oa4mp,2012:/client/a4b78549990 you would issue

ls /myproxy:oa4mp,2012:/client/a4b78549990

Note that since there is no canonical ordering of objects in a store, you should always issue an ls before using the index. It is generally always safer to use the unique identifier.

Setting a default id

If you are working on a single item in the store, you may sinple set a default id to be used for all operations with the

    set_id id

command. Related is the get_id to display the current id and the clear_id to clear it. If you have an default id set, you do not need to specify one.

    oa4mp>set_id foo:bar/34456
    oa4mp>ls -l
  name : My test thing
    id : foo:bar/34456

In this case, the details (long form) for the listing of the object are displayed. Had you issued

    ls -l /fnord:46456

The long form for this object would be displayed, since the final argument overrides the default.

Options supported in each storage tool.

Each of these in turn will be discussed in detail.

Common commands for all storage components.

Getting The Number Of Objects In a Component.

This will print the number of elements in the current storage component.
Example. For showing the number of clients in a store you would have something like the following
oa4mp>use clients
  clients >size
    Current store has 22 entries
  clients >exit
exiting ...
oa4mp>

Listing Object or Details of an Object.

    ls [-l | -E | -v] | [-key key] | [-keys [key1,key2,...] ] [id]

This lists every element in the current component. The default is to sort by the given identifier, except clients which are sorted by creation date. At this point, sorting behavior cannot be changed. The default format is the short form where the identifier and a little information is shown. This is for quick perusal. No argument prints out everything in the store. Giving the optional index or unique identifier will print a listing of the object. Here are the command line switches that control how much is printed:

>
  • -l = line form of the listing. This means that entries longer than a line are truncated.
  • -E = (default) list ALL the elements in the store. Default is short form
  • -v = verbose form of the listing. Entries longer than a line will be wrapped.
Example. Listing a specific object by unique identifier
oa4mp >use approvals
  approvals >ls
    0. (?) myproxy:oa4mp,2012:/adminClient/1105e0486abc6cab6d2450893d9394f6/1586446927641 by "junit" on 2020-04-09 10:42:08.0
    1. (?) myproxy:oa4mp,2012:/adminClient/11daaae5d58ec7e5ba6bb88189fc3bb8/1586446926819 by "junit" on 2020-04-09 10:42:07.0
    2. (?) myproxy:oa4mp,2012:/adminClient/131d41a607efa5be7cd611f56d1060c6/1581113647370 by "junit" on 2020-02-07 16:14:07.0
    3. (?) myproxy:oa4mp,2012:/adminClient/1332c1e8775d827e66e77c5c4c47d23f/1581513221096 by "junit" on 2020-02-12 07:13:41.0
  approvals >

By default (no arguments) it prints out the short form of every item in the store. This is the same as using the -E option. To now print out the details on number 2:

>
    approvals >ls -l 2
    approval_ts : 2020-02-07 16:14:07.0
       approved : true
       approver : junit
      client_id : myproxy:oa4mp,2012:/adminClient/131d41a607efa5be7cd611f56d1060c6/1581113647370
     public_key : k0CPPSzW0Wv5lIZWnEX5yEAOjJTZn838wSrwa-Wh6xF4Vvi4MnBJ7eQuz0QLorDoKYtLEW6LV-u7sC...
         status : none
We could have done the exact same thing by simply using the unique identifier:
    approvals>ls -l /myproxy:oa4mp,2012:/adminClient/131d41a607efa5be7cd611f56d1060c6/1581113647370
And finally, here is what a verbose version looks like for a client (we set the id first so we don't have to keep typing it in.)
  clients> set_id oa4mp:/client_id/52d39e92ab5347c880fa19f3b9cb4204
  clients> ls -v
    callback_uri : ["https://client.example.org/callback","https://client.example.org/callback2"]
             cfg : {"new":
                      "config"}
       client_id : oa4mp:/client_id/52d39e92ab5347c880fa19f3b9cb4204
     creation_ts : 2020-05-04 13:47:41.0
last_modified_ts : 2020-05-04 13:48:21.0
            name : New Test name
   proxy_limited : false
   public_client : true
      public_key : ZOq88bMz4wIxYDbqfPPSzW0Wv5lIZWnEX5yE
     rt_lifetime : 0
          scopes : ["openid"]
     sign_tokens : true
     approved by : my:adminclient/42

Other forms of ls

You may also print out a single key with the -key switch and the name of the key (list_keys will list what's available). You may also make a custom listing by specifying which keys you want to display. Using the above example with the id already set:

    clients>ls -key cfg
   cfg:
 {"new": "config"}
and to list the name, creation time and refresh token lifetime:
    clients>ls -keys [name,creation_ts,rt_lifetime]
   creation_ts : 2020-05-04 13:47:41.0
          name : New Test name
   rt_lifetime : 0

One last note is that the keys are always sorted in any listing and the default for single keys is the verbose form.

Updating (aka Editing) Objects.

    update [-key k [-value v]] [-keys [k0,k1,...] [id]
This will let you update or edit an existing object or the properties in one. If you supply no argument except an id, you will be sequentially prompted for every property of the object. The general function is shown below. Current object values are shown in [ ]'s and simply hitting return will preserve the current value.
Example. Edit a client record. In this case we want to make this client use limited proxies.
    clients >set_id testScheme:oa4md,2018:/client_id/79d0237f26f56ee831b033b7eec5865d
    clients >ls -l
        callback_uri : ["https:/baz.foo.edu/client2/7Vgs2kO-sF4/ready1","https:/baz.foo.edu/client2/7Vgs2kO-sF4/ready2"]
                 cfg : {"version":"aOfSNXcKu7VU3qPqc_lnvQ"}
           client_id : testScheme:oa4md,2018:/client_id/79d0237f26f56ee831b033b7eec5865d
         creation_ts : 2020-04-10 08:42:33.0
               email : bob@7Vgs2kO-sF4.foo.bar
           error_url : https://baz.foo.edu/home/7Vgs2kO-sF4/error
            home_url : https://baz.foo.edu/7Vgs2kO-sF4/home
    last_modified_ts : 2020-04-10 08:42:33.0
                ldap : [{"ldap":{"id":"","name":"","enabled":false,"failOnError":false,"notifyOnFail":false,"address":"foo.bar.edu","port":-...
                name : Test client 7Vgs2kO-sF4
       proxy_limited : false
       public_client : false
          public_key : t37mvHp25SkDRQqZ1mx74TiC4qIn_TRtgh5PA9P8VcTORzfJD1cKngU9yiXHMjdTNqYbaoa8vfyNWx0LfQ652A
         rt_lifetime : 456767875477
              scopes : ["openid","email","profile","org.cilogon.userinfo"]
         sign_tokens : true
    clients >

Let's change the name:

    update -key name -value "My other test client"

and let's look at the value directly

    clients>ls -key name
    My other test client

You can also update several properties at once by nassing in a list:

    clients>update -keys [sign_tokens,name]

and you will be prompted in turn for each.

Creating a new object

To create a new object, you must supply a unique identifier. This does not need to be escaped with a /. Once created, you will be given the option of editing it. You may enter any valid identifier as long as it is a valid URI
Example. Crreating a new admin client. This will create a new admin client with a specified identifier. If no identifier were supplied a random one would have been created.
    admins>create my:new:client
    Created object with id "my:new:client"
The object already exists in the store by this point. We will be given the option to edit it (equivalent to issuing the update command against its unique id):
      edit [y/n]?y
    Update the values. A return accepts the existing or default value in []'s
      enter the identifier[my:new:client]:

and you will be prompted for several more bits of information. Now the completed, new client is displayed and we are prompted if we want to keep the changes. Rather than drag you through each update, here is the finished client.

    here is the complete client:
          admin_id : foo:bar
       creation_ts : Thu Apr 23 17:19:36 CDT 2020
             email : bob@foo.bar
  last_modified_ts : 2020-04-23 17:19:36.752
       max_clients : 50
              name : My client
            secret : 126a1356375daf2951e4c8660a098f7264579853
    save [y/n]?y
    client updated.

Removing Objects From the Store.

    rm [-key k] [id]

This will remove either the value in a property or the entire client. If you specify a key, only thatproperty will be affected. If you specify the id of an object, you will be prompted to continue.

Removing an object. This will remove the object completely from the store. In the case of clients you do not need to remove the corresponding approval -- it will be done automatically. However, if you remove the approval record then the client is in effect unapproved until you re-approve it. To remove an admin client would look like the next

> Example. Removing an object by id
    admins>rm /myproxy:oa4mp,2012:/adminClient/13be753b6e816c801e212f0f1cff9d81/1587161772297
      Are you sure you want to remove this client(y/n)[n]:y
    Done. object with id = myproxy:oa4mp,2012:/adminClient/13be753b6e816c801e212f0f1cff9d81/1587161772297 has been removed from the store.
      Removing approval record
      Done. Client approval with id = myproxy:oa4mp,2012:/adminClient/13be753b6e816c801e212f0f1cff9d81/1587161772297 has been removed from the store

Note The remove command will happily remove objects by index, but remember that the indices of all other objects change, so best practice is to only remove by identifier.

Finally, to remove a value, if we had th following value

    admins>ls -key vo
    https://myorg.bigstate.edu

we could removed it by issuing

    admins>rm -key vo

Note that unlike for properties there is no prompt.

Storing objects on disk

    serialize [-file path] [id]

A common enough task is to want to do a considerable amount of editing which the direct CLI is not so well suited for, e.g. twiddling extensive lists of callbacks. This is always component specific, so you must use a comnponent for this to be available. You may serialize any object to disk in XML format, edit it with any standard text editor and then simply read it back in with the deserialize command. The basic syntax is

In this case, the index is as per any other component. The file is optional in the sense that if it is omitted, the result will be dumped at the command line. The file will be overwritten, so make sure you have it right.
Example. Serializing a client.
    cli> use clients
      clients> serialize -file /path/to/my/client.xml /client:sdfsdf:erg98540j034/456eythw456
    done!
Firing up a text editor shows the file which starts to look like this:
    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
    <properties>
    <entry key="name">Updated Test client 42</entry>
    <entry key="sign_tokens">true</entry>
    <entry key="creation_ts">2018-06-28T13:06:28.000Z</entry>
    <entry key="public_key">j75OY1FoPf1AzW5v9KDqTkxrslD1VQhQ5wdVfqUu7pO7SRoMtEwRXqBdFFtNfwmX0Z4l4vbiVRYpq9zGtoMKYw</entry>
    <entry key="rt_lifetime">456767875477</entry>
    <entry key="public_client">false</entry>
    <entry key="client_id">testScheme:oa4md,2018:/client_id/756a9e899981a4cf93f97f40a9da345a</entry>
    <entry key="home_url">https://baz.foo.edu/H2w3GevCrOU/home</entry>
    <entry key="cfg">{
      "config": "updated by converter from old LDAP entry",
      "claims": {
        "sourceConfig": [
          {
     ... lots more
Note that the format is very simple. A key is given (you cannot change these) and then the value is given as the contents of the element.

Reading objects from disk

    deserialize [-new] -file path

This will read an object from a file. This argument is required. This is always component specific, so you must use a comnponent for this to be available. You may specify it as being new, which will also tell the system to create a new identifier for it or it will reject the object if an existing identifier exists. NOTE: This will replace the object, not just update a few attributes. This means that if you just want change the value of an attribute, you have to do it manually.

Example Deserializing a file
    clients> deserialize -file /path/to/my/file
    done!

This will take the given file and replace the contents. A not uncommon use is to serialize a file, edit it and issue deserialization commands against it repeatedly as you debug it.

Invoking the components.

Clients

Invoke as
oa4mp> use clients
Operations allowed in addition to the standard ones are
  • approve [index | /uniqueID]
  • create_hash [string | -file path]
The first will prompt you for the information needed to make a new approval for the client with the given id. The second is used to create a hash for a secret that you supply. This is needed for OAuth2/OIDC clients since we store a hash and not the secret. You may either specify a file containing the secret or type in manually at the command line.

An example for setting the password

This sets the password for the client with id foo:/bar. Thanks to historical reasons, the secret has the key public_key. Its current secret is 7fd560c4169d4ed6cf08acc87d54aed28b6b92ac

clients>create_hash "my secret password"
 creating hash of my secret password
  ca90213b8b911f48a180abf8002934658bf63828
clients>update -key public_key /foo:/bar
Enter new value for public_key [7fd560c4169d4ed6cf08acc87d54aed28b6b92ac]:ca90213b8b911f48a180abf8002934658bf63828
clients>

If you list the client, you will see the updated secret.

Approvals

Invoke as
oa4mp> use approvals
There are no extra options beyond the standard ones, however, the create command takes the unique id of the client record you are going to approve. Once created, you may enter the approval information directly.

Copy

Invoke as
oa4mp>use copy
Supported operation is
  • cp source target [-verify] - copy source to target optionally verifying
This works over the entire store. It has a single command that will allow you to make a copy of one store (the source) that completely replaces another (the target). This is destructive, meaning that the every object in the target is deleted, so no merge takes place. If the -verify option is used then the two stores are compared once the copy has been completed. Note that this can be very time consuming for a large store! However, if there is any question it is at times very useful.

Appendix

Getting and installing it manually

You can just download the jar and configure it manually, though that is a lot more work than the installer. One cogent argument is incorporating the CLI into your own custome scripts. This section is how to run the CLI fromthe jara manually. If you use the installer, skip this section.

The latest version of the OA4MP command line tool is available at cli.jar You should also get the script that runs this too, cli

If you are pointing to the server configuration, you will probably need to run this as root to get access top that file. If you invoke the CLI with no arguments, it will tell you there is no configuration, but you can use all of the online help. You need the configuration file and name of the configuration generally to get a working CLI.

Running the CLI jar

invocation syntax is just like any other executable jar

java -jar $OA4MP_SERVER/lib/cli.jar  options

Command line options for the jar.

You should run this from the script, but if you want or need to specify invoke the jar directly, the invocation options they are listed here.
Argument Value required? Required? Description
-cfg Y Y The full path to the configuration file.
-name F F The name of the configuration within the file. If there is a single configuration in the file, then no name is needed. If there are multiple configurations you must specify which one to use.
-log F F The full path log file. If this is not specified then a file named "log.xml" is dumped into the invocation directory.
-use F F Specify the component to use. Rather than loading the whole CLI then use-ing a components (such as clients) you may simply specify that here.
-v F F No argument. Turns on verbose logging so that much more information is dumped onto the console. This is useful for debugging or just if you want to see what it is up to.
    java -jar cli.jar -cfg /path/to/cfg.xml -name myConfig -log /path/to/mylog.xml -use approvals
This would load the configuration named "myConfig" from the given file, write the log to a file called "mylog.xml" and bring up the approvals component.