Gateway Documentation for submitting Gateway Attributes

XSEDE requires all gateways to report the number of unique users per quarter who have executed jobs on XSEDE resources. XSEDE collects this information through a simple REST API that is integrated into the job submission process. It enables a gateway to submit the gateway user id of a job, the local job ID (obtained from the local resource manager), the submission time (also obtained from the local resource manager), and the submission host (configured by the local service provider) to special, restricted tables in the XSEDE Central Database (XDCDB). The REST API can also be used to submit optional software, usage, and VM id attributes (if applicable).

Getting started

An API key is needed to submit gateway attributes. Generate it as follows

Generate an API Key for your Gateway

  1. Logon to the Gateway attribute API interface using your XSEDE portal credentials
  2. Click Add Gateway button then set name for your Gateway (choose an appropriate name, as this cannot be changed), then click the Finalize button that will register your gateway and generate a API key.
  3. You may generate API Keys for multiple gateways that you administer
  4. You must save the API key immediately following its generation, as it cannot be retrieved later.

Save your API Key

Save your apikey to a file and protect that file as follows
$ cat > $HOME/.xsede-gateway-attributes-apikey
  apikey=YOUR_API_KEY  <-- paste apikey (note the "apikey=" at the start of the line)
  ^D <-- hit ctrl-d twice
$ chmod 0600 $HOME/.xsede-gateway-attributes-apikey <-- set permissions so nobody else can read the file
Ensure that this file is readable only by you and the API key does not appear on the command line, command line history, or logs.

Insecure API key usage

The API key must not be included directly in the command line as this is not secure. For example:
$ curl -XPOST --data-urlencode apikey=example-123456abcd

Reset your API Key

If your gateway API key needs to be regenerated, the interface provides a RESET button. That will generate a new key which will replace the existing one. You will have to then save the new key (as described above)

Using the REST API

The following example shows how to use the REST API in your gateway after subitting a batch job to a remote resource:
  $sbatch mpi.job
  ...
  Submitted batch job 4937919
From your gateway (where you stored your API key), then run the following cur command (or any HPTPP compliant client API) to submit the needed attributes to the API:
  $ curl -XPOST --data @$HOME/.xsede-gateway-attributes-apikey  \
  --data-urlencode "gatewayuser=John Doe"  \
  --data-urlencode "xsederesourcename=comet.sdsc.xsede"  \
  --data-urlencode "jobid=4937919"  \
  --data-urlencode "submittime=2019-05-31 14:32 PST" \
  https://xsede-xdcdb-api.xsede.org/gateway/v2/job_attributes
Job attributes:
  Job: 4937919
  Job submitted: 2019-05-31 14:32:00-0700
  Resource Name: comet.sdsc.xsede
  Gateway Name: xsede-example-gateway
  Gateway User: John Doe
If the job attributes were successfully submitted to the API, the API will return a summary of the attributes as shown in the example above. The attributes will be stored to a staging table in the XDCDB where it will be later matched with AMIE accounting records coming from the site. To verify that the information is matched correctly, you can use xdusage, a command-line tool available on SP resources to query the status of your XSEDE allocations. Run xdusage with "-ja" option as shown in the example below:
  $ xdusage -j -ja -s 2019-05-31 -e 2019-06-01 -p TG-STA110011S
  . . .
  job id=4939827 resource=comet.sdsc.xsede
  submit=2019-05-31@14:32:28 start=2019-05-31@17:47:28
  end=2015-05-31@17:57:37 nodecount=2 processors=32 queue=normal charge=24.89
  job-attr id= 4939827 name=gateway-user value=John Doe

Best practices

If the REST API is down for any reason, your gateway code will need to handle the resubmission of the attributes at a later time. A gateway may consider calling the REST API within a for loop until it successfully submits or marking failed submissions in a database or file and running a daily cron job to resubmit any job attributes.

List resource names

Get a list of valid resource names as follows
$ curl -XPOST --data @$HOME/.xsede-gateway-attributes-apikey \
  --data-urlencode "xsederesourcename=unknown" https://xsede-xdcdb-api.xsede.org/gateway/v2/job_attributes

Sample resource names

 bridges-gpu.psc.xsede
 bridges.psc.xsede
 comet-gpu.sdsc.xsede
 comet.sdsc.xsede
 jetstream.tacc.xsede
 stampede2.tacc.xsede

Test gateway attribute submission

Test submissions must pass the debug parameter. If set (the value doesn’t matter), the attributes are verified and reported (as described below), but they are not staged, and hence will not be associated with any job.
$ curl -XPOST --data @$HOME/.xsede-gateway-attributes-apikey  \
  --data-urlencode "gatewayuser=John Doe"  \
  --data-urlencode "xsederesourcename=comet-gpu.sdsc.xsede"  \
  --data-urlencode "jobid=bogus"  \
  --data-urlencode "submittime=2019-01-31 23:45 PST" \
  --data-urlencode "debug=x"  \
  https://xsede-xdcdb-api.xsede.org/gateway/v2/job_attributes

Sample response

 Job: bogus
 Job summited: 2019-01-31 23:45:00-0800
 Resource Name: comet-gpu.sdsc.xsede
 Gateway Name: Tesseract
 Gateway User: John Doe
 Debug specified. Attributes not staged

Review submitted gateway attributes

The last 10 submitted Gateway Attributes can be retrieved as follows (Note: The url is different than the one used for submission)
$ curl -XPOST --data @$HOME/.xsede-gateway-attributes-apikey https://xsede-xdcdb-api.xsede.org/gateway/v2/jobs

Supported gateway attributes

A set of gateway attributes are supported, the following table provides information about the required and optional attributes

Required attributes

Field Description Format Example
apikey Key assigned to the gateway String GatewayName-xyz
jobid Batch submission job id String 12345 or use.jetstream-cloud.org-51511649
submittime Approx. time when job was submitted
or the VM was created on JetStream
Date time 2019-04-01 4:25 PST
xsederesourcename Name of the resource Formatted string as: <resource>.<site>.xsede (See "List rsource names" section for the latest list of valid names)
gatewayuser First and Last name or email or username
of the gateway user for that submitted the job
String John Doe or john@example.com or jdoe

Optional attributes

Field Description Format Example
software List of software used by the job Formatted string as:
Single colon delimited key value pair
or Multiple key value pairs in a comma separated list
See "Reporting software attributes" below for format of key value pair
vmid Virtual machine ID String 5707f61b-c53b-4b1d-9059-ae6639c0f2e8-10840642
usage Time consumed by job on a virtual machine Integer (seconds) 3600
debug Flag for testing String Any value: debug=x

Reporting software attributes

Software used by the jobs may be optionally reported. Use a colon delimited key pair of the format:
  <short common software package name>:<long full software package name>
Each gateway can define their own common software names. Use a comma to separate multiple softwaere packages. For example:
  MATLAB:MATLAB R2019a, NumPy:NumPy v1.17

Using vmid and usage attributes

Vmid and usage attributes are available for reporting on non-batch systems such as JetStream, where multiple jobs may run for different gateway users under the same virtual machine. Since this granular usage per job is not known to the resource provider, it needs to be tracked by gateways, and subsequently reported.

a) usage must be reported in seconds as an integer
b) Refer to Jetstream documentation on how to fetch jobid and vmid

Examples

Sample usage for a batch system

A sample command below submits attributes for a Gateway called Tesseract for a job submitted by John Doe, a user on the gateway; which ran on XSEDE resource comet-gpu.sdsc.xsede; with a batch job id 12345; and submitted to the batch queue on 2019-01-31 23:45 PST.
$ curl -XPOST --data @$HOME/.xsede-gateway-attributes-apikey  \
  --data-urlencode "gatewayuser=John Doe"  \
  --data-urlencode "xsederesourcename=comet-gpu.sdsc.xsede"  \
  --data-urlencode "jobid=12345"  \
  --data-urlencode "submittime=2019-01-31 23:45 PST" \
  --data-urlencode "software=MATLAB:MATLAB R2019a, NumPy:NumPy v1.17"  \
  https://xsede-xdcdb-api.xsede.org/gateway/v2/job_attributes

Sample response

 Job: 12345
 Job summited: 2019-01-31 23:45:00-0800
 Resource Name: comet-gpu.sdsc.xsede
 Gateway Name: Tesseract
 Gateway User: John Doe
 Software: MATLAB:MATLAB R2019a, NumPy:NumPy v1.17

Sample usage for JetStream

A sample command below submits attributes for a Gateway called Tesseract for a job submitted by John Doe, a user on the gateway; which ran on XSEDE resource jetstream.tacc.xsede; with job id use.jetstream-cloud.org-51511649; and vmid 5707f61b-c53b-4b1d-9059-ae6639c0f2e8-108406422; and submitted to the batch queue on 2019-01-31 23:45 PST and the job used 3600 seconds of compute time.
$ curl -XPOST --data @$HOME/.xsede-gateway-attributes-apikey  \
  --data-urlencode "xsederesourcename=jetstream.tacc.xsede"  \
  --data-urlencode "jobid=use.jetstream-cloud.org-51511649"  \
  --data-urlencode "gatewayuser=John Doe" \
  --data-urlencode "submittime=2019-01-31 23:45 PST" \
  --data-urlencode "software=MATLAB:MATLAB R2019a, NumPy:NumPy v1.17"  \
  --data-urlencode "vmid=5707f61b-c53b-4b1d-9059-ae6639c0f2e8-108406422"  \
  --data-urlencode "usage=3600"  \
  https://xsede-xdcdb-api.xsede.org/gateway/v2/job_attributes

Sample response

 Job: use.jetstream-cloud.org-51511649
 Job summited: 2019-01-31 23:45:00-0800
 Resource Name: jetstream.tacc.xsede
 Gateway Name: Tesseract
 Gateway User: John Doe
 Software: MATLAB:MATLAB R2019a, NumPy:NumPy v1.17
 vmid: 5707f61b-c53b-4b1d-9059-ae6639c0f2e8-108406422
 usage: 3600

See description of vmid and usage in the Optional Atrributes above

Sample usage with incorrect input

The API responds back in plain text indicating problems with the submission, for example the following command does not include required attributes, and response shows the missing fields
$ curl -XPOST --data @$HOME/.xsede-gateway-attributes-apikey https://xsede-xdcdb-api.xsede.org/gateway/v2/job_attributes

Sample response

Errors found. Job attributes not submitted!
* Missing fields:
 jobid
 xsederesourcename
 submittime
 gatewayuser

Feedback

Please provide feedback to the XSEDE team by filling out this brief survey

Request help

If you need help with any of of this, submit a ticket via email to help@xsede.org and include 'Gateway Attribute Submission' and the server url https://xsede-xdcdb-api.xsede.org/ in your message.