cc-backend

• 1: Command Line
• 2: Configuration
• 3: Environment
• 4: REST API
• 5: Authentication Handbook
• 6: Job Archive Handbook
• 7: Schemas
• 7.1: Application Config Schema
• 7.2: Cluster Schema
• 7.3: Job Data Schema
• 7.4: Job Statistics Schema
• 7.5: Unit Schema
• 7.6: Job Archive Metadata Schema
• 7.7: Job Archive Metrics Data Schema
• 8: Tools
• 8.1: archive-manager
• 8.2: archive-migration
• 8.3: convert-pem-pubkey
• 8.4: gen-keypair
• 8.5: binaryCheckpointReader
• 8.6: grepCCLog.pl
• 8.7: Metric Generator Script

1 - Command Line

This page describes the command line options for the cc-backend executable.

-add-user <username>:[admin,support,manager,api,user]:<password>

Function: Add a new user. Only one role can be assigned.

Example: -add-user abcduser:manager:somepass

  -apply-tags

Function: Run taggers on all completed jobs and exit.

  -cleanup-checkpoints

Function: Trigger checkpoint cleanup without starting the server, then exit. Useful for maintenance windows or automated cleanup scripts.

  -config <path>

Function: Specify alternative path to config.json.

Default: ./config.json

Example: -config ./configfiles/configuration.json

  -del-user <username>

Function: Remove an existing user.

Example: -del-user abcduser

  -dev

Function: Enable development components: GraphQL Playground and Swagger UI.

  -force-db

Function: Force database version, clear dirty flag and exit.

  -gops

Function: Listen via github.com/google/gops/agent (for debugging).

  -import-job <path-to-meta.json>:<path-to-data.json>, ...

Function: Import a job. Argument format: <path-to-meta.json>:<path-to-data.json>,...

Example: -import-job ./to-import/job1-meta.json:./to-import/job1-data.json,./to-import/job2-meta.json:./to-import/job2-data.json

  -init

Function: Setup var directory, initialize sqlite database file, config.json and .env.

  -init-db

Function: Go through job-archive and re-initialize the job, tag, and jobtag tables (all running jobs will be lost!).

  -jwt <username>

Function: Generate and print a JWT for the user specified by its username.

Example: -jwt abcduser

  -logdate

Function: Set this flag to add date and time to log messages.

  -loglevel <level>

Function: Sets the logging level.

Arguments: debug | info | warn | err | crit

Default: warn

Example: -loglevel debug

  -migrate-db

Function: Migrate database to supported version and exit.

  -optimize-db

Function: Run SQLite VACUUM and ANALYZE to optimize the database, then exit.

  -revert-db

Function: Migrate database to previous version and exit.

  -server

Function: Start a server, continues listening on port after initialization and argument handling.

  -sync-ldap

Function: Sync the hpc_user table with ldap.

  -version

Function: Show version information and exit.

2 - Configuration

cc-backend requires a JSON configuration file. The configuration files is structured into sections. Every section is configured either in a separate JSON object or using a separate file. Sections are split into two categories:

• Required sections define integral settings, which are required for cc-backend to start, and work, properly.
• Optional Sections define additional options for specific use-cases or on-site requirements. We recommend to read through the available optional settings, e.g. the file archive config.

When a section is put in a separate file the section key has to have a -file suffix, example:

"auth-file": "./var/auth.json"

To override the default config file path, specify the location of a JSON configuration file with the -config <file path> command line option.

Configuration Options

Required Sections

Primary configuration sections, which key (e.g. main) has to exist on cc-backend start, or the application will shut down with an error.

Subsequent settings within the primary sections might be optional.

Section main

• addr: Type string (Optional). Address where the http (or https) server will listen on (for example: ‘0.0.0.0:80’). Default localhost:8080.
• api-allowed-ips: Type array of strings (Optional). IPv4 addresses from which the secured administrator API endpoint functions /api/* can be reached. Default: No restriction. The previous * wildcard is still supported but obsolete.
• user: Type string (Optional). Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.
• group: Type string. Drop root permissions once .env was read and the port was taken. Only applicable if using privileged port.
• disable-authentication: Type bool (Optional). Disable authentication (for everything: API, Web-UI, …). Default false.
• embed-static-files: Type bool (Optional). If all files in web/frontend/public should be served from within the binary itself (they are embedded) or not. Default true.
• static-files: Type string (Optional). Folder where static assets can be found, if embed-static-files is false. No default.
• db: Type string (Optional). The db file path. Default: ./var/job.db.
• enable-job-taggers: Type bool (Optional). Enable automatic job taggers for application and job class detection. Requires to provide tagger rules. Default: false.
• validate: Type bool (Optional). Validate all input JSON documents against JSON schema. Default: false.
• session-max-age: Type string (Optional). Specifies for how long a session shall be valid as a string parsable by time.ParseDuration(). If 0 or empty, the session/token does not expire! Default 168h.
• https-cert-file and https-key-file: Type string (Optional). If both those options are not empty, use HTTPS using those certificates. Default: No HTTPS.
• redirect-http-to: Type string (Optional). If not the empty string and addr does not end in “:80”, redirect every request incoming at port 80 to that url.
• stop-jobs-exceeding-walltime: Type int (Optional). If not zero, automatically mark jobs as stopped running X seconds longer than their walltime. Only applies if walltime is set for job. Default 0.
• short-running-jobs-duration: Type int (Optional). Do not show running jobs shorter than X seconds. Default 300.
• emission-constant: Type integer (Optional). Energy Mix CO2 Emission Constant [g/kWh]. If entered, UI displays estimated CO2 emission for job based on jobs’ total Energy.
• resampling: Type object (Optional). If configured, will enable dynamic downsampling of metric data using the configured values.
  • minimum-points: Type integer. This option allows user to specify the minimum points required for resampling; Example: 600. If minimum-points: 600, assuming frequency of 60 seconds per sample, then a resampling would trigger only for jobs > 10 hours (600 / 60 = 10).
  • resolutions: Type array [integer]. Array of resampling target resolutions, in seconds; Example: [600,300,60].
  • trigger: Type integer. Trigger next zoom level at less than this many visible datapoints.
• machine-state-dir: Type string (Optional). Where to store MachineState files. Used for persisting machine state between restarts.
• systemd-unit: Type string (Optional). Systemd unit name used for the system log viewer integration. Default: clustercockpit.
• api-subjects: Type object (Optional). NATS subjects configuration for subscribing to job and node events. When configured, the REST API endpoints for start_job and stop_job are disabled in favor of NATS messaging. Default: No NATS API.
  • subject-job-event: Type string (required). NATS subject for job events (start_job, stop_job).
  • subject-node-state: Type string (required). NATS subject for node state updates.
  • job-concurrency: Type integer (Optional). Number of concurrent worker goroutines for processing job events. Default: 8.
  • node-concurrency: Type integer (Optional). Number of concurrent worker goroutines for processing node state events. Default: 2.
• nodestate-retention: Type object (Optional). Configuration for automatic cleanup of old node state records from the database. Runs daily. Default: No retention (node states accumulate indefinitely).
  • policy: Type string (required). Retention policy. Possible values: delete (remove old records), move (archive to Parquet format then delete).
  • age: Type integer (Optional). Retention age in hours. Records older than this are affected. Default: 24.
  • target-kind: Type string (Optional). Target storage kind for Parquet archiving: file or s3. Only applicable for move policy. Default: file.
  • target-path: Type string (Optional). Filesystem path for Parquet file storage. Only applicable for target-kind file.
  • target-endpoint: Type string (Optional). S3 endpoint URL. Only applicable for target-kind s3.
  • target-bucket: Type string (Optional). S3 bucket name. Only applicable for target-kind s3.
  • target-access-key: Type string (Optional). S3 access key. Only applicable for target-kind s3.
  • target-secret-key: Type string (Optional). S3 secret key. Only applicable for target-kind s3.
  • target-region: Type string (Optional). S3 region. Only applicable for target-kind s3.
  • target-use-path-style: Type bool (Optional). Use path-style S3 addressing. Required for MinIO and some S3-compatible services. Only applicable for target-kind s3.
  • max-file-size-mb: Type integer (Optional). Maximum Parquet file size in MB before splitting into a new file. Default: 128.
• db-config: Type object (Optional). SQLite database tuning options.
  • cache-size-mb: Type integer (Optional). SQLite page cache size per connection in MB. Default: 2048.
  • soft-heap-limit-mb: Type integer (Optional). Process-wide SQLite soft heap limit in MB. Default: 16384.
  • max-open-connections: Type integer (Optional). Maximum number of open database connections. Default: 4.
  • max-idle-connections: Type integer (Optional). Maximum number of idle database connections. Default: 4.
  • max-idle-time-minutes: Type integer (Optional). Maximum idle time for a connection in minutes. Default: 10.
  • busy-timeout-ms: Type integer (Optional). SQLite busy timeout in milliseconds. When a write is blocked, SQLite retries with backoff for up to this duration before returning SQLITE_BUSY. Default: 60000.

Section auth

• jwts: Type object (required). For JWT Authentication.
  • max-age: Type string (required). Configure how long a token is valid. As string parsable by time.ParseDuration().
  • cookie-name: Type string (Optional). Cookie that should be checked for a JWT token.
  • validate-user: Type bool (Optional). Deny login for users not in database (but defined in JWT). Overwrite roles in JWT with database roles.
  • trusted-issuer: Type string (Optional). Issuer that should be accepted when validating external JWTs.
  • sync-user-on-login: Type bool (Optional). Add non-existent user to DB at login attempt with values provided in JWT.
  • update-user-on-login: Type bool (Optional). Update existent user in DB at login attempt with values provided in JWT. Name, Roles (excluding admin) and Projects are updated.
• ldap: Type object (Optional). For LDAP Authentication and user synchronisation. Default nil.
  • url: Type string (required). URL of LDAP directory server.
  • user-base: Type string (required). Base DN of user tree root.
  • search-dn: Type string (required). DN for authenticating LDAP admin account with general read rights.
  • user-bind: Type string (required). Expression used to authenticate users via LDAP bind. Must contain uid={username}.
  • user-filter: Type string (required). Filter to extract users for syncing.
  • username-attr: Type string (Optional). Attribute with full user name. Defaults to gecos if not provided.
  • sync-interval: Type string (Optional). Interval used for syncing local user table with LDAP directory. Parsed using time.ParseDuration.
  • uid-attr: Type string (Optional). LDAP attribute used as login username. Defaults to uid if not provided.
  • sync-del-old-users: Type bool (Optional). Delete obsolete users in database.
  • sync-user-on-login: Type bool (Optional). Add non-existent user to DB at login attempt if user exists in LDAP directory.
  • update-user-on-login: Type bool. Update existent user in DB at login attempt with values provided. Name, Roles (excluding admin) and Projects are updated.
• oidc: Type object (Optional). For OpenID Connect Authentication. Default nil.
  • provider: Type string (required). OpenID Connect provider URL.
  • sync-user-on-login: Type bool. Add non-existent user to DB at login attempt with values provided.
  • update-user-on-login: Type bool. Update existent user in DB at login attempt with values provided. Name, Roles (excluding admin) and Projects are updated.

Section metric-store

• retention-in-memory: Type string (required). Keep the metrics within memory for given time interval. Retention for X hours, then the metrics would be freed. Buffers that are still used by running jobs will be kept.
• memory-cap: Type integer (required). If memory used exceeds value in GB, buffers still used by long running jobs will be freed.
• num-workers: Type integer (Optional). Number of concurrent workers for checkpoint and archive operations. Default: If not set defaults to min(runtime.NumCPU()/2+1, 10)
• checkpoints: Type object (required). Configuration for checkpointing the metrics buffers
  • file-format: Type string (Optional). Format to use for checkpoint files. Can be json (human-readable, periodic) or wal (binary snapshot + Write-Ahead Log, crash-safe). Default: wal.
  • directory: Type string (Optional). Path in which the checkpoints should be placed. Default: ./var/checkpoints.
  • max-wal-size: Type integer (Optional). Maximum size in bytes for a single host’s WAL file. When exceeded, the WAL is force-rotated to prevent unbounded disk growth. Default: 0 (unlimited).
• cleanup: Type object (Optional). Configuration for the cleanup process. The cleanup interval always equals the retention-in-memory interval. If not set, the mode defaults to delete.
  • mode: Type string (Optional). The mode for cleanup. Can be delete or archive. Default: delete.
  • directory: Type string (required if mode is archive). Directory where to put the archive files.
• nats-subscriptions: Type array (Optional). List of NATS subjects the metric store should subscribe to. Items are of type object with the following attributes:
  • subscribe-to: Type string (required). NATS subject to subscribe to.
  • cluster-tag: Type string (Optional). Allow lines without a cluster tag, use this as default.

Section cron

• commit-job-worker: Type string. Frequency of commit job worker. Default: 2m
• duration-worker: Type string. Frequency of duration worker. Default: 5m
• footprint-worker: Type string. Frequency of footprint. Default: 10m

Optional Sections

Secondary configuration sections, which key (e.g. nats) can be missing from the configuration without interfering with cc-backend starts.

Subsequent settings within the secondary sections might be optional.

Section archive

If section is not provided, the default is kind set to file with path set to ./var/job-archive.

• kind: Type string (required). Set archive backend. Supported values: file, s3, sqlite.
• path: Type string (Optional). Path to the job-archive. Only applicable for file backend. Default: ./var/job-archive.
• db-path: Type string (Optional). Path to SQLite database file. Only applicable for sqlite backend.
• endpoint: Type string (Optional). S3 endpoint URL. Only applicable for s3 backend. Required for S3-compatible services like MinIO.
• access-key: Type string (Optional). S3 access key ID. Only applicable for s3 backend.
• secret-key: Type string (Optional). S3 secret access key. Only applicable for s3 backend.
• bucket: Type string (Optional). S3 bucket name. Only applicable for s3 backend.
• region: Type string (Optional). S3 region. Only applicable for s3 backend.
• use-path-style: Type bool (Optional). Use path-style S3 URLs. Required for MinIO and some S3-compatible services. Only applicable for s3 backend.
• compression: Type integer (Optional). Setup automatic compression for jobs older than number of days. Default: 7.
• retention: Type object (Optional). Enable retention policy for archive and database. Retention jobs run once daily at fixed times.
  • policy: Type string (required). Retention policy. Possible values: none (disabled), delete (remove from archive and optionally DB), copy (copy to target without removing source), move (copy to target then remove source).
  • format: Type string (Optional). Output format for copy and move policies. Possible values: json (standard archive format, default), parquet (columnar Parquet format for long-term storage).
  • include-db: Type bool (Optional). Also remove jobs from database when deleting from archive. Default: true.
  • omit-tagged: Type string (Optional). Control which tagged jobs are skipped by the retention policy. Possible values: none (apply retention to all jobs, default), all (skip any job that has at least one tag), user (skip jobs that have user-created tags; auto-tagger tags of type app or jobClass do not count as user tags).
  • age: Type integer (Optional). Act on jobs with startTime older than age (in days). Default: 7.
  • target-kind: Type string (Optional). Target storage kind for copy and move policies: file or s3. Default: file.
  • target-path: Type string (Optional). Filesystem path for the target storage. Only applicable for target-kind file.
  • target-endpoint: Type string (Optional). S3 endpoint URL for target. Only applicable for target-kind s3.
  • target-bucket: Type string (Optional). S3 bucket name for target. Only applicable for target-kind s3.
  • target-access-key: Type string (Optional). S3 access key for target. Only applicable for target-kind s3.
  • target-secret-key: Type string (Optional). S3 secret key for target. Only applicable for target-kind s3.
  • target-region: Type string (Optional). S3 region for target. Only applicable for target-kind s3.
  • target-use-path-style: Type bool (Optional). Use path-style S3 URLs for target. Only applicable for target-kind s3.
  • max-file-size-mb: Type integer (Optional). Maximum Parquet file size in MB before splitting into a new file. Only applicable when format is parquet. Default: 512.

Section nats

• address: Type string. Address of the NATS server (e.g., nats://localhost:4222).
• username: Type string (Optional). Username for NATS authentication.
• password: Type string (Optional). Password for NATS authentication (optional).
• creds-file-path: Type string (Optional). Path to NATS credentials file for authentication (optional).

Section metric-store-external

Configures external cc-metric-store instances for reading metric data. This is an array of objects, each mapping a scope (cluster name or * wildcard) to an external metric store URL. When configured alongside the internal metric-store section, the external stores extend the available metric sources.

Each array entry has the following properties:

• scope: Type string (required). Scope identifier for routing metric queries. Use a cluster name to route queries for that specific cluster, or * as a default fallback for any unmatched cluster.
• url: Type string (required). URL of the external cc-metric-store endpoint (e.g., http://host:8082).
• token: Type string (required). Authentication token (JWT) for the external metric store.

Example:

"metric-store-external": [
  {
    "scope": "*",
    "url": "http://metricstore-default:8082",
    "token": "eyJhbGci..."
  },
  {
    "scope": "fritz",
    "url": "http://metricstore-fritz:8084",
    "token": "eyJhbGci..."
  }
]

Section ui

The ui section specifies defaults for the web user interface. The defaults which metrics to show in different views can be overwritten per cluster or subcluster.

• job-list: Type object (Optional). Job list defaults. Applies to user and jobs views.
  • use-paging: Type bool (Optional). If classic paging is used instead of continuous scrolling by default.
  • show-footprint: Type bool (Optional). If footprint bars are shown as first column by default.
• node-list: Type object (Optional). Node list defaults. Applies to node list view.
  • use-paging: Type bool (Optional). If classic paging is used instead of continuous scrolling by default.
• job-view: Type object (Optional). Job view defaults.
  • show-polar-plot: Type bool (Optional). If the job metric footprints polar plot is shown by default.
  • show-footprint: Type bool (Optional). If the annotated job metric footprint bars are shown by default.
  • show-roofline: Type bool (Optional). If the job roofline plot is shown by default.
  • show-stat-table: Type bool (Optional). If the job metric statistics table is shown by default.
• metric-config: Type object (Optional). Global initial metric selections for primary views of all clusters.
  • job-list-metrics: Type array [string] (Optional). Initial metrics shown for new users in job lists (User and jobs view).
  • job-view-plot-metrics: Type array [string] (Optional). Initial metrics shown for new users as job view metric plots.
  • job-view-table-metrics: Type array [string] (Optional). Initial metrics shown for new users in job view statistics table.
  • clusters: Type array of objects (Optional). Overrides for global defaults by cluster and subcluster.
    • name: Type string (required). The name of the cluster.
    • job-list-metrics: Type array [string] (Optional). Initial metrics shown for new users in job lists (User and jobs view) for this cluster.
    • job-view-plot-metrics: Type array [string] (Optional). Initial metrics shown for new users as job view timeplots for this cluster.
    • job-view-table-metrics: Type array [string] (Optional). Initial metrics shown for new users in job view statistics table for this cluster.
    • sub-clusters: Type array of objects (Optional). The array of overrides per subcluster.
      • name: Type string (required). The name of the subcluster.
      • job-list-metrics: Type array [string] (Optional). Initial metrics shown for new users in job lists (User and jobs view) for subcluster.
      • job-view-plot-metrics: Type array [string] (Optional). Initial metrics shown for new users as job view timeplots for subcluster.
      • job-view-table-metrics: Type array [string] (Optional). Initial metrics shown for new users in job view statistics table for subcluster.
• plot-configuration: Type object (Optional). Initial settings for plot render options.
  • color-background: Type bool (Optional). If the metric plot backgrounds are initially colored by threshold limits.
  • plots-per-row: Type integer (Optional). How many plots are initially rendered per row. Applies to job, single node, and analysis views.
  • line-width: Type integer (Optional). Initial thickness of rendered plotlines. Applies to metric plot, job compare plot and roofline.
  • color-scheme: Type array [string] (Optional). Initial colorScheme to be used for metric plots.

3 - Environment

All security-related configurations, e.g. keys and passwords, are set using environment variables. It is supported to set these by means of a .env file in the project root.

Environment Variables

• JWT_PUBLIC_KEY and JWT_PRIVATE_KEY: Base64 encoded Ed25519 keys used for JSON Web Token (JWT) authentication. You can generate your own keypair using go run ./tools/gen-keypair/. The release binaries also include the gen-keypair tool for x86-64. For more information, see the JWT documentation.
• SESSION_KEY: Some random bytes used as secret for cookie-based sessions
• LDAP_ADMIN_PASSWORD: The LDAP admin user password (optional)
• CROSS_LOGIN_JWT_PUBLIC_KEY: Base64 encoded Ed25519 public key for accepting JWTs generated by an external authentication service (optional). Keys in PEM format can be converted using tools/convert-pem-pubkey.
• CROSS_LOGIN_JWT_HS512_KEY: Used for token based logins via another authentication service (optional)
• OID_CLIENT_ID: OpenID connect client id (optional)
• OID_CLIENT_SECRET: OpenID connect client secret (optional)

Template .env file

Below is an example .env file. Copy it as .env into the project root and adapt it for your needs.

# Base64 encoded Ed25519 keys (DO NOT USE THESE TWO IN PRODUCTION!)
# You can generate your own keypair using `go run tools/gen-keypair/main.go`
JWT_PUBLIC_KEY="kzfYrYy+TzpanWZHJ5qSdMj5uKUWgq74BWhQG6copP0="
JWT_PRIVATE_KEY="dtPC/6dWJFKZK7KZ78CvWuynylOmjBFyMsUWArwmodOTN9itjL5POlqdZkcnmpJ0yPm4pRaCrvgFaFAbpyik/Q=="

# Base64 encoded Ed25519 public key for accepting externally generated JWTs
# Keys in PEM format can be converted, see `tools/convert-pem-pubkey/Readme.md`
CROSS_LOGIN_JWT_PUBLIC_KEY=""

# Some random bytes used as secret for cookie-based sessions (DO NOT USE THIS ONE IN PRODUCTION)
SESSION_KEY="67d829bf61dc5f87a73fd814e2c9f629"

# Password for the ldap server (optional)
LDAP_ADMIN_PASSWORD="mashup"

4 - REST API

REST API Authorization

In ClusterCockpit JWTs are signed using a public/private key pair using ED25519. Because tokens are signed using public/private key pairs, the signature also certifies that only the party holding the private key is the one that signed it. JWT tokens in ClusterCockpit are not encrypted, means all information is clear text. Expiration of the generated tokens can be configured in config.json using the max-age option in the auth.jwts object. Example:

"auth": {
  "jwts": {
    "max-age": "168h"
  }
}

The party that generates and signs JWT tokens has to be in possession of the private key and any party that accepts JWT tokens must possess the public key to validate it. cc-backed therefore requires both keys, the private one to sign generated tokens and the public key to validate tokens that are provided by REST API clients.

Generate ED25519 key pairs

We provide a tool as part of cc-backend to generate a ED25519 keypair. The tool is called gen-keypair and provided as part of the release binaries. You can easily build it yourself in the cc-backend source tree with:

go build tools/gen-keypair

To use it just call it without any arguments:

./gen-keypair

Usage of Swagger UI documentation

Swagger UI is a REST API documentation and testing framework. To use the Swagger UI for testing you have to run an instance of cc-backend on localhost (and use the default port 8080):

./cc-backend -server

You may want to start the demo as described here . This Swagger UI is also available as part of cc-backend if you start it with the dev option:

./cc-backend -server -dev

You may access it at this URL.

Conditional Endpoints

When api-subjects is configured in the main section of config.json (i.e., NATS messaging is enabled for job events), the REST API endpoints /api/jobs/start_job/ and /api/jobs/stop_job/ are disabled. Job start/stop operations are then handled exclusively via NATS. All other REST endpoints remain available regardless of NATS configuration.

API Endpoint Groups

The REST API is organized into several route groups:

• Admin API (/api/): Full job and cluster management, requires admin/API role JWT.
• User API (/userapi/): Read-only job query endpoints for regular users.
• Metric Store API (/metricstore/): Metric data ingestion, health checks, and debugging endpoints.
• Config API (/config/): User management and configuration, uses session authentication.
• Frontend API (/frontend/): JWT generation and user config updates, uses session authentication.

Swagger API Reference

5 - Authentication Handbook

Introduction

cc-backend supports the following authentication methods:

• Local login, with credentials stored in SQL database
• LDAP login, with authentication to a LDAP directory
• OpenID Connect login, with authentication against a KeyCloak instance
• JWT login, with authentication via JSON Web Token:
  • With token provided in HTML request header
  • With token provided in cookie

All above methods create a session cookie that is then used for subsequent authentication of requests. Multiple authentication methods can be configured at the same time. If LDAP is enabled it takes precedence over local authentication. The OpenID Connect method against a KeyCloak instance enables many more authentication methods using the ability of KeyCloak to act as an Identity Broker.

The REST API uses stateless authentication via a JWT token, which means that every requests must be authenticated.

Authorization control

cc-backend uses roles to decide if a user is authorized to access certain information. The roles and their rights are described in more detail here.

General configuration options

All configuration is part of the cc-backend configuration file config.json. The primary key for authentication configuration options is auth. All security sensitive options as passwords and tokens are passed in terms of environment variables. cc-backend supports to read an .env file upon startup and set the environment variables contained there.

Duration of session

Per default the maximum duration of a session is 7 days. To change this the option main.session-max-age has to be set to a string that can be parsed by the Golang time.ParseDuration() function. For most use cases the largest unit h is the only relevant option.

To enable unlimited session duration set main.session-max-age either to 0 or empty string.

Example

"main": {
  "session-max-age": "24h",
}

Local authentication

No configuration is required for local authentication.

Usage

You can add an user on the command line using the flag -add-user:

./cc-backend -add-user <username>:<roles>:<password>

Example:

./cc-backend -add-user fritz:admin,api:myPass

Roles can be admin, support, manager, api, and user.

Users can be deleted using the flag -del-user:

./cc-backend -del-user fritz

LDAP authentication

Configuration

To enable LDAP authentication the following set of options are required as attributes of the auth.ldap JSON object:

• url: URL of the LDAP directory server. This must be a complete URL including the protocol and not only the host name. Example: ldaps://ldsrv.mydomain.com.
• user-base: Base DN of user tree root. Example: ou=people,ou=users,dc=rz,dc=mydomain,dc=com.
• search-dn: DN for authenticating an LDAP admin account with general read rights. This is required for the sync on login and the sync options. Example: cn=monitoring,ou=adm,ou=profile,ou=manager,dc=rz,dc=mydomain,dc=com
• user-bind: Expression used to authenticate users via LDAP bind. Must contain uid={username}. Example: uid={username},ou=people,ou=users,dc=rz,dc=mydomain,dc=com.
• user-filter: Filter to extract users for syncing. Example: (&(objectclass=posixAccount)).

Optional configuration options are:

• username-attr: Attribute with full user name. Defaults to gecos if not provided.
• sync-interval: Interval used for syncing SQL user table with LDAP directory. Parsed using time.ParseDuration. The sync interval is always relative to the time cc-backend was started. Example: 24h.
• sync-del-old-users: Type boolean. Delete users in SQL database if not in LDAP directory anymore. This of course only applies to users that were added from LDAP.
• sync-user-on-login: Type boolean. Add non-existent user to database at login attempt if user exists in LDAP directory. This option enables that users can login at once after they are added to the LDAP directory. Does not update user on recurring LDAP logins.
• update-user-on-login: Type boolean. Update existent users in DB at login attempt if user exists in LDAP directory. This option updates changed source attributes, for example the name, if the database value differs. Does not add users on first-time LDAP login.

Example

"auth": {
  "ldap": {
    "url": "ldaps://ldsrv.mydomain.com",
    "user-base": "ou=people,ou=users,dc=rz,dc=mydomain,dc=com",
    "search-dn": "cn=monitoring,ou=adm,ou=profile,ou=manager,dc=rz,dc=mydomain,dc=com",
    "user-bind": "uid={username},ou=people,ou=users,dc=rz,dc=mydomain,dc=com",
    "user-filter": "(&(objectclass=posixAccount))"
  },
}

Environment

The LDAP authentication method requires the environment variable LDAP_ADMIN_PASSWORD for the search-dn account that is used to sync users.

Usage

If LDAP is configured it is the first authentication method that is tried if a user logs in using the login form. A sync with the LDAP directory can also be triggered from the command line using the flag -sync-ldap.

OpenID Connect authentication

KeyCloak configuration

The OpenID Connect implementation is always tested against the latest KeyCloak (currently 26.5.6) provider. The KeyCloak admin UI frequently changes, if you are on an older version use web search to find the settings described below.

Steps to setup KeyCloak:

• Create a new realm. This will determine the provider URL.
• Create a new OpenID Connect client
  • Set a Client ID: This a arbitrary string, e.g. cc
  • For Capability config settings set:
    • Enable Client authentication
    • Leave Authentication flow at Standard flow
    • For PKCE Method choose S256
  • For Login settings set:
    • Root URL: This is the base URL of your cc-backend instance.
    • Valid redirect URLs: Set this to oidc-callback.
      • Add an additional URL including the full HTTP path, e.g. http://monitoring.example.com/oidc-callback
      • If HTTPS is used, also add the HTTPS path, e.g. https://monitoring.example.com/oidc-callback
    • Web origins: Set this also to the base URL of your cc-backend instance.

Everything else can be left to the default.

Do not forget to create users in your realm before testing.

Per default all OpenID Connect authenticated users will have the user role only. If you want to also set elevated roles via OpenID Connect, you need to adapt the default configuration in KeyCloak. KeyCloak is a very flexible tool and there are many ways to set this up. One way is to setup a mapper that sets the realm_access.roles token claim and adds it to the ID token. This is not the default and has to be explicitly configured. And of course the ClusterCockpit roles have to be also assigned to the user.

cc-backend configuration

To enable OpenID Connect authentication create a oidc object below the top-level auth key and set the following attributes:

• provider: The base URL of your OpenID Connect provider. Example: https://auth.example.com/realms/clustercockpit.

Optional configuration options are:

• sync-user-on-login: Type boolean. Add non-existent user to DB at login attempt if user exists in KeyCloak realm. This option enables that users can login at once after they are added to the KeyCloak realm. Does not update user on recurring OIDC logins.
• update-user-on-login: Type boolean. Update existent users in DB at login attempt if user exists in KeyCloak realm. This option updates changed source attributes, for example the name, if the database value differs. Does not add users on first-time OIDC login.

Configuration Example

"auth": {
  "oidc": {
    "provider": "https://auth.server.com:8080/realms/clustercockpit",
    "sync-user-on-login": true,
    "update-user-on-login": true
  },
}

Environment

Furthermore the following environment variables have to be set (in the .env file):

• OID_CLIENT_ID: Set this to the Client ID you configured in Keycloak.
• OID_CLIENT_SECRET: Set this to the Client ID secret available in your Keycloak Open ID Client configuration at the Credentials tab.

Usage

If the auth.oidc config key is correctly set and the required environment variables are available, an additional button for OpenID Connect Login is shown below the login mask. If pressed this button will redirect to the OpenID Connect login.

[... CONTENT ...]
<button type="submit" class="btn btn-success">Submit</button>
{{if .Infos.hasOpenIDConnect}}
<a class="btn btn-primary" href="/oidc-login">OpenID Connect Login</a>
{{end}} [...CONTENT...]

JWT token authentication

JSON web tokens are a standardized method for representing encoded claims securely between two parties. In ClusterCockpit they are used for authorization to use REST APIs as well as a method to delegate authentication to a third party. This section only describes JWT based authentication for initiating a user session.

Two variants exist:

• [1] Session Authenticator: Passes JWT token in the HTTP header Authorization using the Bearer prefix or using the query key login-token.

Example for Authorization header:

Authorization: Bearer S0VLU0UhIExFQ0tFUiEK

Example for query key used as form action in external application:

<form
  method="post"
  action="$CCROOT/jwt-login?login-token=S0VLU0UhIExFQ0tFUiEK"
  target="_blank"
>
  <button type="submit">Access CC</button>
</form>

• [2] Cookie Session Authenticator: Reads the JWT token from a named cookie provided by the request, which is deleted after the session was successfully initiated. This is a more secure alternative to the standard header based solution.

JWT Configuration

• [0] Basic required configuration:

In order to enable JWT based transactions generally, the following has to be true:

• The jwts JSON object has to exist within config.json, even if no other attribute is set within.
  • We recommend to set max-age attribute: Specifies for how long a JWT token shall be valid, defined as a string parsable by time.ParseDuration().
  • This will only affect JWTs generated by ClusterCockpit, e.g. for the use with REST-API endpoints.

In addition, the the following environment variables are used:

• JWT_PRIVATE_KEY: The applications own private key to be used with JWT transactions. Required for cookie based logins and REST-API communication.

• JWT_PUBLIC: The applications own public key to be used with JWT transactions. Required for cookie based logins and REST-API communication.

• [1] Configuration for JWT Session Authenticator:

Compatible signing methods are: HS256, HS512

Only a shared (symmetric) key saved as environment variable CROSS_LOGIN_JWT_HS512_KEY is required.

• [2] Configuration for JWT Cookie Session Authenticator:

Tokens are signed with: Ed25519/EdDSA

To enable JWT authentication via cookie the following set of options are required as attributes of the jwts JSON object:

• cookie-name (String): Specifies which cookie should be checked for a JWT token (if no authorization header is present)
• trusted-issuer (String): Specifies which issuer should be accepted when validating external JWTs (iss-claim)

In addition, the Cookie Session Authenticator method requires the following environment variable:

• CROSS_LOGIN_JWT_PUBLIC_KEY: Primary public key for this method, validates identity of tokens received from trusted-issuer and must therefore match accordingly.

• [3] Optional configuration attributes of the jwts JSON object, valid for both [1] and [2], are:

• validate-user (Bool): Load user by username encoded in sub-claim from database, including roles, denying login if not matched in database. Ignores all other claims. By design not combinable with both sync-user-on-login and/or update-user-on-login options.

• sync-user-on-login (Bool): If user encoded in token does not exist in database, add a new user entry. Does not update user on recurring JWT logins.

• update-user-on-login (Bool): If user encoded in token does exist in database, update the user entry with all encoded information. Does not add users on first-time JWT login.

JWT Usage

• [1] Usage for JWT Session Authenticator:

The endpoint for initiating JWT logins in ClusterCockpit is /jwt-login

For login with JWT Header, the header has to include the Authorization: Bearer $TOKEN information when accessing this endpoint. For login with JWT request parameter, the external website has to submit an action with the parameter ?login-token=$TOKEN (See example above).

In both cases, the JWT should contain the following parameters:

• sub: The subject, in this case this is the username. Will be used for user matching if validate-user is set.

• exp: Expiration in Unix epoch time. Can be small as the token is only used during login.

• name: The full name of the person assigned to this account. Will be used to update user table.

• roles: String array with roles of user.

• projects: [Optional] String array with projects of user. Relevant if user has manager-role.

• [2] Usage for JWT Cookie Session Authenticator:

The token must be set within a cookie with a name matching the configured cookie-name.

The JWT should then contain the following parameters:

• sub: The subject, in this case this is the username. Will be used for user matching if validate-user is set.
• exp: Expiration in Unix epoch time. Can be small as the token is only used during login.
• name: The full name of the person assigned to this account. Will be used to update user table.
• roles: String array with roles of user.

6 - Job Archive Handbook

The job archive specifies an exchange format for job meta and performance metric data. It consists of two parts:

• a Json file format
• a Directory hierarchy / Key specification

By using an open, portable and simple specification based on JSON objects it is possible to exchange job performance data for research and analysis purposes as well as use it as a robust way for archiving job performance data.

The current release supports new SQLite and S3 object store based job archive backends. Those are still experimental and for production we still recommend to use the proven file based job archive. One major disadvantage of the file based job archive backend is that for large job counts it will consume a lot of inodes.

Trying the new job-archive backends

We provide the tool archive-manager that allows to convert between different job-archive formats. This allows to convert your existing file-based job-archive into either a SQLite or S3 variant. Please be aware that for large archives this may take a long time. You can find details about how to use this tool in the archive-manager reference documentation.

Specification for file path / key

To manage the number of directories within a single directory a tree approach is used splitting the integer job ID. The job id is split in junks of 1000 each. Usually 2 layers of directories is sufficient but the concept can be used for an arbitrary number of layers.

For a 2 layer schema this can be achieved with (code example in Perl):

$level1 = $jobID/1000;
$level2 = $jobID%1000;
$dstPath = sprintf("%s/%s/%d/%03d", $trunk, $destdir, $level1, $level2);

While for the SQLite and S3 object store based backend the systematic to introduce layers is obsolete we kept it to keep the naming consistent. This means what is the path in case of the file based backend is used as a object key and column value there.

Example

For the job ID 1034871 on cluster large with start time 1768978339 the key is ./large/1034/871/1768978339.

Create a Job archive from scratch

In case you place the job-archive in the ./var folder create the folder with:

mkdir -p ./var/job-archive

The job-archive is versioned, the current version is documented in the Release Notes. Currently you have to create the version file manually when initializing the job-archive:

echo 3 > ./var/job-archive/version.txt

Directory layout

ClusterCockpit supports multiple clusters, for each cluster you need to create a directory named after the cluster and a cluster.json file specifying the metric list and hardware partitions within the clusters. Hardware partitions are subsets of a cluster with homogeneous hardware (CPU type, memory capacity, GPUs) that are called subclusters in ClusterCockpit.

For above configuration the job archive directory hierarchy looks like the following:

./var/job-archive/
     version.txt
     fritz/
        cluster.json
     alex/
        cluster.json
     woody/
        cluster.json

You find help how-to create a cluster.json file in the How to create a cluster.json file guide.

Json file format

Overview

Every cluster must be configured in a cluster.json file.

The job data consists of two files:

• meta.json: Contains job meta information and job statistics.
• data.json: Contains complete job data with time series

The description of the json format specification is available as [[json schema|https://json-schema.org/]] format file. The latest version of the json schema is part of the cc-backend source tree. For external reference it is also available in a separate repository.

Specification cluster.json

The json schema specification in its raw format is available at the cc-lib GitHub repository. A variant rendered for better readability is found in the references.

Specification meta.json

The json schema specification in its raw format is available at the cc-lib GitHub repository. A variant rendered for better readability is found in the references.

Specification data.json

The json schema specification in its raw format is available at the cc-lib GitHub repository. A variant rendered for better readability is found in the references.

Metric time series data is stored for a fixed time step. The time step is set per metric. If no value is available for a metric time series data timestamp null is entered.

7 - Schemas

ClusterCockpit Schema References for

• Application Configuration
• Cluster Configuration
• Job Data
• Job Statistics
• Units
• Job Archive Job Metadata
• Job Archive Job Metricdata

The schemas in their raw form can be found in the ClusterCockpit GitHub repository.

7.1 - Application Config Schema

A detailed description of each configuration option can be found in the configuration reference.

The configuration is split into sections. Each section is validated against its own JSON schema defined in the corresponding Go package inside the cc-backend repository.

Section main

Source: internal/config/schema.go

resampling

api-subjects

nodestate-retention

db-config

Section auth

Source: internal/auth/schema.go

auth.jwts

auth.ldap

auth.oidc

Section metric-store

Source: pkg/metricstore/configSchema.go

metric-store.checkpoints

metric-store.cleanup

metric-store.nats-subscriptions items

Section cron

Source: internal/taskmanager/taskManager.go

Section archive

Source: pkg/archive/ConfigSchema.go

archive.retention

Section nats

Source: cc-lib (external library)

Section metric-store-external

Source: internal/metricdispatch/configSchema.go

An array of external cc-metric-store instances for reading metric data. Each entry maps a scope (cluster name or * wildcard) to an external metric store.

Section ui

Source: web/configSchema.go

ui.job-list

ui.node-list

ui.job-view

ui.metric-config

Global initial metric selections for all clusters (overridable per cluster/subcluster).

ui.metric-config.clusters[].sub-clusters items

ui.plot-configuration

7.2 - Cluster Schema

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

HPC cluster description

• 1. Property HPC cluster description > name
• 2. Property HPC cluster description > metricConfig
  • 2.1. HPC cluster description > metricConfig > metricConfig items
    • 2.1.1. Property HPC cluster description > metricConfig > metricConfig items > name
    • 2.1.2. Property HPC cluster description > metricConfig > metricConfig items > unit
    • 2.1.3. Property HPC cluster description > metricConfig > metricConfig items > scope
    • 2.1.4. Property HPC cluster description > metricConfig > metricConfig items > timestep
    • 2.1.5. Property HPC cluster description > metricConfig > metricConfig items > aggregation
    • 2.1.6. Property HPC cluster description > metricConfig > metricConfig items > footprint
    • 2.1.7. Property HPC cluster description > metricConfig > metricConfig items > energy
    • 2.1.8. Property HPC cluster description > metricConfig > metricConfig items > lowerIsBetter
    • 2.1.9. Property HPC cluster description > metricConfig > metricConfig items > peak
    • 2.1.10. Property HPC cluster description > metricConfig > metricConfig items > normal
    • 2.1.11. Property HPC cluster description > metricConfig > metricConfig items > caution
    • 2.1.12. Property HPC cluster description > metricConfig > metricConfig items > alert
    • 2.1.13. Property HPC cluster description > metricConfig > metricConfig items > subClusters
      • 2.1.13.1. HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items
        • 2.1.13.1.1. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > name
        • 2.1.13.1.2. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > footprint
        • 2.1.13.1.3. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > energy
        • 2.1.13.1.4. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > lowerIsBetter
        • 2.1.13.1.5. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > peak
        • 2.1.13.1.6. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > normal
        • 2.1.13.1.7. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > caution
        • 2.1.13.1.8. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > alert
        • 2.1.13.1.9. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > remove
• 3. Property HPC cluster description > subClusters
  • 3.1. HPC cluster description > subClusters > subClusters items
    • 3.1.1. Property HPC cluster description > subClusters > subClusters items > name
    • 3.1.2. Property HPC cluster description > subClusters > subClusters items > processorType
    • 3.1.3. Property HPC cluster description > subClusters > subClusters items > socketsPerNode
    • 3.1.4. Property HPC cluster description > subClusters > subClusters items > coresPerSocket
    • 3.1.5. Property HPC cluster description > subClusters > subClusters items > threadsPerCore
    • 3.1.6. Property HPC cluster description > subClusters > subClusters items > flopRateScalar
      • 3.1.6.1. Property HPC cluster description > subClusters > subClusters items > flopRateScalar > unit
      • 3.1.6.2. Property HPC cluster description > subClusters > subClusters items > flopRateScalar > value
    • 3.1.7. Property HPC cluster description > subClusters > subClusters items > flopRateSimd
      • 3.1.7.1. Property HPC cluster description > subClusters > subClusters items > flopRateSimd > unit
      • 3.1.7.2. Property HPC cluster description > subClusters > subClusters items > flopRateSimd > value
    • 3.1.8. Property HPC cluster description > subClusters > subClusters items > memoryBandwidth
      • 3.1.8.1. Property HPC cluster description > subClusters > subClusters items > memoryBandwidth > unit
      • 3.1.8.2. Property HPC cluster description > subClusters > subClusters items > memoryBandwidth > value
    • 3.1.9. Property HPC cluster description > subClusters > subClusters items > nodes
    • 3.1.10. Property HPC cluster description > subClusters > subClusters items > topology
      • 3.1.10.1. Property HPC cluster description > subClusters > subClusters items > topology > node
        • 3.1.10.1.1. HPC cluster description > subClusters > subClusters items > topology > node > node items
      • 3.1.10.2. Property HPC cluster description > subClusters > subClusters items > topology > socket
        • 3.1.10.2.1. HPC cluster description > subClusters > subClusters items > topology > socket > socket items
          • 3.1.10.2.1.1. HPC cluster description > subClusters > subClusters items > topology > socket > socket items > socket items items
      • 3.1.10.3. Property HPC cluster description > subClusters > subClusters items > topology > memoryDomain
        • 3.1.10.3.1. HPC cluster description > subClusters > subClusters items > topology > memoryDomain > memoryDomain items
          • 3.1.10.3.1.1. HPC cluster description > subClusters > subClusters items > topology > memoryDomain > memoryDomain items > memoryDomain items items
      • 3.1.10.4. Property HPC cluster description > subClusters > subClusters items > topology > die
        • 3.1.10.4.1. HPC cluster description > subClusters > subClusters items > topology > die > die items
          • 3.1.10.4.1.1. HPC cluster description > subClusters > subClusters items > topology > die > die items > die items items
      • 3.1.10.5. Property HPC cluster description > subClusters > subClusters items > topology > core
        • 3.1.10.5.1. HPC cluster description > subClusters > subClusters items > topology > core > core items
          • 3.1.10.5.1.1. HPC cluster description > subClusters > subClusters items > topology > core > core items > core items items
      • 3.1.10.6. Property HPC cluster description > subClusters > subClusters items > topology > accelerators
        • 3.1.10.6.1. HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items
          • 3.1.10.6.1.1. Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > id
          • 3.1.10.6.1.2. Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > type
          • 3.1.10.6.1.3. Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > model

Title: HPC cluster description

Description: Meta data information of a HPC cluster

1. Property HPC cluster description > name

Description: The unique identifier of a cluster

2. Property HPC cluster description > metricConfig

Description: Metric specifications

2.1. HPC cluster description > metricConfig > metricConfig items

2.1.1. Property HPC cluster description > metricConfig > metricConfig items > name

Description: Metric name

2.1.2. Property HPC cluster description > metricConfig > metricConfig items > unit

Description: Metric unit

2.1.3. Property HPC cluster description > metricConfig > metricConfig items > scope

Description: Native measurement resolution

2.1.4. Property HPC cluster description > metricConfig > metricConfig items > timestep

Description: Frequency of timeseries points

2.1.5. Property HPC cluster description > metricConfig > metricConfig items > aggregation

Description: How the metric is aggregated

Must be one of:

• “sum”
• “avg”

2.1.6. Property HPC cluster description > metricConfig > metricConfig items > footprint

Description: Is it a footprint metric and what type

Must be one of:

• “avg”
• “max”
• “min”

2.1.7. Property HPC cluster description > metricConfig > metricConfig items > energy

Description: Is it used to calculate job energy

Must be one of:

• “power”
• “energy”

2.1.8. Property HPC cluster description > metricConfig > metricConfig items > lowerIsBetter

Description: Is lower better.

2.1.9. Property HPC cluster description > metricConfig > metricConfig items > peak

Description: Metric peak threshold (Upper metric limit)

2.1.10. Property HPC cluster description > metricConfig > metricConfig items > normal

Description: Metric normal threshold

2.1.11. Property HPC cluster description > metricConfig > metricConfig items > caution

Description: Metric caution threshold (Suspicious but does not require immediate action)

2.1.12. Property HPC cluster description > metricConfig > metricConfig items > alert

Description: Metric alert threshold (Requires immediate action)

2.1.13. Property HPC cluster description > metricConfig > metricConfig items > subClusters

Description: Array of cluster hardware partition metric thresholds

2.1.13.1. HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items

2.1.13.1.1. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > name

Description: Hardware partition name

2.1.13.1.2. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > footprint

Description: Is it a footprint metric and what type. Overwrite global setting

Must be one of:

• “avg”
• “max”
• “min”

2.1.13.1.3. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > energy

Description: Is it used to calculate job energy. Overwrite global

Must be one of:

• “power”
• “energy”

2.1.13.1.4. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > lowerIsBetter

Description: Is lower better. Overwrite global

2.1.13.1.5. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > peak

2.1.13.1.6. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > normal

2.1.13.1.7. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > caution

2.1.13.1.8. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > alert

2.1.13.1.9. Property HPC cluster description > metricConfig > metricConfig items > subClusters > subClusters items > remove

Description: Remove this metric for this subcluster

3. Property HPC cluster description > subClusters

Description: Array of cluster hardware partitions

3.1. HPC cluster description > subClusters > subClusters items

3.1.1. Property HPC cluster description > subClusters > subClusters items > name

Description: Hardware partition name

3.1.2. Property HPC cluster description > subClusters > subClusters items > processorType

Description: Processor type

3.1.3. Property HPC cluster description > subClusters > subClusters items > socketsPerNode

Description: Number of sockets per node

3.1.4. Property HPC cluster description > subClusters > subClusters items > coresPerSocket

Description: Number of cores per socket

3.1.5. Property HPC cluster description > subClusters > subClusters items > threadsPerCore

Description: Number of SMT threads per core

3.1.6. Property HPC cluster description > subClusters > subClusters items > flopRateScalar

Description: Theoretical node peak flop rate for scalar code in GFlops/s

3.1.6.1. Property HPC cluster description > subClusters > subClusters items > flopRateScalar > unit

Description: Metric unit

3.1.6.2. Property HPC cluster description > subClusters > subClusters items > flopRateScalar > value

3.1.7. Property HPC cluster description > subClusters > subClusters items > flopRateSimd

Description: Theoretical node peak flop rate for SIMD code in GFlops/s

3.1.7.1. Property HPC cluster description > subClusters > subClusters items > flopRateSimd > unit

Description: Metric unit

3.1.7.2. Property HPC cluster description > subClusters > subClusters items > flopRateSimd > value

3.1.8. Property HPC cluster description > subClusters > subClusters items > memoryBandwidth

Description: Theoretical node peak memory bandwidth in GB/s

3.1.8.1. Property HPC cluster description > subClusters > subClusters items > memoryBandwidth > unit

Description: Metric unit

3.1.8.2. Property HPC cluster description > subClusters > subClusters items > memoryBandwidth > value

3.1.9. Property HPC cluster description > subClusters > subClusters items > nodes

Description: Node list expression

3.1.10. Property HPC cluster description > subClusters > subClusters items > topology

Description: Node topology

3.1.10.1. Property HPC cluster description > subClusters > subClusters items > topology > node

Description: HwTread lists of node

3.1.10.1.1. HPC cluster description > subClusters > subClusters items > topology > node > node items

3.1.10.2. Property HPC cluster description > subClusters > subClusters items > topology > socket

Description: HwTread lists of sockets

3.1.10.2.1. HPC cluster description > subClusters > subClusters items > topology > socket > socket items

3.1.10.2.1.1. HPC cluster description > subClusters > subClusters items > topology > socket > socket items > socket items items

3.1.10.3. Property HPC cluster description > subClusters > subClusters items > topology > memoryDomain

Description: HwTread lists of memory domains

3.1.10.3.1. HPC cluster description > subClusters > subClusters items > topology > memoryDomain > memoryDomain items

3.1.10.3.1.1. HPC cluster description > subClusters > subClusters items > topology > memoryDomain > memoryDomain items > memoryDomain items items

3.1.10.4. Property HPC cluster description > subClusters > subClusters items > topology > die

Description: HwTread lists of dies

3.1.10.4.1. HPC cluster description > subClusters > subClusters items > topology > die > die items

3.1.10.4.1.1. HPC cluster description > subClusters > subClusters items > topology > die > die items > die items items

3.1.10.5. Property HPC cluster description > subClusters > subClusters items > topology > core

Description: HwTread lists of cores

3.1.10.5.1. HPC cluster description > subClusters > subClusters items > topology > core > core items

3.1.10.5.1.1. HPC cluster description > subClusters > subClusters items > topology > core > core items > core items items

3.1.10.6. Property HPC cluster description > subClusters > subClusters items > topology > accelerators

Description: List of of accelerator devices

3.1.10.6.1. HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items

3.1.10.6.1.1. Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > id

Description: The unique device id

3.1.10.6.1.2. Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > type

Description: The accelerator type

Must be one of:

• “Nvidia GPU”
• “AMD GPU”
• “Intel GPU”

3.1.10.6.1.3. Property HPC cluster description > subClusters > subClusters items > topology > accelerators > accelerators items > model

Description: The accelerator model

Generated using json-schema-for-humans on 2024-12-04 at 16:45:59 +0100

7.3 - Job Data Schema

The following schema in its raw form can be found in the ClusterCockpit GitHub repository.

Job metric data list

• 1. Property Job metric data list > mem_used
  • 1.1. Property Job metric data list > mem_used > node
• 2. Property Job metric data list > flops_any
  • 2.1. Property Job metric data list > flops_any > node
  • 2.2. Property Job metric data list > flops_any > socket
  • 2.3. Property Job metric data list > flops_any > memoryDomain
  • 2.4. Property Job metric data list > flops_any > core
  • 2.5. Property Job metric data list > flops_any > hwthread
• 3. Property Job metric data list > mem_bw
  • 3.1. Property Job metric data list > mem_bw > node
  • 3.2. Property Job metric data list > mem_bw > socket
  • 3.3. Property Job metric data list > mem_bw > memoryDomain
• 4. Property Job metric data list > net_bw
  • 4.1. Property Job metric data list > net_bw > node
• 5. Property Job metric data list > ipc
  • 5.1. Property Job metric data list > ipc > node
  • 5.2. Property Job metric data list > ipc > socket
  • 5.3. Property Job metric data list > ipc > memoryDomain
  • 5.4. Property Job metric data list > ipc > core
  • 5.5. Property Job metric data list > ipc > hwthread
• 6. Property Job metric data list > cpu_user
  • 6.1. Property Job metric data list > cpu_user > node
  • 6.2. Property Job metric data list > cpu_user > socket
  • 6.3. Property Job metric data list > cpu_user > memoryDomain
  • 6.4. Property Job metric data list > cpu_user > core
  • 6.5. Property Job metric data list > cpu_user > hwthread
• 7. Property Job metric data list > cpu_load
  • 7.1. Property Job metric data list > cpu_load > node
• 8. Property Job metric data list > flops_dp
  • 8.1. Property Job metric data list > flops_dp > node
  • 8.2. Property Job metric data list > flops_dp > socket
  • 8.3. Property Job metric data list > flops_dp > memoryDomain
  • 8.4. Property Job metric data list > flops_dp > core
  • 8.5. Property Job metric data list > flops_dp > hwthread
• 9. Property Job metric data list > flops_sp
  • 9.1. Property Job metric data list > flops_sp > node
  • 9.2. Property Job metric data list > flops_sp > socket
  • 9.3. Property Job metric data list > flops_sp > memoryDomain
  • 9.4. Property Job metric data list > flops_sp > core
  • 9.5. Property Job metric data list > flops_sp > hwthread
• 10. Property Job metric data list > vectorization_ratio
  • 10.1. Property Job metric data list > vectorization_ratio > node
  • 10.2. Property Job metric data list > vectorization_ratio > socket
  • 10.3. Property Job metric data list > vectorization_ratio > memoryDomain
  • 10.4. Property Job metric data list > vectorization_ratio > core
  • 10.5. Property Job metric data list > vectorization_ratio > hwthread
• 11. Property Job metric data list > cpu_power
  • 11.1. Property Job metric data list > cpu_power > node
  • 11.2. Property Job metric data list > cpu_power > socket
• 12. Property Job metric data list > mem_power
  • 12.1. Property Job metric data list > mem_power > node
  • 12.2. Property Job metric data list > mem_power > socket
• 13. Property Job metric data list > acc_utilization
  • 13.1. Property Job metric data list > acc_utilization > accelerator
• 14. Property Job metric data list > acc_mem_used
  • 14.1. Property Job metric data list > acc_mem_used > accelerator
• 15. Property Job metric data list > acc_power
  • 15.1. Property Job metric data list > acc_power > accelerator
• 16. Property Job metric data list > clock
  • 16.1. Property Job metric data list > clock > node
  • 16.2. Property Job metric data list > clock > socket
  • 16.3. Property Job metric data list > clock > memoryDomain
  • 16.4. Property Job metric data list > clock > core
  • 16.5. Property Job metric data list > clock > hwthread
• 17. Property Job metric data list > eth_read_bw
  • 17.1. Property Job metric data list > eth_read_bw > node
• 18. Property Job metric data list > eth_write_bw
  • 18.1. Property Job metric data list > eth_write_bw > node
• 19. Property Job metric data list > filesystems
  • 19.1. Job metric data list > filesystems > filesystems items
    • 19.1.1. Property Job metric data list > filesystems > filesystems items > name
    • 19.1.2. Property Job metric data list > filesystems > filesystems items > type
    • 19.1.3. Property Job metric data list > filesystems > filesystems items > read_bw
      • 19.1.3.1. Property Job metric data list > filesystems > filesystems items > read_bw > node
    • 19.1.4. Property Job metric data list > filesystems > filesystems items > write_bw
      • 19.1.4.1. Property Job metric data list > filesystems > filesystems items > write_bw > node
    • 19.1.5. Property Job metric data list > filesystems > filesystems items > read_req
      • 19.1.5.1. Property Job metric data list > filesystems > filesystems items > read_req > node
    • 19.1.6. Property Job metric data list > filesystems > filesystems items > write_req
      • 19.1.6.1. Property Job metric data list > filesystems > filesystems items > write_req > node
    • 19.1.7. Property Job metric data list > filesystems > filesystems items > inodes
      • 19.1.7.1. Property Job metric data list > filesystems > filesystems items > inodes > node
    • 19.1.8. Property Job metric data list > filesystems > filesystems items > accesses
      • 19.1.8.1. Property Job metric data list > filesystems > filesystems items > accesses > node
    • 19.1.9. Property Job metric data list > filesystems > filesystems items > fsync
      • 19.1.9.1. Property Job metric data list > filesystems > filesystems items > fsync > node
    • 19.1.10. Property Job metric data list > filesystems > filesystems items > create
      • 19.1.10.1. Property Job metric data list > filesystems > filesystems items > create > node
    • 19.1.11. Property Job metric data list > filesystems > filesystems items > open
      • 19.1.11.1. Property Job metric data list > filesystems > filesystems items > open > node
    • 19.1.12. Property Job metric data list > filesystems > filesystems items > close
      • 19.1.12.1. Property Job metric data list > filesystems > filesystems items > close > node
    • 19.1.13. Property Job metric data list > filesystems > filesystems items > seek
      • 19.1.13.1. Property Job metric data list > filesystems > filesystems items > seek > node

Title: Job metric data list

Description: Collection of metric data of a HPC job

1. Property Job metric data list > mem_used

Description: Memory capacity used

1.1. Property Job metric data list > mem_used > node

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

2. Property Job metric data list > flops_any

Description: Total flop rate with DP flops scaled up

2.1. Property Job metric data list > flops_any > node

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

2.2. Property Job metric data list > flops_any > socket

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

2.3. Property Job metric data list > flops_any > memoryDomain

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

2.4. Property Job metric data list > flops_any > core

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

2.5. Property Job metric data list > flops_any > hwthread

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

3. Property Job metric data list > mem_bw

Description: Main memory bandwidth

3.1. Property Job metric data list > mem_bw > node

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

3.2. Property Job metric data list > mem_bw > socket

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

3.3. Property Job metric data list > mem_bw > memoryDomain

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

4. Property Job metric data list > net_bw

Description: Total fast interconnect network bandwidth

4.1. Property Job metric data list > net_bw > node

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

5. Property Job metric data list > ipc

Description: Instructions executed per cycle

5.1. Property Job metric data list > ipc > node

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

5.2. Property Job metric data list > ipc > socket

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

5.3. Property Job metric data list > ipc > memoryDomain

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

5.4. Property Job metric data list > ipc > core

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

5.5. Property Job metric data list > ipc > hwthread

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

6. Property Job metric data list > cpu_user

Description: CPU user active core utilization

6.1. Property Job metric data list > cpu_user > node

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

6.2. Property Job metric data list > cpu_user > socket

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

6.3. Property Job metric data list > cpu_user > memoryDomain

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

6.4. Property Job metric data list > cpu_user > core

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

6.5. Property Job metric data list > cpu_user > hwthread

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

7. Property Job metric data list > cpu_load

Description: CPU requested core utilization (load 1m)

7.1. Property Job metric data list > cpu_load > node

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

8. Property Job metric data list > flops_dp

Description: Double precision flop rate

8.1. Property Job metric data list > flops_dp > node

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

8.2. Property Job metric data list > flops_dp > socket

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

8.3. Property Job metric data list > flops_dp > memoryDomain

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

8.4. Property Job metric data list > flops_dp > core

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

8.5. Property Job metric data list > flops_dp > hwthread

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

9. Property Job metric data list > flops_sp

Description: Single precision flops rate

9.1. Property Job metric data list > flops_sp > node

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

9.2. Property Job metric data list > flops_sp > socket

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

9.3. Property Job metric data list > flops_sp > memoryDomain

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

9.4. Property Job metric data list > flops_sp > core

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

9.5. Property Job metric data list > flops_sp > hwthread

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

10. Property Job metric data list > vectorization_ratio

Description: Fraction of arithmetic instructions using SIMD instructions

10.1. Property Job metric data list > vectorization_ratio > node

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

10.2. Property Job metric data list > vectorization_ratio > socket

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

10.3. Property Job metric data list > vectorization_ratio > memoryDomain

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

10.4. Property Job metric data list > vectorization_ratio > core

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

10.5. Property Job metric data list > vectorization_ratio > hwthread

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

11. Property Job metric data list > cpu_power

Description: CPU power consumption

11.1. Property Job metric data list > cpu_power > node

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

11.2. Property Job metric data list > cpu_power > socket

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

12. Property Job metric data list > mem_power

Description: Memory power consumption

12.1. Property Job metric data list > mem_power > node

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

12.2. Property Job metric data list > mem_power > socket

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

13. Property Job metric data list > acc_utilization

Description: GPU utilization

13.1. Property Job metric data list > acc_utilization > accelerator

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

14. Property Job metric data list > acc_mem_used

Description: GPU memory capacity used

14.1. Property Job metric data list > acc_mem_used > accelerator

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

15. Property Job metric data list > acc_power

Description: GPU power consumption

15.1. Property Job metric data list > acc_power > accelerator

Description: 😅 ERROR in schema generation, a referenced schema could not be loaded, no documentation here unfortunately 🏜️

16. Property Job metric data list > clock

Description: Average core frequency