Documentation

Deep Reinforcement Learning Algorithm

Swaps the arguments of HaskTorch's foldLoop around

Because snake_case sucks

Default limits for nanToNum

Default limits for nanToNum (0.0)

GPU Tensor filled with Float value

Select index with [Int] from GPU tensor

Torch.where' with fixed type for where'

Syntactic sugar for HaskTorch's repeatInterleave so it can more easily be fmapped.

Apply same function to both Left and Right

Apply same function to both Left and Right and put back into Either

Range from 0 to n - 1

First of triple

Helper function creating split indices

Helper function creating split indices as gpu int tensor

Split Tensor into list of Tensors along dimension

Current Timestamp as formatted string

Current Timestamp with default formatting: "%Y%m%d-%H%M%S"

Create a model archive directory for the given algorithm

Optimizer moments at given prefix

Load Optimizer State

Calculate weight Limits based on Layer Dimensions

Type of weight initialization

Weights for a layer given limits and dimensions.

Initialize Weights of Linear Layer

Initialize Weights and Bias of Linear Layer

Initialize weights uniformally given upper and lower bounds

Initialize weights uniformally based on Fan In

Initialize weights normally given mean and std bounds

Initialize weights normally based on Fan In

Softly update parameters from Online Net to Target Net

Softly copy parameters from Online Net to Target Net

Hard Copy of Parameter from one net to the other

GPU 1

CPU 0

Default Tensor Data Type

Convert an Array to a Tensor on GPU

Convert an Array to a Tensor on CPU

Convert an Array to a Tensor on GPU

Convert an Array to a Tensor on CPU

Create an empty Float Tensor on GPU

Create an empty Float Tensor on CPU

Convert a Scalar to a Tensor on GPU

Convert a Scalar to a Tensor on CPU

Convert model to Double on GPU

Convert model to Double on CPU

Convert model to Float on CPU

Convert model to Float on CPU

Generate a Tensor of random Integers on GPU

Generate a Tensor of random Integers on CPU

Generate Normally Distributed Random values given dimensions

Generate Normally Distributed Random values given μs and σs

Generate Uniformally distributed values in a given range

Rescale tensor s.t. mean = 0.0 and std = 1.0

HTTP options for Hym server communication, sometimes simulations can take a while, therefore we wait ...

Info object gotten form stepping

Single Environment Step

Base Route to Hym Server

Possible Action Spaces

Convert a Map to a Tensor where Pool index is a dimension on GPU

Convert a Map to a Tensor where Pool index is a dimension on CPU

Convert Tensor to Map (Continuous action spaces) on GPU

Convert Tensor to Map (Discrete action spaces) on CPU

Convert the Pooled Step Map to a Tuple on GPU

Convert the Pooled Step Map to a Tuple on CPU

Generic HTTP GET Request to Hym Server

Send a POST Request to a Hym Server

Convert a JSON Response from an ACE Server to a Map: String Float

Convert a JSON Response from an ACE Server to a Map: String Bool

Convert a JSON Response from an ACE Server to a Float-List

Convert a JSON Response from an ACE Server to a String-List

Reset Pooled Environments on a Hym server

Get Random Actions from all Pooled Environments

Perform Random Actions in all Pooled Environments

Take Steps in All Environments (Continuous)

Take Steps in All Environments (Discrete)

Generate URL to a Hym-GACE server from meta information

Generate URL to a Hym-Gym server from meta information

Send a GET Request to a GACE Server Obtain the Target of Pooled GACE Environments

Send a GET Request to a GACE Server Obtain the Target Predicate of Pooled GACE Environments

Get minmax estimates for performances andor targets

Action Keys from GACE Server

Observation Keys from GACE Server

Get the SHACE logging path as a dict

Get the SHACE logging path

Reset a Vectorized Environment Pool

Reset selected Environments from Pool

Shorthand for getting keys of pooled same envs

Shorthand for getting keys of pooled same envs

Number of Environments in Pool

Get Info without stepping

Get Targets for all envs in Pool

Get Targets for all envs in Pool and process them

Target parameter keys

Implying same target params for all envs in pool.

Implying same target params for all envs in pool and we want all of them.

Step in a Control Environment

Step in a Discrete Environment

Take a random Step an Environment

Get a set of random actions from the current environment

Create Boolean Mask Tensor from list of indices.

Create Boolean Mask Tensor from list of indices on CPU.

Create a Boolean Mask Tensor from index Tensor

Process Targets for HER

Convenience, takes output straight from stepPool.

Process for HER returns processed observations, the target and the augmented target

Process for HER returns processed observations, the target and the augmented target

Standardize state over all parallel envs

Process / Sanitize the Observations from GACE

Scale reward to center

Normalize feature x s.t. x' ∈ [a,b]

Convenience: Normalize feature x s.t. x' ∈ [-1.0,1.0]

Normalize given some arbitrary min / max values

Normalize feature x' ∈ [a,b] given the original min/max

Sanatize JSON for MLFlow: Names may only contain alphanumerics, underscores (_), dashes (-), periods (.), spaces ( ), and slashes (/).

Data Logging to MLFlow Trackign Server

Retrieve a run ID

Make new Tracker given a Tracking Server URI

Make new Tracker given a Hostname and Port

Create a new Experiment with rng suffix

Create a new Experiment

Create a new run with a set of given paramters

New run with algorithm id and #envs as log params

End a run

End all runs of a Tracker

End all runs and discard tracker

Write Loss to Tracking Server

Write Reward to Tracking Server

Filter Performance of all envs

Write Current state of the Environment to Trackign Server