API

Power Electronic Converter generalized block diagram models (pecblocks).

These models are based on a Hammerstein-Wiener framework for photovoltaics (HWPV). The training data may come from electromagnetic transient (EMT) simulations or laboratory tests of PV inverters, either grid-forming (GFM) or grid-following (GFL). The inverters may be three-phase or single-phase.

The required packages include dynonet, harold, control, and h5py. These in turn require torch, pandas, scipy, numpy, and matplotlib.

Contains these implemented modules:

  • common.py: supports a PyTorch dataloader customized to fitting generalized block diagram models. Implements PVInvDataset class.

  • pv3_functions.py: analyzes the closed-loop stability of trained Norton and Thevenin HWPV models.

  • pv3_poly.py: the main class for training, exporting, and evaluating HWPV models. Implements pv3_poly class.

  • util.py: reads training data from HDF5 and CSV files into Pandas DataFrames.

See https://pecblocks.readthedocs.io/en/latest/ for documentation.

common

PyTorch dataloader customized to fitting generalized block diagram models. Implements PVInvDataset class.

class pecblocks.common.PVInvDataset(data, input_dim, out_dim, pre_pad=0, bLog=True)

Helper class to load training datasets for PyTorch.

A list of Pandas dataframes is converted to a torch tensor for training the HWPV models. The tensor can be pre-padded with initial conditions before training the model.

data

data [case index, time point, channel index]

Type:

tensor

len

number of time points per channel

Type:

int

n_in

number of input channels

Type:

int

n_out

number of output channels

Type:

int

__getitem__(idx)

Retrieve data for a case or event

Parameters:

idx (int) – zero-based case number

Returns:

tensor – case data [number of time points, number of input channels plus output channels]

__init__(data, input_dim, out_dim, pre_pad=0, bLog=True)

Constructor method

Total number of channels must be input_dim + output_dim.

Parameters:

  • data (list(DataFrame)) – list of Pandas DataFrames from the HDF5 or CSV source (see util module).

  • input_dim (int) – number of input channels

  • output_dim (int) – number of output channels

  • pre_pad (int) – number of initial condition points to pre-pend, suggest 10-20% of the total number of time points.

__len__()

Retrieve number of cases or events

Returns:

int – number of cases (first dimension) in data

pecblocks.common.read_csv_files_to_dflist(path, pattern='.csv', time_range=None)

Helper function to load CSV files into a list of Pandas DataFrames.

Each CSV file should use the comma as separator, column names in the first row. The Pandas read_csv function is called on each CSV file.

Parameters:

  • path (str) – this must be a path to glob for files with pattern in the name.

  • pattern (list(str)) – the file extension to look for. Not a regular expression.

Returns:

list (DataFrame) – List of Pandas DataFrames, one per CSV file.

pv3_functions

Functions to perform sensitivity analysis on Thevenin and Norton controlled sources containing generalized block diagram models.

pecblocks.pv3_functions.analyze_case(model, idx, bPrint=False)

Scans a case to update the ranges of grid resistance and parameter changes in a model’s training dataset. Internal

Parameters:

  • model (pv3_poly) – The pv3_poly instance with data loaded and normalized.

  • idx (int) – the zero-based case number to scan.

  • bPrint (bool) – request diagnostic output of step changes identified in this case.

Yields:

Updates to global module variables minRd, maxRd, minRq, maxRq, nRd, nG, nUd, and nUq.

Returns:

None

pecblocks.pv3_functions.build_baselines(bases, step_vals, cfg, keys, indices, lens, level)

Recursive function to add a set of operating points to the sensitivity evaluation set. Uses a depth-first approach. When the last channel number is processed, the recursion will back up to a previous channel number that was not fully processed yet. Internal

Parameters:

  • bases (list(float)[]) – array of step_vals for operating points in the sensitivity evaluation set

  • step_vals (list(float)) – input channel values for a pv3_poly model steady-state operating point

  • cfg (dict) – the sensitivity member of a pv3_poly configuration, which incluces a member sets contained keyed channel names

  • keys (list(str)) – list of channel names from the sets member of cfg, each of these corresponds to a level of recursion

  • indices (list(int)) – keeps track of the channel number to resume processing whenever level reachs the last key

  • lens (list(int)) – the number of operating point values for each named channel in keys

  • level (int) – enters with 0, backs up at the length of keys minus 1

Yields:

Appending to bases. Updates counter in each call.

Returns:

None

pecblocks.pv3_functions.build_step_vals(T0, G0, F0, Ud0, Uq0, Vd0, Vq1, GVrms, Ctl)

Assembles input values into a properly ordered array for a model’s forward evaluation, excluding any not provided, as indicated by NaN. Internal

Parameters:

  • T0 (float) – temperature, may be NaN

  • G0 (float) – solar irradiance

  • F0 (float) – control frequency, may be NaN

  • Ud0 (float) – direct-axis voltage control index

  • Uq0 (float) – quadrature-axis voltage control index

  • Vd0 (float) – direct-axis terminal voltage

  • Vq1 (float) – quadrature-axis terminal voltage

  • GVrms (float) – polynomial input feature

  • Ctl (float) – control-mode input feature

Returns:

list (float) – array of input values for model.steady_state_response

pecblocks.pv3_functions.clamp_loss(model, bPrint)

Estimates the clamping loss for a model.

Clamping loss occurs when a forward-evaluated output channel exceeds its configured limits. If clamp is not part of the model configuration, there is no clamping loss.

Parameters:

  • model (pv3_poly) – The pv3_poly instance with data loaded and normalized, and the model previously exported.

  • bPrint (bool) – request diagnostic output of clamping loss by case and output channel.

Returns:

float – sum of clamping loss over all cases and output channels

pecblocks.pv3_functions.counter = 0

Tracks the number of recursive calls to build_baselines

pecblocks.pv3_functions.find_channel_indices(targets, available)

Pick out the training dataset channel numbers used in sensitivity evaluations. Call separately for the input channels and the output channesl. Internal

Parameters:

  • targets (list(str)) – array of channel names used in the sensitivity evaluation set

  • available (list(str)) – array of channel names available in a model’s training dataset

Returns:

  • int – length of the next return array, equal to len(targets)

  • list (int) – array of training dataset channel numbers

pecblocks.pv3_functions.general_sensitivity_analysis(model, bPrint, dThresh, bLog=False)

Estimates the sensitivity of an exported model in z domain, Norton or Thev, without G.

The sensitivity is estimated from steady-state response of grid output variables (OutD, OutQ) with respect to grid input variables (InD, InQ). These variables must be indentified in the model cfg sensitivity data structure. Steady-state operating points are estimated at 7 values of each input variables, as determined by autoranging. There is no special handling of G.

Parameters:

  • model (pv3_poly) – The pv3_poly instance with data loaded and normalized, and the model previously exported.

  • bPrint (bool) – print the maximum values of the four partial derivatives of OutD, OutQ with respect to InD, InQ.

  • dThresh (float) – when printing, show the number of cases within each range that have sigma exceeding this threshold.

  • bLog (bool) – print diagnostics of the operating points and perturbations over the sensitivity evaluation set.

Returns:

float – the maximum of four partial derivatives of OutD, OutQ with respect to InD, InQ

pecblocks.pv3_functions.get_gvrms(G, Vd, Vq, k)

Calculates the polynomial feature GVrms. Internal

Parameters:

  • G (float) – solar irradiance.

  • Vd (float) – direct-axis voltage.

  • Vq (float) – quadrature-axis voltage.

  • k (float) – sqrt(1.5) for three-phase inverters, 1.0 for single-phase inverters.

Returns:

float – value of GVrms

pecblocks.pv3_functions.maxRd = 0.0

Maximum resistance Rd found in a model’s training dataset

pecblocks.pv3_functions.maxRq = 0.0

Maximum resistance Rq found in a model’s training dataset

pecblocks.pv3_functions.minRd = 1000000000.0

Minimum resistance Rd found in a model’s training dataset

pecblocks.pv3_functions.minRq = 1000000000.0

Minimum resistance Rq found in a model’s training dataset

pecblocks.pv3_functions.model_sensitivity(model, bPrint)

Calculates the maximum sensitivity of an exported Norton model in z domain.

The model configuration must include a sensitivy structure that was used in training. This determines the sensitivity evaluation set.

See also

sensitivity_analysis()

Parameters:

  • model (pv3_poly) – The pv3_poly instance with data loaded and normalized, and the model previously exported.

  • bPrint (bool) – print the maximum sensitivity (return value).

Yields:

Printed information about the sensitivity evaluation set and columns.

Returns:

float – The maximum derivative of Id, Iq with respect to Vd, Vq.

pecblocks.pv3_functions.nG = 0

Number of G changes found in a model’s training dataset

pecblocks.pv3_functions.nRd = 0

Number of Rd changes found in a model’s training dataset

pecblocks.pv3_functions.nUd = 0

Number of Ud changes found in a model’s training dataset

pecblocks.pv3_functions.nUq = 0

Number of Uq changes found in a model’s training dataset

pecblocks.pv3_functions.norton_sensitivity_analysis(model, bPrint, bLog=False, bAutoRange=False, dThresh=0.1, cfgKRMS=None)

Estimates the sensitivity of an exported model in z domain.

The sensitivity is estimated for grid interface output variables (Id, Iq) with respect to the closed loop feedback variables (Vd, Vq). The sensitivity is examined over a set of operating points in G, Ctl, Ud, Uq, Fc, Vd, and Vq. At each operating point, Vd and Vq is perturbed separately by a small amount, and then GVrms is updated. The model.steady_state_response function is used to estimate the changes in Id and Iq for the perturbations in Vd and Vq.

Parameters:

  • model (pv3_poly) – The pv3_poly instance with data loaded and normalized, and the model previously exported.

  • bPrint (bool) – print the maximum values of the four partial derivatives of Id, Iq with respect to Vd, Vq.

  • bLog (bool) – print diagnostics of the operating points and perturbations over the sensitivity evaluation set.

  • bAutoRange (bool) – if True, use the minima and maxima of the model training set to establish the input channel bounds. If False, use a built-in set of bounds for the GridLink SDI lab tests.

  • dThresh (float) – when printing, show the number of cases within each range that have sigma exceeding this threshold.

  • cfgKRMS (float) – pass KRMS from the model config file if available, otherwise sqrt(1.5) is the default

Yields:

Printed output of the auto-range boundaries and the sensitivity evaluation set.

Returns:

float – the maximum of four partial derivatives of Id, Iq with respect to Vd, Vq.

pecblocks.pv3_functions.thevenin_sensitivity_analysis(model, bPrint, bLog=False, bReducedSet=False, dThresh=10.0, cfgKRMS=None)

Calculates the maximum sensitivity of an exported Thevenin model in z domain.

This function uses a fixed sensitivity evaluation set for the GridLink SDI lab tests.

See also

sensitivity_analysis()

Parameters:

  • model (pv3_poly) – The pv3_poly instance with data loaded and normalized, and the model previously exported.

  • bPrint (bool) – print the maximum values of each partial derivative of Vd, Vq with respect to Id, Iq

  • bLog (bool) – print the four partial derivatives of Vd, Vq, w.r.t. Id, Iq sat each operating point in the sensitivity evaluation set.

  • bReducedSet (bool) – use a reduced sensitivity evaluation set of 72 cases instead of the full set of 60,500 cases

  • dThresh (float) – when printing, show the number of cases within each range that have sigma exceeding this threshold.

  • cfgKRMS (float) – pass KRMS from the model config file if available, otherwise sqrt(1.5) is the default

Yields:

Printed information about the sensitivity evaluation set and columns.

Returns:

float – The maximum derivative of Vd, Vq with respect to Id, Iq.

pv3_poly

The pv3_poly class supports training and evaluation of generalized block diagram models of three-phase and single-phase inverters.

class pecblocks.pv3_poly.pv3(training_config=None, sim_config=None)

Implementation of HWPV model training, export, and forward evaluation.

This class is configured from a JSON file. The normal usage is to load a training dataset and normalize it. Then perform one or more of the following operations:

  • Train the model

  • Export the model

  • Evaluate model metrics

  • Forward evaluation, or simulation

eps

numerical stability parameter for the Adam optimizer in PyTorch

Type:

float

lr

learning rate for PyTorch

Type:

float

num_iter

number of iterations (epochs) to perform in the next training run

Type:

int

continue_iterations

if True, start the next set of training epochs from the existing model saved in pkl files. If False, reset the initial model to randomized parameters.

Type:

bool

print_freq

epoch interval for printing status information during model training

Type:

int

batch_size

the data loader’s batch size for training in PyTorch

Type:

int

n_validation_pct

percentage of cases in data_train to reserve for validation loss

Type:

int

n_validation_seed

a random number to choose the cases reserved for validation loss

Type:

int

n_loss_skip

number of decimated-in-time points to exclude from loss evaluation during training, used to exclude the initialization transients.

Type:

int

n_pad

number of decimated-in-time points to pre-pad the training data with t=0 initial values, used to mitigate dynoNet’s behavior of always starting H1 from rest. Due to bias coefficients from F1, the initial H1 inputs are generally non-zero.

Type:

int

gtype

type of H1 block, may be iir, fir, stable2ndx. (stable2nd is deprecated because it does not allow distinct real poles, only complex conjugates.)

Type:

str

n_skip

number of decimated-in-time points to exclude from the beginning of each event in data_train

Type:

int

n_trunc

number of decimated-in-time points to exclude from the end of each event in data_train

Type:

int

n_dec

decimation-in-time interval for data_train, e.g., 1 for every point, 5 for every fifth point. Decimation may have already been done in preparing the input data for the HDF5 file.

Type:

int

na

number of learnable denominator coefficients in H1, not including a0=1

Type:

int

nb

number of learnable numerator coefficients in H1

Type:

int

nk

number of delay cells in H1 (never tested in pecblocks)

Type:

int

activation

activation function for the F1 and F2 blocks, may be tanh (preferred), sigmoid or relu

Type:

str

nh1

number of hidden cells in F1

Type:

int

nh2

number of hidden cells in F2

Type:

int

COL_T

time channel name of data_train; should be an array of length 1

Type:

list(str)

COL_U

input channel names of data_train

Type:

list(str)

COL_Y

output channel names of data_train

Type:

list(str)

idx_in

channel indices of data_train corresponding to COL_U

Type:

list(int)

idx_out

channel indices of data_train corresponding to COL_Y

Type:

list(int)

data_train

three-dimensional Numpy array of training data loaded from an HDF5 file. First dimension is the case number, second dimension is the time point index, third dimension is the channel index.

Type:

array_like(float, ndim=3)

normfacs

the min, max, mean, offset, and scaling factor for each channel in data_train

Type:

dict

t

the series of time points in data_train

Type:

float

n_cases

number of cases (or events) in data_train

Type:

int

t_step

the decimated time step in data_train

Type:

float

Lf

inverter’s output filter inductance, before Cf

Type:

float

Lc

inverter’s output filter inductance, at the PCC

Type:

float

Cf

inverter’s output filter capacitance

Type:

float

model_folder

folder to save trained and exported model data

Type:

str

model_root

base name for the exported model

Type:

str

data_path

path and file name to the hdf5 input data for data_train

Type:

str

h5grp_prefix

prefix for the 0-based indexing of cases in data_train

Type:

str

clamps

configuration for clamping losses

Type:

dict

sensitivity

configuration for dV/dI or dI/dV sensitivity losses

Type:

dict

d_key

automatically determined as Id for a Norton model or Vd for a Thevenin model

Type:

str

q_key

automatically determined as Iq for a Norton model or Vq for a Thevenin model

Type:

str

sens_counter

tracks the number of recursive function calls to build sensitivity evaluation sets

Type:

int

__init__(training_config=None, sim_config=None)

Constructor method. If called with no parameters, then load_training_config or load_sim_config must be called later before use.

See also

load_training_config() load_sim_config()

Parameters:

  • training_config (str) – JSON file name with a configuration for model training.

  • sim_config (str) – JSON file name with an exported configuration for model evaluation. These file nams generally end in _fhf.json

__weakref__

list of weak references to the object (if defined)

append_2nd(model, label, H)

Adds H1 as a stable 2nd-order block to the model export, limited to complex conjugate poles.

Parameters:

  • model (dict) – configuration that will be exported to a JSON file

  • label (str) – should be ‘H1’ to label the block

  • H (StableSecondOrderMimoLinearDynamicalOperator) – H1 containing the parameters

append_2ndx(model, label, H)

Adds H1 as a stable 2nd-order block to the model export, allows distinct real and complex conjugate poles.

Parameters:

  • model (dict) – configuration that will be exported to a JSON file

  • label (str) – should be ‘H1’ to label the block

  • H (StableSecondOrderMimoLinearDynamicalOperatorX) – H1 containing the parameters

append_fir(model, label, H)

Adds H1 as a finite impulse response block to the model export.

Parameters:

  • model (dict) – configuration that will be exported to a JSON file

  • label (str) – should be ‘H1’ to label the block

  • H (MimoFirLinearDynamicalOperator) – H1 containing the parameters

append_lti(model, label, H)

Adds H1 to the model export.

Checks gtype and if not FIR or stable 2nd-order, treat as IIR.

Parameters:

  • model (dict) – configuration that will be exported to a JSON file

  • label (str) – should be ‘H1’ to label the block

  • H (MimoLinearDynamicalOperator) – H1 containing the IIR parameters

append_net(model, label, F)

Adds F1 or F2 to the model export.

Parameters:

  • model (dict) – configuration that will be exported to a JSON file

  • label (str) – either ‘F1’ or ‘F2’ to label the block

  • F (MimoStaticNonLinearity) – either F1 or F2 containing the parameters

applyAndSaveNormalization()

Scans data_train to identify the range of each channel, then normalize them.

Yields:

  • Each channel normalized to vary from 0.0 to 1.0 over all cases and times.

  • Normalization factors and ranges saved in normfacs

applyNormalization()

Normalizes data_train using normfacs

build_sens_baselines(bases, step_vals, cfg, keys, indices, lens, level)

Recursive function to add a set of operating points to the sensitivity evaluation set. Uses a depth-first approach. When the last channel number is processed, the recursion will back up to a previous channel number that was not fully processed yet. Internal

Parameters:

  • bases (list(float)[]) – array of step_vals for operating points in the sensitivity evaluation set

  • step_vals (list(float)) – input channel values for a pv3_poly model steady-state operating point

  • cfg (dict) – the sensitivity member of a pv3_poly configuration, which incluces a member sets contained keyed channel names

  • keys (list(str)) – list of channel names from the sets member of cfg, each of these corresponds to a level of recursion

  • indices (list(int)) – keeps track of the channel number to resume processing whenever level reachs the last key

  • lens (list(int)) – the number of operating point values for each named channel in keys

  • level (int) – enters with 0, backs up at the length of keys minus 1

Yields:

Appending to bases. Updates sens_counter in each call.

Returns:

None

calc_sensitivity_losses(bPrint=False)

Calculate a sensitivity loss term that can be added to the fitting loss.

The torch functions are used to facilitate optimization during training. This function is called after the fitting of each batch, because the model coefficients were updated. It is not called for each dataset, because the sensitivity only depends on model coefficients.

Parameters:

bPrint (bool) – if True, print some diagnostics

Returns:

  • float – weighted sensitivity loss; add this to the fitting loss

  • float – maximum sensitivity, normalized

check_poles()

Verify that the H(z) transfer function poles are stable.

All poles must lie within the unit circle in the Z plane. This does not guarantee that the poles of H1(s) will be stable. Uses the control package TransferFunction class to check the poles.

See also

make_H1Q1s()

Yields:

Printed output will contain the word ‘UNSTABLE’ if any unstable poles were found.

clampingErrors(bByCase=False)

Summarize the clamping errors in a trained model

Parameters:

bByCase (bool) – True if the individual clamping loss per case is desired, otherwise just the total is calculated.

Returns:

  • list (float) – total clamping loss over all cases and times, by output channel

  • array_like (float,ndim=2) – case clamping loss over all times. First dimension is case number, second dimension is output channel. None if bByCase=False

clamping_losses()

Calculate the clamping losses with torch functions that support optimization.

Deprecated. If any output channel departs from its clamped range, the clamping losses are non-zero. However, this creates step changes that lead to oscillations in the total loss. The normal fitting losses, together with improvements in the initialization of H1, do a better job of constraining outputs to the expected range.

Returns:

list (float) – total clamping loss over all cases and times, by output channel

de_normalize(val, fac)

De-normalizes one channel from the trained blocks to the user.

Parameters:

  • list (float) – normalized channel data

  • fac (dict) – a member of normfacs with scale and offset

Returns:

list (float) – de-normalized channel data

exportModel(filename)

Writes the trained model coefficients to a JSON file.

Includes all the training configuration parameters, and both continuous-time and discrete-time variants of H1.

Parameters:

filename (str) – path and file name to the exported JSON file; typically ends in _fhf.json

find_sens_channel_indices(targets, available)

Pick out the training dataset channel numbers used in sensitivity evaluations. Call separately for the input channels and the output channesl. Internal

Parameters:

  • targets (list(str)) – array of channel names used in the sensitivity evaluation set

  • available (list(str)) – array of channel names available in a model’s training dataset

Returns:

  • int – length of the next return array, equal to len(targets)

  • list (int) – array of training dataset channel numbers

init_to_none()

Initializes class attributes to a default value, usually None

initializeModelStructure()

Set the blocks up for training or forward evaluation.

Yields:

F1, H1, and F2 constructed to match numbers of channels and gtype, random initial coefficients.

loadAndApplyNormalization(filename=None, bSummary=False)

Reads normfacs from a JSON file, then applies them to data_train

Parameters:

  • filename (str) – name of JSON file with a ‘normfacs’ section. If None, try ‘normfacs.json’

  • bSummary (bool) – if True, print a summary of the unnormalized and normalized data

Yields:

normfacs will be updated and applied to data_train.

loadModelCoefficients()

Load the block coefficients from F1.pkl, F2.pkl and H1.pkl in model_folder.

loadNormalization(filename=None)

Retrieves the saved normalization facgtors

Parameters:

filename (str) – name of JSON file with a ‘normfacs’ section. If None, try ‘normfacs.json’

Yields:

normfacs will be updated but not applied to the data.

loadTrainingData(data_path)

Read the HDF5 training records into a NumPy array.

Parameters:

data_path (str) – path and file name to the HDF5 data file.

Yields:

data_train loaded as a 3-dimensional NumPy array.

load_sim_config(filename, model_only=True)

Loads a reduced configuration set for model evaluation.

Only the name, type, H1, F1, and F2 members from model export are essential. If available, other configuration members will be included.

Parameters:

filename (str) – JSON file name with the exported model configuration parameters. It will generally end with _fhf.json.

load_training_config(filename)

Loads a configuration set for model training.

The configuration may include optional sensitivity and clamps members for enhanced loss evaluations. The JSON file may include other members that will be ignored here, e.g., differently named data_path attributes from differeng computing platforms. For another example, a sensitivity member could be renamed as disable_sensitivity, which has the effect of excluding sensitivity from the loss function without discarding its configuration data from the JSON file. This function does not load exported model parameters from the JSON file, but these parameters may be available from existing pkl files in the model_folder

Parameters:

filename (str) – JSON file name with the configuration parameters

make_H1Q1s(Hz)

Convert discrete-time H1(z) to continuous-time H1(s)

Uses the Harold package to convert each transfer function between pairs of input and output channels. The transfer functions are undiscretized using the ‘forward euler’ method. Each continuous-time transfer function is checked for unstable poles, and if any are found, a warning message is printed. If such warning messages appear, the continuous-time model should not be used. One option is to re-train the blocks using the ‘stable2ndx’ gtype.

Parameters:

Hz (MimoLinearDynamicalOperator) – should be H1

Returns:

  • dict – H1(s) as rational transfer function coefficient arrays, keyed by ‘a_i_j’ for denominator and ‘b_i_j’ for numerator, where i is the H1 output channel number and j is the H1 input channel number

  • dict – Q1(s) as the state transition matrices for evaluating x_dot = Ax + Bu; y = Cx + Du, keyed by ‘A_i_j’, ‘B_i_j’, ‘C_i_j’, ‘D_i_j’, where i is the H1 output channel number and j is the H1 input channel number

make_mimo_block(gtype, n_in, n_out, n_a, n_b, n_k)

Create the H1 block of a requested type and dimension.

The multiple-input multiple-output block will comprise a matrix of discrete H(z) transfer functions between each input and output pair, each having the same order.

Parameters:

  • gtype (str) – use stable2ndx for stable 2nd-order, iir for infinite impulse response, or fir for finite impulse response

  • n_in (int) – number of input channels (F1 outputs)

  • n_out (int) – number of output channels (F2 inputs)

  • n_a (int) – number of learnable denominator coefficients for each H(z)

  • n_b (int) – number of learnable numerator coefficients for each H(z)

  • n_k (int) – number of delay cells for each H(z)

Returns:

MimoLinearDynamicalOperator – a subclass from dynoNet

make_mimo_ylin(y_non)

Evaluates H1 from the F1 output, used in training or evaluation. The dynoNet function signature depends on gtype.

Parameters:

y_non (MimoStaticNonLinearity) – F1 evaluated

Returns:

MimoLinearDynamicalOperator – H1 evaluated

normalize(val, fac)

Normalizes one channel for the trained blocks.

Parameters:

  • list (float) – un-normalized channel data

  • fac (dict) – a member of normfacs with scale and offset

Returns:

list (float) – normalized channel data

printStateDicts()

Debugging output of the F1, F2, and H1 block parameters in memory.

read_lti(config)

Creates H1 from the JSON configuration

Parameters:

config (dict) – configuration section for H1 from JSON file, which must include the exported model members

Returns:

MimoLinearDynamicalOperator – H1 of the correct subclass

read_net(config)

Creates F1 or F2 from the JSON configuration

Parameters:

config (dict) – configuration section for F1 or F2 from JSON file, which must include the exported model members

Returns:

MimoStaticNonLinearity – F1 or F2

saveModelCoefficients(prefix='')

Save the current block coefficients

Default file names are F1.pkl, F2.pkl, and H1.pkl in model_folder.

Parameters:

prefix (str) – set as ‘best’ for saving the epoch with lowest fitting loss.

sensitivity_response(vals, bPrint=False)

Calculate the maximum d-axis and q-axis model sensitivity, using pre-calculated sens_mat for H1

Parameters:

vals (list(float)) – input vector for ub

Returns:

  • float – maximum d-axis sensitivity, de-normalized

  • float – maximum q-axis sensitivity, de-normalized

set_LCL_filter(Lf, Cf, Lc)

Set the inverter output filter parameters

These may be used to back-calculate voltages and currents at the voltage-source converter (VSC) terminals, based on voltages and currents at the AC terminals, i.e., at the point of commong coupling. However, this is not a causal operation.

Parameters:

  • Lf (float) – filter inductance between the VSC and filter capacitor

  • Cf (float) – filter capacitance

  • Lc (float) – filter inductance between the filter capacitor and the PCC

set_idx_in_out()

Populates idx_in, idx_out, d_key, and q_key.

The indices are set sequentially for the channels as ordered in COL_U and COL_Y. If COL_Y includes Id and Iq, d_key = Id and q_key = Iq for a Norton model. If COL_Y includes Vd and Vq, d_key = Vd and q_key = Vq for a Thevenin model.

set_sim_config(config, model_only=True)

Loads the attributes for simulation by forward evaluation

Parameters:

  • config (dict) – configuration from JSON file, which must include the exported model parameters

  • model_only (bool) – if True, don’t load the original model training configuration. If False, load configuration parameters that are sufficient for testing against the original data_train

setup_clamping_losses()

Normalize the output channel clamping limits and identify the clamped channel numbers, in preparation for adding clamping losses to fitting losses in model training.

setup_sensitivity_losses()

Parses the sensitivity configuration data, constructs the operating points for the sensitivity evaluation set. The existence of sensitivity should be checked before calling.

simulateVectors(T, G, Fc, Md, Mq, Vrms, GVrms, Ctl, npad)

Forward evaluation by processing individual vectors of input.

Deprecated: this function was only used with early Norton models that used Vrms instead of Vd and Vq inputs.

Parameters:

  • T (list(float)) – array of input temperature vs. time

  • G (list(float)) – array of input solar irradiance vs. time

  • Fc (list(float)) – array of input frequency vs. time

  • Md (list(float)) – array of input d-axis voltage control index vs. time

  • Mq (list(float)) – array of input q-axis voltage control index vs. time

  • Vrms (list(float)) – array of input RMS voltage vs. time

  • GVrms (list(float)) – array of input polynomial feature vs. time

  • Ctl (list(float)) – array of input control model vs. time

  • npad (int) – number of decimated-in-time points to pre-pad with initial values

Returns:

DC voltage vs. time list(float): DC current vs. time list(float): Id vs. time list(float): Iq vs. time

Return type:

list(float)

start_sensitivity_simulation(bPrint=False)

Build sens_mat from the H1 coefficients with z=-1, for efficiency in the sensitivity evaluations.

Parameters:

bPrint (bool) – if True, print out the H1 coefficients and sens_mat

start_simulation(bPrint=False)

Sets up the H1 history terms for an efficient IIR simulation of the trained model

Parameters:

bPrint (bool) – if True, print some information about the H1 IIR setup

Returns:

list (str) – names of input columns to help the user construct input vectors at each discrete time step

steady_state_response(vals)

Calculate steady-state block outputs using the IIR coefficients of H1.

Parameters:

vals (list(float)) – vector of de-normalized inputs, in order to match COL_U

Returns:

  • float – steady-state Id for a Norton model or Vd for a Thevenin model

  • float – steady-state Iq for a Norton model or Vq for a Thevenin model

  • float – steady-state DC voltage (if present in the output, NOT INCLUDED for now)

  • float – steady-state DC current (if present in the output, NOT INCLUDED for now)

stepOneCase(case_idx)

Compare the estimated and actual output for one of the cases used for training and validation.

This function uses the IIR filter coefficients and history terms to evaluate H1. The history terms allow proper initialization of H1 without pre-padding the data with initial values.

See also

testOneCase()

Parameters:

case_idx (int) – zero-based case number from data_train to test

Returns:

  • float – root mean square error for this case

  • float – mean absolute error for this case

  • array_like (float,ndim=2) – estimated output (y_hat) for this case, first dimension is time, second dimension is channel

  • array_like (float,ndim=2) – true output (y) for this case, first dimension is time, second dimension is channel

  • array_like (float,ndim=2) – input (u) for this case, first dimension is time, second dimension is channel

step_simulation(vals, nsteps=1)

Simulate one or more discrete time steps of a trained model using the IIR coefficients of H1

Parameters:

  • vals (list(float)) – vector of de-normalized inputs, in order to match COL_U

  • nsteps (int) – number of discrete time steps to simulate. This can be > 1 for brute-force initialization, but afterward it is usually 1.

Returns:

  • float – DC voltage

  • float – DC current

  • float – Id for a Norton model or Vd for a Thevenin model

  • float – Iq for a Norton model or Vq for a Thevenin model

testOneCase(case_idx, npad, bUseTorchDS=False, bLog=True)

Compare the estimated and actual output for one of the cases used for training and validation.

This function uses PyTorch to evaluate H1.

See also

stepOneCase()

Parameters:

  • case_idx (int) – zero-based case number from data_train to test

  • npad (int) – number of decimated time points to pre-pad with initial condition values

  • bUseTorchDS (bool) – if True, use the torch tensor DataLoader, which matches the training process but takes longer and does not properly match the initial conditions. If False, perform test on data_train directly.

  • bLog (bool) – if True, output diagnostics on the dataset sizes

Returns:

  • float – root mean square error for this case

  • float – mean absolute error for this case

  • array_like (float,ndim=2) – estimated output (y_hat) for this case, first dimension is time, second dimension is channel

  • array_like (float,ndim=2) – true output (y) for this case, first dimension is time, second dimension is channel

  • array_like (float,ndim=2) – input (u) for this case, first dimension is time, second dimension is channel

trainModelCoefficients(bMAE=False)

Supervises the model fitting and validation using Adam optimizer.

A PVInvDataset is created from data_train, which supports DataLoader*s from *PyTorch for training and validation. Batch sizes, continuation from a previous training run, and other options may be configured in the JSON file. Sensitivity and clamping may be added to the fitting loss, if configured in the JSON file.

Parameters:

bMAE (bool) – if True, optimize the mean absolute error (MAE) instead of root mean square error (RMSE)

Yields:

  • Block coefficients are trained in pkl files in the model_folder, and ready to export.

  • Training loss components written to ‘Loss.npy’ in the model_folder

  • After each epoch, if the current fitting loss is the best so far, model coefficients will be saved to bestF1.pkl, bestF2.pkl, and bestH1.pkl

Returns:

  • float – total training time elapsed between calls to time.time()

  • list (float) – fitting loss for each epoch

  • list (float) – validation loss for each epoch

  • list (float) – sensitivity loss for each epoch

trainingErrors(bByCase=False)

Summarize the fitting errors in a trained model.

Evaluates both root mean square error (RMSE) and mean absolute error (MAE).

Parameters:

bByCase (bool) – True if the individual fitting loss per case is desired, otherwise just the total is calculated.

Returns:

  • list (float) – total fitting RMSE over all cases and times, by output channel

  • list (float) – total fitting MAE over all cases and times, by output channel

  • array_like (float,ndim=2) – case RMSE over all times. First dimension is case number, second dimension is output channel. None if bByCase=False

  • array_like (float,ndim=2) – case MAE over all times. First dimension is case number, second dimension is output channel. None if bByCase=False

trainingLosses()

Evaluates the fitting loss of a trained model.

Returns:

float – total RMSE over all cases, times, and output channels

util

File support functions.

pecblocks.util.read_csv_files(path, pattern='.csv')

Adds a set of CSV files to a list of Pandas dataframes.

Each CSV file should use the comma as separator, column names in the first row. The Pandas read_csv function is called on each CSV file.

Parameters:

  • path (str) – this can be the name of a zip file, or a path to glob for files with pattern in the name.

  • pattern (list(str)) – the file extension to look for. Ignored for zip file input, and not a regular expression.

Returns:

list (DataFrame) – List of Pandas DataFrames, one per CSV file.

pecblocks.util.read_hdf5_file(filename, cols, n_dec=1, n_skip=0, n_trunc=0, prefix=None)

Adds each group to a list of Pandas dataframes.

The HDF5 file should include 0..n groups, each with the same column keys, record length, and sample interval. One column key should be ‘t’.

Parameters:

  • filename (str) – name of HDF5 file, one group per event record, groups indexed from 0

  • cols (list(str)) – list of column keys to extract from each group

  • n_dec (int) – decimation, i.e., take every n_dec point

  • n_skip (int) – number of samples, after decimation, to exclude from beginning of each event

  • n_trunc (int) – number of samples, after decimation, to exclude from end of each event

  • prefix (str) – optional prefix to the group number

Returns:

list (DataFrame) – List of Pandas DataFrames, one per group.