Standard Convolutional Models#

Models that use standard (non-dilated) 1D convolutions with BasicEncodingMixin for simpler inference.

Series2Vec#

Temporal encoding via SoftDTW-based contrastive loss.

class chronocratic.models.convolutional.standard.series2vec.model.Series2Vec(input_dims: int, embedding_dims: int = 16, num_heads: int = 8, feedforward_dims: int = 256, representation_dims: int = 320, dropout_rate: float = 0.01, encoder_kernel_size: int = 8, learning_rate: float = 0.001, soft_dtw_gamma: float = 0.1, *, sync_dist: bool = False, optimizer_name: str = 'RAdam', weight_decay: float = 0.0)#

Bases: LightningModule, BasicEncodingMixin

Lightning wrapper for Series2Vec pretraining.

The public input shape is (batch, time, channels).

This model was implemented based on the code available on this GitHub repo Navidfoumani/Series2Vec.

forward(x: Tensor) → Tensor#

Return Series2Vec representations for x.

property encoder: Module#

Return the Series2Vec network for inspection and checkpointing.

training_step(batch: Tensor, _batch_idx: int) → Tensor#

Compute and log the Series2Vec pretraining loss for one batch.

validation_step(batch: Tensor, _batch_idx: int) → Tensor#

Compute and log the Series2Vec validation loss for one batch.

configure_optimizers() → Optimizer#

Return the configured optimizer for Series2Vec pretraining.

property representation_dim: int#

Flattened representation size (temporal + frequency concatenated).

Returns:

2 * representation_dims — the output dimension of Series2VecNetwork.encode().

Configuration for the Series2Vec model.

Provides Series2VecModelParameters with all settings for the dual time/frequency Series2Vec encoder, soft-DTW target computation, and optimizer choice.

class chronocratic.models.convolutional.standard.series2vec.config.Series2VecModelParameters(*, input_dims: int, embedding_dims: int = 16, num_heads: int = 8, feedforward_dims: int = 256, representation_dims: int = 320, dropout_rate: float = 0.01, encoder_kernel_size: int = 8, learning_rate: float = 0.001, soft_dtw_gamma: float = 0.1, sync_dist: bool = False, optimizer_name: Literal['Adam', 'RAdam', 'AdamW'] = 'RAdam', weight_decay: float = 0.0)#

Bases: object

Configuration for the Series2Vec model.

Parameters:

  • input_dims – Number of input features (channels) in the time series.

  • embedding_dims – Token embedding dimensionality. Defaults to 16.

  • num_heads – Number of attention heads in the transformer encoder. Defaults to 8.

  • feedforward_dims – Hidden dimensionality of the transformer feed-forward block. Defaults to 256.

  • representation_dims – Output dimensionality of the projection head used for pretraining. Defaults to 320.

  • dropout_rate – Dropout probability applied throughout the network. Defaults to 0.01.

  • encoder_kernel_size – Kernel size of the convolutional tokenizer.

  • learning_rate – Base learning rate for the optimizer.

  • soft_dtw_gamma – Smoothing parameter for the soft-DTW distance used as the temporal target.

  • sync_dist – Whether to synchronize logged metrics across distributed processes.

  • optimizer_name – Optimizer to use; one of 'Adam', 'RAdam', or 'AdamW'.

  • weight_decay – L2 weight-decay coefficient passed to the optimizer.

TSTCC#

Temporal contrastive clustering for representation learning.

class chronocratic.models.convolutional.standard.tstcc.model.TSTCC(input_dims: int, conv_kernel_size: int, stride: int, features_len: int, num_classes: int, output_dims: int = 128, encoder_channels: tuple[int, ...] = (32, 64), encoder_inner_kernels: tuple[int, ...] = (8, 8), dropout_rate: float = 0.35, temporal_contrast_hidden_dim: int = 100, temporal_contrast_timesteps: int = 6, temperature: float = 0.2, *, use_cosine_similarity: bool = True, learning_rate: float = 0.0003, temporal_loss_weight: float = 1.0, contextual_loss_weight: float = 0.7, sync_dist: bool = False, augmentation: AugmentationProducer[ViewPair] | None = None)#

Bases: LightningModule, BasicEncodingMixin

PyTorch Lightning module for TS-TCC (self-supervised pretraining only).

Single-purpose model for temporal + contextual contrastive pre-training on augmented views. Labels are ignored during pretraining.

Batch format: (data, labels) where labels is ignored. Two augmented views of data are produced by the injected AugmentationProducer[ViewPair] (e.g. _default_tstcc_pair()), which provides Gaussian scaling (weak) and segment-permutation + jitter (strong) views.

Uses automatic_optimization = False because two separate optimizers (one per sub-module) must be stepped independently.

For downstream classification or regression, use SupervisedModule from chronocratic.models.supervised.

This model was implemented based on the code available on this GitHub repo emadeldeen24/TS-TCC under MIT License.

forward(x: Tensor) → tuple[Tensor, Tensor]#

Run the encoder. Returns (logits, features).

training_step(batch: tuple[Tensor, Tensor], _batch_idx: int) → Tensor#

Manual optimization step for both sub-module optimizers.

validation_step(batch: tuple[Tensor, Tensor], _batch_idx: int) → Tensor#

Compute and log validation loss.

configure_optimizers() → OptimizerLRScheduler#

Return one Adam optimizer per sub-module (encoder and TC model).

property encoder: Module#

Return the TCC encoder for inspection and checkpointing.

property representation_dim: int#

Flattened pre-logits feature size of the TCC encoder.

Returns:

The input dimension of the encoder’s logits nn.Linear layer, which equals output_dims * features_len.

Configuration for the TS-TCC model.

Provides TSTCCModelParameters with all settings for the TCC encoder, temporal contrast head, and NT-Xent contextual loss for self-supervised pretraining.

class chronocratic.models.convolutional.standard.tstcc.config.TSTCCModelParameters(*, input_dims: int, conv_kernel_size: int, stride: int, output_dims: int = 128, encoder_channels: tuple[int, ...] = (32, 64), encoder_inner_kernels: tuple[int, ...] = (8, 8), features_len: int, num_classes: int, dropout_rate: float = 0.35, temporal_contrast_hidden_dim: int = 100, temporal_contrast_timesteps: int = 6, temperature: float = 0.2, use_cosine_similarity: bool = True, learning_rate: float = 0.0003, temporal_loss_weight: float = 1.0, contextual_loss_weight: float = 0.7, sync_dist: bool = False)#

Bases: object

Configuration for the TS-TCC model.

Parameters:

  • input_dims – Number of input features (dimensions) in the time series.

  • conv_kernel_size – Convolutional kernel size used in the TCC encoder.

  • stride – Convolutional stride used in the TCC encoder.

  • output_dims – Number of channels produced by the final encoder block (also used as the temporal-contrast input dim).

  • encoder_channels – Tuple of channel counts for the first two encoder convolution blocks.

  • encoder_inner_kernels – Tuple of kernel sizes for the inner convolution blocks (block 2 and block 3).

  • features_len – Length of the encoder feature map fed into the logits head.

  • num_classes – Number of output classes for the encoder logits head.

  • dropout_rate – Dropout probability applied inside the TCC encoder.

  • temporal_contrast_hidden_dim – Hidden dimensionality of the temporal contrast module.

  • temporal_contrast_timesteps – Number of future timesteps the temporal contrast module predicts.

  • temperature – Temperature scaling for the NT-Xent contextual loss.

  • use_cosine_similarity – Whether the NT-Xent loss uses cosine similarity (otherwise dot-product).

  • learning_rate – Base learning rate for the two Adam optimizers (encoder and temporal-contrast).

  • temporal_loss_weight – Weight of the temporal-contrast loss term in the self-supervised objective.

  • contextual_loss_weight – Weight of the contextual NT-Xent loss term in the self-supervised objective.

  • sync_dist – Whether to synchronize logged metrics across distributed processes.

FCN (MCL)#

Multi-scale contrastive learning with a minimal FCN architecture.

class chronocratic.models.convolutional.standard.mcl.model.FCN(input_dims: int, output_dims: int = 320, alpha: float = 1.0, learning_rate: float = 0.001, encoder_channels: tuple[int, ...] = (128, 256, 128), encoder_kernels: tuple[int, ...] = (7, 5, 3), encoder_dilations: tuple[int, ...] = (2, 4, 8), projection_dims: int = 128, sync_dist: bool = False)#

Bases: LightningModule, BasicEncodingMixin

FCN encoder for Mixup Contrastive Learning (MCL).

This model was implemented based on the code available on this GitHub repo Wickstrom/MixupContrastiveLearning.

property encoder: Module#

Return the FCN encoder.

forward(x: Tensor) → Tensor#

Return projected MCL representations for x.

training_step(batch: Tensor, _batch_idx: int) → Tensor#

Compute and log the training loss for one batch.

validation_step(batch: Tensor, _batch_idx: int) → Tensor#

Compute and log the validation loss for one batch.

configure_optimizers() → Optimizer#

Return the Adam optimizer used to train MCL.

Configuration for the MCL (MixUp Contrastive Learning) model.

Provides MCLModelParameters with MCL-specific settings for the FCN encoder and MixUp contrastive criterion.

class chronocratic.models.convolutional.standard.mcl.config.MCLModelParameters(*, input_dims: int, output_dims: int = 320, alpha: float = 1.0, learning_rate: float = 0.001, encoder_channels: tuple[int, ...] = (128, 256, 128), encoder_kernels: tuple[int, ...] = (7, 5, 3), encoder_dilations: tuple[int, ...] = (2, 4, 8), projection_dims: int = 128, sync_dist: bool = False)#

Bases: object

Configuration for the MCL model.

Parameters:

  • input_dims – Number of input features (channels) in the time series.

  • output_dims – Number of output features produced by the encoder.

  • alpha – Beta-distribution parameter controlling the MixUp interpolation coefficient.

  • learning_rate – Base learning rate for the Adam optimizer.

  • encoder_channels – Tuple of channel counts for each Conv1d block in the FCN encoder.

  • encoder_kernels – Tuple of kernel sizes for each Conv1d block in the FCN encoder.

  • encoder_dilations – Tuple of dilation rates for each Conv1d block in the FCN encoder.

  • projection_dims – Hidden dimension of the projection head used for contrastive learning.

  • sync_dist – Whether to synchronize metrics across distributed processes during logging.