Standard Convolutional Models#

Models that use standard (non-dilated) 1D convolutions with BasicEncodingMixin for simpler inference.

Series2Vec#

Temporal encoding via SoftDTW-based contrastive loss.

class chronocratic.models.convolutional.standard.series2vec.model.Series2Vec(*, input_dim: int, embedding_dim: int = 16, num_heads: int = 8, feedforward_dim: int = 256, representation_dim: int = 320, dropout_rate: float = 0.01, encoder_kernel_size: int = 8, learning_rate: float = 0.001, soft_dtw_gamma: float = 0.1, singleton_split_count: int = 3, normalization_layer_type: NormalizationLayerType = NormalizationLayerType.CHANNEL, sync_dist: bool = False, optimizer_name: str = 'RAdam', weight_decay: float = 0.0)#

Bases: LightningModule, BasicEncodingMixin

Lightning wrapper for Series2Vec pretraining.

Learns representations by aligning pairwise distance matrices between temporal and frequency-domain encodings using soft-DTW and Euclidean losses. Dual-branch architecture with separate conv encoders for time and frequency domains, followed by transformer-based cross-attention.

The public input shape is (batch, time, channels).

The encoder defaults to GroupNorm (normalization_layer_type=CHANNEL), ensuring correct gradient flow at batch_size=1 (unlike BatchNorm, which degenerates with zero variance statistics for single-sample batches). Pass normalization_layer_type=BATCH to reproduce the upstream BatchNorm architecture exactly.

Parameters:

  • input_dim – Number of input features (channels).

  • embedding_dim – Token embedding dimensionality.

  • num_heads – Number of attention heads in the transformer encoder.

  • feedforward_dim – Hidden dimensionality of the transformer feed-forward block.

  • representation_dim – Output dimensionality of the encoding (temporal + frequency concatenated). Must be even.

  • dropout_rate – Dropout probability applied throughout the network.

  • encoder_kernel_size – Kernel size of the convolutional tokenizer.

  • learning_rate – Base learning rate for the optimizer.

  • soft_dtw_gamma – Smoothing parameter for the soft-DTW distance used as the temporal target.

  • singleton_split_count – Number of contiguous windows to split a singleton batch into for pairwise loss computation.

  • normalization_layer_type – Normalization strategy for the DisjoinEncoder. CHANNEL (default) uses GroupNorm for batch_size=1 safety. BATCH uses BatchNorm.

  • sync_dist – Whether to synchronize logged metrics across distributed processes.

  • optimizer_name – Optimizer to use; one of 'Adam', 'RAdam', or 'AdamW'.

  • weight_decay – L2 weight-decay coefficient passed to the optimizer.

This model was implemented based on the code available on this GitHub repo Navidfoumani/Series2Vec.

forward(x: Tensor) → Tensor#

Return Series2Vec representations for x.

property encoder: Module#

Return the Series2Vec network for inspection and checkpointing.

training_step(batch: Tensor, _batch_idx: int) → Tensor#

Compute and log the Series2Vec pretraining loss for one batch.

validation_step(batch: Tensor, _batch_idx: int) → Tensor#

Compute and log the Series2Vec validation loss for one batch.

configure_optimizers() → Optimizer#

Return the configured optimizer for Series2Vec pretraining.

property representation_dim: int#

Flattened representation size (temporal + frequency concatenated).

Returns:

representation_dim — the output dimension of Series2VecNetwork.encode(), matching the constructor parameter.

Configuration for the Series2Vec model.

Provides Series2VecModelParameters with all settings for the dual time/frequency Series2Vec encoder, soft-DTW target computation, and optimizer choice.

class chronocratic.models.convolutional.standard.series2vec.config.Series2VecModelParameters(*, input_dim: int, embedding_dim: int = 16, num_heads: int = 8, feedforward_dim: int = 256, representation_dim: int = 320, dropout_rate: float = 0.01, encoder_kernel_size: int = 8, learning_rate: float = 0.001, soft_dtw_gamma: float = 0.1, sync_dist: bool = False, optimizer_name: Literal['Adam', 'RAdam', 'AdamW'] = 'RAdam', weight_decay: float = 0.0, normalization_layer_type: NormalizationLayerType = NormalizationLayerType.CHANNEL, singleton_split_count: int = 3)#

Bases: object

Configuration for the Series2Vec model.

Parameters:

  • input_dim – Number of input features (channels) in the time series.

  • embedding_dim – Token embedding dimensionality. Defaults to 16.

  • num_heads – Number of attention heads in the transformer encoder. Defaults to 8.

  • feedforward_dim – Hidden dimensionality of the transformer feed-forward block. Defaults to 256.

  • representation_dim – Output dimensionality of the projection head used for pretraining. Defaults to 320.

  • dropout_rate – Dropout probability applied throughout the network. Defaults to 0.01.

  • encoder_kernel_size – Kernel size of the convolutional tokenizer.

  • learning_rate – Base learning rate for the optimizer.

  • soft_dtw_gamma – Smoothing parameter for the soft-DTW distance used as the temporal target.

  • sync_dist – Whether to synchronize logged metrics across distributed processes.

  • optimizer_name – Optimizer to use; one of 'Adam', 'RAdam', or 'AdamW'.

  • weight_decay – L2 weight-decay coefficient passed to the optimizer.

  • normalization_layer_type – Normalization strategy for encoder and projection head. CHANNEL (default) uses GroupNorm for batch_size=1 safety. BATCH uses BatchNorm.

  • singleton_split_count – Number of contiguous windows to split a singleton batch into for pairwise loss computation. Defaults to 3 to ensure sufficient gradients at batch_size=1.

TSTCC#

Temporal contrastive clustering for representation learning.

class chronocratic.models.convolutional.standard.tstcc.model.TSTCC(*, input_dim: int, conv_kernel_size: int = 8, stride: int = 1, representation_dim: int = 128, encoder_channels: tuple[int, ...] = (32, 64), encoder_inner_kernels: tuple[int, ...] = (8, 8), dropout_rate: float = 0.35, temporal_contrast_hidden_dim: int = 100, temporal_contrast_timesteps: int = 6, temperature: float = 0.2, use_cosine_similarity: bool = True, learning_rate: float = 0.0003, temporal_loss_weight: float = 1.0, contextual_loss_weight: float = 0.7, weight_decay: float = 0.0003, sync_dist: bool = False, normalization_layer_type: NormalizationLayerType = NormalizationLayerType.CHANNEL, augmentation: AugmentationProducer[ViewPair] | None = None)#

Bases: LightningModule, BasicEncodingMixin

PyTorch Lightning module for TS-TCC (self-supervised pretraining only).

Single-purpose model for temporal + contextual contrastive pre-training on augmented views. Labels are ignored during pretraining.

Batch format: (data, labels) where labels is ignored. Two augmented views of data are produced by the injected AugmentationProducer[ViewPair] (e.g. _default_tstcc_pair()), which provides Gaussian scaling (weak) and segment-permutation + jitter (strong) views.

Uses automatic_optimization = False because two separate optimizers (one per sub-module) must be stepped independently.

For downstream classification or regression, use SupervisedModule from chronocratic.models.supervised.

This model was implemented based on the code available on this GitHub repo emadeldeen24/TS-TCC under MIT License.

Parameters:

  • input_dim – Number of input features (channels).

  • conv_kernel_size – Kernel size for the first convolution block.

  • stride – Stride for the first convolution block.

  • representation_dim – Number of output channels from the encoder.

  • encoder_channels – Channel counts for the first two conv blocks.

  • encoder_inner_kernels – Kernel sizes for the second and third conv blocks.

  • dropout_rate – Dropout rate applied after the first conv block.

  • temporal_contrast_hidden_dim – Hidden dimension for the transformer and projection head in TemporalContrast.

  • temporal_contrast_timesteps – Number of timesteps for temporal contrastive prediction.

  • temperature – Temperature parameter for NT-Xent loss.

  • use_cosine_similarity – Whether to use cosine similarity in NT-Xent.

  • learning_rate – Learning rate for Adam optimizers.

  • temporal_loss_weight – Weight for the temporal contrastive loss.

  • contextual_loss_weight – Weight for the contextual NT-Xent loss.

  • weight_decay – Weight decay for Adam optimizers.

  • sync_dist – Whether to synchronize metrics across processes.

  • normalization_layer_type – Normalization strategy. CHANNEL uses GroupNorm(1, C) in the encoder and LayerNorm in the projection head, which is batch-size independent and avoids degeneracy at small batch sizes. BATCH uses BatchNorm1d. Defaults to CHANNEL.

  • augmentation – Optional custom augmentation producer. Defaults to the standard TSTCC pair (Gaussian scaling + segment permutation with jitter).

forward(x: Tensor) → Tensor#

Run the encoder. Returns convolutional feature map (B, C, L').

training_step(batch: tuple[Tensor, Tensor], _batch_idx: int) → Tensor#

Manual optimization step for both sub-module optimizers.

validation_step(batch: tuple[Tensor, Tensor], _batch_idx: int) → Tensor#

Compute and log validation loss.

configure_optimizers() → OptimizerLRScheduler#

Return one Adam optimizer per sub-module (encoder and TC model).

property encoder: Module#

Return the TCC encoder for inspection and checkpointing.

property representation_dim: int#

Representation dimension after global average pooling.

Returns:

The encoder’s representation_dim, matching the pooled feature shape (B, representation_dim).

Configuration for the TS-TCC model.

Provides TSTCCModelParameters with all settings for the TCC encoder, temporal contrast head, and NT-Xent contextual loss for self-supervised pretraining.

class chronocratic.models.convolutional.standard.tstcc.config.TSTCCModelParameters(*, input_dim: int, conv_kernel_size: int = 8, stride: int = 1, representation_dim: int = 128, encoder_channels: tuple[int, ...] = (32, 64), encoder_inner_kernels: tuple[int, ...] = (8, 8), dropout_rate: float = 0.35, temporal_contrast_hidden_dim: int = 100, temporal_contrast_timesteps: int = 6, temperature: float = 0.2, use_cosine_similarity: bool = True, learning_rate: float = 0.0003, temporal_loss_weight: float = 1.0, contextual_loss_weight: float = 0.7, weight_decay: float = 0.0003, sync_dist: bool = False, normalization_layer_type: NormalizationLayerType = NormalizationLayerType.CHANNEL)#

Bases: object

Configuration for the TS-TCC model.

Parameters:

  • input_dim – Number of input features (dimensions) in the time series.

  • conv_kernel_size – Convolutional kernel size used in the TCC encoder. Defaults to 8 matching the source TS-TCC implementation.

  • stride – Convolutional stride used in the TCC encoder. Defaults to 1 matching the source TS-TCC implementation.

  • representation_dim – Number of channels produced by the final encoder block (also used as the temporal-contrast input dim).

  • encoder_channels – Tuple of channel counts for the first two encoder convolution blocks.

  • encoder_inner_kernels – Tuple of kernel sizes for the inner convolution blocks (block 2 and block 3).

  • dropout_rate – Dropout probability applied inside the TCC encoder.

  • temporal_contrast_hidden_dim – Hidden dimensionality of the temporal contrast module.

  • temporal_contrast_timesteps – Number of future timesteps the temporal contrast module predicts.

  • temperature – Temperature scaling for the NT-Xent contextual loss.

  • use_cosine_similarity – Whether the NT-Xent loss uses cosine similarity (otherwise dot-product).

  • learning_rate – Base learning rate for the two Adam optimizers (encoder and temporal-contrast).

  • temporal_loss_weight – Weight of the temporal-contrast loss term in the self-supervised objective.

  • contextual_loss_weight – Weight of the contextual NT-Xent loss term in the self-supervised objective.

  • weight_decay – Weight decay for the Adam optimizers.

  • sync_dist – Whether to synchronize logged metrics across distributed processes.

  • normalization_layer_type – Normalization strategy for TCCEncoder and TemporalContrast. CHANNEL (default) uses GroupNorm for encoder conv blocks and LayerNorm for the projection head — safe at batch_size=1. BATCH uses BatchNorm1d.

FCN (MCL)#

Multi-scale contrastive learning with a minimal FCN architecture.

class chronocratic.models.convolutional.standard.mcl.model.MCL(*, input_dim: int, representation_dim: int = 128, alpha: float = 1.0, learning_rate: float = 0.001, encoder_channels: tuple[int, ...] = (128, 256, 128), encoder_kernels: tuple[int, ...] = (7, 5, 3), encoder_dilations: tuple[int, ...] = (2, 4, 8), projection_dim: int = 128, sync_dist: bool = False, normalization_layer_type: NormalizationLayerType = NormalizationLayerType.CHANNEL)#

Bases: LightningModule, BasicEncodingMixin

FCN-based encoder for Mixup Contrastive Learning (MCL).

This model was implemented based on the code available on this GitHub repo Wickstrom/MixupContrastiveLearning.

Parameters:

  • input_dim – Number of input feature channels.

  • representation_dim – Dimension of the flat encoder output.

  • alpha – Beta-distribution parameter for MixUp interpolation.

  • learning_rate – Base learning rate for the Adam optimizer.

  • encoder_channels – Tuple of channel counts for each Conv1d block.

  • encoder_kernels – Tuple of kernel sizes for each Conv1d block.

  • encoder_dilations – Tuple of dilation rates for each Conv1d block.

  • projection_dim – Hidden dimension of the projection head.

  • sync_dist – Whether to synchronize metrics across processes.

  • normalization_layer_type – Normalization strategy for encoder and projection head. Use CHANNEL for GroupNorm (batch_size=1 safe) or BATCH for BatchNorm1d. Defaults to CHANNEL.

property representation_dim: int#

Return the feature dim of encode()’s output.

property encoder: Module#

Return the encoder.

forward(x: Tensor) → Tensor#

Return projected MCL representations for x.

training_step(batch: Tensor, _batch_idx: int) → Tensor#

Compute and log the training loss for one batch.

validation_step(batch: Tensor, _batch_idx: int) → Tensor#

Compute and log the validation loss for one batch.

configure_optimizers() → Optimizer#

Return the Adam optimizer used to train MCL.

Configuration for the MCL (MixUp Contrastive Learning) model.

Provides MCLModelParameters with MCL-specific settings for the FCN encoder and MixUp contrastive criterion.

class chronocratic.models.convolutional.standard.mcl.config.MCLModelParameters(*, input_dim: int, representation_dim: int = 128, alpha: float = 1.0, learning_rate: float = 0.001, encoder_channels: tuple[int, ...] = (128, 256, 128), encoder_kernels: tuple[int, ...] = (7, 5, 3), encoder_dilations: tuple[int, ...] = (2, 4, 8), projection_dim: int = 128, sync_dist: bool = False, normalization_layer_type: NormalizationLayerType = NormalizationLayerType.CHANNEL)#

Bases: object

Configuration for the MCL model.

Parameters:

  • input_dim – Number of input features (channels) in the time series.

  • representation_dim – Width of the vector encode() returns. The representation size of the encoder output.

  • alpha – Beta-distribution parameter controlling the MixUp interpolation coefficient.

  • learning_rate – Base learning rate for the Adam optimizer.

  • encoder_channels – Tuple of channel counts for each Conv1d block in the FCN encoder.

  • encoder_kernels – Tuple of kernel sizes for each Conv1d block in the FCN encoder.

  • encoder_dilations – Tuple of dilation rates for each Conv1d block in the FCN encoder.

  • projection_dim – Hidden dimension of the projection head used for contrastive learning.

  • sync_dist – Whether to synchronize metrics across distributed processes during logging.

  • normalization_layer_type – Normalization strategy for encoder and projection head. CHANNEL (default) uses GroupNorm for batch_size=1 safety. BATCH uses BatchNorm1d.