experanto.utils.FastSessionDataLoader

class FastSessionDataLoader(dataset, batch_size=1, shuffle=False, num_workers=0, pin_memory=False, drop_last=False, seed=None, **kwargs)

Bases: object

Optimized multi-session dataloader with state tracking.

Provides efficient data loading across multiple sessions with guarantees:

• Each session appears exactly once before repeating

• Epoch ends when the longest session is exhausted

• Perfect alignment between sessions and batches is maintained

• State is properly tracked and can be restored

Parameters:

• dataset (SessionConcatDataset) – Concatenated dataset with session tracking.

• batch_size (int, default=1) – Number of samples per batch.

• shuffle (bool, default=False) – Whether to shuffle samples within each session.

• num_workers (int, default=0) – Number of worker processes for data loading.

• pin_memory (bool, default=False) – Whether to pin memory for GPU transfer.

• drop_last (bool, default=False) – Whether to drop incomplete batches.

• seed (int, optional) – Random seed for reproducibility.

• **kwargs – Additional arguments passed to underlying DataLoaders.

session_names

Names of all sessions in the dataset.

Type:

list

batches_per_session

Number of batches in each session.

Type:

dict

See also

SessionConcatDataset

Dataset that tracks session membership.

LongCycler

Simpler alternative without state tracking.

Methods

__init__(dataset[, batch_size, shuffle, ...])
get_state()	Return the current state of the dataloader.
set_state(state)	Restore the dataloader state.

__init__(dataset, batch_size=1, shuffle=False, num_workers=0, pin_memory=False, drop_last=False, seed=None, **kwargs)

__len__()

Return the total number of batches in an epoch.

get_state()

Return the current state of the dataloader.

set_state(state)

Restore the dataloader state.

__iter__()

Iterate through sessions, cycling through them until all are exhausted.

The iteration scheme ensures: 1. Each session appears exactly once in each cycle 2. Samples within a session are properly batched and optionally shuffled 3. The epoch ends when the longest session is exhausted