mdsuite.memory_management package

Submodules

Module contents

MDSuite: A Zincwarecode package.

License

This program and the accompanying materials are made available under the terms of the Eclipse Public License v2.0 which accompanies this distribution, and is available at https://www.eclipse.org/legal/epl-v20.html

SPDX-License-Identifier: EPL-2.0

Copyright Contributors to the Zincwarecode Project.

Contact Information

email: zincwarecode@gmail.com github: https://github.com/zincware web: https://zincwarecode.com/

Citation

If you use this module please cite us with:

Summary

class mdsuite.memory_management.MemoryManager(data_path: Optional[list] = None, database: Optional[Database] = None, parallel: bool = False, memory_fraction: float = 0.2, scale_function: Optional[dict] = None, gpu: bool = False, offset: int = 0)[source]

Bases: object

A class to manage memory during MDSuite operations.

In most MDSuite operations the memory requirements of a complete procedure will be greater than what is available to the average computer. Therefore, some degree of memory management is required for smooth running. This class should be instantiated during any operation so it can update the operation on what the current memory capabilities are.

The class can work with several tensor_values-sets in the case an analysis requires this much tensor_values.

data_path
Type:

list

database
Type:

Database

parallel
Type:

bool

memory_fraction
Type:

float

scale_function
Type:

dict

gpu
Type:

bool

get_batch_size() → tuple[source]

Calculate the batch size of an operation.

This method takes the tensor_values requirements of an operation and returns how big each batch of tensor_values should be for such an operation.

Returns:

  • batch_size (int) – number of elements that should be loaded at one time

  • number_of_batches (int) – number of batches which can be loaded.

  • remainder (int) – number of elements that will be left unloaded after a loop over all batches. This amount can then be loaded to collect unused tensor_values.

get_ensemble_loop(data_range: int, correlation_time: int = 1) → Tuple[int, bool][source]

Get the tensor_values range partition quantity.

Not only does tensor_values need to be batched, it then needs to be looped over in order to calculate some property. This method will return the number of loops possible given the tensor_values range and the correlation time

Parameters:

  • data_range (int) – Data range to be used in the analysis.

  • correlation_time (int) – Correlation time to be considered when looping over the tensor_values

Returns:

data_range_partitions – Number of time the batch can be looped over with given data_range and correlation time.

Return type:

int

static hdf5_load_time(n: int)[source]

Describes the load time of a hdf5 database i.e. O(log N).

Parameters:

n (int) – Amount of data to be loaded

Returns:

load_time – Load time of N data points from a hdf5 database.

Return type:

float