elfes.io.hdf5

Read and write the ELFES HDF5 format.

An ELFES HDF5 file stores a non-empty collection of physical Sample objects in quantity-major continuous arrays. Each file is self-contained: it carries the geometry, basis definitions, electronic-data descriptions, and dataset metadata needed to interpret its samples. elfes.data.HDF5Dataset may concatenate several files as ordered homogeneous shards, but sharding is not part of the single-file format.

Shape names

• n_samples: number of samples.
• n_atoms: total number of atoms across all samples.
• n_species: number of species in one basis set.
• n_shells: total number of shells in one basis set.
• n_blocks: total stored blocks for one block-sparse electronic quantity.
• n_values: total stored value rows for one electronic quantity.
• extra_shape: the quantity's fixed trailing shape, possibly empty.

File structure

/
├── metadata/                         # string attributes
├── samples/
│   ├── ids                           string [n_samples]
│   ├── atom_offsets                  int64 [n_samples + 1]
│   ├── atomic_numbers                int32 [n_atoms]
│   ├── positions                     float64 [n_atoms, 3]
│   ├── cells                         float64 [n_samples, 3, 3]
│   ├── pbc                           bool [n_samples, 3]
│   └── magmoms                       float64 [n_atoms] or [n_atoms, 3]  # optional
├── basis/
│   └── <role>/                       # e.g. ao, aux, or paw_coupled
└── electronic_data/
    └── <name>/                       # one named electronic quantity

atom_offsets[s:s+2] gives the half-open atom range of sample s. Geometry arrays are therefore continuous across samples, while cells and pbc have one row per sample. Optional magmoms contains input atomic magnetic moments in μB. Its absence means no sample has this input; when present, every sample is collinear with shape [n_atoms] or every sample is noncollinear with shape [n_atoms, 3]. HDF5Reader.magmom_components reports this dataset-level definition as None, "z", or "xyz". Dataset metadata is stored as string attributes of /metadata.

Basis sets

Role names form an open string namespace. ao, aux, and paw_coupled are common conventions rather than an exhaustive list; the role is otherwise only used to associate electronic data with its basis set.

Every /basis/<role> group has a basis_type attribute with value "layout", "spline_numerical", "uniform_numerical", or "gaussian". All four types contain the common shell layout:

atomic_numbers                    int32 [n_species]
shell_offsets                    int64 [n_species + 1]
angmoms                          int32 [n_shells]
names                            string [n_species]

atomic_numbers[i] identifies species i, and shell_offsets[i:i+2] selects its shells. Empty entries in names mean that the corresponding atomic basis has no source-level name. A "layout" basis ends here and records only shell angular momenta and ordering.

The three types containing radial functions additionally have a basis-set attribute name; an empty string means that the basis set has no name. Their numerical arrays always use float64, independently of the precision selected for electronic-data values.

Spline numerical basis

A "spline_numerical" basis stores an explicit radial grid for each species together with the sampled radial functions:

radial_grid_offsets              int64 [n_species + 1]
radial_grid                      float64 [n_radial_points]
radial_value_offsets             int64 [n_species + 1]
radial_values                    float64 [n_radial_values]

For species i, adjacent radial_grid_offsets select its radial knots in Å, and adjacent radial_value_offsets select the C-order flattening of its [n_atomic_shells, n_grid_points] radial-value array. Here n_atomic_shells = shell_offsets[i + 1] - shell_offsets[i] and n_grid_points is the selected radial-grid length. Radial values are bare radial functions in Å\(^{-3/2}\).

Uniform numerical basis

A "uniform_numerical" basis has a float64 attribute radial_spacing in Å, shared by all species in the basis set, and stores only the sampled radial functions:

radial_value_offsets             int64 [n_species + 1]
radial_values                    float64 [n_radial_values]

For species i, adjacent offsets select the C-order flattening of its [n_atomic_shells, n_grid_points] array. Different species may have different numbers of grid points; the selected value count divided by n_atomic_shells gives n_grid_points. Radial knot j is always at j * radial_spacing. The file does not store the resulting radial-grid arrays or shell cutoff indices; each cutoff is derived from the corresponding radial function's exact zero tail. Radial values are bare radial functions in Å\(^{-3/2}\).

Gaussian basis

A "gaussian" basis stores flattened contracted-Gaussian data in global shell order:

primitive_offsets               int64 [n_shells + 1]
primitive_exponents             float64 [n_primitives]
contraction_coefficients        float64 [n_primitives]

Adjacent primitive_offsets select the primitive exponents and contraction coefficients of one shell. Primitive exponents use Å\(^{-2}\), and the dimensionless coefficients multiply individually normalized primitive radial functions.

Electronic data

Each /electronic_data/<name> group describes one quantity shared by every sample. The name carries the physical role chosen by the dataset, such as hamiltonian or overlap; storage is selected independently by these string attributes:

• data_type: "block_sparse_orb_matrix", "herm_block_sparse_orb_matrix", "orb_matrix", "herm_orb_matrix", "triu_orb_matrix", "orb_vector", "uniform_volumetric", or "quadrature_volumetric".
• basis_role: the corresponding open role name (commonly ao, aux, or paw_coupled), or an empty string for volumetric data.
• pauli: the ordered Pauli components, or an empty string when absent.

All quantities contain values_real with shape [n_values, *extra_shape]. Complex quantities also contain values_imag with the same shape and dtype; real quantities omit it. Both use the float32 or float64 precision selected for the file. The leading axis contains flattened spatial or orbital values, while extra_shape preserves all fixed leading axes of the physical object. When pauli is present, its component axis is the final axis of extra_shape.

Block-sparse orbital matrices

data_type="block_sparse_orb_matrix" and data_type="herm_block_sparse_orb_matrix" contain:

block_offsets                    int64 [n_samples + 1]
atom_pairs                       int32 [n_blocks, 2]
pair_shifts                      int32 [n_blocks, 3]
value_offsets                    int64 [n_blocks + 1]
values_real                      float32|float64 [n_values, *extra_shape]
values_imag                      float32|float64 [n_values, *extra_shape]  # complex only

block_offsets[s:s+2] selects the stored blocks of sample s. atom_pairs[b] = (i, j) identifies the row and column atoms, and pair_shifts[b] identifies the cell image of atom j. The block shape is derived from the sample's atoms and the named basis role; its C-order-flattened spatial values occupy value_offsets[b:b+2]. extra_shape may contain arbitrary fixed leading dimensions. When pauli is present, its component axis is the final axis of extra_shape.

For block_sparse_orb_matrix, every stored key is an actual directed block and a missing key represents zero. For herm_block_sparse_orb_matrix, only the lexicographically first key from each Hermitian-partner pair is stored; the omitted partner is the conjugate transpose, and onsite blocks are themselves Hermitian.

Dense orbital matrices

All dense orbital-matrix data types contain sample-bounded value_offsets with shape [n_samples + 1] and the common values arrays. The orbital count is derived from each sample's atoms and the named basis role.

• data_type="orb_matrix" stores all n_orbitals**2 matrix entries in C order. The reader reshapes them into the full matrix.
• data_type="herm_orb_matrix" stores n_orbitals * (n_orbitals + 1) // 2 entries from the upper triangle, including the diagonal, in NumPy triu_indices row-major order. The reader reconstructs the strict lower triangle by conjugate transpose.
• data_type="triu_orb_matrix" uses the same packed upper order and value count. The reader fills the strict lower triangle with zero.

Orbital vectors

data_type="orb_vector" contains sample-bounded value_offsets with shape [n_samples + 1] and the common values arrays. Each sample contributes n_orbitals rows, which reconstruct an OrbVector with shape [*extra_shape, n_orbitals].

Uniform volumetric data

data_type="uniform_volumetric" is real and contains:

value_offsets                    int64 [n_samples + 1]
values_real                      float32|float64 [n_values, *extra_shape]
origins                          float64 [n_samples, 3]
step_vectors                     float64 [n_samples, 3, 3]
grid_shapes                      int64 [n_samples, 3]

Each sample's three grid axes are flattened in C order into its values range. The grid periodicity is the owning sample's pbc and is not stored again.

Quadrature volumetric data

data_type="quadrature_volumetric" is real and contains:

value_offsets                    int64 [n_samples + 1]
values_real                      float32|float64 [n_values, *extra_shape]
coordinates                     float64 [n_values, 3]
weights                         float64 [n_values]

The values range of each sample is also its quadrature-point range, so every value row is aligned with one coordinate and weight.

Offsets and public access

Every offsets array starts at zero, has one terminal entry, and selects half-open ranges from the corresponding continuous arrays. Counts are recovered by taking adjacent differences and are not stored separately.

Use write_hdf5() to stream (sample_id, sample) pairs into a file. The ID is the stable dataset-member key and is not part of the physical Sample. Use HDF5Reader to inspect IDs and dataset-level definitions or reconstruct selected physical data as complete Sample objects.

ElectronicDataDescription(
    data_type: ElectronicDataType,
    basis_role: str | None,
    pauli: str | None,
    is_complex: bool,
    precision: int,
    extra_shape: tuple[int, ...],
)

HDF5Reader(path: StrPath)

continuous_counts(electronic_data_names: Collection[str] | None = None) -> ArrayCounts

read_continuous_into(arrays: HDF5Arrays, starts: ArrayStarts) -> None

write_hdf5(
    path: StrPath,
    items: Iterable[tuple[str, Sample]],
    *,
    precision: Precision = 64,
    compression: Compression = "none",
    compression_level: int = 4,
    metadata: dict[str, str] | None = None,
    overwrite: bool = False,
) -> int