{ "cells": [ { "cell_type": "markdown", "id": "dd717fcf-ebca-48f1-ba1c-7cfb3f064fdc", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "# HDF5 specification in Ubermag\n", "\n", "## Version 0.1\n", "\n", "This document is valid for HDF5 files written with `ubermag >= 2023.10`.\n", "\n", "**attributes**\n", "\n", "Each HDF5 file written with Ubermag has three standard attributes in the root group.\n", "\n", "- `file-creation-time-UTC: str`: Date and time (UTC) when the file was created in ISO 8601 format\n", "- `type: str`: The type of data in that file, expressed in terms of which class in Ubermag it represents. Currently specified are:\n", " - `discretisedfield.Field`: hdf5 file contains a single Field\n", "- `ubermag-hdf5-file-version: str`: Version of this file format specification.\n", "\n", "The versions of all Ubermag libraries involved in writing the file are saved as additional attributes:\n", "\n", "- `discretisedfield.__version__` (`str`): the version of `discretisedfield` used to save the Field.\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "fa41b2d5-ca27-456d-9273-1c4e999a6b4d", "metadata": { "tags": [] }, "outputs": [], "source": [ "import discretisedfield as df\n", "import h5glance" ] }, { "cell_type": "markdown", "id": "e2c8ac23-40d5-49e4-958d-6f5ff484ff71", "metadata": {}, "source": [ "### Single field: `discretisedfield`\n", "\n", "An HDF5 file with `type: discretisedfield.Field` contains a single field.\n", "It can be written with `field.to_file('.hdf5|.h5')`.\n", "\n", "The file has three nested groups, called `field`, `mesh`, and `region`.\n", "\n", "Example:" ] }, { "cell_type": "code", "execution_count": 2, "id": "5ecfc2b4-d472-47af-b157-df5d7ebb0aa1", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "nbval-ignore-output" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[94mhdf5_examples/discretisedfield-Field.h5\u001b[0m\n", "├4 attributes:\n", "│ ├discretisedfield.__version__: '0.90.0'\n", "│ ├file-creation-time-UTC: '2023-10-20T13:00:38'\n", "│ ├type: 'discretisedfield.Field'\n", "│ └ubermag-hdf5-file-version: '0.1'\n", "└\u001b[94mfield\u001b[0m\n", " ├3 attributes:\n", " │ ├nvdim: 3\n", " │ ├unit: 'A/m'\n", " │ └vdims: ['mx' 'my' 'mz']\n", " ├\u001b[1marray\u001b[0m\t[float64: 24 × 10 × 4 × 3]\n", " ├\u001b[94mmesh\u001b[0m\n", " │ ├2 attributes:\n", " │ │ ├bc: 'xy'\n", " │ │ └n: [24 10 4]\n", " │ ├\u001b[94mregion\u001b[0m\n", " │ │ └6 attributes:\n", " │ │ ├dims: ['x' 'y' 'z']\n", " │ │ ├ndim: 3\n", " │ │ ├pmax: [1.0e-07 2.5e-08 1.0e-08]\n", " │ │ ├pmin: [-2.0e-08 -2.5e-08 -1.0e-08]\n", " │ │ ├tolerance_factor: 1e-12\n", " │ │ └units: ['m' 'm' 'm']\n", " │ ├\u001b[1msubregion_names\u001b[0m\t[UTF-8 string: 2]\n", " │ └\u001b[1msubregions\u001b[0m\t[float64: 2 × 6]\n", " └\u001b[1mvalid\u001b[0m\t[enum (FALSE, TRUE): 24 × 10 × 4]\n" ] } ], "source": [ "!h5glance --attrs hdf5_examples/discretisedfield-Field.h5" ] }, { "cell_type": "markdown", "id": "e7fdb951-8d0a-4ad0-b182-b95f54b02033", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "#### The first group is called `field`. It contains:\n", "\n", "**attributes**\n", "\n", "- `nvdim` (`int`): Number of Value DIMensions of the field\n", "- `unit` (`str`): unit of the field; if not set it contains the value `'None'`\n", "- `vdims` (`list[str], length nvdim`): names of the vdims\n", "\n", "**datasets**\n", "\n", "- `array` (`array[mesh.n, nvdim], numbers.Number`): field data\n", "- `valid` (`array[mesh.n], bool`): boolean mask to mark (in)valid parts of `array`\n", "\n", "**groups**\n", "\n", "- `mesh` (`h5py.Group`): mesh on which the field is defined\n", "\n", "#### The second group is called `mesh`. It contains:\n", "\n", "\n", "**attributes**\n", "\n", "- `bc` (`str`): information about boundary conditions as defined in `discretisedfield.Mesh.bc`\n", "- `n` (`list[int], length region.ndim`): number of cells along the different spatial directions\n", "\n", "**datasets** (optional, only present if subregions are defined)\n", "\n", "- `subregion_names` (`list[str]`): names of the individual subregions defined for the field, only present if subregions are defined\n", "- `subregions` (`array[, region.ndim*2]`): coordinates of `discretisedfield.Region.pmin` and `discretisedfield.Region.pmax`, subregions and subregion names are combined based on their order, only present if subregions are defined\n", "\n", "**groups**\n", "\n", "- `region` (`h5py.Group`): region on which the mesh is defined\n", "\n", "#### The third group is called `region`. It contains:\n", "\n", "**attributes**\n", "\n", "- `dims` (`list[str], length ndim`): names of spatial dimensions\n", "- `ndim` (`int`): number of spatial dimensions\n", "- `pmin` (`list[number], length ndim`): coordinates of the minimum point `discretisedfield.Region.pmin`\n", "- `pmax` (`list[number], length ndim`): coordinates of the minimum point `discretisedfield.Region.pmin`\n", "- `tolerance_factor` (`float`): `discretisedfield.Region.tolerance_factor`\n", "- `units` (`list[str], length ndim`): units of the spatial dimensions\n", "\n", "#### Examples:\n", "**1d field with 1 vdim**" ] }, { "cell_type": "code", "execution_count": 3, "id": "cf5faefe-a6ce-44f5-bd21-5854d6c0f4a1", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "Field\n", "" ], "text/plain": [ "Field(Mesh(Region(pmin=[0], pmax=[20], dims=['x'], units=['m']), n=[20]), nvdim=1)" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mesh = df.Mesh(p1=0, p2=20, n=20)\n", "field_1d_scalar = df.Field(mesh, nvdim=1, value=0)\n", "field_1d_scalar" ] }, { "cell_type": "code", "execution_count": 4, "id": "20be65b4-8b8b-414e-99bd-d84585867752", "metadata": { "tags": [] }, "outputs": [], "source": [ "field_1d_scalar.to_file(\"field_1d_scalar.h5\")" ] }, { "cell_type": "code", "execution_count": 5, "id": "a692b81d-4a73-4058-8087-0b205329b27b", "metadata": { "tags": [ "nbval-ignore-output" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[94mfield_1d_scalar.h5\u001b[0m\n", "├4 attributes:\n", "│ ├discretisedfield.__version__: '0.90.0'\n", "│ ├file-creation-time-UTC: '2023-10-20T13:03:32'\n", "│ ├type: 'discretisedfield.Field'\n", "│ └ubermag-hdf5-file-version: '0.1'\n", "└\u001b[94mfield\u001b[0m\n", " ├3 attributes:\n", " │ ├nvdim: 1\n", " │ ├unit: 'None'\n", " │ └vdims: 'None'\n", " ├\u001b[1marray\u001b[0m\t[float64: 20 × 1]\n", " ├\u001b[94mmesh\u001b[0m\n", " │ ├2 attributes:\n", " │ │ ├bc: ''\n", " │ │ └n: [20]\n", " │ └\u001b[94mregion\u001b[0m\n", " │ └6 attributes:\n", " │ ├dims: ['x']\n", " │ ├ndim: 1\n", " │ ├pmax: [20]\n", " │ ├pmin: [0]\n", " │ ├tolerance_factor: 1e-12\n", " │ └units: ['m']\n", " └\u001b[1mvalid\u001b[0m\t[enum (FALSE, TRUE): 20]\n" ] } ], "source": [ "!h5glance --attrs field_1d_scalar.h5" ] }, { "cell_type": "code", "execution_count": 6, "id": "e3d5347e-8963-4008-9226-1b0e6c61eb6f", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.Field.from_file(\"field_1d_scalar.h5\") == field_1d_scalar" ] }, { "cell_type": "markdown", "id": "29ce5769-b8ad-480d-b13d-d955f1ca12ed", "metadata": {}, "source": [ "**1d field with 2 vdims**" ] }, { "cell_type": "code", "execution_count": 7, "id": "e803a1a9-414a-43d2-af54-89f01c6b344b", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "Field\n", "" ], "text/plain": [ "Field(Mesh(Region(pmin=[0], pmax=[20], dims=['x'], units=['m']), n=[20]), nvdim=2, vdims: (x, y))" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mesh = df.Mesh(p1=0, p2=20, n=20)\n", "field_1d = df.Field(mesh, nvdim=2, value=(0, 1))\n", "field_1d" ] }, { "cell_type": "code", "execution_count": 8, "id": "389578de-999c-461d-8f9c-ca1c3814663e", "metadata": { "tags": [] }, "outputs": [], "source": [ "field_1d.to_file(\"field_1d_vector.h5\")" ] }, { "cell_type": "code", "execution_count": 9, "id": "c34ead93-d1f1-4077-893e-e7afd8f165fe", "metadata": { "tags": [ "nbval-ignore-output" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[94mfield_1d_vector.h5\u001b[0m\n", "├4 attributes:\n", "│ ├discretisedfield.__version__: '0.90.0'\n", "│ ├file-creation-time-UTC: '2023-10-20T13:03:32'\n", "│ ├type: 'discretisedfield.Field'\n", "│ └ubermag-hdf5-file-version: '0.1'\n", "└\u001b[94mfield\u001b[0m\n", " ├3 attributes:\n", " │ ├nvdim: 2\n", " │ ├unit: 'None'\n", " │ └vdims: ['x' 'y']\n", " ├\u001b[1marray\u001b[0m\t[float64: 20 × 2]\n", " ├\u001b[94mmesh\u001b[0m\n", " │ ├2 attributes:\n", " │ │ ├bc: ''\n", " │ │ └n: [20]\n", " │ └\u001b[94mregion\u001b[0m\n", " │ └6 attributes:\n", " │ ├dims: ['x']\n", " │ ├ndim: 1\n", " │ ├pmax: [20]\n", " │ ├pmin: [0]\n", " │ ├tolerance_factor: 1e-12\n", " │ └units: ['m']\n", " └\u001b[1mvalid\u001b[0m\t[enum (FALSE, TRUE): 20]\n" ] } ], "source": [ "!h5glance --attrs field_1d_vector.h5" ] }, { "cell_type": "code", "execution_count": 10, "id": "71704a48-5251-450f-af08-887dcd7c6d70", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.Field.from_file(\"field_1d_vector.h5\") == field_1d" ] }, { "cell_type": "markdown", "id": "7b96f41b-8c13-487e-993f-3b2f6fc79e1b", "metadata": {}, "source": [ "**3d field with 3 vdims**" ] }, { "cell_type": "code", "execution_count": 11, "id": "95ae0a14-3131-4cf6-a8d4-6e1b57cb6ef4", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "Field\n", "" ], "text/plain": [ "Field(Mesh(Region(pmin=[-2e-08, -2.5e-08, -1e-08], pmax=[1e-07, 2.5e-08, 1e-08], dims=['x', 'y', 'z'], units=['m', 'm', 'm']), n=[24, 10, 4], bc=xy, subregions: (Region`sr1`(pmin=[0.0, 0.0, 0.0], pmax=[5e-09, 5e-09, 5e-09], dims=['x', 'y', 'z'], units=['m', 'm', 'm']), Region`sr2`(pmin=[-2e-08, -2.5e-08, -1e-08], pmax=[1e-07, 2.5e-08, 1e-08], dims=['x', 'y', 'z'], units=['m', 'm', 'm']))), nvdim=3, vdims: (mx, my, mz), unit=A/m)" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "region = df.Region(p1=(-20e-9, -25e-9, -10e-9), p2=(100e-9, 25e-9, 10e-9))\n", "mesh = df.Mesh(\n", " region=region,\n", " cell=(5e-9, 5e-9, 5e-9),\n", " bc=\"xy\",\n", " subregions={\n", " \"sr1\": df.Region(p1=(0, 0, 0), p2=(5e-9, 5e-9, 5e-9)),\n", " \"sr2\": region,\n", " },\n", ")\n", "\n", "\n", "def init_norm(p):\n", " x, y, _ = p\n", " if x > 50e-9 and abs(y) > 15e-9:\n", " return 0\n", " return 1e4\n", "\n", "\n", "field_3d = df.Field(\n", " mesh,\n", " nvdim=3,\n", " value=mesh.coordinate_field(),\n", " norm=init_norm,\n", " vdims=[\"mx\", \"my\", \"mz\"],\n", " valid=\"norm\",\n", " unit=\"A/m\",\n", ")\n", "field_3d" ] }, { "cell_type": "code", "execution_count": 12, "id": "adba5b6b-a62b-41ff-af50-e392e60431d3", "metadata": { "tags": [] }, "outputs": [], "source": [ "field_3d.to_file(\"field_3d.h5\")" ] }, { "cell_type": "code", "execution_count": 13, "id": "a4959219-69db-4455-b289-5889a5d8ad05", "metadata": { "tags": [ "nbval-ignore-output" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[94mfield_3d.h5\u001b[0m\n", "├4 attributes:\n", "│ ├discretisedfield.__version__: '0.90.0'\n", "│ ├file-creation-time-UTC: '2023-10-20T13:03:33'\n", "│ ├type: 'discretisedfield.Field'\n", "│ └ubermag-hdf5-file-version: '0.1'\n", "└\u001b[94mfield\u001b[0m\n", " ├3 attributes:\n", " │ ├nvdim: 3\n", " │ ├unit: 'A/m'\n", " │ └vdims: ['mx' 'my' 'mz']\n", " ├\u001b[1marray\u001b[0m\t[float64: 24 × 10 × 4 × 3]\n", " ├\u001b[94mmesh\u001b[0m\n", " │ ├2 attributes:\n", " │ │ ├bc: 'xy'\n", " │ │ └n: [24 10 4]\n", " │ ├\u001b[94mregion\u001b[0m\n", " │ │ └6 attributes:\n", " │ │ ├dims: ['x' 'y' 'z']\n", " │ │ ├ndim: 3\n", " │ │ ├pmax: [1.0e-07 2.5e-08 1.0e-08]\n", " │ │ ├pmin: [-2.0e-08 -2.5e-08 -1.0e-08]\n", " │ │ ├tolerance_factor: 1e-12\n", " │ │ └units: ['m' 'm' 'm']\n", " │ ├\u001b[1msubregion_names\u001b[0m\t[UTF-8 string: 2]\n", " │ └\u001b[1msubregions\u001b[0m\t[float64: 2 × 6]\n", " └\u001b[1mvalid\u001b[0m\t[enum (FALSE, TRUE): 24 × 10 × 4]\n" ] } ], "source": [ "!h5glance --attrs field_3d.h5" ] }, { "cell_type": "code", "execution_count": 14, "id": "1878ff3d-9103-4d2d-903f-51185fefca1b", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.Field.from_file(\"field_3d.h5\") == field_3d" ] }, { "cell_type": "markdown", "id": "9f554613-662d-4e95-8a77-396276923bbd", "metadata": {}, "source": [ "### Multiple fields or time series: `micromagneticdata`\n", "\n", "**Currently not specified**\n", "\n", "#### Internal notes:\n", "\n", "The top-level name of the field, which for discretisedfield.Field is fixed to `field`, is arbitrary and has to be defined in the new code. The internal structure of the field group matches the one described above with one difference: `array` can have an additional variable first dimension: `array[, mesh.n, nvdim]`, which can e.g. be use for time-series information. Writing fields can be done with the private methods `Field._h5_save_structure`, and `Field._h5_save_data`. The former saves all \"metadata\" such as the mesh and valid and should only be called once. It requires a `h5py.group` for the field, which has to be defined separately, and the `array` shape as inputs. The latter can write data of individual Fields into parts of the `array` dataset. It should be called for each field in the series." ] }, { "cell_type": "markdown", "id": "ded27c4a-9ef4-4a05-80a5-fcd06349fa23", "metadata": {}, "source": [ "## Changes\n", "\n", "- HDF5 files written with `ubermag<=0.66.1` have a different structure and lack significant parts of the metadata, e.g. subregion information and valid. `discretisedfield` can read these files with the same method." ] }, { "cell_type": "code", "execution_count": 15, "id": "e4f4438d-203a-42a4-b44d-1fc244e23252", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [ "# cleanup\n", "!rm field_1d_scalar.h5 field_1d_vector.h5 field_3d.h5" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12" } }, "nbformat": 4, "nbformat_minor": 5 }