TileDB Python API Reference¶

Warning

The Python interface to TileDB is still under development and the API is subject to change.

Modules¶

Typical usage of the Python interface to TileDB will use the top-level module tiledb, e.g.

import tiledb

There is also a submodule libtiledb which contains the necessary bindings to the underlying TileDB native library. Most of the time you will not need to interact with tiledb.libtiledb unless you need native-library specific information, e.g. the version number:

import tiledb
tiledb.libtiledb.version()  # Native TileDB library version number

Exceptions¶

exception tiledb.TileDBError¶

TileDB Error Exception

Captures and raises error return code (TILEDB_ERR) messages when calling libtiledb functions. The error message that is raised is the last error set for the tiledb.Ctx

A Python MemoryError is raised on TILEDB_OOM

message¶

The TileDB error message string

Return type:	str
Returns:	error message

Context¶

class tiledb.Ctx(config=None)¶

Class representing a TileDB context.

A TileDB context wraps a TileDB storage manager.

Parameters:	config (tiledb.Config or dict) – Initialize Ctx with given config parameters

config(self)¶: Returns the Config instance associated with the Ctx

set_tag(self, key, value)¶: Sets a string:string “tag” on the Ctx

Config¶

class tiledb.Config(params=None, path=None)¶

TileDB Config class

See: https://docs.tiledb.io/en/stable/tutorials/config.html#summary-of-parameters

Unknown parameters will be ignored!

Parameters:	params (dict) – Set parameter values from dict like object path (str) – Set parameter values from persisted Config parameter file

clear(self)¶: Unsets all Config parameters (returns them to their default values)

dict(self, prefix=u'')¶

Returns a dict representation of a Config object

Parameters:	prefix (str) – return only parameters with a given prefix
Return type:	dict
Returns:	Config parameter / values as a a Python dict

from_file(self, path)¶

Update a Config object with from a persisted config file

Parameters:	path – A local Config file path

get(self, key, *args)¶

Gets the value of a config parameter, or a default value.

Parameters:	key (str) – Config parameter args – return arg if Config does not contain parameter key
Returns:	Parameter value, arg or None.

items(self, prefix=u'')¶

Returns an iterator object over Config parameters, values

Parameters:	prefix (str) – return only parameters with a given prefix
Return type:	ConfigItems
Returns:	iterator over Config parameter, value tuples

keys(self, prefix=u'')¶

Returns an iterator object over Config parameters (keys)

Parameters:	prefix (str) – return only parameters with a given prefix
Return type:	ConfigKeys
Returns:	iterator over Config parameter string keys

static load(uri)¶

Constructs a Config class instance from config parameters loaded from a local Config file

Parameters:	uri (str) – a local URI config file path
Return type:	tiledb.Config
Returns:	A TileDB Config instance with persisted parameter values
Raises:	TypeError – uri cannot be converted to a unicode string
Raises:	`tiledb.TileDBError`

save(self, uri)¶

Persist Config parameter values to a config file

Parameters:	uri (str) – a local URI config file path
Raises:	TypeError – uri cannot be converted to a unicode string
Raises:	`tiledb.TileDBError`

update(self, odict)¶

Update a config object with parameter, values from a dict like object

Parameters:	odict – dict-like object containing parameter, values to update Config.

values(self, prefix=u'')¶

Returns an iterator object over Config values

Parameters:	prefix (str) – return only parameters with a given prefix
Return type:	ConfigValues
Returns:	iterator over Config string values

Array Schema¶

class tiledb.ArraySchema(domain=None, attrs=(), cell_order='row-major', tile_order='row-major', capacity=0, coords_filters=None, offsets_filters=None, allows_duplicates=False, sparse=False, Ctx ctx=None)¶

Schema class for TileDB dense / sparse array representations

Parameters:	attrs (tuple(tiledb.Attr, ..)) – one or more array attributes cell_order ('row-major' or 'C', 'col-major' or 'F') – TileDB label for cell layout tile_order ('row-major' or 'C', 'col-major' or 'F', 'unordered') – TileDB label for tile layout capacity (int) – tile cell capacity coords_filters (tiledb.FilterList) – (default None) coordinate filter list offsets_filters (tiledb.FilterList) – (default None) offsets filter list allow_duplicates (bool) – True if duplicates are allowed sparse (bool) – True if schema is sparse, else False (set by SparseArray and DenseArray derived classes) ctx (tiledb.Ctx) – A TileDB Context
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

allows_duplicates¶: Returns True if the (sparse) array allows duplicates.

attr(self, key)¶

Returns an Attr instance given an int index or string label

Parameters:	key (int or str) – attribute index (positional or associative)
Return type:	tiledb.Attr
Returns:	The ArraySchema attribute at index or with the given name (label)
Raises:	TypeError – invalid key type

attr_or_dim_dtype(self, unicode name)¶

capacity¶

The array capacity

Return type:	int
Raises:	`tiledb.TileDBError`

cell_order¶: The cell order layout of the array.

coords_compressor¶

The compressor label and level for the array’s coordinates.

Return type:	tuple(str, int)
Raises:	`tiledb.TileDBError`

coords_filters¶

The FilterList for the array’s coordinates

Return type:	tiledb.FilterList
Raises:	`tiledb.TileDBError`

domain¶

The Domain associated with the array.

Return type:	tiledb.Domain
Raises:	`tiledb.TileDBError`

dump(self)¶: Dumps a string representation of the array object to standard output (stdout)

has_attr(self, name)¶

Returns true if the given name is an Attribute of the ArraySchema

Parameters:	name – attribute name
Return type:	boolean

static load(uri, Ctx ctx=None, key=None)¶

nattr¶

The number of array attributes.

Return type:	int
Raises:	`tiledb.TileDBError`

ndim¶

The number of array domain dimensions.

Return type:	int

offsets_compressor¶

The compressor label and level for the array’s variable-length attribute offsets.

Return type:	tuple(str, int)
Raises:	`tiledb.TileDBError`

offsets_filters¶

The FilterList for the array’s variable-length attribute offsets

Return type:	tiledb.FilterList
Raises:	`tiledb.TileDBError`

shape¶

The array’s shape

Return type:	tuple(numpy scalar, numpy scalar)
Raises:	TypeError – floating point (inexact) domain

sparse¶

True if the array is a sparse array representation

Return type:	bool
Raises:	`tiledb.TileDBError`

tile_order¶

The tile order layout of the array.

Return type:	str
Raises:	`tiledb.TileDBError`

Attribute¶

class tiledb.Attr(name=u'', dtype=np.float64, var=False, filters=None, Ctx ctx=None)¶

Class representing a TileDB array attribute.

Parameters:	ctx (tiledb.Ctx) – A TileDB Context name (str) – Attribute name, empty if anonymous dtype (bool) – Attribute value datatypes var – Attribute is variable-length (automatic for byte/string types) filters (FilterList) – List of filters to apply
Raises:	TypeError – invalid dtype
Raises:	`tiledb.TileDBError`

compressor¶

String label of the attributes compressor and compressor level

Return type:	tuple(str, int)
Raises:	`tiledb.TileDBError`

dtype¶

Return numpy dtype object representing the Attr type

Return type:	numpy.dtype

dump(self)¶: Dumps a string representation of the Attr object to standard output (stdout)

filters¶

FilterList of the TileDB attribute

Return type:	tiledb.FilterList
Raises:	`tiledb.TileDBError`

isanon¶

True if attribute is an anonymous attribute

Return type:	bool

isvar¶

True if the attribute is variable length

Return type:	bool
Raises:	`tiledb.TileDBError`

name¶

Attribute string name, empty string if the attribute is anonymous

Return type:	str
Raises:	`tiledb.TileDBError`

ncells¶

The number of cells (scalar values) for a given attribute value

Return type:	int
Raises:	`tiledb.TileDBError`

Filters¶

class tiledb.FilterList(filters=None, chunksize=None, Ctx ctx=None)¶

An ordered list of Filter objects for filtering TileDB data.

FilterLists contain zero or more Filters, used for filtering attribute data, the array coordinate data, etc.

Parameters:	ctx (tiledb.Ctx) – A TileDB context filters – An iterable of Filter objects to add. chunksize (int) – (default None) chunk size used by the filter list in bytes

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     # Create several filters
...     gzip_filter = tiledb.GzipFilter()
...     bw_filter = tiledb.BitWidthReductionFilter()
...     # Create a filter list that will first perform bit width reduction, then gzip compression.
...     filters = tiledb.FilterList([bw_filter, gzip_filter])
...     a1 = tiledb.Attr(name="a1", dtype=np.int64, filters=filters)
...     # Create a second attribute filtered only by gzip compression.
...     a2 = tiledb.Attr(name="a2", dtype=np.int64,
...                      filters=tiledb.FilterList([gzip_filter]))
...     schema = tiledb.ArraySchema(domain=dom, attrs=(a1, a2))
...     tiledb.DenseArray.create(tmp + "/array", schema)

append(self, Filter filter)¶

Appends filter to the end of filter list

Parameters:	filter (Filter) – filter object to add
Returns:	None

chunksize¶: The chunk size used by the filter list.

nfilters¶

Returns:	Number of filters in the filter list
Return type:	int

class tiledb.libtiledb.CompressionFilter(tiledb_filter_type_t filter_type, level, Ctx ctx=None)¶

Base class for filters performing compression.

All compression filters support a compression level option, although some (such as RLE) ignore it.

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     a1 = tiledb.Attr(name="a1", dtype=np.int64,
...                      filters=tiledb.FilterList([tiledb.GzipFilter(level=10)]))
...     schema = tiledb.ArraySchema(domain=dom, attrs=(a1,))
...     tiledb.DenseArray.create(tmp + "/array", schema)

level¶

The compression level setting for the filter.

Every compressor interprets this value differently (some ignore it, such as RLE).

Returns:	compression level
Return type:	int

class tiledb.GzipFilter(level=None, Ctx ctx=None)¶

Filter that compresses using gzip.

Parameters:	ctx (tiledb.Ctx) – TileDB Ctx level (int) – (default None) If not None set the compressor level

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     a1 = tiledb.Attr(name="a1", dtype=np.int64,
...                      filters=tiledb.FilterList([tiledb.GzipFilter()]))
...     schema = tiledb.ArraySchema(domain=dom, attrs=(a1,))
...     tiledb.DenseArray.create(tmp + "/array", schema)

class tiledb.ZstdFilter(level=None, Ctx ctx=None)¶

Filter that compresses using zstd.

Parameters:	ctx (tiledb.Ctx) – TileDB Ctx level (int) – (default None) If not None set the compressor level

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     a1 = tiledb.Attr(name="a1", dtype=np.int64,
...                      filters=tiledb.FilterList([tiledb.ZstdFilter()]))
...     schema = tiledb.ArraySchema(domain=dom, attrs=(a1,))
...     tiledb.DenseArray.create(tmp + "/array", schema)

class tiledb.LZ4Filter(level=None, Ctx ctx=None)¶

Filter that compresses using lz4.

Parameters:	ctx (tiledb.Ctx) – TileDB Ctx level (int) – (default None) If not None set the compressor level

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     a1 = tiledb.Attr(name="a1", dtype=np.int64,
...                      filters=tiledb.FilterList([tiledb.LZ4Filter()]))
...     schema = tiledb.ArraySchema(domain=dom, attrs=(a1,))
...     tiledb.DenseArray.create(tmp + "/array", schema)

class tiledb.Bzip2Filter(level=None, Ctx ctx=None)¶

Filter that compresses using bzip2.

Parameters:	level (int) – (default None) If not None set the compressor level

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     a1 = tiledb.Attr(name="a1", dtype=np.int64,
...                      filters=tiledb.FilterList([tiledb.Bzip2Filter()]))
...     schema = tiledb.ArraySchema(domain=dom, attrs=(a1,))
...     tiledb.DenseArray.create(tmp + "/array", schema)

class tiledb.RleFilter(Ctx ctx=None)¶

Filter that compresses using run-length encoding (RLE).

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     a1 = tiledb.Attr(name="a1", dtype=np.int64,
...                      filters=tiledb.FilterList([tiledb.RleFilter()]))
...     schema = tiledb.ArraySchema(domain=dom, attrs=(a1,))
...     tiledb.DenseArray.create(tmp + "/array", schema)

class tiledb.DoubleDeltaFilter(Ctx ctx=None)¶

Filter that performs double-delta encoding.

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     a1 = tiledb.Attr(name="a1", dtype=np.int64,
...                      filters=tiledb.FilterList([tiledb.DoubleDeltaFilter()]))
...     schema = tiledb.ArraySchema(domain=dom, attrs=(a1,))
...     tiledb.DenseArray.create(tmp + "/array", schema)

class tiledb.BitShuffleFilter(Ctx ctx=None)¶

Filter that performs a bit shuffle transformation.

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     a1 = tiledb.Attr(name="a1", dtype=np.int64,
...                      filters=tiledb.FilterList([tiledb.BitShuffleFilter()]))
...     schema = tiledb.ArraySchema(domain=dom, attrs=(a1,))
...     tiledb.DenseArray.create(tmp + "/array", schema)

class tiledb.ByteShuffleFilter(Ctx ctx=None)¶

Filter that performs a byte shuffle transformation.

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     a1 = tiledb.Attr(name="a1", dtype=np.int64,
...                      filters=tiledb.FilterList([tiledb.ByteShuffleFilter()]))
...     schema = tiledb.ArraySchema(domain=dom, attrs=(a1,))
...     tiledb.DenseArray.create(tmp + "/array", schema)

class tiledb.BitWidthReductionFilter(window=None, Ctx ctx=None)¶

Filter that performs bit-width reduction.

param ctx: A TileDB Context

type ctx: tiledb.Ctx

param window: (default None) max window size for the filter

type window: int

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     a1 = tiledb.Attr(name="a1", dtype=np.int64,
...                      filters=tiledb.FilterList([tiledb.BitWidthReductionFilter()]))
...     schema = tiledb.ArraySchema(domain=dom, attrs=(a1,))
...     tiledb.DenseArray.create(tmp + "/array", schema)

window¶

Returns:	The maximum window size used for the filter
Return type:	int

class tiledb.PositiveDeltaFilter(window=None, Ctx ctx=None)¶

Filter that performs positive-delta encoding.

Parameters:	ctx (tiledb.Ctx) – A TileDB Context window (int) – (default None) the max window for the filter

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     a1 = tiledb.Attr(name="a1", dtype=np.int64,
...                      filters=tiledb.FilterList([tiledb.PositiveDeltaFilter()]))
...     schema = tiledb.ArraySchema(domain=dom, attrs=(a1,))
...     tiledb.DenseArray.create(tmp + "/array", schema)

window¶

Returns:	The maximum window size used for the filter
Return type:	int

Dimension¶

class tiledb.Dim(name=u'__dim_0', domain=None, tile=None, dtype=np.uint64, Ctx ctx=None)¶

Class representing a dimension of a TileDB Array.

Parameters:	name (str) – the dimension name, empty if anonymous domain (tuple(int, int) or tuple(float, float)) – tile (int or float) – Tile extent ctx (tiledb.Ctx) – A TileDB Context
Dtype:	the Dim numpy dtype object, type object, or string that can be corerced into a numpy dtype object
Raises:	ValueError – invalid domain or tile extent TypeError – invalid domain, tile extent, or dtype type
Raises:	`TileDBError`

domain¶

The dimension (inclusive) domain.

The dimension’s domain is defined by a (lower bound, upper bound) tuple.

Return type:	tuple(numpy scalar, numpy scalar)

dtype¶

Numpy dtype representation of the dimension type.

Return type:	numpy.dtype

isanon¶

True if the dimension is anonymous

Return type:	bool

isvar¶

True if the dimension is variable length

Return type:	bool
Raises:	`tiledb.TileDBError`

name¶

The dimension label string.

Anonymous dimensions return a default string representation based on the dimension index.

Return type:	str

shape¶

The shape of the dimension given the dimension’s domain.

Note: The shape is only valid for integer and datetime dimension domains.

Return type:	tuple(numpy scalar, numpy scalar)
Raises:	TypeError – floating point (inexact) domain

size¶

The size of the dimension domain (number of cells along dimension).

Return type:	int
Raises:	TypeError – floating point (inexact) domain

tile¶

The tile extent of the dimension.

Return type:	numpy scalar or np.timedelta64

Domain¶

class tiledb.Domain(Ctx ctx=None, *dims)¶

Class representing the domain of a TileDB Array.

Parameters:	dims – one or more tiledb.Dim objects up to the Domain’s ndim ctx (tiledb.Ctx) – A TileDB Context
Raises:	TypeError – All dimensions must have the same dtype
Raises:	`TileDBError`

dim(self, dim_id)¶

Returns a Dim object from the domain given the dimension’s index or name.

Parameters:	dim_d – dimension index (int) or name (str)
Raises:	`tiledb.TileDBError`

dtype¶

The numpy dtype of the domain’s dimension type.

Return type:	numpy.dtype

dump(self)¶: Dumps a string representation of the domain object to standard output (STDOUT)

has_dim(self, name)¶

Returns true if the Domain has a Dimension with the given name

Parameters:	name – name of Dimension
Return type:	bool
Returns:

homogeneous¶: Returns True if the domain’s dimension types are homogeneous.

ndim¶

The number of dimensions of the domain.

Return type:	int

shape¶

The domain’s shape, valid only for integer domains.

Return type:	tuple
Raises:	TypeError – floating point (inexact) domain

size¶

The domain’s size (number of cells), valid only for integer domains.

Return type:	int
Raises:	TypeError – floating point (inexact) domain

Array¶

class tiledb.libtiledb.Array(uri, mode='r', key=None, timestamp=None, attr=None, Ctx ctx=None)¶

Base class for TileDB array objects.

Defines common properties/functionality for the different array types. When an Array instance is initialized, the array is opened with the specified mode.

Parameters:	uri (str) – URI of array to open mode (str) – (default ‘r’) Open the array object in read ‘r’ or write ‘w’ mode key (str) – (default None) If not None, encryption key to decrypt the array timestamp (int) – (default None) If not None, open the array at a given TileDB timestamp ctx (Ctx) – TileDB context

attr(self, key)¶

Returns an Attr instance given an int index or string label

Parameters:	key (int or str) – attribute index (positional or associative)
Return type:	`Attr`
Returns:	The array attribute at index or with the given name (label)
Raises:	TypeError – invalid key type

close(self)¶: Closes this array, flushing all buffered data.

consolidate(self, Config config=None, key=None)¶

Consolidates fragments of an array object for increased read performance.

Parameters:	config (tiledb.Config) – The TileDB Config with consolidation parameters set key (str or bytes) – (default None) encryption key to decrypt an encrypted array
Raises:	`tiledb.TileDBError`

coords_dtype¶

Returns the numpy record array dtype of the array coordinates

Return type:	numpy.dtype
Returns:	coord array record dtype

create(type cls, uri, ArraySchema schema, key=None, Ctx ctx=None)¶

Creates a persistent TileDB Array at the given URI

Parameters:	uri (str) – URI at which to create the new empty array. schema (ArraySchema) – Schema for the array key (str) – (default None) Encryption key to use for array Ctx (ctx) – (default None) Optional TileDB Ctx used when creating the array, by default uses the ArraySchema’s associated context.

dim(self, dim_id)¶

Returns a Dim instance given a dim index or name

Parameters:	key (int or str) – attribute index (positional or associative)
Return type:	`Attr`
Returns:	The array attribute at index or with the given name (label)
Raises:	TypeError – invalid key type

domain¶: The Domain of this array.

dtype¶: The NumPy dtype of the specified attribute

dump(self)¶

isopen¶: True if this array is currently open.

iswritable¶: This array is currently opened as writable.

static load_typed(uri, mode='r', key=None, timestamp=None, attr=None, Ctx ctx=None)¶

mode¶: The mode this array was opened with.

nattr¶: The number of attributes of this array.

ndim¶: The number of dimensions of this array.

nonempty_domain(self)¶

Return the minimum bounding domain which encompasses nonempty values.

Return type:	tuple(tuple(numpy scalar, numpy scalar), ..)
Returns:	A list of (inclusive) domain extent tuples, that contain all nonempty cells

reopen(self, timestamp=None)¶

Reopens this array.

This is useful when the array is updated after it was opened. To sync-up with the updates, the user must either close the array and open again, or just use reopen() without closing. Reopening will be generally faster than the former alternative.

schema¶: The ArraySchema for this array.

shape¶: The shape of this array.

subarray(self, selection, attrs=None, coords=False, order=None)¶

timestamp¶

Returns the timestamp the array is opened at

Return type:	int
Returns:	tiledb timestamp at which point the array was opened

uri¶: Returns the URI of the array

tiledb.consolidate(uri, key=None, Config config=None, Ctx ctx=None)¶

Consolidates TileDB array fragments for improved read performance

Parameters:	uri (str) – URI to the TileDB Array key (str) – (default None) Key to decrypt array if the array is encrypted config (tiledb.Config) – The TileDB Config with consolidation parameters set ctx (tiledb.Ctx) – (default None) The TileDB Context
Return type:	str or bytes
Returns:	path (URI) to the consolidated TileDB Array
Raises:	TypeError – cannot convert path to unicode string
Raises:	`tiledb.TileDBError`

Dense Array¶

class tiledb.DenseArray¶

__getitem__(selection)¶

Retrieve data cells for an item or region of the array.

Parameters:	selection (tuple) – An int index, slice or tuple of integer/slice objects, specifying the selected subarray region for each dimension of the DenseArray.
Return type:	`numpy.ndarray` or `collections.OrderedDict`
Returns:	If the dense array has a single attribute than a Numpy array of corresponding shape/dtype is returned for that attribute. If the array has multiple attributes, a `collections.OrderedDict` is returned with dense Numpy subarrays for each attribute.
Raises:	IndexError – invalid or unsupported index selection
Raises:	`tiledb.TileDBError`

Example:

>>> import tiledb, numpy as np, tempfile
>>> with tempfile.TemporaryDirectory() as tmp:
...     # Creates array 'array' on disk.
...     A = tiledb.DenseArray.from_numpy(tmp + "/array",  np.ones((100, 100)))
...     # Many aspects of Numpy's fancy indexing are supported:
...     A[1:10, ...].shape
...     A[1:10, 20:99].shape
...     A[1, 2].shape
(9, 100)
(9, 79)
()
>>> # Subselect on attributes when reading:
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64))
...     schema = tiledb.ArraySchema(domain=dom,
...         attrs=(tiledb.Attr(name="a1", dtype=np.int64),
...                tiledb.Attr(name="a2", dtype=np.int64)))
...     tiledb.DenseArray.create(tmp + "/array", schema)
...     with tiledb.DenseArray(tmp + "/array", mode='w') as A:
...         A[0:10] = {"a1": np.zeros((10)), "a2": np.ones((10))}
...     with tiledb.DenseArray(tmp + "/array", mode='r') as A:
...         # Access specific attributes individually.
...         A[0:5]["a1"]
...         A[0:5]["a2"]
array([0, 0, 0, 0, 0])
array([1, 1, 1, 1, 1])

__setitem__(selection, value)¶

Set / update dense data cells

Parameters:	selection (tuple) – An int index, slice or tuple of integer/slice objects, specifiying the selected subarray region for each dimension of the DenseArray. value (dict or `numpy.ndarray`) – a dictionary of array attribute values, values must able to be converted to n-d numpy arrays. if the number of attributes is one, than a n-d numpy array is accepted.
Raises:	IndexError – invalid or unsupported index selection ValueError – value / coordinate length mismatch
Raises:	`tiledb.TileDBError`

Example:

>>> import tiledb, numpy as np, tempfile
>>> # Write to single-attribute 2D array
>>> with tempfile.TemporaryDirectory() as tmp:
...     # Create an array initially with all zero values
...     with tiledb.DenseArray.from_numpy(tmp + "/array",  np.zeros((2, 2))) as A:
...         pass
...     with tiledb.DenseArray(tmp + "/array", mode='w') as A:
...         # Write to the single (anonymous) attribute
...         A[:] = np.array(([1,2], [3,4]))
>>>
>>> # Write to multi-attribute 2D array
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(
...         tiledb.Dim(domain=(0, 1), tile=2, dtype=np.uint64),
...         tiledb.Dim(domain=(0, 1), tile=2, dtype=np.uint64))
...     schema = tiledb.ArraySchema(domain=dom,
...         attrs=(tiledb.Attr(name="a1", dtype=np.int64),
...                tiledb.Attr(name="a2", dtype=np.int64)))
...     tiledb.DenseArray.create(tmp + "/array", schema)
...     with tiledb.DenseArray(tmp + "/array", mode='w') as A:
...         # Write to each attribute
...         A[0:2, 0:2] = {"a1": np.array(([-3, -4], [-5, -6])),
...                        "a2": np.array(([1, 2], [3, 4]))}

Sparse Array¶

class tiledb.SparseArray¶

__getitem__(selection)¶

Retrieve nonempty cell data for an item or region of the array

Parameters:	selection (tuple) – An int index, slice or tuple of integer/slice objects, specifying the selected subarray region for each dimension of the SparseArray.
Return type:	`collections.OrderedDict`
Returns:	An OrderedDict is returned with “coords” coordinate values being the first key. “coords” is a Numpy record array representation of the coordinate values of non-empty attribute cells. Nonempty attribute values are returned as Numpy 1-d arrays.
Raises:	IndexError – invalid or unsupported index selection
Raises:	`tiledb.TileDBError`

Example:

>>> import tiledb, numpy as np, tempfile
>>> # Write to multi-attribute 2D array
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(
...         tiledb.Dim(name="y", domain=(0, 9), tile=2, dtype=np.uint64),
...         tiledb.Dim(name="x", domain=(0, 9), tile=2, dtype=np.uint64))
...     schema = tiledb.ArraySchema(domain=dom, sparse=True,
...         attrs=(tiledb.Attr(name="a1", dtype=np.int64),
...                tiledb.Attr(name="a2", dtype=np.int64)))
...     tiledb.SparseArray.create(tmp + "/array", schema)
...     with tiledb.SparseArray(tmp + "/array", mode='w') as A:
...         # Write in the twp cells (0,0) and (2,3) only.
...         I, J = [0, 2], [0, 3]
...         # Write to each attribute
...         A[I, J] = {"a1": np.array([1, 2]),
...                    "a2": np.array([3, 4])}
...     with tiledb.SparseArray(tmp + "/array", mode='r') as A:
...         # Return an OrderedDict with values and coordinates
...         A[0:3, 0:10]
...         # Return just the "x" coordinates values
...         A[0:3, 0:10]["x"]
OrderedDict([('a1', array([1, 2])), ('a2', array([3, 4])), ('y', array([0, 2], dtype=uint64)), ('x', array([0, 3], dtype=uint64))])
array([0, 3], dtype=uint64)

With a floating-point array domain, index bounds are inclusive, e.g.:

>>> # Return nonempty cells within a floating point array domain (fp index bounds are inclusive):
>>> # A[5.0:579.9]

__setitem__(selection, value)¶

Set / update sparse data cells

Parameters:	selection (tuple) – N coordinate value arrays (dim0, dim1, …) where N in the ndim of the SparseArray, The format follows numpy sparse (point) indexing semantics. value (dict or `numpy.ndarray`) – a dictionary of nonempty array attribute values, values must able to be converted to 1-d numpy arrays. if the number of attributes is one, than a 1-d numpy array is accepted.
Raises:	IndexError – invalid or unsupported index selection ValueError – value / coordinate length mismatch
Raises:	`tiledb.TileDBError`

Example:

>>> import tiledb, numpy as np, tempfile
>>> # Write to multi-attribute 2D array
>>> with tempfile.TemporaryDirectory() as tmp:
...     dom = tiledb.Domain(
...         tiledb.Dim(domain=(0, 1), tile=2, dtype=np.uint64),
...         tiledb.Dim(domain=(0, 1), tile=2, dtype=np.uint64))
...     schema = tiledb.ArraySchema(domain=dom, sparse=True,
...         attrs=(tiledb.Attr(name="a1", dtype=np.int64),
...                tiledb.Attr(name="a2", dtype=np.int64)))
...     tiledb.SparseArray.create(tmp + "/array", schema)
...     with tiledb.SparseArray(tmp + "/array", mode='w') as A:
...         # Write in the corner cells (0,0) and (1,1) only.
...         I, J = [0, 1], [0, 1]
...         # Write to each attribute
...         A[I, J] = {"a1": np.array([1, 2]),
...                    "a2": np.array([3, 4])}

Object Management¶

tiledb.group_create(uri, Ctx ctx=None)¶

Create a TileDB Group object at the specified path (URI)

Parameters:	uri (str) – URI of the TileDB Group to be created ctx (tiledb.Ctx) – The TileDB Context
Return type:	str
Returns:	The URI of the created TileDB Group
Raises:	TypeError – cannot convert path to unicode string
Raises:	`tiledb.TileDBError`

tiledb.object_type(uri, Ctx ctx=None)¶

Returns the TileDB object type at the specified path (URI)

Parameters:	path (str) – path (URI) of the TileDB resource ctx (tiledb.Ctx) – The TileDB Context
Return type:	str
Returns:	object type string
Raises:	TypeError – cannot convert path to unicode string

tiledb.remove(uri, Ctx ctx=None)¶

Removes (deletes) the TileDB object at the specified path (URI)

Parameters:	uri (str) – URI of the TileDB resource ctx (tiledb.Ctx) – The TileDB Context
Raises:	TypeError – uri cannot be converted to a unicode string
Raises:	`tiledb.TileDBError`

tiledb.move(old_uri, new_uri, Ctx ctx=None)¶

Moves a TileDB resource (group, array, key-value).

Parameters:	ctx (tiledb.Ctx) – The TileDB Context old_uri (str) – path (URI) of the TileDB resource to move new_uri (str) – path (URI) of the destination
Raises:	TypeError – uri cannot be converted to a unicode string
Raises:	`TileDBError`

tiledb.ls(path, func, Ctx ctx=None)¶

Lists TileDB resources and applies a callback that have a prefix of path (one level deep).

Parameters:	path (str) – URI of TileDB group object func (function) – callback to execute on every listed TileDB resource, URI resource path and object type label are passed as arguments to the callback ctx (tiledb.Ctx) – TileDB context
Raises:	TypeError – cannot convert path to unicode string
Raises:	`tiledb.TileDBError`

tiledb.walk(path, func, order='preorder', Ctx ctx=None)¶

Recursively visits TileDB resources and applies a callback to resources that have a prefix of path

Parameters:	path (str) – URI of TileDB group object func (function) – callback to execute on every listed TileDB resource, URI resource path and object type label are passed as arguments to the callback ctx (tiledb.Ctx) – The TileDB context order (str) – ‘preorder’ (default) or ‘postorder’ tree traversal
Raises:	TypeError – cannot convert path to unicode string ValueError – unknown order
Raises:	`tiledb.TileDBError`

VFS¶

class tiledb.VFS(Config config=None, Ctx ctx=None)¶

TileDB VFS class

Encapsulates the TileDB VFS module instance with a specific configuration (config).

Parameters:	ctx (tiledb.Ctx) – The TileDB Context config (tiledb.Config or dict) – Override ctx VFS configurations with updated values in config.

close(self, FileHandle fh)¶

Closes a VFS FileHandle object

Parameters:	fh (FileHandle) – An opened VFS FileHandle
Return type:	FileHandle
Returns:	closed VFS FileHandle
Raises:	`tiledb.TileDBError`

config(self)¶: Returns the Config instance associated with the VFS

create_bucket(self, uri)¶

Create an object store bucket at the given URI

Parameters:	uri (str) – full URI of bucket resource to be created.
Return type:	str
Returns:	created bucket URI
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

create_dir(self, uri)¶

Create a VFS directory at the given URI

Parameters:	uri (str) – URI of directory to be created
Return type:	str
Returns:	URI of created VFS directory
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

dir_size(self, uri)¶

Returns the size (in bytes) of a VFS directory at the given URI

Parameters:	uri (str) – URI of a VFS directory resource
Return type:	int
Returns:	dir size in number of bytes
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

empty_bucket(self, uri)¶

Empty an object store bucket of all objects at the given URI

This function blocks until all objects are verified to be removed from the given bucket.

Parameters:	uri (str) – URI of bucket resource to be emptied
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

file_size(self, uri)¶

Returns the size (in bytes) of a VFS file at the given URI

Parameters:	uri (str) – URI of a VFS file resource
Return type:	int
Returns:	file size in number of bytes
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

is_bucket(self, uri)¶

Returns True if the URI resource is a valid object store bucket

Parameters:	uri (str) – URI of bucket resource
Return type:	bool
Returns:	True if given URI is a valid object store bucket, False otherwise
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

is_dir(self, uri)¶

Returns True if the given URI is a VFS directory object

Parameters:	uri (str) – URI of the directory resource
Return type:	bool
Returns:	True if uri is a VFS directory, False otherwise
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

is_empty_bucket(self, uri)¶

Returns true if the object store bucket is empty (contains no objects).

If the bucket is versioned, this returns the status of the latest bucket version state.

Parameters:	uri (str) – URI of bucket resource
Return type:	bool
Returns:	True if bucket at given URI is empty, False otherwise
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

is_file(self, uri)¶

Returns True if the given URI is a VFS file object

Parameters:	uri (str) – URI of the file resource
Return type:	bool
Returns:	True if uri is a VFS file, False otherwise
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

ls(self, uri)¶

Lists contents of directory at the given URI. Raises TileDBError for non-existent directory.

Parameters:	uri (str) – URI of a VFS directory resource
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

move_dir(self, old_uri, new_uri)¶

Moves a VFS dir from old URI to new URI

Parameters:	old_uri (str) – Existing VFS file or directory resource URI new_uri (str) – URI to move existing VFS resource to force (bool) – if VFS resource at new_uri exists, delete the resource and overwrite
Return type:	str
Returns:	new URI of VFS resource
Raises:	TypeError – cannot convert old_uri/new_uri to unicode string
Raises:	`tiledb.TileDBError`

move_file(self, old_uri, new_uri)¶

Moves a VFS file from old URI to new URI

Parameters:	old_uri (str) – Existing VFS file or directory resource URI new_uri (str) – URI to move existing VFS resource to force (bool) – if VFS resource at new_uri exists, delete the resource and overwrite
Return type:	str
Returns:	new URI of VFS resource
Raises:	TypeError – cannot convert old_uri/new_uri to unicode string
Raises:	`tiledb.TileDBError`

open(self, uri, mode='rb')¶

Opens a VFS file resource for reading / writing / appends at URI

If the file did not exist upon opening, a new file is created.

Parameters:	uri (str) – URI of VFS file resource str (mode) – ‘rb’ for opening the file to read, ‘wb’ to write, ‘ab’ to append
Return type:	FileHandle
Returns:	VFS FileHandle
Raises:	TypeError – cannot convert uri to unicode string ValueError – invalid mode
Raises:	`tiledb.TileDBError`

read(self, FileHandle fh, offset, nbytes)¶

Read nbytes from an opened VFS FileHandle at a given offset

Parameters:	fh (FileHandle) – An opened VFS FileHandle in ‘r’ mode offset (int) – offset position in bytes to read from nbytes (int) – number of bytes to read
Return type:	`bytes()`
Returns:	read bytes
Raises:	`tiledb.TileDBError`

readinto(self, FileHandle fh, const unsigned char[:] buffer, offset, nbytes)¶

Read nbytes from an opened VFS FileHandle at a given offset into a preallocated bytes buffer

Parameters:	fh (FileHandle) – An opened VFS FileHandle in ‘r’ mode buffer (bytes) – A preallocated bytes buffer object offset (int) – offset position in bytes to read from nbytes (int) – number of bytes to read
Returns:	bytes buffer
Raises:	ValueError – invalid offset or nbytes values
Raises:	`tiledb.TileDBError`

remove_bucket(self, uri)¶

Remove an object store bucket at the given URI

Parameters:	uri (str) – URI of bucket resource to be removed.
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

..note:: Consistency is not enforced for bucket removal so although this function will return immediately on success, the actual removal of the bucket make take some (indeterminate) amount of time.

remove_dir(self, uri)¶

Removes a VFS directory at the given URI

Parameters:	uri (str) – URI of the directory resource to remove
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

remove_file(self, uri)¶

Removes a VFS file at the given URI

Parameters:	uri (str) – URI of a VFS file resource
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

supports(self, scheme)¶

Returns true if the given URI scheme (storage backend) is supported

Parameters:	scheme (str) – scheme component of a VFS resource URI (ex. ‘file’ / ‘hdfs’ / ‘s3’)
Return type:	bool
Returns:	True if the linked libtiledb version supports the storage backend, False otherwise
Raises:	ValueError – VFS storage backend is not supported

sync(self, FileHandle fh)¶

Sync / flush an opened VFS FileHandle to storage backend

Parameters:	fh (FileHandle) – An opened VFS FileHandle in ‘w’ or ‘a’ mode
Raises:	`tiledb.TileDBError`

touch(self, uri)¶

Creates an empty VFS file at the given URI

Parameters:	uri (str) – URI of a VFS file resource
Return type:	str
Returns:	URI of touched VFS file
Raises:	TypeError – cannot convert uri to unicode string
Raises:	`tiledb.TileDBError`

write(self, FileHandle fh, buff)¶

Writes buffer to opened VFS FileHandle

Parameters:	fh (FileHandle) – An opened VFS FileHandle in ‘w’ mode buff – a Python object that supports the byte buffer protocol
Raises:	TypeError – cannot convert buff to bytes
Raises:	`tiledb.TileDBError`

Version¶

tiledb.libtiledb.version()¶

Return the version of the linked libtiledb shared library

Return type:	tuple
Returns:	Semver version (major, minor, rev)

Statistics¶

tiledb.stats_enable()¶: Enable TileDB internal statistics.

tiledb.stats_disable()¶: Disable TileDB internal statistics.

tiledb.stats_reset()¶: Reset all TileDB internal statistics to 0.

tiledb.stats_dump()¶: Prints all TileDB internal statistics values to standard output.