TileDB Python API Reference¶
Warning
The Python interface to TileDB is still under development and the API is subject to change.
Modules¶
Typical usage of the Python interface to TileDB will use the top-level module tiledb
, e.g.
import tiledb
There is also a submodule libtiledb
which contains the necessary bindings to the underlying TileDB native library. Most of the time you will not need to interact with tiledb.libtiledb
unless you need native-library specific information, e.g. the version number:
import tiledb
tiledb.libtiledb.version() # Native TileDB library version number
Exceptions¶
-
exception
tiledb.
TileDBError
¶ TileDB Error Exception
Captures and raises error return code (
TILEDB_ERR
) messages when callinglibtiledb
functions. The error message that is raised is the last error set for thetiledb.Ctx
A Python
MemoryError
is raised onTILEDB_OOM
Context¶
-
class
tiledb.
Ctx
(config=None)¶ Class representing a TileDB context.
A TileDB context wraps a TileDB storage manager.
Parameters: config (tiledb.Config or dict) – Initialize Ctx with given config parameters -
config
(self)¶ Returns the Config instance associated with the Ctx
-
set_tag
(self, key, value)¶ Sets a string:string “tag” on the Ctx
-
Config¶
-
class
tiledb.
Config
(params=None, path=None)¶ TileDB Config class
See: https://docs.tiledb.io/en/stable/tutorials/config.html#summary-of-parameters
Unknown parameters will be ignored!
Parameters: -
clear
(self)¶ Unsets all Config parameters (returns them to their default values)
-
dict
(self, prefix=u'')¶ Returns a dict representation of a Config object
Parameters: prefix (str) – return only parameters with a given prefix Return type: dict Returns: Config parameter / values as a a Python dict
-
from_file
(self, path)¶ Update a Config object with from a persisted config file
Parameters: path – A local Config file path
-
get
(self, key, *args)¶ Gets the value of a config parameter, or a default value.
Parameters: - key (str) – Config parameter
- args – return arg if Config does not contain parameter key
Returns: Parameter value, arg or None.
-
items
(self, prefix=u'')¶ Returns an iterator object over Config parameters, values
Parameters: prefix (str) – return only parameters with a given prefix Return type: ConfigItems Returns: iterator over Config parameter, value tuples
-
keys
(self, prefix=u'')¶ Returns an iterator object over Config parameters (keys)
Parameters: prefix (str) – return only parameters with a given prefix Return type: ConfigKeys Returns: iterator over Config parameter string keys
-
static
load
(uri)¶ Constructs a Config class instance from config parameters loaded from a local Config file
Parameters: uri (str) – a local URI config file path Return type: tiledb.Config Returns: A TileDB Config instance with persisted parameter values Raises: TypeError – uri cannot be converted to a unicode string Raises: tiledb.TileDBError
-
save
(self, uri)¶ Persist Config parameter values to a config file
Parameters: uri (str) – a local URI config file path Raises: TypeError – uri cannot be converted to a unicode string Raises: tiledb.TileDBError
-
update
(self, odict)¶ Update a config object with parameter, values from a dict like object
Parameters: odict – dict-like object containing parameter, values to update Config.
-
Array Schema¶
-
class
tiledb.
ArraySchema
(domain=None, attrs=(), cell_order='row-major', tile_order='row-major', capacity=0, coords_filters=None, offsets_filters=None, sparse=False, Ctx ctx=None)¶ Schema class for TileDB dense / sparse array representations
Parameters: - attrs (tuple(tiledb.Attr, ..)) – one or more array attributes
- cell_order ('row-major' or 'C', 'col-major' or 'F') – TileDB label for cell layout
- tile_order ('row-major' or 'C', 'col-major' or 'F', 'unordered') – TileDB label for tile layout
- capacity (int) – tile cell capacity
- coords_filters (tiledb.FilterList) – (default None) coordinate filter list
- offsets_filters (tiledb.FilterList) – (default None) offsets filter list
- sparse (bool) – True if schema is sparse, else False (set by SparseArray and DenseArray derived classes)
- ctx (tiledb.Ctx) – A TileDB Context
Raises: TypeError – cannot convert uri to unicode string
Raises: -
attr
(self, key)¶ Returns an Attr instance given an int index or string label
Parameters: key (int or str) – attribute index (positional or associative) Return type: tiledb.Attr Returns: The ArraySchema attribute at index or with the given name (label) Raises: TypeError – invalid key type
-
attr_or_dim_dtype
(self, unicode name)¶
-
capacity
¶ The array capacity
Return type: int Raises: tiledb.TileDBError
-
cell_order
¶ The cell order layout of the array.
-
coords_compressor
¶ The compressor label and level for the array’s coordinates.
Return type: tuple(str, int) Raises: tiledb.TileDBError
-
coords_filters
¶ The FilterList for the array’s coordinates
Return type: tiledb.FilterList Raises: tiledb.TileDBError
-
domain
¶ The Domain associated with the array.
Return type: tiledb.Domain Raises: tiledb.TileDBError
-
dump
(self)¶ Dumps a string representation of the array object to standard output (stdout)
-
has_attr
(self, name)¶ Returns true if the given name is an Attribute of the ArraySchema
Parameters: name – attribute name Return type: boolean
-
static
load
(uri, Ctx ctx=None, key=None)¶
-
nattr
¶ The number of array attributes.
Return type: int Raises: tiledb.TileDBError
-
offsets_compressor
¶ The compressor label and level for the array’s variable-length attribute offsets.
Return type: tuple(str, int) Raises: tiledb.TileDBError
-
offsets_filters
¶ The FilterList for the array’s variable-length attribute offsets
Return type: tiledb.FilterList Raises: tiledb.TileDBError
-
shape
¶ The array’s shape
Return type: tuple(numpy scalar, numpy scalar) Raises: TypeError – floating point (inexact) domain
-
sparse
¶ True if the array is a sparse array representation
Return type: bool Raises: tiledb.TileDBError
-
tile_order
¶ The tile order layout of the array.
Return type: str Raises: tiledb.TileDBError
Attribute¶
-
class
tiledb.
Attr
(name=u'', dtype=np.float64, var=False, filters=None, Ctx ctx=None)¶ Class representing a TileDB array attribute.
Parameters: - ctx (tiledb.Ctx) – A TileDB Context
- name (str) – Attribute name, empty if anonymous
- dtype (bool) – Attribute value datatypes
- var – Attribute is variable-length (automatic for byte/string types)
- filters (FilterList) – List of filters to apply
Raises: TypeError – invalid dtype
Raises: -
compressor
¶ String label of the attributes compressor and compressor level
Return type: tuple(str, int) Raises: tiledb.TileDBError
-
dtype
¶ Return numpy dtype object representing the Attr type
Return type: numpy.dtype
-
dump
(self)¶ Dumps a string representation of the Attr object to standard output (stdout)
-
filters
¶ FilterList of the TileDB attribute
Return type: tiledb.FilterList Raises: tiledb.TileDBError
-
isvar
¶ True if the attribute is variable length
Return type: bool Raises: tiledb.TileDBError
-
name
¶ Attribute string name, empty string if the attribute is anonymous
Return type: str Raises: tiledb.TileDBError
-
ncells
¶ The number of cells (scalar values) for a given attribute value
Return type: int Raises: tiledb.TileDBError
Filters¶
-
class
tiledb.
FilterList
(filters=None, chunksize=None, Ctx ctx=None)¶ An ordered list of Filter objects for filtering TileDB data.
FilterLists contain zero or more Filters, used for filtering attribute data, the array coordinate data, etc.
Parameters: - ctx (tiledb.Ctx) – A TileDB context
- filters – An iterable of Filter objects to add.
- chunksize (int) – (default None) chunk size used by the filter list in bytes
Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... # Create several filters ... gzip_filter = tiledb.GzipFilter() ... bw_filter = tiledb.BitWidthReductionFilter() ... # Create a filter list that will first perform bit width reduction, then gzip compression. ... filters = tiledb.FilterList([bw_filter, gzip_filter]) ... a1 = tiledb.Attr(name="a1", dtype=np.int64, filters=filters) ... # Create a second attribute filtered only by gzip compression. ... a2 = tiledb.Attr(name="a2", dtype=np.int64, ... filters=tiledb.FilterList([gzip_filter])) ... schema = tiledb.ArraySchema(domain=dom, attrs=(a1, a2)) ... tiledb.DenseArray.create(tmp + "/array", schema)
-
append
(self, Filter filter)¶ Appends filter to the end of filter list
Parameters: filter (Filter) – filter object to add Returns: None
-
chunksize
¶ The chunk size used by the filter list.
-
class
tiledb.libtiledb.
CompressionFilter
(tiledb_filter_type_t filter_type, level, Ctx ctx=None)¶ Base class for filters performing compression.
All compression filters support a compression level option, although some (such as RLE) ignore it.
Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... a1 = tiledb.Attr(name="a1", dtype=np.int64, ... filters=tiledb.FilterList([tiledb.GzipFilter(level=10)])) ... schema = tiledb.ArraySchema(domain=dom, attrs=(a1,)) ... tiledb.DenseArray.create(tmp + "/array", schema)
-
class
tiledb.
GzipFilter
(level=None, Ctx ctx=None)¶ Filter that compresses using gzip.
Parameters: - ctx (tiledb.Ctx) – TileDB Ctx
- level (int) – (default None) If not None set the compressor level
Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... a1 = tiledb.Attr(name="a1", dtype=np.int64, ... filters=tiledb.FilterList([tiledb.GzipFilter()])) ... schema = tiledb.ArraySchema(domain=dom, attrs=(a1,)) ... tiledb.DenseArray.create(tmp + "/array", schema)
-
class
tiledb.
ZstdFilter
(level=None, Ctx ctx=None)¶ Filter that compresses using zstd.
Parameters: - ctx (tiledb.Ctx) – TileDB Ctx
- level (int) – (default None) If not None set the compressor level
Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... a1 = tiledb.Attr(name="a1", dtype=np.int64, ... filters=tiledb.FilterList([tiledb.ZstdFilter()])) ... schema = tiledb.ArraySchema(domain=dom, attrs=(a1,)) ... tiledb.DenseArray.create(tmp + "/array", schema)
-
class
tiledb.
LZ4Filter
(level=None, Ctx ctx=None)¶ Filter that compresses using lz4.
Parameters: - ctx (tiledb.Ctx) – TileDB Ctx
- level (int) – (default None) If not None set the compressor level
Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... a1 = tiledb.Attr(name="a1", dtype=np.int64, ... filters=tiledb.FilterList([tiledb.LZ4Filter()])) ... schema = tiledb.ArraySchema(domain=dom, attrs=(a1,)) ... tiledb.DenseArray.create(tmp + "/array", schema)
-
class
tiledb.
Bzip2Filter
(level=None, Ctx ctx=None)¶ Filter that compresses using bzip2.
Parameters: level (int) – (default None) If not None set the compressor level Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... a1 = tiledb.Attr(name="a1", dtype=np.int64, ... filters=tiledb.FilterList([tiledb.Bzip2Filter()])) ... schema = tiledb.ArraySchema(domain=dom, attrs=(a1,)) ... tiledb.DenseArray.create(tmp + "/array", schema)
-
class
tiledb.
RleFilter
(Ctx ctx=None)¶ Filter that compresses using run-length encoding (RLE).
Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... a1 = tiledb.Attr(name="a1", dtype=np.int64, ... filters=tiledb.FilterList([tiledb.RleFilter()])) ... schema = tiledb.ArraySchema(domain=dom, attrs=(a1,)) ... tiledb.DenseArray.create(tmp + "/array", schema)
-
class
tiledb.
DoubleDeltaFilter
(Ctx ctx=None)¶ Filter that performs double-delta encoding.
Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... a1 = tiledb.Attr(name="a1", dtype=np.int64, ... filters=tiledb.FilterList([tiledb.DoubleDeltaFilter()])) ... schema = tiledb.ArraySchema(domain=dom, attrs=(a1,)) ... tiledb.DenseArray.create(tmp + "/array", schema)
-
class
tiledb.
BitShuffleFilter
(Ctx ctx=None)¶ Filter that performs a bit shuffle transformation.
Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... a1 = tiledb.Attr(name="a1", dtype=np.int64, ... filters=tiledb.FilterList([tiledb.BitShuffleFilter()])) ... schema = tiledb.ArraySchema(domain=dom, attrs=(a1,)) ... tiledb.DenseArray.create(tmp + "/array", schema)
-
class
tiledb.
ByteShuffleFilter
(Ctx ctx=None)¶ Filter that performs a byte shuffle transformation.
Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... a1 = tiledb.Attr(name="a1", dtype=np.int64, ... filters=tiledb.FilterList([tiledb.ByteShuffleFilter()])) ... schema = tiledb.ArraySchema(domain=dom, attrs=(a1,)) ... tiledb.DenseArray.create(tmp + "/array", schema)
-
class
tiledb.
BitWidthReductionFilter
(window=None, Ctx ctx=None)¶ Filter that performs bit-width reduction.
param ctx: A TileDB Context type ctx: tiledb.Ctx param window: (default None) max window size for the filter type window: int Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... a1 = tiledb.Attr(name="a1", dtype=np.int64, ... filters=tiledb.FilterList([tiledb.BitWidthReductionFilter()])) ... schema = tiledb.ArraySchema(domain=dom, attrs=(a1,)) ... tiledb.DenseArray.create(tmp + "/array", schema)
-
class
tiledb.
PositiveDeltaFilter
(window=None, Ctx ctx=None)¶ Filter that performs positive-delta encoding.
Parameters: - ctx (tiledb.Ctx) – A TileDB Context
- window (int) – (default None) the max window for the filter
Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... a1 = tiledb.Attr(name="a1", dtype=np.int64, ... filters=tiledb.FilterList([tiledb.PositiveDeltaFilter()])) ... schema = tiledb.ArraySchema(domain=dom, attrs=(a1,)) ... tiledb.DenseArray.create(tmp + "/array", schema)
Dimension¶
-
class
tiledb.
Dim
(name=u'__dim_0', domain=None, tile=None, dtype=np.uint64, Ctx ctx=None)¶ Class representing a dimension of a TileDB Array.
Parameters: Dtype: the Dim numpy dtype object, type object, or string that can be corerced into a numpy dtype object
Raises: - ValueError – invalid domain or tile extent
- TypeError – invalid domain, tile extent, or dtype type
Raises: -
domain
¶ The dimension (inclusive) domain.
The dimension’s domain is defined by a (lower bound, upper bound) tuple.
Return type: tuple(numpy scalar, numpy scalar)
-
dtype
¶ Numpy dtype representation of the dimension type.
Return type: numpy.dtype
-
isvar
¶ True if the dimension is variable length
Return type: bool Raises: tiledb.TileDBError
-
name
¶ The dimension label string.
Anonymous dimensions return a default string representation based on the dimension index.
Return type: str
-
shape
¶ The shape of the dimension given the dimension’s domain.
Note: The shape is only valid for integer and datetime dimension domains.
Return type: tuple(numpy scalar, numpy scalar) Raises: TypeError – floating point (inexact) domain
-
size
¶ The size of the dimension domain (number of cells along dimension).
Return type: int Raises: TypeError – floating point (inexact) domain
-
tile
¶ The tile extent of the dimension.
Return type: numpy scalar or np.timedelta64
Domain¶
-
class
tiledb.
Domain
(Ctx ctx=None, *dims)¶ Class representing the domain of a TileDB Array.
Parameters: - dims – one or more tiledb.Dim objects up to the Domain’s ndim
- ctx (tiledb.Ctx) – A TileDB Context
Raises: TypeError – All dimensions must have the same dtype
Raises: -
dim
(self, dim_id)¶ Returns a Dim object from the domain given the dimension’s index or name.
Parameters: dim_d – dimension index (int) or name (str) Raises: tiledb.TileDBError
-
dtype
¶ The numpy dtype of the domain’s dimension type.
Return type: numpy.dtype
-
dump
(self)¶ Dumps a string representation of the domain object to standard output (STDOUT)
-
has_dim
(self, name)¶ Returns true if the Domain has a Dimension with the given name
Parameters: name – name of Dimension Return type: bool Returns:
Array¶
-
class
tiledb.libtiledb.
Array
(uri, mode='r', key=None, timestamp=None, attr=None, Ctx ctx=None)¶ Base class for TileDB array objects.
Defines common properties/functionality for the different array types. When an Array instance is initialized, the array is opened with the specified mode.
Parameters: - uri (str) – URI of array to open
- mode (str) – (default ‘r’) Open the array object in read ‘r’ or write ‘w’ mode
- key (str) – (default None) If not None, encryption key to decrypt the array
- timestamp (int) – (default None) If not None, open the array at a given TileDB timestamp
- ctx (Ctx) – TileDB context
-
attr
(self, key)¶ Returns an
Attr
instance given an int index or string labelParameters: key (int or str) – attribute index (positional or associative) Return type: Attr
Returns: The array attribute at index or with the given name (label) Raises: TypeError – invalid key type
-
close
(self)¶ Closes this array, flushing all buffered data.
-
consolidate
(self, Config config=None, key=None)¶ Consolidates fragments of an array object for increased read performance.
Parameters: - config (tiledb.Config) – The TileDB Config with consolidation parameters set
- key (str or bytes) – (default None) encryption key to decrypt an encrypted array
Raises:
-
coords_dtype
¶ Returns the numpy record array dtype of the array coordinates
Return type: numpy.dtype Returns: coord array record dtype
-
create
(type cls, uri, ArraySchema schema, key=None, Ctx ctx=None)¶ Creates a persistent TileDB Array at the given URI
Parameters: - uri (str) – URI at which to create the new empty array.
- schema (ArraySchema) – Schema for the array
- key (str) – (default None) Encryption key to use for array
- Ctx (ctx) – (default None) Optional TileDB Ctx used when creating the array, by default uses the ArraySchema’s associated context.
-
dim
(self, dim_id)¶ Returns a
Dim
instance given a dim index or nameParameters: key (int or str) – attribute index (positional or associative) Return type: Attr
Returns: The array attribute at index or with the given name (label) Raises: TypeError – invalid key type
-
domain
¶ The
Domain
of this array.
-
dtype
¶ The NumPy dtype of the specified attribute
-
dump
(self)¶
-
isopen
¶ True if this array is currently open.
-
iswritable
¶ This array is currently opened as writable.
-
static
load_typed
(uri, mode='r', key=None, timestamp=None, attr=None, Ctx ctx=None)¶
-
mode
¶ The mode this array was opened with.
-
nattr
¶ The number of attributes of this array.
-
ndim
¶ The number of dimensions of this array.
-
nonempty_domain
(self)¶ Return the minimum bounding domain which encompasses nonempty values.
Return type: tuple(tuple(numpy scalar, numpy scalar), ..) Returns: A list of (inclusive) domain extent tuples, that contain all nonempty cells
-
reopen
(self, timestamp=None)¶ Reopens this array.
This is useful when the array is updated after it was opened. To sync-up with the updates, the user must either close the array and open again, or just use
reopen()
without closing. Reopening will be generally faster than the former alternative.
-
schema
¶ The
ArraySchema
for this array.
-
shape
¶ The shape of this array.
-
subarray
(self, selection, attrs=None, coords=False, order=None)¶
-
timestamp
¶ Returns the timestamp the array is opened at
Return type: int Returns: tiledb timestamp at which point the array was opened
-
uri
¶ Returns the URI of the array
-
tiledb.
consolidate
(uri=None, Config config=None, key=None, Ctx ctx=None)¶ Consolidates a TileDB Array updates for improved read performance
Parameters: - config (tiledb.Config) – The TileDB Config with consolidation parameters set
- uri (str) – URI to the TileDB Array
- str – (default None) Key to decrypt array if the array is encrypted
- ctx (tiledb.Ctx) – The TileDB Context
Return type: Returns: path (URI) to the consolidated TileDB Array
Raises: TypeError – cannot convert path to unicode string
Raises:
Dense Array¶
-
class
tiledb.
DenseArray
¶ -
__getitem__
(selection)¶ Retrieve data cells for an item or region of the array.
Parameters: selection (tuple) – An int index, slice or tuple of integer/slice objects, specifiying the selected subarray region for each dimension of the DenseArray. Return type: numpy.ndarray
orcollections.OrderedDict
Returns: If the dense array has a single attribute than a Numpy array of corresponding shape/dtype is returned for that attribute. If the array has multiple attributes, a collections.OrderedDict
is returned with dense Numpy subarrays for each attribute.Raises: IndexError – invalid or unsupported index selection Raises: tiledb.TileDBError
Example:
>>> import tiledb, numpy as np, tempfile >>> with tempfile.TemporaryDirectory() as tmp: ... # Creates array 'array' on disk. ... A = tiledb.DenseArray.from_numpy(tmp + "/array", np.ones((100, 100))) ... # Many aspects of Numpy's fancy indexing are supported: ... A[1:10, ...].shape ... A[1:10, 20:99].shape ... A[1, 2].shape (9, 100) (9, 79) () >>> # Subselect on attributes when reading: >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain(tiledb.Dim(domain=(0, 9), tile=2, dtype=np.uint64)) ... schema = tiledb.ArraySchema(domain=dom, ... attrs=(tiledb.Attr(name="a1", dtype=np.int64), ... tiledb.Attr(name="a2", dtype=np.int64))) ... tiledb.DenseArray.create(tmp + "/array", schema) ... with tiledb.DenseArray(tmp + "/array", mode='w') as A: ... A[0:10] = {"a1": np.zeros((10)), "a2": np.ones((10))} ... with tiledb.DenseArray(tmp + "/array", mode='r') as A: ... # Access specific attributes individually. ... A[0:5]["a1"] ... A[0:5]["a2"] array([0, 0, 0, 0, 0]) array([1, 1, 1, 1, 1])
-
__setitem__
(selection, value)¶ Set / update dense data cells
Parameters: - selection (tuple) – An int index, slice or tuple of integer/slice objects, specifiying the selected subarray region for each dimension of the DenseArray.
- value (dict or
numpy.ndarray
) – a dictionary of array attribute values, values must able to be converted to n-d numpy arrays. if the number of attributes is one, than a n-d numpy array is accepted.
Raises: - IndexError – invalid or unsupported index selection
- ValueError – value / coordinate length mismatch
Raises: Example:
>>> import tiledb, numpy as np, tempfile >>> # Write to single-attribute 2D array >>> with tempfile.TemporaryDirectory() as tmp: ... # Create an array initially with all zero values ... with tiledb.DenseArray.from_numpy(tmp + "/array", np.zeros((2, 2))) as A: ... pass ... with tiledb.DenseArray(tmp + "/array", mode='w') as A: ... # Write to the single (anonymous) attribute ... A[:] = np.array(([1,2], [3,4])) >>> >>> # Write to multi-attribute 2D array >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain( ... tiledb.Dim(domain=(0, 1), tile=2, dtype=np.uint64), ... tiledb.Dim(domain=(0, 1), tile=2, dtype=np.uint64)) ... schema = tiledb.ArraySchema(domain=dom, ... attrs=(tiledb.Attr(name="a1", dtype=np.int64), ... tiledb.Attr(name="a2", dtype=np.int64))) ... tiledb.DenseArray.create(tmp + "/array", schema) ... with tiledb.DenseArray(tmp + "/array", mode='w') as A: ... # Write to each attribute ... A[0:2, 0:2] = {"a1": np.array(([-3, -4], [-5, -6])), ... "a2": np.array(([1, 2], [3, 4]))}
-
Sparse Array¶
-
class
tiledb.
SparseArray
¶ -
__getitem__
(selection)¶ Retrieve nonempty cell data for an item or region of the array
Parameters: selection (tuple) – An int index, slice or tuple of integer/slice objects, specifying the selected subarray region for each dimension of the SparseArray. Return type: collections.OrderedDict
Returns: An OrderedDict is returned with “coords” coordinate values being the first key. “coords” is a Numpy record array representation of the coordinate values of non-empty attribute cells. Nonempty attribute values are returned as Numpy 1-d arrays. Raises: IndexError – invalid or unsupported index selection Raises: tiledb.TileDBError
Example:
>>> import tiledb, numpy as np, tempfile >>> # Write to multi-attribute 2D array >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain( ... tiledb.Dim(name="y", domain=(0, 9), tile=2, dtype=np.uint64), ... tiledb.Dim(name="x", domain=(0, 9), tile=2, dtype=np.uint64)) ... schema = tiledb.ArraySchema(domain=dom, sparse=True, ... attrs=(tiledb.Attr(name="a1", dtype=np.int64), ... tiledb.Attr(name="a2", dtype=np.int64))) ... tiledb.SparseArray.create(tmp + "/array", schema) ... with tiledb.SparseArray(tmp + "/array", mode='w') as A: ... # Write in the twp cells (0,0) and (2,3) only. ... I, J = [0, 2], [0, 3] ... # Write to each attribute ... A[I, J] = {"a1": np.array([1, 2]), ... "a2": np.array([3, 4])} ... with tiledb.SparseArray(tmp + "/array", mode='r') as A: ... # Return an OrderedDict with cell coordinates ... A[0:3, 0:10] ... # Return just the "x" coordinates values ... A[0:3, 0:10]["x"] OrderedDict([('a1', array([1, 2])), ('a2', array([3, 4])), ('y', array([0, 2], dtype=uint64)), ('x', array([0, 3], dtype=uint64))]) array([0, 3], dtype=uint64)
With a floating-point array domain, index bounds are inclusive, e.g.:
>>> # Return nonempty cells within a floating point array domain (fp index bounds are inclusive): >>> # A[5.0:579.9]
-
__setitem__
(selection, value)¶ Set / update sparse data cells
Parameters: - selection (tuple) – N coordinate value arrays (dim0, dim1, …) where N in the ndim of the SparseArray, The format follows numpy sparse (point) indexing semantics.
- value (dict or
numpy.ndarray
) – a dictionary of nonempty array attribute values, values must able to be converted to 1-d numpy arrays. if the number of attributes is one, than a 1-d numpy array is accepted.
Raises: - IndexError – invalid or unsupported index selection
- ValueError – value / coordinate length mismatch
Raises: Example:
>>> import tiledb, numpy as np, tempfile >>> # Write to multi-attribute 2D array >>> with tempfile.TemporaryDirectory() as tmp: ... dom = tiledb.Domain( ... tiledb.Dim(domain=(0, 1), tile=2, dtype=np.uint64), ... tiledb.Dim(domain=(0, 1), tile=2, dtype=np.uint64)) ... schema = tiledb.ArraySchema(domain=dom, sparse=True, ... attrs=(tiledb.Attr(name="a1", dtype=np.int64), ... tiledb.Attr(name="a2", dtype=np.int64))) ... tiledb.SparseArray.create(tmp + "/array", schema) ... with tiledb.SparseArray(tmp + "/array", mode='w') as A: ... # Write in the corner cells (0,0) and (1,1) only. ... I, J = [0, 1], [0, 1] ... # Write to each attribute ... A[I, J] = {"a1": np.array([1, 2]), ... "a2": np.array([3, 4])}
-
Object Management¶
-
tiledb.
group_create
(uri, Ctx ctx=None)¶ Create a TileDB Group object at the specified path (URI)
Parameters: - uri (str) – URI of the TileDB Group to be created
- ctx (tiledb.Ctx) – The TileDB Context
Return type: Returns: The URI of the created TileDB Group
Raises: TypeError – cannot convert path to unicode string
Raises:
-
tiledb.
object_type
(uri, Ctx ctx=None)¶ Returns the TileDB object type at the specified path (URI)
Parameters: - path (str) – path (URI) of the TileDB resource
- ctx (tiledb.Ctx) – The TileDB Context
Return type: Returns: object type string
Raises: TypeError – cannot convert path to unicode string
-
tiledb.
remove
(uri, Ctx ctx=None)¶ Removes (deletes) the TileDB object at the specified path (URI)
Parameters: - uri (str) – URI of the TileDB resource
- ctx (tiledb.Ctx) – The TileDB Context
Raises: TypeError – uri cannot be converted to a unicode string
Raises:
-
tiledb.
move
(old_uri, new_uri, Ctx ctx=None)¶ Moves a TileDB resource (group, array, key-value).
Parameters: - ctx (tiledb.Ctx) – The TileDB Context
- old_uri (str) – path (URI) of the TileDB resource to move
- new_uri (str) – path (URI) of the destination
Raises: TypeError – uri cannot be converted to a unicode string
Raises:
-
tiledb.
ls
(path, func, Ctx ctx=None)¶ Lists TileDB resources and applies a callback that have a prefix of
path
(one level deep).Parameters: - path (str) – URI of TileDB group object
- func (function) – callback to execute on every listed TileDB resource, URI resource path and object type label are passed as arguments to the callback
- ctx (tiledb.Ctx) – TileDB context
Raises: TypeError – cannot convert path to unicode string
Raises:
-
tiledb.
walk
(path, func, order='preorder', Ctx ctx=None)¶ Recursively visits TileDB resources and applies a callback to resources that have a prefix of
path
Parameters: - path (str) – URI of TileDB group object
- func (function) – callback to execute on every listed TileDB resource, URI resource path and object type label are passed as arguments to the callback
- ctx (tiledb.Ctx) – The TileDB context
- order (str) – ‘preorder’ (default) or ‘postorder’ tree traversal
Raises: - TypeError – cannot convert path to unicode string
- ValueError – unknown order
Raises:
VFS¶
-
class
tiledb.
VFS
(Config config=None, Ctx ctx=None)¶ TileDB VFS class
Encapsulates the TileDB VFS module instance with a specific configuration (config).
Parameters: - ctx (tiledb.Ctx) – The TileDB Context
- config (tiledb.Config or dict) – Override ctx VFS configurations with updated values in config.
-
close
(self, FileHandle fh)¶ Closes a VFS FileHandle object
Parameters: fh (FileHandle) – An opened VFS FileHandle Return type: FileHandle Returns: closed VFS FileHandle Raises: tiledb.TileDBError
-
config
(self)¶ Returns the Config instance associated with the VFS
-
create_bucket
(self, uri)¶ Create an object store bucket at the given URI
Parameters: uri (str) – full URI of bucket resource to be created. Return type: str Returns: created bucket URI Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError
-
create_dir
(self, uri)¶ Create a VFS directory at the given URI
Parameters: uri (str) – URI of directory to be created Return type: str Returns: URI of created VFS directory Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError
-
empty_bucket
(self, uri)¶ Empty an object store bucket of all objects at the given URI
This function blocks until all objects are verified to be removed from the given bucket.
Parameters: uri (str) – URI of bucket resource to be emptied Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError
-
file_size
(self, uri)¶ Returns the size (in bytes) of a VFS file at the given URI
Parameters: uri (str) – URI of a VFS file resource Return type: int Returns: file size in number of bytes Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError
-
is_bucket
(self, uri)¶ Returns True if the URI resource is a valid object store bucket
Parameters: uri (str) – URI of bucket resource Return type: bool Returns: True if given URI is a valid object store bucket, False otherwise Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError
-
is_dir
(self, uri)¶ Returns True if the given URI is a VFS directory object
Parameters: uri (str) – URI of the directory resource Return type: bool Returns: True if uri is a VFS directory, False otherwise Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError
-
is_empty_bucket
(self, uri)¶ Returns true if the object store bucket is empty (contains no objects).
If the bucket is versioned, this returns the status of the latest bucket version state.
Parameters: uri (str) – URI of bucket resource Return type: bool Returns: True if bucket at given URI is empty, False otherwise Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError
-
is_file
(self, uri)¶ Returns True if the given URI is a VFS file object
Parameters: uri (str) – URI of the file resource Return type: bool Returns: True if uri is a VFS file, False otherwise Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError
-
ls
(self, uri)¶ Lists contents of directory at the given URI. Raises TileDBError for non-existent directory.
Parameters: uri (str) – URI of a VFS directory resource Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError
-
move_dir
(self, old_uri, new_uri)¶ Moves a VFS dir from old URI to new URI
Parameters: Return type: Returns: new URI of VFS resource
Raises: TypeError – cannot convert old_uri/new_uri to unicode string
Raises:
-
move_file
(self, old_uri, new_uri)¶ Moves a VFS file from old URI to new URI
Parameters: Return type: Returns: new URI of VFS resource
Raises: TypeError – cannot convert old_uri/new_uri to unicode string
Raises:
-
open
(self, uri, mode='rb')¶ Opens a VFS file resource for reading / writing / appends at URI
If the file did not exist upon opening, a new file is created.
Parameters: - uri (str) – URI of VFS file resource
- str (mode) – ‘rb’ for opening the file to read, ‘wb’ to write, ‘ab’ to append
Return type: FileHandle
Returns: VFS FileHandle
Raises: - TypeError – cannot convert uri to unicode string
- ValueError – invalid mode
Raises:
-
read
(self, FileHandle fh, offset, nbytes)¶ Read nbytes from an opened VFS FileHandle at a given offset
Parameters: Return type: bytes()
Returns: read bytes
Raises:
-
readinto
(self, FileHandle fh, const unsigned char[:] buffer, offset, nbytes)¶ Read nbytes from an opened VFS FileHandle at a given offset into a preallocated bytes buffer
Parameters: Returns: bytes buffer
Raises: ValueError – invalid offset or nbytes values
Raises:
-
remove_bucket
(self, uri)¶ Remove an object store bucket at the given URI
Parameters: uri (str) – URI of bucket resource to be removed. Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError
- ..note:
- Consistency is not enforced for bucket removal so although this function will return immediately on success, the actual removal of the bucket make take some (indeterminate) amount of time.
-
remove_dir
(self, uri)¶ Removes a VFS directory at the given URI
Parameters: uri (str) – URI of the directory resource to remove Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError
-
remove_file
(self, uri)¶ Removes a VFS file at the given URI
Parameters: uri (str) – URI of a VFS file resource Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError
-
supports
(self, scheme)¶ Returns true if the given URI scheme (storage backend) is supported
Parameters: scheme (str) – scheme component of a VFS resource URI (ex. ‘file’ / ‘hdfs’ / ‘s3’) Return type: bool Returns: True if the linked libtiledb version supports the storage backend, False otherwise Raises: ValueError – VFS storage backend is not supported
-
sync
(self, FileHandle fh)¶ Sync / flush an opened VFS FileHandle to storage backend
Parameters: fh (FileHandle) – An opened VFS FileHandle in ‘w’ or ‘a’ mode Raises: tiledb.TileDBError
-
touch
(self, uri)¶ Creates an empty VFS file at the given URI
Parameters: uri (str) – URI of a VFS file resource Return type: str Returns: URI of touched VFS file Raises: TypeError – cannot convert uri to unicode string Raises: tiledb.TileDBError