matx package¶
- matx.Device¶
alias of
DeviceOp
- class matx.Dict(seq=None, **kwargs)[source]¶
Bases:
Object- matx.Dict implemented refering to python built-in dict,
supports common methods of built-in list and some custom methods.
- Dict() -> construct empty dict
>>> import matx >>> d = matx.Dict() >>> print(d) {}
- Dict(mapping) -> construct dict from mapping
>>> import matx >>> d = matx.Dict({'a': 1, 'b':2}) >>> print(d) {a: 1, b: 2}
- bucket_count()[source]¶
Returns the number of slots in the hash table.
- Returns:
int
Examples
>>> d = matx.Dict({'a': 1, 'b': 2}) >>> print(d.bucket_count()) 4
- clear()[source]¶
Remove all items.
- Returns:
None
Examples
>>> import matx >>> d = matx.Dict({'a': 1, 'b': 2}) >>> d {a: 1, b: 2} >>> d.clear() >>> d {}
- get(k, d=None)[source]¶
Return the value for key if key is in the dictionary, d.
- Parameters:
k (item) –
d (item) – defautl return value when k is not in dict
- Returns:
item
Examples
>>> import matx >>> d = matx.Dict({'a': 1, 'b': 2}) >>> d.get('a') 1 >>> d.get('a', 3) 1 >>> d.get('c', 3) 3
- items()[source]¶
Return a key-value iterable (matx.Iterator).
- Returns:
matx.Iterator
Examples
>>> import matx >>> d = matx.Dict({'a': 1, 'b': 2}) >>> it = d.items() >>> type(it) <class 'matx.runtime._container._iterator.Iterator'> >>> for k, v in it: ... print(k, v) ... a 1 b 2
- keys()[source]¶
Return a key iterable.
- Returns:
matx.Iterator
Examples
>>> import matx >>> d = matx.Dict({'a': 1, 'b': 2}) >>> it = d.keys() >>> type(it) <class 'matx.runtime._container._iterator.Iterator'> >>> for k in it: ... print(k) ... a b
- pop(k, *args)[source]¶
- .pop(k[,d]) -> v, remove specified key and return the corresponding value.
If key is not found, d is returned if given, otherwise Exception is raised
- Parameters:
k (item) –
- Returns:
item
Examples
>>> import matx >>> d = matx.Dict({'a': 1, 'b': 2, 'c': 3}) >>> d.pop('a') 1 >>> d.pop('a', 100) 100 >>> d.pop('a') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/data00/shubaihan/repos/matx4/python/matx/runtime/_container/_dict.py", line 223, in pop return _ffi_api.DictPop(self, k, *args) TypeError: MATXError: dict.pop KeyError Stack trace: File "/data00/shubaihan/repos/matx4/src/runtime/container/dict_ref.cc", line 305 [bt] (0) /data00/shubaihan/repos/matx4/lib/libmatx.so(+0x33c9ed) [0x7f70d4ad39ed] [bt] (1) /data00/shubaihan/repos/matx4/lib/libmatx.so(matx::runtime::Dict::pop(matx::runtime::PyArgs)+0x92) [0x7f70d4ad5362] [bt] (2) /data00/shubaihan/repos/matx4/lib/libmatx.so(+0x33f434) [0x7f70d4ad6434] [bt] (3) /data00/shubaihan/repos/matx4/lib/libmatx.so(MATXFuncCall_PYTHON_C_API+0x45) [0x7f70d4ab8c45]
- class matx.File(path, mode='r', encoding='utf-8')[source]¶
Bases:
ObjectA simple file class, which only supports reading by lines now.
File(path, mode, encoding) -> similar to builtins.open
- class matx.List(seq=())[source]¶
Bases:
Object- matx.List implemented refering to python built-in list,
supports common methods of built-in list and some custom methods.
- List() -> construct empty list
>>> import matx >>> l = matx.List() >>> print(l) []
- List(iterable) -> construct list from iterable object
>>> import matx >>> l = matx.List([1, 2, 3]) >>> print(l) [1, 2, 3] >>> l1 = matx.List([1, 2, 3]) >>> l2 = matx.List(l1) >>> print(l2) [1, 2, 3]
- append(p_object)[source]¶
Append object to the end of the list.
- Parameters:
p_object –
- Returns:
None
Examples
>>> import matx >>> l = matx.List() >>> l.append(1) >>> l [1]
- capacity()[source]¶
Return the number of elements that the list has currently allocated space for.
- Returns:
int
Examples
>>> import matx >>> l = matx.List([1, 2, 3]) >>> print(l) [1, 2, 3] >>> print(l.capacity()) 4
- clear()[source]¶
Remove all items from list.
- Returns:
None
Examples
>>> import matx >>> l = matx.List([1, 2, 3]) >>> l [1, 2, 3] >>> l.clear() >>> l []
- extend(iterable)[source]¶
Extend list by appending elements from the iterable.
- Parameters:
iterable – iterable
- Returns:
None
Examples
>>> import matx >>> l = matx.List([1, 2, 3]) >>> l.extend(matx.List([1, 2, 3])) >>> print(l) [1, 2, 3, 1, 2, 3] >>> l.extend([1, 2, 3]) >>> print(l) [1, 2, 3, 1, 2, 3, 1, 2, 3]
- index(x, start=None, end=None)[source]¶
Return zero-based index in the list of the first item whose value is equal to x.
- Parameters:
x (value type) –
start (int) –
end (int) –
- Raises:
a ValueError if there is no such item. –
- Returns:
None
- insert(index, value)[source]¶
Insert an item at a given position. The first argument is the index of the element before which to insert
- Parameters:
index (int) –
value (value type) –
- Returns:
None
- pop(index=-1)[source]¶
- Remove and return item at index (default last).
Raises Exception if list is empty or index is out of range.
- Parameters:
index (int, optional) –
- Returns:
item
- Raises:
Exception –
Examples
>>> import matx >>> l = matx.List([1, 2, 3]) >>> l.pop() 3 >>> import matx >>> l = matx.List([1, 2, 3, 4, 5]) >>> l.pop() 5 >>> l [1, 2, 3, 4] >>> l.pop(1) 2 >>> l [1, 3, 4]
- remove(value)[source]¶
- Remove first occurrence of value.
Raises Exception if the value is not present.
- Parameters:
value –
- Returns:
None
- Raises:
Exception –
Examples
>>> import matx >>> l = matx.List(["a", "b", "c", "d", "c"]) >>> l.remove("c") >>> l [a, b, d, c]
- reserve(new_size)[source]¶
Increase the capacity of the list to a value that’s greater or equal to new_size.
- Parameters:
new_size (int) –
- Returns:
None
Examples
>>> import matx >>> l = matx.List([1, 2, 3]) >>> print(l) [1, 2, 3] >>> print(l.capacity()) 4 >>> l.reserve(8) >>> print(l) [1, 2, 3] >>> print(l.capacity()) 8
- class matx.NDArray(arr: Union[int, float, list, List], shape: Union[list, List], dtype: str, device: str = 'cpu')[source]¶
Bases:
ObjectLightweight NDArray implementation for matx runtime
The structure is currently just a container for a multi-dimensional array, without defining various types of transformations and arithmetic methods. The goal of the structure is to serve as a bridge tool and other machine learning frameworks (pytorch tensorflow) for the conversion of multidimensional arrays
- Parameters:
- Construction method 1
arr is a one-dimensional List, the shape is not empty, producing NDArray with the content of arr and the shape of the given shape
- Examples:
>>> import matx >>> nd = matx.NDArray([1,2,3,4,5,6], [3,2], "int32") >>> nd [ [ 1 2 ] [ 3 4 ] [ 5 6 ] ]
- Construction method 2
arr is a List of arbitrary dimensions, shape is an empty List, producing NDArray with the same shape as arr and the same content as arr
- Examples:
>>> import matx >>> nd = matx.NDArray([[1, 2, 3,], [4, 5, 6]], [], "int32") >>> nd [ [ 1 2 3 ] [ 4 5 6 ] ]
>>> nd = matx.NDArray([1, 2, 3, 4, 5, 6], [], "int32") >>> nd [1, 2, 3, 4, 5, 6]
- Construction method 3
arr is empty, shape is not empty, return a NDArray corresponding to the random initialization content of the shape
- Examples:
>>> imoprt matx >>> nd = matx.NDArray([], [2, 3], "float32") >>> nd [ [ 4.46171e-08 3.08678e-41 2.25609e-43 ] [ 0 8.15743e-06 4.5723e-41 ] ]
- __init__(arr: Union[int, float, list, List], shape: Union[list, List], dtype: str, device: str = 'cpu')[source]¶
- asnumpy()[source]¶
Construct a numpy.ndarray from the current NDArray. Note! This method cannot be compiled for use in matx.script
- Returns:
numpy.ndarray
Examples
>>> import matx >>> import numpy >>> nd = matx.NDArray([[1,2,3],[4,5,6]], [], "int32") >>> nd [ [1, 2, 3] [4, 5, 6] ] >>> arr = nd.asnumpy() >>> arr array([[1, 2, 3], [4, 5, 6]], dtype=int32)
- contiguous()[source]¶
Returns a copy of the ndarray with contiguous memory if the adarray is not contiguous. Otherwise, the original one is returned.
- Returns:
matx.NDArray
Examples
>>> import matx >>> x = matx.NDArray([1,2,3,4], [2,2], "int32") >>> y = x.transpose() >>> z = y.contiguous() [[1, 3], [2, 4]] >>> z.is_contiguous() 1
- dim()[source]¶
Returns the number of array dimensions. Unlike numpy, this is a method and not a property.
- Returns:
int
Examples
>>> import matx >>> x = matx.NDArray([], [3,2], "int32") >>> x.dim() 2
- dtype()[source]¶
Returns the dtype of the current NDArray as a string
- Returns:
“int32” “int64” “float32” “float64”
- Return type:
str
Examples
>>> import matx >>> x = matx.NDArray([], [3,2], "int32") >>> x.dtype() 'int32'
- from_numpy(source_array)[source]¶
- Copy data from a numpy.ndarray to the current NDArray, requiring both to have the same size.
Note! This method cannot be compiled for use in matx.script
- Parameters:
source_array (numpy.ndarray) –
- Raises:
ValueError –
- Returns:
self
- Return type:
Examples
>>> import matx >>> import numpy >>> arr = numpy.random.rand(2, 3) >>> arr array([[0.0402096 , 0.99905783, 0.85840985], [0.89764146, 0.25342801, 0.566187 ]]) >>> nd = matx.NDArray([], arr.shape, str(arr.dtype)) >>> nd.from_numpy(arr) [ [0.0402096 , 0.99905783, 0.85840985] [0.89764146, 0.25342801, 0.566187 ] ]
- is_contiguous()[source]¶
Returns a int indicating if the underlying data is contiguous in memory. The continuity of array changes when its stride changes.
- Returns:
int
Examples
>>> import matx >>> x = matx.NDArray([1,2,3,4], [2,2], "int32") >>> y = x.transpose() >>> y.is_contiguous() 0
- numpy()[source]¶
Construct a numpy.ndarray from the current NDArray. Note! This method cannot be compiled for use in matx.script
- Returns:
numpy.ndarray
Examples
>>> import matx >>> import numpy >>> nd = matx.NDArray([[1,2,3],[4,5,6]], [], "int32") >>> nd [ [1, 2, 3] [4, 5, 6] ] >>> arr = nd.numpy() >>> arr array([[1, 2, 3], [4, 5, 6]], dtype=int32)
- shape()[source]¶
Returns the current NDArray’s shape, unlike numpy, this is a method and not a property
- Returns:
matx.List
Examples
>>> import matx >>> x = matx.NDArray([], [3,2], "int32") >>> x.shape() [3, 2]
- stride()[source]¶
Returns List of bytes to step in each dimension when traversing an array.
- Returns
matx.List
Examples
>>> import matx >>> x = matx.NDArray([1,2,3,4], [2,2], "int32") >>> y = x.transpose() >>> y.stride() [1, 2]
- to_list()[source]¶
Convert a NDArray to a matx.List corresponding to the shape
- Returns:
matx.List
Examples
>>> import matx >>> nd = matx.NDArray([], [2, 3], "float32") >>> nd [ [ 1.35134e-24 4.55548e-41 1.35134e-24 ] [ 4.55548e-41 7.41622e+14 3.06142e-41 ] ] >>> l = nd.to_list() >>> l [[1.35134e-24, 4.55548e-41, 1.35134e-24], [4.55548e-41, 7.41622e+14, 3.06142e-41]]
- tolist()[source]¶
Convert a NDArray to a matx.List corresponding to the shape
- Returns:
matx.List
Examples
>>> import matx >>> nd = matx.NDArray([], [2, 3], "float32") >>> nd [ [ 1.35134e-24 4.55548e-41 1.35134e-24 ] [ 4.55548e-41 7.41622e+14 3.06142e-41 ] ] >>> l = nd.tolist() >>> l [[1.35134e-24, 4.55548e-41, 1.35134e-24], [4.55548e-41, 7.41622e+14, 3.06142e-41]]
- torch(copy=True)[source]¶
convert NDArray to torch.Tensor, make sure NDArray is synchronized
- Returns:
torch.Tensor
- class matx.Regex(pattern, ignore_case=False, dotall=False, extended=False, anchored=False, ucp=True)[source]¶
Bases:
ObjectRegular class implemented using pcre.
- Parameters:
pattern (str) – Str types. Regular expression pattern.
ignore_case (bool) – Booleans. Perform case-insensitive matching. The default is false
dotall (bool) – Booleans. “.” matches any character at all, including the newline. The default is false
extended (bool) – Booleans. Most white space in the pattern (other than in a character class), and characters between a # outside a character class and the next newline, inclusive, are ignored. An escaping backslash can be used to include a white space or # character as part of the pattern. The default is false.
anchored (bool) – Booleans. Matches only at the beginning of the subject. The default is false.
ucp (bool) – Booleans. Sequences such as “d” and “w” use Unicode properties to determine character types, instead of recognizing only characters with codes less than 128 via a lookup table. The default is false.
Examples
>>> import matx >>> regex = matx.Regex("(?<first>.*) are (?<second>.*?) .*") >>> regex Object(0x55c11322a200)
- __init__(pattern, ignore_case=False, dotall=False, extended=False, anchored=False, ucp=True)[source]¶
- match(string: Union[str, bytes], offset: int = 0)[source]¶
Try to apply the pattern at the start of the string, returning a tuple containing the matched string. If grouping version of regular pattern is used, then the text of all groups are returned.
- Parameters:
string (str|bytes) – The source string.
offset (int) – Offset in the subject at which to start matching
- Returns:
The matched groups. The first element in the tuple is indexed groups. The second element in the tuple is named groups.
- Return type:
Examples
>>> import matx >>> regex = matx.Regex("(?<first>.*) are (?<second>.*?) .*") >>> matched_result = regex.match("Cats are smarter than dogs") >>> matched_result[0] ['Cats are smarter than dogs', 'Cats', 'smarter'] >>> matched_result[1] {'first': 'Cats', 'second': 'smarter'}
- replace(string: Union[str, bytes], repl: Union[str, bytes])[source]¶
Return the string obtained by replacing the leftmost non-overlapping occurrences of the pattern in the input string by the replacement repl.
- Parameters:
string (str|bytes) – The source string.
repl (str|bytes) – The replacement string.
- Returns:
The replaced string. If no match was found, returning the source string.
- Return type:
str|bytes
Examples
>>> import matx >>> regex = matx.Regex("name") >>> new_str = regex.replace("mynameisHE", "NAME") >>> new_str myNAMEisHE
- split(string: Union[str, bytes])[source]¶
Split a string by the occurrences of a pattern.
- Parameters:
string (str|bytes) – The source string.
- Returns:
A list containing the resulting substrings. If no match was found,returning a list containing only the source string, i.e. [input].
- Return type:
List[str|bytes]
Examples
>>> import matx >>> regex = matx.Regex("name") >>> tokens = regex.split("mynameisHE") >>> tokens ['my', 'isHE']
- class matx.Set(seq=())[source]¶
Bases:
Object- matx.Set: matx.Set implemented refering to python built-in dict,
supports common methods of built-in list and some custom methods.
set() -> construct empty set .. rubric:: Examples
>>> import matx >>> s = matx.Set() >>> print(s) {}
set(iterable) -> construct set from iterable .. rubric:: Examples
>>> import matx >>> s = matx.Set(['a', 1, 'b']) >>> print(s) {a, b, 1}
- add(p_object)[source]¶
- Add an element to a set.
This has no effect if the element is already present.
- Returns
None
Examples
>>> import matx >>> s = matx.Set(['a', 1, 'b']) >>> s.add('c') >>> print(s) {a, c, b, 1}
- bucket_count()[source]¶
Returns the number of slots in the hash table.
- Returns:
int
Examples
>>> import matx >>> s = matx.Set(['a', 1, 'b']) >>> print(s.bucket_count()) 8
- clear()[source]¶
Remove all elements.
- Returns:
None
Examples
>>> import matx >>> s = matx.Set(['a', 1, 'b']) >>> print(s) {a, b, 1} >>> s.clear() >>> print(s) {}
- difference(*args)[source]¶
Return the difference of two or more sets as a new set.
- Returns:
matx.Set
Examples
>>> import matx >>> s = matx.Set({'a', 'b', 'c', 'd', 'e', 'f'}) >>> s.difference(matx.Set({'a', 'b'}), {'d', 1}, ['f'], matx.List([2])) {'c', 'e'} >>> s {'a', 'd', 'c', 'b', 'f', 'e'}
- difference_update(*args)[source]¶
Remove all elements of another set from this set.
- Returns:
None
Examples
>>> s = matx.Set({'a', 'b', 'c', 'd', 'e', 'f'}) >>> s.difference_update(matx.Set({'a', 'b'}), {'d', 1}, ['f'], matx.List([2])) >>> s {'c', 'e'}
- discard(item)[source]¶
- Remove an element from a set if it is a member.
If the element is not a member, do nothing.
- Returns:
None
Examples
>>> import matx >>> s = matx.Set({'a', 'b', 'c'}) >>> s.discard('a') >>> s {'c', 'b'} >>> s.discard('d') >>> s {'c', 'b'}
- reserve(new_size)[source]¶
Increase the capacity of the set to a value that’s greater or equal to new_size.
- Parameters:
new_size (int) –
- Returns:
None
Examples
>>> import matx >>> s = matx.Set(['a', 1, 'b']) >>> print(s.bucket_count()) 8 >>> s.reserve(10) >>> print(s.bucket_count()) 32
- exception matx.TError[source]¶
Bases:
RuntimeErrorDefault error thrown by packed functions.
TError will be raised if you do not give any error type specification,
- class matx.Trie(dic=None)[source]¶
Bases:
ObjectDouble Array Trie Object
- Parameters:
dic (Dict[str, int]) – The key is word and the value is id
Examples
>>> import matx >>> tree = matx.Trie({"hello": 1, "hello w": 2, "good": 3}) >>> tree Trie(addr: 0x5601feb2aca0) >>> tree = matx.Trie() >>> tree Trie(addr: 0x5601feba5b90)
- prefix_search(w, pos=0)[source]¶
Find the longest substring of w[pos:] in the trie tree
- Parameters:
w (str) – The input word
pos (int, optional) – The start position
- Returns:
- Return the length and id of the matched substring.
If not found, return (0, -1)
- Return type:
Tuple(int, int)
Examples
>>> import matx >>> tree = matx.Trie({"hello": 1, "hello w": 2, "good": 3}) >>> tree.prefix_search("hello world") (7, 2) >>> tree.prefix_search("hellf") (0, -1)
- prefix_search_all(w, pos=0)[source]¶
Find all substring of w[pos:] in the trie tree
- Parameters:
w (str) – The input word
pos (int, optional) – The start position
- Returns:
- Return a list of the length and id of the matched substring.
If not found, return []
- Return type:
Examples
>>> import matx >>> trie = matx.Trie({'hello': 1, 'hello world': 2, 'hello hello world': 3}) >>> trie.prefix_search_all("hello hello world") [(5, 1), (17, 3)]
- update(w, index=-1)[source]¶
Insert a word and corresponding id into the trie tree
- Parameters:
w (str) – The input word
index (int, optional) – id, -1 for default
Examples
>>> import matx >>> tree = matx.Trie() >>> tree.update("hello", 1) >>> tree.update("hello w", 2) >>> tree.update("good", 3) >>> tree.prefix_search("hello world") (7, 2) >>> tree.prefix_search("hellf") (0, -1) >>> tree.update("hell", 10) >>> tree.prefix_search("hellf") (4, 10)
- matx.get_global_func(name, allow_missing=False)[source]¶
Get a global function by name
- Parameters:
name (str) – The name of the global function
allow_missing (bool) – Whether allow missing function or raise an error.
- Returns:
func – The function to be returned, None if function is missing.
- Return type:
- matx.ir_module(obj)¶
- matx.list_heap_pushpop(l, item, comp=None)¶
- matx.list_heap_replace(l, item, comp=None)¶
- matx.list_heapify(l, comp=None)¶
- matx.list_nth_element(l, n, comp=None)¶
- matx.register_func(func_name, f=None, override=False)[source]¶
Register global function
- Parameters:
func_name (str or function) – The function name
f (function, optional) – The function to be registered.
override (boolean optional) – Whether override existing entry.
- Returns:
fregister – Register function if f is not specified.
- Return type:
function
Examples
The following code registers my_packed_func as global function. Note that we simply get it back from global function table to invoke it from python side. However, we can also invoke the same function from C++ backend, or in the compiled TVM code.
targs = (10, 10.0, "hello") @matx.register_func def my_packed_func(*args): assert(tuple(args) == targs) return 10 # Get it out from global function table f = matx.get_global_func("my_packed_func") assert isinstance(f, matx.PackedFunc) y = f(*targs) assert y == 10
- matx.script(compiling_obj, *args, backend=None, **kwargs)[source]¶
Entry function for compiling. Given a python object including function, simple class, compile it to a matx object which mostly keep the behavior of the original python object.
- Parameters:
compiling_obj (([function, class])) – [input python object to be compiled.]
args – not used
backend ([str, None]) – ‘torch’ or ‘pytorch’ or None
kwargs – keyword arguments passed into different script backend
- Returns:
obj – the compiled object.
- Return type:
OpKernel
- matx.trace(func, *args, **kwargs)[source]¶
Trace a function and return an executable module that will be optimized using just-in-time compilation.
- Parameters:
func (callable) – A Python function or a matx Symbol(s) that will be run with args. func arguments and return values must be Operators returned from Script.
args – func inputs
kwargs – func inputs
- Returns:
module – an executable module
- Return type:
JITModule
Subpackages¶
- matx.runtime package
- Submodules
- matx.runtime.container module
- matx.runtime.cuda_stream module
- matx.runtime.file module
- matx.runtime.module module
- matx.runtime.msgpack module
- matx.runtime.ndarray module
- matx.runtime.object module
- matx.runtime.object_generic module
- matx.runtime.packed_func module
- matx.runtime.picke module
- matx.runtime.regex module
- matx.runtime.trie module
- Submodules
- matx.text package
- matx.vision package
AutoContrastOpAverageBlurOpCastOpCenterCropOpChannelReorderOpColorLinearAdjustOpConv2dOpCropOpCvtColorOpEdgeDetectOpEmbossOpFlipOpGammaContrastOpGaussNoiseOpGaussianBlurOpHistEqualizeOpImdecodeNoExceptionOpImdecodeNoExceptionRandomCropOpImdecodeOpImdecodeRandomCropOpImencodeNoExceptionOpImencodeOpInvertOpLaplacianBlurOpMeanOpMedianBlurOpMixupImagesOpNormalizeOpPadOpPadWithBorderOpPosterizeOpRandomDropoutOpRandomResizedCropOpResizeOpRotateOpSaltAndPepperOpSharpenOpSolarizeOpSplitOpStackOpSumOpTransposeNormalizeOpTransposeOpWarpAffineOpWarpPerspectiveOp- Subpackages
- matx.vision.tv_transforms package
CenterCropColorJitterComposeConvertImageDtypeCvtColorDecodeGaussianBlurGrayscaleNormalizePadRandomAdjustSharpnessRandomAffineRandomAutocontrastRandomCropRandomEqualizeRandomGrayscaleRandomHorizontalFlipRandomInvertRandomPerspectiveRandomPosterizeRandomResizedCropRandomRotationRandomSolarizeRandomVerticalFlipResizeStackToTensorTransposeset_device- Submodules
- matx.vision.tv_transforms package