matx package¶
- matx.Device¶
alias of
DeviceOp
- class matx.Dict(seq=None, **kwargs)[source]¶
Bases:
Object
- matx.Dict implemented refering to python built-in dict,
supports common methods of built-in list and some custom methods.
- Dict() -> construct empty dict
>>> import matx >>> d = matx.Dict() >>> print(d) {}
- Dict(mapping) -> construct dict from mapping
>>> import matx >>> d = matx.Dict({'a': 1, 'b':2}) >>> print(d) {a: 1, b: 2}
- bucket_count()[source]¶
Returns the number of slots in the hash table.
- Returns:
int
Examples
>>> d = matx.Dict({'a': 1, 'b': 2}) >>> print(d.bucket_count()) 4
- clear()[source]¶
Remove all items.
- Returns:
None
Examples
>>> import matx >>> d = matx.Dict({'a': 1, 'b': 2}) >>> d {a: 1, b: 2} >>> d.clear() >>> d {}
- get(k, d=None)[source]¶
Return the value for key if key is in the dictionary, d.
- Parameters:
k (item) –
d (item) – defautl return value when k is not in dict
- Returns:
item
Examples
>>> import matx >>> d = matx.Dict({'a': 1, 'b': 2}) >>> d.get('a') 1 >>> d.get('a', 3) 1 >>> d.get('c', 3) 3
- items()[source]¶
Return a key-value iterable (matx.Iterator).
- Returns:
matx.Iterator
Examples
>>> import matx >>> d = matx.Dict({'a': 1, 'b': 2}) >>> it = d.items() >>> type(it) <class 'matx.runtime._container._iterator.Iterator'> >>> for k, v in it: ... print(k, v) ... a 1 b 2
- keys()[source]¶
Return a key iterable.
- Returns:
matx.Iterator
Examples
>>> import matx >>> d = matx.Dict({'a': 1, 'b': 2}) >>> it = d.keys() >>> type(it) <class 'matx.runtime._container._iterator.Iterator'> >>> for k in it: ... print(k) ... a b
- pop(k, *args)[source]¶
- .pop(k[,d]) -> v, remove specified key and return the corresponding value.
If key is not found, d is returned if given, otherwise Exception is raised
- Parameters:
k (item) –
- Returns:
item
Examples
>>> import matx >>> d = matx.Dict({'a': 1, 'b': 2, 'c': 3}) >>> d.pop('a') 1 >>> d.pop('a', 100) 100 >>> d.pop('a') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/data00/shubaihan/repos/matx4/python/matx/runtime/_container/_dict.py", line 223, in pop return _ffi_api.DictPop(self, k, *args) TypeError: MATXError: dict.pop KeyError Stack trace: File "/data00/shubaihan/repos/matx4/src/runtime/container/dict_ref.cc", line 305 [bt] (0) /data00/shubaihan/repos/matx4/lib/libmatx.so(+0x33c9ed) [0x7f70d4ad39ed] [bt] (1) /data00/shubaihan/repos/matx4/lib/libmatx.so(matx::runtime::Dict::pop(matx::runtime::PyArgs)+0x92) [0x7f70d4ad5362] [bt] (2) /data00/shubaihan/repos/matx4/lib/libmatx.so(+0x33f434) [0x7f70d4ad6434] [bt] (3) /data00/shubaihan/repos/matx4/lib/libmatx.so(MATXFuncCall_PYTHON_C_API+0x45) [0x7f70d4ab8c45]
- class matx.File(path, mode='r', encoding='utf-8')[source]¶
Bases:
Object
A simple file class, which only supports reading by lines now.
File(path, mode, encoding) -> similar to builtins.open
- class matx.List(seq=())[source]¶
Bases:
Object
- matx.List implemented refering to python built-in list,
supports common methods of built-in list and some custom methods.
- List() -> construct empty list
>>> import matx >>> l = matx.List() >>> print(l) []
- List(iterable) -> construct list from iterable object
>>> import matx >>> l = matx.List([1, 2, 3]) >>> print(l) [1, 2, 3] >>> l1 = matx.List([1, 2, 3]) >>> l2 = matx.List(l1) >>> print(l2) [1, 2, 3]
- append(p_object)[source]¶
Append object to the end of the list.
- Parameters:
p_object –
- Returns:
None
Examples
>>> import matx >>> l = matx.List() >>> l.append(1) >>> l [1]
- capacity()[source]¶
Return the number of elements that the list has currently allocated space for.
- Returns:
int
Examples
>>> import matx >>> l = matx.List([1, 2, 3]) >>> print(l) [1, 2, 3] >>> print(l.capacity()) 4
- clear()[source]¶
Remove all items from list.
- Returns:
None
Examples
>>> import matx >>> l = matx.List([1, 2, 3]) >>> l [1, 2, 3] >>> l.clear() >>> l []
- extend(iterable)[source]¶
Extend list by appending elements from the iterable.
- Parameters:
iterable – iterable
- Returns:
None
Examples
>>> import matx >>> l = matx.List([1, 2, 3]) >>> l.extend(matx.List([1, 2, 3])) >>> print(l) [1, 2, 3, 1, 2, 3] >>> l.extend([1, 2, 3]) >>> print(l) [1, 2, 3, 1, 2, 3, 1, 2, 3]
- index(x, start=None, end=None)[source]¶
Return zero-based index in the list of the first item whose value is equal to x.
- Parameters:
x (value type) –
start (int) –
end (int) –
- Raises:
a ValueError if there is no such item. –
- Returns:
None
- insert(index, value)[source]¶
Insert an item at a given position. The first argument is the index of the element before which to insert
- Parameters:
index (int) –
value (value type) –
- Returns:
None
- pop(index=-1)[source]¶
- Remove and return item at index (default last).
Raises Exception if list is empty or index is out of range.
- Parameters:
index (int, optional) –
- Returns:
item
- Raises:
Exception –
Examples
>>> import matx >>> l = matx.List([1, 2, 3]) >>> l.pop() 3 >>> import matx >>> l = matx.List([1, 2, 3, 4, 5]) >>> l.pop() 5 >>> l [1, 2, 3, 4] >>> l.pop(1) 2 >>> l [1, 3, 4]
- remove(value)[source]¶
- Remove first occurrence of value.
Raises Exception if the value is not present.
- Parameters:
value –
- Returns:
None
- Raises:
Exception –
Examples
>>> import matx >>> l = matx.List(["a", "b", "c", "d", "c"]) >>> l.remove("c") >>> l [a, b, d, c]
- reserve(new_size)[source]¶
Increase the capacity of the list to a value that’s greater or equal to new_size.
- Parameters:
new_size (int) –
- Returns:
None
Examples
>>> import matx >>> l = matx.List([1, 2, 3]) >>> print(l) [1, 2, 3] >>> print(l.capacity()) 4 >>> l.reserve(8) >>> print(l) [1, 2, 3] >>> print(l.capacity()) 8
- class matx.NDArray(arr: Union[int, float, list, List], shape: Union[list, List], dtype: str, device: str = 'cpu')[source]¶
Bases:
Object
Lightweight NDArray implementation for matx runtime
The structure is currently just a container for a multi-dimensional array, without defining various types of transformations and arithmetic methods. The goal of the structure is to serve as a bridge tool and other machine learning frameworks (pytorch tensorflow) for the conversion of multidimensional arrays
- Parameters:
- Construction method 1
arr is a one-dimensional List, the shape is not empty, producing NDArray with the content of arr and the shape of the given shape
- Examples:
>>> import matx >>> nd = matx.NDArray([1,2,3,4,5,6], [3,2], "int32") >>> nd [ [ 1 2 ] [ 3 4 ] [ 5 6 ] ]
- Construction method 2
arr is a List of arbitrary dimensions, shape is an empty List, producing NDArray with the same shape as arr and the same content as arr
- Examples:
>>> import matx >>> nd = matx.NDArray([[1, 2, 3,], [4, 5, 6]], [], "int32") >>> nd [ [ 1 2 3 ] [ 4 5 6 ] ]
>>> nd = matx.NDArray([1, 2, 3, 4, 5, 6], [], "int32") >>> nd [1, 2, 3, 4, 5, 6]
- Construction method 3
arr is empty, shape is not empty, return a NDArray corresponding to the random initialization content of the shape
- Examples:
>>> imoprt matx >>> nd = matx.NDArray([], [2, 3], "float32") >>> nd [ [ 4.46171e-08 3.08678e-41 2.25609e-43 ] [ 0 8.15743e-06 4.5723e-41 ] ]
- __init__(arr: Union[int, float, list, List], shape: Union[list, List], dtype: str, device: str = 'cpu')[source]¶
- asnumpy()[source]¶
Construct a numpy.ndarray from the current NDArray. Note! This method cannot be compiled for use in matx.script
- Returns:
numpy.ndarray
Examples
>>> import matx >>> import numpy >>> nd = matx.NDArray([[1,2,3],[4,5,6]], [], "int32") >>> nd [ [1, 2, 3] [4, 5, 6] ] >>> arr = nd.asnumpy() >>> arr array([[1, 2, 3], [4, 5, 6]], dtype=int32)
- contiguous()[source]¶
Returns a copy of the ndarray with contiguous memory if the adarray is not contiguous. Otherwise, the original one is returned.
- Returns:
matx.NDArray
Examples
>>> import matx >>> x = matx.NDArray([1,2,3,4], [2,2], "int32") >>> y = x.transpose() >>> z = y.contiguous() [[1, 3], [2, 4]] >>> z.is_contiguous() 1
- dim()[source]¶
Returns the number of array dimensions. Unlike numpy, this is a method and not a property.
- Returns:
int
Examples
>>> import matx >>> x = matx.NDArray([], [3,2], "int32") >>> x.dim() 2
- dtype()[source]¶
Returns the dtype of the current NDArray as a string
- Returns:
“int32” “int64” “float32” “float64”
- Return type:
str
Examples
>>> import matx >>> x = matx.NDArray([], [3,2], "int32") >>> x.dtype() 'int32'
- from_numpy(source_array)[source]¶
- Copy data from a numpy.ndarray to the current NDArray, requiring both to have the same size.
Note! This method cannot be compiled for use in matx.script
- Parameters:
source_array (numpy.ndarray) –
- Raises:
ValueError –
- Returns:
self
- Return type:
Examples
>>> import matx >>> import numpy >>> arr = numpy.random.rand(2, 3) >>> arr array([[0.0402096 , 0.99905783, 0.85840985], [0.89764146, 0.25342801, 0.566187 ]]) >>> nd = matx.NDArray([], arr.shape, str(arr.dtype)) >>> nd.from_numpy(arr) [ [0.0402096 , 0.99905783, 0.85840985] [0.89764146, 0.25342801, 0.566187 ] ]
- is_contiguous()[source]¶
Returns a int indicating if the underlying data is contiguous in memory. The continuity of array changes when its stride changes.
- Returns:
int
Examples
>>> import matx >>> x = matx.NDArray([1,2,3,4], [2,2], "int32") >>> y = x.transpose() >>> y.is_contiguous() 0
- numpy()[source]¶
Construct a numpy.ndarray from the current NDArray. Note! This method cannot be compiled for use in matx.script
- Returns:
numpy.ndarray
Examples
>>> import matx >>> import numpy >>> nd = matx.NDArray([[1,2,3],[4,5,6]], [], "int32") >>> nd [ [1, 2, 3] [4, 5, 6] ] >>> arr = nd.numpy() >>> arr array([[1, 2, 3], [4, 5, 6]], dtype=int32)
- shape()[source]¶
Returns the current NDArray’s shape, unlike numpy, this is a method and not a property
- Returns:
matx.List
Examples
>>> import matx >>> x = matx.NDArray([], [3,2], "int32") >>> x.shape() [3, 2]
- stride()[source]¶
Returns List of bytes to step in each dimension when traversing an array.
- Returns
matx.List
Examples
>>> import matx >>> x = matx.NDArray([1,2,3,4], [2,2], "int32") >>> y = x.transpose() >>> y.stride() [1, 2]
- to_list()[source]¶
Convert a NDArray to a matx.List corresponding to the shape
- Returns:
matx.List
Examples
>>> import matx >>> nd = matx.NDArray([], [2, 3], "float32") >>> nd [ [ 1.35134e-24 4.55548e-41 1.35134e-24 ] [ 4.55548e-41 7.41622e+14 3.06142e-41 ] ] >>> l = nd.to_list() >>> l [[1.35134e-24, 4.55548e-41, 1.35134e-24], [4.55548e-41, 7.41622e+14, 3.06142e-41]]
- tolist()[source]¶
Convert a NDArray to a matx.List corresponding to the shape
- Returns:
matx.List
Examples
>>> import matx >>> nd = matx.NDArray([], [2, 3], "float32") >>> nd [ [ 1.35134e-24 4.55548e-41 1.35134e-24 ] [ 4.55548e-41 7.41622e+14 3.06142e-41 ] ] >>> l = nd.tolist() >>> l [[1.35134e-24, 4.55548e-41, 1.35134e-24], [4.55548e-41, 7.41622e+14, 3.06142e-41]]
- torch(copy=True)[source]¶
convert NDArray to torch.Tensor, make sure NDArray is synchronized
- Returns:
torch.Tensor
- class matx.Regex(pattern, ignore_case=False, dotall=False, extended=False, anchored=False, ucp=True)[source]¶
Bases:
Object
Regular class implemented using pcre.
- Parameters:
pattern (str) – Str types. Regular expression pattern.
ignore_case (bool) – Booleans. Perform case-insensitive matching. The default is false
dotall (bool) – Booleans. “.” matches any character at all, including the newline. The default is false
extended (bool) – Booleans. Most white space in the pattern (other than in a character class), and characters between a # outside a character class and the next newline, inclusive, are ignored. An escaping backslash can be used to include a white space or # character as part of the pattern. The default is false.
anchored (bool) – Booleans. Matches only at the beginning of the subject. The default is false.
ucp (bool) – Booleans. Sequences such as “d” and “w” use Unicode properties to determine character types, instead of recognizing only characters with codes less than 128 via a lookup table. The default is false.
Examples
>>> import matx >>> regex = matx.Regex("(?<first>.*) are (?<second>.*?) .*") >>> regex Object(0x55c11322a200)
- __init__(pattern, ignore_case=False, dotall=False, extended=False, anchored=False, ucp=True)[source]¶
- match(string: Union[str, bytes], offset: int = 0)[source]¶
Try to apply the pattern at the start of the string, returning a tuple containing the matched string. If grouping version of regular pattern is used, then the text of all groups are returned.
- Parameters:
string (str|bytes) – The source string.
offset (int) – Offset in the subject at which to start matching
- Returns:
The matched groups. The first element in the tuple is indexed groups. The second element in the tuple is named groups.
- Return type:
Examples
>>> import matx >>> regex = matx.Regex("(?<first>.*) are (?<second>.*?) .*") >>> matched_result = regex.match("Cats are smarter than dogs") >>> matched_result[0] ['Cats are smarter than dogs', 'Cats', 'smarter'] >>> matched_result[1] {'first': 'Cats', 'second': 'smarter'}
- replace(string: Union[str, bytes], repl: Union[str, bytes])[source]¶
Return the string obtained by replacing the leftmost non-overlapping occurrences of the pattern in the input string by the replacement repl.
- Parameters:
string (str|bytes) – The source string.
repl (str|bytes) – The replacement string.
- Returns:
The replaced string. If no match was found, returning the source string.
- Return type:
str|bytes
Examples
>>> import matx >>> regex = matx.Regex("name") >>> new_str = regex.replace("mynameisHE", "NAME") >>> new_str myNAMEisHE
- split(string: Union[str, bytes])[source]¶
Split a string by the occurrences of a pattern.
- Parameters:
string (str|bytes) – The source string.
- Returns:
A list containing the resulting substrings. If no match was found,returning a list containing only the source string, i.e. [input].
- Return type:
List[str|bytes]
Examples
>>> import matx >>> regex = matx.Regex("name") >>> tokens = regex.split("mynameisHE") >>> tokens ['my', 'isHE']
- class matx.Set(seq=())[source]¶
Bases:
Object
- matx.Set: matx.Set implemented refering to python built-in dict,
supports common methods of built-in list and some custom methods.
set() -> construct empty set .. rubric:: Examples
>>> import matx >>> s = matx.Set() >>> print(s) {}
set(iterable) -> construct set from iterable .. rubric:: Examples
>>> import matx >>> s = matx.Set(['a', 1, 'b']) >>> print(s) {a, b, 1}
- add(p_object)[source]¶
- Add an element to a set.
This has no effect if the element is already present.
- Returns
None
Examples
>>> import matx >>> s = matx.Set(['a', 1, 'b']) >>> s.add('c') >>> print(s) {a, c, b, 1}
- bucket_count()[source]¶
Returns the number of slots in the hash table.
- Returns:
int
Examples
>>> import matx >>> s = matx.Set(['a', 1, 'b']) >>> print(s.bucket_count()) 8
- clear()[source]¶
Remove all elements.
- Returns:
None
Examples
>>> import matx >>> s = matx.Set(['a', 1, 'b']) >>> print(s) {a, b, 1} >>> s.clear() >>> print(s) {}
- difference(*args)[source]¶
Return the difference of two or more sets as a new set.
- Returns:
matx.Set
Examples
>>> import matx >>> s = matx.Set({'a', 'b', 'c', 'd', 'e', 'f'}) >>> s.difference(matx.Set({'a', 'b'}), {'d', 1}, ['f'], matx.List([2])) {'c', 'e'} >>> s {'a', 'd', 'c', 'b', 'f', 'e'}
- difference_update(*args)[source]¶
Remove all elements of another set from this set.
- Returns:
None
Examples
>>> s = matx.Set({'a', 'b', 'c', 'd', 'e', 'f'}) >>> s.difference_update(matx.Set({'a', 'b'}), {'d', 1}, ['f'], matx.List([2])) >>> s {'c', 'e'}
- discard(item)[source]¶
- Remove an element from a set if it is a member.
If the element is not a member, do nothing.
- Returns:
None
Examples
>>> import matx >>> s = matx.Set({'a', 'b', 'c'}) >>> s.discard('a') >>> s {'c', 'b'} >>> s.discard('d') >>> s {'c', 'b'}
- reserve(new_size)[source]¶
Increase the capacity of the set to a value that’s greater or equal to new_size.
- Parameters:
new_size (int) –
- Returns:
None
Examples
>>> import matx >>> s = matx.Set(['a', 1, 'b']) >>> print(s.bucket_count()) 8 >>> s.reserve(10) >>> print(s.bucket_count()) 32
- exception matx.TError[source]¶
Bases:
RuntimeError
Default error thrown by packed functions.
TError will be raised if you do not give any error type specification,
- class matx.Trie(dic=None)[source]¶
Bases:
Object
Double Array Trie Object
- Parameters:
dic (Dict[str, int]) – The key is word and the value is id
Examples
>>> import matx >>> tree = matx.Trie({"hello": 1, "hello w": 2, "good": 3}) >>> tree Trie(addr: 0x5601feb2aca0) >>> tree = matx.Trie() >>> tree Trie(addr: 0x5601feba5b90)
- prefix_search(w, pos=0)[source]¶
Find the longest substring of w[pos:] in the trie tree
- Parameters:
w (str) – The input word
pos (int, optional) – The start position
- Returns:
- Return the length and id of the matched substring.
If not found, return (0, -1)
- Return type:
Tuple(int, int)
Examples
>>> import matx >>> tree = matx.Trie({"hello": 1, "hello w": 2, "good": 3}) >>> tree.prefix_search("hello world") (7, 2) >>> tree.prefix_search("hellf") (0, -1)
- prefix_search_all(w, pos=0)[source]¶
Find all substring of w[pos:] in the trie tree
- Parameters:
w (str) – The input word
pos (int, optional) – The start position
- Returns:
- Return a list of the length and id of the matched substring.
If not found, return []
- Return type:
Examples
>>> import matx >>> trie = matx.Trie({'hello': 1, 'hello world': 2, 'hello hello world': 3}) >>> trie.prefix_search_all("hello hello world") [(5, 1), (17, 3)]
- update(w, index=-1)[source]¶
Insert a word and corresponding id into the trie tree
- Parameters:
w (str) – The input word
index (int, optional) – id, -1 for default
Examples
>>> import matx >>> tree = matx.Trie() >>> tree.update("hello", 1) >>> tree.update("hello w", 2) >>> tree.update("good", 3) >>> tree.prefix_search("hello world") (7, 2) >>> tree.prefix_search("hellf") (0, -1) >>> tree.update("hell", 10) >>> tree.prefix_search("hellf") (4, 10)
- matx.get_global_func(name, allow_missing=False)[source]¶
Get a global function by name
- Parameters:
name (str) – The name of the global function
allow_missing (bool) – Whether allow missing function or raise an error.
- Returns:
func – The function to be returned, None if function is missing.
- Return type:
- matx.ir_module(obj)¶
- matx.list_heap_pushpop(l, item, comp=None)¶
- matx.list_heap_replace(l, item, comp=None)¶
- matx.list_heapify(l, comp=None)¶
- matx.list_nth_element(l, n, comp=None)¶
- matx.register_func(func_name, f=None, override=False)[source]¶
Register global function
- Parameters:
func_name (str or function) – The function name
f (function, optional) – The function to be registered.
override (boolean optional) – Whether override existing entry.
- Returns:
fregister – Register function if f is not specified.
- Return type:
function
Examples
The following code registers my_packed_func as global function. Note that we simply get it back from global function table to invoke it from python side. However, we can also invoke the same function from C++ backend, or in the compiled TVM code.
targs = (10, 10.0, "hello") @matx.register_func def my_packed_func(*args): assert(tuple(args) == targs) return 10 # Get it out from global function table f = matx.get_global_func("my_packed_func") assert isinstance(f, matx.PackedFunc) y = f(*targs) assert y == 10
- matx.script(compiling_obj, *args, backend=None, **kwargs)[source]¶
Entry function for compiling. Given a python object including function, simple class, compile it to a matx object which mostly keep the behavior of the original python object.
- Parameters:
compiling_obj (([function, class])) – [input python object to be compiled.]
args – not used
backend ([str, None]) – ‘torch’ or ‘pytorch’ or None
kwargs – keyword arguments passed into different script backend
- Returns:
obj – the compiled object.
- Return type:
OpKernel
- matx.trace(func, *args, **kwargs)[source]¶
Trace a function and return an executable module that will be optimized using just-in-time compilation.
- Parameters:
func (callable) – A Python function or a matx Symbol(s) that will be run with args. func arguments and return values must be Operators returned from Script.
args – func inputs
kwargs – func inputs
- Returns:
module – an executable module
- Return type:
JITModule
Subpackages¶
- matx.runtime package
- Submodules
- matx.runtime.container module
- matx.runtime.cuda_stream module
- matx.runtime.file module
- matx.runtime.module module
- matx.runtime.msgpack module
- matx.runtime.ndarray module
- matx.runtime.object module
- matx.runtime.object_generic module
- matx.runtime.packed_func module
- matx.runtime.picke module
- matx.runtime.regex module
- matx.runtime.trie module
- Submodules
- matx.text package
- matx.vision package
AutoContrastOp
AverageBlurOp
CastOp
CenterCropOp
ChannelReorderOp
ColorLinearAdjustOp
Conv2dOp
CropOp
CvtColorOp
EdgeDetectOp
EmbossOp
FlipOp
GammaContrastOp
GaussNoiseOp
GaussianBlurOp
HistEqualizeOp
ImdecodeNoExceptionOp
ImdecodeNoExceptionRandomCropOp
ImdecodeOp
ImdecodeRandomCropOp
ImencodeNoExceptionOp
ImencodeOp
InvertOp
LaplacianBlurOp
MeanOp
MedianBlurOp
MixupImagesOp
NormalizeOp
PadOp
PadWithBorderOp
PosterizeOp
RandomDropoutOp
RandomResizedCropOp
ResizeOp
RotateOp
SaltAndPepperOp
SharpenOp
SolarizeOp
SplitOp
StackOp
SumOp
TransposeNormalizeOp
TransposeOp
WarpAffineOp
WarpPerspectiveOp
- Subpackages
- matx.vision.tv_transforms package
CenterCrop
ColorJitter
Compose
ConvertImageDtype
CvtColor
Decode
GaussianBlur
Grayscale
Normalize
Pad
RandomAdjustSharpness
RandomAffine
RandomAutocontrast
RandomCrop
RandomEqualize
RandomGrayscale
RandomHorizontalFlip
RandomInvert
RandomPerspective
RandomPosterize
RandomResizedCrop
RandomRotation
RandomSolarize
RandomVerticalFlip
Resize
Stack
ToTensor
Transpose
set_device
- Submodules
- matx.vision.tv_transforms package