# README
etensor
Docs: GoDoc
etensor provides a basic set of tensor data structures (n-dimensional arrays of data), based on apache arrow tensor and intercompatible with those structures, and also with a gonum matrix interface.
The etensor.Tensor has all major data types available, and supports float64 and string access for all types. It provides the basis for the etable.Table columns.
The Shape of the tensor is a distinct struct that the tensor embeds, supporting row major ordering by default, but also column major or any other arbitrary ordering. To construct a tensor, use SetShape method.
Differences from arrow
- pure simple unidimensional Go slice used as the backing data array, auto allocated
- fully modifiable data -- arrow is designed to be read-only
- Shape struct is fully usable separate from the tensor data
- Everything exported, e.g., Offset method on Shape
- int used instead of int64 to make everything easier -- target platforms are all 64bit and have 64bit int in Go by default
Updating generated code
Run the following:
go install github.com/goki/stringer@latest github.com/apache/arrow/go/arrow/_tools/tmpl@latest
PATH=$(go env GOPATH)/bin:$PATH make generate
go generate
The go generate updates type_string using the goki version of stringer.
Note that the float64.go, int.go, string.go and bits.go types have some amount of custom code relative to the numeric.gen.go.tmpl template, and thus must be updated manually with any changes.
# Functions
AddShapes returns a new shape by adding two shapes one after the other.
No description provided by the author
ColMajorStrides returns strides for shape where the first dimension is inner-most and subsequent dimensions are progressively outer.
CopyDense copies a gonum mat.Dense matrix into given Tensor using standard Float64 interface.
CopyInts makes a copy of an int slice.
CopyInts64 makes a copy of an int64 slice to an int slice.
CopyStrings makes a copy of a string slice.
EqualInts compares two int slices and returns true if they are equal.
No description provided by the author
No description provided by the author
IntTo64 converts an []int slice to an []int64 slice.
New returns a new Tensor of given type, using our Type specifier which is isomorphic with arrow.Type.
NewBits returns a new n-dimensional array of bits If strides is nil, row-major strides will be inferred.
NewBitsShape returns a new n-dimensional array of bits If strides is nil, row-major strides will be inferred.
NewFloat32 returns a new n-dimensional array of float32s.
NewFloat32Shape returns a new n-dimensional array of float32s.
NewFloat64 returns a new n-dimensional array of float64s.
NewFloat64Shape returns a new n-dimensional array of float64s.
NewInt returns a new n-dimensional array of ints.
NewInt16 returns a new n-dimensional array of int16s.
NewInt16Shape returns a new n-dimensional array of int16s.
NewInt32 returns a new n-dimensional array of int32s.
NewInt32Shape returns a new n-dimensional array of int32s.
NewInt64 returns a new n-dimensional array of int64s.
NewInt64Shape returns a new n-dimensional array of int64s.
NewInt8 returns a new n-dimensional array of int8s.
NewInt8Shape returns a new n-dimensional array of int8s.
NewIntShape returns a new n-dimensional array of ints.
NewShape returns a new shape object initialized with params.
NewString returns a new n-dimensional array of strings If strides is nil, row-major strides will be inferred.
NewStringShape returns a new n-dimensional array of strings from given shape.
NewUint16 returns a new n-dimensional array of uint16s.
NewUint16Shape returns a new n-dimensional array of uint16s.
NewUint32 returns a new n-dimensional array of uint32s.
NewUint32Shape returns a new n-dimensional array of uint32s.
NewUint64 returns a new n-dimensional array of uint64s.
NewUint64Shape returns a new n-dimensional array of uint64s.
NewUint8 returns a new n-dimensional array of uint8s.
NewUint8Shape returns a new n-dimensional array of uint8s.
OpenCSV reads a tensor from a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg), using the Go standard encoding/csv reader conforming to the official CSV standard.
Prjn2DCoords returns the corresponding full-dimensional coordinates that go into the given row, col coords for a 2D projection of the given tensor, collapsing higher dimensions down to 2D (and 1D up to 2D).
Prjn2DIdx returns the flat 1D index for given row, col coords for a 2D projection of the given tensor shape, collapsing higher dimensions down to 2D (and 1D up to 2D).
Prjn2DSet sets a float64 value at given row, col coords for a 2D projection of the given tensor, collapsing higher dimensions down to 2D (and 1D up to 2D).
Prjn2DShape returns the size of a 2D projection of the given tensor Shape, collapsing higher dimensions down to 2D (and 1D up to 2D).
Prjn2DVal returns the float64 value at given row, col coords for a 2D projection of the given tensor, collapsing higher dimensions down to 2D (and 1D up to 2D).
ReadCSV reads a tensor from a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg), using the Go standard encoding/csv reader conforming to the official CSV standard.
RowMajorStrides returns strides for shape where the first dimension is outer-most and subsequent dimensions are progressively inner.
SaveCSV writes a tensor to a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg).
SetFloat64SliceLen is a utility function to set given slice of float64 values to given length, reusing existing where possible and making a new one as needed.
No description provided by the author
WriteCSV writes a tensor to a comma-separated-values (CSV) file (where comma = any delimiter, specified in the delim arg).
# Constants
Bool is a 1 bit, LSB bit-packed ordering.
FLOAT16 is a 2-byte floating point value.
FLOAT32 is a 4-byte floating point value.
FLOAT64 is an 8-byte floating point value.
INT is a Signed 64-bit little-endian integer -- should only use on 64bit machines!.
INT16 is a Signed 16-bit little-endian integer.
INT32 is a Signed 32-bit little-endian integer.
INT64 is a Signed 64-bit little-endian integer.
INT8 is a Signed 8-bit little-endian integer.
Null type having no physical storage.
STRING is a UTF8 variable-length string.
No description provided by the author
UINT16 is an Unsigned 16-bit little-endian integer.
UINT32 is an Unsigned 32-bit little-endian integer.
UINT64 is an Unsigned 64-bit little-endian integer.
UINT8 is an Unsigned 8-bit little-endian integer.
# Variables
No description provided by the author
# Structs
etensor.Bits is a tensor of bits backed by a bitslice.Slice for efficient storage of binary data.
No description provided by the author
Float32 is an n-dim array of float32s.
Float64 is an n-dim array of float64s.
Int is an n-dim array of ints, which are assumed to be 64bit in various places so usage on non-64bit platforms is not recommended.
Int16 is an n-dim array of int16s.
Int32 is an n-dim array of int32s.
Int64 is an n-dim array of int64s.
Int8 is an n-dim array of int8s.
Shape manages a tensor's shape information, including strides and dimension names and can compute the flat index into an underlying 1D data storage array based on an n-dimensional index (and vice-versa).
etensor.String is a tensor of strings backed by a []string slice.
Uint16 is an n-dim array of uint16s.
Uint32 is an n-dim array of uint32s.
Uint64 is an n-dim array of uint64s.
Uint8 is an n-dim array of uint8s.
# Interfaces
Tensor is the general interface for n-dimensional tensors.
# Type aliases
AggFunc is an aggregation function that incrementally updates agg value from each element in the tensor in turn -- returns new agg value that will be passed into next item as agg.
EvalFunc is an evaluation function that computes a function on each element value, returning the computed value.
Type is a logical type -- the subset supported by etable.