package
17.0.0
Repository: https://github.com/apache/arrow.git
Documentation: pkg.go.dev

# Functions

ColumnPathFromNode walks the parents of the given node to construct it's column path.
DefaultSortOrder returns the default stat sort order for the given physical type.
FromParquet converts a slice of thrift Schema Elements to the correct node type.
GetLogicalSortOrder returns the default sort order for this logical type or falls back to the default sort order for the physical type if not valid.
GetSortOrder defaults to the sort order based on the physical type if convert is ConvertedTypes.None, otherwise determines the sort order by the converted type.
ListOf is a convenience helper function to create a properly structured list structure according to the Parquet Spec.
ListOf is a convenience helper function to create a properly structured list structure according to the Parquet Spec.
MapOf is a convenience helper function to create a properly structured parquet map node setup according to the Parquet Spec.
Must is a convenience function for the NewNode functions that return a Node and an error, panic'ing if err != nil or returning the node.
MustGroup is like Must, except it casts the node to a *GroupNode, which will panic if it is a primitive node.
MustPrimitive is like Must except it casts the node to *PrimitiveNode which will panic if it is a group node.
NewBooleanNode is a convenience factory for constructing an Boolean Primitive Node.
NewByteArrayNode is a convenience factory for constructing an Byte Array Primitive Node.
NewColumn returns a new column object for the given node with the provided maximum definition and repetition levels.
NewDecimalLogicalType returns a Decimal logical type with the given precision and scale.
NewFixedLenByteArrayNode is a convenience factory for constructing an Fixed Length Byte Array Primitive Node of the given length.
NewFloat32Node is a convenience factory for constructing an Float Primitive Node.
NewFloat64Node is a convenience factory for constructing an Double Primitive Node.
NewGroupNode constructs a new group node with the provided fields, but with converted type None and No Logical Type.
NewGroupNodeConverted constructs a group node with the provided fields and converted type, determining the logical type from that converted type.
NewGroupNodeLogical constructs a group node with the provided fields and logical type, determining the converted type from the provided logical type.
NewInt32Node is a convenience factory for constructing an Int32 Primitive Node.
NewInt64Node is a convenience factory for constructing an Int64 Primitive Node.
NewInt96Node is a convenience factory for constructing an Int96 Primitive Node.
NewIntLogicalType creates an integer logical type of the desired bitwidth and whether it is signed or not.
NewPrimitiveNode constructs a primitive node with the ConvertedType of None and no logical type.
NewPrimitiveNodeConverted constructs a primitive node from the given physical type and converted type, determining the logical type from the converted type.
NewPrimitiveNodeLogical constructs a Primitive node using the provided logical type for a given physical type and typelength.
NewSchema constructs a new Schema object from a root group node.
NewSchemaFromStruct generates a schema from an object type via reflection of the type and reading struct tags for "parquet".
NewStructFromSchema generates a struct type as a reflect.Type from the schema by using the appropriate physical types and making things either pointers or slices based on whether they are repeated/optional/required.
NewTimeLogicalType returns a time type of the given unit.
NewTimestampLogicalType returns a logical timestamp type with "forceConverted" set to false.
NewTimestampLogicalTypeForce returns a timestamp logical type with "forceConverted" set to true.
NewTimestampLogicalTypeWithOpts creates a new TimestampLogicalType with the provided options.
PrintSchema writes a string representation of the tree to w using the indent width provided.
ToThrift converts a GroupNode to a slice of SchemaElements which is used for thrift serialization.
WithTSForceConverted enable force converted mode.
WithTSFromConverted enable the timestamp logical type to be constructed from a converted type.
WithTSIsAdjustedToUTC sets the IsAdjustedToUTC field of the timestamp type.
WithTSTimeUnitType sets the time unit for the timestamp type.

# Constants

the available constants for NodeType.
the available constants for NodeType.
Constants for the Stat sort order definitions.
Constants for the Stat sort order definitions.
Constants for the Stat sort order definitions.
Constants for the TimeUnitType.
Constants for the TimeUnitType.
Constants for the TimeUnitType.
Constants for the TimeUnitType.

# Variables

ConvertedTypes is a struct containing the constants for the types to make it easy to reference them while making it clear what they are.

# Structs

BSONLogicalType represents a binary JSON string in the byte array.
Column encapsulates the information necessary to interpret primitive column data in the context of a particular schema.
DateLogicalType is an int32 representing the number of days since the Unix Epoch 1 January 1970.
DecimalLogicalType is used to represent a decimal value of a given precision and scale.
DecimalMetadata is a struct for managing scale and precision information between converted and logical types.
EnumLogicalType is for representing an enum, which should be a byte array type.
Float16LogicalType can only be used with a FixedLength byte array column that is exactly 2 bytes long.
GroupNode is for managing nested nodes like List, Map, etc.
IntervalLogicalType is not yet in the thrift spec, but represents an interval time and needs to be a fixed length byte array of 12 bytes.
IntLogicalType represents an integer type of a specific bit width and is either signed or unsigned.
JSONLogicalType represents a byte array column which is to be interpreted as a JSON string.
ListLogicalType is used for columns which are themselves nested lists.
MapLogicalType represents a mapped type.
A PrimitiveNode is a type that is one of the primitive Parquet storage types.
Schema is the container for the converted Parquet schema with a computed information from the schema analysis needed for file reading * Column index to Node * Max repetition / definition levels for each primitive node The ColumnDescriptor objects produced by this class can be used to assist in the reconstruction of fully materialized data structures from the repetition-definition level encoding of nested data.
StringLogicalType is a UTF8 string, only usable with ByteArray and FixedLenByteArray.
TimeLogicalType is a time type without a date and must be an int32 for milliseconds, or an int64 for micro or nano seconds.
TimestampLogicalType represents an int64 number that can be decoded into a year, month, day, hour, minute, second, and subsecond.
UnknownLogicalType is a type that is essentially a placeholder for when we don't know the type.
UUIDLogicalType can only be used with a FixedLength byte array column that is exactly 16 bytes long.

# Interfaces

LogicalType is the descriptor that defines the usage of a physical primitive type in the schema, such as an Interval, Date, etc.
Node is the interface for both Group and Primitive Nodes.
TemporalLogicalType is a smaller interface for Time based logical types like Time / Timestamp.
Visitor is an interface for creating functionality to walk the schema tree.

# Type aliases

ConvertedType corresponds to the ConvertedType in the parquet.Thrift, with added values of None and NA for handling when these values are not set in the metadata.
FieldList is an alias for a slice of Nodes.
NodeType describes whether the Node is a Primitive or Group node.
SortOrder mirrors the parquet.thrift sort order type.
TimestampOpt options used with New Timestamp Logical Type.
TimeUnitType is an enum for denoting whether a time based logical type is using milliseconds, microseconds or nanoseconds.