ColumnBasedInsertMsgToInsertData converts an InsertMsg msg into InsertData based on provided CollectionSchema collSchema.

DeserializeStats deserialize @blobs as []*PrimaryKeyStats.

GetDimFromParams get dim from params.

TODO: string type.

GetTimestampFromInsertData returns the Int64FieldData for timestamp field.

ListAllChunkWithPrefix is a helper function to list all objects with same @prefix by using `ListWithPrefix`.

Locations returns a list of hash locations representing a data item.

MergeFieldData merge field into data.

MergeInsertData append the insert datas to the original buffer.

NewBinlogReader creates binlogReader to read binlog file.

NewDataDefinitionCodec is constructor for DataDefinitionCodec.

NewDDLBinlogWriter creates DDLBinlogWriter to write binlog file.

NewDeleteBinlogWriter creates DeleteBinlogWriter to write binlog file.

NewDeleteCodec returns a DeleteCodec.

NewIndexCodec creates IndexCodec.

NewIndexFileBinlogCodec is constructor for IndexFileBinlogCodec.

NewIndexFileBinlogWriter returns a new IndexFileBinlogWriter with provided parameters.

NewInsertBinlogIterator creates a new iterator Deprecated: use storage.NewBinlogDeserializeReader instead.

NewInsertBinlogWriter creates InsertBinlogWriter to write binlog file.

NewInsertCodec creates an InsertCodec.

NewInsertCodecWithSchema creates an InsertCodec with provided collection meta.

NewLocalChunkManager create a new local manager object.

NewMergeIterator return a new MergeIterator.

NewNullableInt creates a new NullableInt instance.

NewRemoteChunkManagerForTesting is used for testing.

Open opens file as os.Open works, also converts the os errors to Milvus errors.

ParseSegmentIDByBinlog parse segment id from binlog paths if path format is not expected, returns error.

PrintBinlogFiles call printBinlogFile in turn for the file list specified by parameter fileList.

ReadBinary read data in bytes and write it into receiver.

ReadDataFromAllRowGroups iterates all row groups of file.Reader, and convert column to E.

ReadDescriptorEvent reads a descriptorEvent from buffer.

ReadFile reads file as os.ReadFile works, also converts the os errors to Milvus errors.

TransferColumnBasedInsertDataToRowBased transfer column-based insert data to row-based rows.

#nosec G103 */.

#nosec G103 */.

#nosec G103 */.

#nosec G103 */.

#nosec G103 */.

#nosec G103 */.

#nosec G103 */.

#nosec G103 */.

WriteFile writes file as os.WriteFile works， also converts the os errors to Milvus errors.

BM25 BinlogType for bm25 stats data.

CompundStatsType log save multiple stats and bloom filters to one file.

EventTypeCode definitions.

EventTypeCode definitions.

DDL is blob key "ddl".

DDLBinlog BinlogType for DDL.

DeleteBinlog BinlogType for delete data.

EventTypeCode definitions.

EventTypeCode definitions.

EventTypeCode definitions.

EventTypeCode definitions.

EventTypeCode definitions.

IndexFileBinlog BinlogType for index.

EventTypeCode definitions.

IndexParamsKey is blob key "indexParams".

InsertBinlog BinlogType for insert data.

EventTypeCode definitions.

InvalidUniqueID is used when the UniqueID is not set (like in return with err).

MagicNumber used in binlog.

mark useMultiFieldFormat if there are multi fields in a log file.

StatsBinlog BinlogType for stats data.

Ts is blob key "ts".

ErrDisposed is the error that the iterator is disposed.

ErrNoMoreRecord is the error that the iterator does not have next record.

BinlogReader is an object to read binlog file.

Blob is a pack of key&value.

TODO: fill it info for each blob.

BlobReader is implemented because Azure's stream body does not have ReadAt and Seek interfaces.

ChunkObjectInfo is to store object info.

DataDefinitionCodec serializes and deserializes the data definition Blob key example: ${tenant}/data_definition_log/${collection_id}/ts/${log_idx} ${tenant}/data_definition_log/${collection_id}/ddl/${log_idx}.

DataSorter sorts insert data.

DDLBinlogWriter is an object to write binlog file which saves ddl information.

DeleteBinlogWriter is an object to write binlog file which saves delete data.

DeleteCodec serializes and deserializes the delete data.

DeleteData saves each entity delete message represented as <primarykey,timestamp> map.

DeltaData stores delta data currently only delete tuples are stored.

DescriptorEventDataFixPart is a memory struct saves events' DescriptorEventData.

DataType_Double.

EventReader is used to parse the events contained in the Binlog file.

FieldStats contains statistics data for any column todo: compatible to PrimaryKeyStats.

FieldStatsReader reads stats.

FieldStatsWriter writes stats to buffer.

DataType_Float.

IndexCodec can serialize and deserialize index.

IndexFileBinlogWriter is an object to write binlog file which saves index files.

InsertBinlogIterator is the iterator of binlog.

InsertBinlogWriter is an object to write binlog file which saves insert data.

InsertCodec serializes and deserializes the insert data Blob key example: ${tenant}/insert_log/${collection_id}/${partition_id}/${segment_id}/${field_id}/${log_idx}.

InsertData example row_schema: {float_field, int_field, float_vector_field, string_field} Data {<0, row_id>, <1, timestamp>, <100, float_field>, <101, int_field>, <102, float_vector_field>, <103, string_field>} system filed id: 0: unique row id 1: timestamp 100: first user field id 101: second user field id 102: ...

DataType_Int16.

DataType_Int32.

DataType_Int64.

DataType_Int8.

LocalChunkManager is responsible for read and write local file.

LocationsCache is a helper struct caching pk bloom filter locations.

MergeIterator merge iterators.

PayloadReader reads data from payload.

pkStatistics contains pk field statistic information.

PrimaryKeyStats contains rowsWithToken data for pk column.

RemoteChunkManager is responsible for read and write data stored in minio.

StatsReader reads stats.

StatsWriter writes stats to buffer.

Value is the return value of Next.

ChunkManager is to manager chunks.

EventWriter abstracts event writer.

FieldData defines field data interface.

Iterator is the iterator interface.

PayloadReaderInterface abstracts PayloadReader.

PayloadWriterInterface abstracts PayloadWriter.

PrimaryKeys is the interface holding a slice of PrimaryKey.

BinlogType is to distinguish different files saving different data.

BlobList implements sort.Interface for a list of Blob.

ChunkObjectWalkFunc is the callback function for walking objects.

EventTypeCode represents event type by code.

Option is used to config the retry function.