ArchiveInfoFromRemotePath returns the archive type and directory from the remotePath from either ns_server or test code, or logs and returns an error if the type is missing or unrecognized.

Handler for /api/v1/bucket/<bucket-name>/<function-name>.

Compare two arrays of byte arrays and find out diff of which byte entry needs to be deleted and which needs to be inserted Both the byte arrays are expected to have unique entries.

CreateMutationStreamReader creates a new mutation stream and starts a reader to listen and process the mutations.

Return true if x < y.

flip bits in-place for a given byte slice.

GetBucketFromKeyspaceId will return the bucket name from either a 1-part ("bucket") or 3-part ("bucket:scope:collection") key.

GetCurrentKVTs gets the current KV timestamp vector for the specified number of vBuckets.

GetGlobalTopology does a scatter-gather to all available Index Service nodes to get their full index topologies.

Return encoded key with docid without size check.

This has to follow the pattern in IndexPath function defined above.

This has to follow the pattern in IndexPath function defined above.

GetPauseMgr atomically gets the global PauseMgr pointer.

Get slice snaspshot.

Host2key converts a host:httpPort IPV4 or IPV6 string to a key for the metaCache and statsCache.

Return true if a < b.

Return true if x < y.

Key2host converts a host_httpPort cache key to the original IPV4 or IPV6 host:httpPort URL form, assuming the first char of an IPV6 key is "[" as this is required in URLs using IPV6.

NewAtomicMutationQueue allocates a new Atomic Mutation Queue and initializes it.

NewRowBuffer initializes a new queue with a given size and pre-allocated Rows.

NewAutofailoverServiceManager is the constructor for the AutofailoverServiceManager class.

NewClustMgrAgent creates a new ClustMgrAgent.

NewCpuThrottle is the constructor for the CpuThrottle class.

Constructor.

NewFlusher returns new instance of flusher.

NewForestDBSlice initiailizes a new slice with forestdb backend.

NewGenericServiceManager is the constructor for the GenericServiceManager class.

NewHashedSliceContainer initializes a new HashedSliceContainer and returns.

NewIndexer is the constructor for the Indexer interface implemented by the indexer class.

DEPRECATED!! NewMasterServiceManager is the constructor for the MasterServiceManager class.

NewMemDBSlice is the constructor for memdbSlice.

NewMutationManager creates a new Mutation Manager which listens for commands fromIndexer.

NewNodeLoad is the constructor for the NodeLoad class.

NewPauseObjutil is the constructor for the PauseObjutil class.

NewPauser creates a Pauser instance to execute the given task.

NewPauseServiceManager is the constructor for the PauseServiceManager class.

Constructor.

NewRebalancer creates the Rebalancer object that will run the rebalance on this node and starts go routines that perform it asynchronously.

NewRebalanceServiceManager is the constructor for the RebalanceServiceManager class.

NewRequestHandlerCache is the constructor for the requestHandlerCache class.

NewResumer creates a Resumer instance to execute the given task.

NewScanCoordinator returns an instance of scanCoordinator or err message It listens on supvCmdch for command and every command is followed by a synchronous response on the supvCmdch.

used only in bhive scan path.

NewServerlessManager is the constructor for the ServerlessManager class.

NewSettingsManager is the settingsManager constructor.

NewSlabManager returns a slabManager struct instance with an initialized Arena.

NewStatsManager is the constructor for statsManager.

NewStorageManager returns an instance of storageMgr or err message It listens on supvCmdch for command and every command is followed by a synchronous response of the supvCmdch.

NewTaskObj is the constructor for the taskObj class.

NewTimekeeper returns an instance of timekeeper or err message.

NewTopKRowHeap creates a new TopKRowHeap with the given capacity and heap type.

NewWorkerPool creates a new WorkerPool.

RegisterRequestHandler is the main entry point of the request handler.

RemoveDuplicateIndexes function gets input of hosts and index definitions to be removed from hosts, these are indexes that have been detected as duplicate and are being removed as part of rebalance.

Helper method to pretty print timestamp.

SetPauseMgr atomically sets the global PauseMgr pointer.

SplitKeyspaceId will return all three parts of a bucket:scope:collection key.

Start Cpu collection .

Upload wraps the tools-common/objstore/objutil/upload.go Upload method.

ADMIN_MGR.

RangeReq is a span which is Range on the entire index without composite index filtering FilterRangeReq is a span request which needs composite index filtering.

Azure Cloud Storage.

local filesystem.

Google Cloud Storage.

undefined.

AWS S3 bucket.

increment by +100 -- see above.

XXX_LEVEL constants are used in the constraints.level field to indicate at what level of the hierarchy index stats are requested.

Catchup Topic Name.

CBQ_BRIDGE_SHUTDOWN.

CLUSTER_MGR.

TODO add the Prefix in Indexer settings.

XXX_LEVEL constants are used in the constraints.level field to indicate at what level of the hierarchy index stats are requested.

Backup corrupt index data files.

CPU_THROTTLE_EXPIRY_SEC is # of seconds after last HealthCheck call to asssume Autofailover is disabled and thus CPU throttling should be turned off.

Default cluster address This can ONLY be used for indexer main as default value for command line argument.

Constants for unlimiting entry size.

Default Pool Name.

Slab Manager Specific constants.

CLUSTER_MGR_AGENT.

INDEXER.

KVSender.

metering throttling mgr.

Mutation Manager.

Mutation Queue.

ScanCoordinator.

Slab Manager.

STORAGE_MGR.

Stream Reader.

Timekeeper.

generated internally.

FILENAME_METADATA is the name of the file to write containing the index metadata from ONE node.

FILENAME_PAUSE_METADATA is the name of the file to write containing the PauseMetadata.

FILENAME_STATS is the name of the file to write containing persisted index stats from ONE node.

Range with filtering.

XXX_LEVEL constants are used in the constraints.level field to indicate at what level of the hierarchy index stats are requested.

INDEXER.

XXX_LEVEL constants are used in the constraints.level field to indicate at what level of the hierarchy index stats are requested.

Deletes are not normalized with numInserts while tenant management.

Initial Stream Topic Name.

No batching.

In Milliseconds.

op_monitor constants.

KVSender.

LOAD_XXX enum are node load levels returned by NodeLoad.getLoad.

LOAD_XXX enum are node load levels returned by NodeLoad.getLoad.

RangeReq is a span which is Range on the entire index without composite index filtering FilterRangeReq is a span request which needs composite index filtering.

Maintenance Topic Name.

Max number of vbuckets supported in the system.

Max number of snapshot to be retained per index.

max msec to adjust throttleDelayMs by at one time.

max msec to delay an action due to throttling.

32 bytes for SHA256 encode and 4 bytes for number of such entries.

to grep for all the following logXyz messages.

threshold for warning of slow message processing.

General Messages.

MUTATION_MANAGER.

number of past CPU stats to keep.

Range with no filtering.

stop sending mutations for anything in keyspace.

timeout in milliseconds to batch the vbuckets together for repair message.

Picked up for MTR.

Not Used.

Marked for repair || KVReq sent for restart.

KVReq sent for shutdown & restart.

KVResp received tried shutdown.

same collection as a prior index in this batch (implies shared).

not replica repair and not sharing a prior stream.

same partition as a prior index in this batch (implies shared).

replica repair not sharing a prior stream.

replica repair sharing a prior stream in this batch.

SCAN COORDINATOR.

XXX_LEVEL constants are used in the constraints.level field to indicate at what level of the hierarchy index stats are requested.

Internal Buffer Size for Each Slice to store incoming requests.

Ready for operations.

Slice is warming up(open db files etc), not ready for operations.

Marked for deletion.

Constants for stats persistence in snapshot meta.

STORAGE_MANAGER.

Stream is active i.e.

Stream is inactive i.e.

Prepare is done before recovery.

Stream is preparing for recovery(i.e.

STREAM_READER.

Stream is using a Catchup to recover.

suffix for temporary filename to be atomically renamed.

Large row scans - Say every iteration is 1 RU - 200 Iterations.

Small row scans.

TIMEKEEPER.

In Milliseconds.

COMMON.

VECTOR_TODO: Check if this is needed.

Supervisor's channel capacity to buffer requests from workers.

Intermediate message buffer channel size.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Errors.

Seconds.

Special index keys.

Special index keys.

Special index keys.

PauseMgr points to the singleton of this class, which will still be nil early in user ScanCoordinator's lifecycle, hence the need for atomics.

Milliseconds.

Milliseconds.

Seconds.

Milliseconds.

Milliseconds.

Seconds.

Seconds.

Seconds.

AtomicRowBuffer is a thread-safe queue for Row pointers.

AutofailoverServiceManager provides the implementation of the ns_server RPC interface AutofailoverManager (defined in cbauth/service/interface.go).

Range for a single field in composite index.

CpuThrottle is a class to provide a CPU throttling delay to other components.

DDLServiceMgr Definition.

FlatFileStatsPersister implements the StatsPersister interface.

ForestDBIterator taken fromhttps://github.com/couchbaselabs/bleve/blob/master/index/store/goforestdb/iterator.go.

GenericServiceManager provides the implementation of the generic subset of the ns_server RPC Manager interface (defined in cbauth/service/interface.go).

GetTaskListResponse is the REST return payload of testGetTaskList.

hashedSliceContainer provides a hash based implementationfor SliceContainer.

IndexMutationQueue comprising of a mutation queue and a slab manager.

IndexerStats contains both indexer stats at top level and individual index stats under the indexes map.

These structures are used for validating items_count across multiple replicas to identify if an index is corrupt or not.

Holder for IndexInstMap.

Holder for IndexPartnMap.

A point in index and the corresponding filter the point belongs to either as high or low.

IndexStats holds statistics for a single index instance.

Represents storage stats for an index instance.

KeyspaceStats tracks statistics of all indexes in a given keyspace in a stream.

KeyspaceStatsMapHolder holds an atomic pointer to a KeyspaceStatsMap.

MasterServiceManager is used to work around cbauth's monolithic service manager architecture that requires a singleton to implement all the different interfaces, as cbauth requires this to be registered only once.

MergeOperator will read from recvQ, based on distances and either stores in heap or forwards row to down stream.

ADD_INDEX_INSTANCE.

INDEXER_CANCEL_MERGE_PARTITION CLUST_MGR_BUILD_INDEX_DDL CLUST_MGR_BUILD_RECOVERED_INDEXES.

CLUST_MGR_BUILD_INDEX_DDL_RESPONSE.

INDEXER_CANCEL_MERGE_PARTITION.

CLUST_MGR_CLEANUP_PARTITION.

CLUST_MGR_GET_LOCAL CLUST_MGR_SET_LOCAL CLUST_MGR_DEL_LOCAL.

CLUST_MGR_MERGE_PARTITION.

CLUST_MGR_PRUNE_PARTITION.

CLUST_MGR_RESET_INDEX_ON_ROLLBACK.

CLUST_MGR_RESET_INDEX_ON_UPGRADE.

CLUST_MGR_GET_GLOBAL_TOPOLOGY.

CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX.

CBQ_CREATE_INDEX_DDL CLUST_MGR_CREATE_INDEX_DDL.

CBQ_DROP_INDEX_DDL CLUST_MGR_DROP_INDEX_DDL.

Error Message.

Generic Message.

INDEXER_PAUSE_MOI INDEXER_RESUMEMOI INDEXER_PREPARE_UNPAUSE_MOI INDEXER_UNPAUSE_MOI INDEXER_BOOTSTRAP.

TK_GET_KEYSPACE_HWT STREAM_READER_HWT.

KV_STREAM_REPAIR.

INDEXER_MERGE_PARTITION.

MUT_MGR_FLUSH_DONE MUT_MGR_ABORT_DONE STORAGE_SNAP_DONE.

MUT_MGR_PERSIST_MUTATION_QUEUE MUT_MGR_ABORT_PERSIST MUT_MGR_DRAIN_MUTATION_QUEUE.

MUT_MGR_GET_MUTATION_QUEUE_HWT MUT_MGR_GET_MUTATION_QUEUE_LWT.

POOL_CHANGE.

INDEXER_INIT_PREP_RECOVERY INDEXER_PREPARE_RECOVERY INDEXER_PREPARE_DONE INDEXER_INITIATE_RECOVERY INDEXER_RECOVERY_DONE INDEXER_KEYSPACE_NOT_FOUND INDEXER_MTR_FAIL INDEXER_ABORT_RECOVERY.

KV_SENDER_REPAIR_ENDPOINTS.

INDEXER_RESET_INDEX_DONE.

KV_SENDER_RESTART_VBUCKETS.

KV_SENDER_RESTART_VBUCKETS_RESPONSE.

STATS_PERSISTER_START STATS_PERSISTER_STOP STATS_PERSISTER_FORCE_PERSIST STATS_PERSISTER_CONFIG_UPDATE STATS_READ_PERSISTED_STATS STATS_LOG_AT_EXIT.

Stream Reader Message.

Stream Error Message.

STREAM_READER_CONN_ERROR STREAM_REQUEST_DONE.

OPEN_STREAM ADD_INDEX_LIST_TO_STREAM REMOVE_KEYSPACE_FROM_STREAM REMOVE_INDEX_LIST_FROM_STREAM CLOSE_STREAM CLEANUP_STREAM CLEANUP_PRJ_STATS INDEXER_UPDATE_BUILD_TS RESET_STREAM.

Success Message.

Success Message.

MUT_MGR_STREAM_CLOSE.

Success Message.

Timestamp Message.

TK_INIT_BUILD_DONE TK_INIT_BUILD_DONE_ACK TK_INIT_BUILD_DONE_NO_CATCHUP_ACK TK_ADD_INSTANCE_FAIL.

TK_MERGE_STREAM TK_MERGE_STREAM_ACK.

TK_STABILITY_TIMESTAMP.

TK_ENABLE_FLUSH TK_DISABLE_FLUSH.

UPDATE_FLUSHER_MAPS.

UPDATE_INDEX_INSTANCE_MAP.

STREAM_READER_UPDATE_QUEUE_MAP.

UPDATE_KEYSPACE_STATS_MAP.

UPDATE_INDEX_PARTITION_MAP.

STORAGE_UPDATE_SNAP_MAP.

UPDATE_MAP_WORKER.

MutationKeys holds the Secondary Keys from a single KV Mutation.

MutationMeta represents meta information for a KV Mutation.

MutationSnapshot represents snapshot information of KV.

NodeLoad holds stats about each Index node retained by a rebal, used to estimate current load.

Contains the mapping between nodeUUID to hostname of a KV node.

NodeUUIDsResponse is used to return a list of Index Service NodeUUIDs from the /getCachedIndexerNodeUUIDs REST API.

PartitionInst contains the partition definition and a SliceContainerto manage all the slices storing the partition's data.

this information should be considered read only during resume created by pause leader node.

PauseObjutil object holds the state for a session of file transfer activity.

Pauser object holds the state of Pause orchestration.

PauseServiceManager provides the implementation of the Pause-Resume-specific APIs of ns_server RPC Manager interface (defined in cbauth/service/interface.go).

RebalanceServiceManager implements the ns_server cbauth/service.Manager interface, which drives Rebalance and Failover through RPC.

Resumer object holds the state of Resume orchestration.

RowHeap is a heap of *Row based on the dist field.

ScanWorker will scan a range, calculate distances, add distance and send data down the line.

ServerlessManager provides the implementation of the ns_server RPC interface ServerlessManager(defined in cbauth/service/interface.go).

ShardRebalancer embeds Rebalancer struct to reduce code duplication across common functions.

---------------------------------------------------------------------------- A map that holds the stats.

Fragementation is calculated based on DataSizeOnDisk and LogSpace stat a.

Holder for StreamKeyspaceIdInstList.

Holder for StreamKeyspaceIdInstList.

TaskResponse is the REST return payload of testCancelTask, testPause, and testResume, as they only need generic fields.

TopKRowHeap is a heap that maintains a fixed size.

Pool of ScanWokers.

ClustMgrAgent provides the mechanism to talk to Index Coordinator.

********* CLONE of COPIER obj in Plasma***************.

Counter is a class of algorithms that return total node count efficiently.

Exister is a class of algorithms that allow testing if a key exists in the index.

Flusher is the only component which does read/dequeue from a MutationQueue.

Generic index entry abstraction (primary or secondary) Represents a row in the index.

Generic index key abstraction (primary or secondary) Represents a key supplied by the user for scan operation.

Abstract context implemented by storage subsystem.

IndexSnapshot is an immutable data structure that provides point-in-time snapshot of an index instance held by an indexer.

KVSender provides the mechanism to talk to KV(projector, router etc).

Looker is a class of algorithms that allow looking up a key in an index.

MutationManager handles messages from Indexer to manage Mutation Streamsand flush mutations from mutation queues.

MutationQueue interface specifies methods which a mutation queue for indexerneeds to implement.

MutationStreamReader reads a Dataport and stores the incoming mutations in mutation queue.

RangeCounter is a class of algorithms that can count a range efficiently.

Ranger is a class of algorithms that can extract a range of keys from the index.

ReadAtSeeker is a composition of the reader/seeker/reader at interfaces.

Slice represents the unit of physical storage for index.

SliceContainer contains all slices for an index partitionand provides methods to determine how datais distributed in multiple slices for a single partition.

Snapshot interface.

A helper data stucture for in-memory snapshot info list.

STATS PERSISITER INTERFACE.

Timekeeper manages the Stability Timestamp Generation and also keeps track of the HWTimestamp for each keyspaceId.

ArchiveEnum defines types of storage archives for Pause-Resume (following Archive_XXX constants).

redefine regulator constants so that we dont need to import regulator module elsewhere.

Implements sort Interface.

Inclusion controls how the boundaries values of a range are treated.

IndexPartnMap maps a IndexInstId to PartitionInstMap.

Implements sort Interface.

IndexQueueMap is a map between IndexId and IndexerMutationQueue.

Map from keyspaceId to mutation queue.

Map from keyspaceId to flusher stop channel.

KeyspaceStatsMap holds a map of stream IDs to maps of keyspace IDs to per-keyspace stats.

NodeLoadSlice implements sort.Interface to enable sorting by load (which is really just two categories: low and high load).

PartitionInstMap maps a PartitionId to PartitionInst.

Called at the end of the pause lifecycle.

SortOrder characterizes if the algorithm emits keys in a predictable order.

Stability Timestamp.

---------------------------------------------------------------------------- For many int64 stats, aggregation of the stats across partitions is done for reporting the stat values.

list of seqno per vbucket.

KeyspaceId -> map of vbucket to hostname.

Function to write data downstream from MergeOperator.