AddParam adds single parameter to SQL statement.

CheckPattern is a generic functino to check given key value within Lexicon map.

CheckQueryParameters checks query parameters against API parameters map.

CleanStatement cleans the given SQL statement to remove empty strings, etc.

CreateInvalidParamError creates the error for parameter validation.

Date provides default date for DB records.

DBStats returns database stats.

DecodeValidatorError provides uniform error representation of DBRecord validation errors.

helper function to create dbs error.

FlatLumis perform flat operation for given lumis lists.

GetBlocks returns list of blocks for a given url and block/dataset input.

GetChunks helper function to get ORACLE chunks from provided list of values.

GetID function fetches table primary id for a given value.

get list of migration blocks in order of processing (first parents then children).

GetParentBlocks returns parent blocks for given url and block namegocyclo:ignore.

GetParentDatasetBlocks returns full list of parent blocks associated with given datasetgocyclo:ignore.

GetParents returns list of parents for given block or dataset.

GetRecID function fetches table primary id for a given value and insert it if necessary.

GetTestData executes simple query to ensure that connection to DB is valid.

HttpClient is HTTP client for urlfetch server.

IfExist check if given rid, attr exists in given table for provided value conditions.

IfExistMulti checks if given rid exists in given table for provided value conditions.

IncrementSequence API returns single unique ID for a given sequence.

IncrementSequences API provide a way to get N unique IDs for given sequence name.

InsertFileLumisTxViaChunks DBS APIgocyclo:ignore.

LastInsertID returns last insert id of given table and idname parameter.

LoadApiParameters loads Api parameters and constructs ApiParameters map.

LoadPatterns loads CMS Lexion patterns from given file the format of the file is a list of the following dicts: [ {"name": <name>, "patterns": [list of patterns], "length": int},...].

LoadSQL function loads DBS SQL statements with Owner.

LoadTemplateSQL function loads DBS SQL templated statements.

MigrationCleanupServer represents migration cleanup daemon..

MigrationRequests fetches migration requests from migration table.

MigrationServer represent migration server.

OperatorValue function generates operator and value pair for a given argument.

ParseDBFile function parses given file name and extracts from it dbtype and dburi file should contain the "dbtype dburi" string.

ParseRuns parse run_num parameter and convert it to run list.

QueryRow function fetches results from given table.

RunsConditions function to handle runs conditions.

StartMigrationRequest starts asynchronously migration request process via goroutine with timeout context the code is based on the following example: https://medium.com/geekculture/timeout-context-in-go-e88af0abd08d.

TokenCondition provides proper condition statement for TokenGenerator.

TokenGenerator creates a SQL token generator statement.

TokenGeneratorORACLE creates a SQL token generator statement using ORACLE syntax https://betteratoracle.com/posts/20-how-do-i-bind-a-variable-in-list.

TokenGeneratorSQLite creates a SQL token generator statement using SQLite syntax https://stackoverflow.com/questions/67372811/what-is-equivalent-of-token-generator-oracle-sql-statement-in-sqlite.

Validate provides validation of all input parameters of HTTP request.

Validate POST/PUT Parameters.

ValidatePostPayload function to validate POST request.

WhereClause function construct proper SQL statement from given statement and list of conditions.

136 AcquisitionEra does not exist in DBS.

128 block xxx already exists in DBS.

106 db commit error.

MigrationCodes represents all migration codes.

117 content type error.

101 database error.

139 DatasetAccessType does not exist in DBS.

140 Dataset does not exist in DBS.

131 DatasetParent does not exist in DBS.

137 DataTier does not exist in DBS.

115 decode error.

116 encode error.

MigrationCodes represents all migration codes.

MigrationCodes represents all migration codes.

129 FileDataTypes does not exist in DBS.

130 FileParent does not exist in DBS.

generic DBS error.

109 get id db error.

124 HTTP request error.

MigrationCodes represents all migration codes.

110 db insert error.

127 invalid request error.

last available DBS error code.

112 db last insert error.

108 loading error, e.g.

123 marshal error.

125 Migration error.

119 not implemented API error.

118 parameters error.

107 parser error.

114 pattern error.

MigrationCodes represents all migration codes.

138 PhysicsGroup does not exist in DBS.

134 PrimaryDataset does not exist in DBS.

133 PrimaryDatasetType does not exist in DBS.

132 ProcessedDataset does not exist in DBS.

135 ProcessingEra does not exist in DBS.

103 query error.

MigrationCodes represents all migration codes.

120 io reader error.

126 remove error.

104 row scan error.

105 db session error.

MigrationCodes represents all migration codes.

102 transaction error.

122 json unmarshal error.

111 update error.

113 validation error.

121 io writer error.

ApiParameterFile represents API parameter file.

ApiParamMap an object which holds API parameter records.

Cert represents DBS X509 cert used by HttpClient.

Ckey represents DBS X509 key used by HttpClient.

ConcurrencyErr represents generic concurrency error.

ConcurrentBulkBlocks defines if code should use concurrent bulkblocks API.

ContentTypeErr represents generic content-type error.

DatabaseErr represents generic database error.

DB represents sql DB pointer.

DBOWNER represents DBS DB owner.

DBSQL represents DBS SQL record.

DBTYPE represents DBS DB type, e.g.

DRYRUN allows to skip query execution and printout DB statements along with passed parameters.

FileChunkSize controls size of chunk for []File insertion.

FileLumiChunkSize controls chunk size for FileLumi list insertion.

FileLumiInsertMethod controls which method to use for insertion of FileLumi list.

FileLumiMaxSize controls max size for FileLumi list insertion.

GenericErr represents generic dbs error.

InvalidParamErr represents generic error for invalid input parameter.

InvalidRequestErr represents generic invalid request error.

LexiconPatterns represents CMS Lexicon patterns.

MigrateURL holds URL of DBSMigrate server.

MigrationAsyncTimeout defines timeout of asynchrounous migration request process.

MigrationCleanupInterval defines migration cleanup server interval.

MigrationCleanupOffset defines offset in seconds to delete migration requests.

MigrationDB points to migration DB.

MigrationProcessTimeout defines migration process timeout.

MigrationRetries specifies total number of migration retries.

MigrationServerInterval defines migration process timeout.

NotImplementedApiErr represents generic not implemented api error.

RecordErr represents generic record error.

RecordValidator pointer to validator Validate method.

Timeout represents DBS timeout used by HttpClient.

TlsRefreshInterval represents refresh interval for Tls proxy.

TotalCompleted represents total number of completed migration requests.

TotalExistInDB represents total number of exist in db migration requests.

TotalFailed represents total number of failed migration requests.

TotalInProgress represents total number in progress migration requests.

TotalMigrationRequests counts total number of migration requests processed by this server.

TotalPending represents total number pending migration requests.

TotalQueued represents total number of queued migration requests.

TotalTermFailed represents total number of terminally failed migration requests.

ValidationErr represents generic validation error.

AcquisitionEra represents AcquisitionEra structure use in BulkBlocks structure.

AcquisitionEras represents Acquisition Eras DBS DB table.

API structure represents DBS API.

ApiParameters rerepresents a API parameters record.

ApplicationExecutables structure describe associative table in DBS DB.

Block represents Block structure used in BulkBlocks structure.

BlockDumpRecord represents input block record used in BlockDump and InsertBlockDump APIs.

BlockParent represents block parent structure used in BulkBlocks structure.

BlockParents structure represents block parents table in DBS DB.

BlockRecord represents input record for insert blocks API.

BlockResponse represents block response structure used in GetParentBlocks.

Blocks represents Blocks DBS DB table.

BranchHashes represents Branch Hashes DBS DB table.

BulkBlocks represents bulk block structure used by `/bulkblocks` DBS API.

Dataset represents dataset structure used in BulkBlocks structure.

DatasetAccessTypes represents Dataset Access Types DBS DB table.

DatasetConfig represents dataset config structure used in BulkBlocks structure.

DatasetOutputModConfigs represents dataset output mod configs DBS DB table.

DatasetParent represents dataset parent structure used in BulkBlocks structure.

DatasetParents represents Dataset Parents DBS DB table.

DatasetRecord we receive for InsertDatasets API.

DatasetResponse represents response of processDatasetBlocks API.

Datasets represents Datasets DBS DB table.

DatasetShortRecord represents short dataset record.

DataTiers represents data tiers DBS DB table.

DBInfo represents entire database information.

DBSError represents common structure for DBS errors.

File represents file structure used in BulkBlocks structure.

FileConfig represents file config structure used in BulkBlocks structure.

FileDataTypes represents File Data Types DBS DB table.

FileLumi represents file lumi structure used in File structure of BulkBlocks structure.

FileLumis represents File Lumis DBS DB table.

FileOutputModConfigRecord represents file output mod config input record.

FileOutputModConfigs represents file output mod config DBS DB table.

FileParentBlockRecord represents file parent DBS record BlockName: name of the block ChildParentIDList: list of child and parent file ids MissingFiles: Number of missing files in the child and parent file list.

FileParentLFNRecord represents file parent recoord supplied in file parent list of FileRecord.

FileParentRecord represents file parent DBS record used by bulkblocks API NOTE: bulkblocks API should return this_logical_file_name as it is used by DBS migrate while users, e.g.

FileParents represents file parents DBS DB table.

FileRecord represents input record for insert files API.

Files represents Files DBS DB table.

Lexicon represents single lexicon pattern structure.

LexiconPattern represents single lexicon compiled pattern structure.

MigrationBlock represent block with migration order.

MigrationBlocks represents migration blocks table.

MigrationRemoveRequest represents migration remove request object.

MigrationReport represents migration report returned by the migration API.

MigrationRequest represent MigrationRequest table.

MigrationStatusRequest defines status request structure.

OutputConfigRecord represents input to InsertOutputConfigs API.

OutputConfigs represents Output Configs DBS DB table.

ParameterSetHashes represents Parameter Set Hashes DBS DB table.

PhysicsGroups represents Physics Groups DBS DB table.

PrimaryDataset represents primary dataset structure used in BulkBlocks structure.

PrimaryDatasetRecord represents primary dataset record.

PrimaryDatasets represents Primary Datasets DBS DB table.

PrimaryDSTypes represents primary ds types DBS DB table.

ProcessedDatasets represents Processed Datasets DBS DB table.

ProcessingEra represents processing era structure used in BulkBlocks structure.

ProcessingEras represents Processing Eras DBS DB table.

PyFileRecord represents DBS python input file record structure.

ReleaseVersions represents Relases Versions DBS DB table.

SchemaIndex represents schema index details.

SchemaInfo represents schema details.

StrPattern represents string object pattern.

TableIndex represents individual table index sizes.

TableInfo represents individual table info.

TempFileRecord contains all relevant attribute to insert File records.

TLSCertsManager manages TLS certificates.

DBRecord interface represents general DB record used by DBS APIs.

ObjectPattern represents interface to check different objects.

ApiParametersMap represents data type of api parameters.

BlockParentList represents BlockParent records.

DatasetConfigList represents DatasetConfig records.

DatasetParentList represents list of dataset parents.

FileConfigList represents FileConfig records.

FileList represents list of File records.

FileParentList represents FileParent records.

Record represents DBS record.