package
2.6.9+incompatible
Repository: https://github.com/oneconcern/datamon.git
Documentation: pkg.go.dev
# Functions
BundleContributor defines a single contributor for a bundle descriptor.
BundleContributors defines the list of contributors for a bundle descriptor.
Deduplication defines the deduplication scheme for a bundle descriptor.
DiamondClone clones from a DiamondDescriptor.
DiamondID sets the DiamondID of a DiamondDescriptor.
DiamondMode sets the conflict resolution mode for a diamond.
DiamondTag sets an informative tag on the diamond.
GenerateCheckpointPath builds a path in the dataset to save checkpointed files.
GenerateConflictPath builds a path in the dataset to save conflicting files.
GetArchivePathComponents yields all metadata components from a parsed archive path.
GetArchivePathPrefixToBundles yields a path to all bundles in a repo.
GetArchivePathPrefixToContexts returns the path to the list of contexts.
GetArchivePathPrefixToDiamonds yields a path to all diamonds in a repo.
GetArchivePathPrefixToLabels yields the path to labels in a repo, given some prefixes.
GetArchivePathPrefixToRepos yields the path to all repos.
GetArchivePathPrefixToSplits yields a path to all splits in a diamond in a repo.
GetArchivePathToBundle yields a path in a repo to the descriptor of a bundle
Example:
bundles/{repo}/{bundleID}/bundle.yaml.
GetArchivePathToBundleFileList yields a path to the list of the files in a bundle
Example:
bundles/{repo}/{bundleID}/bundle-files-{index}.yaml.
GetArchivePathToDiamond yields a path in a repo to the descriptor of a diamond in any state.
GetArchivePathToFinalDiamond yields a path in a repo to the descriptor of a diamond in a final state.
GetArchivePathToFinalSplit yields a path in a repo to the descriptor of a split in a final state.
GetArchivePathToInitialDiamond yields a path in a repo to the descriptor of a diamond in an initialized state.
GetArchivePathToInitialSplit yields a path in a repo to the descriptor of a split in a running state.
GetArchivePathToLabel gets the path to the label descriptor.
GetArchivePathToRepoDescriptor yields a path in a repo to the descriptor of a bundle
Example:
repos/{repo}/{bundleID}/repo.yaml.
GetArchivePathToSplit yields a path in a repo to the descriptor of a split in any state.
GetArchivePathToSplitFileList yields a path to the list of the files in a diamond split
Example:
diamonds/{repo}/{diamond}/splits/{split}/{generation}/bundle-files-{index}.yaml.
GetBlobName yields the name of the Blob store.
GetBundleTimeStamp yields the current UTC time.
GetConsumablePathToBundle yields a path to some bundle metadata.
GetConsumablePathToBundleFileList yields a path to some bundle file.
GetConsumableStorePathMetadata is the inverse of GetConsumablePath* functions.
GetMetadataName yields the name of the Metadata store.
GetPathToCategory returns the path to the descriptor for a Category.
GetPathToContainer returns the path to the descriptor Container for a container part of a stage in a run.
GetPathToContext returns the path to the context descriptor.
GetPathToDataSetIn returns the path to the descriptor DataSetIn for a container part of a stage in a run.
GetPathToDataSetOut returns the path to the descriptor DataSetOut for a container part of a stage in a run.
GetPathToRun returns the path to the descriptor for a Run.
GetPathToRunStatus returns the path to the descriptor RunStatus for a run.
GetReadLogName yields the name of the Read Log store.
GetVMetadataName yields the name of the Versioning Metadata store.
GetWALName yields the name of the Write Ahead Log store.
IsGeneratedFile indicates if some file comes from auto-generation (e.g.
LabelContributor sets a single contributor for the label.
LabelContributors sets a list of contributors for the label.
LabelName sets a name for the label.
MarshalContext marshals a context as a YAML descriptor.
MarshalWAL marshals a WAL entry as a YAML descriptor.
Message defines the message of the bundle descriptor.
NewBundleDescriptor builds a new default bundle descriptor.
NewDiamondDescriptor builds a new DiamondDescriptor.
NewEntry creates a new entry for the WAL.
NewLabelDescriptor builds a new label descriptor.
NewSplitDescriptor builds a new SplitDescriptor.
Parents defines the parents for a bundle descriptor.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
SplitClone clones from a SplitDescriptor.
SplitContributor defines a single contributor for a SplitDescriptor.
SplitContributors defines the list of contributors for a SplitDescriptor.
SplitID sets the splitID of a SplitDescriptor.
SplitTag sets an informative tag on the split.
UnmarshalContext unmarshals a context from a YAML descriptor.
UnmarshalWAL unmarshals a WAL entry from a YAML descriptor.
ValidateContext checks the context is valid, i.e.
ValidateLabel validates a label descriptor.
ValidateRepo validates a repository descriptor: name and description are required, name only contains letters, digits or '-'.
# Constants
ConsumableStorePathTypeDescriptor defines consumable store metadata of type "descriptor".
ConsumableStorePathTypeFileList defines consumable store metadata of type "file list".
ContextVersion indicates the version of the context model.
CurrentBundleVersion indicates the version of the bundle model
Note that version numbering is an integer, not a semver string.
DiamondCanceled indicates the diamond has completed with a cancel.
DiamondDone indicates the diamond has completed with a successful commit.
DiamondInitialized is the state of an initialized diamond.
EnableCheckpoints is the diamond mode with which conflicts are explicitly handled and saved as "checkpoints" (incremental upload).
EnableConflicts is the diamond mode with which conflicts are detected and saved as "conflicts".
ForbidConflicts is the diamond mode with which conflicts result in failure to commit.
IgnoreConflicts is the diamond mode with which conflicts are not handled (latest win, no track kept of clobbered files).
SplitDone is the state of a completed split.
SplitRunning is the state of running split.
TokenGeneratorPath is the path to the WAL token generator.
No description provided by the author
# Structs
ArchivePathComponents defines the unique path parts to retrieve a file in a bundle.
BundleDescriptor represents a commit which is a file tree with the changes to the repository.
BundleEntries are the files in this bundle.
BundleEntry describes a file in the bundle.
Category of run declares the kind of run it is.
ConsumableStorePathMetadata defines the metadata associated to a consumable store.
ConsumableStorePathMetadataErr defines errors related to consumable store metadata.
Container describes the details for a container in a stage.
Context defines the details for a datamon context.
Contributor who created the object.
DataSetIn describes the details for the data set that went into a container.
DataSetOut describes the details for the data set generated by a container.
DiamondDescriptor models a diamond's metadata.
Entry defines a Write Ahead Log entry.
FS captures the details for the filesystem mount of datasets.
LabelDescriptor describes a label.
RepoDescriptor represents a commit which is a file tree with the changes to the repository.
Run is an unit of computation that is executed.
RunEngine describes the details for the type of Run being executed and a runfile describing the run.
RunStatus records the final status for a run.
SplitDescriptor models the metadata about a given split within a diamond.
# Type aliases
BundleDescriptorOption is a functor to build a bundle descriptor with some options.
BundleDescriptors is a sortable slice of BundleDescriptor.
ConflictMode indicates the conflict detection mode defined for a diamond.
DiamondDescriptorOption defines an option to build a DiamondDescriptor.
DiamondDescriptors is a sortable slice of DiamondDescriptor.
DiamondState models the running status of an ungoing diamond workflow.
LabelDescriptorOption is a functor to build label descriptors.
LabelDescriptors is a sortable slice of LabelDescriptor.
RepoDescriptors is a sortable slice of RepoDescriptor.
SplitDescriptorOption defines an option to build a SplitDescriptor.
SplitDescriptors is a sortable slice of SplitDescriptor.
SplitState models the running status of an ungoing diamond split.