package
8.2.0+incompatible
Repository: https://github.com/pingcap/tiflow.git
Documentation: pkg.go.dev

# README

Structure of Master

Structure

The master module consists of several components:

  • Master Server.
    • provide Grpc/http api, including
      • operate cluster
        • register/delete executor server
        • handle heartbeat (keep alive)
      • user interface
        • submit job
        • query job(s)
        • cancel job
      • schedule and failover jobs
  • ExecutorManager
    • Handle Heartbeat
      • Notify Resource Manager updates the status infos
    • Maintain the aliveness of Executors.
      • Check liveness
        • Executor Manager check every executor whether the heartbeat has been timeout.
        • Once an executor is offline by heartbeat timeout, it should notify the job manager to reschedule all the tasks on it.
  • Executor Client
    • Embeds two independent interfaces
      • ExecutorServiceClient, which is used to dispatch task to executor.
      • BrokerServiceClient, which is used to manage resource belongs to an executor.
  • JobManager
    • Receive SubmitJob Request, Check the type of Job, Create the JobMaster.
    • JobMaster (per job)
      • Generate the Tasks for the job
      • Schedule and Dispatch Tasks

# Packages

No description provided by the author
No description provided by the author
No description provided by the author

# Functions

GetDefaultMasterConfig returns a default master config.
NewDefaultBusinessMetaConfig return the default business metastore config.
NewExecutorManagerImpl creates a new ExecutorManagerImpl instance.
NewJobFsm creates a new job fsm.
NewJobManagerImpl creates a new JobManagerImpl instance.
NewMetaStoreManager creates a new metaStoreManagerImpl instance.
NewServer creates a new master-server.

# Constants

DefaultBusinessMetaID is the ID for default business metastore.
FrameMetaID is the ID for frame metastore.

# Structs

Config is the configuration for server-master.
Executor records the status of an executor instance.
ExecutorManagerImpl holds all the executors' info, including liveness, status, resource usage.
JobFsm manages state of all job masters, job master state forms a finite-state machine.
JobHolder holds job meta and worker handle for a job.
JobManagerImpl is a special job master that manages all the job masters, and notify the offline executor to them.
Server handles PRC requests for df master.

# Interfaces

ExecutorManager defines an interface to manager all executors.
JobManager defines manager of job master.
JobStats defines a statistics interface for JobFsm.
MetaStoreManager defines an interface to manage metastore.