What is transformer

We can apply stateless transformation on our inflight data. This is based on our data model

Each batch of changes can be transformed into new batch of changes:

How to add new transformer

1. Create new package
2. Implemenet abstract.Transformer interface
3. Add proto mesage for transformer config
4. Register implementation

Example:

Implementation of abstract.Transformer

type DummyTransformer struct {
}

func (r *DummyTransformer) Apply(input []abstract.ChangeItem) abstract.TransformerResult {
	return abstract.TransformerResult{
		Transformed: input,
		Errors:      nil,
	}
}

func (r *DummyTransformer) Suitable(table abstract.TableID, schema abstract.TableColumns) bool {
	return true
}

func (r *DummyTransformer) ResultSchema(original abstract.TableColumns) abstract.TableColumns {
	return original
}

func (r *DummyTransformer) Description() string {
	return "this transformer do nothing"
}

Proto interface

message DummyTransformer {
	// here some params can be placed
}

Registration

func init() {
	transformer.Register(new(api.DummyTransformer), func(protoConfig any, lgr log.Logger) (abstract.Transformer, error) {
		return &DummyTransformer{}, nil
	})
}

Prefillment on transfer create

If you want your transformers to prefill on transfer create, you should insert your typeswitch in this function called prefillTransformationDefaults in transfer_service.go: https://a.yandex-team.ru/arcadia/transfer_manager/go/pkg/controlplane/transfer_service.go?rev=r11799915#L127