Simple Data Service - shared library

This package presents code to be used by each data resource in order to have a standard API used by every service.

Code Organisation

The following approach is used:

1. Application domain types go in the root - api.go, server.go, util.go, etc
2. Implementation of the application domain go in subpackages - badger, sqlite, etc
3. Everything is tied together in the cmd subpackages - cmd/shared

There is no internal package here as there need be no hiding of code from implementation code.

Usage

Add to project

go get github.com/rhythmicsound/sdsshared

Then you can create a new service by picking your connector and running something like:

package main

import (
	"fmt"
	"log"

	sdsshared "github.com/RhythmicSound/sds-shared"
	badgerconnector "github.com/RhythmicSound/sds-shared/badgerConnector"
)

func main() {

	connector := badgerconnector.New(sdsshared.ResourceServiceName, sdsshared.DatasetURI)

	log.Fatalln(sdsshared.StartServer(connector, "", 0))
}

Using default type values for the arguments to StartServer allows service name and ports to be set using environment variables at runtime.

An example execute command is:

debug=false \
name="postcodeUK-Service" \
database_uri="working/databases/postcodeUKdb" \
dataset_uri="https://storage.cloud.google.com/simple-data-service/datasets/postcodesUK.zip" \ 
objectname="datasets/postcodesUK.zip" \
go run cmd/dummy.go

Settings

Settings for services created from this library can be hardcoded or set using environment variables

Writing new backend storage connectors

Implement DataResource interface

Example:

type impl struct{
	ResourceName string
	Database     *badger.DB
	versioner    sdsshared.VersionManager
}

func NewImpl(resourceName, datasetArchiveLocation string) *impl{
  return &impl{
    	ResourceName: resourceName,
		versioner: sdsshared.VersionManager{
			Repo:           sdsshared.DatasetURI,
			LastUpdated:    "",
			CurrentVersion: 0,
			DataSources:    make([]string, 0),
		},
  }
}

(im impl) Startup()error{
  ... //Startup scripts
}

vt := &VersionManager{
    Repo = repoURIAddress
}

(im impl) UpdateDataset(vt)(*VersionManager,error){
  ... // Logic to keep the database synced with a master versioned dataset archive somehwere

    //Passing a VersionManager as an arg should overwrite internal VM created in New. 
    //Must accept nil to use default
}

(im impl) Retrieve(someSearchString)(sdsshared.SimpleData,error){
   ... //Fetching data from the database based on given key value. 
    //Use the latest VersionManager to complete the Meta elements of SimpleData response object
}

(im impl) Shutdown()error{
  ... //Shutdown scripts
}

See the badgerConnector package for best practise