gocraft/work v2

Please see cmd/ for enqueuer and worker demo.

Improvements

• queue backend abstraction
  • redis is still the default, but the new design allows custom queue implementation.
• simplify the keyspace design of redis queue backend
  • The new design uses 1 redis hash per job, and 1 redis sorted set for queue.
  • Interesting read
• modular
  • The core only catches panics, retries on failure, and waits if a queue is empty.
  • All other functionalities are either removed or moved to separate middlewares.
• support binary payload/args with message pack.
• replace built-in UI with prometheus metrics (use grafana if you want dashboard).
• additional optimizations (alloc + bulk queue ops)

  BenchmarkWorkerRunJob/work_v1_1-8         	    3000	    515957 ns/op
  BenchmarkWorkerRunJob/work_v2_1-8         	    5000	    284516 ns/op
  BenchmarkWorkerRunJob/work_v1_10-8        	    1000	   2136546 ns/op
  BenchmarkWorkerRunJob/work_v2_10-8        	    5000	    367997 ns/op
  BenchmarkWorkerRunJob/work_v1_100-8       	     100	  18234023 ns/op
  BenchmarkWorkerRunJob/work_v2_100-8       	    1000	   1759186 ns/op
  BenchmarkWorkerRunJob/work_v1_1000-8      	      10	 162110100 ns/op
  BenchmarkWorkerRunJob/work_v2_1000-8      	     100	  12646080 ns/op
  BenchmarkWorkerRunJob/work_v1_10000-8     	       1	1691287122 ns/op
  BenchmarkWorkerRunJob/work_v2_10000-8     	      10	 144923087 ns/op
  BenchmarkWorkerRunJob/work_v1_100000-8    	       1	17515722574 ns/op
  BenchmarkWorkerRunJob/work_v2_100000-8    	       1	1502468637 ns/op
  PASS
  ok  	github.com/taylorchu/work	87.901s
  

• http server
  • delete job
  • create job
  • get job status
  • get queue metrics (kubernetes autoscaler integration with keda metrics api scaler)
  • OpenAPI spec

License