Categorygithub.com/Laisky/go-utils/v2
modulepackage
2.3.0
Repository: https://github.com/laisky/go-utils.git
Documentation: pkg.go.dev
Overview
Versions
5
Dependencies
46
Dependents
10
Files
2.6k SLOC

# README

Go-Utils

Many useful golang tools

VersionSupport Go
1.x>= v1.16
2.x>= v1.18
3.x>= v1.19

License: MIT Commitizen friendly Go Report Card GoDoc Build Status codecov

Use as command-line executable binary

Install

go install github.com/Laisky/go-utils/v2/cmd/gutils@latest

Add $HOME/.go/bin to your PATH.

Usage

# find and delete duplicate files/ similar images
gutils remove-dup examples/images --dry

# move files to hash-based hierach directories
gutils md5dir -i examples/md5dir/ --dry

# generate tls cert
gutils gentls --host 1.2.3.4

# encrypt by aes
gutils encrypt aes -i <file_path> -s <password>

# sign or verify by rsa
gutils rsa sign
gutils rsa verify

Use as SDK

Install

go get github.com/Laisky/go-utils/v2@latest

Usage

import (
    gutils "github.com/Laisky/go-utils/v2"
)

Modules

Contains some useful tools in different directories:

  • settings: move go github.com/Laisky/go-config
  • color.go: colorful code
  • compressor.go: compress and extract dir/files
  • email/: SMTP email sdk
  • encrypt/: some tools for encrypt and decrypt, support AES, RSA, ECDSA, MD5, SHA128, SHA256
    • configserver.go: load configs from file or config-server
  • fs.go: some tools to read, move, walk dir/files
  • http.go: some tools to send http request
  • jwt/: some tools to generate and parse JWT
  • log/: enhanched zap logger
  • math.go: some math tools to deal with int, round
  • net.go: some tools to deal with tcp/udp
  • random.go: generate random string, int
  • sort.go: easier to sort
  • sync.go: some locks depends on atomic
  • throttle.go: faster rate limiter
  • time.go: faster clock (if you do not enable vdso)
  • utils: some useful tools

# Packages

Package algorithm contains some useful algorithms.
Package cmd some useful tools for command argument.
Package compress contains some useful tools to compress/decompress data or files.
Package counter contains varias counter tools.
Package email simple email sender.
Package encrypt contains some useful tools to deal with encryption/decryption.
Package gorm some useful tools for gorm.
Package jwt all in one JWT sdk.
Package log enhanced zap logger.
No description provided by the author

# Functions

AbsInt32 abs(v) ignore int exceeds limit error, abs(MinInt32) == MaxInt32.
AbsInt64 abs(v) ignore int exceeds limit error, abs(MinInt64) == MaxInt64.
AutoGC auto trigger GC when memory usage exceeds the custom ration default to /sys/fs/cgroup/memory/memory.limit_in_bytes.
Base64Decode decode string to bytes use base64.
Base64Encode encode bytes to string use base64.
Bytes2Str unsafe convert bytes to str.
CheckResp check HTTP response's status code and return the error with body message.
Color wrap with ANSI color inspired by github.com/fatih/color.
Contains if collection contains ele.
ConvertMap2StringKey convert any map to `map[string]any`.
CopyFile copy file content from src to dst.
CostSecs convert duration to string like `0.25s`.
Dedent removes leading whitespace or tab from the beginning of each line will replace all tab to 4 blanks.
DeepClone deep clone a struct will ignore all unexported fields.
DirSize calculate directory size inspired by https://stackoverflow.com/a/32482941/2368737.
EmptyAllChans receive all thins in all chans.
FallBack return the fallback when orig got error utils.FallBack(func() any { return getIOStatMetric(fs) }, &IOStat{}).(*IOStat).
FileExists is path a valid file.
FileMD5 read file and calculate MD5.
FileSHA1 read file and calculate sha1 return hashed string in 40 bytes.
FlattenMap make embedded map into flatten map.
ForceGCBlocking force to run blocking manual gc.
ForceGCUnBlocking trigger GC unblocking.
GetFuncName return the name of func.
GetStructFieldByName get struct field by name.
GracefulCancel is a function that will be called when the process is about to be terminated.
HasField check is struct has field inspired by https://mrwaggel.be/post/golang-reflect-if-initialized-struct-has-member-method-or-fields/.
HasMethod check is struct has method inspired by https://mrwaggel.be/post/golang-reflect-if-initialized-struct-has-member-method-or-fields/.
HTTPInvalidStatusError return error about status code.
HumanReadableByteCount convert bytes to human readable string Args: - bytes: - si: `si ? 1024 : 1000` Example: `HumanReadableByteCount(1005, false) -> "1.01KB"`.
InArray if collection contains ele Deprecated: use Contains instead.
IntersectSortedChans return the intersection of multiple sorted chans.
IsDir is path exists as dir.
IsDirWritable if dir is writable.
IsEmpty is empty.
IsFile is path exists as file.
IsFileATimeChanged check is file's atime equal to expectATime.
IsPanic is `f()` throw panic.
IsPtr check if t is pointer.
IsRemoteUDPPortOpen check is remote udp port open.
JumpHash fatest consistent hashing created by google.
ListFilesInDir list files in dir.
Max return the maximal value.
MD5JSON calculate md5(jsonify(data)).
Min return the minimal value.
MoveFile move file from src to dst by copy sometimes move file by `rename` not work.
NewTask new async task.
NewClock create new Clock.
NewExpCache new cache manager.
NewExpiredRLock new ExpiredRLock.
NewFlock new file lock.
NewGoroutineTest new test for goroutine any fail will call cancel().
NewHTTPClient create http client.
NewLRUExpiredMap new ExpiredMap.
NewMutex create new mutex.
NewPermissionTree new permission tree only contains root node.
NewRand new individual random to aviod global mutex.
NewSingleItemExpCache new expcache contains single data.
NewThrottleWithCtx create new Throttle 90x faster than `rate.NewLimiter`.
NewTmpFileForContent write content to tmp file and return path.
NilInterface make sure data is nil interface or another type with nil value Example: type foo struct{} var f *foo var v any v = f v == nil // false NilInterface(v) // true.
NotEmpty val should not be empty, with pretty error msg.
OptionalVal return optionval if not empty.
Overwrite overwrite file if target existed.
PanicIfErr panic if err is not nil.
ParseHex2UTC parse hex to UTC time.
ParseHexNano2UTC parse hex contains nano to UTC time.
ParseUnix2String can parse unix timestamp(int64) to string.
ParseUnix2UTC convert unix to UTC time.
ParseUnixNano2UTC convert unixnano to UTC time.
Pipeline run f(v) for all funcs.
PrettyBuildInfo get build info in formatted json Print: { "Path": "github.com/Laisky/go-ramjet", "Version": "v0.0.0-20220718014224-2b10e57735f1", "Sum": "h1:08Ty2gR+Xxz0B3djHVuV71boW4lpNdQ9hFn4ZIGrhec=", "Replace": null }.
Race return when any goroutine returned Deprecated: use RaceErr instead.
RaceErr return when any goroutine returned.
RaceErrWithCtx return when any goroutine returned or ctx canceled.
RandomBytesWithLength generate random bytes.
RandomStringWithLength generate random string with specific length.
Recursive list files recursively.
RegexNamedSubMatch extract key:val map from string by group match.
RemoveEmpty remove duplicate string in slice.
RenderTemplate render template with args.
RenderTemplateFile render template file with args.
RequestJSON request JSON and return JSON by default client.
RequestJSONWithClient request JSON and return JSON with specific client.
Round round float64 Round(1.005, 2) -> 1.01.
RunCMD run command script.
RunCMDWithEnv run command with environments # Args - envs: []string{"FOO=BAR"}.
RunWithTimeout run func with timeout.
SecRandInt generate security int.
SecRandomBytesWithLength generate crypto random bytes.
SecRandomStringWithLength generate random string with specific length.
SetInternalClock set internal Clock with refresh interval.
SetStructFieldsBySlice set field value of structs slice by values slice.
SilentClose close and ignore error Example defer SilentClose(fp).
SilentFlush flush and ignore error.
SleepWithContext sleep duration with context, if context is done, return.
SortBiggest sort from biggest to smallest.
SortSmallest sort from smallest to biggest.
StopSignal registered for SIGTERM and SIGINT.
Str2Bytes unsafe convert str to bytes.
TemplateWithMap replace `${var}` in template string.
TemplateWithMapAndRegexp replace `${var}` in template string.
TrimEleSpaceAndRemoveEmpty remove duplicate string in slice.
UniqueStrings remove duplicate string in slice.
URLMasking masking password in url.
UTCNow get current time in utc.
ValidateFileHash validate file content with hashed string Args: - filepath: file path to check - hashed: hashed string, like `sha256: xxxx`.
WatchFileChanging watch file changing when file changed, callback will be called, callback will only received fsnotify.Write no matter what happened to changing a file.
WithFileFlag how to write dst file.
WithFileMode if create new dst file, set the file's mode.
WithGCMemLimitFilePath set memory limit file.
WithGCMemRatio set mem ratio trigger for GC default to 85.
WithHTTPClientInsecure set http client igonre ssl issue default to false.
WithHTTPClientMaxConn set http client max connection default to 20.
WithHTTPClientTimeout set http client timeout default to 30s.
WithReplaceTabBySpaces replace tab to spaces.
WithStopSignalCloseSignals set signals that will trigger close.

# Constants

Background text colors.
Background text colors.
Background text colors.
Background text colors.
Background Hi-Intensity text colors.
Background Hi-Intensity text colors.
Background Hi-Intensity text colors.
Background Hi-Intensity text colors.
Background Hi-Intensity text colors.
Background Hi-Intensity text colors.
Background Hi-Intensity text colors.
Background Hi-Intensity text colors.
Background text colors.
Background text colors.
Background text colors.
Background text colors.
Base attributes.
Base attributes.
Base attributes.
Base attributes.
Base attributes.
ANSIColorEscape escape string for ANSI color.
Base attributes.
Foreground text colors.
Foreground text colors.
Foreground text colors.
Foreground text colors.
Foreground Hi-Intensity text colors.
Foreground Hi-Intensity text colors.
Foreground Hi-Intensity text colors.
Foreground Hi-Intensity text colors.
Foreground Hi-Intensity text colors.
Foreground Hi-Intensity text colors.
Foreground Hi-Intensity text colors.
Foreground Hi-Intensity text colors.
Foreground text colors.
Foreground text colors.
Foreground text colors.
Foreground text colors.
Base attributes.
Base attributes.
Base attributes.
Base attributes.
AsyncTaskStatusDone task done.
AsyncTaskStatusFailed task failed.
AsyncTaskStatusPending task pending.
BaseHex 16.
BitSize64 64.
HTTPHeaderContentType HTTP header name.
HTTPHeaderContentTypeValJSON HTTP header value.
HTTPHeaderHost HTTP header name.
HTTPHeaderReferer HTTP header name.
Nano2Sec 1e9.
TimeFormatDate "2006-01-02".

# Variables

Clock high performance time utils, replace Clock1.
ErrAsyncTask root error for async tasks.
ForceGC force to start gc blocking.
JSON effective json.
ParseTs2String can parse unix timestamp(int64) to string.
ParseTs2Time can parse unix timestamp(int64) to time.Time.
ParseTs2UTC can parse unix timestamp(int64) to time.Time.
TimeZoneShanghai timezone Shanghai TimeZoneShanghai = time.FixedZone("Asia/Shanghai", 8*3600).
TimeZoneUTC timezone UTC.
TriggerGC force to start gc unblocking.
UnixNano2UTC convert unixnano to UTC time.

# Structs

AsyncTaskResult result of async task.
ClockT high performance ClockT with lazy refreshing.
ExpCache cache with expires can Store/Load like map.
ExpiredRLock Lock with expire time.
GoroutineTest testing.T support goroutine.
LRUExpiredMap map with expire time, auto delete expired item.
Mutex mutex that support unblocking lock.
RBACPermissionElem element node of permission tree the whole permission tree can represented by the head node.
RequestData http request.
SingleItemExpCache single item with expires.
Throttle current limitor.
ThrottleCfg Throttle's configuration.

# Interfaces

asyncTask async task.
AsyncTaskStore persistency storage for async task.
ClockItf high performance lazy clock.
FLock lock by file.
Number is a number type.
Sortable Data types that can be compared by >, <, ==.
SortItemItf interface of sort item.

# Type aliases

AsyncTaskStatus status of async task.
CopyFileOptionFunc set options for copy file.
DedentOptFunc dedent option.
GcOptFunc option for GC utils.
HTTPClientOptFunc http client options.
ListFilesInDirOptionFunc options for ListFilesInDir.
PairList array of sort items.
RBACPermFullKey key with ancesters.
RBACPermKey permission identity keyword format like `a`.
StopSignalOptFunc options for StopSignal.