log

• wrapper of zap

• wrapper of logrus

• middleware tracing log for Echo, Gin, or net/http (Gorilla mux, Go-chi)

  • useful for Financial Products (to comply with PCI-DSS standards) and Healthcare (to comply with HIPAA regulations)

• You can refer to middleware-log-tracing at my Linked In for more details.

A typical micro service

• When you zoom one micro service, the flow is as below, and you can see "log (zap, logrus)" and "middleware" in the full picture:

Content for logging

Request

Features

• Log Request Method and URL: Log the HTTP method (GET, POST, etc.) and the requested URL.
• Log Request Headers: Option to log request headers for debugging purposes.
• Log Request Body: Option to log the request body (with configurable size limits to avoid logging large payloads).

Benefits

• Debugging: Helps in tracing and debugging issues by providing complete information about incoming requests.
• Monitoring: Provides visibility into the types of requests being received.

Response

Features

• Log Response Status Code: Log the HTTP status code of the response.
• Log Response Headers: Option to log response headers.
• Log Response Body: Option to log the response body (with configurable size limits to avoid logging large payloads).

Benefits

• Debugging: Assists in diagnosing issues by providing complete information about the responses sent by the server.
• Auditing: Helps in auditing and reviewing server responses for compliance and monitoring purposes.

Response Time

Features

• Log Response Time: Calculate and log the time taken to process each request.

Benefits

• Performance Monitoring: Helps in identifying slow requests and performance bottlenecks.
• Optimization: Provides data to optimize and improve server response times.

Response Size

Features

• Log Response Size: Log the size of the response payload in bytes.

Benefits

• Bandwidth Monitoring: Helps in monitoring and managing bandwidth usage.
• Optimization: Provides insights into the response sizes to optimize payloads and improve performance.

Features

Middleware Integration

Features

• Middleware Function: Designed to integrate seamlessly with existing Go libraries: Echo, Gin, or net/http (Gorilla mux, Go-chi).
  • Sample for Echo is at go-echo-sql-sample
  • Sample for Gin is at go-gin-sql-sample
  • Sample for Gorilla mux is at go-sql-sample
• Context Handling: Pass context to handle request-specific data throughout the middleware chain.

Benefits

• Ease of Use: Simplifies the integration of logging into existing web applications.
• Consistency: Ensures consistent logging across different parts of the application.

Logging Libraries Integration

• Do not depend on any logging libraries.
• Already supported to integrate with zap, logrus
• Can be integrated with any logging library.

Enable/Disable Logging

Features

• Enable/Disable Logging: Allow users to turn on or off logging for requests, responses, headers, and bodies independently.
• Logging Levels: Support different logging levels (e.g., INFO, DEBUG, ERROR) to control the verbosity of logs.

Benefits

• Flexibility: Provides users with the flexibility to configure logging based on their needs and environment.
• Efficiency: Reduces overhead by allowing selective logging, especially in production environments.

Asynchronous Logging

Features

• Non-Blocking Logs: Implement asynchronous logging to ensure that logging does not block request processing.
• Log Buffering: Use buffering to improve logging performance and reduce latency.

Benefits:

• Performance: Improves the overall performance of the application by reducing logging overhead.
• Scalability: Allows the application to handle high-throughput logging without performance degradation.

Sensitive Data Encryption

Features

• Mask/Encrypt sensitive data in the request and response bodies.
  • Sample for Echo is at go-echo-sql-sample
  • Sample for Gin is at go-gin-sql-sample
  • Sample for Gorilla mux is at go-sql-sample
• Sensitive Data Identification: identify and encrypt specific fields in JSON payloads.

Benefits:

• Security: Protects sensitive information from being exposed in logs.
• Compliance: Helps meet security and compliance requirements by safeguarding sensitive data.
• Ease of Use: Simplifies the integration of encryption/masking into any existing applications.
• Consistency: Ensures that sensitive data is consistently encrypted or masked across all logged requests and responses

Use Cases of sensitive data masking/encrypting

Financial Transactions

• Benefit: Encrypting sensitive financial data, such as credit card numbers and transaction details, helps comply with PCI-DSS standards and secures financial transactions from exposure in logs.

Healthcare

• Benefit: Encrypting patient data such as medical records and health information in logs ensures compliance with HIPAA regulations and protects patient privacy.

E-commerce

• Benefit: Protecting customer information, such as addresses and payment details, enhances customer trust and protects the e-commerce platform from potential data breaches.

Benefits to Developers

Enhanced Debugging

• Provides detailed logs for requests and responses, aiding in troubleshooting and debugging issues.

Performance Monitoring

• Logs response times and sizes, allowing developers to monitor and optimize application performance.

Flexibility

• Configurable logging settings enable developers to tailor the logging behavior to their needs.

Ease of Integration

• Middleware function can be easily integrated into existing web frameworks, simplifying the setup process.

Improved Maintainability

• Centralized logging logic ensures consistency and makes the codebase easier to maintain.

Security:

• Encrypt or mask sensitive data in logs, reducing the risk of data exposure and meeting compliance requirements.

Conclusion

By implementing these features, you provide a comprehensive logging solution that enhances the visibility, performance, and maintainability of any GO applications.

Installation

Please make sure to initialize a Go module before installing core-go/log:

go get -u github.com/core-go/log

Import:

import "github.com/core-go/log"

Summary of middleware log tracing

log tracing at middleware

middleware log for

• http
• echo
• gin

Support to turn on, turn off

• request
• response
• duration
• http response status code
• response content length

Support to mask or encrypt fields

• support to mask or encrypt fields, such as mobileNumber, creditCardNumber
• useful for Financial Products (to comply with PCI-DSS standards) and Healthcare (to comply with HIPAA regulations)

Appendix

Microservice Architect

Cross-cutting concerns

• "log" in the full picture of cross-cutting concerns