# README
LiteFS
LiteFS is a FUSE-based file system for replicating SQLite databases across a cluster of machines. It works as a passthrough file system that intercepts writes to SQLite databases in order to detect transaction boundaries and record changes on a per-transaction level in LTX files.
This project is actively maintained but is currently in a beta state. Please report any bugs as an issue on the GitHub repository.
You can find a Getting Started guide on LiteFS' documentation site. Please see the ARCHITECTURE.md design document for details about how LiteFS works.
SQLite TCL Test Suite
It's a goal of LiteFS to pass the SQLite TCL test suite, however, this is currently a work in progress. LiteFS doesn't have database deletion implemented yet so that causes many tests to fail during teardown.
To run a test from the suite against LiteFS, you can use the Dockerfile.test
to run it in isolation. First build the Dockerfile:
docker build -t litefs-test -f Dockerfile.test .
Then run it with the filename of the test you want to run. In this case, we
are running select1.test:
docker run --device /dev/fuse --cap-add SYS_ADMIN -it litefs-test select1.test
Contributing
LiteFS contributions work a little different than most GitHub projects. If you have a small bug fix or typo fix, please PR directly to this repository.
If you would like to contribute a feature, please follow these steps:
- Discuss the feature in an issue on this GitHub repository.
- Create a pull request to your fork of the repository.
- Post a link to your pull request in the issue for consideration.
This project has a roadmap and features are added and tested in a certain order. Additionally, it's likely that code style, implementation details, and test coverage will need to be tweaked so it's easier to for me to grab your implementation as a starting point when implementing a feature.
# Functions
ContainsLockType returns true if a contains typ.
FormatNodeID formats a node identifier as a 16-character uppercase hex string.
GenerateClusterID returns a new, randomly-generated cluster ID.
JournalChecksum returns the checksum used by the journal format.
NewDB returns a new instance of DB.
NewFileBackupClient returns a new instance of FileBackupClient.
JournalReader returns a new instance of JournalReader.
NewStaticLeaser returns a new instance of StaticLeaser.
NewStore returns a new instance of Store.
NewWALReader returns a new instance of WALReader.
ParseDatabaseLockRange returns a list of SQLite database locks that are within a range.
ParseNodeID parses a 16-character hex string into a node identifier.
ParseSHMLockRange returns a list of SQLite WAL locks that are within a range.
ReadStreamFrame reads a the stream type & frame from the reader.
TrimName removes "-journal", "-shm" or "-wal" from the given name.
ValidateClusterID returns nil if id is a valid cluster ID.
WALChecksum computes a running SQLite WAL checksum over a byte slice.
WriteStreamFrame writes the stream type & frame to the writer.
# Constants
ChecksumBlockSize is the number of pages that are grouped into a single checksum block.
ClusterIDLen is the length of a cluster ID.
ClusterIDPrefix is the prefix for every cluster ID.
Database journal modes.
Database journal modes.
Default store settings.
Default store settings.
Default store settings.
Default store settings.
Default store settings.
Default store settings.
Default store settings.
Default store settings.
Default store settings.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
Database file types.
Database file types.
Database file types.
Database file types.
Database file types.
Database file types.
Database file types.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
LiteFS-specific lock byte.
1073741824.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
1073741825.
1073741826.
SHM file locks.
MaxBackupLTXFileN is the number of LTX files that can be compacted together at a time when sending data to the backup service.
No description provided by the author
SQLite rollback journal lock constants.
SQLite rollback journal lock constants.
RWMutexInterval is the time between reattempting lock acquisition.
No description provided by the author
No description provided by the author
No description provided by the author
SQLite rollback journal lock constants.
SQLite rollback journal lock constants.
No description provided by the author
Location of the database size, in pages, in the main database file.
Size of the journal header, in bytes.
Magic header string that identifies a SQLite journal header.
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
TraceLogFlags are the flags to be used with TraceLog.
WaitInterval is the time between checking if the DB has reached a position in DB.Wait().
SQLite WAL lock constants.
SQLite WAL lock constants.
SQLite WAL lock constants.
SQLite WAL lock constants.
SQLite WAL lock constants.
SQLite WAL lock constants.
SQLite WAL lock constants.
SQLite WAL lock constants.
SQLite constants.
SQLite constants.
SQLite constants.
SQLite constants.
# Variables
LiteFS errors.
LiteFS errors.
LiteFS errors.
ErrInvalidClusterID is returned when a cluster ID is invalid.
No description provided by the author
LiteFS errors.
LiteFS errors.
LiteFS errors.
LiteFS errors.
LiteFS errors.
LiteFS errors.
No description provided by the author
GlobalStore represents a single store used for metrics collection.
Global log level.
NativeEndian is always set to little endian as that is the only endianness used by supported platforms for LiteFS.
TraceLog is a log for low-level tracing.
# Structs
ChangeSetSubscriber subscribes to changes to databases in the store.
DB represents a SQLite database.
DropDBStreamFrame notifies replicas that a database has been deleted.
No description provided by the author
Event represents a generic event.
EventSubscriber subscribes to generic store events.
FileBackupClient is a reference implemenation for BackupClient.
GuardSet represents a set of mutex guards by a single owner.
HaltLock represents a lock remotely held on the primary.
No description provided by the author
HeartbeatStreamFrame informs replicas that there have been no recent transactions.
HWMStreamFrame propagates the high-water mark to replica nodes.
No description provided by the author
JouralReader represents a reader of the SQLite journal file format.
No description provided by the author
NodeInfo represents basic info about a node.
No description provided by the author
PrimaryInfo is the JSON object stored in the Consul lease value.
No description provided by the author
RWMutex is a reader/writer mutual exclusion lock.
RWMutexGuard is a reference to a mutex.
StaticLease represents a lease for a fixed primary.
StaticLeaser always returns a lease to a static primary.
Store represents a collection of databases.
No description provided by the author
WALReader wraps an io.Reader and parses SQLite WAL frames.
# Interfaces
No description provided by the author
Client represents a client for connecting to other LiteFS nodes.
Environment represents an interface for interacting with the host environment.
Invalidator is a callback for the store to use to invalidate the kernel page cache.
Lease represents an acquired lease from a Leaser.
Leaser represents an API for obtaining a lease for leader election.
OS represents an interface for os package calls so they can be mocked for testing.
Stream represents a stream of frames.
No description provided by the author
# Type aliases
DBMode represents either a rollback journal or WAL mode.
FileType represents a type of SQLite file.
JournalMode represents a SQLite journal mode.
LockType represents a SQLite lock type.
RWMutexState represents the lock state of an RWMutex or RWMutexGuard.
No description provided by the author
No description provided by the author