Categorygithub.com/colinmarc/sequencefile
modulepackage
1.0.0
Repository: https://github.com/colinmarc/sequencefile.git
Documentation: pkg.go.dev

# README

Sequencefile

Go Reference

This is a native Go implementation of Hadoop's SequenceFile format.

Usage

sf, err := sequencefile.Open("foo.sequencefile")
if err != nil {
  log.Fatal(err)
}

// Iterate through the file.
for sf.Scan() {
  // Do something with sf.Key() and sf.Value()
}

if sf.Err() != nil {
  log.Fatal(err)
}

Reading files written by Hadoop

Hadoop adds another layer of serialization for individual keys and values, depending on the class used, like BytesWritable. By default, this library will return the raw key and value bytes, still serialized. You can use the following methods to unwrap them:

func BytesWritable(b []byte) []byte
func Text(b []byte) string
func IntWritable(b []byte) int32
func LongWritable(b []byte) int64

# Functions

BytesWritable unwraps a hadoop BytesWritable and returns the actual bytes.
IntWritable unwraps an IntWritable and returns the deserialized int32.
LongWritable unwraps an LongWritable and returns the deserialized int64.
New returns a new Reader for a SequenceFile, reading data from r.
New returns a new Reader for a SequenceFile, reading data from r.
NewWritableWriter gets a WritableWriter for a given Hadoop class name.
NewWriter constructs a new Writer.
Open opens a SequenceFile on disk and immediately reads the header.
ReadVInt reads an int64 encoded in hadoop's "VInt" format, described and implemented here: https://goo.gl/1h4mrG.
Text unwraps a Text and returns the deserialized string.
WriteVInt writes an int64 encoded in Hadoop's "VInt" format.

# Constants

No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author

# Structs

A Header represents the information contained in the header of the SequenceFile.
A Reader reads key/value pairs from a SequenceFile input stream.
A Writer writes key/value pairs to a sequence file output stream.
A WriterConfig specifies the configuration for a Writer.

# Type aliases

No description provided by the author
No description provided by the author
A WritableWriter knows how to write data wrapped in Hadoop Writables.