Categorygithub.com/00security/grammes
modulepackage
1.3.6
Repository: https://github.com/00security/grammes.git
Documentation: pkg.go.dev

# README

Grammes

Grammes

GoDoc Go Report Card License

Grammes is an API/Wrapper for Gremlin and Janusgraph. It's written purely in Golang and allows for easy use of Gremlin without touching the Gremlin terminal.

Table of Contents

Local Setup

You need to setup all of the following tools to run the service locally

  • Go 1.12
  • Git
  • Elastic Search
  • Cassandra
    • Java 8

Cloning Grammes

Begin by opening up a terminal or command prompt and clone the grammes repository.

go get -u github.com/00security/grammes

Setting up JanusGraph

if you have decided to use another graph database then you may move on to project setup

First off, direct your terminal to the Grammes' scripts directory.

cd $GOPATH/src/github.com/00security/grammes/scripts

In here you can find the gremlin.sh and janusgraph.sh scripts. To set up JanusGraph just run the janusgraph.sh script.

./janusgraph.sh

This should download and/or begin the graph and TinkerPop server.

To make sure that everything is running try running gremlin.sh.

$ ./gremlin.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/<username>/projects/nm/gocode/src/github.com/00security/grammes/bin/janusgraph-0.3.1-hadoop2/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/<username>/projects/nm/gocode/src/github.com/00security/grammes/bin/janusgraph-0.3.1-hadoop2/lib/logback-classic-1.1.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
15:05:59 WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
gremlin>

Once Gremlin starts then you may begin by running this command.

gremlin> :remote connect tinkerpop.server conf/remote.yaml
===>Configured localhost/127.0.0.1:8182

If you see the message that Gremlin was configured to the localhost then quit Gremlin.

gremlin> :exit

Finally, run the janusgraph.sh script again, but this time with the status flag.

./janusgraph.sh status

Using Grammes

Once you have cloned the repository then you may begin implementing it into a project. Let's begin with creating a place for your code in the $GOPATH, i.e.,

$GOPATH/src/github.com/<username-here>/<project-name-here>

Next, you'll want to create a main.go file. For this example I will be using MS Code, but you may use any editor you prefer.

code main.go

In this file we can begin by making it a typical empty main.go file like this:

package main

func main() {
}

Next, import the grammes package and begin using it by connecting your client to a gremlin server.

package main

import (
    "log"

    "github.com/00security/grammes"
)

func main() {
    // Creates a new client with the localhost IP.
    client, err := grammes.DialWithWebSocket("ws://127.0.0.1:8182")
    if err != nil {
        log.Fatalf("Error while creating client: %s\n", err.Error())
    }

    // Executing a basic query to assure that the client is working.
    res, err := client.ExecuteStringQuery("1+3")
    if err != nil {
        log.Fatalf("Querying error: %s\n", err.Error())
    }

    // Print out the result as a string
    for _, r := range res {
        log.Println(string(r))
    }
}

Once the client is created then you can begin querying the gremlin server via the .ExecuteQuery method in the client. To use this function you must put in a Query which is an interface for any kind of Query-able type in the package. These include: graph.String and traversal.String. For an example of querying the gremlin server for all of the Vertex labels:

package main

import (
    "log"

    "github.com/00security/grammes"
)

func main() {
    // Creates a new client with the localhost IP.
    client, err := grammes.DialWithWebSocket("ws://127.0.0.1:8182")
    if err != nil {
        log.Fatalf("Error while creating client: %s\n", err.Error())
    }

    // Executing a query to add a vertex to the graph.
    client.AddVertex("testingvertex")

    // Create a new traversal string to build your traverser.
    g := grammes.Traversal()

    // Executing a query to fetch all of the labels from the vertices.
    res, err := client.ExecuteQuery(g.V().Label())
    if err != nil {
        log.Fatalf("Querying error: %s\n", err.Error())
    }

    // Log out the response.
    for _, r := range res {
        log.Println(string(r))
    }
}

After this is all written you may run this file by saving it and hopping back on to your terminal. After starting your Gremlin Server and graph database in the terminal let's run this command to run the file:

go run main.go

For more examples look in the examples/ directory of the project. In there you'll find multiple examples on how to use the Grammes package.

Testing Grammes

Grammes uses goconvey by smartystreets for its tests. Before trying to run the unit tests in Grammes you should update your version of this repository using this command.

go get -u github.com/smartystreets/goconvey/convey

Once you have this downloaded you may run the tests in Grammes how you normally would in Golang.

go test ./...

Additional Resources

Documentation on Gremlin

To learn more about how to use Gremlin I highly recommend looking through their Tinkerpop3 documentation. It's full of examples and documentation on every traversal step available.

Examples

To find examples look in the examples/ directory of Grammes. In there you'll find plenty of examples related to how to use this package. Make sure you're running Janusgraph before you begin running the examples.

Troubleshooting

Fixing time outs when starting Janusgraph

If Nodetool times out or any other part of the setup times out then the most common issue is that Cassandra is already running on your machine. To fix this run this command.

# only for Unix at the moment.
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.cassandra.plist

# Packages

No description provided by the author
Package gremconnect has the connection related structs and functions.
Package gremerror holds custom error types to return in the Grammes package.
Package logging provides some zap loggers for the use of a Grammes client.
Package manager contains the graph managers to query the database.
Package model holds structs of the Gremlin counterparts and related methods.
Package query contains the interface used to define a graph database query.
Package quick is used for when you want to execute queries to the graph database without having to use a Grammes client.

# Functions

CustomTraversal could be used when you need to specifically need to change some property of the traversal.
Dial returns a working client with the given dialer and configurations.
DialWithWebSocket returns a new client with a websocket dialer and possible client configurations.
Traversal will return a new traversal string ready to use when executing a query.
VerboseTraversal will return a new graph traversal string ready to use when executing a query.
WithAuthUserPass sets the authentication credentials within the dialer.
WithCompression sets the compression flag for websocket connections.
WithErrorChannel will assign an error channel to send connection errors through for the user to handle.
WithGremlinVersion sets the version of the gremlin traversal language being used by the client.
WithHandshakeTimeout sets the websocket handshake timeout.
WithHTTPAuth sets the authentication provider within the dialer.
WithLogger will replace the default zap.Logger with a custom logger that implements the logging.Logger interface.
WithMaxConcurrentMessages sets the limit as to how many requests can be stored in the requests buffer.
WithMaxConcurrentRequests sets the limit as to how many requests can done simultaneously.
WithPingInterval sets the interval of ping sending for know is connection is alive and in consequence the client is connected.
WithReadBufferSize sets the max read buffer size for the websocket frame.
WithReadingWait sets the time to wait when reading with the dialer in seconds.
WithRequestTimeout sets the timeout when reading a request from the gremlin server.
WithTimeout sets the timeout to wait when dialing with the dialer in seconds.
WithTLS sets the TLS config.
WithWriteBufferResizing enables dynamic write buffer expansion, effectively disabling websocket frame fragmentation (which TinkerPop doesn't support).
WithWriteBufferSize sets the max write buffer size for the websocket frame.
WithWritingWait sets the time to wait when writing with the dialer in seconds.

# Variables

NewProperty returns a property struct meant for adding it to a vertex.
NewVertex returns a vertex struct meant for adding it.
NewWebSocketDialer returns websocket with established connection.
UnmarshalEdgeList is a utility to unmarshal a list or array of edges properly.
UnmarshalPropertyList is a utility to unmarshal a list or array of IDs properly.
UnmarshalVertexList is a utility to unmarshal a list or array of vertices properly.

# Structs

Client is used to handle the graph, schema, connection, and basic debug logging when querying the graph database.

# Type aliases

APIData is used to get quick access to the model.APIData without having to import it everywhere in the grammes package.
ClientConfiguration is the type used for configuring and changing values in the client and the dialer.
Data is used to get quick access to the model.Data without having to import it everywhere in the grammes package.
Edge is used to get quick access to the model.Edge without having to import it everywhere in the grammes package.
EdgeList is used to get quick access to the model.EdgeList without having to import it everywhere in the grammes package.
IDList is used to get quick access to the model.IDList without having to import it everywhere in the grammes package.
Property is used to get quick access to the model.Property without having to import it everywhere in the grammes package.
PropertyList is used to get quick access to the model.PropertyList without having to import it everywhere in the grammes package.
PropertyMap is used to get quick access to the model.PropertyMap without having to import it everywhere in the grammes package.
PropertyValue is used to get quick access to the model.PropertyValue without having to import it everywhere in the grammes package.
SimpleValue is used to get quick access to the model.SimpleValue without having to import it everywhere in the grammes package.
Vertex is used to get quick access to the model.Vertex without having to import it everywhere in the grammes package.
VertexList is used to get quick access to the model.VertexList without having to import it everywhere in the grammes package.
VertexValue is used to get quick access to the model.VertexValue without having to import it everywhere in the grammes package.