Work with DSE Graph in Your Web Applications Part (2/2) : Visualization

This post is the second and last part of a series digging into integration of Graph Databases with web applications. In Part 1, we created a data access object (DAO) to implement basic CRUD operations for both vertices and edges. Here, we will have a closer look at graph visualization in user interfaces. One of the coolest features of graph is the capability to browse the data to identiy patterns and extract information from relationships. DataStax Studio provides with great visual rendering but, why not having the same visualization in your own applications?

What's the idea?

For an overview of the solution, please look at the diagram on the left. When it comes to architecture, a picture is worth a thousand words.

Here are the steps and components we will review.

1. User POSTs a Gremlin query to the REST API
2. The query is converted from String to GraphStatement and executed at the DAO level
3. Driver returns a GraphResultSet
4. Mapper translates the result set to web beans with the structure expected by the front end
5. Graph is rendered using the Javascript library VisJs
6. Specialisation and customisation can be done in both Mapper and CSS.

Implementation

Rendering charts in user interface

When you need to deal with rich user interfaces, Javascript is a no brainer, alternative solutions like Flash or applets are long since obsolete. You can find dozens of frameworks to render charts with JS. I have used a couple myself in the past  and am very keen on D3js and HighCharts. You can see and run my experiments on github if interested using the following :

git clone https://github.com/clun/sandboxes.git
cd clu-dataviz
mvn jetty:run

Then accessing http://localhost:8181 you should get something like :

But when it comes to rendering graphs and networks, I've found nothing better than VisJS (which is also used in DataStax studio). It provides tons of options to customize exactly as you wish. It is also aware of events coming from users (such as mouse clicks), allowing you to implement cool features like interactive graph browsing (c.f. Datastax Studio). This framework expects JSON data provided with a specific structure such as this:

var nodes = new vis.DataSet([
    {id: 1, label: 'Node 1'},
    {id: 2, label: 'Node 2'},
    {id: 3, label: 'Node 3'},
    {id: 4, label: 'Node 4'},
    {id: 5, label: 'Node 5'}
  ]);

  // create an array with edges
  var edges = new vis.DataSet([
    {from: 1, to: 3},
    {from: 1, to: 2},
    {from: 2, to: 4},
    {from: 2, to: 5},
    {from: 3, to: 3}
  ]);

Creating a REST API

For the backend our objective is to create a REST API accepting a Gremlin Query as input and generating the JSON Format expected by VisJS. The first thing to do is to create plain old java objects (POJOs) to reflect the Json structure. Here is a UML representation generated by ObjectAid available for Eclipse.

We chose to use SpringMVC included in SpringBoot (what else ?). The DAO generated in first blogpost can be reused and injected into the REST resources.

@RestController
@RequestMapping("/api/v1/graphs")
public class GraphResource {
   
    @Autowired
    protected GraphDao graphDao;

    @RequestMapping(value = "/{graphName}/{gremlinQuery}", method = GET,  produces = APPLICATION_JSON_VALUE)
    public ResponseEntity<VizJsGraph> executeGremlinQuery(
            @PathVariable(value = "graphName") String graphName, 
            @PathVariable(value = "gremlinQuery") String gremlinQuery) {
        return new ResponseEntity<VizJsGraph>(
                              graphDao.executeGremlinQuery(graphName, gremlinQuery, true), 
                              HttpStatus.ACCEPTED);
    }

}

Note : executeGremlinQuery has not been defined yet, we will do this in the next section.

Implementing the DAO

As we have seen in the previous post, the DataStax Java driver accepts Gremlin queries as raw Strings (how convenient). The driver will convert each query into a GraphStatement and execute against DSE Graph.

This is what it looks like :

public VizJsGraph executeGremlinQuery(String graphName, String gremlinQuery, boolean populateEdges) {
  Assert.hasText(graphName, "'graphName' is required here");
  Assert.hasText(gremlinQuery, "'gremlinQuery' is required here");
  LOGGER.info("Executing query {} on graph {}", gremlinQuery, graphName);
        
  // QUERY FOR VERTICES
  GraphStatement graphStatement = new SimpleGraphStatement(gremlinQuery).setGraphName(graphName);
  GraphResultSet gras = dseSession.executeGraph(graphStatement);
        
  // MAPPING AS VERTICES IN UI
  VizJsGraph vizGraph    = new VizJsGraph();
  List<Object> vertexIds = new ArrayList<>();
  for (GraphNode gn : gras) {
     if (populateEdges && gn.isVertex()) vertexIds.add(gn.asVertex().getId());
     populateGraphVizJs(gn, vizGraph);
   }
   
   // SOMETHING IS MISSING 
   //--> HERE <--
   return vizGraph;
}

This works quite well, but there is more. Have you ever try to execute the g.V() query in DataStax Studio ? It will display the whole graph (both vertices and edges)...but it shouldn't. The gremlin query returns Vertices (V for Vertices) and only the Vertices so, what is the trick for how to collect edges as well?

The way to retrieve edges between vertices in Gremlin is described in the following documentation. If we want to display a nice graph with the edges in the user interface, we need to fire an extra query where ids are the unique identifier of your vertices. 

Here is the code. I inlined the code a bit, YOU are an expert now after all.

// Add Edges to the results
dseSession.executeGraph(new SimpleGraphStatement("g.V(ids.toArray()).outE().where(inV().id().is(within(ids)))")
                              .set("ids", vertexIds)
                              .setGraphName(graphName))
          .all()
          .stream()
          .forEach(gn -> populateGraphVizJs(gn, vizGraph));

Populating the visualization via Mappers

Last missing part to have our sample running is mapping of GraphResultSet to VizJsGraph. Straightforward, this is what it looks like:

private void populateGraphVizJs(GraphNode gn, VizJsGraph graph) {
  if (gn.isVertex()) {
     Vertex v = gn.asVertex();
     graph.addVertex(new GraphVertex().id(v.getId().toString()).label(v.getLabel()););
  } else if (gn.isEdge()) {
    Edge e = gn.asEdge();
     graph.addEdge(new GraphEdge(e.getOutV().toString(), e.getInV().toString()));
  }
}

As usual, the full working application can be found on github please have a look and run a sample command. And the result is ....

This....is... a bit disapointing, isn't it? After all we have done it is still quite ugly, and it's pretty difficult to extract value from that representation. You cannot give that to the business can you?

Don't worry, we are not finished yet.

Customisation (Mappers, part 2)

First, update the mapper to select which properties will be displayed in the graph, and how they will be displayed. You can use the label of the vertex or any business condition to filter. Then, define which group each Vertex will be part of. The concept of a group is used by the VisJs library in order to apply specific CSS styles for rendering. This is what it looks like in our example :

Update the Mapper :

if ("video".equals(v.getLabel())) {
  VertexProperty videoName = v.getProperty("name");
  if (videoName != null) {
    gv.setLabel(videoName.getValue().asString());
  }
} else if ("tag".equals(v.getLabel())) {
  VertexProperty videoName = v.getProperty("name");
  if (videoName != null) {
    gv.setLabel(videoName.getValue().asString());
  }   
} else if ("user".equals(v.getLabel())) {
  VertexProperty email = v.getProperty("email");
  if (email != null) {
    gv.setLabel(email.getValue().asString());
  } 
} // Important
gv.setGroup(v.getLabel());,

Update Javascript Function & apply CSS styles :

function executeGremlin(graphName) {
        $.get('api/v1/graphs/' + graphName + '/' + document.getElementById('query').value,
          function(graph) {
            var container = document.getElementById('mynetwork');
            var data      = { nodes: graph.nodes, edges: graph.edges };
            var options = {
                    groups: {
                      video: {
                        shape: 'icon',
                        icon: {
                          face: 'FontAwesome',
                          code: '\uf03d',
                          size: 50,
                          color: '#57169a'
                        }
                      },
                      user: {
                        shape: 'icon',
                        icon: {
                          face: 'FontAwesome',
                          code: '\uf007',
                          size: 50,
                          color: '#0890D0'
                        }
                      },
                      tag: {
                        shape: 'icon',
                        icon: {
                         face: 'FontAwesome',
                          code: '\uf02b',
                              size: 50,
                              color: '#00ab8b'
                            }
                          }
                    }
                  };
            var network = new vis.Network(container, data, options);
        });
    }

If now we execute the same query we get:

Isn't that cool?

Conclusion and takeaways

In this series of two articles, we have seen how simple it was to connect, interact and render data coming from graph databases. From this template and the code provided on github we expect you to build AWESOME applications.

If you want to improve with Gremlin Language or Graph data modeling please consider taking the FREE courses DS330 and Gremlin Recipes available here on DataStax Academy.

Happy coding!