Update docs (#422)

Author: tomasonjo

doc/betweenness-centrality.adoc

@@ -31,6 +31,10 @@ betweenness centrality as an analysis measure, it indicates a potential gate kee
 
 It differs from the other centrality measures. A node can have quite low degree, be connected to others that have low degree, even be a long way from others on average, and still have high betweenness. Consider a node A that lies on a bridge between two groups of vertices within a network. Since any path between nodes in different groups must go through this bridge, node A acquires high betweenness even though it is not well connected (it lies at the periphery of both groups).
 
+Node betweenness centrality is relevant to problems such as identifying important nodes that control flows of information between separate parts of the network and identifying causal nodes to influence other entities behavior, such as genes in genomics or customers in marketing studies.
+Betweenness centrality has been used to analyze social networks and protein networks, to identify significant nodes in wireless ad hoc networks, to study the importance and activity of nodes in mobile phone call networks and interaction patterns of players on massively multiplayer online games, to study online expertise sharing communities such as physicians, to identify and analyze linking behavior of key bloggers in dynamic networks of blog posts and to measure network traffic in communication networks.[1]
+
+
 == Constraints / when not to use it
 
 
@@ -42,33 +46,19 @@ People with high betweenness tend to be the innovators and brokers in any networ
 .Create sample graph
 [source,cypher]
 ----
-CREATE (nAlice:User {id:'Alice'})
-,(nBridget:User {id:'Bridget'})
-,(nCharles:User {id:'Charles'})
-,(nDoug:User {id:'Doug'})
-,(nMark:User {id:'Mark'})
-,(nMichael:User {id:'Michael'})
-CREATE (nAlice)-[:MANAGE]->(nBridget)
-,(nAlice)-[:MANAGE]->(nCharles)
-,(nAlice)-[:MANAGE]->(nDoug)
-,(nMark)-[:MANAGE]->(nAlice)
-,(nCharles)-[:MANAGE]->(nMichael)
-
+include::scripts/betweenness-centrality.cypher[tag=create-sample-graph]
 ----
 
 .Running algorithm and streaming results
 [source,cypher]
 ----
-CALL algo.betweenness.stream('User','MANAGE',{direction:'out'}) 
-YIELD nodeId, centrality 
-RETURN nodeId,centrality order by centrality desc limit 20
+include::scripts/betweenness-centrality.cypher[tag=stream-sample-graph]
 ----
 
 .Running algorithm and writing back results
 [source,cypher]
 ----
-CALL algo.betweenness('User','MANAGE', {direction:'out',write:true, writeProperty:'centrality'}) 
-YIELD nodes, minCentrality, maxCentrality, sumCentrality, loadMillis, computeMillis, writeMillis
+include::scripts/betweenness-centrality.cypher[tag=write-sample-graph]
 ----
 
 .Results
@@ -164,10 +154,7 @@ Set `graph:'cypher'` in the config.
 
 [source,cypher]
 ----
-CALL algo.betweenness(
-'MATCH (p:User) RETURN id(p) as id',
-'MATCH (p1:User)-[:MANAGE]->(p2:User) RETURN id(p1) as source, id(p2) as target',
-{graph:'cypher', write: true});
+include::scripts/betweenness-centrality.cypher[tag=cypher-loading]
 ----
 
 == Versions 
@@ -176,14 +163,14 @@ We support the following versions of the betweenness centrality algorithm:
 
 * [x] directed, unweighted
 
-- loading incoming relationships: 'INCOMING','IN','I' or '<'
-- loading outgoing relationships: 'OUTGOING','OUT','O' or '>'
+** loading incoming relationships: 'INCOMING','IN','I' or '<'
+** loading outgoing relationships: 'OUTGOING','OUT','O' or '>'
 
 * [ ] directed, weighted
 
 * [x] undirected, unweighted
 
-- direction:'both' or '<>'
+** direction:'both' or '<>'
 
 * [ ] undirected, weighted 
 
@@ -221,6 +208,9 @@ We support the following versions of the betweenness centrality algorithm:
 
 * http://iima.org/wp/wp-content/uploads/2017/04/Curriculum-Structure-and-Assessment-Placement_Lightfoot.pdf
 
+* [1] https://arxiv.org/pdf/1702.06087.pdf
+
+
 ifdef::implementation[]
 // tag::implementation[]
 

doc/closeness-centrality.adoc

@@ -24,7 +24,10 @@ Our algorithm returns *normalized closeness centrality* score.
 
 == When to use it / use-cases
 
-The most central nodes according to closeness centrality can quickly interact to all others because they are close to all others.
+As an example, one can consider identifying the location within a city where to place a new public service, so that it is easily accessible for everyone. 
+Similarly, identifying central people that have ideal social network location for the purpose of information dissemination or network influence. 
+In such kind of applications, the nodes who can access the entire network faster need to be selected.
+
 This measure is preferable to degree centrality, because it does not take into account only direct connections among nodes, but also indirect connections.
 For closeness centrality in directed networks, we split the concept into in-closeness
 and out-closeness. 
@@ -45,33 +48,20 @@ image::{img}/closeness_centrality.png[]
 .Create sample graph
 [source,cypher]
 ----
-CREATE (a:Node{id:"A"}),
-       (b:Node{id:"B"}),
-       (c:Node{id:"C"}),
-       (d:Node{id:"D"}),
-       (e:Node{id:"E"})
-CREATE (a)-[:LINK]->(b),
-       (b)-[:LINK]->(a),
-       (b)-[:LINK]->(c),
-       (c)-[:LINK]->(b),
-       (c)-[:LINK]->(d),
-       (d)-[:LINK]->(c),
-       (d)-[:LINK]->(e),
-       (e)-[:LINK]->(d)
+include::scripts/closeness-centrality.cypher[tag=create-sample-graph]
 ----
 
 .Running algorithm and streaming results
 [source,cypher]
 ----
-CALL algo.closeness.stream('Node', 'LINKS') YIELD nodeId, centrality
-RETURN nodeId,centrality order by centrality desc limit 20 - yields centrality for each node
+include::scripts/closeness-centrality.cypher[tag=stream-sample-graph]
+- yields centrality for each node
 ----
 
 .Running algorithm and writing back results
 [source,cypher]
 ----
-CALL algo.closeness('Node', 'LINK', {write:true, writeProperty:'centrality'}) 
-YIELD nodes,loadMillis, computeMillis, writeMillis 
+include::scripts/closeness-centrality.cypher[tag=write-sample-graph]
 - calculates closeness centrality and potentially writes back
 ----
 
@@ -170,10 +160,7 @@ Set `graph:'cypher'` in the config.
 
 [source,cypher]
 ----
-CALL algo.closeness(
-'MATCH (p:Node) RETURN id(p) as id',
-'MATCH (p1:Node)-[:LINK]->(p2:Node) RETURN id(p1) as source, id(p2) as target',
-{graph:'cypher', write: true});
+include::scripts/closeness-centrality.cypher[tag=cypher-loading]
 ----
 
 == Versions 

doc/connected-components.adoc

@@ -1,4 +1,4 @@
-= Connected Components
+= Community detection: Connected Components
 
 _Connected Components_ or _UnionFind_ basically finds sets of connected nodes where each node is reachable from any other node in the same set. 
 In graph theory, a connected component of an undirected graph is a subgraph in which any two vertices are connected to each other by paths, and which is connected to no additional vertices in the graph.
@@ -37,32 +37,20 @@ image::{img}/connected_components.png[]
 .Create sample graph
 [source,cypher]
 ----
-CREATE (nAlice:User {id:'Alice'})
-,(nBridget:User {id:'Bridget'})
-,(nCharles:User {id:'Charles'})
-,(nDoug:User {id:'Doug'})
-,(nMark:User {id:'Mark'})
-,(nMichael:User {id:'Michael'})
-CREATE (nAlice)-[:FRIEND{weight:0.5}]->(nBridget)
-,(nAlice)-[:FRIEND{weight:4}]->(nCharles)
-,(nMark)-[:FRIEND{weight:1}]->(nDoug)
-,(nMark)-[:FRIEND{weight:2}]->(nMichael)
-
+include::scripts/connected-components.cypher[tag=create-sample-graph]
 ----
 
 === Unweighted version:
 
 .Running algorithm and streaming results
 [source,cypher]
 ----
-CALL algo.unionFind.stream('User', 'FRIEND', {}) 
-YIELD nodeId,setId
+include::scripts/connected-components.cypher[tag=unweighted-stream-sample-graph]
 ----
 .Running algorithm and writing back results
 [source,cypher]
 ----
-CALL algo.unionFind('User', 'FRIEND', {write:true, partitionProperty:"partition"}) 
-YIELD nodes, setCount, loadMillis, computeMillis, writeMillis
+include::scripts/connected-components.cypher[tag=unweighted-write-sample-graph]
 ----
 .Results
 [opts="header",cols="1,1"]
@@ -83,9 +71,7 @@ Results show us, that we have two distinct group of users that have no link betw
 .We can also easily check the number and size of partitions using cypher.
 [source,cypher]
 ----
-MATCH (u:User)
-RETURN u.partition as partition,count(*) as size_of_partition 
-ORDER by size_of_partition DESC LIMIT 20 
+include::scripts/connected-components.cypher[tag=check-results-sample-graph]
 ----
 === Weighted version:
 
@@ -94,14 +80,12 @@ If you define the property that holds the weight(weightProperty) and the thresho
 .Running algorithm and streaming results
 [source,cypher]
 ----
-CALL algo.unionFind.stream('User', 'FRIEND', {weightProperty:'weight', defaultValue:0.0, threshold:1.0}) 
-YIELD nodeId,setId
+include::scripts/connected-components.cypher[tag=weighted-stream-sample-graph]
 ----
 .Running algorithm and writing back results
 [source,cypher]
 ----
-CALL algo.unionFind('User', 'FRIEND', {write:true, partitionProperty:"partition",weightProperty:'weight', defaultValue:0.0, threshold:1.0}) 
-YIELD nodes, setCount, loadMillis, computeMillis, writeMillis
+include::scripts/connected-components.cypher[tag=weighted-write-sample-graph]
 ----
 
 .Results
@@ -120,6 +104,32 @@ We can observe, that because the weight of the relationship betwen Bridget and A
 
 == Example Usage
 
+As said Connected Components are an essential step in preprocessing your data. 
+One reason is that most centralities suffer from disconnected components or you just want to find disconnected groups of nodes. 
+Yelp's social network will be used to demonstrate how to proceed when dealing with real world data.
+A typical social network consist of one big component and a number of small disconnected components.
+
+.Get the count of connected components
+[source,cypher]
+----
+include::scripts/connected-components.cypher[tag=count-component-yelp]
+----
+
+We get back count of disconnected components being 18512 if we do not count users without friends. 
+Let's now check the size of top 20 components to get a better picture
+
+.Get the size of top 20 components
+[source,cypher]
+----
+include::scripts/connected-components.cypher[tag=top-20-component-yelp]
+----
+
+The biggest component has 8938630 out of total 8981389 (99,5%).
+It is quite high, but not shocking as we have a friendship social network, where we can expect small world effect and 6 degree of separation rule, where you can get to any person in a social network, just depends how long is the path.
+
+We can now move on to next step of analysis and run centralities only on the biggest components, so that our results will be more accurate.
+We just simply write back the results to the node, and use centralities with cypher loading or set a new label for the biggest component.
+
 == Syntax
 
 .Running algorithm and writing back results
@@ -193,9 +203,7 @@ Set `graph:'cypher'` in the config.
 
 [source,cypher]
 ----
-CALL algo.unionFind('MATCH (p:User) RETURN id(p) as id',
-'MATCH (p1:User)-[:FRIEND]->(p2:User) RETURN id(p1) as source, id(p2) as target',
-{graph:'cypher',write:true});
+include::scripts/connected-components.cypher[tag=cypher-loading]
 ----
 == Implementations
 

doc/label-propagation.adoc

@@ -38,7 +38,6 @@ Moreover, it is computationally expensive to determine the community size.
 Hence, algorithms that are able to detect communities from complex networks without having any knowledge about the number of communities or the size of communities are essential. 
 _Label Propagation_ algorithms (LPA) are such type of algorithms that detects community structure that uses network structure alone as its guide and requires neither a pre-defined objective function nor prior information about the communities. 
 
-
 == Constraints / when not to use it
 
 == Algorithm explanation on simple sample graph
@@ -48,29 +47,13 @@ image::{img}/label_propagation.png[]
 .Create sample graph
 [source,cypher]
 ----
-CREATE (nAlice:User {id:'Alice'})
-,(nBridget:User {id:'Bridget'})
-,(nCharles:User {id:'Charles'})
-,(nDoug:User {id:'Doug'})
-,(nMark:User {id:'Mark'})
-,(nMichael:User {id:'Michael'})
-CREATE (nAlice)-[:FOLLOW]->(nBridget)
-,(nAlice)-[:FOLLOW]->(nCharles)
-,(nMark)-[:FOLLOW]->(nDoug)
-,(nBridget)-[:FOLLOW]->(nMichael)
-,(nDoug)-[:FOLLOW]->(nMark)
-,(nMichael)-[:FOLLOW]->(nAlice)
-,(nAlice)-[:FOLLOW]->(nMichael)
-,(nBridget)-[:FOLLOW]->(nAlice)
-,(nMichael)-[:FOLLOW]->(nBridget)
-,(nCharles)-[:FOLLOW]->(nDoug)
+include::scripts/label-propagation.cypher[tag=create-sample-graph]
 ----
 
 .Running algorithm and writing back results
 [source,cypher]
 ----
-CALL algo.labelPropagation('User', 'FOLLOW','OUTGOING', {iterations:10,partitionProperty:'partition', write:true}) 
-YIELD nodes, iterations, loadMillis, computeMillis, writeMillis, write, partitionProperty 
+include::scripts/label-propagation.cypher[tag=write-sample-graph]
 ----
 
 == Example Usage
@@ -120,11 +103,11 @@ We support the following versions of the label propagation algorithms:
 
 * [x] directed, unweighted:  
 
-- weightProperty: null
+** weightProperty: null
 
 * [x] directed, weighted 
 
-- weightProperty : 'weight'
+** weightProperty : 'weight'
 
 * [ ] undirected, unweighted
 

doc/louvain.adoc

@@ -88,36 +88,19 @@ image::{img}/louvain.png[]
 .Create sample graph
 [source,cypher]
 ----
-CREATE (nAlice:User {id:'Alice'})
-,(nBridget:User {id:'Bridget'})
-,(nCharles:User {id:'Charles'})
-,(nDoug:User {id:'Doug'})
-,(nMark:User {id:'Mark'})
-,(nMichael:User {id:'Michael'})
-CREATE (nAlice)-[:FRIEND]->(nBridget)
-,(nAlice)-[:FRIEND]->(nCharles)
-,(nMark)-[:FRIEND]->(nDoug)
-,(nBridget)-[:FRIEND]->(nMichael)
-,(nCharles)-[:FRIEND]->(nMark)
-,(nAlice)-[:FRIEND]->(nMichael)
-,(nCharles)-[:FRIEND]->(nDoug)
+include::scripts/louvain.cypher[tag=create-sample-graph]
 ----
 
 .Running algorithm and streaming results
 [source,cypher]
 ----
-CALL algo.clustering.louvain.stream('User', 'FRIEND', 
-{}) 
-YIELD nodeId, setId
-RETURN nodeId, setId LIMIT 20
+include::scripts/louvain.cypher[tag=stream-sample-graph]
 ----
 
 .Running algorithm and writing back results
 [source,cypher]
 ----
-CALL algo.clustering.louvain('User', 'FRIEND', 
-{write:true, writeProperty:'community'}) 
-YIELD nodes, communityCount, iterations, loadMillis, computeMillis, writeMillis 
+include::scripts/louvain.cypher[tag=write-sample-graph]
 ----
 
 .Results
@@ -134,6 +117,9 @@ YIELD nodes, communityCount, iterations, loadMillis, computeMillis, writeMillis
 
 == Example Usage
 
+
+
+
 == Syntax
 
 .Running algorithm and writing back results
@@ -201,11 +187,11 @@ YIELD nodeId, setId - yields a setId to each node id
 
 * [x] undirected, unweighted
 
-- weightProperty: null
+** weightProperty: null
 
 * [x] undirected, weighted 
 
-- weightProperty : 'weight'
+** weightProperty : 'weight'
 
 == References