In real world observations, we can conclude that there are a few topics, which have many subscribers, and a few nodes subscribe to many topics. This kind of distributions as Fig. 5 we call “power-law”, and we assume the topic popularity and subscriptions per node follow power-law distribution [4].
Fig. 5 Power law
We also assume that topic subscriptions are correlated, so the subscription sets may overlap. If subscriptions are uncorrelated, the system will fall into an overlay-per topic solution and disconnected components will occur in every single overlay, which will degrades the performance.
real world.
3.2 Design Rationale
For a topic-based publish/subscribe system, following properties should take into concern. These properties affect reliability and effectiveness [5].
• Completeness
All events for a topic should publish to all nodes, which subscribe to it. No nodes will miss these events for the subscribed topic.
• Accuracy
Nodes should not receive the event from the topic it does not subscribe to.
• Node scalability
The nodes subscribe to a topic should scale well as the number of nodes grows large.
• Topic scalability
The topics should scale well as the number of topic grows large.
• Connectivity
All nodes are reachable from any other node, which can ensure completeness.
• Clustering coefficient
A measure of the graph to show how the nodes cluster together. It is the ratio between the number of links that exist among the neighbors of a node and the
total number of links that may exist. This measurement quantifies how close the neighbors are. For practice purpose, it is related to the dissemination cost, because highly clustered sections of the overlay will produce redundant messages. For fault tolerance, highly clustered sections of the overlay tend to easily become disconnected from each other as the graph change, which will degrade overall connectivity. The calculation of clustering coefficient is shown as Fig. 6.
3.3 Link Management
For link management, we always can use global knowledge of the whole system to optimize the overlaps; however, to minimize the number of physical links is shown to be NP-complete [6]. That is, the scalability of the system cannot be satisfied.
The main idea of tStaN is simple: each node samples the subscribers of all the overlays it belongs to in a random manner periodically. In our assumption, subscriptions of a node are correlated, so these sample nodes will likely overlap in different overlays, in other words, topics. For each overlay, the node chooses a set of neighbors from the sampled set deterministically. As mentioned above, the nodes with highly probability will share across overlays; therefore, different logical link among overlays will map to single physical link and share this link. When the amount of topics grows large, the physical links of the network may grow slowly as logical links grows rapidly.
In such design, the correlation of subscriptions will lead to an undesired goal, clustering. It will impact the fitness of the overlay, because nodes will gather together with same subscriptions and increase fault rate. To face this problem, we should uniformly establish links when selecting neighbors, but not in random. By Uniformity for preserving good properties of random overlay [7] and determinism for consistency amount overlays, we can use a deterministic method to decide the nodes we select, and it will usually be the same candidates in different overlays. We can use a weight function to represent the link from node p to node q, and the value of the function is calculated by hash function [8] of the sum of the identifiers in string type. Unlike consistent hashing [9], which takes node distance into account; this value is asymmetry so it can avoid clustering.
3.4 Time-concerned content
For every node, the subscription to a topic can vary in time; it may cancel it now and re-join it later. So, in our assumption, we randomly let these nodes to join and leave for each topic in the future as Fig. 7, and record it as a timeslot to let other nodes to know it. This change make every node should take this into concern when it select neighbors, by compare this timeslot from other nodes to it self’s time slot. Therefore, this selection may guarantee the higher reliability in the future.
Fig. 7 random join-and-leave time slot
3.5 Algorithm
Following is the proposed algorithm, the node periodically choose a node from its neighbors randomly in a topic (called view size), check the time slot is matched and add this node into candidate and pass to other neighbors, it a given TTL to maintain the collecting cost. At last, the set is sent to original node and the new view size is regenerated. By the hash function, node will always choose the same node in different overlay, so the purpose of link sharing is achieved.
tStaN protocol(node p)
periodically
foreach topic t א p.topics do
q = RANDOMNODE(p.views[t])
call q.COLLECTWALK(p, {}, TTL, t, p.slot[t] )
function COLLECTWALK(src, set, ttl, topic,slot) if slot is in p.slot[topic] then
set = set {p}
if ttl>0 then
q = RANDOMNODE(p.views[topic])
call q.COLLECTWALK(src, set, TTL-1, t, slot ) else
call src.COLLECTREPLY(src, topic)
function COLLECTREPLY(set, topic) ViewSize = size(p.views[topic])
List = {q א set p.views[topic] sorted by WEIGHT(q)}
p.views[topic] = first ViewSize nodes from List
function WEIGHT(q)
return HASH( STRINGIFY(p.id) + STRINGIFY(q.id) )