the "iceberg visibility" model

1. Introduction

    Search engines, like the popular Google, indexing billions of web resources, on the one hand, people and organisations publishing on the web as many pages as they can in order to be known as broadly as possible, on the other hand, both build the wrong idea that any information or knowledge should be shared publicly.
    The real world is less ideal: a large amount of valuable information or useful knowledge is neither directly accessible by anonymous readers nor by web crawlers. Portals limiting access to registered users and databases giving answers only to well formed queries are the main technical impediments to free browsing or crawling. They are many good economic and social reasons for that.
    The real world wide web is then already divided between a "surface web" and a "deep web". See http://en.wikipedia.org/wiki/Invisible_web. The former serves interchanges between identified partners, the later helps to contact new partners. Even between identified partners all information or knowledge is obviously not be shared totally. Moreover the degree of sharing should be adjustable according to reciprocal trust which could vary in the lifetime of partners.
    Our work which encompass database, portal and weblog technologies for p2p networks, wants to address this requirement. In part 2 we present a model we call "iceberg visibility" to specify views on hierarchies of categories that each pair could use to semantically index web resources of his/her own and to define what he/she wants to share with which other (group of) pair. In part 3 we present a web application that demonstrates the simplicity and the utility of the model for a personalised and privacy preserving weblogging.

2. The « iceberg visibility » model

    We would like to deal with publishers who do not compete firstly for the broadest public – popularity – or market share – sales – but for faithful readers they eventually consider as partners for the high exchange and communication rate – mutual quotations, comments and appraisals – established with.
    Such publishers try to keep in mind distinctions between “the public” and qualified groups of readers and between groups of readers and named – identified – readers. This groups/persons hierarchy should be dynamically set up and updated according to experience of trust in publisher-reader relationship.

    So the first object they will require should be a freely partitionable directory groups/readers per publisher with the following data structure and methods :

a subset of the power set of all registred readers : every reader belonging to a group belongs to all its super-groups too (necessarily then to the top group called “public” or “everyone”)
the groups hierarchy is a tree with the semantics of a class hierarchy
method “create a sub-group” : creates a new named group as a specialisation of an already existing group (top group is created at the publisher’s creation time)
method “register a new reader” : creates a reader’s identification relatively to the publisher’s identification and registers him or her in the top group
method “insert a reader in a group” : registers him/her in this group AND in all super-groups between this one and the parent group where he/she was formerly registered. Exception : this group should not be a super-group of the group where the reader is already registered (if so, go to next method)
method “move up a reader” : deletes registrations of a reader from a sub-group and sub-groups between this one and a new targeted one upward. It is a king of rollback.
method “drop a reader” : deletes all his or her registrations cascading.
all readers of a given group will share the same visibility over publisher’s readable resources
It is worth noticing that it is only the content of a group (list of its readers and cardinality) that makes two sibling sub-groups in the hierarchy (i.e. of the same level) either exclusive or not. There is no other semantics for groups before they will be assigned views on readable publisher’s resources.

The second object, that “selective” publishers would need, is a “topic map” to index their pieces of publication – short pages called hereafter “posts”. One could dream, for this map, of a graphical ontology with many semantic links between topics. Not only it would be complex to manage and display but, what is worse, it will prevent reader for an easy top-down recognition and browsing. Even if a publisher, for an expert usage, would rather deal with a graph structure, he or she should refrain himself or herself in order to offer readers firstly classical tree hierarchy as external views to be easily visited by users broadly used to the expand-collapse method associated to this pattern.
Then, for external specification purpose, let us consider only a topics/posts hierarchy per publisher with the same structure and methods as a directories/files hierarchy where files could have aliases to belong to several sub-directories :

a publisher is created with three default topics: “Transit”, “Top” and “Junk” (or “Trash”)
“Transit” and “Junk” could not have sub-topic, “Transit” will be the container for not yet indexed posts, “Junk” the container for posts detached from all topics
method “create a sub-topic” is straightforward (a topic identification should be an URI, it should have a parent topic (may be “Top”), a topic would be locally named by the publisher who could refers to an external well known topic or term via its URI)
method “edit a topic” too as it only allows to update topic’s attributes, except mandatory URI and parent URI
method “delete a topic” should carefully warn from or should coherently execute cascading
method “create a post” would take advantage of sticking to classical weblogging requirements: it should be identified by an URI with a timestamp in its definition; it can not be deleted, only edited for corrections or comments; by default it is attached to “Transit” topic
method “attach post to topic” will create new indexing topic link without deleting formerly created links
method “detach post from topic” will delete only the link with the selected topic, if it is the only one a new link with the “Junk” topic will be created
method “move topic” is equivalent to “attach to (a new) topic” + “detach from (old) topic”
the semantics of the indexing is entirely related to a “contextual” counting of posts “accessible from” a topic: are accessible from a selected topic all posts attached to this topic or to its parent topics. This semantic would be coherent with a master/detail meaning of topic (like in a table of contents) but could not prohibit by itself attaching very peculiar post to a very general topic. That will remain application defined.

Now our “selective” publisher should be helped for creating views on his or her topics/posts hierarchy, assigning views to groups and then, by union of views build views of groups and views of readers.
We will answer to the publisher needs by a unique structure type for all views: a “tree cut” or “shear” which will define our third, and last, class of objects.
Let us define a “shear view” practically:

For each branch of the topic tree define the deepest visible topic belonging to the view, then all parent topics will belong to the view
The so called "shear" of the topic tree is the sequence of deepest visible topics along the sequence of branches visited in "depth first" order.

a shear view of a topic tree

Propositions:

union of shears is a shear - proof element: there is always a highest-deepest visible topic on every branch;
because it does not exist constraint on assignement of views to (sub-)groups, it could exist some sub-group with a shear view not lower than (i.e. not including) some view of some parent super-group - proof: assign any branch shear view to the top group "everyone" and then assign the single topic "Top" for view of a sub-group of "everyone" - but...
the shear view of a sub-group user is lower than (i.e. includes) the shear view of any user of parents super-groups - proof element: see above the method "insert a reader in a group" !

3. Demonstration prototypes

first prototype (2005), with a hierarchy of reader groups per publisher/pair : the semblog2p project. This prototype was designed only to test the coherence of the model against many scenarii.

current prototype (2006), without hierarchy for reader groups, considering that this hierarchy would be created a posteriori thanks to view definitions on the publisher/pair topic hierarchy : the webograph project. This prototype intend to focus on the most useful property of the model: view definitions by shears allow topic tree restructuration like subtree deplacements, i.e. they are automatically updated like logical views should behave.

4. Conclusion

We think our "iceberg visibility model", with classes of objects defined here above, could be implemented at the database schema level - in an object model or a relational model with integrity constraints - and thus could free the application level from checking the coherency of the publisher policy toward his or her readers. Furthermore it could help - by "try and check" - the publisher to put each of his or her reader in the group appropriate to the view he or she wants to give to him or her. Defining (1) group hierarchy as a subset of the power set of registred users, (2) visible topics as all topics above a choosen topic within every branch of the topic tree hierarchy and (3) "shear" view of any user as the union of views all the groups he or she belongs to have been assigned, are the basic - necessary and sufficient - requirements of our model.

firstly edited on 2005/09/26