SWI-Prolog/XPCE Semantic Web Library
Jan Wielemaker
SWI,
University of Amsterdam
The Netherlands
E-mail: jan@swi.psy.uva.nl
Abstract
This document describes a library for dealing with standards from the
W3C standard for the Semantic Web.
Like the standards themselves (RDF, RDFS and OWL) this infrastructure is
modular. It consists of Prolog packages for reading, querying and
storing semantic web documents as well as XPCE libraries that provide
visualisation and editing. The Prolog libraries can be used without the
XPCE GUI modules. The library can handle upto about 2 million
RDF triples on current commonly used hardware (256MB memory,
Pentium 1.5Ghz).
|
SWI-Prolog has started support for web-documents with the development
of a small and fast SGML/XML parser, followed by an RDF parser (early
2000). With the semweb
library we provide more high level
support for manipulating semantic web documents. The semantic web is a
likely point of orientation for knowledge representation in the future,
making a library designed in its spirit promising.
Central to this library is the module rdf_db.pl
,
providing storage and basic querying for RDF triples. This triple store
is filled using the RDF parser realised by rdf.pl
. The
storage module can quickly save and load (partial) databases. The
modules rdfs.pl
and owl.pl
add querying in
terms of the more powerful RDFS and OWL languages. Module rdf_edit.pl
adds editing, undo, journaling and change-forwarding. Finally, a variety
of XPCE modules visualise and edit the database. Figure figure
1 summarised the modular design.
Figure 1 : Modules for the Semantic Web library |
The central module is called rdf_db
. It provides storage
and indexed querying of RDF triples. Triples are stored as a quintuple.
The first three elements denote the RDF triple. File and
Line provide information about the origin of the triple.
{Subject Predicate Object File Line}
The actual storage is provided by the foreign language (C)
module rdf_db.c
. Using a dedicated C-based implementation
we can reduced memory usage and improve indexing capabilities. (1)
Currently the following indexing is provided.
- Any of the 3 fields of the triple
- Subject + Predicate and Predicate + Object
- Predicates are indexed on the highest property.
In other words, if predicates are related through
subPropertyOf
predicates indexing happens on the most
abstract predicate. This makes calls to rdf_has/4
very efficient.
- Literal Objects are indexed case-insensitive to make
case-insensitive queries fully indexed. See rdf/3.
- rdf(?Subject, ?Predicate,
?Object)
-
Elementary query for triples. Subject and Predicate
are atoms representing the fully qualified URL of the resource. Object
is either an atom representing a resource or
literal(Text)
if the object is a literal value. (2)
If a value of the form NameSpaceID : LocalName
is provided
it is expanded to a ground atom using expand_goal/2.
This implies you can use this construct in compiled code without paying
a preformance penalty. See also section 3.4.
For querying purposes, Object can be of the form
literal(+Query, -Value)
, where Query is one of
- exact(+Text)
-
Perform exact, but case-insensitive match. This query is fully indexed.
- substring(+Text)
-
Match any literal that contains Text as a case-insensitive
substring. The query is not indexed on Object.
- word(+Text)
-
Match any literal that contains Text delimited by a non
alpha-numeric character, the start or end of the string. The query is
not indexed on Object.
- prefix(+Text)
-
Match any literal that starts with Text. This call is
intended for completion. The query is not indexed on Object.
- rdf(?Subject, ?Predicate,
?Object, ?Source)
-
As rdf/3 but in addition
return the source-location of the triple. The source is either a plain
atom or a term of the format
Atom : Integer
where Atom is intended to be used
as filename or URL and Integer for representing the
line-number.
- rdf_has(?Subject,
?Predicate, ?Object, -TriplePred)
-
This query exploits the RDFS
subPropertyOf
relation. It
returns any triple whose stored predicate equals Predicate or
can reach this by following the recursive subPropertyOf
relation. The actual stored predicate is returned in TriplePred.
The example below gets all subclasses of an RDFS (or OWL) class, even if
the relation used is not rdfs:subClassOf
, but a
user-defined sub-property thereof. (3)
subclasses(Class, SubClasses) :-
findall(S, rdf_has(S, rdfs:subClassOf, Class), SubClasses).
|
- rdf_has(?Subject,
?Predicate, ?Object)
-
Same as
rdf_has(Subject, Predicate, Object, _)
.
- rdf_reachable(?Subject,
+Predicate, ?Object)
-
Is true if Object can be reached from Subject
following the transitive predicate Predicate or a
sub-property thereof. When used with either Subject or Object
unbound, it first returns the origin, followed by the reachable nodes in
breath-first search-order. It never generates the same node twice and is
robust against cycles in the transitive relation. With all arguments
instantiated it succeeds deterministically of the relation if a path can
be found from Subject to Object. Searching starts
at Subject, assuming the branching factor is normally lower.
A call with both Subject and Object unbound raises
an instantiation error. The following example generates all subclasses
of
rdfs:Resource
:
?- rdf_reachable(X, rdfs:subClassOf, rdfs:'Resource').
X = 'http://www.w3.org/2000/01/rdf-schema#Resource' ;
X = 'http://www.w3.org/2000/01/rdf-schema#Class' ;
X = 'http://www.w3.org/1999/02/22-rdf-syntax-ns#Property' ;
...
|
- rdf_subject(?Subject)
-
Enumerate resources appearing as a subject in a triple. The main reason
for this predicate is to generate the known subjects without
duplicates as one gets using
rdf(Subject, _, _)
.
As depicted in figure 1, there are two
levels of modification. The rdf_db
module simply modifies,
where the rdf_edit
library provides transactions and undo
on top of this. Applications that wish to use the rdf_edit
layer must never use the predicates from this section directly.
- rdf_assert(+Subject,
+Predicate, +Object)
-
Assert a new triple into the database. This is equivalent to
rdf_assert/4
using SourceRef
user
. Subject and
Predicate are resources. Object is either a
resource or a term literal(Value)
. All arguments are
subject to name-space expansion (see section 3.4).
- rdf_assert(+Subject,
+Predicate, +Object, +SourceRef)
-
As rdf_assert/3,
adding SourceRef to specify the orgin of the triple. SourceRef
is either an atom or a term of the format
Atom:Int where Atom normally refers to
a filename and Int to the line-number where the description
starts.
- rdf_retractall(?Subject,
?Predicate, ?Object)
-
Removes all matching triples from the database. Previous Prolog
implementations also provided a backtracking rdf_retract/3, but
this proved to be rarely used and could always be replaced with
rdf_retractall/3.
As rdf_retractall/4
using an unbound SourceRef.
- rdf_retractall(?Subject,
?Predicate, ?Object, ?SourceRef)
-
As rdf_retractall/4,
also matching on the SourceRef. This is particulary useful to
update all triples coming from a loaded file.
- rdf_update(+Subject,
+Predicate, +Object, +Action)
-
Replaces one of the three fields on the matching triples depending on Action:
- subject(Resource)
-
Changes the first field of the triple.
- predicate(Resource)
-
Changes the second field of the triple.
- object(Object)
-
Changes the last field of the triple to the given resource or
literal(Value)
.
The rdf_db
module can read and write RDF-XML for import
and export as well as a binary format built for quick load and save
described in section 3.3.2. Here are the
predicates for portable RDF load and save.
- rdf_load(+In)
-
Load triples from In, which is either a stream opened for
reading or an atom specifying a filename. This predicate calls
process_rdf/3 to read the source
one description at a time, avoiding limits to the size of the input. If In
is a file, rdf_load/1
provides for caching the results for quick-load using rdf_load_db/1
described below. Caching is activated by creating a directory
.cache
(or _cache
on Windows) in the directory
holding the .rdf
files. Cached RDF files are loaded at
approx. 25 times the speed of RDF-XML files.
- rdf_save(+File)
-
Save all known triples to the given File.
- rdf_save(+File, +FileRef)
-
Save all triples whose file-part of their SourceRef matches
FileRef to the given File. Saving arbitrary
selections is possible using predicates from section
3.3.1.
- rdf_source(?File)
-
Test or enumerate the files loaded using rdf_load/1.
- rdf_make
-
Re-load all RDF sourcefiles (see rdf_source/1)
that have changed since they were loaded the last time. This implies all
triples that originate from the file are removed and the file is
re-loaded. If the file is cached a new cache-file is written. Please
note that the new triples are added at the end of the database, possibly
changing the order of (conflicting) triples.
Sometimes it is necessary to make more arbitrary selections of
material to be saved or exchange RDF descriptions over an open network
link. The predicates in this section provide for this.
- rdf_save_header(+Stream,
?FileRef)
-
Save an RDF header, with the XML header,
DOCTYPE
,
ENTITY
and opening the rdf:RDF
element with
appropriate namespace declarations. It uses the primitives from section
3.4 to generate the required namespaces and desired short-name.
- rdf_save_footer(+Stream)
-
Close the work opened with rdf_save_header/2.
- rdf_save_subject(+Stream,
+Subject, +FileRef)
-
Save everything known about Subject that matches FileRef.
Using an variable for FileRef saves all triples with
Subject.
Loading and saving RDF format is relatively slow. For this reason we
designed a binary format that is more compact, avoids the complications
of the RDF parser and avoids repetitive lookup of (URL) identifiers.
Especially the speed improvement of about 25 times is worth-while when
loading large databases. These predicates are used for caching by
rdf_load/1 under
certain conditions.
- rdf_save_db(+File)
-
Save all known triples into File. The saved version includes
the
SourceRef information.
- rdf_save_db(+File,
+FileRef)
-
Save all triples with SourceRef FileRef,
regardless of the line-number. For example, using
user
all
information added using rdf_assert/3
is stored in the database.
- rdf_load_db(+File)
-
Load triples from File.
Prolog code often contains references to constant resources in a
known XML namespace. For example,
http://www.w3.org/2000/01/rdf-schema#Class
refers to the
most general notion of a class. Readability and maintability concerns
require for abstraction here. The dynamic and multifile predicate
rdf_db:ns/2 maintains a mapping between short meaningful names and
namespace locations very much like the XML xmlns
construct.
The initial mapping contains the namespaces required for the semantic
web languages themselves:
ns(rdf, 'http://www.w3.org/1999/02/22-rdf-syntax-ns#').
ns(rdfs, 'http://www.w3.org/2000/01/rdf-schema#').
ns(owl, 'http://www.w3.org/2002/7/owl#').
ns(xsd, 'http://www.w3.org/2000/10/XMLSchema#').
ns(dc, 'http://purl.org/dc/elements/1.1/').
ns(eor, 'http://dublincore.org/2000/03/13/eor#').
|
All predicates for the semweb libraries use goal_expansion/2
rules to make the SWI-Prolog compiler rewrite terms of the form
Id : Local
into the fully qualified URL. In addition, the
following predicates are supplied:
- rdf_equal(Resource1,
Resource2)
-
Defined as
Resource1, Resource2 = Resource1, Resource2
As
this predicate is subject to goal-expansion it can be used to obtain or
test global URL values to readable values. The following goal unifies X
with
http://www.w3.org/2000/01/rdf-schema#Class
without more
runtime overhead than normal Prolog unification.
rdf_equal(rdfs:'Class', X)
|
- rdf_register_ns(+Alias,
+URL)
-
Register Alias as a shorthand for URL. Note that
the registration must be done before loading any files using them as
namespace aliases are handled at compiletime through goal_expansion/2.
- rdf_global_id(?Alias:Local,
?Global)
-
Runtime translation between Alias and Local and a
Global URL. Expansion is normally done at compiletime. This
predicate is often used to turn a global URL into a more readable term.
- rdf_global_term(+Term0,
-Term)
-
Expands all Alias:Local in Term0 and
return the result in Term. Use infrequently for runtime
expansion of namespace identifiers.
- rdf_split_url(?Base,
?Local, ?URL)
-
Split a URL into a prefix and local part if used in mode -,-,+ or simply
behave as atom_concat/3 in other
modes. The URL is split on the last
#
or /
character.
This section describes the remaining predicates of the rdf_db
module.
- rdf_node(-Id)
-
Generate a unique reference. The returned atom is guaranteed not to
occur in the current database in any field of any triple.
- rdf_source_location(+Subject,
-SourceRef)
-
Return the source-location as File:Line of the
first triple that is about Subject.
- rdf_generation(-Generation)
-
Returns the Generation of the database. Each modification to
the database increments the generation. It can be used to check the
validity of cached results deduced from the database.
- rdf_statistics(?Statistics)
-
Report statistics collected by the
rdf_db
module. Defined
values for Statistics are:
- lookup(?Index, -Count)
-
Number of lookups using a pattern of instantiated fields. Index
is a term
rdf(S,P,O)
, where S, P and O
are either +
or -
. For example rdf(+,+,-)
returns the lookups with subject and predicate specified and object
unbound.
- properties(-Count)
-
Number of unique values for the second field of the triple set.
- sources(-Count)
-
Number of files loaded through rdf_load/1.
- subjects(-Count)
-
Number of unique values for the first field of the triple set.
- triples(-Count)
-
Total number of triples in the database.
- rdf_match_label(+Method,
+Search, +Atom)
-
True if Search matches Atom as defined by Method.
All matching is performed case-insensitive. Defines methods are:
- exact
-
Perform exact, but case-insensitive match.
- substring
-
Search is a sub-string of Text.
- word
-
Search appears as a whole-word in Text.
- prefix
-
Text start with Search.
This RDF low-level module has been created after two year
experimenting with a plain Prolog based module and a brief evaluation of
a second generation pure Prolog implementation. The was to be able to
handle upto aboud 2 million triples on standard (notebook) hardware and
deal efficiently with subPropertyOf
which was identified as
a crucial feature of RDFS to realise fusion of different data-sets.
The following issues are identified and not solved in suitable
manner.
- Logical update
-
as provided by Prolog means that active queries are not affected by
subsequent modification of the database. The current C-based
implementation adheres the immediate update model, mainly
because the current foreign language interface does not provide the
required information to realise logical updates in C.
- Property hierarchy
-
The system currently cannot deal with properties that have multiple
parents if not all parents ultimately have the save root. I.e. there
must be a single root property for each property hierarchy. Although the
design accomodates for this case, it has not yet been implemented.
subPropertyOf
of subPropertyOf
-
is not supported.
The library(rdfs)
library
adds interpretation of the triple store in terms of concepts from
RDF-Schema (RDFS).
The predicates in this section explore the rdfs:subPropertyOf
,
rdfs:subClassOf
and rdf:type
relations. Note
that the most fundamental of these, rdfs:subPropertyOf
, is
also used by rdf_has/[3,4].
- rdfs_subproperty_of(?SubProperty,
?Property)
-
True if SubProperty is equal to Property or Property
can be reached from SubProperty following the
rdfs:subPropertyOf
relation. It can be used to test as well
as generate sub-properties or super-properties. Note that the commonly
used semantics of this predicate is wired into rdf_has/[3,4]. (4). (5)
- rdfs_subclass_of(?SubClass,
?Class)
-
True if SubClass is equal to Class or Class
can be reached from SubClass following the
rdfs:subClassOf
relation. It can be used to test as well as
generate sub-classes or super-classes. (6).
- rdfs_class_property(+Class,
?Property)
-
True if the domain of Property includes Class.
Used to generate all properties that apply to a class.
- rdfs_individual_of(?Resource,
?Class)
-
True if Resource is an indivisual of Class. This
implies
Resource has an
rdf:type
property that refers to
Class or a sub-class thereof. Can be used to test, generate
classes Resource belongs to or generate individuals described
by Class.
The
RDF construct rdf:parseType
=Collection
constructs a list using the rdf:first
and rdf:next
relations.
- rdfs_member(?Resource,
+Set)
-
Test or generate the members of Set. Set is either
an individual of
rdf:List
or rdf:Container
.
- rdfs_list_to_prolog_list(+Set,
-List)
-
Convert Set, which must be an individual of
rdf:List
into a Prolog list of objects.
Textual search is partly handled by the predicates from the
library(rdf_db)
module and its underlying C-library. For
example, literal objects are hashed case-insensitive to speed up the
commonly used case-insensitive search.
- rdfs_label(?Resource,
?Label)
-
Extract the label from Resource or generate all resources
with the given Label. The label is either associated using a
sub-property of
rdfs:label
or it is extracted from the URL
using rdf_split_url/3.
- rdfs_ns_label(?Resource,
?Label)
-
Similar to rdfs_label/2,
but prefixes the result using the declared namespace alias (see section
3.4) to facilitate user-friendly labels in applications using
multiple namespaces that may lead to confusion.
- rdfs_find(+String,
+Description, +Properties, +Method, -Subject)
-
Find (on backtracking) Subjects
that satisfy a search specification for textual attributes. String
is the string searched for. Description is an OWL description
(see section ??) specifying candidate resources. Properties
is a list of properties to search for literal objects where rdfs:label
is replaced by a call to rdfs_label/2
and finally, Method defines the textual matching algorithm.
All textual mapping is performed case-insensitive. The matching-methods
are described with rdf_match_label/3.
The
module rdf_edit.pl
is a layer than encasulates the
modification predicates from section 3.2
for use from a (graphical) editor of the triple store. It adds the
following features:
- Transaction management
Modifications are grouped into transactions to safeguard the
system from failing operations as well as provide meaningfull chunks for
undo and journalling.
- Undo
Undo and redo-transactions using a single mechanism to support
user-friendly editing.
- Journalling
Record all actions to support analysis, versioning, crash-recovery and
an alternative to saving.
Transactions group low-level modification actions together.
- rdfe_transaction(:Goal)
-
Run Goal, recording all modifications to the triple store
made through section 5.2. Execution is
performed as in once/1. If
Goal succeeds the changes are committed. If Goal
fails or throws an exception the changes are reverted.
Transactions may be nested. A failing nested transaction only reverts
the actions performed inside the nested transaction. If the outer
transaction succeeds it is committed normally. Contrary, if the outer
transaction fails, comitted nested transactions are reverted as well.
A successful outer transaction (`level-0') may be undone using
rdfe_undo/0.
The following predicates encapsulate predicates from the rdf_db
module that modify the triple store. These predicates can only be called
when inside a transaction. See rdfe_transaction/1.
- rdfe_assert(+Subject,
+Predicate, +Object)
-
Encapsulates rdf_assert/3.
- rdfe_retractall(?Subject,
?Predicate, ?Object)
-
Encapsulates rdf_retractall/3.
- rdfe_update(+Subject,
+Predicate, +Object, +Action)
-
Encapsulates rdf_update/4.
- rdf_load(+In)
-
Encapsulates rdf_load/1.
This section describes a (yet very incomplete) set of more high-level
operations one would like to be able to perform. Eventually this set may
include operations based on RDFS and OWL.
- rdfe_delete(+Resource)
-
Delete all traces of resource. This implies all triples where
Resource appears as subject, predicate or
object. This predicate starts a transation.
Undo aims at user-level undo operations
from a (graphical) editor.
- rdfe_undo
-
Revert the last outermost (`level 0') transaction (see
rdfe_transaction/1).
Successive calls go further back in history. Fails if there is no more
undo information.
- rdfe_redo
-
Revert the last rdfe_undo/0.
Successive calls revert more rdfe_undo/0
operations. Fails if there is no more redo information.
- rdfe_can_undo
-
Test if there is another transaction that can be reverted. Used for
activating menus in a graphical environment.
- rdfe_can_redo
-
Test if there is another undo that can be reverted. Used for activating
menus in a graphical environment.
Optionally, every action through this
module is immediately send to a
journal-file. The journal provides a full log of all actions
with a time-stamp that may be used for inspection of behaviour, version
management, crash-recovery or an alternative to regular save operations.
- rdfe_open_journal(+File,
+Mode)
-
Open a existing or new journal. If Mode equala
append
and File exists, the journal is first replayed. See
rdfe_replay_journal/1.
If Mode is write
the journal is truncated if it
exists.
- rdfe_close_journal
-
Close the currently open journal.
- rdfe_current_journal(-Path)
-
Test whether there is a journal and to which file the actions are
journalled.
- rdfe_replay_journal(+File)
-
Read a jorunal, replaying all actions in it. To do so, the system reads
the journal a transaction at a time. If the transaction is closed with a commit
it executes the actions inside the journal. If it is closed with a rollback
or not closed at all due to a crash the actions inside the journal are
discarded. Using this predicate only makes sense to inspect the state at
the end of a journal without modifying the journal. Normally a journal
is replayed using the
append
mode of rdfe_open_journal/2.
To realise
a modular graphical interface for editing the triple store, the system
must use some sort of event mechanism. This is implemented by
the XPCE library library(broadcast)
which is described in
the XPCE
User Guide. In this section we describe the terms brodcasted by the
library.
- rdf_transaction(+Id)
-
A `level-0' transaction has been committed. The system passes the
identifier of the transaction in Id. In the current
implementation there is no way to find out what happened inside the
transaction. This is likely to change in time.
If a transaction is reverted due to failure or exception no
event is broadcasted. The initiating GUI element is supposed to handle
this possibility itself and other components are not affected as the
triple store is not changed.
- rdf_undo(+Type, +Id)
-
This event is broadcasted after an rdfe_undo/0
or rdfe_redo/0.
Type is one of
undo
or redo
and Id
identifies the transaction as above.
- note-1
-
The orginal implementation was in Prolog. This version was implemented
in 3 hours, where the C-based implementation costed a full week. The
C-based implementation requires about half the memory and provides about
twice the performance.
- note-2
-
The current implementation has no provisions for XML-Schema typed
literals.
- note-3
-
This predicate realises semantics defined in RDF-Schema rather than RDF.
It is part of the
library(rdf_db)
module because the
indexing of this module incorporates the rdfs:subClassOf
predicate.
- note-4
-
BUG: The current implementation cannot deal with cycles
- note-5
-
BUG: The current implementation cannot deal with predicates that are an
rdfs:subPropertyOf
of rdfs:subPropertyOf
, such as owl:samePropertyAs
.
- note-6
-
BUG: The current implementation cannot deal with cycles
- A
-
- atom_concat/3
-
3.4
- B
-
- broadcast
-
5.6
- C
-
- Collection,parseType
-
4.2
- E
-
- event
-
5.6
- expand_goal/2
-
3.1
- G
-
- goal_expansion/2
-
3.4 3.4
- J
-
- journal
-
5 5.5
- O
-
- once/1
-
5.1
- P
-
- parseType,Collection
-
4.2
- process_rdf/3
-
3.3
- R
-
- rdf/3
-
3 3.1
- rdf/4
-
- rdf_assert/3
-
3.2 3.3.2 5.2
- rdf_assert/4
-
3.2
- rdfe_assert/3
-
- rdfe_can_redo/0
-
- rdfe_can_undo/0
-
- rdfe_close_journal/0
-
- rdfe_current_journal/1
-
- rdfe_delete/1
-
- rdfe_open_journal/2
-
5.5
- rdf_equal/2
-
- rdfe_redo/0
-
5.6
- rdfe_replay_journal/1
-
5.5
- rdfe_retractall/3
-
- rdfe_transaction/1
-
5.2 5.4
- rdfe_undo/0
-
5.1 5.4 5.4 5.6
- rdfe_update/4
-
- rdf_generation/1
-
- rdf_global_id/2
-
- rdf_global_term/2
-
- rdf_has/3
-
- rdf_has/4
-
3
- rdf_has/[3,4]
-
4.1 4.1
- rdf_load/1
-
3.3 3.3 3.3.2 3.5 5.2
- rdf_load_db/1
-
3.3
- rdf_make/0
-
- rdf_match_label/3
-
4.3
- rdf_node/1
-
- rdf_reachable/3
-
- rdf_register_ns/2
-
- rdf_retractall/3
-
3.2 5.2
- rdf_retractall/4
-
3.2 3.2
- rdf_save/1
-
- rdf_save/2
-
- rdf_save_db/1
-
- rdf_save_footer/1
-
- rdf_save_header/2
-
3.3.1
- rdf_save_subject/3
-
- RDF-Schema
-
4
- rdfs_class_property/2
-
- rdfs_find/5
-
- rdfs_individual_of/2
-
- rdfs_label/2
-
4.3 4.3
- rdfs_list_to_prolog_list/2
-
- rdfs_member/2
-
- rdfs_ns_label/2
-
- rdf_source/1
-
3.3
- rdf_source_location/2
-
- rdf_split_url/3
-
4.3
- rdfs_subclass_of/2
-
- rdfs_subproperty_of/2
-
- rdf_statistics/1
-
- rdf_subject/1
-
- rdf_update/4
-
5.2
- S
-
- search
-
4.3
- T
-
- transactions
-
5
- U
-
- undo
-
5 5.4