1
2
3 """
4 This module is brute force implementation of the 'finite' version of U{RDFS semantics<http://www.w3.org/TR/rdf-mt/>}
5 and of U{OWL 2 RL<http://www.w3.org/TR/owl2-profiles/#Reasoning_in_OWL_2_RL_and_RDF_Graphs_using_Rules>}
6 on the top of RDFLib (with some caveats, see below). Some extensions to these are also implemented.
7 Brute force means that, in all cases, simple forward chaining rules are used to extend (recursively) the incoming graph with all triples
8 that the rule sets permit (ie, the "deductive closure" of the graph is computed).
9 There is an extra options whether the axiomatic triples are added to the graph (prior to the forward chaining step).
10 These, typically set the domain and range for properties or define some core classes.
11 In the case of RDFS, the implementation uses a 'finite' version of the axiomatic triples only (as proposed, for example,
12 by Herman ter Horst). This means that it adds only those C{rdf:_i} type predicates that do appear in the original graph,
13 thereby keeping this step finite. For OWL 2 RL, OWL 2 does not define axiomatic triples formally; but they can be deduced from the
14 U{OWL 2 RDF Based Semantics<http://www.w3.org/TR/owl2-rdf-based-semantics/>} document and are listed in Appendix 6 (though informally).
15 Note, however, that this implementation adds only those triples that refer to OWL terms that are meaningful for the OWL 2 RL case.
16
17
18 Package Entry Points
19 ====================
20
21 The main entry point to the package is via the L{DeductiveClosure<DeductiveClosure>} class. This class should be initialized to control
22 the parameters of the deductive closure; the forward chaining is done via the L{expand<DeductiveClosure.expand>} method.
23 The simplest way to use the package from an RDFLib application is as follows::
24
25 graph = Graph() # creation of an RDFLib graph
26 ...
27 ... # normal RDFLib application, eg, parsing RDF data
28 ...
29 DeductiveClosure(OWLRL_Semantics).expand(graph) # calculate an OWL 2 RL deductive closure of graph
30 # without axiomatic triples
31
32 The first argument of the C{DeductiveClosure} initialization can be replaced by other classes, providing different
33 types of deductive closure; other arguments are also possible. For example::
34
35 DeductiveClosure(OWLRL_Extension, rdfs_closure = True, axiomatic_triples = True, datatype_axioms = True).expand(graph)
36
37 will calculate the deductive closure including RDFS and some extensions to OWL 2 RL, and with all possible axiomatic
38 triples added to the graph (this is about the maximum the package can do…)
39
40 The same instance of L{DeductiveClosure<DeductiveClosure>} can be used for several graph expansions. In other words, the expand function does not change any state.
41
42 For convenience, a second entry point to the package is provided in the form of a function called L{convert_graph<convert_graph>},
43 that expects a directory with various options, including a file name. The function parses the file, creates the expanded graph, and serializes the result into RDF/XML or
44 Turtle. This function is particularly useful as an entry point for a CGI call (where the HTML form parameters are in a directory) and
45 is easy to use with a command line interface. The package distribution contains an example for both.
46
47 There are major closure type (ie, semantic closure possibilities); these can be controlled through the appropriate
48 parameters of the L{DeductiveClosure<DeductiveClosure>} class:
49
50 - using the L{RDFS_Semantics<RDFSClosure.RDFS_Semantics>} class, implementing the U{RDFS semantics<http://www.w3.org/TR/rdf-mt/>}
51 - using the L{OWLRL_Semantics<OWLRL.OWLRL_Semantics>} class, implementing the U{OWL 2 RL<http://www.w3.org/TR/owl2-profiles/#Reasoning_in_OWL_2_RL_and_RDF_Graphs_using_Rules>}
52 - using L{RDFS_OWLRL_Semantics<CombinedClosure.RDFS_OWLRL_Semantics>} class, implementing a combined semantics of U{RDFS semantics<http://www.w3.org/TR/rdf-mt/>} and U{OWL 2 RL<http://www.w3.org/TR/owl2-profiles/#Reasoning_in_OWL_2_RL_and_RDF_Graphs_using_Rules>}
53
54 In all three cases there are other dimensions that can control the exact closure being generated:
55
56 - for convenience, the so called axiomatic triples (see, eg, the U{axiomatic triples in RDFS<http://www.w3.org/TR/rdf-mt/#rdfs_interp>}) are, by default, I{not} added to the graph closure to reduce the number of generated triples. These can be controlled through a separate initialization argument
57 - similarly, the axiomatic triples for D-entailment are separated
58
59 Some Technical/implementation aspects
60 =====================================
61
62 The core processing is done in the in the L{Core<Closure.Core>} class, which is subclassed by the L{RDFS<RDFS_Semantics>} and
63 the L{OWL 2 RL<OWLRL_Semantics>} classes (these two are then, on their turn, subclassed by the
64 L{RDFS + OWL 2 RL Semantics<CombinedClosure.RDFS_OWLRL_Semantics>}) class). The core implements the core functionality of cycling
65 through the rules, whereas the rules themselves are defined and implemented in the subclasses. There are also methods that are executed only once either
66 at the beginning or at the end of the full processing cycle. Adding axiomatic triples is handled separately, which allows a finer user control over
67 these features.
68
69 Literals must be handled separately. Indeed, the functionality relies on 'extended' RDF graphs, that allows literals
70 to be in a subject position, too. Because RDFLib does not allow that, processing begins by exchanging all literals in the
71 graph for bnodes (identical literals get the same associated bnode). Processing occurs on these bnodes; at the end of the process
72 all these bnodes are replaced by their corresponding literals if possible (if the bnode occurs in a subject position, that triple
73 is removed from the resulting graph). Details of this processing is handled in the separate L{Literals Proxies<RDFClosure.Literals.LiteralProxies>}
74 class.
75
76 The OWL specification includes references to datatypes that are not in the core RDFS specification, consequently not
77 directly implemented by RDFLib. These are added in a separate module of the package.
78
79
80 Problems with Literals with datatypes
81 -------------------------------------
82
83 The current distribution of RDFLib is fairly poor in handling datatypes, particularly in checking whether a lexical form
84 of a literal is "proper" as for its declared datatype. A typical example is::
85 "-1234"^^xsd:nonNegativeInteger
86 which should not be accepted as valid literal. Because the requirements of OWL 2 RL are much stricter in this respect, an alternative set of datatype handling (essentially, conversions) had to be implemented (see the L{XsdDatatypes} module).
87
88 The L{DeductiveClosure<DeductiveClosure>} class has an additional instance variable whether
89 the default RDFLib conversion routines should be exchanged against the new ones. If this flag is set to True and instance creation (this is
90 the default), then the conversion routines are set back
91 to the originals once the expansion is complete, thereby avoiding to influence older application that may not work properly with the
92 new set of conversion routines.
93
94 If the user wants to use these alternative lexical conversions everywhere in the application, then
95 the L{use_improved_datatypes_conversions<DeductiveClosure.use_improved_datatypes_conversions>} method can be invoked. That method changes
96 the conversion routines and, from that point on, all usage of L{DeductiveClosure<DeductiveClosure>} instances will use the
97 improved conversion methods without resetting them. Ie, the code structure can be something like::
98 DeductiveClosure().use_improved_datatypes_conversions()
99 ... RDFLib application
100 DeductiveClosure().expand(graph)
101 ...
102 The default situation can be set back using the L{use_rdflib_datatypes_conversions<DeductiveClosure.use_improved_datatypes_conversions>} call.
103
104 It is, however, not I{required} to use these methods at all. Ie, the user can use::
105 DeductiveClosure(improved_datatypes=False).expand(graph)
106 which will result in a proper graph expansion except for the datatype specific comparisons which will be incomplete.
107
108
109
110 Problems with Literals with datatypes
111 -------------------------------------
112
113 The current distribution of RDFLib is fairly poor in handling datatypes, particularly in checking whether a lexical form
114 of a literal is "proper" as for its declared datatype. A typical example is::
115 "-1234"^^xsd:nonNegativeInteger
116 which should not be accepted as valid literal. Because the requirements of OWL 2 RL are much stricter in this respect, an alternative set of datatype handling (essentially, conversions) had to be implemented (see the L{XsdDatatypes} module).
117
118 The L{DeductiveClosure<DeductiveClosure>} class has an additional instance variable whether
119 the default RDFLib conversion routines should be exchanged against the new ones. If this flag is set to True and instance creation (this is
120 the default), then the conversion routines are set back
121 to the originals once the expansion is complete, thereby avoiding to influence older application that may not work properly with the
122 new set of conversion routines.
123
124 If the user wants to use these alternative lexical conversions everywhere in the application, then
125 the L{use_improved_datatypes_conversions<DeductiveClosure.use_improved_datatypes_conversions>} method can be invoked. That method changes
126 the conversion routines and, from that point on, all usage of L{DeductiveClosure<DeductiveClosure>} instances will use the
127 improved conversion methods without resetting them. Ie, the code structure can be something like::
128 DeductiveClosure().use_improved_datatypes_conversions()
129 ... RDFLib application
130 DeductiveClosure().expand(graph)
131 ...
132 The default situation can be set back using the L{use_rdflib_datatypes_conversions<DeductiveClosure.use_improved_datatypes_conversions>} call.
133
134 It is, however, not I{required} to use these methods at all. Ie, the user can use::
135 DeductiveClosure(improved_datatypes=False).expand(graph)
136 which will result in a proper graph expansion except for the datatype specific comparisons which will be incomplete.
137
138
139 @requires: U{RDFLib<https://github.com/RDFLib/rdflib>}, 4.0.0 and higher
140 @requires: U{rdflib_jsonld<https://github.com/RDFLib/rdflib-jsonld>}
141 @license: This software is available for use under the U{W3C Software License<http://www.w3.org/Consortium/Legal/2002/copyright-software-20021231>}
142 @organization: U{World Wide Web Consortium<http://www.w3.org>}
143 @author: U{Ivan Herman<a href="http://www.w3.org/People/Ivan/">}
144
145 """
146
147
148
149 __version__ = "5.0"
150 __author__ = 'Ivan Herman'
151 __contact__ = 'Ivan Herman, ivan@w3.org'
152 __license__ = u'W3C® SOFTWARE NOTICE AND LICENSE, http://www.w3.org/Consortium/Legal/2002/copyright-software-20021231'
153
154 import StringIO
155 from types import *
156
157
158 import rdflib
159
160 from rdflib import Literal as rdflibLiteral
161
162 from rdflib import Graph
163
164 import DatatypeHandling, Closure
165 from OWLRLExtras import OWLRL_Extension, OWLRL_Extension_Trimming
166 from OWLRL import OWLRL_Semantics
167 from RDFSClosure import RDFS_Semantics
168 from CombinedClosure import RDFS_OWLRL_Semantics
169 from OWL import imports
170
171
172 RDFXML = "xml"
173 TURTLE = "turtle"
174 JSON = "json"
175 AUTO = "auto"
176 RDFA = "rdfa"
177
178 NONE = "none"
179 RDF = "rdf"
180 RDFS = "rdfs"
181 OWL = "owl"
182 FULL = "full"
183
184 try:
185 from rdflib_jsonld.parser import JsonLDParser
186 from rdflib_jsonld.serializer import JsonLDSerializer
187 from rdflib.plugin import register, Serializer, Parser
188 register('json-ld', Parser, 'rdflib_jsonld.parser', 'JsonLDParser')
189 register('json-ld', Serializer, 'rdflib_jsonld.serializer', 'JsonLDSerializer')
190 json_ld_available = True
191 except:
192 json_ld_available = False
238
241 """Interpret the owl import statements. Essentially, recursively merge with all the objects in the owl import statement, and remove the corresponding
242 triples from the graph.
243
244 This method can be used by an application prior to expansion. It is I{not} done by the the L{DeductiveClosure} class.
245
246 @param iformat: input format; can be one of L{AUTO}, L{TURTLE}, or L{RDFXML}. L{AUTO} means that the suffix of the file name or URI will decide: '.ttl' means Turtle, RDF/XML otherwise.
247 @param graph: the RDFLib Graph instance to parse into.
248 """
249 while True:
250
251 all_imports = [t for t in graph.triples((None, imports, None))]
252 if len(all_imports) == 0 :
253
254 return
255
256 for t in all_imports : graph.remove(t)
257
258 for (s, p, uri) in all_imports :
259
260
261 if isinstance(uri, rdflibLiteral):
262 __parse_input(iformat, str(uri), graph)
263 else :
264 __parse_input(iformat, uri, graph)
265
269 """
270 Return the right semantic extension class based on three possible choices (this method is here to help potential users, the result can be
271 fed into a L{DeductiveClosure} instance at initialization)
272 @param owl_closure: whether OWL 2 RL deductive closure should be calculated
273 @type owl_closure: boolean
274 @param rdfs_closure: whether RDFS deductive closure should be calculated. In case C{owl_closure==True}, this parameter should also be used in the initialization of L{DeductiveClosure}
275 @type rdfs_closure: boolean
276 @param owl_extras: whether the extra possibilities (rational datatype, etc) should be added to an OWL 2 RL deductive closure. This parameter has no effect in case C{owl_closure==False}.
277 @param trimming : whether extra trimming is done on the OWL RL + Extension output
278 @return: deductive class reference or None
279 @rtype: Class type
280 """
281 if owl_closure:
282 if owl_extras:
283 if trimming:
284 return OWLRL_Extension_Trimming
285 else :
286 return OWLRL_Extension
287 else :
288 if rdfs_closure:
289 return RDFS_OWLRL_Semantics
290 else :
291 return OWLRL_Semantics
292 elif rdfs_closure:
293 return RDFS_Semantics
294 else :
295 return None
296
300 """
301 Entry point to generate the deductive closure of a graph. The exact choice deductive
302 closure is controlled by a class reference. The important initialization parameter is the C{closure_class}: a Class object referring to a
303 subclass of L{Closure.Core}. Although this package includes a number of
304 such subclasses (L{OWLRL_Semantics}, L{RDFS_Semantics}, L{RDFS_OWLRL_Semantics}, and L{OWLRL_Extension}), the user can use his/her own if additional rules are
305 implemented.
306
307 Note that owl:imports statements are I{not} interpreted in this class, that has to be done beforehand on the graph that is to be expanded.
308
309 @ivar rdfs_closure: Whether the RDFS closure should also be executed. Default: False.
310 @type rdfs_closure: boolean
311 @ivar axiomatic_triples: Whether relevant axiomatic triples are added before chaining, except for datatype axiomatic triples. Default: False.
312 @type axiomatic_triples: boolean
313 @ivar datatype_axioms: Whether further datatype axiomatic triples are added to the output. Default: false.
314 @type datatype_axioms: boolean
315 @ivar closure_class: the class instance used to expand the graph
316 @type closure_class: L{Closure.Core}
317 @cvar improved_datatype_generic: Whether the improved set of lexical-to-Python conversions should be used for datatype handling I{in general}, ie, not only for a particular instance and not only for inference purposes. Default: False.
318 @type improved_datatype_generic: boolean
319 """
320 improved_datatype_generic = False
321
322 - def __init__(self, closure_class, improved_datatypes=True, rdfs_closure=False, axiomatic_triples=False, datatype_axioms=False) :
323 """
324 @param closure_class: a closure class reference.
325 @type closure_class: subclass of L{Closure.Core}
326 @param rdfs_closure: whether RDFS rules are executed or not
327 @type rdfs_closure: boolean
328 @param axiomatic_triples: Whether relevant axiomatic triples are added before chaining, except for datatype axiomatic triples. Default: False.
329 @type axiomatic_triples: boolean
330 @param datatype_axioms: Whether further datatype axiomatic triples are added to the output. Default: false.
331 @type datatype_axioms: boolean
332 @param improved_datatypes: Whether the improved set of lexical-to-Python conversions should be used for datatype handling. See the introduction for more details. Default: True.
333 @type improved_datatypes: boolean
334 """
335 if closure_class is None :
336 self.closure_class = None
337 else :
338 if not isinstance(closure_class, ClassType):
339 raise ValueError("The closure type argument must be a class reference")
340 else :
341 self.closure_class = closure_class
342 self.axiomatic_triples = axiomatic_triples
343 self.datatype_axioms = datatype_axioms
344 self.rdfs_closure = rdfs_closure
345 self.improved_datatypes = improved_datatypes
346
361
362 @staticmethod
369
370 @staticmethod
377
383 """
384 Entry point for external scripts (CGI or command line) to parse an RDF file(s), possibly execute OWL and/or RDFS closures,
385 and serialize back the result in some format.
386 Note that this entry point can be used requiring no entailment at all;
387 because both the input and the output format for the package can be RDF/XML or Turtle, such usage would
388 simply mean a format conversion.
389
390 If OWL 2 RL processing is required, that also means that the owl:imports statements are interpreted. Ie,
391 ontologies can be spread over several files. Note, however, that the output of the process would then include all
392 imported ontologies, too.
393
394 @param options: object with specific attributes, namely:
395 - options.sources: list of uris or file names for the source data; for each one if the name ends with 'ttl', it is considered to be turtle, RDF/XML otherwise (this can be overwritten by the options.iformat, though)
396 - options.text: direct Turtle encoding of a graph as a text string (useful, eg, for a CGI call using a text field)
397 - options.owlClosure: can be yes or no
398 - options.rdfsClosure: can be yes or no
399 - options.owlExtras: can be yes or no; whether the extra rules beyond OWL 2 RL are used or not.
400 - options.axioms: whether relevant axiomatic triples are added before chaining (can be a boolean, or the strings "yes" or "no")
401 - options.daxioms: further datatype axiomatic triples are added to the output (can be a boolean, or the strings "yes" or "no")
402 - options.format: output format, can be "turtle" or "rdfxml"
403 - options.iformat: input format, can be "turtle", "rdfa", "json", "rdfxml", or "auto". "auto" means that the suffix of the file is considered: '.ttl'. '.html', 'json' or '.jsonld' respectively with 'xml' as a fallback
404 - options.trimming: whether the extension to OWLRL should also include trimming
405 @param closureClass: explicit class reference. If set, this overrides the various different other options to be used as an extension.
406 """
407
408 def __check_yes_or_true(opt) :
409 return opt is True or opt == "yes" or opt == "Yes" or opt == "True" or opt == "true"
410
411 import warnings
412
413 warnings.filterwarnings("ignore")
414 if len(options.sources) == 0 and (options.text is None or len(options.text.strip()) == 0) :
415 raise Exception("No graph specified either via a URI or text")
416
417 graph = Graph()
418
419
420
421 iformat = AUTO
422 try :
423 iformat = options.iformat
424 except :
425
426 pass
427
428
429 try :
430 if options.source is not None:
431 options.sources.append(options.source)
432 except:
433
434 pass
435
436
437
438 for inp in set(options.sources):
439 __parse_input(iformat, inp, graph)
440
441
442 if options.text is not None:
443 graph.parse(StringIO.StringIO(options.text), format="n3")
444
445
446
447 owlClosure = __check_yes_or_true(options.owlClosure)
448
449 rdfsClosure = __check_yes_or_true(options.rdfsClosure)
450
451 owlExtras = __check_yes_or_true(options.owlExtras)
452 try:
453 trimming = __check_yes_or_true(options.trimming)
454 except :
455 trimming = False
456 axioms = __check_yes_or_true(options.axioms)
457 daxioms = __check_yes_or_true(options.daxioms)
458
459 if owlClosure:
460 interpret_owl_imports(iformat, graph)
461
462
463 graph.bind("owl", "http://www.w3.org/2002/07/owl#")
464 graph.bind("xsd", "http://www.w3.org/2001/XMLSchema#")
465
466
467
468 if closureClass is not None :
469 closure_class = closureClass
470 else :
471 closure_class = return_closure_class(owlClosure, rdfsClosure, owlExtras, trimming)
472
473 DeductiveClosure(closure_class, improved_datatypes=True, rdfs_closure=rdfsClosure, axiomatic_triples=axioms, datatype_axioms=daxioms).expand(graph)
474
475 if options.format == TURTLE:
476 return graph.serialize(format="turtle")
477 elif options.format == JSON:
478 if json_ld_available :
479 return graph.serialize(format="json-ld")
480 else:
481 raise Exception("JSON-LD serializer is not available")
482 else:
483 return graph.serialize(format="pretty-xml")
484