Having executed these methods with the shift+enter or Run command from the main menu, we can now do a quick inspection to see if everything has loaded properly. We again use a listing of subclasses of the Products class:

Like for classes, there are multiple methods for creating a new instance. Here is another form:

The .instances() class method can be used to iterate through all Instances of a class (including its subclasses). It returns a generator:

Go ahead and substitute ‘Products‘ for the class name above to verify it also works on subclasses.

As we learned in the last installment, we can also delete the instance and all of its internal references:

Data and Object Properties

Recall that the basic statement (assertion) in RDF, the base language of our ontologies, is a ‘triple‘. The triple has the form of subject – property – object (or s-p-o; also, predicate or relation is a synonym for property). Data and object properties may be inferred over in a W3C ontology. Object properties require a defined and named object (instance or class) as its object. Data properties require a datatype as its object, including strings. Annotation properties (see below) also take strings or URIs as their object, but can not be inferred over.

One creates properties in a similar manner to classes. In this example, we are creating the has_color sub-property of the qualities property, itself a sub-property of an ObjectProperty:

You will note in this example that we used the underscore character to separate our names to make the property name more readable. Actually, in KKO (and KBpedia) our preferred convention is to use CamelCase with classes initially capitalized, and properties initially lower case. We do not use the underscore. What is important to note here, however, is that these are only best-practice conventions. There is no absolute requirement for classes, individuals or properties to assume a particular naming form.

To see that our new property has been properly added, we can do our standard list method:

We can delete properties and all internal references with the destroy_entity method. Further, we can make domain and range assignments and other specifications of properties, some addressed below, the others covered in the Additional Documentation section.

We can confirm the properties assigned to the computer instance with the get_properties method:

We can also see the actual altLabels were properly assigned:

You will note that the first label in the example above is shown with double-quotes in order to properly account for the single quote in the label for possession.

Like all properties in the system we can get a listing of the properties and classes that have assigned values for that property with this form of .get_relations method:

• When using a cell in Markdown mode for narratives, it is sometimes useful to be able to add HTML code directly. A nice Jupyter Notebook WYSIWYG assistant is:
  • https://github.com/genepattern/jupyter-wysiwyg; install via:

    
    
    conda install -c genepattern jupyter-wysiwyg 

    However, after a period of time I reversed that position, since I found using the assistant caused all of the cell code to be converted to HTML vs Markdown. It is actually easier to use Markdown for simple HTML formatting
• I tend to keep only one or two Notebook pages active at a time (closing out by first File → Save and Checkpoint, and then File → Close and Halt), because not properly closing a Notebook page means it is shown as open next you open the Notebook
• When working with notebook files, running certain cells that cause long lists to be generated or large data arrays to be analyzed can cause the notebook file when saved to grow into a very large size. To keep notebook file sizes manageable, invoke Cell → Current Output → Clear on the offending cells
• When starting a new installment, I tend to first set up the environment by loading all of the proper knowledge bases in the environment, then I am able to start working on new routines.

or type:

Directories and files

Any legal directory or file name is accepted by Python. For Windows, there is often automatic conversion of URI slashes. But non-Linux systems should investigate the specific compatibilities their operating systems have for Python. The differences and nuances are small, but can be a source of frustration if overlooked.

Here are some tips about directories and files:

• Don’t put a """comment""" in the middle of a dictionary listing
• A """comment""" header is best practice for a function likely to be used multiple times
• When reading code, the real action tends to occur at the end of a listing, meaning that understanding the code is often easier working bottom up, as references higher up in the code are more often preliminary or condition setting
• Similarly, many routines tend to build from the inside out. At the core are the key processing and conversion steps. Prior to that is set-up, after that is staging output
• Follow best practices for directory set ups in Python packages (see CWPK #37).

Modules and libraries

A module name must be a valid Python name, limited to letters, digits and ‘_’s.

Modules are the largest construct in Python programs. Python programs consist of multiple module files, either included in the base package, or imported into the current program. Each module has its own container of variables. Variable names may be duplicated across modules, but are distinguished and prevented from name clashes by invoking them with the object (see earlier discussion about the ‘dot notation’). You can also assign imported variables to local ones to keep the dot notation to a minimum and to promote easier to read code.

Namespaces

Python is an object-oriented language, wherein each object in the system has a name identifier. To prevent name conflicts, Python has a namespace construct wherein any grouping of existing object names may be linked to a namespace. The only constraint, within Python’s naming conventions, is that two objects may not share the same name within a given namespace. They may share names between namespaces, but not within.

The namespace construct is both assigned automatically based on certain Python activities, and may also be directly set by assignment. Import events or import artifacts, like knowledge graphs or portions thereto, are natural targets for this convenient namespace convention.

When items are declared and how they are declared informs the basis of a namespace for a given item. If it is only a variable declared in a local context, it has that limited scope. But the variable may be declared in different ways or with different specified scope, in a progression that goes from local to enclosed to global and then built-in. This progression is known as the LEGB scope (see next).

All of this is logical and seemingly straightforward, but what is required by Python in a given context is dependent on just that: context. Sometimes it is difficult to know within a Python program or routine exactly where one is with regard to the LEGB scope. In some cases, prefixes are necessary to cross scope boundaries; in other cases, they are not. About the best rule of thumb I have been able to derive for my own experience is to be aware the the ‘dot notation’ hierarchies in my program objects, and if I have difficulties getting a specific value, to add or reduce more scope definitions in the ‘dot notation’.

To gain a feel for namespace scope, I encourage you to test and run these examples: https://www.programiz.com/python-programming/namespace.

Namespaces may relate to Python concepts like classes, functions, inner functions, variables, exceptions, comprehensions, built-in functions, standard data structures, knowledge bases, and knowledge graphs.

To understand the Python objects within your knowledge graph namespace, you can Run the following cell, which will bring up a listing of objects in each of the imported files via its associated namespace:

You’ll see that each of the major namespaces (sometimes ontologies) list out their internal objects as imported. You may pick any of these objects, and then inspect its attributes:

You can always return to this page to get a global sense of what is in the knowledge graph. Similarly, you may import the cowpoke package (to be defined soon!) or any other Python package and inspect its code contents in the same manner. So, depending on how you lay out your namespaces, you may readily segregate code from knowledge graph from knowledge base, or whatever distinctions make sense for your circumstance.

LEGB Rule

The scope of a name or variable depends on the place in your code where you create that variable. The Python scope concept is generally presented using a rule known as the LEGB rule. The letters in the acronym LEGB stand for Local, Enclosing, Global, and Built-in scopes. A variable is evaluated in sequence in order across LEGB, and its scope depends on the context in which it was initially declared. A variable does not apply beyond the scope in which it was defined. One typical mistake, for example, is to declare a local variable and then assume it applies outside of its local scope. Another typical mistake is to declare a local variable that has the same name as one in a broader context, and then to wonder why it does not operate as declared when in a broader scope.

Probably the safest approach to the LEGB scope is to be aware of variables used in the core Python functions (‘built-ins’) or those in imported modules (the ‘global’ ones) and to avoid them in new declarations. Then, be cognizant that what you declare in local routines only apply there, unless you take explicit steps (through declaration mechanisms) or the use of namespace and dot notation to make your intentions clear.

Setting Proper Data Type

Within owlready2, classes and properties are defined and treated as Python classes. Thus, when you retrieve an item or want to manipulate an item, the item needs to be specified as a proper Python class to the system. However, in moving from format to format or doing various conformance checks, the representation may come into the system as a string or list object. Knowing what representation the inputs are compared with the desired outputs is critical for certain activities in cowpoke. So, let’s look at the canoncial means of shifting data types when dealing with listings of KBpedia classes.

From Python Class to String

Much of the staging of extractions is manipulating labels as strings after retrieving the objects as classes from the system. There is a simple iterator that allows us to obtain sets of classes, loop over them, and convert each item to a string in the process:

 new_str_items = []
 for item in loop:
     a_item = item.curResource             # Retrieves curResource property for item
     a_item = str(a_item)                  # Converts item to string                
     new_str_items.append(a_item)          # Adds to new string list

If you have nested items within loops, you can pick them up using the enumerate in the loop specification.

From Sting to Python Class

The reverse form has us specifying a string and a namespace, from which we obtain the class data type:

 for item in loop:
     var1 = getattr(rc, str_item)   # Need to remove prefix and get from proper namespace (RC)
     var2 = getattr(rc, str_parent) # May need to do so for parent or item for property
     var1.is_a.append(var2)

The general challenge in this form is to make sure that items and parents are in the form of strings without namespaces, and that the proper namespace is referenced when retrieving the actual attribute value. Many code examples throughout show examples of how to test for trap multiple namespaces.

Our previous installments of the Cooking with Python and KBpedia series relied on the full knowledge graph, kbpedia_reference_concepts.owl. That approach was useful to test out whether our current Python and Jupyter Notebook configurations were adequate to handle the entire 58,000 reference concepts (RCs) in KBpedia. However, a file that large makes finding and navigating stuff a bit harder. For this installment, and a few that come thereafter, we will restrict our example to the much smaller upper ontology to KBpedia, KKO (Kbpedia Knowledge Ontology). This ontology only has hundreds of concepts, but has the full suite of functionality and component types found in the full system.

In today’s installment we will apply some of the basic commands in owlready2 we learned in the last installment. Owlready2 is the API to our OWL2 knowledge graphs. In today’s installment we will explore the standard CRUD (create-read-update-delete) actions against the classes (reference concepts) in our graph. Since our efforts to date have focused on the R in CRUD (for reading), our emphasis today will be on class creation, updates and deletions.

Remember, you may find the KKO reference file that we use for this installment, kko.owl where you first stored your KBpedia reference files. (What I call main in the code snippet below.)

Again, you shift+enter or pick Run from the main menu to execute the cell contents. (If you chose to post the code lines to screen, you may clear the file listing from the screen by choosing Cell → All Output → Clear.)

We will next consolidate multiple steps from the prior installment to make absolute file references for the imported SKOS ontology and then to actually load the files:

We can also confirm that the two additional ontologies have been properly imported:

Further, we know that KKO has a class called Products. We also want to see if our file load has properly captured its subClassOf relationships. (In its baseline configuration KKO Products has three sub-classes: Primary ..., Secondary ..., and Tertiary ....) We will return to this cell below multiple times to confirm some of the later steps:

In the second method, we bypass the initial Thing assignment and directly assign the new class WooFoo:

In the third of our examples, we instead use the native Python method of ‘types‘ to do the assignment directly. This can be a useful approach when we are wanting to process longer lists of assignments directly:

Since updates tend to occur more for object properties and values, we discuss these options further two installments from now.

Of course, other functions are available for the use of classes and individuals. See the Additional Documentation below for links explaining these options.

Note during a save specification that you may also indicate the format of the written ontology. We have been using ‘rdfxml‘ as our standard format, but you may also use ‘ntriples‘ (or others that may arise over time for the application).

After you shift+enter to execute the cell contents, or pick Run from the main menu, you will see a rather long file listing. To clear it from the screen, run Cell → All Output → Clear.

The KBpedia full ontology also depends on (‘imports’) two other files: the KBpedia upper ontology, KKO (kko.owl), and the SKOS language that we use for annotations (http://www.w3.org/2004/02/skos/core). The KBpedia full ontology also depends on (‘imports’) two other files: the KBpedia upper ontology, KKO (kko.owl), and the SKOS language that we use for annotations (http://www.w3.org/2004/02/skos/core). As we described in the prior CWPK installment, you will need to download the raw version of kko.owl, and store it in the same location as kbpedia_reference_concepts.owl downloaded in the prior installment. There is no need to download the skos file.

Again, we give names to their absolute file addresses, only this time we do not echo their file listings to screen:

Note one file (kko) is on disk and we use the native Windows file path (with backward slashes) by invoking the ‘r‘ (raw) command switch: r'C:\1-PythonProjects\kbpedia\sandbox\kko.owl'. The other skos file is found off of the Web.

MyBinder Option

If you are running this notebook online, do NOT run the above routines, since the load routines vary somewhat. Please use the following instructions.

To view this file, run the next cell, otherwise, run the following cells to load all graphs.

The asterisk (*) after the import statement means that we want to load all modules in the owlready2 package. One of those functions is get_ontology. We have also given our main ontology the name of onto

We can also place all three commands into a single cell. Note, however, that if you run this consolidated command, that you will see double listings when you invoke print(onto.imported_ontologies) below. In other words, while there is no harm to loading a file again, you are better off only doing the consolidated alone or the three separate loads alone.

You can run the command below to see this behavior. You should then log off or Kernal → Restart & Clear Output and then only load one set before proceeding further.

Let’s now test to make sure the ontologies have been loaded OK by checking to see if the base IRIs for all graphs are recognized by the system:

We can also confirm that the two additional ontologies have been properly imported:

Now, this listing above is interesting. You will see a few warning messages at the top of the listing. This is due to the fact that a very few KBpedia RCs share the same subClasses, making them essentially equivalent classes. The ones listed here are purposeful and do no harm; generally, however, you want to avoid such duplication.

Next, if you scroll down to the bottom you will see that the listing is actually truncated at the bottom with the [...] indicator. If you were to copy the output listing to an editor you would see there are exactly 1000 items in this list, the default limit for such listings when you first install Jupyter Python. But we know there are more than 58,000 RCs (classes) in the system. To see a complete listing we will need to make a change to our notebook’s configuration file.

First, you need to exit this notebook and at your command line create an IPython config file if you don’t already have one:

ipython profile create

Once generated, this config file can be found in your user’s directory (C:\Users\mike.ipython\profile_default\ipython_config.py in my case). Make sure and note the dot in the .ipython directory. Navigate to your appropriate directory and edit this config file to remove the comment for this line and to set the value to ‘0’:

c.PlainTextFormatter.max_seq_length = 0

(which, again, is set to 1000 by default.)

Restart your notebook instance after making this change and run the cells above again. You will now see the full listing of RCs in KBpedia. (A hat tip to StackOverflow for this snippet.)

But, hmmm, where are the classes from the kko and skos imports? All we are showing in this list are the RCs from the default ontology of main.

One of the interesting aspects of owlready2 is its use of what it calls ‘worlds’ to isolate given items into separate namespaces. This allows different projects to be held in memory at one time, including even different open and closed reasoning that may be applied. (A topic for a later installment.)

Now, it is the case that we loaded all three of our ontologies without specifying different ‘world’ namespaces. To see the three combined, let’s actually specify the ‘world’ that is currently active, which is called ‘default’ if not otherwise specified. Rather than our previous onto namespace, let’s now explicitly refer to this default:

Note the full IRI listings for the kko classes at the bottom of the listing. This signals a bit of a namespace recognition problem, that we will address in a few moments.

We can use this same dot approach of namespace.method for other components in our system. (See Additional Documentation below for a listing of such methods available in owlready2.)

In the two examples below we list the annotation properties for both the main (onto) and skos namespaces:

Now, one of the RCs we see in our listing is for Eutheria, the technical name for mammals. Using the same format as above we want to get a listing of its subclasses, but adding our rc. prefix in fact does not work:

Hmmm, it is not immediately clear why we get the error that ‘rc‘ is not defined.

If we look at the KBpedia Web site we see, in fact, that our prefix to individual RCs is actually http://kbpedia.org/kko/rc/ with the trailing ‘/’. owlready2 documentation indicates it uses the ‘#’ sign as its default separator for vocabulary terms. Since our system uses the ‘/’, perhaps something got screwed up in how owlready2 recognizes our namespace.

One way to deal with this potential problem explicitly is to make a direct assignment for what the IRI prefix should be. We do so via this command, where we are directing the prefix ‘rc‘ to recognize our standard base IRI:

With this new namespace assignment, we run the command again:

Voilà! We now have the proper registry of the namespace in our system.

Along with absolute file names that we reference via assigned names, it is also good practice to make sure that our namespaces are properly registered before beginning work with owlready2. (I may discover better tips as we move ahead that I will post when and if discovered.)

We can also check out the properties for this concept:

And, pull out our definition for the concept:

There is also the alternate IRI sytax for identifying a class, in this case to obtain all of the children, grandchildren, etc., of our Mammal concept (which produces a long list of mammals!):

Since we have used the Python list method quite a bit, we can get direct assistance about the method with the standard command:

We can also conduct searches of our RCs, including with wildcards (*) in either the pre or post positions:

Another helpful command in owlready2 is to instruct the system what paths to search when it looks for referenced names:

And, we can get a listing of all such paths registered:

Lastly, for this installment, we can close by saving the ontology we have been working on, being careful in this instance to give a different file name to prevent inadvertent overwriting of our initial input file:

To properly exit the notebook, first pick File → Save and Checkpoint, then File → Close and Halt. This will return you to the main Jupyter files screen, from which you Quit and then close the tab. Return to the command window and finish shutting down the service with ctrl+c. That will return you to the command prompt, from which you may again start up a ‘Jupyter Notebook‘ instance.