SIOC Post profile for "AI3:::Adaptive Information"
type: Document
description: A SIOC profile describes the structure and contents of a weblog in a machine readable form. For more information please refer to http://sioc-project.org/.
primary topic: CWPK #41: Optimizations and Property Structure Ingest
generator agent: http://rdfs.org/sioc/wp-sioc.php

CWPK #41: Optimizations and Property Structure Ingest
type: Post
has container: weblog
has creator: Mike Bergman
maker: Mike Bergman
created: 2020-09-23T14:42:00Z
content: Trying to Overcome Some Performance Problems and Extend into Property Structure Up until the last installment in this Cooking with Python and KBpedia series, everything we did performed quickly in our interactive tests. However, we have encountered some build bottlenecks with our full build routines. If we use the single Generals typology (under which all other KBpedia typologies reside and includes all class structure definitions), the full build requires 700 minutes! Worse, if we actually loop over all of the constituent typology files (and exclude the Generals typology), the full build requires 620 minutes! Wow, that is unacceptable. Fortunately, we do not need to loop over all typology files, but this poor performance demands some better ways to approach things. So, as we continue our detour to a full structure build, I wanted to test some pretty quick options. I also thought some of these tests have use in their own right apart from these performance questions. Tests with broader usefulness we will add to a new utils module in cowpoke. Some of the tests we will look at include: Add a memory add-in to Jupyter Notebook Use sqlite3 datastore rather than entirely in-memory Saving and reloading between passes Removing duplicates in our input build files Creating a unique union of class specifications across typologies, or Some other ideas that we are deferring for now. After we tweak the system based on these tests, we resume our structure building quest, now including KBpedia properties, to complete today's CWPK installment. Standard Start-up We invoke our standard start-up functions. We have updated our 'full' switch to 'start', and have cleaned out some earlier initiatizations that were not actually needed: Which environment? The specific load routine you should choose below depends on whether you are using the online MyBinder service or local files. The example below is based on using local files, which given the complexity of the routines that are emerging, is …
encoded: Trying to Overcome Some Performance Problems and Extend into Property Structure Up until the last installment in this Cooking with Python and KBpedia series, everything we did performed quickly in our interactive tests. However, we have encountered some build bottlenecks with our full build routines. If we use the single Generals typology (under which all other KBpedia typologies reside and includes all class structure definitions), the full build requires 700 minutes! Worse, if we actually loop over all of the constituent typology files (and exclude the Generals typology), the full build requires 620 minutes! Wow, that is unacceptable. Fortunately, we do not need to loop over all typology files, but this poor performance demands some better ways to approach things. So, as we continue our detour to a full structure build, I wanted to test some pretty quick options. I also thought some of these tests have use in their own right apart from these performance questions. Tests with broader usefulness we will add to a new utils module in cowpoke. Some of the tests we will look at include: Add a memory add-in to Jupyter Notebook Use sqlite3 datastore rather than entirely in-memory Saving and reloading between passes Removing duplicates in our input build files Creating a unique union of class specifications across typologies, or Some other ideas that we are deferring for now. After we tweak the system based on these tests, we resume our structure building quest, now including KBpedia properties, to complete today’s CWPK installment. Standard Start-up We invoke our standard start-up functions. We have updated our ‘full’ switch to ‘start’, and have cleaned out some earlier initiatizations that were not actually needed: Which environment? The specific load routine you should choose below depends on whether you are using the online MyBinder service or local files. The example below is based on using local files, which given the complexity of the routines that are emerging, is …
topic: CWPK
topic: KBpedia
topic: Semantic Web Tools
links to: Cooking with Python and KBpedia
links to: cb1-1
links to: cb1-2
links to: cb1-3
links to: cb1-4
links to: cb1-5
links to: cb1-6
links to: cb1-7
links to: cb1-8
links to: cb1-9
links to: cb1-10
links to: cb1-11
links to: cb1-12
links to: cb1-13
links to: cb1-14
links to: cb1-15
links to: cb1-16
links to: cb1-17
links to: cb1-18
links to: cb1-19
links to: cb1-20
links to: cb1-21
links to: cb1-22
links to: cb1-23
links to: cb1-24
links to: cb1-25
links to: cb1-26
links to: cb2-1
links to: cb2-2
links to: cb2-3
links to: cb2-4
links to: cb2-5
links to: cb2-6
links to: cb2-7
links to: cb2-8
links to: cb2-9
links to: cb2-10
links to: cb3-1
links to: nbreuse
links to: Jean-Baptiste Lamy
links to: cb5-1
links to: cb5-2
links to: cb5-3
links to: cb5-4
links to: cb5-5
links to: cb5-6
links to: cb5-7
links to: cb5-8
links to: cb5-9
links to: cb5-10
links to: cb5-11
links to: cb5-12
links to: cb5-13
links to: cb5-14
links to: cb5-15
links to: cb5-16
links to: cb5-17
links to: cb5-18
links to: cb5-19
links to: cb5-20
links to: cb5-21
links to: cb5-22
links to: cb5-23
links to: cb5-24
links to: cb6-1
links to: CWPK #28
links to: cb7-1
links to: cb7-2
links to: cb7-3

Mike Bergman
type: User

Mike Bergman
type: Person

Cooking with Python and KBpedia
label: CWPK listing

Clojure

DSL

transducers-python

ReadTheDocs documentation
label: Documentation

Transducers are Coming

python-transducers

PyPi

ReadTheDocs documentation

Understanding Transducers through Python

pandas

vaex

owlready2 mailing list

keystone

couple

KBpedia

direct download

notify me