How to – and how not to – build ontologies: the hard road to semantic interoperability Information Models and Ontologies to Enable Digital Engineering Research Workshop -- May 23-24, 2023 Chris Partridge Chief Ontologist , BORO Solutions, UK Structure Background (quick BORO introduction) Information (cultural) evolution: the current scaffolding (the context) Radically new practices (the way forward?) Background Lightning tour BORO Acronym Business Object Reference Ontology BORO is (both): a foundational (or upper) ontology – BORO - and a closely intertwined methodology – bCLEARer for formalising information and eventually grounding it under the foundational ontology Two mutually supporting frameworks The two frameworks validate and inform each other top-down BORO foundational ontology guides the bottom-up framework bottom-up bCLEARer framework validates the whole model early de-risking validates the top-down framework; its completeness and fit BORO FO - top-down framework bCLEARer - bottom-up framework Visualising BORO’s two components a repeated sequence of automated processes: a scalable way to systematically improve semantic maturity can be seen, from an evolutionary perspective, as navigating a fitness landscape data collect collect collect increasing semantic maturity reuse reuse Foundational ontology bCLEARer process leading to the BORO Top Ontology Brief history BORO (ontology and methodology) was originally conceived in the late 1980s to address a particular need for a solid legacy re-engineering process and evolved to resolve semantic issues around interoperating enterprise systems aiming to … enabling higher levels of reuse and, as a consequence, capable of reducing the effort and cost of (re-)developing, maintaining and interoperating enterprise systems. It was eventually publicly documented in (Partridge, 1996). Significant theme that has emerged over time is: the bCLEARer methodology requires new practices the practices lead to (among other things) systematic paradigm shifting examples (references in next slide) 4D extensionalism accounting, agentology/epistemology, constructionalism and nomenclature. Paradigm Shifting --- References {5C22544A-7EE6-4342-B048-85BDC9FD1C3A} 4d extensionalism 1996 Partridge C. Business Objects: Re - Engineering for re - use. Butterworth Heinemann, Oxford accounting 2003 Partridge, C. Shifting the Ontological Foundations of Accounting's Conceptual Scheme. ECAIS accounting 2018 PARTRIDGE, C., KHAN, M., DE CESARE, S., GAILLY, F., VERDONCK, M. AND MITCHELL, A.. Thoroughly Modern Accounting: Shifting to a De Re Conceptual Pattern for Debits and Credits. agentology/ epistemology 2018 PARTRIDGE, C., DECESARE, S., MITCHELL, A., LEON, A., GAILLY, F. AND KHAN, M. Ontology then Agentology: A Finer Grained Framework for Enterprise Modelling. agentology/ epistemology 2009 Lycett and Partridge. The challenge of epistemic divergence in IS development. Mark Lycett, Chris Partridge. Communications of the ACM. Volume 52 Issue 6, June. constructionalism 2017 PARTRIDGE, C., DECESARE, S., MITCHELL, A., GAILLY, F. AND KHAN, M. Developing an ontological sandbox: investigating multi-level modelling’s possible Metaphysical Structures. constructionalism 2019 Partridge, Chris, Andrew Mitchell, Michael Loneragan, et al. Coordinate Systems: Level Ascending Ontological Options". nomenclature 2019 Partridge, C., Mitchell, A., de Cesare, S. Grounding for an Enterprise Computing Nomenclature Ontology. OMG 2010 BORO Top Ontology – some standards 1990 Ongoing BORO development 2000 ISO 15926 : Part 2 IDEAS MODAF/MODEM UPDM 2 2020 UAF DODAF DM2 BORO (FO & bCLEARer) exploitation routes 10 transform assess build has the application already been deployed? is there a commercial application available in the market? has it already been selected? standard needs to be implemented in application? sustain target requires a standard? requirement semantics composition mereology classification external identifiers content standardise configure various entry points general tool – multiple applications Programme Example: Routes Exploitation Breakdown 11 Programme Example: Rough analysis of effort 12 BORO’s evolving challenge In the early 90s we had a challenge we knew how to build a kind of ‘semantic interoperability’, we had the practice but we didn’t know how to share this practice easily with others when showing our practices we were met with a blank, incredulous stare Our practices needed be made shareable common challenge for innovative practices First response (evolutionary adaption (?)): providing explanatory scaffolding Textual explanation: Partridge (1996) Business Objects: Re - Engineering for re - use. Butterworth Heinemann How to evolve from ‘practices’ to ‘praxis’ Question: How to evolve from ‘practices’ to ‘praxis’ What we can do now: practice (n.) early 15c., practise , “practical aspect or application,” … ; from Old French pratiser , from Medieval Latin practicare …). … Sense of “habit, frequent or customary performance” is from c. 1500. … Sense of “action, the process of accomplishing or carrying out” (opposed to speculation or theory ) is from 1530s. …( https://www.etymonline.com/word/practice ) Where we would like to be: praxis (n.) 1580s, “practice or discipline for a specific purpose,” from Medieval Latin praxis “practice, exercise, action” (mid-13c., opposite of theory), from Greek praxis “practice, action, doing,” from stem of prassein , prattein "to do, to act" (see practical). From 1610s as "a collection of examples for practice.” Information (cultural) evolution: the current scaffolding Overview Overall context Context the evolution of life (on the planet Earth) The ‘phylum’ of interest ‘(symbolic) information’ Plainly, this involves cultural evolution Though the early stages also involve biological evolution Evolutionary transmission (over time) Inheritance (persistence) – of characteristics Biological – vertical – genetic – parent to child Cultural – horizontal/oblique – social New characteristics emerge through new practices (ways of doing) And are inherited if the practices are successfully shared E.g. Jablonka , Eva, and Marion J. Lamb. (2005) Evolution in Four Dimensions: Genetic, Epigenetic, Behavioral , and Symbolic Variation in the History of Life Macroevolutionary perspective: Major Evolutionary Transitions Useful to take a transition rather than trends view Major transition == game-changing alterations From our symbolic information perspective Major Evolutionary Transitions Pre-(symbolic)-information Brain (assuming an internal ‘symbolic’ language) Speech (Spoken language) Writing (Written language) Computing (Language processing) E.g. Maynard Smith, John and Eörs Szathmáry , 1995, The Major Transitions in Evolution Macroevolutionary perspective: Transition Characteristics Clear association of information with technology Technology that shifts the symbol processing (in stages) outside the brain/body Co-evolution rather than replacement We still use our brains, talk and write New technology reshapes the old E.g. primary versus secondary orality (Ong, 1977, Rhetoric, Romance and Technology) One technology shift, two stage transition: Technology then interoperability Key point: Advances in technology do not automatically bring its interoperability it is the basis for interoperability but interoperability is not built in Can be a ‘hard road’ to achieving interoperability The classic example for speech is the Tower of Babel narrative in Genesis 11:1–9 This speaks to the power of interoperability and to the curse of not having it. And the LORD said, "Look, they are one people, and they have all one language, and this is only the beginning of what they will do; nothing that they propose to do will now be impossible for them . Come, let us go down and confuse their language so they will not understand each other.” In this myth, implausibly, the journey is ‘backwards’ from being interoperable to not being interoperable A better historical example, where the journey is forward, is printing speech Pre-printing textual practices were oral 20 T he western medieval world was oral “ We know a good deal about the actual procedures that Thomas Aquinas followed in composing his works, thanks ... to the full accounts we have from the hearings held for his canonization. … ... Still stronger is the testimony of Reginald his socius and of his pupils and of those who wrote to his dictation, who all declare that he used to dictate in his cell to three secretaries, and even occasionally to four, on different subjects at the same time . . . No one could dictate simultaneously so much various material without a special grace. Nor did he seem to be searching for things as yet unknown to him; he seemed simply to let his memory pour out its treasures ...” Mary Carruthers, The Book of Memory, 1992. “… composing a text was not writing at all but composing mentally and performing orally and, on occasion, dictating from memory. Carruthers (Carruthers, 1990, p. 6) argues that Aquinas' multi-volume Summa Theologica was produced in just this way: … these highly literate medieval churchmen did their work orally, relying on memory for examining, criticizing and developing ideas rather than relying, as is usually assumed, on the written text. Sermons were composed in the mind and sometimes written down later. Texts were not scrutinized so much as used as a record against which to check memory. Reading was not so much a matter of studying a text as ingesting or internalizing it. Once ingested, it could become the object of meditation and reflection. The scrutinized object was in the mind not in the text.” Olson, The world on paper, 1994. Post-printing textual practices were ‘literate’ What is literacy? “Literacy in Western cultures is not just learning the abc's ; it is learning to use the resources of writing for a culturally defined set of tasks and procedures. All writers agree on this point.” Olson, The world on paper, 1994. In other words, literacy is the communal cultural practice of producing and exploiting textual resources more specifically it involves both collating and creating common access to historical textual information as well as creating new textual information it is a property of societies rather than individuals. from a modern computing perspective, this is a definition of textual semantic interoperability As we have seen, for the emergence of literacy it is necessary but not sufficient for writing technology to emerge A further step is needed 21 Printing: the disruption that enabled literate interoperability Printing enabled radical new practices to emerge allowing the production, checking and revising of historic and new texts to scale. a key element of this was significantly reducing the cost of producing multiple ‘exact’ copies at scale. {5C22544A-7EE6-4342-B048-85BDC9FD1C3A} text directions of communication writing printing brain-to-external (text) writing printing external (text)-to-brain reading (pre-printing) reading (post-printing) legend disruption Lorem ipsum dolor sit amet , consectetur adipiscing elit , sed do Lorem ipsum dolor sit amet , consectetur adipiscing elit , sed do eiusmod tempor incididunt ut labore et Computeracy as an analogue of literacy 23 what is computeracy (computing semantic interoperability)? computeracy is not just the ability to process data; it is the communal cultural practice of producing and exploiting data resources – more specifically it involves both collating and creating access to historical information as well as creating new information as shareable data. It is a property of societies rather than individuals. in a similar situation to writing and printing, the mere availability of computing technology is necessary, but not sufficient to produce computeracy. This requires the development of data sharing techniques and their deployment in cultural practices where both historical and new information is produced as shareable data. The computeracy interoperability test System A System A System B System B System A System A System B System B In a computerate ecosystem, the costs of intra- and inter-operability are roughly comparable, If they are not comparable, this encourages silos intra-operability inter-operability machines talking to other machines machines talking to themselves Economics of interoperability Printing changed the economics of the interoperability of text Significant shifts at many levels Reducing cost of accuracy dramatically changes the landscape What is going to change the economics of data exchange? One way to look at the economics of data exchange: A Coasian theory of the economics of system’s data interoperability Treat a data exchange between systems as a Coasian transaction Reducing the cost of these exchanges changes the nature of the systems Coase, R. H. (November 1937). "The Nature of the Firm". Economica . Macroevolutionary perspective: The evolutionary contingency question If we want the benefits of computing semantic interoperability (computeracy) one key question is: Is the evolutionary opportunity for computeracy contingent*? Will it necessarily evolve? Can we just saunter along and it will eventually happen? Some historical evidence, suggests this kind of stage transition may be contingent: In the prior case of printing, (Olson 1994) compares the evolution of literacy in Western Europe and China He notes it evolved in Western Europe, it didn’t evolve in China Even if contingent, maybe we don’t need to be concerned We might already be on the right evolutionary trajectory? however, it doesn’t look good. early signs of progress are patchy. So, how can we adopt a strategy of facilitating the evolution of computeracy? To put ourselves on the ‘right’ evolutionary trajectory One possibility: promote the adoption of the practices that encourage that trajectory * See e.g. : https://plato.stanford.edu/entries/macroevolution/#HistCont - 6. Historical Contingency Macroevolutionary perspective: Persisting transition – planning inheritance systems To be successful a transition needs to persist – be inherited We can plan for it to persist – be inherited The planning will involve ‘inheritance systems’* such as: Genetic (Epigenetic) (mostly vertical) Behavioural (both horizontal and vertical) Symbolic (both horizontal and vertical) ** During the transition (in our experience) Knowledge is not explicit enough to put enough of it into symbolic inheritance systems Some inheritance needs to be behavioural (imitation) transmitting Information through social learning reconstructing and imitating * See e.g. : https://plato.stanford.edu/entries/inheritance-systems/ ** Or, Jablonka, E., & Lamb, M. J. ( 2005 ) Evolution in Four Dimensions: Genetic, Epigenetic, Behavioural, and Symbolic Variation in the History of Life Radically New Practices Kuratowski - strangeness leading to distrust The stimulus to the investigations from which the theory of sets grew was given by problems of analysis, the establishing of the foundations of the theory of irrational numbers, the theory of trigonometric series, etc. However, the further development of set theory went initially in an abstract direction, little connected with other branches of mathematics . This fact, together with a certain strangeness of the methods of set theory which were entirely different from those applied up to that time, caused many mathematicians to regard this new branch of mathematics initially with a certain degree of distrust and reluctance . In the course of years, however, when set theory showed its usefulness in many branches of mathematics such as the theory of analytic functions or theory of measure, and when it became an indispensable basis for new mathematical disciplines (such as topology, the theory of functions of a real variable, the foundations of mathematics), it became an especially important branch and tool of modern mathematics. K. Kuratowski , Introduction to Set Theory and Topology, p. 21 Proposed new practices - adoption “Learning” can be defined in a very general way as: an adaptive (usually) change in behaviour that is the result of experience. “Social learning” or, more precisely, “socially mediated learning” is therefore: a change in behaviour that is the result of social interactions with other individuals, usually of the same species. Proposed new practices Some are more strange than others You can judge the strangeness for yourselves and your communities Most are already used in some form or other But no community uses many of them, yet … So, individually not really new But using them together is Starting to adopt any one of them is not particularly difficult Deploying them all effectively is more difficult – a hard road Practices – a little more organised Can be seen as falling under three intertwined headings: Shifting scope Evolution Economics of complex accuracy Shifting scope: Life cycle separation of concerns approach Manage form: Managing the formalisation process Taking control of the form The role of top ontology Work with ALL the data Directed Evolution Factor then refactor Managing the economics of accuracy: Run a paradigm shift factory Scale down to scale up Life cycle separation of concerns approach Radically New Practices: Shifting Scope Life Cycle Separation of concerns approach Common principle in software development System Development Life Cycle e.g. OMG’s MDA Early/left life cycle concerns are not usually well-separated E.g. not many Computation-independent Models (CIM) Unfortunate, as these are where real benefits can be found Where the hard work of ontology lies Life cycle separation of concerns approach: An information framework perspective 34 Information is implemented as a stored representation. Representation Domain Information is typically about a domain Good practice in the development of information systems, is to separate concerns. (See e.g. OMG’s MDA divisions) style.visibility style.visibility style.visibility Manage Form Radically New Practices: Shifting Scope Manage Form: two aspects Two aspects: Managing the formalisation process Taking control of the form In the BORO context bCLEARer is an example of an approach to both aspects Implementations of the BORO Foundation are example of the second aspect Managing the formalisation process Radically New Practices: Shifting Scope: Manage Form Managing the formalisation process Formalisation is: An iterative, (discovery,) empirical exercise should aim to ‘let the data speak’ In evolutionary terms, exploring the paths through the fitness landscape Issues The formalisation process is largely unexamined (AKA unmanaged) typically uncritically gives control of the form to the syntax Potential characterisations formalised dialogue – see Dutilh Novaes , C. (2020). The Dialogical Roots of Deduction: Historical, Cognitive, and Philosophical Perspectives on Reasoning. prediction machines – see Clark A. (2015). Surfing uncertainty: Prediction, action, and the embodied mind. Taking control of the form Radically New Practices: Shifting Scope: Manage Form Taking control of the form: Example: Dissolve the boundaries between schema and data syntax segregates schema and data; Typically; schema is assumed to fix the general semantics even though semantically murky hence the schema dictates the (semantic) form of the data – syntax driven semantics Example issues with current syntax-driven approaches Typology trumps mereology Instantiation is more embedded in the syntax than parthood Single level typology Higher-order types ignored E.g. Partridge et al. (2020) Implicit requirements for ontological multi-level types in the UNICLASS classification Manage form: Taking control of form Need the same control of form as one has for data Simple way around this Situate form in the data {5C22544A-7EE6-4342-B048-85BDC9FD1C3A} Dogs {5C22544A-7EE6-4342-B048-85BDC9FD1C3A} Cats {5C22544A-7EE6-4342-B048-85BDC9FD1C3A} Objects Objects Dogs Cats ? Once one has control of the form, what does one do? One approach is to choose a top ontology {5C22544A-7EE6-4342-B048-85BDC9FD1C3A} Dogs Name Breed Etc. Fido Corgi {5C22544A-7EE6-4342-B048-85BDC9FD1C3A} Objects Objects Dogs Name Breed Fido Corgi Situating the tables in the data Situating the whole schema in the data Finding an ‘appropriate’ top ontology Radically New Practices: Shifting Scope: Manage Form: Taking control of the form A framework for assessing an ontology’s architectural choices https://www.repository.cam.ac.uk/handle/1810/313452 Appendix E: Summary of Framework Assessment Matrix Results 31 ontological choices 37 top ontologies shortlisted and assessed The ontological choices shape the architecture of the ontology web-based: https://digitaltwinhub.co.uk/a-survey-of-top-level-ontologies/#a_survey_of_TLOs_contents 43 Example architectural choice: whether to stratify or unify “4.2.2 Horizontal aspects: stratification versus unification There is a group of fundamental choices that impact the ontological architecture which involves whether or not to make a distinction. If one chooses not to make the distinction, one only introduces a single type. If one chooses to make the distinction, one introduces two types; one for each alternative. The choice boils down to whether to horizontally stratify or unify. One can describe choosing to make the distinction as ‘separating one potentially unified type into two’, creating a horizontal stratification in the hierarchy – and not making the distinction, ‘unifying the potentially separated two types into one’.” 44 Horizontal stratification choices 45 {5C22544A-7EE6-4342-B048-85BDC9FD1C3A} Label Unified type Separate types Stratifying relation spacetime spatio-temporal locations spatial locations, temporal locations spaces are multiply located at times (though this is often a derived relation – from an occupying object’s links to both space and time) locations supersubstantival objects (physical) objects, locations objects are (exactly) located at their locations properties objects substances, properties substances are bearers of properties endurants perdurants continuants, occurrents occurrent is dependent upon continuant immaterial (physical) objects material objects, immaterial objects immaterial objects are part of material objects Spacetime stratification: unifying or stratifying A key choice: unifying or stratifying space and time temporal entities spatial entities entities more stratified supersubstantive objects more unified happens at 46 Horizontal Aspects: Stratification: unifying and separated 47 The progenitors of the IMF’s TLO From: The Approach to Develop the Foundation Data Model for the Information Management Framework https://www.cdbb.cam.ac.uk/files/250221_the_choice_of_start_point_for_the_foundation_data_model_for_the_information_management_framework_1.pdf 48 Two (of many) levels of unifying-stratifying temporal locations spatial locations spatio -temporal locations more stratified more unified happens at locations in space and locations in time are unified as locations in space-time supersubstantival objects spatio -temporal locations material entities supersubstantival objects located at matter and the space-time it is located in are unified as supersubstantival objects – matter is then a way space-time can be 49 Comparing BFO and IDEAS’s stratification journey Reminder: 4D ontologies’ unifying choices {5C22544A-7EE6-4342-B048-85BDC9FD1C3A} category type choice 4D Ontologies horizontal aspect spacetime unifying or separating unifying locations unifying or separating unifying properties unifying or separating unifying endurants unifying or separating unifying immaterial unifying or separating unifying 4D ontologies are maximally unifying Motivation: the perceived benefits of parsimony and cost of separation the easy fit with plenitude 51 https://digitaltwinhub.co.uk/a-survey-of-top-level-ontologies/#a_survey_of_TLOs_contents Work with ALL the data Radically New Practices: Shifting Scope: Work with ALL the data ‘Let the data speak ’ Benzécri J (1973). L’Analyse des Données. Tome II: The model must follow the data, not the other way around … What we need is a rigorous method which extracts structures from the data. Historically, there were technical constraints on modelling with large datasets So, techniques for working with just schemas (or subsets of schemas emerged) These constraints no longer exist But the habits do Embedded into our tools Work with ALL the data : extreme shift-left A commitment to ‘Work with ALL the data’ leads to different ways of working an example is shift-left testing (Smith, L., 2001) ( Bahrs , 2014) ( Firesmith , 2015) it is an agile approach much favoured in DevOps, in which one aims to test earlier than usual in the lifecycle. there, it is often described as based upon the first half of the maxim " test early and often ”. In our context, the equivalent is shift-left analysis This naturally evolves into an extreme version which enables data-based testing (at scale) from the start of the development 54 Factor then refactor Radically New Practices: Directed Evolution: Directed Evolution: Factor then refactor Evolutionary claim is that there are two stages: one first automates/formalises somehow and then tidies up This is a salvaging approach for more detail see various papers, including: Ontology Mining versus Ontology Speculation ( https://www.academia.edu/95095494/ ) Semantic Modernisation: Layering, Harvesting and Interoperability ( https://www.academia.edu/27732898/ ) Why (and how) to use a metaphysicalist foundational ontology ( https://www.academia.edu/101235440/ ) in a nutshell: start with fielded systems – the more fielded the better As always, not a totally new idea Refactoring common in software context Martin Fowler (2018) Refactoring: Improving the Design of Existing Code but not always a clear direction of travel A modern philosophy context: “de novo conceptual engineering (designing a new concept) as well as conceptual re-engineering (fixing an old concept)” David J. Chalmers (2020) What is conceptual engineering and what should it be? Rudolf Carnap – rational reconstruction “The construction does not represent the actual process of cognition in its concrete manifestations, but… it is intended to give a rational reconstruction (“rationale Nachkonstruktion ”) of the formal structure of this process.” (1928, Der logische Aufbau der Welt. §143) More poetically, Hegel (1821) The Philosophy of Right: "the owl of Minerva spreads its wings only with the falling of the dusk". The claim is that one can only understand in hindsight. Systematic Paradigm Shifting Radically New Practices: Economics of Complex Accuracy: What is a paradigm shift? Key difference: With a paradigm shift the journey is typically one-way. When you shift to the new perspective, it is difficult to shift back In this case, it is a different, significantly better way of looking at the same data same image different interpretation Analogous gestalt shift – Rubin vase example physical implementation conceptual model Economics of Complex Accuracy: Systematic Paradigm Shifting To (economically) support the levels of accurate complexity required for semantic interoperability, There is a need to shift to richer (more complex) paradigms Similar pressure in science: gives rise to a series of paradigm shifts Not a completely new idea For example, the ‘T’ in ETL often has elements of paradigm shifting In our context, the economics require the shifts to be generated systematically at scale The factory needs a paradigm shift production line One powerful engine for generating these is ‘taking control of the form’ If both enables and suggests paradigm shifts Example Paradigm Shift: Four dimensionalism 60 Objects as ‘space-time worms’ Scale down to scale up Radically New Practices: Economics of Complex Accuracy: Scale down to scale up Two stage process: Break down wholes into new (first-class) components Reassemble components into wholes Properties of wholes emerge from properties of the components Not completely new Can see this as: Componentisation or Modularity Both well-established engineering practices in software and elsewhere Classic scaling down example: Periodic table 1807 - Dalton’s ‘billiard ball’ atoms 1869 - Mendeleev - Periodic table 1913 Bohr's shell model of the atom (electrons and shells as components) Scale down to reusable components 64 A The car has a tyre (A). When the car was built, a tyre (B) was ‘installed’ on the car. At some point in time, this tyre (B) was taken off and a new tyre (C) installed. B C A 2 1 Two new first class components (1 & 2) Summary Summary One way of understanding these radically new practices is through cultural evolutionary eyes This raises questions of: contingency If we don’t adapt in the right way, we not evolve adaption and inheritance if we have some adaptions (practices) adapted how do we (horizontally) inherit The road to semantic interoperability (computeracy) is hard because it requires the inheritance of a set of radically new practices at the moment it requires a shift from practices to praxis At least, we have a rough idea of what some of the practices are. questions 67