Data View
The big picture
Semantic data model
Conceptual model
Semi-structured semantic data
A semi-structured data model is a model which represents the data and the rules about the structure of the data in the same database. The Simantics semantic data model is a semi-structured graph of statements about resources.
The following statements about the Simantics semantic data model apply
Each resource can contain at most one attached primitive value
The native access model is localized e.g. list predicates for subject or objects for (subject, predicate) pair
Some edges in the graph have a corresponding reverse edge but this is optional
The set of statements is partitioned and the partitions can be independently processed
The semantics of the model are built on layered ontologies
Primitive data
Some data does not benefit from semantic representation. This is the case for example when
Data is numeric or textual
Data has a clear primitive structure e.g. array
There is really a lot data
The data should be treated with a different granularity of versioning than semantic data.
Simantics includes its own primitive data modelling system. The system
Specifies primitive data type schemas
Selected semantic information can also be embedded into the primitive data model
Specifies powerful primitive data structure access and manipulation interfaces
Arbitrary data sources can be exposed through these interfaces
The semantic graph can be used to store one piece of primitive data per resource. This data is manipulated atomically and versioned alongside the semantic graph.
Configuration and Valuations
A general modelling contract in Simantics involves the following concepts
Configuration. A semantic graph which describes the structure of a model.
Variable. An identifier constructed from the configuration.
Valuation. An assignment of values for a set of variables.
Value. A primitive data valued function of time. Can be sampled or accessed as a time series.
The contract is that all primitive data for a model should be accessed from some valuation.
Simantics exposes the valuations and their structure using the semantic data model. The persistency model (database, workspace or memory) of semantic valuations is application specific.
The semantic valuation is needed as a uniform interface for accessing the data model produced by a semantic model.
The configuration can be expressive and terse
A simple configuration can produce a complex set of variables e.g. even an infinite set. The valuation can be lazily produced.
The complex variable formation logic is hidden behind the valuation
The model user only operates on the results
There are usually multiple values for a given variable
E.g. default, known good configurations and several experiments
The valuation directly supports this by specifying a valuation identifier which is inserted in the middle of the variable identifier
The values can come from arbitrary sources
E.g. semantic configuration, running experiment, historical archive, sensor
The model user accesses data by value using supplied interfaces. There is no need to know where the data came from.
Computational variables can be freely implemented
E.g. expressions based on variable identities can be evaluated by value or they can even be sent to the simulator as expressions.
The implementation for new values does not know how dependent values are produced
Variable typing
Some general semantics for variables are established even at the Layer0 level. These semantics can be freely extended by applications. Variables can belong to some of the following categories
Fixed. In many cases e.g. model component names are unique and persistent across all valuations.
Configuration. Some variables do not change value during simulation, but there can be several known configuration values e.g. initial states for experiments.
Simulator. Some variables have their values produced by a black box simulator process but other e.g. initial values are also stored semantically within Simantics
Simulator only. Some simulator variables are only available from the black box simulator. Simantics stores initial values for these variables only as blobs, which are transferred to the simulator as black boxes.
Derived. Some variables have values computed from other variable values.
Dynamic. Some variables are semantically declared but their occurrence in valuations is optional.
Fully dynamic. Some variables are not semantically declared can occur in valuations only. These variables can be analysed based on their primitive data content only.
Valuation stack
A valuation does not have to supply values for all variables in a model configuration. A model user accesses values using a valuation stack which is guaranteed to produce values for all variables.
Large primitives
Some models contain large pieces (> megabytes) of primitive data which needs to be specially processed. Key questions are
Partial reads and writes (e.g. of a terabyte file)
Data transfer between server and client
File access support (e.g. native state files of simulators stored in the semantic database and accessed by simulators as files)
Client models
Sessions and databases
The semantic database is accessed by using a session which provides shared access to a working copy of the database for multiple users. Key points are
Concurrent use models. The session provides both transaction-locked concurrent use and publish/synchronize based operation.
Authentication and access control. These are not yet supported in Simantics 1.0
Database client access interface
The database session interface allows reading and writing the semantic database using requests. Key points are
Transaction model. The client provides a traditional read-write locking scheme. Reads are processed concurrently while writes require exclusive locking. Fairness requirements for the locking scheme are not specified.
Caching of requests. Often-used read requests can be cached to achieve necessary read performance.
Change listening. The client interface supports change listening with automatic dependency analysis. This mechanism allows the database to be used in reactive programming style.
Threading model. The client supports both synchronous and asynchronous request formulation. Asynchronous model automatically distributes the computation on available processor cores.
The connection between change sets and requests. Write requests form change sets, which can be further annotated with metadata.
Runtime client extensions
Adapters
The adaption of resources into Java interfaces is a key method of enforcing the semantics of the data model. The adapter model includes
A way to declare adapter interfaces in plugins
A way to contribute adapter implementations in plugins
A way to retrieve an adapter for a resource
The mechanism is described in Resource Adaptation.
Virtual graph
The Simantics database client can extend the persistent semantic graph with more transient resources and statements. This mechanism can be used to e.g. expose valuations and other workspace-persistent data.
The following statements apply
The life cycle of transient contributions is managed by a workspace-persistent entity
Transient resources can be persistent within the life cycle of the workspace
This entity ensures that co-existing subgraphs share the available identifier space
Transient contributions are dynamically attached and detached to a database Session.
The subgraph is applied on top of the persistent graph and below the query layer
Generic Relation Indexing
Layer0 Generic Relations can be used to construct per model indices of useful data, which then can be quickly searched using the Lucene search engine.
Semantics
Ontology language (Layer0)
Layer0 describes the baseline semantics for Simantics applications. Layer0 involves
Generic concepts used in all Simantics modelling
Common activities encountered in all Simantics modelling
Adapter interfaces, which implement the intended semantics of the concepts and the activities
Concepts
Type. A resource can be an instance of several types. The types assert structural constraints, contribute relations and specify adapters.
Relation. All predicates in the semantic graph are part of a relation hierarchy. Structural constraints involve relations and basic queries support relation hierarchies.
Requirement. A validity requirement, which can used to check semantic instances against their specification
Data type. Specifies the structure and units of a piece of primitive data.
User. Users are needed in change management, documentation and access control.
Adapter. Adapter specifies a requirement for the client to supply an implementation of a certain interface.
URI. A URI can be used to identify and locate resources within a semantic database.
Subgraph. Specifies a set of statements for a resource. Needed in e.g. copy and delete activities.
Library. Libraries are used to structure large amounts of data. Libraries are browsed.
Project. Private data is organised in projects. A project specifies its required execution environment.
Model. A model spans a variable space.
Variable. A model determines a set of variables which have primitive data values.
Valuation. A valuation contains values for variables.
Label. A label is a textual representation of a resource.
Viewpoint. A viewpoint specifies means to browse the semantic graph.
Predicate. A logical predicate as e.g. in Prolog.
Function. A function with n inputs and a single output.
Index. A workspace-persistent realization of a predicate
Activation. Used to activate and deactivate a concept.
Operation. A runnable action, which makes a modification to the semantic graph.
Extent. For determining a local extent of an instance. For determining subgraphs.
<!-- -->Activities
Adaption. Retrieves an implementation of an interface for a resource.
Analysis. Computes some result based on the semantic graph.
Validation. Checks whether some constraints are satisfied.
Access determination. Determines whether access to data is granted.
Structure determination. Determines allowed structures in the graph.
Shared identification. Establishes identity across semantic databases.
Instantiation. Creates new instances based on types.
Copying. Creates a copy of a subgraph in the same database or in some other database.
Deletion. Deletes a subgraph.
Activation. Transfers a concept between active and inactive states. Activation can involve displaying and initiation of certain processes.
Mapping. Enforces rules between models.
Editing. Transforms a model based on input.
Variable discovery. Determination of the variable space based on the configuration structure of the model.
Instance discovery. Finding e.g. shared instances of a type from the database.
Browsing. Traversing the semantic graph using tree representations.
Layer 0 is further specified in Layer0.pdf.
Project management
The top level organization or a Simantics database is depicted in the image below.
Projects
Each Simantics workspace is started into a specific project. Simantics projects use features for associating life-cycle behavior to projects, much like project natures in Eclipse.
Subgraph transfer mechanism
The subgraph transfer mechanism is needed for identifying a set of statements to transfer from and to a database.
Use-cases, where the mechanism is needed:
Copy-paste
Export-import
Ontology evolution
There have been at least two different prototype implementations of subgraph transfer (org.simantics.layer0.utils.extent and org.simantics.layer0.utils.extent2). Both have their own shortcomings.
There are two major technical challenges:
How to define which statements belong to the subgraph
First approach is based on Includes- and Propagates-relations.
Second approach is based on classification of relations and defining parent resources for all resources.
See also Subgraph Extents
We have so far tried to find a generic solution for this problem. There are however often application specific concerns. A robust solution should probably involve some generic solution that can be customized with adapters.
Even if adapters solve the problem of local propagation of extents, we still need a conceptual model that defines the whole extent.
How to serialize resource references
Several serialization formats have been defined (but the format is not really the problem):
The most important issue is serializability of resources.
Validation
To maximally utilize the increased semantic content of the semantic database a validation mechanism is defined for
Checking the semantic configuration against semantic rules
Checking the validity of the semantic implications of the semantic configuration (e.g. checking the flattening of a structural model)
Checking real time data against semantic rules
Browsing
The following statements describe the browsing solution in Simantics
The browser builds up its representation using Contributions e.g. ViewpointContributions and Labelers.
Viewpoints and Labelers are determined using Evaluators bound to Java class of input objects.
The ontology for browsing specifies a viewpoint made up from contributions to named contexts (e.g. browser identity or model) with adapter-modelled acceptance criteria
New contributions are described in ontologies and implemented as Java adapters.
Each browser has its own unique identifier string
Contributions are contributed and bound to contexts using Eclipse extension points
Concepts and their contracts
- Evaluator
A Java object which determines e.g. Viewpoints and Labelers for given input.
- Viewpoint
Determines a set of child objects for given input object.
- Labeler
Determines a label for an object.
- Context
A set of keys and associated values which are used to steer the browsing process. A mandatory INPUT object is used in Evaluators.
Framework ontologies
Mapping
Mapping framework in Simantics consists of several different levels. From the lowest to the highest:
- Triggers
Provides a uniform interface to mappings and other graph modifying software components. This specification defines how a mapping is attached to a resource and how it is used. Actual implementation can be written directly in Java or using higher level mechanisms.
- SCL mappings
A datalog based language that is meant for writing queries and transformations.
- Data driven mappings
A mapping defined using lower level definitions that can be customized with specific relations. For example Modeling.pgraph defines this kind of mapping. Graphical user intefaces can be built for defining this kind of mappings.
Structural modelling
The structural ontology models hierarchically decomposed and connected components and their flattening.
The data model is described in Structural Ontology.
Diagramming
Diagrams are used to configure models in various domains. The diagram model is a self-sufficient graphical model which is mapped into domain specific simulation models directly or via structural ontology.
Address Space
Address Space is the structure of the data when a model is exposed in an communication interface. It consists of nodes and variables.
There are two types of address spaces:
Configuration Configuration model consists of initialization values, such as parameters.
Runtime Runtime model consists of variables and constants.
There are two structures of address spaces:
Structural address space is a complicated model of instances and classes. The structure is a graph.
Flattened address space is a flattened simplification of the previous model. It is a tree.
There is no <u>{=html}structural</u>{=html} address space of runtime
model.
The <u>{=html}structure</u>{=html} of configuration flat model is
equal or a sub-set of the a runtime flat model.