Mapping: Lowering Research

Lowering RDF

TL;DR: Mapping languages:

Javascript (JSON & string (& XML))
Simple mapping (JSON & XML, not good enough)
Handbars (XML)
VueJS (XML)
JSX (XML)
XQuery (JSON & XML & string)
XSLT (JSON & XML)
XSPARQL (XML & string)

My favourites:

Javascript for JSON + JSX for XML
Xquery JSON + XML

All languages but XSPARQL aren't really RDF compliant (see rdf-compliance). Possible solutions at Possible solutions.

Sample input action: (JSON-LD)

{
  "@context": "http://schema.org/",
  "object": {
    "@type": "Person",
    "name": "John Doe",
    "address": {
      "postalCode": "2111"
    },
    "birthDate": {
      "@type": "xs:dateTime",
      "@value": "2020-03-18T08:31:04+0000"
    },
    "email": ["john.deo@personal.com", "john.deo@work.com"],
    "knows": [
      {
        "@type": "Person",
        "name": "Max Mustermann"
      },
      {
        "@type": "Person",
        "name": "Jane Doe"
      }
    ],
    "condition": true
  }
}

Wanted JSON/XML request

{
  "id": 1234,
  "name": "John Doe",
  "p_code": 2111,
  "bday": "2020-03-18",
  "emails": ["john.deo@personal.com", "john.deo@work.com"],
  "rel": [
    {
      "name": "Max Mustermann"
    },
    {
      "name": "Jane Doe"
    }
  ],
  "Person": true,  // key=object.@type, value= object.@type == Person
  "cond": "yes" // optional, only if object.condition === true
}

<?xml version="1.0" encoding="UTF-8" ?>
<root>
  <id>1234</id>
  <name>John Doe</name>
  <p_code>2111</p_code>
  <bday>2020-03-18</bday>
  <emails>john.deo@personal.com</emails>
  <emails>john.deo@work.com</emails>
  <rel>
    <name>Max Mustermann</name>
  </rel>
  <rel>
    <name>Jane Doe</name>
  </rel>
  <Person>true</Person>
  <cond>yes</cond>
</root>

"Simple" JSON/XML mapping

The lowering mapping I had implemented

{
  "id": 1234,
  "name": "$.object.name",
  "p_code": "$.object.address.postalCode |> (x => toNumber(x))",
  "bday": "$.object.birthDate.@value |> (x => parseDate(x))",
  "emails": "$.object.email"
}

missing: rel, Person

<?xml version="1.0" encoding="UTF-8"?>
<root>
  <id>1234</id>
  <name>$.object.name</name>
  <p_code>$.object.address.postalCode</p_code>
  <bday>$.object.birthDate.@value |> (x => parseDate(x))</bday>
</root>

missing: rel, emails, Person

+ Easy to use

+ Mapping itself is valid JSON/XML

- Not RDF compliant

- No array/list support

Vanilla JS

To JSON

({
  "id": 1234,
  "name": $.object.name,
  "p_code": toNumber($.object.address.postalCode),
  "bday": parseDate($.object.birthDate['@value']),
  "emails": $.object.email.map(e => e),
  "rel": $.object.knows.map(p => ({
    name: p.name
  })),
  [$.object['@type']]: $.object['@type'] === 'Person',
  ...($.object.condition && {cond: 'yes'})
})

To XML

(`<root>
  <id>1234</id>
  <name>${$.object.name}</name>
  <p_code>${toNumber($.object.address.postalCode)}</p_code>
  <bday>${parseDate($.object.birthDate['@value'])}</bday>
  ${$.object.email.map(e =>
  `<emails>${e}</emails>`
  ).join('\n')}
  ${$.object.knows.map(p => `
  <rel>
    <name>${p.name}</name>
  </rel>`
  ).join('\n')}
  <${$.object['@type']}> ${ $.object['@type'] === 'Person'} </${$.object['@type']}>
  ${$.object.condition ? '<cond>yes</cond>': ''}
</root>`)

+ Known/Popular

+ NP-Complete / can express anything

+- need to eval with JS Engine

- Not RDF compliant

- Difficulty expressing XML (need strings or function to convert json to xml)

Mustache / Handlebars

templating language

handlebars is extension to mustache

handlebars main use is to generate html, can use for xml mapping

from http://mustache.github.io & http://handlebarsjs.com/

To XML: (in handlebars)

Handlebars.registerHelper('parseDate', (s) => s.substring(0,10));

Handlebars.registerHelper('get', function (path, opts) {
    return get(opts, `data.root.${path}`)
});

Handlebars.registerHelper('eq', function(arg1, arg2) {
    return (arg1 == arg2);
});

<root>
  <id>1234</id>
  <name>{{object.name}}</name>
  <p_code>{{object.address.postalCode}}</p_code>
  <bday>{{ parseDate (get "object.birthDate.@value")}}</bday>
  {{#each object.email}}
  <emails>{{this}}</emails>
  {{/each}}
  {{#each object.knows}}
  <rel>
    <name>{{this.name}}/name>
  </rel>
  {{/each}}
  <{{get "object.@type"}}> {{ eq (get "object.@type") "Person"}} </{{get "object.@type"}}>
  {{#if object.condition}}
  <cond>yes</cond>
  {{/if}}
</root>

+ simple

+ can use javascript functions

- not RDF compliant

- made for xml (html) output, can technically output JSON

Vue Templates

Although intended for something else, vue can generate xml

To XML:

<template>
    <root>
    <id>1234</id>
    <name>{{object.name}}</name>
    <p_code>{{object.address.postalCode}}</p_code>
    <bday>{{object.birthDate['@value'].substring(0, 10)}}</bday>
    <emails v-for="email in object.email">
        {{ email }}
    </emails>
    <rel v-for="p in object.knows">
        <name>{{ p.name }}</name>
    </rel>
    <component :is="type">
        {{object['@type'] == 'Person'}}
    </component>
    <cond v-if="object.condition">yes</span>
    </root>
</template>

<script>
module.exports = {
    data: {JSONLD_Object}
    computed: {
        type() {
            return this.object["@type"];
        }
    }
}
</script>

+ popular

+ simple

+ JS functions

- Vue is more than just its templates

- not RDF compliant

- only xml output

JSX (React)

JSX is Javascript extension for HTML (XML)

<root>
  <id>1234</id>
  <name>{$.object.name}</name>
  <p_code>{toNumber($.object.address.postalCode)}</p_code>
  <bday>{parseDate($.object.birthDate['@value'])}</bday>
  {$.object.email.map(e =>
    <emails>{e}</emails>
  )}
  {$.object.knows.map(p => 
  <rel>
    <name>{p.name}</name>
  </rel>
  )}
  <${$.object['@type']}> ${ $.object['@type'] === 'Person'} </${$.object['@type']}>
  {$.object.condition && <cond>yes</cond>}
  {React.createElement($.object["@type"], null, ($.object["@type"] === "Person").toString())}
</root>

+ popular

+ basically extended Javascript with XML support

- Only xml output

- not RDF compliant

XQuery 3.1

3.1 for JSON support, see http://docs.basex.org/wiki/XQuery_3.1

With Saxon HE: (saxonica.com)

java -cp D:\Libraries\Downloads\SaxonHE10-0J\saxon-he-10.0.jar net.sf.saxon.Query -q:"query.xq" (with -qs="xquerystring" directly from string)

With fontoXpath: (npm library) See xquery.js

3 options:

parse json in xquery - input is xquery maps/arrays
transform json to xml in xquery, use xpath to query
transform JSON-LD in a previous step to xml (rdf-xml or any xml representation of the json)

1 st option:

To XML:

declare namespace map = "http://www.w3.org/2005/xpath-functions/map";
declare namespace array = "http://www.w3.org/2005/xpath-functions/array";
declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization";
declare option output:method "adaptive";

declare function local:parseDate($p as xs:string?)
as xs:string?
{
substring($p, 1, 10)
};

let $json := parse-json('{"@context":"http://schema.org/","object":{"@type":"Person","name":"John Doe","address":{"postalCode":"2111"},"birthDate":{"@type":"xs:dateTime","@value":"2020-03-18T08:31:04+0000"},"email":["john.deo@personal.com","john.deo@work.com"],"knows":[{"@type":"Person","name":"Max Mustermann"},{"@type":"Person","name":"Jane Doe"}],"condition":true}}')

let $obj := map:get($json, "object")

return
<root>
  <id>1234</id>
  <name>{$obj("name")}</name>
  <p_code>{$obj("address")("postalCode")}</p_code>
  <bday>{local:parseDate($obj("birthDate")("@value"))}</bday>
  {array:for-each(
    $obj("email"), 
    function($value) {
      <emails>{$value}</emails>
    }
  )}
  {array:for-each(
    $obj("knows"), 
    function($value) {
      <rel><name>{$value("name")}</name></rel>
    }
  )}
  {element {$obj("@type")} {$obj("@type") = "Person"}}
  {if($obj("condition")) then (
      <cond>yes</cond>
  ) else ()}
</root>

To JSON:

(*same header as for xml*)

let $map := 
map {
  "id": 1234,
  "name": $obj("name"),
  "p_code": number($obj("address")("postalCode")),
  "bday": local:parseDate($obj("birthDate")("@value")),
  "emails": array:for-each(
    $obj("email"), 
    function($value) {
      $value
    }
  ),
  "rel": array:for-each(
    $obj("knows"), 
    function($value) {
      map {"name": $value("name")}
    }
  ),
  $obj("@type"): $obj("@type") = "Person"
}

let $ret := if($obj("condition")) then map:put($map, "cond", "yes") else $map

return $ret

2nd option: Parse json to xml xquery - input now acts as xml: (only show XML->XML, XML->JSON would work similarly)

To JSON: (only works in Saxon, not fontoxpath npm). Result of json-to-xml is bit weird, kind of annoying to query.

let $xml := json-to-xml('...')

let $obj := $xml/fn:map/fn:map[@key="object"]

return
<root>
  <id>1234</id>
  <name>{$obj/fn:string[@key="name"]/text()}</name>
  <p_code>{$obj/fn:map[@key="address"]/fn:string[@key="postalCode"]/text()}</p_code>
  <bday>{local:parseDate($obj/fn:map[@key="birthDate"]/fn:string[@key="@value"]/text())}</bday>
  {for $e in $obj/fn:array[@key="email"]/fn:string
    return <emails>{$e/text()}</emails>
  }
  {for $e in $obj/fn:array[@key="knows"]/fn:map
    return <rel><name>{$e/fn:string[@key="name"]/text()}</name></rel>
  }
  {element {$obj/fn:string[@key="@type"]/text()} {$obj/fn:string[@key="@type"]/text() = "Person"}}
  {if(xs:boolean($obj/fn:boolean[@key="condition"])) then (
      <cond>yes</cond>
  ) else ()}
</root>

+ transformation language (not primary purpose, but eh..)

+ similar to traditional programming languages (like javascript)

+ single language for XML/JSON/any other string output

+- Need to write functions in xquery syntax (could do javascript functions with fontoxpath)

- JAVA needed for Saxon (might want to try Saxon C implementation)

- Not RDF compliant

- Need to learn language

XSLT 3.0

3.0 again for json support

Using Saxon (Java)

Calling: java -cp D:\Libraries\Downloads\SaxonHE10-0J\saxon-he-10.0.jar net.sf.saxon.Transform -xsl:"q.xslt" -it:"n" input="<escaped-json-string>"

Escaping json in browser:

JSON.stringify({json}).replace(/"/g, '\\"')

To Json: (with json->xml->json / xquery option 2)

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:math="http://www.w3.org/2005/xpath-functions/math"
    xmlns:xd="http://www.oxygenxml.com/ns/doc/xsl"
    xmlns:emp="http://www.semanticalllc.com/ns/employees#"
    xmlns:h="http://www.w3.org/1999/xhtml"
    xmlns:fn="http://www.w3.org/2005/xpath-functions"
    xmlns:j="http://www.w3.org/2005/xpath-functions"
    xmlns:local="http://myfunc.com"
    exclude-result-prefixes="xs math xd h emp"
    version="3.0"
    expand-text="yes"
    >
    <xsl:param name="input" as="xs:string"/>
    <xsl:output method="text" indent="yes" media-type="text/json" omit-xml-declaration="yes"/>
    <xsl:variable name="json" select="json-to-xml($input)"/>
    <xsl:template name="n">
        <xsl:variable name="out">
           <xsl:apply-templates select="$json/*"/>
        </xsl:variable>
        {xml-to-json($out)}
    </xsl:template>
    <xsl:function name="local:parseDate" as="xs:string">
        <xsl:param name="p" as="xs:string"/>
        <xsl:sequence select="substring($p, 1, 10)"/>
    </xsl:function>
    <xsl:template match="j:map" xpath-default-namespace="http://www.w3.org/2005/xpath-functions">
        <xsl:variable name="obj" select="map[@key='object']"/>
        <j:map>
            <j:string key="id">1234</j:string>
            <j:string key="name">{$obj/string[@key="name"]}</j:string>
            <j:string key="p_code">{$obj/map[@key="address"]/string[@key="postalCode"]/text()}</j:string>
            <j:string key="bday">{local:parseDate($obj/map[@key="birthDate"]/string[@key="@value"]/text())}</j:string>
            <j:array key="emails">
                <xsl:for-each select="$obj/array[@key='email']/*">
                    <j:string>{text()}</j:string>
                </xsl:for-each>
            </j:array>
            <j:array key="rel">
                <xsl:for-each select="$obj/array[@key='knows']/*">
                    <j:map>
                        <j:string key="name">{string[@key="name"]}</j:string>
                    </j:map>
                </xsl:for-each>
            </j:array>
            <j:boolean key="{$obj/string[@key='@type']}">{$obj/string[@key='@type']="Person"}</j:boolean>
            <xsl:if test="$obj/boolean[@key='condition']/text() = 'true'"> 
                <j:string key="cond">yes</j:string>
            </xsl:if>
        </j:map>
    </xsl:template>
</xsl:stylesheet>

To XML example very similar, e.g. <name>{$obj/string[@key="name"]}<name> instead of <j:string key="name">{$obj/string[@key="name"]}</j:string>.

+ Transformation language

+ mapping document is xml

+ JSON/XML output

- more difficult to learn than xquery (IMO)

- not RDF compliant

XSparql

from https://github.com/semantalytics/xsparql

doc: https://www.w3.org/Submission/xsparql-language-specification/

calling:

java -jar D:\Libraries\Downloads\xsparql-cli-jar-with-dependencies.jar q.xs

prefix schema: <http://schema.org/>

declare function local:parseDate($p as xs:string?)
as xs:string?
{
substring($p, 1, 10)
};

declare function local:getType($p as xs:string?)
as xs:string?
{
tokenize($p, '/')[4]
};

for $Name $Code $Bday $Type from <data.ttl>
where { 
    <http://action.com> schema:object $Obj .
    $Obj a $Type .
    $Obj schema:name $Name .
    $Obj schema:address/schema:postalCode $Code .
    $Obj schema:birthDate $Bday .
}
return 
<root>
    <id>{"1234"}</id>
    <name>{$Name}</name>
    <p_code>{$Code}</p_code>
    <bday>{local:parseDate($Bday)}</bday>
    {
        for $Email from <data.ttl>
        where { 
            $Obj schema:email $Email .
        }
        return <emails>{$Email}</emails>
    }
    {
        for $Ref from <data.ttl>
        where { 
            $Obj schema:knows/schema:name $Ref .
        }
        return <rel><name>{$Ref}</name></rel>
    }
    {element {local:getType($Type)} {local:getType($Type) = "Person"}}
    {if($Cond = "true") then (
      <cond>yes</cond>
    ) else ()}
</root>

+ RDF compliant

+ can use XQuery

- slow (example takes 1.85 sec for this small example)

- no JSON output (as far as I have seen)

- need to convert JSON-LD input to ttl/tripples

- not very known language

- own syntax - no syntax highlighting

- Java jar

- oldish project (last real commit 3y ago)

Other

Also took a look at Sparql Templates (STTL) (http://ns.inria.fr/sparql-template/), would be RDF compliant and technically turing complete, but too verbose/complicated.

RDF Compliance

So far all mappings could only work with the fixed JSON-LD structure. The same mapping with the semantically same data in a different structure would not work.

E.g. the following JSON-LD

{
    "birthDate": {
      "@value": "2020-03-18T08:31:04+0000"
    },
    "email": "john.deo@personal.com"
}

is equivalent to

{
    "birthDate": "2020-03-18T08:31:04+0000",
    "email": ["john.deo@personal.com"]
}

Yet the mapping would need to query the data differently.

Possible solutions:

1. Use SPARQL to query the data in a first step (something like to SAWSDL https://www.w3.org/TR/sawsdl/#lifting) and query the SPARQL results instead. Downsides: mapping more verbose (now with sparql queries), not sure how well this works with arrays/lists containing nodes, as sparql results are "flat" (e.g csv lines). Using sparql just to get properties seems a bit overkill. Similar to XSparql.

2. Transform RDF (JSON-LD) input to some fixed form. E.g. transform to expanded or flattened document form ("https://json-ld.org/spec/latest/json-ld/#expanded-document-form") (maybe still with context for easier queries) and create mapping for the new json structure. Also maybe replace insert nodes where they are used (e.g replace the "address": {"@id": "_:b1"} references with the actual addresses) (Replacing doesn't work with circular dependencies though). Or convert to RDF/XML for easier Xpath/Xquery/XSLT querying. (although RDF XML can also be represented in different ways with the same semantic meaning) Downside is querying the data will be more verbose/complicated.

with JS Mapping: instead of

{
    foo: $.object.name
}

becomes

{
    // with a context/namespaces
    foo: $.object[0].name[0]['@value']
    // without context
    foo: $['http://schema.org/object'][0]['http://schema.org/name'][0]['@value'] 
}

With xquery mapping:

let $obj := $json(1)("http://schema.org/object")(1)

return
map {
  "name": $obj("http://schema.org/name")(1)("@value"),
  "p_code": $obj("http://schema.org/address")(1)("http://schema.org/postalCode")(1)("@value")
}

3. Query with custom functions. Instead of querying directly the data (with xpath, xsl maps/arrays, Javascript objects) use a custom function to get the wanted rdf properties.

e.g.

function getPath(rdfPropertyPath: string): string | string[]

Would only work with Javascript mappings (and javascript xquery implementation - fontoxpath).

Function could maybe support SPARQL propertypaths (https://www.w3.org/TR/sparql11-query/#propertypaths) or even full SPARQL with https://github.com/linkeddata/rdflib.js .

with JS Mapping:

{
    name: getPath('/object/name') // with sparql PP ('schema:object/schema:name') / (':object/:name')
}

with xquery (js-fontoxpath):

map {
  "name": local:getPath("/object/name")
}

with handlebars:

<name>{{ getPath "/object/name"}}</name>

Only downside: most probably would not work with xslt.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mapping: Lowering Research

Lowering RDF

"Simple" JSON/XML mapping

Vanilla JS

Mustache / Handlebars

Vue Templates

JSX (React)

XQuery 3.1

XSLT 3.0

XSparql

Other

RDF Compliance

Possible solutions:

Clone this wiki locally