Marim Data Services Platform

Manual

Contents

Introduction

Welcome to the Marim manual! Before reading it, we strongly recommend you follow the tutorial to have an overview of how Marim works.

Data sources

The source keyword lets you specify data sources. For example, the snippet below specifies a data source named DvdRental:

source DvdRental
    dialect  postgresql
    url      "jdbc:postgresql://localhost:5432/dvdrental"
    property "user"     "postgres"
    property "password" "postgres"
Dialects

You specify to which DBMS (Database Management System) a data source refers to through the dialect keyword. Marim supports the following dialects:

Dialect DBMS
calcite Apache Calcite, not really a DBMS, actually a framework that lets you query CSV file content through SQL (Structured Query Language)
db2 IBM DB2
mysql MySQL
oracle Oracle Database
postgresql PostgreSQL
sqlserver SQL Server
teradata Teradata Database
URLs

You give Marim the JDBC (Java Database Connectivity) URL of the database the data source refers to through the url keyword. The table below shows the JDBC driver Marim bundles for each dialect, and gives links to the sections in their documentations that discusses their URL formats.

Dialect GroupId ArtifactId Version More information
calcite org.apache.calcite calcite-csv 1.34.0 https://calcite.apache.org/docs/adapter.html#drivers
db2 com.ibm.db2 jcc 11.5.8.0 https://www.ibm.com/docs/en/db2/11.5?topic=cdsudidsdjs-url-format-data-server-driver-jdbc-sqlj-type-4-connectivity
mysql com.mysql mysql-connector-j 8.0.33 https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-reference-jdbc-url-format.html
oracle com.oracle.database.jdbc ojdbc11 21.7.0.0 https://docs.oracle.com/en/database/oracle/oracle-database/23/jjdbc/data-sources-and-URLs.html#GUID-C4F2CA86-0F68-400C-95DA-30171C9FB8F0
postgresql org.postgresql postgresql 42.5.4 https://jdbc.postgresql.org/documentation/use/#connecting-to-the-database
sqlserver com.microsoft.sqlserver mssql-jdbc 11.2.3.jre17 https://learn.microsoft.com/en-us/sql/connect/jdbc/building-the-connection-url?view=sql-server-ver16
teradata com.teradata.jdbc terajdbc 20.00.00.10 https://teradata-docs.s3.amazonaws.com/doc/connectivity/jdbc/reference/current/frameset.html
Properties

The property keyword must be followed by a property name and value. The JDBC specification requires that every driver must support the properties user and password, which define the name and password of the user with which the driver will connect to the database.

URL and property value sources

The file keyword, which can be used together with the url and property keywords, instruct Marim to set the content of a file as the URL of a data source or the value of a property. The same applies to the variable keyword. In this case, however, Marim will set the content of an environment variable as the URL of a data source or the value of a property. For example, in the snippet below, Marim will set the content of the DVD_RENTAL_URL environment variable as the URL of the DvdRental data source, and the content of the /var/run/secrets/dvd_rental/password.txt file as the value of the password property.

source DvdRental
    dialect  postgresql
    url      variable "DVD_RENTAL_URL"
    property "user"          "postgres"
    property "password" file "/var/run/secrets/dvd_rental/password.txt"

Queries

The query keyword lets you specify a query. A query specification is comprised, at least, of a SQL statement and the data source in which it will be executed.

source DvdRental
    dialect  postgresql
    url      "jdbc:postgresql://localhost:5432/dvdrental"
    property "user"     "postgres"
    property "password" "postgres"
    
query Categories
    source DvdRental

    statement                  
        "select category_id as Id, 
                name        as Name,
                last_update as LastUpdate
           from category"
Anonymous data sources

The specification of the data source in which a query will be executed can be provided directly in the query specification. In this case, however, the source keyword cannot be followed by an identifier, rendering it an anonymous data source, and preventing other queries from referring to it. For example, in the snippet below, the data source of the Categories query is an anonymous one:

source DvdRental
    dialect  postgresql
    url      "jdbc:postgresql://localhost:5432/dvdrental"
    property "user"     "postgres"
    property "password" "postgres"

query Categories
    source
        dialect  postgresql
        url      "jdbc:postgresql://localhost:5432/dvdrental"
        property "user"     "postgres"
        property "password" "postgres"

    statement                  
        "select category_id as Id, 
                name        as Name,
                last_update as LastUpdate
           from category"
Automatic parameters: _skip and _top

Unless you state otherwise, Marim automatically adds the parameters _skip and top to every query. You can confirm this by checking the Open API specification Marim generates from your queries. For example, Marim generates the following Open API specification for the previous snippet:

...
     "parameters" : [ {
        "name" : "_skip",
        "in" : "query",
        "description" : "Number of elements that will be excluded from the response start",
        "schema" : {
          "type" : "integer",
          "format" : "int64",
          "maximum" : 9223372036854775807,
          "minimum" : 0
        }
      }, {
        "name" : "_top",
        "in" : "query",
        "description" : "Maximum number of elements the response will have",
        "schema" : {
          "type" : "integer",
          "format" : "int32",
          "maximum" : 2147483647,
          "minimum" : 1
        }
      } ]
...

To prevent Marim from adding the _skip and/or _top parameters to a query, use the result keyword followed by the unskippable and/or unlimitable keyword, as in the snippet below:

source DvdRental
    dialect  postgresql
    url      "jdbc:postgresql://localhost:5432/dvdrental"
    property "user"     "postgres"
    property "password" "postgres"

query Categories
    result
        unskippable
        unlimitable

    source DvdRental

    statement
        "select category_id as Id, 
                name        as Name,
                last_update as LastUpdate
           from category"

You can instruct Marim to add the parameters _skip and/or _top again with the skippable and/or limitable keywords. For example, the snippet below instructs Marim to add only the _top parameter to the Categories query:

source DvdRental
    dialect  postgresql
    url      "jdbc:postgresql://localhost:5432/dvdrental"
    property "user"     "postgres"
    property "password" "postgres"

query Categories
    result
        unskippable
        limitable

    source DvdRental

    statement
        "select category_id as Id, 
                name        as Name,
                last_update as LastUpdate
           from category"
Result structures

Marim lets you specify the tabular structure of a query result through the table, column and type keywords, as in the snippet below:

source DvdRental
    dialect  postgresql
    url      "jdbc:postgresql://localhost:5432/dvdrental"
    property "user"     "postgres"
    property "password" "postgres"

query Categories
    result
        table
            column Id         type integer
            column Name       type string
            column LastUpdate type timestamp		

    source DvdRental	
  
    statement
        "select category_id  as Id, 
                name         as Name,
                last_update  as LastUpdate
           from category"

Marim supports the following column types:

Type keyword Description
integer 64 bit integer
decimal Arbitrary-precision signed decimal numbers
string Arbitraty lengh character string
date Date
time Time
timestamp Date and time

Marim incorporates the metadata you provide through the table, column and type keywords in the Open API specification of your data service. For example, Marim generates the following Open API specification for the previous snippet:

...
    "/Categories" : {
      "get" : {
        "tags" : [ "Categories" ],
        "responses" : {
          "200" : {
            "description" : "OK",
            "content" : {
              "application/json" : {
                "schema" : {
                  "type" : "array",
                  "items" : {
                    "type" : "object",
                    "properties" : {
                      "Id" : {
                        "type" : "integer",
                        "format" : "int64"
                      },
                      "Name" : {
                        "type" : "string"
                      },
                      "LastUpdate" : {
                        "type" : "string",
                        "format" : "date-time"
                      }
                    }
                  }
                }
              },
...
Automatic parameters: _select, _filter and _orderby

Unless you state otherwise, if you specify the structure of a query result, then Marim automatically adds the parameters _select, _filter and _orderby to it. For example, Marim generates the following Open API specification for the previous snippet:

...
      "parameters" : [ {
        "name" : "_select",
        "in" : "query",
        "description" : "Names of the properties/columns the response will have",
        "style" : "form",
        "explode" : false,
        "schema" : {
          "type" : "array",
          "items" : {
            "type" : "string",
            "enum" : [ "Id", "Name", "LastUpdate" ]
          }
        }
      }, {
        "name" : "_filter",
        "in" : "query",
        "description" : "RSQL (https://github.com/jirutka/rsql-parser#examples) predicate the response elements must satisfy",
        "schema" : {
          "type" : "string"
        }
      }, {
        "name" : "_orderby",
        "in" : "query",
        "description" : "Response sort criteria",
        "style" : "form",
        "explode" : false,
        "schema" : {
          "type" : "array",
          "items" : {
            "type" : "string",
            "enum" : [ "Id asc", "Id desc", "Name asc", "Name desc", "LastUpdate asc", "LastUpdate desc" ]
          }
        }
      },
...

To prevent Marim from adding the _select, _filter and/or _orderby parameters to a query, use the unprojectable, unfilterable and/or unsortable keywords, as in the snippet below:

source DvdRental
    dialect  postgresql
    url      "jdbc:postgresql://localhost:5432/dvdrental"
    property "user"     "postgres"
    property "password" "postgres"

query Categories
    result
        table
            column Id         type integer
            column Name       type string
            column LastUpdate type timestamp
        unprojectable
        unfilterable
        unsortable

    source DvdRental
  
    statement
        "select category_id as Id, 
                name        as Name,
                last_update as LastUpdate
          from category"

You can instruct Marim to add the parameters _select, _filter, and/or _top again with the projectable, filterable and/or sortable keywords. For example, the snippet below instructs Marim to add only the _skip and _top parameters to the Categories query:

source DvdRental
    dialect  postgresql
    url      "jdbc:postgresql://localhost:5432/dvdrental"
    property "user"     "postgres"
    property "password" "postgres"

query Categories
    result
        table
            column Id         type integer
            column Name       type string
            column LastUpdate type timestamp		
        unprojectable
        unfilterable
        unsortable
        skippable
        limitable

    source DvdRental	
  
    statement
        "select category_id as Id, 
                name        as Name,
                last_update as LastUpdate
           from category"
Named result structures

The table keyword also lets you name a tabular structure and refer to it through this name. For example, the snippet below specifies the CategoryTable structure, which the Categories query refers to, and is equivalent to the previous snippet.

table CategoryTable
    column Id         type integer
    column Name       type string
    column LastUpdate type timestamp	
    
source DvdRental
    dialect  postgresql
    url      "jdbc:postgresql://localhost:5432/dvdrental"
    property "user"     "postgres"
    property "password" "postgres"	

query Categories
    result
        table CategoryTable
        unprojectable
        unfilterable
        unsortable        
        skippable
        limitable

    source DvdRental	
  
    statement
        "select category_id as Id, 
                name        as Name,
                last_update as LastUpdate
           from category"
Parameters

Open Telemetry

Command-line interface tool